Skip to content

docs: refine documentation tone, eliminate ai artifacts, and expand technical depth - #25

Merged
MichalRedm merged 4 commits into
mainfrom
docs/refine-docs-professionalism
Oct 11, 2026
Merged

MichalRedm merged 4 commits into
mainfrom
docs/refine-docs-professionalism

Conversation

@MichalRedm

Copy link
Copy Markdown
Collaborator

Summary

  • Refactored the entire documentation suite to eliminate AI-generated artifacts, decorative emojis in headings/bullets, promotional boilerplate, and superficial placeholders.
  • Expanded user guides across all framework analytical layers (Data Layer, Model Layer, Decision Boundary Analysis, Feature Importance, Clustering Dynamics, Recurring Concepts, and Dashboard) into comprehensive, publication-grade academic prose with formal mathematical formulations.
  • Upgraded the documentation landing page (docs/index.md) to reflect the published ECML PKDD 2026 paper, Springer chapter citation, NCN grant funding, and three-layer synchronous processing architecture.
  • Enriched all 10 API reference pages under docs/api/ with module architectural overviews, key class/function matrices, and standard import workflows preceding the autogenerated mkdocstrings directives.

Motivation

Following pull request #22 which established the MkDocs infrastructure and autogenerated API reference, the documentation retained visible signs of AI-generated content (excessive emojis in headings, shallow 30-line skeleton pages, formulaic bullet lists, and conversational filler). Drawing inspiration from the evolution of README.md—which transitioned from an emoji-heavy draft to a rigorous, academic presentation for ECML PKDD 2026—this PR elevates the documentation to match the project's peer-reviewed research caliber.

Key Changes

1. Landing Page & Quickstart

  • Replaced emoji headings (🌟 Overview, 🏛️ Architecture, 🚀 Quick Navigation) and centered HTML banner in docs/index.md with an academic header, publication badges, formal problem decomposition, three-layer architectural narrative, and clean navigation tables.
  • Overhauled docs/getting-started/quickstart.md to present a cohesive, realistic end-to-end stream forensics workflow (Hyperplane drift generation $\to$ statistical two-sample tests $\to$ sequential error tracking with CUSUM lookback $\to$ temporal discriminator feature attribution $\to$ dashboard launch).
  • Expanded docs/getting-started/installation.md with explicit virtual environment setup and a detailed capability extras table.

2. Analytical User Guides

  • Data Layer (docs/user-guide/data-layer.md): Added formal mathematical formulations for Kolmogorov-Smirnov ($D_{KS}$), Anderson-Darling ($A^2$), and 1D Wasserstein distance ($\mathcal{W}_1$), along with practical guidance on window size selection and multiple hypothesis testing.
  • Model Layer (docs/user-guide/model-layer.md): Detailed River's DDM Bernoulli process formulation ($p_t \pm 2s/3s$ thresholds), CUSUM/gradient lookback onset localization mechanics, and the DriftDescription dataclass schema.
  • Decision Boundaries (docs/user-guide/boundary-analysis.md): Explained why standard dimensionality reduction lacks continuous inverse mapping ($\mathbb{R}^2 \to \mathbb{R}^D$), detailed SSNP's composite reconstruction and neighborhood preservation loss, and documented disagreement decision tree rule extraction.
  • Feature Importance (docs/user-guide/feature-importance.md): Formalized the temporal discriminator paradigm, explained the distinction between data drift ($Z = X$) and real concept drift ($Z = [X, Y]$), and documented conformal selective prediction with abstention.
  • Clustering Dynamics (docs/user-guide/clustering-dynamics.md): Documented dynamic $K$ selection via X-means BIC score, formulated centroid matching as optimal bipartite assignment via the Hungarian algorithm ($O(K^3)$), and detailed cluster velocity, birth, and death metrics.
  • Recurring Concepts (docs/user-guide/recurring-concepts.md): Formalized prototype tree representations (ProTree), pairwise distance matrices ("minimal_distance", "mean_distance"), and HDBSCAN regime clustering with outlier correction.
  • Interactive Dashboard (docs/user-guide/dashboard.md): Detailed the decoupled architecture, session state isolation, computation gating, full tab-by-tab workflow, and Shaker dataset stitcher dialog.

3. Theory & Developer Standards

  • Drift Taxonomy (docs/theory/drift-taxonomy.md): Formalized Bayesian decomposition $P(X, Y) = P(X)P(Y \mid X) = P(Y)P(X \mid Y)$, temporal velocity profiles (abrupt, gradual sigmoidal mixture, incremental, recurring), and mapped them to STRIDE's three layers with formal citations.
  • Mathematical Formulations (docs/theory/mathematical-formulations.md): Comprehensive mathematical reference including Kantorovich-Rubinstein duality, asymptotic ECDF distributions, SSNP loss formulation, and conformal prediction guarantees.
  • Contributing & Agent Standards (docs/developer/): Outlined architectural invariants (pure core, headless plotting), Git conventional commit protocols, and codified Google-style docstring invariants with type deduplication rules.

4. API Reference Manuals

  • Enriched all 10 API reference documents (api/datasets.md, api/drift.md, api/exceptions.md, api/models.md, api/plotting.md, api/xai-boundary.md, api/xai-clustering.md, api/xai-importance.md, api/xai-recurrence.md, api/xai-stats.md) with module architectural contexts and summary tables of exported classes, exceptions, and methods.

Verification

  • ruff check . passed with 0 errors (All checks passed!)
  • ruff format --check . passed cleanly with 0 formatting warnings (113 files already formatted)
  • python -m unittest discover tests passed with 42/42 tests passing (OK)
  • mkdocs build --strict built documentation with 0 warnings or errors in strict mode
  • Verified zero remaining emojis across all markdown files in docs/

@MichalRedm
MichalRedm merged commit 7215706 into main Oct 11, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant