Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 8 additions & 8 deletions .github/CODEOWNERS
Original file line number Diff line number Diff line change
Expand Up @@ -3,17 +3,17 @@
# Order matters: the last matching pattern wins.

# Default owner for everything in the repo.
* @shifulegend
* @sherwin-casem

# Build + CI configuration.
/Makefile @shifulegend
/CMakeLists.txt @shifulegend
/.github/ @shifulegend
/Makefile @sherwin-casem
/CMakeLists.txt @sherwin-casem
/.github/ @sherwin-casem

# Core engine hot paths (kernels, weight loading, transformer).
/src/math/ @shifulegend
/src/core/ @shifulegend
/src/transformer/ @shifulegend
/src/math/ @sherwin-casem
/src/core/ @sherwin-casem
/src/transformer/ @sherwin-casem

# Project memory / AI-dev system (canonical docs).
/docs/ai/ @shifulegend
/docs/ai/ @sherwin-casem
2 changes: 1 addition & 1 deletion .github/FUNDING.yml
Original file line number Diff line number Diff line change
@@ -1 +1 @@
github: shifulegend
github: sherwin-casem
6 changes: 3 additions & 3 deletions .github/ISSUE_TEMPLATE/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,11 @@
blank_issues_enabled: true
contact_links:
- name: 💬 Questions & Discussion
url: https://github.com/shifulegend/project-zero/discussions
url: https://github.com/sherwin-casem/project-zero/discussions
about: Ask usage questions, propose ideas, or discuss architecture here.
- name: 🏁 Community Benchmark Challenge
url: https://github.com/shifulegend/project-zero/discussions/3
url: https://github.com/sherwin-casem/project-zero/discussions/3
about: Run the engine on your hardware and share your tok/s result.
- name: 🔐 Report a Security Vulnerability
url: https://github.com/shifulegend/project-zero/security/advisories/new
url: https://github.com/sherwin-casem/project-zero/security/advisories/new
about: Please report vulnerabilities privately — do not open a public issue.
6 changes: 3 additions & 3 deletions .github/ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -116,7 +116,7 @@ Verified on [OpenBenchmarking.org](https://openbenchmarking.org/result/2606063-S
| Phase | Feature | Status | Notes |
|-------|---------|--------|-------|
| **34** | GGUF reader — metadata, tokenizer, tensor loading | ✅ | DeepSeek-V2-Lite-Chat Q4_K_S runs correctly |
| **34-MoE** | MoE expert weight scatter — contiguous access at inference time | 🆘 **BLOCKING** | 13× behind llama.cpp. Root cause: non-contiguous DRAM offsets. [Discussion #1](https://github.com/shifulegend/project-zero/discussions/1) |
| **34-MoE** | MoE expert weight scatter — contiguous access at inference time | 🆘 **BLOCKING** | 13× behind llama.cpp. Root cause: non-contiguous DRAM offsets. [Discussion #1](https://github.com/sherwin-casem/project-zero/discussions/1) |
| **35** | GGUF MLA+MoE architecture router — deepseek2 tensor mapping, stacked expert slicing | ❌ | Fully specified in `IMPLEMENTATION_PLAN.md §35` |
| **37.1** | Q2_K dequant | ✅ | Needed for DeepSeek-V2-Lite-Chat-Q2_K |
| **37.2** | Q3_K dequant | ❌ | 256 elems, 110 bytes/block |
Expand Down Expand Up @@ -172,7 +172,7 @@ The two highest-impact contributions that unblock all MoE-related phases:

**Fix:** Repack expert weights at model load time so the top-K experts for common activation patterns are physically contiguous. llama.cpp does this in `llama_model_load_internal()`. The challenge: preserving Q4_K superblock boundaries during the repack.

**Discussion + full profiling data:** [Discussion #1](https://github.com/shifulegend/project-zero/discussions/1)
**Discussion + full profiling data:** [Discussion #1](https://github.com/sherwin-casem/project-zero/discussions/1)

### 2. Native Q4_K Matmul Kernel

Expand All @@ -184,7 +184,7 @@ The two highest-impact contributions that unblock all MoE-related phases:

## Community Benchmarks

Run the engine on your hardware and add your result: [Discussion #3](https://github.com/shifulegend/project-zero/discussions/3)
Run the engine on your hardware and add your result: [Discussion #3](https://github.com/sherwin-casem/project-zero/discussions/3)

---

Expand Down
4 changes: 2 additions & 2 deletions CITATION.cff
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,10 @@ abstract: >-
tools, and RAG persistent memory. No GPU and no ML framework.
type: software
authors:
- alias: shifulegend
- alias: sherwin-casem
name: "Project Zero Contributors"
license: MIT
repository-code: "https://github.com/shifulegend/project-zero"
repository-code: "https://github.com/sherwin-casem/project-zero"
keywords:
- llm
- inference
Expand Down
8 changes: 4 additions & 4 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Thank you for your interest in contributing! Project Zero is a from-scratch CPU-

1. **Read [`GOLDEN_RULES.md`](GOLDEN_RULES.md)** — mandatory for all contributors and AI agents. No hardcoding, test after every change, document every step.
2. **Read [`DEVELOPER_ONBOARDING.md`](DEVELOPER_ONBOARDING.md)** — architecture overview, testing protocol, branching strategy.
3. **Check open Discussions** — especially [Discussion #1](https://github.com/shifulegend/project-zero/discussions/1) (MoE help wanted) and [Discussion #2](https://github.com/shifulegend/project-zero/discussions/2) (phase roadmap).
3. **Check open Discussions** — especially [Discussion #1](https://github.com/sherwin-casem/project-zero/discussions/1) (MoE help wanted) and [Discussion #2](https://github.com/sherwin-casem/project-zero/discussions/2) (phase roadmap).

---

Expand Down Expand Up @@ -60,8 +60,8 @@ The full phase roadmap is at [`.github/ROADMAP.md`](.github/ROADMAP.md). The two

| Area | Current state | Target | Discussion | Skill needed |
|---|---|---|---|---|
| **MoE expert weight repacking** | DeepSeek MoE at ~1 tok/s — 86% L3 miss rate due to non-contiguous expert offsets in GGUF | ≥ 9 tok/s | [Discussion #1](https://github.com/shifulegend/project-zero/discussions/1) | C, GGUF Q4_K layout, memory layout |
| **Native Q4_K matmul kernel** | Dense layers dequant Q4_K → FP32 before matmul | 4× speedup on DeepSeek dense layers | [Discussion #1](https://github.com/shifulegend/project-zero/discussions/1) | C, AVX-512 intrinsics |
| **MoE expert weight repacking** | DeepSeek MoE at ~1 tok/s — 86% L3 miss rate due to non-contiguous expert offsets in GGUF | ≥ 9 tok/s | [Discussion #1](https://github.com/sherwin-casem/project-zero/discussions/1) | C, GGUF Q4_K layout, memory layout |
| **Native Q4_K matmul kernel** | Dense layers dequant Q4_K → FP32 before matmul | 4× speedup on DeepSeek dense layers | [Discussion #1](https://github.com/sherwin-casem/project-zero/discussions/1) | C, AVX-512 intrinsics |

See [`MOE_RESEARCH_AND_FIX_PLAN.md`](docs/architecture/MOE_RESEARCH_AND_FIX_PLAN.md) for 8 previous fix attempts (P1–P8) with profiling data — read this before starting on expert repacking.

Expand Down Expand Up @@ -112,7 +112,7 @@ These phases are completely spec'd in [`IMPLEMENTATION_PLAN.md`](docs/architectu

## 💬 Questions?

Open a [Discussion](https://github.com/shifulegend/project-zero/discussions) — we prefer discussions over issues for questions.
Open a [Discussion](https://github.com/sherwin-casem/project-zero/discussions) — we prefer discussions over issues for questions.

---

Expand Down
14 changes: 7 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ Project Zero runs [PrismML's Ternary-Bonsai-27B](https://huggingface.co/prism-ml
Run it yourself — one binary, no Python (model download ~7.2 GB):

```bash
git clone https://github.com/shifulegend/project-zero && cd project-zero && make release
git clone https://github.com/sherwin-casem/project-zero && cd project-zero && make release
curl -fL -o models/Ternary-Bonsai-27B-Q2_0.gguf \
https://huggingface.co/prism-ml/Ternary-Bonsai-27B-gguf/resolve/main/Ternary-Bonsai-27B-Q2_0.gguf
./adaptive_ai_engine --model models/Ternary-Bonsai-27B-Q2_0.gguf \
Expand Down Expand Up @@ -160,7 +160,7 @@ All 16 screenshots (t=1..8 × 2 engines): [`benchmark_results/sweep_2026-06-21/s
|---|---|
| [![Xeon result](docs/openbenchmarking_xeon_vs_bitnetcpp.png)](https://openbenchmarking.org/result/2606207-SHIF-PROJECT42) | [![i5 result](docs/openbenchmarking_i5_vs_llamacpp.png)](https://openbenchmarking.org/result/2606208-SHIF-PROJECT03) |

**Run it yourself and post your result:** [Discussion #3 — community benchmarks](https://github.com/shifulegend/project-zero/discussions/3)
**Run it yourself and post your result:** [Discussion #3 — community benchmarks](https://github.com/sherwin-casem/project-zero/discussions/3)

### Community Baselines & Known Bottlenecks

Expand Down Expand Up @@ -356,7 +356,7 @@ Full RCA, methodology, and status: [`docs/ai/mistakes.md`](docs/ai/mistakes.md)
**Option A — pre-built binary (Linux x86-64, no compiler needed):**

```bash
wget https://github.com/shifulegend/project-zero/releases/download/v0.1.0/adaptive_ai_engine-0.1.0-x86_64-linux.tar.gz
wget https://github.com/sherwin-casem/project-zero/releases/download/v0.1.0/adaptive_ai_engine-0.1.0-x86_64-linux.tar.gz
tar xf adaptive_ai_engine-0.1.0-x86_64-linux.tar.gz
./adaptive_ai_engine --model models/bitnet-b1.58-2B-4T.bin \
--tokenizer models/bitnet-b1.58-2B-4T_tokenizer_proper.bin \
Expand All @@ -366,7 +366,7 @@ tar xf adaptive_ai_engine-0.1.0-x86_64-linux.tar.gz
**Option B — build from source (60 seconds):**

```bash
git clone https://github.com/shifulegend/project-zero.git
git clone https://github.com/sherwin-casem/project-zero.git
cd project-zero
make demo # builds engine + downloads SmolLM2-135M + runs a test prompt
```
Expand All @@ -383,7 +383,7 @@ No GPU. No Python at runtime. No API key. GCC or Clang + `make` + `curl` — not

> If you like my work or if it helped you, buy me a coffee or A100😉
>
> [![Sponsor Nitika](https://img.shields.io/badge/Sponsor-%E2%9D%A4-red)](https://github.com/sponsors/shifulegend)
> [![Sponsor Nitika](https://img.shields.io/badge/Sponsor-%E2%9D%A4-red)](https://github.com/sponsors/sherwin-casem)

## Help Wanted

Expand All @@ -393,10 +393,10 @@ Two open problems where outside expertise would make a real difference:
|---|---|---|
| **MoE expert weight repacking** | DeepSeek-V2-Lite runs at 1.90 tok/s — 7× behind `llama.cpp`. Top-K expert weights sit at non-contiguous GGUF offsets: **~86% L3 cache miss rate per token**. Fix: repack selected expert weights into contiguous memory at load time, matching llama.cpp's interleaved layout. | ≥ 9 tok/s |
| **Native Q4_K matmul kernel** | Current dense-model path dequants Q4_K → F32 before multiply. A fused mixed-precision kernel would close the remaining gap to `llama.cpp` on dense 4-bit GGUF models. | — |
| **Re-benchmark classifier INT8/INT4 throughput post-quality-fix** | `matmul_i4_task`/`matmul_i8_task` (`src/math/parallel_matmul.c`) were rewritten 2026-07-31 to fix a real quantization-quality gap (one scale per row → one scale per 32-element block with error-minimizing search — see `docs/ai/decision-log.md`, [GitHub issue #27](https://github.com/shifulegend/project-zero/issues/27)). That rewrite dropped the VNNI `dpbusds`/VBMI fast paths (their row-wide bias-correction trick doesn't hold once every block has its own scale) in favor of a portable per-block FMA path — the specific SSE-interleave-unpack throughput issue this row used to describe no longer applies to the current code, but the *net* throughput effect of the whole rewrite hasn't been re-measured against the historical sweep above. | Confirm INT8/INT4 throughput post-fix; a genuine multi-row/multi-block SIMD kernel (not full VNNI dpbusds, which the new per-block scaling structurally can't reuse) could recover some of the dropped throughput without regressing quality |
| **Re-benchmark classifier INT8/INT4 throughput post-quality-fix** | `matmul_i4_task`/`matmul_i8_task` (`src/math/parallel_matmul.c`) were rewritten 2026-07-31 to fix a real quantization-quality gap (one scale per row → one scale per 32-element block with error-minimizing search — see `docs/ai/decision-log.md`, [GitHub issue #27](https://github.com/sherwin-casem/project-zero/issues/27)). That rewrite dropped the VNNI `dpbusds`/VBMI fast paths (their row-wide bias-correction trick doesn't hold once every block has its own scale) in favor of a portable per-block FMA path — the specific SSE-interleave-unpack throughput issue this row used to describe no longer applies to the current code, but the *net* throughput effect of the whole rewrite hasn't been re-measured against the historical sweep above. | Confirm INT8/INT4 throughput post-fix; a genuine multi-row/multi-block SIMD kernel (not full VNNI dpbusds, which the new per-block scaling structurally can't reuse) could recover some of the dropped throughput without regressing quality |

Existing SIMD work documented in [`docs/KERNEL_INTERNALS.md`](docs/KERNEL_INTERNALS.md).
MoE repacking thread: [Discussion #1](https://github.com/shifulegend/project-zero/discussions/1)
MoE repacking thread: [Discussion #1](https://github.com/sherwin-casem/project-zero/discussions/1)

---

Expand Down
2 changes: 1 addition & 1 deletion SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@

**Please do NOT open a public GitHub issue for security vulnerabilities.**

To report a security vulnerability, please open a **[GitHub Security Advisory](https://github.com/shifulegend/project-zero/security/advisories/new)** (private disclosure).
To report a security vulnerability, please open a **[GitHub Security Advisory](https://github.com/sherwin-casem/project-zero/security/advisories/new)** (private disclosure).

Include:
- Description of the vulnerability
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# PR #31 verification — `--moe-threading=rowsplit` vs `=legacy`

**Date:** 2026-07-22 · **PR under test:** [#31](https://github.com/shifulegend/project-zero/pull/31) (`Amlaach/project-zero@fba3225`)
**Date:** 2026-07-22 · **PR under test:** [#31](https://github.com/sherwin-casem/project-zero/pull/31) (`Amlaach/project-zero@fba3225`)

## Setup

Expand Down
2 changes: 1 addition & 1 deletion docs/GROWTH_STRATEGY.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ Project Zero is a pure-C CPU LLM engine. On an Intel Xeon (2.10 GHz, AVX-512 VNN
the same binary hits 40.4 tok/s on BitNet b1.58-2B-4T and 142 tok/s on SmolLM2-135M.

Run it on YOUR CPU and drop your numbers here:
git clone https://github.com/shifulegend/project-zero && cd project-zero && make demo
git clone https://github.com/sherwin-casem/project-zero && cd project-zero && make demo

Please include: CPU model, core/thread count, RAM type, the `[gen] … tok/s` line,
and the auto-detected SIMD backend. I'll keep a leaderboard table in this thread.
Expand Down
2 changes: 1 addition & 1 deletion docs/KERNEL_INTERNALS.md
Original file line number Diff line number Diff line change
Expand Up @@ -155,7 +155,7 @@ At model load, sort and interleave expert weights so that any set of top-K activ

The challenge with Q4_K: each "superblock" in Q4_K quantization contains 256 weights with a shared scale+offset. Repacking must preserve superblock boundaries or recompute scales. See `ggml_type_size(GGML_TYPE_Q4_K)` and the `block_q4_K` struct in ggml-quants.h for the layout.

**Discussion with more context:** https://github.com/shifulegend/project-zero/discussions/1
**Discussion with more context:** https://github.com/sherwin-casem/project-zero/discussions/1

If you've worked on ggml Q4_K layout or llama.cpp's expert repacking, your input there would be extremely valuable.

Expand Down
Binary file modified docs/QA_Testing_Strategy_Report.pdf
Binary file not shown.
2 changes: 1 addition & 1 deletion docs/ai/change-trace.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
> Update after each meaningful sub-step. Last updated: 2026-08-04.

### 2026-08-04 — DeepSeek MoE benchmark run: project-zero vs colibri vs llama.cpp
- What: user asked to run DeepSeek MoE on project-zero, colibri (`shifulegend/colibri`), and
- What: user asked to run DeepSeek MoE on project-zero, colibri (`sherwin-casem/colibri`), and
llama.cpp and report results. Found colibri has no DeepSeek-arch loader (only GLM-5.2/Inkling/
Kimi K3/OLMoE, one engine per family — DeepSeek referenced only as prior-art in `colibri.c`)
and project-zero has no OLMoE-arch loader (`gguf_loader.c`'s MoE dispatch only special-cases
Expand Down
2 changes: 1 addition & 1 deletion docs/reports/PR21_K6_LUT_KERNEL_REVIEW.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# PR #21 Review: K-6 LUT Ternary Kernel — Bug Fix & Benchmark Report

**PR:** [shifulegend/project-zero#21](https://github.com/shifulegend/project-zero/pull/21) — adds a new
**PR:** [sherwin-casem/project-zero#21](https://github.com/sherwin-casem/project-zero/pull/21) — adds a new
5-trit-per-byte ("K-6 LUT") AVX-512BW ternary matmul kernel for BitNet-style models.
**Reviewed:** 2026-07-04 · **Fix commit:** `8a4905f` on `bench/lut-k6-20260627`
**Test rig:** this session's sandbox — Intel Xeon Scalable (cloud vCPU), 4 cores, AVX-512 VNNI + AVX-512BW (see hardware table below).
Expand Down
2 changes: 1 addition & 1 deletion docs/reports/SECURITY_FINDINGS.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Security and Functional Test Report: project-zero

**Repository:** https://github.com/shifulegend/project-zero.git
**Repository:** https://github.com/sherwin-casem/project-zero.git
**Assessment Date:** 2026-03-12 (updated)
**Branch assessed:** `master`
**HEAD commit:** See latest commit on branch
Expand Down
2 changes: 1 addition & 1 deletion docs/reports/qa_report.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
---

**Prepared by:** Independent Principal QA Architect / Senior Software Security Auditor
**Repository:** https://github.com/shifulegend/project-zero
**Repository:** https://github.com/sherwin-casem/project-zero
**Language / Runtime:** Pure C99, POSIX + Win32 abstraction, zero external runtime dependencies
**Assessed Phases:** 0–9 (Phases 10+ planned but not implemented)
**Policy:** Zero-Trust — all tests designed from first principles; no existing test artefacts consulted
Expand Down
4 changes: 2 additions & 2 deletions docs/reports/test_execution_report.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Test Execution Report — project-zero
**Status:** [TO FIX] — Unmitigated risks identified
**Repository:** https://github.com/shifulegend/project-zero
**Repository:** https://github.com/sherwin-casem/project-zero
**Engine:** CPU LLM Ternary Inference Engine (C99)
**Execution Date:** March 13, 2026
**Policy:** Zero-Trust — all tests written from scratch, existing tests treated as untrusted
Expand Down Expand Up @@ -293,4 +293,4 @@ It is registered in CMake and runs as test #19 under `ctest`.
---

*End of Report — Prepared under Zero-Trust policy. Total: 3,367 tests executed, 3,367 passed, 0 failed, 0 ASAN/UBSAN violations.*
*Source: https://github.com/shifulegend/project-zero*
*Source: https://github.com/sherwin-casem/project-zero*
Loading