Field hardening: link graph fix, weekly lint, nightly backup + embed split, reflection retry

CHANGELOG.md

@@ -6,7 +6,13 @@ Format inspired by [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 
 ## [Unreleased]
 
-_Nothing yet — open an issue to suggest._
+### Added
+- **Weekly lint** (`engine/nightly/gbrain-lint.sh`, launchd Monday 08:00) — verify-and-surface pass over the whole pipeline: doctor, vault lint, orphans, anomalies, back-links, stats, and a "did the nightly actually run in the last 48h" check. 🟢/🟠/🔴 verdict pinned in `Profile/memory.md` (idempotent marker block), full report in `Profile/lint.md`. Pure CLI, no LLM — a silently-failing maintenance job no longer looks healthy.
+- **Link graph actually builds now** — `link_resolution.global_basename` is enabled at install (and idempotently by the nightly for existing installs). Without it, every skeleton dir (`Team/`, `Agents/`, `Decisions/`, `Skills/`, `Journal/`…) is outside gbrain's entity-dir whitelist and all wikilinks were silently dropped: empty graph, forever. Field-tested on a 562-page vault: 0 → 185 edges. Skeleton `CLAUDE.md`s document the convention: bare-basename wikilinks (`[[Page-Name]]`, no path, no `.md`) + a short `## See also` per page.
+
+### Changed
+- **Nightly**: vault is pushed to its git remote after the nightly commit (best-effort, never blocks — local commits are worth little if the disk dies). Sync and embed are now split (`sync --no-embed` + `embed --stale`): sync's inline embed path fails against `zembed-1` with a misleading parse error and silently stops ingesting the vault.
+- **Reflection**: the headless `claude -p` run is retried (3 attempts, 60s apart) — transient API failures ("Connection closed mid-response") were silently losing whole days of journal. Failures are logged to `daily-reflection-errors.log`.
 
 ## [0.1.0] — 2026-05-26 — Initial public release
 

README.md

@@ -13,6 +13,7 @@ Everyone gets the **same engine**, with **their own data** — everything stays
 | **GBrain** (`~/.gbrain`) | Semantic search over the vault (`gbq query "..."`) — ZeroEntropy embeddings |
 | **Nightly** (launchd 04:00) | commit vault → sync → dream cycle (dedup, facts, consolidation) → self-update |
 | **Reflection** (launchd 12:00 + 23:00) | LLM summary of the day's sessions → `Journal/` + rolling 15-day `memory.md` |
+| **Weekly lint** (launchd Monday 08:00) | Verifies the whole pipeline (doctor, lint, orphans, nightly freshness) → 🟢/🟠/🔴 verdict pinned in `memory.md` + report in `Profile/lint.md` |
 
 It all runs inside Claude Code (terminal/IDE). No bot, no server, no shared cloud.
 

company-skeleton/CLAUDE.md

@@ -25,5 +25,6 @@ Each member's nightly `git pull`s this vault + re-indexes. The dream cycle dedup
 
 ## Rules
 - **Zero secrets** (shared vault + on GitHub). Locations only.
+- Link related pages with bare-basename wikilinks (`[[Page-Name]]`, no path, no `.md`) — that's what builds the shared graph. End substantial pages with a short `## See also`.
 - 1 decision = 1 dated file in `Decisions/` (`YYYY-MM-DD-topic.md`), clear title, the "why".
 - Factual, no personal drafts (those go in your personal vault `~/Documents/Brain`).

engine/hooks/daily-reflection.py

@@ -54,20 +54,31 @@
 {big}
 """
 
-try:
-    home_claude = Path.home() / ".local" / "bin" / "claude"
-    claude_bin = (
-        os.environ.get("CLAUDE_BIN")
-        or (str(home_claude) if home_claude.exists() else None)
-        or shutil.which("claude")
-        or str(home_claude)
-    )
-    subprocess.run(
-        [claude_bin, "-p", "--permission-mode", "acceptEdits", prompt],
-        cwd=str(BRAIN),
-        timeout=600,
-        check=False,
-    )
-except Exception as e:
-    (LOGS / "daily-reflection-errors.log").open("a").write(f"{time.strftime(chr(37)+chr(70)+chr(84)+chr(37)+chr(84))} {e}" + chr(10))
+home_claude = Path.home() / ".local" / "bin" / "claude"
+claude_bin = (
+    os.environ.get("CLAUDE_BIN")
+    or (str(home_claude) if home_claude.exists() else None)
+    or shutil.which("claude")
+    or str(home_claude)
+)
+# Transient API failures ("Connection closed mid-response") kill headless
+# claude runs often enough that an unretried cron silently loses whole days
+# of journal. 3 attempts, 60s apart; a non-zero exit counts as a failure.
+for attempt in range(3):
+    try:
+        r = subprocess.run(
+            [claude_bin, "-p", "--permission-mode", "acceptEdits", prompt],
+            cwd=str(BRAIN),
+            timeout=600,
+            check=False,
+        )
+        if r.returncode == 0:
+            break
+        err = f"exit {r.returncode}"
+    except Exception as e:
+        err = f"{type(e).__name__}: {e}"
+    (LOGS / "daily-reflection-errors.log").open("a").write(
+        f"{time.strftime(chr(37)+chr(70)+chr(84)+chr(37)+chr(84))} attempt {attempt + 1}/3 failed: {err}" + chr(10))
+    if attempt < 2:
+        time.sleep(60)
 sys.exit(0)

engine/nightly/com.USER.gbrain-lint.plist.template

@@ -0,0 +1,28 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<plist version="1.0">
+<dict>
+    <key>Label</key>
+    <string>com.__USER__.gbrain-lint</string>
+    <key>ProgramArguments</key>
+    <array>
+        <string>/bin/zsh</string>
+        <string>__HOME__/.claude/hooks/brain/gbrain-lint.sh</string>
+    </array>
+    <key>StartCalendarInterval</key>
+    <dict>
+        <key>Weekday</key>
+        <integer>1</integer>
+        <key>Hour</key>
+        <integer>8</integer>
+        <key>Minute</key>
+        <integer>0</integer>
+    </dict>
+    <key>RunAtLoad</key>
+    <false/>
+    <key>StandardOutPath</key>
+    <string>__HOME__/.gbrain/lint.launchd.out</string>
+    <key>StandardErrorPath</key>
+    <string>__HOME__/.gbrain/lint.launchd.err</string>
+</dict>
+</plist>

engine/nightly/gbrain-lint.sh

@@ -0,0 +1,178 @@
+#!/bin/zsh
+# brain-in-a-box weekly lint (launchd: com.<user>.gbrain-lint, Monday 08:00).
+#
+# The nightly MAINTAINS the brain (sync/dream) but writes to a log nobody
+# reads — a silently-failing maintenance job looks exactly like a healthy one.
+# This job VERIFIES and surfaces the result where it gets read:
+#   - <vault>/Profile/lint.md  : full report (replaced every run)
+#   - <vault>/Profile/memory.md : one-line verdict between <!-- LINT --> markers
+# No LLM pass — pure CLI, immune to transient API failures. Read-only on the
+# vault except those two files.
+
+export PATH="$HOME/.bun/bin:/opt/homebrew/bin:/usr/bin:/bin"
+GBRAIN="$HOME/.bun/bin/gbrain"
+VAULT="$HOME/Documents/Brain"
+LOG="$HOME/.gbrain/lint.log"
+TMP=$(mktemp -d); trap 'rm -rf "$TMP"' EXIT
+
+# doctor is cwd-sensitive (its resolver check targets the vault's skills)
+cd "$VAULT" || exit 1
+
+echo "===== $(date '+%F %T') lint start =====" >> "$LOG"
+
+# last week's report quotes raw issues → lint would recount them; remove it
+# before the pass (regenerated at the end of the run)
+rm -f "$VAULT/Profile/lint.md"
+
+"$GBRAIN" doctor --fast --json           > "$TMP/doctor.json"   2>>"$LOG"
+"$GBRAIN" lint "$VAULT"                  > "$TMP/lint.txt"      2>&1
+"$GBRAIN" orphans --count                > "$TMP/orphans.txt"   2>>"$LOG"
+"$GBRAIN" anomalies --since 7d           > "$TMP/anomalies.txt" 2>&1
+"$GBRAIN" check-backlinks check "$VAULT" > "$TMP/backlinks.txt" 2>&1
+"$GBRAIN" stats                          > "$TMP/stats.txt"     2>&1
+grep -E 'nightly (start|done)' "$HOME/.gbrain/nightly.log" 2>/dev/null | tail -6 > "$TMP/nightly.txt"
+
+python3 - "$TMP" "$VAULT" <<'PY' >> "$LOG" 2>&1
+import json, re, sys
+from datetime import datetime
+from pathlib import Path
+
+tmp, vault = Path(sys.argv[1]), Path(sys.argv[2])
+today = datetime.now().strftime("%Y-%m-%d")
+
+def read(name):
+    p = tmp / name
+    return p.read_text(errors="replace") if p.exists() else ""
+
+# --- doctor ---
+doctor_status, doctor_score, doctor_fails = "unknown", "?", []
+try:
+    d = json.loads(read("doctor.json"))
+    doctor_status = d.get("status", "unknown")
+    doctor_score = d.get("health_score", "?")
+    doctor_fails = [c["name"] for c in d.get("checks", []) if c.get("status") == "fail"]
+except Exception:
+    pass
+
+# --- lint ---
+lint_txt = read("lint.txt")
+m = re.search(r"(\d+) pages scanned\. (\d+) issue\(s\)", lint_txt)
+lint_pages, lint_issues = (int(m.group(1)), int(m.group(2))) if m else (0, -1)
+
+# --- orphans ---
+m = re.search(r"^(\d+)$", read("orphans.txt"), re.M)
+orphans = int(m.group(1)) if m else -1
+
+# --- anomalies / backlinks ---
+anom_txt = read("anomalies.txt").strip()
+anomalies_clean = "(no anomalies" in anom_txt or not anom_txt
+back_txt = read("backlinks.txt")
+backlinks_clean = "No missing back-links" in back_txt
+
+# --- nightly freshness ---
+nightly_txt = read("nightly.txt")
+last_done, nightly_age_h = None, None
+for line in nightly_txt.splitlines():
+    m = re.search(r"(\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}) nightly done", line)
+    if m:
+        last_done = m.group(1)
+if last_done:
+    nightly_age_h = (datetime.now() - datetime.strptime(last_done, "%Y-%m-%d %H:%M:%S")).total_seconds() / 3600
+
+# --- stats ---
+stats_txt = read("stats.txt").strip()
+m = re.search(r"Links:\s+(\d+)", stats_txt)
+links = int(m.group(1)) if m else -1
+m = re.search(r"Pages:\s+(\d+)", stats_txt)
+db_pages = int(m.group(1)) if m else -1
+
+# --- verdict ---
+red = (nightly_age_h is None or nightly_age_h > 48) or (isinstance(doctor_score, int) and doctor_score < 50) or lint_issues < 0
+orange = doctor_status != "healthy" or lint_issues > 0 or not backlinks_clean or not anomalies_clean
+verdict = "🔴" if red else ("🟠" if orange else "🟢")
+
+reasons = []
+if nightly_age_h is None:
+    reasons.append("nightly: no 'done' entry found in the log")
+elif nightly_age_h > 48:
+    reasons.append(f"nightly DEAD for {nightly_age_h:.0f}h")
+else:
+    reasons.append(f"nightly OK (last run {last_done})")
+reasons.append(f"doctor {doctor_status} ({doctor_score}/100" + (f", fail: {', '.join(doctor_fails)}" if doctor_fails else "") + ")")
+if lint_issues >= 0:
+    reasons.append(f"{lint_issues} frontmatter/artifact issue(s) across {lint_pages} pages")
+reasons.append(f"{orphans} orphan pages / {db_pages} in DB, {links} links")
+if not anomalies_clean:
+    reasons.append("statistical anomalies detected (see report)")
+if not backlinks_clean:
+    reasons.append("missing back-links (see report)")
+
+summary = " · ".join(reasons)
+
+# --- full report -> Profile/lint.md ---
+report = f"""---
+title: Weekly brain lint
+created: {today}
+type: report
+---
+
+# Weekly lint — {today} {verdict}
+
+{summary}
+
+Generated by engine/nightly/gbrain-lint.sh (launchd, Monday 08:00).
+Raw output of each pass below.
+
+## doctor --fast
+```json
+{read("doctor.json").strip()}
+```
+
+## lint (frontmatter, LLM artifacts, placeholder dates)
+Bulk-fixable: `gbrain lint <vault> --fix`
+```
+{lint_txt.strip()[-4000:]}
+```
+
+## anomalies (7 days)
+```
+{anom_txt or "(empty)"}
+```
+
+## back-links
+```
+{back_txt.strip()[-1500:]}
+```
+
+## stats
+```
+{stats_txt}
+```
+
+## nightly (recent runs)
+```
+{nightly_txt.strip() or "(nightly log not found)"}
+```
+"""
+(vault / "Profile" / "lint.md").write_text(report)
+
+# --- verdict block -> Profile/memory.md (between markers, idempotent) ---
+mem_path = vault / "Profile" / "memory.md"
+block = (
+    "<!-- LINT:BEGIN -->\n"
+    f"> **🧹 Weekly lint {today} {verdict}** — {summary}. Detail: `Profile/lint.md`.\n"
+    "<!-- LINT:END -->"
+)
+mem = mem_path.read_text() if mem_path.exists() else "# Memory\n"
+if "<!-- LINT:BEGIN -->" in mem:
+    mem = re.sub(r"<!-- LINT:BEGIN -->.*?<!-- LINT:END -->", block, mem, flags=re.S)
+else:
+    lines = mem.splitlines(keepends=True)
+    lines.insert(min(2, len(lines)), "\n" + block + "\n")
+    mem = "".join(lines)
+mem_path.write_text(mem)
+
+print(f"lint {verdict} — {summary}")
+PY
+
+echo "===== $(date '+%F %T') lint done =====" >> "$LOG"

engine/nightly/gbrain-nightly.sh

@@ -63,13 +63,34 @@ if [ -d "$GSTACK/.git" ] && cd "$GSTACK" 2>/dev/null; then
     fi
 fi
 
+# 0ter. Link-graph resolution: idempotent, upgrades installs that predate the
+# flag (see install.sh — without it every skeleton dir is outside gbrain's
+# entity-dir whitelist and wikilinks are silently dropped: empty graph).
+"$GBRAIN" config set link_resolution.global_basename true >> "$LOG" 2>&1
+
 # 1. Commit the personal vault first (sync is git-diff based → without a commit, edits are invisible).
 if cd "$VAULT" 2>/dev/null && [ -n "$(git status --porcelain 2>/dev/null)" ]; then
     git add -A >> "$LOG" 2>&1
     git -c user.email="brain@local" -c user.name="brain" commit -q -m "nightly $(date '+%Y-%m-%d')" >> "$LOG" 2>&1 \
         && echo "[git] personal vault committed" >> "$LOG"
 fi
-"$GBRAIN" sync --repo "$VAULT" --no-pull >> "$LOG" 2>&1
+# 1bis. Back up the vault off-machine. The vault is the only copy otherwise
+# (local commits aren't worth much if the disk dies). Best-effort: a remote may
+# be absent and a cron may lack creds — never block the cycle.
+# GIT_TERMINAL_PROMPT=0 so a missing credential fails fast instead of hanging.
+if cd "$VAULT" 2>/dev/null && git remote get-url origin >/dev/null 2>&1; then
+    GIT_TERMINAL_PROMPT=0 git push origin HEAD >> "$LOG" 2>&1 \
+        && echo "[git] vault pushed to origin" >> "$LOG" \
+        || echo "[git] vault push skipped (no remote creds in cron?)" >> "$LOG"
+fi
+# Import and embed are split on purpose (2026-07-16). `sync` with its built-in
+# embed fails on most files with "[embed(zeroentropyai:zembed-1)] Invalid JSON
+# response", reported as "N file(s) failed to parse" — a lie: the parse is fine.
+# Only sync's inline embed path fails; the same texts embed cleanly on their
+# own. Left unsplit, the RAG silently stops ingesting the vault for weeks.
+# Re-test `sync` alone after a gbrain upgrade; drop this split once fixed.
+"$GBRAIN" sync --repo "$VAULT" --no-pull --no-embed >> "$LOG" 2>&1
+"$GBRAIN" embed --stale >> "$LOG" 2>&1
 
 # 2. COMPANY vault (team mode): pull teammates' contributions + sync the 'company' source.
 if [ -d "$VAULT_CO/.git" ]; then

install.sh

@@ -84,9 +84,10 @@ say "gbq + nightly"
 mkdir -p "$BIN"
 sed "s#__HOME__#$H#g" "$REPO/engine/bin/gbq" > "$BIN/gbq" && chmod +x "$BIN/gbq"; ok "gbq → $BIN/gbq"
 sed "s#__HOME__#$H#g" "$REPO/engine/nightly/gbrain-nightly.sh" > "$HOOKS/gbrain-nightly.sh" && chmod +x "$HOOKS/gbrain-nightly.sh"; ok "gbrain-nightly.sh"
+sed "s#__HOME__#$H#g" "$REPO/engine/nightly/gbrain-lint.sh" > "$HOOKS/gbrain-lint.sh" && chmod +x "$HOOKS/gbrain-lint.sh"; ok "gbrain-lint.sh"
 
-# ── 7. launchd jobs (nightly maintenance + daily reflection) ─────────────────
-say "launchd (nightly 04:00 + reflection 12:00/23:00)"
+# ── 7. launchd jobs (nightly maintenance + daily reflection + weekly lint) ───
+say "launchd (nightly 04:00 + reflection 12:00/23:00 + lint Monday 08:00)"
 mkdir -p "$H/Library/LaunchAgents"
 load_agent() {  # <label> <template>
   local label="$1" tpl="$2" plist="$H/Library/LaunchAgents/$1.plist"
@@ -97,6 +98,7 @@ load_agent() {  # <label> <template>
 }
 load_agent "com.$U.gbrain-nightly"   "com.USER.gbrain-nightly.plist.template"
 load_agent "com.$U.brain-reflection" "com.USER.brain-reflection.plist.template"
+load_agent "com.$U.gbrain-lint"      "com.USER.gbrain-lint.plist.template"
 
 # ── 8. settings.json (merge hooks, non-destructive) ─────────────────────────
 say "Registering hooks (~/.claude/settings.json)"
@@ -164,6 +166,13 @@ os.chmod(p, 0o600)
 print("  ok ZE key written (config.json, 600)")
 PY
 "$GBQ_BIN" config set search.mode balanced >/dev/null 2>&1 || true
+# Link graph: gbrain's extractor only auto-recognizes English "entity" dirs
+# (people/, companies/, projects/…). EVERY skeleton dir (Team/, Agents/,
+# Decisions/, Skills/, Journal/…) is outside that whitelist, so without this
+# flag all wikilinks are silently dropped and the graph stays empty forever.
+# Basename resolution links [[Page-Name]] to the page whose filename matches
+# (case-insensitive), regardless of folder.
+"$GBQ_BIN" config set link_resolution.global_basename true >/dev/null 2>&1 || true
 "$GBQ_BIN" import "$BRAIN" --no-embed >/dev/null 2>&1 && ok "vault imported"
 say "Embedding (may take 1-2 min)…"
 "$GBQ_BIN" embed --stale >/dev/null 2>&1 && ok "embedded" || warn "re-run embed: gbrain embed --stale"

vault-skeleton/CLAUDE.md

@@ -55,6 +55,19 @@ Always use `gbq` — it's the safe universal wrapper for every gbrain command (r
 | `Clients/` | Actifs/, Prospects/ |
 | `Resources/` | Templates/, reusable |
 
+## Wikilinks — how the graph gets built
+
+Link pages with **bare-basename wikilinks**: `[[Page-Name]]` — no folder path, no
+`.md`, case-insensitive. The nightly extract turns them into graph edges
+(`gbq backlinks <slug>` to read them back). Path-qualified links
+(`[[Projects/Page-Name]]`) break resolution on some write paths — avoid them.
+
+When you write or update a page, end it with a short `## See also` section
+(2-5 wikilinks to genuinely related pages, one short reason each). A page
+nobody links to is invisible to graph traversal. Few good links > many weak.
+
+---
+
 ## Routing — where to write
 - Correction → `Profile/lessons.md` (Rule #1)
 - Meaningful decision worth answering "why X?" in 6 months → `Decisions/YYYY-MM-DD-<slug>.md`