[docs-noob-tester] 📚 Documentation Noob Test Report - 2026-05-05

[github-actions[bot]]

Summary

• Date of test: 2026-05-05
• Pages visited:
  • https://github.github.com/gh-aw/ (Home)
  • https://github.github.com/gh-aw/setup/quick-start/ (Quick Start)
  • https://github.github.com/gh-aw/setup/cli/ (CLI Commands)
• Overall impression: The site is visually polished and the Quick Start guide has a clear 4-step flow, but a newcomer who isn't already familiar with GitHub CLI extensions, GitHub Actions, and secret management will hit several friction points before getting their first workflow running.

---

🔴 Critical Issues (Blocking for Beginners)

1. "Authentication" link in prerequisites is a dead end without context

The Quick Start Step 2 references "See Authentication for setup instructions" when describing token setup, but a beginner has no idea what COPILOT_GITHUB_TOKEN is (a separate GitHub token) vs the regular GITHUB_TOKEN. The distinction between these two tokens is never explained on the Quick Start page itself — the user must click away before they can proceed. Many beginners will be confused or stall here.

2. add-wizard command requires the user to know a workflow reference up-front

gh aw add-wizard githubnext/agentics/daily-repo-status

The <owner>/<repo>/<workflow-name> format is only briefly explained inline. There's no "how do I discover available workflows?" link nearby. A beginner may wonder: "What other workflows are there? Where do I find them?". The only pointer is "Explore Peli's Agent Factory" at the very bottom, buried in the What's next? section.

3. "Step 3 - Wait for the workflow to complete" has a visual placeholder

The content says "The report will look something like this:" but there is no image or example shown after that sentence. It appears a screenshot or code block was intended but never added. This leaves a blank-looking gap and breaks beginner trust.

📎 [quick-start.png] — Full Quick Start page:

---

🟡 Confusing Areas (Slow Down Learning)

4. Home page: "Continuous AI" jargon with no quick definition

The hero section uses "Continuous AI" prominently, but never defines it inline. A reader coming from CI/CD background may conflate it with Continuous Integration. There's a link to the FAQ, but beginners rarely click FAQ first.

📎 [home.png] — Home page hero:

5. Prerequisites list: GitHub Actions enablement check is vague

"GitHub Actions enabled — Check in Settings → Actions"

This is fine for an experienced GitHub user, but a beginner may not know which level of settings (repo vs org), whether it's on by default, or what to do if it's disabled. A one-liner note would help.

6. CLI Commands page: Command table has no examples inline

The "Most Common Commands" table lists commands and descriptions but no examples. A beginner reading gh aw audit — "Debug a failed workflow run" — has no idea what arguments it expects. The details are further down the page but aren't linked.

📎 [cli.png] — CLI Commands page:

7. Quick Start Step 4 "Customize" — frontmatter vs body explanation is a wall of text

The note about "two parts: frontmatter and body" is useful, but it's presented as a dense paragraph block. First-time readers often skip over long Note boxes.

📎 [quick-start-top.png] — Quick Start prerequisites/step 1:

8. Top nav: "Peli's Agent Factory" link is unexplained

The primary navigation bar shows "Peli's Agent Factory" as a top-level link next to "Blog". New users have no idea what this is. It looks like a personal page that accidentally landed in the main nav.

---

🟢 What Worked Well

• Estimated time (10 minutes) at the top of Quick Start sets clear expectations. ✅
• Tip boxes (alternative install for auth issues, having trouble link) are well-placed. ✅
• Prerequisites list covers all the required tools with version hints. ✅
• 4-step structure of Quick Start is logical and well-numbered. ✅
• Security/guardrails section on the home page builds trust early. ✅
• CLI Commands "Most Common" table gives a great at-a-glance overview. ✅
• Search bar (Ctrl+K) is present in the header. ✅

---

Recommendations

Quick Wins 🚀

1. Add a missing example/screenshot after "The report will look something like this" in Quick Start Step 3
2. Add a one-sentence inline definition for "Continuous AI" on the home page (e.g., "Continuous AI: systematic, recurring application of AI agents to your repository")
3. Rename or explain "Peli's Agent Factory" in the nav — e.g., rename to "Workflow Gallery" or add a tooltip
4. Link "Authentication" to the specific section that explains the COPILOT_GITHUB_TOKEN vs GITHUB_TOKEN distinction, and add a brief note inline in Step 2

Medium-Term Improvements 📈

5. Add a workflow discovery page — a simple list or gallery of available add-wizard references so beginners know what's available
6. Convert the frontmatter/body explanation in Step 4 into a visual two-panel diagram or annotated code block
7. Add GitHub Actions enablement detail — link to the exact settings page or explain defaults
8. Add per-command examples in the CLI Commands table (or link directly to the example section for each command)

---

References:

• §25358370586

Warning

Firewall blocked 5 domains

The following domains were blocked by the firewall during workflow execution:

• accounts.google.com
• android.clients.google.com
• clients2.google.com
• safebrowsingohttpgateway.googleapis.com
• www.google.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "accounts.google.com"
    - "android.clients.google.com"
    - "clients2.google.com"
    - "safebrowsingohttpgateway.googleapis.com"
    - "www.google.com"

See Network Configuration for more information.

Generated by Documentation Noob Tester · ● 3.4M · ◷

• expires on May 6, 2026, 5:08 AM UTC