Skip to content

beta-loop.md

Latest commit

 

History

History
100 lines (70 loc) · 2.37 KB

beta-loop.md

File metadata and controls

100 lines (70 loc) · 2.37 KB
Raw

BeforeShip Beta Loop

Use this loop with every beta tester or external repository. The goal is to learn where BeforeShip is useful, noisy, or unclear.

1. Pick a Real App

Good beta targets:

  • AI-built SaaS apps close to launch
  • Next.js apps with auth, payments, database access, or env files
  • Supabase, Neon/Postgres, Stripe, Razorpay, Clerk, Auth.js, or Better Auth projects
  • Apps the owner is comfortable scanning locally

Avoid asking users to upload private source code. BeforeShip should run locally in their repo.

2. Run the Local Gate

npx beforeship@latest scan .

Ask the tester to open:

.beforeship/launch-report.html

Then ask them to run:

npx beforeship@latest fix-prompts .

3. Try the CI Gate

Copy .github/workflows/beforeship.yml into the target repository, or add this minimal workflow:

name: BeforeShip

on:
  pull_request:
  workflow_dispatch:

jobs:
  launch-gate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 22
      - run: npx beforeship@latest scan . --no-color --fail-on high --min-score 90

Use --fail-on critical for a softer first rollout, or --fail-on medium --min-score 95 for stricter teams.

For private apps, keep reports as local artifacts unless the owner is comfortable uploading them to GitHub Actions.

4. Capture Feedback

Use this template for each test:

Repo stack:
Command run:
Score/verdict:
Top findings:

True positives:
False positives:
Missed issues:
Confusing wording:
Useful fix prompt? yes/no
HTML report useful? yes/no
Would you add this to CI? yes/no

One thing that would make this more useful:

5. Triage the Result

Turn feedback into one of these outcomes:

  • detector: scanner missed or misread a code pattern
  • wording: finding was valid but unclear or too scary
  • report: HTML/CLI output made the result hard to use
  • docs: user did not know what to run next
  • new-check: beta repo exposed a launch risk not currently covered

Prioritize issues that affect multiple real apps before adding broad new checks.

6. Repeat

Run the loop against 5-10 repositories before the next release. A good release candidate should have:

  • fewer repeated false positives
  • clearer fix prompts
  • stable CI behavior
  • at least one external user who would keep it in their workflow