Skip to content

seo: noindex the username catch-all route - #2678

Closed
0xkkonrad wants to merge 2 commits into
devfrom
seo/noindex-username-catchall
Closed

seo: noindex the username catch-all route#2678
0xkkonrad wants to merge 2 commits into
devfrom
seo/noindex-username-catchall

Conversation

@0xkkonrad

@0xkkonrad 0xkkonrad commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Defect (T3, layer 1)

src/app/[...recipient]/page.tsx is the username catch-all. Its generateMetadata builds a full, indexable "X on Peanut" shell for any username-shaped string — USERNAME_PATTERN = /^[a-z][a-z0-9]{3,11}$/ in src/constants/routes.ts:155 is the only filter. Whether the account exists is checked client-side, after the HTML has already been served with HTTP 200.

Two consequences, both live on peanut.me right now:

$ curl -s https://peanut.me/kkonrad | grep -o '<meta name="robots"[^>]*>'
<meta name="robots" content="index, follow"/>
$ curl -s https://peanut.me/kkonrad | grep -o '<title>[^<]*</title>'
<title>kkonrad on Peanut</title>

$ curl -s -o /dev/null -w '%{http_code}\n' https://peanut.me/zzqx9      # nobody owns this handle
200
$ curl -s https://peanut.me/zzqx9 | grep -o '<meta name="robots"[^>]*>'
<meta name="robots" content="index, follow"/>
$ curl -s https://peanut.me/zzqx9 | grep -o '<title>[^<]*</title>'
<title>zzqx9 on Peanut</title>

So Google is offered an unbounded space of soft-404 "profile" pages, plus every real profile, address, ENS name and request/receipt link — all app surface, none of it a search landing page.

The index, follow comes from the root layout (src/app/layout.tsx:24, robots: IS_PRODUCTION_DOMAIN ? { index: true, follow: true } : …). That is why the empty return {} guard branches are not already safe: with nothing set at page level they inherit index, follow from the root.

Fix

robots: { index: false, follow: false } on all four generateMetadata return paths (count re-verified on current origin/dev @ ad5b61b6grep -n return shows exactly 3 early returns plus the metadata object):

line (pre-patch) branch
26 reserved route
31 no recipient segment
37 !couldBeRecipient(...)
160–186 full metadata object (profiles, addresses, ENS, amounts, receipts)

Extracted as one NOINDEX const so the four branches cannot drift apart.

Per the 12 Aug decision: noindex everything, no carve-outs — real profiles and .eth ENS pages included.

One file, +11/−3. The page component, routing, couldBeRecipient, and 404 behaviour are untouched.

Verification

A/B on a local server, same worktree, only the patch differing. Run with NEXT_PUBLIC_BASE_URL=https://peanut.me — without it the root layout noindexes every page locally and the evidence would be worthless.

Before (git checkout HEAD~1 -- src/app/[...recipient]/page.tsx):

/kkonrad                                     -> HTTP 200 | robots="index, follow"  | kkonrad on Peanut
/zzqx9                                       -> HTTP 200 | robots="index, follow"  | zzqx9 on Peanut
/vitalik.eth                                 -> HTTP 200 | robots="index, follow"  | vitalik.eth is requesting funds
/kkonrad/1usdc                               -> HTTP 200 | robots="index, follow"  | kkonrad is requesting $1 via Peanut
/0x1234567890abcdef1234567890abcdef12345678  -> HTTP 200 | robots="index, follow"  | 0x1234...345678 is requesting funds
/pricing                                     -> HTTP 200 | robots="index, follow"  | pricing on Peanut
/es                                          -> HTTP 200 | robots="noindex"        | (404 page)
/some-random-slug-here                       -> HTTP 200 | robots="noindex"        | (404 page)
/en/send-money-to/brazil                     -> HTTP 200 | robots="index, follow"  | Send Money to Brazil — No IOF Tax | Peanut

After (this branch):

/kkonrad                                     -> HTTP 200 | robots="noindex, nofollow" | kkonrad on Peanut
/zzqx9                                       -> HTTP 200 | robots="noindex, nofollow" | zzqx9 on Peanut
/vitalik.eth                                 -> HTTP 200 | robots="noindex, nofollow" | vitalik.eth is requesting funds
/kkonrad/1usdc                               -> HTTP 200 | robots="noindex, nofollow" | kkonrad is requesting $1 via Peanut
/0x1234567890abcdef1234567890abcdef12345678  -> HTTP 200 | robots="noindex, nofollow" | 0x1234...345678 is requesting funds
/pricing                                     -> HTTP 200 | robots="noindex, nofollow" | pricing on Peanut
/es                                          -> HTTP 200 | robots="noindex"           | (404 page, unchanged)
/some-random-slug-here                       -> HTTP 200 | robots="noindex"           | (404 page, unchanged)
/en/send-money-to/brazil                     -> HTTP 200 | robots="index, follow"     | Send Money to Brazil — No IOF Tax | Peanut

/en/send-money-to/brazil is the control: a real content page, still index, follow in both runs. The blast radius is the catch-all only.

/pricing moving to noindex, nofollow is the intended read of the decision — it is a catch-all shell today, not a real pricing page (isReservedRoute('/pricing') is false; the server-side username list rejects it, which is why nobody owns it). Giving it a real page is PR4's job, and that PR will set its own metadata.

Gates:

typecheck   tsc --noEmit ................ clean, 0 errors
lint        eslint . .................... 0 errors, 64 warnings (all pre-existing, none in the changed file)
test        jest ....................... 229/229 suites, 2925 passed, 3 skipped

CI on this branch: typecheck, eslint, format, unit, e2e, analyze, Deploy-Preview and Vercel all green.

Caveats

  • next build was never completed locally. The dev box has ~12 GB RAM shared with several concurrent agents; every next build --webpack attempt was SIGTERM'd around 70 s at ~2.5 GB RSS with under 1.5 GB left free (seven attempts, including one with webpackBuildWorker: false to halve peak). Nothing in the failures referenced this diff — it never reached compile output. The production build is covered instead by the green Deploy-Preview / Vercel checks above.
  • Playwright was not run locallyglobalSetup needs a live API on :5000 with /dev/test-session, which is not available here. No new specs were written; the repo's existing e2e job passes in CI.
  • The Vercel preview deployment is not usable as proof: it is not peanut.me, so src/app/layout.tsx:24 noindexes every page there — including the Brazil control page. Only the local A/B above, with NEXT_PUBLIC_BASE_URL=https://peanut.me, isolates this change.
  • The local server 404s /en in both the before and after runs, while production serves it as a real page — a local locale-routing artifact, identical on either side of the patch, and untouched by this diff.
  • This is layer 1 of T3. It stops the bleeding for new crawls, but noindex only takes effect once Googlebot re-crawls each URL; already-indexed profile URLs will age out over weeks. Layer 2 (returning a real 404 for handles that do not exist) is PR3's loading.tsx removal plus follow-up work.
  • No open PR touches src/app/[...recipient]/ (checked all 25 open PRs), so no conflict is expected.

🤖 Generated with Claude Code

The [...recipient] catch-all renders an indexable "X on Peanut" shell for
ANY username-shaped string — existence is only checked client-side, so a
non-existent handle still returns HTTP 200 with full metadata. Google has
been indexing those as thin/soft-404 pages.

Set robots: { index: false, follow: false } on all four generateMetadata
return paths (reserved route, missing recipient, !couldBeRecipient, and the
full metadata object). No carve-outs: real profiles, ETH addresses, .eth ENS
names and request/receipt links are app surface, not search landing pages.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@vercel

vercel Bot commented Aug 12, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
peanut-wallet Ready Ready Preview Aug 12, 2026 1:52pm

Request Review

@coderabbitai

coderabbitai Bot commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: b6f470b2-6486-468c-aba8-d439e2a6760a

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Code-analysis diff

Painscore total: 7158 → 7158.35 (+0.35)
Findings: 0 net (+7 new, -7 resolved)

🆕 New findings (7)

  • high complexity — src/app/[...recipient]/page.tsx — CC 49, MI 43.64, SLOC 145
  • high method-complexity — src/app/[...recipient]/page.tsx:30 — generateMetadata CC 42 SLOC 124
  • medium high-mdd — src/app/[...recipient]/page.tsx:30 — generateMetadata: MDD 46.5 (uses across many lines from declarations)
  • medium structural-dup — app/[...recipient]/page.tsx:175 — 28 duplicate lines / 101 tokens with app/invite/page.tsx:58
  • low high-dlt — src/app/[...recipient]/page.tsx:30 — generateMetadata: DLT 18 (calls 18 distinct functions — high context load)
  • low missing-return-type — src/app/[...recipient]/page.tsx:30 — generateMetadata: exported fn missing return type annotation
  • low missing-return-type — src/app/[...recipient]/page.tsx:202 — Page: exported fn missing return type annotation

✅ Resolved (7)

  • src/app/[...recipient]/page.tsx — CC 49, MI 44.48, SLOC 135
  • src/app/[...recipient]/page.tsx:19 — generateMetadata CC 42 SLOC 121
  • src/app/[...recipient]/page.tsx:19 — generateMetadata: MDD 46.3 (uses across many lines from declarations)
  • app/[...recipient]/page.tsx:158 — 32 duplicate lines / 107 tokens with app/invite/page.tsx:54
  • src/app/[...recipient]/page.tsx:19 — generateMetadata: DLT 18 (calls 18 distinct functions — high context load)
  • src/app/[...recipient]/page.tsx:19 — generateMetadata: exported fn missing return type annotation
  • src/app/[...recipient]/page.tsx:189 — Page: exported fn missing return type annotation

@github-actions

github-actions Bot commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

🧪 UI test report — ✅ all green

Suites

  • unit: 2951 ran, 0 failed, 0 skipped, 52.5s

📊 Coverage (unit)

metric %
statements 66.3%
branches 51.2%
functions 56.4%
lines 67.0%
⏱ 10 slowest test cases
time test
4.0s src/components/Card/share-asset/__tests__/shareAssetLayout.test.ts › never places two stickers in heavy overlap (broad seed sweep)
1.2s src/utils/__tests__/demo-api.test.ts › isDemoMode() is false when not running under Capacitor
0.6s src/components/Card/share-asset/__tests__/shareAssetLayout.test.ts › every sticker stays within canvas at any count
0.3s src/utils/__tests__/auth-token.test.ts › is none — never guarded — when only the guarded marker is present
0.3s src/utils/__tests__/sentry.utils.test.ts › defaults to the client budget under a browser global
0.3s src/app/(mobile-ui)/withdraw/__tests__/withdraw-states.test.tsx › Bank withdrawal keeps the $1 minimum for sub-$1 amounts
0.3s src/app/actions/__tests__/api-headers.test.ts › should include Content-Type in validateInviteCode
0.3s src/utils/__tests__/auth-token.test.ts › authReady does not park — hydrates the plain token without an unlock
0.3s src/utils/__tests__/sentry.utils.test.ts › still lets a per-call timeoutMs win over the default
0.3s src/app/actions/__tests__/api-headers-extended.test.ts › should not include apiKey in validateInviteCode body
📍 Inline annotations are in the **Unit test report** check above. Coverage artifact: `coverage-unit`. Generated by `.github/workflows/tests.yml`.

Adversarial-review catch: every branch of this route inherits
alternates.canonical '/' from the root layout, so the noindex was
shipping on pages that canonicalize to the homepage - a combination
Google's canonicalization guidance warns can bleed the noindex into
the cluster head (the homepage itself, sitemap priority 1.0).
alternates: { canonical: null } suppresses the inherited value; the
noindex pages now carry no canonical at all.
@0xkkonrad

Copy link
Copy Markdown
Contributor Author

Adversarial review pass (agent) — HIGH finding, fixed in e46a576.

The noindex was shipping alongside an inherited rel=canonical → homepage (from the root layout), i.e. thousands of URLs saying "don't index me" while naming peanut.me as their canonical — a combination Google's guidance warns can attribute the noindex to the cluster head. Fix: alternates: { canonical: null } on all four branches; the noindex pages now carry no canonical. Verify on the preview: the <link rel="canonical"> tag should be absent on profile-shaped URLs (canonical rendering IS observable there, unlike robots).

Two non-blocking review notes for the record:

  • Post-deploy check: payment-request/receipt links carry the noindex too; most chat unfurlers ignore meta robots, but X/Twitterbot has history of refusing cards on noindex pages. If cards break after deploy, the carve-out is chargeId || recipient.length > 1 (request/receipt shapes only) — the noindex-all-profiles decision stays intact.
  • No test in the repo asserts on robots metadata anywhere; nothing locks this invariant against a future fifth return branch.

🤖 Generated with Claude Code

@0xkkonrad

Copy link
Copy Markdown
Contributor Author

Consolidated into #2685 (one commit per fix, review catches included) at the CTO's request — this PR's evidence and review thread remain the reference for its slice. Branch kept until #2685 merges.

@0xkkonrad 0xkkonrad closed this Aug 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant