[ethanzhoucool]

Summary

Language detection could be made to burn minutes of CPU with a request made of blank lines. Several of the patterns in SIGNATURES start with ^\s* under re.M. \s also matches \n, so from every line start the engine scans all of the whitespace that follows before it gives up. That makes the cost quadratic in the number of blank lines.

Detection runs on anonymous input whenever language is omitted, which is the web UI's default ("auto"), and on POST /v1/detect, which has no size limit at all. The Procfile runs a single uvicorn worker, so one such request ties up the whole service.

Measured locally on Python 3.13 before this change:

input	size	detect_language
"\n" * 20_000	20 KB	37.9 s
" \t\r\n" * 5_000	20 KB	10.9 s
"def a(" + ")->" * 7_000	20 KB	1.3 s
"new A<>" * 6_000	42 KB	0.29 s (grows quadratically)

It's quadratic, so the 200 KB that /v1/explain accepts works out to roughly an hour for the blank-lines case.

After this change, each of those inputs at the full 200 KB takes between 0.05 and 0.19 s.

Changes

• Leading indentation is matched with [^\S\n]* (any whitespace except a newline) instead of \s*. It's factored into one _INDENT prefix so the rule lives in one place.
• The Python def header pattern \(.*\)\s*(->.*)?: re-scanned the rest of the line once for every ). It is now \(.*\)[^)\n]*:, which only scans up to the next ), so the total work stays linear. It still matches every line the old one matched (plain, ->-annotated and multi-paren return types).
• The Java generic-constructor pattern bounds its .* to .{0,200}, so an attempt can't rescan a whole long line.
• A comment above SIGNATURES explains both banned shapes, so a later pattern doesn't bring them back.

Testing

• pytest: 152 passed, 3 skipped (Python 3.13). pytest tests/test_explain.py on Python 3.11: 42 passed. ruff check explain_my_code tests: clean.
• New test_detection_stays_fast_on_adversarial_input (4 cases, each must finish in under 0.5 s). Against the old patterns all 4 fail, and the run takes 40 s. With this change they take milliseconds.
• New test_detection_still_reads_indented_and_crlf_lines checks that indented and CRLF lines are still recognised.
• No change in behaviour: I ran the old and new detect_language on 4,750 files (150 CPython stdlib modules, this repo, web/static/samples.js, and JS/TS/Java/C++/SQL files from other local checkouts). Language, confidence and runner-up were identical for every file.

Notes

/v1/detect and /v1/analyze still have no request-size validator, unlike /v1/explain. I left that out of this PR to keep it to one theme.

---

Generated by Claude Code