[prompt-analysis] Copilot PR Prompt Analysis - 2026-08-28

[github-actions[bot]]

🤖 Copilot PR Prompt Pattern Analysis - 2026-08-28

Summary

Analysis Period: Last 30 days (2026-07-29 to 2026-08-28)
Total PRs: 1000 | Merged: 831 (83.6% of completed) | Closed: 163 (16.4% of completed) | Open: 6

Prompt Categories and Success Rates

Category	Total	Merged	Closed	Success Rate
Bug Fix	904	747	151	83.2%
Feature Addition	81	74	7	91.4%
Testing	6	4	2	66.7%
Documentation	1	1	0	100.0%
Refactoring	1	1	0	100.0%
Other	7	4	3	57.1%

Note: 904 of 1000 PRs matched bug-fix keywords ("fix", "resolve", "error") — this dataset is dominated by automated issue-triage-style bug-fix PRs, so category comparisons for the smaller buckets (testing, other) have low sample sizes and should be read cautiously.

Prompt Analysis

✅ Successful Prompt Patterns

Structural characteristics of merged PRs vs closed PRs:

• Average prompt length: 169 words (merged) vs 192 words (closed) — shorter, more focused prompts tend to merge more often
• Contains a code block (```): 83.0% of merged prompts vs 69.3% of closed prompts
• Title tagged [WIP]: only 2.4% of merged PRs vs 22.7% of closed PRs — WIP-tagged PRs are far more likely to be abandoned/closed
• Bullet-list structure: present in both groups (~98–100%), not a strong differentiator on its own

Example successful prompts:

1. PR Fix broken documentation redirect links #56654: "Fix broken documentation redirect links" (42 words, concise, single clear target) → Merged
2. PR Preserve GitHub expression operators in safe-output JSON #56653: "Preserve GitHub expression operators in safe-output JSON" (55 words, specific technical scope) → Merged
3. PR [eslint-miner] Require try/catch around fs.realpathSync calls #56612: "[eslint-miner] Require try/catch around fs.realpathSync calls" (96 words, references a specific code pattern and location) → Merged

❌ Unsuccessful Prompt Patterns

Common characteristics in closed PRs:

• Longer prompts (avg 192 words) that bundle multiple concerns or extensive background
• Nearly 1 in 4 closed PRs (22.7%) started as [WIP], suggesting exploratory/incomplete tasks are less likely to land
• Lower rate of embedded code blocks (69.3%), suggesting less concrete technical grounding in the prompt

Example unsuccessful prompts:

1. PR [WIP] Fix MCP gateway health timeout races with backend startup timeout #56273: "[WIP] Fix MCP gateway health timeout races with backend startup timeout" (264 words, WIP tag, complex multi-part race-condition description) → Closed
2. PR Consolidate duplicated string-or-slice any coercion into typeutil.NormalizeStringSlice #56058: "Consolidate duplicated string-or-slice any coercion into typeutil.NormalizeStringSlice" (287 words, broad refactor spanning many call sites) → Closed
3. PR Refactor import field extraction into focused parser modules #56053: "Refactor import field extraction into focused parser modules" (258 words, large-scope structural refactor) → Closed

Key Insights

• Pattern 1: Prompts containing a code block (showing the exact code/config to change) merge at 83.0% vs 69.3% without — being concrete about the target code matters more than prose length.
• Pattern 2: [WIP] titled PRs close nearly 10x more often proportionally (22.7% of closed vs 2.4% of merged) — WIP is a strong leading indicator of an incomplete/abandoned task rather than a bug in the prompt itself.
• Pattern 3: Prompts over ~250 words that span multiple files/concerns (broad refactors, multi-part bug investigations) skew toward closure; narrowly scoped, single-issue prompts (~50-100 words) merge more reliably.
• Pattern 4: Bug-fix style prompts dominate volume (904/1000) and merge at a solid 83.2%, while feature-addition prompts have the highest success rate (91.4%) — likely because feature requests in this dataset tend to be smaller, well-defined additions.

Recommendations

Based on today's analysis:

1. DO: Keep prompts concise and scoped to one file/behavior; include a code block or exact snippet showing what needs to change.
2. DO: For bug fixes, reference the specific error, function, or reproduction steps rather than general symptoms.
3. AVOID: Framing tasks as [WIP] or open-ended explorations — these correlate strongly with closure. Prefer a single, well-defined acceptance criterion.
4. AVOID: Bundling large multi-file refactors into one prompt; split into smaller, independently mergeable PRs.

Historical Trends

Prior recorded data points (from repo-memory prompt-analysis-history.jsonl, dataset composition may differ from today's 1000-PR sample):

Date	PRs	Merge Rate	Top Category
2026-08-28 (today)	1000	83.6%	bug_fix
2026-07-07	1000	80.0%	bug_fix
2026-07-06	1000	80.4%	test
2026-07-05	1000	80.8%	bug_fix
2026-07-04	1000	81.1%	bug_fix
2026-07-03	1000	81.8%	bug_fix
2026-07-01	—	81.2%	—

Trend: Merge rate has climbed from ~80-82% (early July) to 83.6% today, a modest but steady improvement. Bug-fix prompts remain the dominant and most consistent category by volume.

Note: This run's structural comparisons (WIP tags, code-block presence, word counts) were computed directly from the PR corpus and were not persisted back to repo-memory due to a read-only filesystem in this run's environment — historical structural trend data could not be appended this cycle. Future runs should retry the persistence step.

---

Generated by Copilot PR Prompt Analysis (Run: 33215233477)

Generated by 🔍 Copilot PR Prompt Pattern Analysis · copilot · auto · 26.1 AIC · ⌖ 7.14 AIC · ⊞ 11.1K · ◷

• expires on Aug 29, 2026, 2:10 PM UTC-08:00