Skip to content

Record plugin searches that find nothing and group them into ideas with Jev - #532

Merged
simonhamp merged 1 commit into
mainfrom
plugin-search-misses
Sep 25, 2026
Merged

simonhamp merged 1 commit into
mainfrom
plugin-search-misses

Conversation

@simonhamp

Copy link
Copy Markdown
Member

When an agent uses the search_plugins MCP tool and nothing matches, we now keep the search so we can see which plugins people want that we don't have yet.

Each missed search is stored with its term, the free/paid filter if one was set, and a new optional use_case argument where the agent can say what the user needs the plugin for. Nothing about who searched is stored. A queued job then asks TypeSafe's Jev whether the search is for a plugin idea we've already had. If it is, the search counts as a vote for that idea. If not, it starts a new one. The ideas show up in the admin under Products > Plugin Ideas, sorted by votes, with the searches behind each one. This adds two tables, plugin_ideas and missed_plugin_searches.

How the matching works:

  • Jev gets one choice question listing up to 254 existing ideas, most votes first, plus a none_of_the_above option. Jev caps choice questions at 255 options.
  • If the probabilities Jev gives the existing ideas add up to 0.5 or more, the vote goes to the likeliest one. Adding them up means a search that falls between two near-duplicate ideas still gets matched instead of starting a third.
  • Jev can't write text, so a new idea takes the search term as its title and the use case as its description. Admins can rename ideas, and later searches are matched against the edited wording.
  • A search term we've seen before goes straight to the same idea without asking Jev. The very first search starts an idea without asking Jev either, since there's nothing to compare it with.
  • The job uses WithoutOverlapping, so an agent firing off a few phrasings of the same search at once doesn't start an idea for each.

laravel/ai was on 0.1.5 and not used anywhere yet. The TypeSafe provider only exists in 1.0, and 1.0 conflicts with laravel/mcp below 1.0, so this also moves laravel/mcp to 1.0 and laravel/boost to 2.10 (the first Boost release that allows mcp 1.0). prism-php/prism drops out because laravel/ai no longer needs it.

Things to watch:

  • The laravel/mcp 1.0 upgrade affects the admin MCP server at /mcp/oauth/admin. Its tests pass and clients that open with initialize keep working, but the server now checks the MCP-Protocol-Version, Mcp-Method and Mcp-Name headers on requests that use the new 2026-07-28 protocol. Worth trying from a real client after deploy.
  • Production needs TYPESAFE_API_KEY. Without it, searches still get saved, but any that need Jev fail and stay ungrouped until the jobs are retried.
  • Only the MCP tool records misses. The REST endpoint at /api/mcp/plugins doesn't. A search filtered to free or paid isn't recorded if a plugin of the other type matches.
  • Recording happens inside rescue(), so a failure gets reported but never breaks the search response.
  • The public search_plugins description now says that searches with no results are kept anonymously.

Testing: 26 new tests cover the MCP tool, the job and the Filament resource, and the full suite passes on the upgraded packages. I also ran it against a throwaway SQLite database: the admin pages render, and a real MCP call was counted as a vote on the right idea.

🤖 Generated with Claude Code

When the search_plugins MCP tool finds nothing, store the search and ask
TypeSafe's Jev whether it's a plugin idea we already have. A match counts
as a vote, otherwise it starts a new idea. Admins see the ideas, sorted by
votes, under Products > Plugin Ideas.

Upgrades laravel/ai to 1.0 for the TypeSafe provider, which also needs
laravel/mcp 1.0 and laravel/boost 2.10.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@simonhamp
simonhamp marked this pull request as ready for review September 25, 2026 16:50
@simonhamp
simonhamp merged commit 9ed72dc into main Sep 25, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant