Repository navigation
feat(devices): name the crawlers production reads generically - #3
Merged
Merged
Conversation
Follows matomo: a WhatsApp Web request no longer claims to run on a Mac, and a Windows launcher that mentions Steam no longer reads as SteamOS.
The fixtures move to eaafe37: 38 383 cases, 82 more. Every new case has its entry in the file named after its fixture; Vivo's Y67L and X9L, Honor's X9b 5G and WhatsApp's iPhone model follow matomo's renaming.
…2026-10-29 Each entry rests on a page its operator publishes, or on a third-party directory where none exists. cert-manager also answers for a build cut from no tag, which says canary where a release says its version.
A client counts as a person only when it is a browser, an application or a PIM on a named platform; libraries and anything naming no platform are tools. The dump predates the rule, so its rows that differ on the type alone are excused.
Matomo reads Headless Chrome, Headless Edge and Lightpanda as browsers, and so does the detection. Detection::is_headless says what the name already holds, for a caller who counts visitors. A headless Chrome sending a plain user agent is still read from its brands.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follows matomo past 6.5.2, and names crawlers, monitors and agents that the detector only read as
Generic Bot, or not at all. Each one was looked up on the web. An entry was written where an operator or a project says what it is. Everything else stays generic.One commit per typology, so a regression bisects to a file.
matomo, at eaafe37
The fixtures move to eaafe37: 38 383 cases, 82 more than at 6.5.2.
WhatsApp/x N) is named and no longer claims a Mac. SteamOS now reads only onSteamDeckorValve Steam, so a Windows launcher that mentions Steam stays Windows.iPhone 4follow matomo's renaming.Bots —
src/devices/bots.ymlMozilla/5.0 (compatible; TalarionSearchBot/1.0; research; contact: …)eplictabot/1.0 (+https://eplicta.se/bot/)ElevenlabsBot/1.0aikido-scan-agent/1.0cert-manager-challenges/canary (linux/amd64) cert-manager/AppVersion = "canary")centreon::plugins::backend::http::useragentcontentsquare static resource scrapertopicsearch-crawler/0.1 (+https://fatgrid.com/crawler; …)ClueWeb-Crawler/1.0 (+https://boston.lti.cs.cmu.edu/CMU-ClueWeb-Crawler/; …)kinsta-botSvix-Webhooks/1.101.0None of them feeds a search index, so none takes the Search bot category.
tests/pinned/production.rspins one case per entry, and each case fails against the database ofmain. Five of the strings left alone are pinned as generic.Traffic comparison types a detection as the log injector now does
examples/compareandexamples/trafficreplay the log injector's currentdevice_type: a client counts as a person only when it is a browser, an application or a PIM on a named platform. Libraries, and anything naming no platform, are tools. Rows of an older dump that differ on the type alone are excused.Detection::is_headless()Matomo reads Headless Chrome, Headless Edge and Lightpanda as browsers (#5691, #7441), and so does the detection.
is_headless()says what the name already holds, for a caller who counts visitors and wants to leave them out. A headless Chrome sending a plain user agent is still read fromSec-CH-UA, as matomo does.