Skip to content

Say which SDKs provide the robots.txt engine - #234

Open
jwrosewell wants to merge 1 commit into
mainfrom
docs/robotstxt-sdk-availability
Open

Say which SDKs provide the robots.txt engine#234
jwrosewell wants to merge 1 commit into
mainfrom
docs/robotstxt-sdk-availability

Conversation

@jwrosewell

Copy link
Copy Markdown
Contributor

The Robots.txt Generator page names the .NET package once in passing and never states that the on-premise engine is .NET only. A developer working in Java can read the whole page, see a package they cannot use, and never learn that the Cloud route works from their language today.

This adds an Availability section near the top saying so plainly, with a table covering the three ways to use the feature.

Checked against the code, not assumed

RobotsTxtEngine appears only in C# across the 51Degrees organisation. A code search returns 7 occurrences in C# and zero in Java, C, C++, Go, Rust, PHP, Node and Python. The files are FiftyOne.DeviceDetection.RobotsTxt/FlowElements/RobotsTxtEngine.cs and RobotsTxtBuilder.cs in device-detection-dotnet.

How this was found

Our website assistant was asked whether the robots.txt engine is available in Java. It declined to answer rather than guess, which was correct, and that is what exposed the gap: nothing published anywhere states which SDKs provide the feature, so neither a reader nor an assistant can tell.

That suggests a wider check is worth doing, comparing what exists in the repositories against what the documentation says exists per language. This PR fixes the one case we hit.

Note on how this was written

Drafted with AI assistance. The availability claim was verified against the code before writing, but the wording should have a human read before merge.

The page named the .NET package once in passing and never said the
on-premise engine is .NET only. A developer working in Java could
read the whole page, see a package they cannot use, and not learn
that the Cloud route works from their language today.

Verified against the code before writing it: RobotsTxtEngine appears
only in C# across the organisation, with no occurrences in Java, C,
C++, Go, Rust, PHP, Node or Python.

Found because our website assistant could not answer 'is this
available in Java'. It correctly refused to guess, which was the
right behaviour and showed the gap: nothing published states which
SDKs have the feature.
@github-actions

github-actions Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Documentation preview 👀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant