Skip to content

More languages, without quietly lending them a rate we never measured #44

Description

@peopleworks

The project is bilingual by accident of having two rule packs, not by design. A French, German, Portuguese or Italian speaker who arrives today finds a tool that says it is for "academic writing" and then only understands English and Spanish. That is a real exclusion and it is worth fixing.

But adding languages naively would break the one thing this project has that others do not, so this issue is as much about the constraint as the work.

What a language actually consists of here

Four separable pieces. Contributors should be able to land them one at a time, and the product must be honest about which are present.

  1. Rules/Packs/rules.{lang}.json — the tells themselves. The hard part, and the only part that needs a native speaker who has read a lot of machine output in that language. The Spanish pack is an original derivation, not a translation of the English one, and any new language deserves the same rather than a translated word list.
  2. wwwroot/i18n/{lang}.json — the interface. Easy, and already falls back gracefully.
  3. Reporting/report.{lang}.json — the report prose, with its mandatory core. Already designed for partial translation; see Docs/TRANSLATING.md.
  4. Calibration texts in that language — the part everybody will want to skip, and the reason this issue exists.

The constraint: a pack without a corpus has no error rate

Docs/CALIBRATION.md supports no threshold for English or Spanish on their own — 65 and 25 texts, best bounds 5.6% and 13.3%. A brand-new language would have zero.

So a French pack must ship saying, on the report and anywhere a number appears: no false-positive rate has been measured for this language. The machinery for that already exists and is tested — a language absent from the corpus gets no threshold, no verdict, and no borrowed aggregate.

What must never happen is a new language inheriting the aggregate figure. That is the exact error this project criticises in every commercial detector, and it would be worse coming from us, because we are the ones who published the sentence saying a rate that holds in one language and fails in another is not one number.

This makes the corpus contribution the valuable one, and the issue should say so out loud: ~75 texts published before 2022 in a language is worth more to that language's users than the rule pack is.

Known blocker, small and verified

src/SignsOfAI.Core/SignsOfAI.Core.csproj:32-33 lists the rule packs one file at a time, while the report resources on line 38 use a wildcard. So a contributor adding rules.fr.json today must also edit the build — which violates the project's own rule that extension points are JSON anyone can send by pull request, never compiled C#.

Fix that first, with the same WithCulture=false + LogicalName treatment, and adding a language becomes what it should be: adding files.

Which languages first

Suggested order, by the size of the underserved academic population and by how well pre-2022 open-access text can be sourced: Portuguese, French, German, Italian. But this should follow whoever actually turns up — a maintained pack from one motivated contributor beats four machine-translated ones.

Definition of done for a language

  • Rule pack derived for the language, not translated from English
  • Interface translation (may be partial)
  • Report resource with its mandatory core (may otherwise be partial)
  • Listed in locales.json
  • Either calibration texts, or the report visibly saying no rate has been measured for it

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions