From a717ffdae5fdbe50cb5332ccf061a46fc980b4cf Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Pedro=20Hern=C3=A1ndez?= Date: Wed, 5 Aug 2026 17:26:48 -0400 Subject: [PATCH] Hand each reader the number measured for their own language MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two things went stale in the teacher package, and only one of them was #32's doing. The syllabus wording said the tool "produces no verdict". That was true when it was written — the report's gate demanded a per-language threshold that no language has, so it withheld the verdict from every document ever analysed. #32 fixed the gate, and above the measured boundary the tool now says "Signs of AI writing", which is a verdict by any reading. A teacher who pasted that sentence into a syllabus would have been contradicted by the tool in front of their own class. It now says what stays true: the tool reaches no conclusion about who wrote anything, and nothing it produces is the basis of a decision. The second is older and worse. The committee template handed the teacher the **pooled** false- positive rate — "under 4.1% overall" — to write into a disciplinary finding, whatever language the work was in. Spanish on its own supports 13.3%: a committee judging a Spanish essay was being handed a tool three times better than the one actually used, in the teacher's own handwriting, in a document that gets read at appeal. This is precisely the harm #36 removed from the report, sitting untouched in the package aimed at the people who take reports to committees. The template now points at the figure the report prints for the work being judged, and says plainly why copying the pooled number from a template is the easiest way to overstate this tool at a hearing. The student sheet had the same defect in a quieter form: both language editions quoted the pooled figure. Each now carries its own — 6 in 100 for English over 65 texts, 13 in 100 for Spanish over 25 — and says that the pooled number is the more flattering one, which is why it is not being used. A Spanish-speaking student was the person in this project most likely to be harmed by a rate measured mostly on English, and the sheet written for them was quoting it. No code changed. Every figure checked against published-calibration.json. --- Docs/Teaching/committee.en.md | 9 ++++++++- Docs/Teaching/committee.es.md | 14 +++++++++++--- Docs/Teaching/student-sheet.en.md | 7 ++++++- Docs/Teaching/student-sheet.es.md | 8 ++++++-- Docs/Teaching/syllabus.en.md | 8 ++++---- Docs/Teaching/syllabus.es.md | 5 +++-- 6 files changed, 38 insertions(+), 13 deletions(-) diff --git a/Docs/Teaching/committee.en.md b/Docs/Teaching/committee.en.md index c2608e2..423d2ea 100644 --- a/Docs/Teaching/committee.en.md +++ b/Docs/Teaching/committee.en.md @@ -93,9 +93,16 @@ not, in the finding itself: > A writing-analysis tool was used to decide which sections to examine. Its output is not evidence of > authorship and was not treated as such. It publishes a false-positive rate on writing known to be -> human of under 4.1% overall, with no threshold supported for either language on its own — the +> human, measured for the language this work is written in and printed on the report itself — the > findings below rest on the source verification and the interview, not on that tool. +**Copy the figure from the report in front of you, not from here.** The tool measures a separate rate +per language and the report prints the one that applies to the work being judged. Quoting the pooled +figure instead is the single easiest way to overstate this tool at a hearing: on the current corpus +the pooled rate is under 4.1%, while Spanish on its own supports only 13.3% — three times worse. A +committee handed the flattering number about a Spanish essay has been given a better tool than the +one that was actually used, and the difference is the sort of thing that surfaces on appeal. + A committee that writes this sentence is in a stronger position than one that omits it, because the sentence is going to be raised at appeal whether or not it appears. diff --git a/Docs/Teaching/committee.es.md b/Docs/Teaching/committee.es.md index e29301e..db20259 100644 --- a/Docs/Teaching/committee.es.md +++ b/Docs/Teaching/committee.es.md @@ -96,9 +96,17 @@ resolución qué es y qué no es: > Se utilizó una herramienta de análisis de escritura para decidir qué secciones examinar. Su > resultado no es prueba de autoría y no se ha tratado como tal. Publica una tasa de falsos positivos -> sobre escritura conocidamente humana inferior al 4,1% en agregado, sin que ningún idioma por -> separado respalde un umbral propio; las conclusiones siguientes se apoyan en la verificación de -> fuentes y en la entrevista, no en esa herramienta. +> sobre escritura conocidamente humana, medida para el idioma en que está escrito este trabajo e +> impresa en el propio informe; las conclusiones siguientes se apoyan en la verificación de fuentes y +> en la entrevista, no en esa herramienta. + +**Copie la cifra del informe que tiene delante, no de aquí.** La herramienta mide una tasa distinta +por idioma y el informe imprime la que corresponde al trabajo que se está juzgando. Citar la cifra +agregada es la forma más fácil de exagerar esta herramienta en una audiencia: en el corpus actual la +agregada queda por debajo del 4,1%, mientras que el español por separado solo respalda un 13,3% — +tres veces peor. A un comité al que se le entrega el número halagador sobre un ensayo en español se +le ha dado una herramienta mejor que la que realmente se usó, y esa diferencia es de las que salen a +la luz en una apelación. Un comité que escribe esa frase queda en mejor posición que uno que la omite, porque la cuestión se va a plantear en apelación aparezca o no. diff --git a/Docs/Teaching/student-sheet.en.md b/Docs/Teaching/student-sheet.en.md index d562602..057a739 100644 --- a/Docs/Teaching/student-sheet.en.md +++ b/Docs/Teaching/student-sheet.en.md @@ -24,7 +24,12 @@ patterns. Software notices patterns in innocent work constantly. It publishes how often it is wrong about writing known to be human — a figure almost no tool in this category will state about itself. On its test corpus of ninety texts published before AI writing existed, it flagged **none of them** at its recommended setting. But that is ninety texts, and the -honest reading is the range around it, not the zero: somewhere **under about 4 in 100**. +honest reading is the range around it, not the zero. + +Of those ninety, sixty-five were in English. This sheet quotes the **English** figure, which is the +one that applies to you: somewhere **under about 6 in 100**. The pooled figure that mixes both +languages is more flattering, and using it here would credit the tool with a precision nobody +measured on writing in your language. It also does something most tools refuse to do: **below its supported threshold it prints no verdict at all.** A low score is not a certificate that a person wrote something, and it does not claim to be diff --git a/Docs/Teaching/student-sheet.es.md b/Docs/Teaching/student-sheet.es.md index 8fbda82..ab06294 100644 --- a/Docs/Teaching/student-sheet.es.md +++ b/Docs/Teaching/student-sheet.es.md @@ -24,8 +24,12 @@ programas detectan patrones en trabajos inocentes todo el tiempo. Publica con qué frecuencia se equivoca sobre escritura que se sabe humana — una cifra que casi ninguna herramienta de esta categoría dice sobre sí misma. En su corpus de noventa textos publicados antes de que existiera la escritura con IA, no marcó **ninguno** en su ajuste recomendado. Pero son -noventa textos, y la lectura honesta es el rango alrededor de ese cero, no el cero: algo **por debajo -de 4 de cada 100**. +noventa textos, y la lectura honesta es el rango alrededor de ese cero, no el cero. + +Y de esos noventa, solo veinticinco estaban en español. Esta hoja cita la cifra del **español**, que +es la que te corresponde: algo **por debajo de 13 de cada 100**. La cifra agregada que mezcla los dos +idiomas es más favorable, y usarla contigo sería atribuirle a la herramienta una precisión que nadie +midió sobre escritura en tu idioma. También hace algo que la mayoría se niega a hacer: **por debajo de su umbral respaldado no imprime ningún veredicto.** Una puntuación baja no es un certificado de que lo escribió una persona, y no diff --git a/Docs/Teaching/syllabus.en.md b/Docs/Teaching/syllabus.en.md index 0b67c18..44e24d2 100644 --- a/Docs/Teaching/syllabus.en.md +++ b/Docs/Teaching/syllabus.en.md @@ -40,10 +40,10 @@ never the reason for a decision about a student.** Everything else is negotiable > Being asked is not an accusation and carries no penalty. It is part of how work is assessed in > this course, and I may ask anyone. > -> I may run submitted work through an offline writing-analysis tool. It produces no verdict, and no -> score from it will ever be the basis of a decision about you. Its purpose is to tell me where to -> read more carefully — the same thing my own eyes do, applied evenly to everyone's work rather than -> only to the students I happen to wonder about. +> I may run submitted work through an offline writing-analysis tool. It reaches no conclusion about +> who wrote anything, and nothing it produces will ever be the basis of a decision about you. Its +> purpose is to tell me where to read more carefully — the same thing my own eyes do, applied evenly +> to everyone's work rather than only to the students I happen to wonder about. ## Option C — Not permitted diff --git a/Docs/Teaching/syllabus.es.md b/Docs/Teaching/syllabus.es.md index 398faad..5c45927 100644 --- a/Docs/Teaching/syllabus.es.md +++ b/Docs/Teaching/syllabus.es.md @@ -39,8 +39,9 @@ detector nunca es el motivo de una decisión sobre un estudiante.** Todo lo dem > Que se lo pida no es una acusación ni conlleva ninguna penalización. Es parte de cómo se evalúa en > esta asignatura, y puedo pedírselo a cualquiera. > -> Puedo analizar los trabajos con una herramienta que funciona sin conexión. No emite veredictos, y -> ninguna puntuación suya será la base de una decisión sobre usted. Sirve para indicarme dónde leer +> Puedo analizar los trabajos con una herramienta que funciona sin conexión. No concluye nada sobre +> quién escribió qué, y nada de lo que produzca será la base de una decisión sobre usted. Sirve para +> indicarme dónde leer > con más atención — lo mismo que hacen mis propios ojos, aplicado por igual a todos los trabajos y > no solo a los estudiantes sobre los que casualmente me pregunte algo.