Commit cdebc4d
committed
test(evals): fix adversarial scenario design after the live run
Two live failures were eval design, not model failure:
- empty-result-no-hallucination rejected valid 'didn't find' / 'wasn't able
to find' phrasing. Broaden the grounding check.
- near-duplicate-names required a userId the prompt never gave, so the model
reasonably asked for it. Put the id in the prompt and the scripted call.1 parent 9d2ec19 commit cdebc4d
1 file changed
Lines changed: 3 additions & 3 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
499 | 499 | | |
500 | 500 | | |
501 | 501 | | |
502 | | - | |
| 502 | + | |
503 | 503 | | |
504 | 504 | | |
505 | 505 | | |
| |||
567 | 567 | | |
568 | 568 | | |
569 | 569 | | |
570 | | - | |
| 570 | + | |
571 | 571 | | |
572 | 572 | | |
573 | 573 | | |
574 | 574 | | |
575 | 575 | | |
576 | 576 | | |
577 | 577 | | |
578 | | - | |
| 578 | + | |
579 | 579 | | |
580 | 580 | | |
581 | 581 | | |
| |||
0 commit comments