Repository navigation
Commit 433fd0f
committed
test(evals): assert grounded behavior instead of exact phrasing in live mode
The first live DeepSeek run exposed brittle assertions, not harness bugs:
the model chained the tools correctly but the checks were case-sensitive and
required an internal order id. Match the retrieved value case-insensitively
and let live runs accept the grounded status rather than the internal id.1 parent af2e6ca commit 433fd0f
1 file changed
Lines changed: 4 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
182 | 182 | | |
183 | 183 | | |
184 | 184 | | |
185 | | - | |
| 185 | + | |
186 | 186 | | |
187 | 187 | | |
188 | 188 | | |
| |||
216 | 216 | | |
217 | 217 | | |
218 | 218 | | |
| 219 | + | |
| 220 | + | |
219 | 221 | | |
220 | 222 | | |
221 | 223 | | |
| |||
245 | 247 | | |
246 | 248 | | |
247 | 249 | | |
248 | | - | |
| 250 | + | |
249 | 251 | | |
250 | 252 | | |
251 | 253 | | |
| |||
0 commit comments