Commit 695d71e
Fix two parser generator bugs exposed by inlined error alternatives
Two groups with the same syntax but different actions or variable
names were deduplicated into a single artificial rule, because the
cache key was based on str(), which ignores actions and variable
names. With the error alternatives inlined before the base
alternatives, the artificial rule for the group in an error
alternative (without an action) could shadow the artificial rule for
the same-looking group in a base alternative (with an action), e.g.
"class C(B): ..." lost its bases. Use repr() in the cache key.
The action of an alternative inserted by a rule extension can return
NULL without setting an error (a conditional action which only
sometimes reports the error, e.g. in invalid_legacy_expression). The
generated code treated this as a result of the whole rule, so the
rule failed (and the failure was memoized) instead of trying the
remaining alternatives. Generate a NULL check which continues with
the next alternative, like a reference to an invalid_* rule returning
NULL behaves.
Also do not insert new alternatives before alternatives inserted by
preceding extensions: they raise an error rather than succeed, so
they cannot shadow the new alternative, and the order of extensions
should define their relative order ("'in' expected after for-loop
variables" must be reported before descending into the for targets).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>1 parent 0ef9162 commit 695d71e
6 files changed
Lines changed: 2387 additions & 1773 deletions
File tree
- Grammar
- Lib/test/test_peg_generator
- Parser
- Tools/peg_generator/pegen
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
429 | 429 | | |
430 | 430 | | |
431 | 431 | | |
432 | | - | |
433 | | - | |
434 | | - | |
435 | | - | |
436 | 432 | | |
437 | 433 | | |
438 | 434 | | |
439 | 435 | | |
| 436 | + | |
| 437 | + | |
| 438 | + | |
| 439 | + | |
440 | 440 | | |
441 | 441 | | |
442 | 442 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
98 | 98 | | |
99 | 99 | | |
100 | 100 | | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
101 | 113 | | |
102 | 114 | | |
103 | 115 | | |
| |||
0 commit comments