Improve debugging ergonomics

[deliciouslytyped]

I've been wondering what it would take to improve the debugging experience for combinators in generalm but here are some specific ideas (TODO: breakdown based on usecases; which part of my code is failing? what data does it see?):

• parse errors show where they failed and what symbol they were expecting, this is somewhat redundant with the "line number:character" location data, but it would still be helpful to show the next few symbols the parser is failing on. Improve debugging: peek show next data in errors #46

• a .debug() method which can be added inline wherever you need to see the output of the previous parser method / the input to the next parser method Inline (explicit) and implicit tracing #49

• It would be good to also have something for things like seq(a, seq(b, c)) where you can insert a debug on the left of b, to see what input it's getting. I.e. you can't use a method at the beginning of a sequence of parsers so it needs to be a function different from .debug() Inline (explicit) and implicit tracing #49

• It would also be cool if you didn't need to annotate with .debug wherever you wanted to debug something, but could get the parser to give information about the relevant things without changing your code. The only way to do this that I've been able to think of is to write an extra wrapper for every function? And for filtering, to be able to say "return trace information between characters x and y"'. Inline (explicit) and implicit tracing #49

• I haven't found trying to use the debugger particularly illuminating (IIRC?) because it never jumps back into user code where I can see which part of my grammar is causing problems. It would be cool if there were ways to make using a debugger more ergonomic but I have no ideas for that...

• It would be helpful if .bind could handle ParseError (stream, index) properly and return the appropriate error information when a ParseError gets passed up to the larger context. Recompute line number for ParseError passed up from .bind #47

[deliciouslytyped]

A recuring issue is "Why isn't my parser consuming more?" -which I assume happens when backtracking runs out of possibilities. I should get an example.

(consequently, "Why is my parser consuming too much?" might also be an issue?)

[deliciouslytyped]

It might be interesting if instead of opaque Parser objects, some kind of AST could be displayed. I don't have any concrete ideas for implementing this offhand.

Allowing some manner of introspection on the parser structure might allow to set conditional breakpoints more effectively, or at all.

Scenario: you need to breakpoint on when a specific alt is run. This is problematic because breakpointing in the combinator code breakpoints in the parser construction stage and not when the parser is run.

Edit: I ended up solving the specific issue by parsing stream[index:] in the conditional to check if I'm in the proximity of what's breaking.
You can also set conditional breakpoints on e.g. named arguments when processing seq

[deliciouslytyped]

It would be good if there's some way to help debug when parsers fail to backtrack.

I'm writing this after the fact so I'm going to mischaracterize whats going on, but I think there might be some difficulties or things that are not clear about handling exceptions in @generator based parsers, and this can lead to backtracking failure (?).

Probably related: https://stackoverflow.com/questions/3997496/python-generator-non-swallowing-exception-in-coroutine

I vaguely recall something about my exceptions being silently swallowed, but that might have been a badly written except clause or not returning fail when I should have. Alternatively I think exceptions might be silently being turned into GeneratorExit, and so being swallowed silently.