Strange peek behaviour/generator exception handling?

[deliciouslytyped]

I've been staring at this for two days and haven't been able to figure it out: I'm pretty sure at this point the problem is as described in the later paragraphs, and I rubber-ducked myself.

Disregarding my weird function naming, why does block_lex_transformer keep trying to parse past the end of the input? I figured peek() would yield a ParseError if I tried to do that, but it doesn't seem to be the case?

peek(any_char).parse_partial("") gives a ParseError.

I don't know much about how coroutines/generators work, my suspicions are that this isn't working because a GeneratorExit exception seems to come from yield peek(seq(sideToken, line(id))) , but I don't know how to deal with it.

I think that happens because googling suggests an exception in a generator will stop the generator, I didn't immediately see anything explaining how to have it continue and catch the exception?

# some context for the following snippets
from dataclasses import dataclass
from parsy import *

@dataclass
class Neutral:
    depth: int

@generate
def neutral():
    return (yield test_item(lambda i: isinstance(i, Neutral), "neutral"))

sideToken = neutral

take = test_item(lambda x: True, "any nested")

def wrap(fn):
    def thing(res):
        try:
            result = fn.parse(res)
            return success(result)
        except ParseError as e:
            return fail("idk, failure '%s'" % e)
    return thing


def line(fn):
    return take.bind(wrap(fn))

@generate
def id():
    return (yield any_char.many().concat())

@generate
def block_lex_transformer():
    token = yield sideToken  # TODO ok?
    curDepth = token.depth

    res = [Neutral(0), (yield line(id))]
    while True:
        try:
            side, ln = yield peek(seq(sideToken, line(id)))  # go to except branch if fail
        except ParseError as e:  # todo havent actually tested this explicitly
            if not e.stream:
                return res
            else:
                raise e
        if side.depth >= curDepth or ln == "":  # as deep or deeper or empty line
            (yield sideToken), (yield line(id))  # actually consume the two tokens we just peeked
            res += [side.__class__(max(side.depth - curDepth, 0)), ln]
        else:
            return res
    return res

block_lex_transformer.parse([Neutral(depth=0), 'test'])

[deliciouslytyped]

Edit: So to restate this in a clearer manner: As far as I can tell, python doesn't let you catch exceptions raised from inside a generator, because they are converted into stopping the generator.

All the stackoverflow posts I've looked at so far suggest yielding an exception object from a generator instead of having it raised, which I don't really like, and might require rearchitecting parsy?

I'm going to try to see if I can write a wrapper function that will catch exceptions and return them in this form, so at least it can be done on a case by case basis, but will look less nice.

[deliciouslytyped]

To be clear, help on the above would be appreciated, I'm winging this.

[deliciouslytyped]

If the above is true I'm not quite sure I get how the normal case works, but being unable to figure out how to implement something better, I found a workaround:

I think I've temporarily worked around this by modifying the peek implementation to signal the end of the stream with None (I should probably create a more descriptive symbol instead);

def peek_with_none(parser): #To work around the cant raise exception from yield issue
    @Parser
    def peek_parser(stream, index):
        if index == len(stream):
            #bail early instead of raising a ParseError, or something
            return Result.success(index, None)

        result = parser(stream, index)
        if result.status:
            return Result.success(index, result.value)
        else:
            return result
    return peek_parser

[deliciouslytyped]

I completely forgot I ran into this exact issue and did it again.
I think the problem is that wrapping a yield in a try/except doesn't catch the error because it's not actually "running" in the context where the try/except is. It's returning a value which gets run somewhere else.

I haven't actually figured out how to make this work but it probably involves writing a @parser. I worked around my issue temporarily by using a peek that wouldnt fail and .map()-ing a conditional function over the result to check if continuing is ok.