Issue with PL/Perl functions

[rabbiveesh]

I have a lot of DB functions running on my Postgres project written in pl/Perl.

I'm finding that this module is failing to handle them correctly, in the following two instances.

1. Each statement inside of the dollar quoted string is getting its semicolon stripped, which is obviously wrong.
2. All braces are getting stripped, so $hash->{value} is rendered into $hash->value, which again, is obviously wrong.

What I fail to understand is why ANY munging is happening inside of the dollar quoted strings. Those are already as atomic as they get!

[rabbiveesh]

This RT issue is related

[rabbiveesh]

A test has been added. It seems like the issue is upstream. In short, the problem lies in the fact that SQL::Tokenizer is breaking up tokens for us, but the tokenizer is doing SQL tokenization for perl code, which is a total mismatch.

This is reminiscent of the perl lexer/parser, where the lexer is influenced by the currently parsed context. We have to think about what to do. We may need to inline the lexer, seeing as it's hasn't seen any activity since 2011, just like this was before we took over....

[rabbiveesh]

An initial attempt at fixing this was in #8, and we'll look at if this is how we want to fix it.

One of the issues that need examination is if we really want to continue lexing with a big regex. There have been issues on SO where the regex takes forever and uses a ridiculous amount of resources.