-
Notifications
You must be signed in to change notification settings - Fork 969
fix: robustly strip psql meta commands #4177
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
ignat980
wants to merge
17
commits into
sqlc-dev:andrew/fix-4065
Choose a base branch
from
ignat980:fix/strip-psql-meta
base: andrew/fix-4065
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
+3,986
−2,831
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Update examples for 1.30.0 * Update tests for v1.30.0 * Update version to v1.30.0
…qlc-dev#4091) Bumps the production-dependencies group with 2 updates: [github.com/spf13/cobra](https://github.com/spf13/cobra) and [github.com/spf13/pflag](https://github.com/spf13/pflag). Updates `github.com/spf13/cobra` from 1.9.1 to 1.10.1 - [Release notes](https://github.com/spf13/cobra/releases) - [Commits](spf13/cobra@v1.9.1...v1.10.1) Updates `github.com/spf13/pflag` from 1.0.7 to 1.0.9 - [Release notes](https://github.com/spf13/pflag/releases) - [Commits](spf13/pflag@v1.0.7...v1.0.9) --- updated-dependencies: - dependency-name: github.com/spf13/cobra dependency-version: 1.10.1 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: production-dependencies - dependency-name: github.com/spf13/pflag dependency-version: 1.0.9 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: production-dependencies ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps the production-dependencies group with 1 update: [github.com/spf13/pflag](https://github.com/spf13/pflag). Updates `github.com/spf13/pflag` from 1.0.9 to 1.0.10 - [Release notes](https://github.com/spf13/pflag/releases) - [Commits](spf13/pflag@v1.0.9...v1.0.10) --- updated-dependencies: - dependency-name: github.com/spf13/pflag dependency-version: 1.0.10 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: production-dependencies ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [actions/setup-go](https://github.com/actions/setup-go) from 5 to 6. - [Release notes](https://github.com/actions/setup-go/releases) - [Commits](actions/setup-go@v5...v6) --- updated-dependencies: - dependency-name: actions/setup-go dependency-version: '6' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps golang from 1.25.0 to 1.25.1. --- updated-dependencies: - dependency-name: golang dependency-version: 1.25.1 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…y with 4 updates (sqlc-dev#4100) Bumps the production-dependencies group with 4 updates in the / directory: [github.com/jackc/pgx/v5](https://github.com/jackc/pgx), [golang.org/x/sync](https://github.com/golang/sync), [google.golang.org/grpc](https://github.com/grpc/grpc-go) and google.golang.org/protobuf. Updates `github.com/jackc/pgx/v5` from 5.7.5 to 5.7.6 - [Changelog](https://github.com/jackc/pgx/blob/master/CHANGELOG.md) - [Commits](jackc/pgx@v5.7.5...v5.7.6) Updates `golang.org/x/sync` from 0.16.0 to 0.17.0 - [Commits](golang/sync@v0.16.0...v0.17.0) Updates `google.golang.org/grpc` from 1.75.0 to 1.75.1 - [Release notes](https://github.com/grpc/grpc-go/releases) - [Commits](grpc/grpc-go@v1.75.0...v1.75.1) Updates `google.golang.org/protobuf` from 1.36.8 to 1.36.9 --- updated-dependencies: - dependency-name: github.com/jackc/pgx/v5 dependency-version: 5.7.6 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: production-dependencies - dependency-name: golang.org/x/sync dependency-version: 0.17.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: production-dependencies - dependency-name: google.golang.org/grpc dependency-version: 1.75.1 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: production-dependencies - dependency-name: google.golang.org/protobuf dependency-version: 1.36.9 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: production-dependencies ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…y with 3 updates (sqlc-dev#4130) Bumps the production-dependencies group with 3 updates in the / directory: [google.golang.org/grpc](https://github.com/grpc/grpc-go), google.golang.org/protobuf and [modernc.org/sqlite](https://gitlab.com/cznic/sqlite). Updates `google.golang.org/grpc` from 1.75.1 to 1.76.0 - [Release notes](https://github.com/grpc/grpc-go/releases) - [Commits](grpc/grpc-go@v1.75.1...v1.76.0) Updates `google.golang.org/protobuf` from 1.36.9 to 1.36.10 Updates `modernc.org/sqlite` from 1.38.2 to 1.39.0 - [Commits](https://gitlab.com/cznic/sqlite/compare/v1.38.2...v1.39.0) --- updated-dependencies: - dependency-name: google.golang.org/grpc dependency-version: 1.76.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: production-dependencies - dependency-name: google.golang.org/protobuf dependency-version: 1.36.10 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: production-dependencies - dependency-name: modernc.org/sqlite dependency-version: 1.39.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: production-dependencies ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…y with 4 updates (sqlc-dev#4127) Bumps the production-dependencies group with 4 updates in the /docs directory: [markupsafe](https://github.com/pallets/markupsafe), [certifi](https://github.com/certifi/python-certifi), [pyparsing](https://github.com/pyparsing/pyparsing) and [sphinxext-rediraffe](https://github.com/sphinx-doc/sphinxext-rediraffe). Updates `markupsafe` from 3.0.2 to 3.0.3 - [Release notes](https://github.com/pallets/markupsafe/releases) - [Changelog](https://github.com/pallets/markupsafe/blob/main/CHANGES.rst) - [Commits](pallets/markupsafe@3.0.2...3.0.3) Updates `certifi` from 2025.8.3 to 2025.10.5 - [Commits](certifi/python-certifi@2025.08.03...2025.10.05) Updates `pyparsing` from 3.2.3 to 3.2.5 - [Release notes](https://github.com/pyparsing/pyparsing/releases) - [Changelog](https://github.com/pyparsing/pyparsing/blob/master/CHANGES) - [Commits](pyparsing/pyparsing@3.2.3...3.2.5) Updates `sphinxext-rediraffe` from 0.2.7 to 0.3.0 - [Release notes](https://github.com/sphinx-doc/sphinxext-rediraffe/releases) - [Commits](sphinx-doc/sphinxext-rediraffe@v0.2.7...v0.3.0) --- updated-dependencies: - dependency-name: markupsafe dependency-version: 3.0.3 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: production-dependencies - dependency-name: certifi dependency-version: 2025.10.5 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: production-dependencies - dependency-name: pyparsing dependency-version: 3.2.5 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: production-dependencies - dependency-name: sphinxext-rediraffe dependency-version: 0.3.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: production-dependencies ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps the production-dependencies group with 1 update: [modernc.org/sqlite](https://gitlab.com/cznic/sqlite). Updates `modernc.org/sqlite` from 1.39.0 to 1.39.1 - [Commits](https://gitlab.com/cznic/sqlite/compare/v1.39.0...v1.39.1) --- updated-dependencies: - dependency-name: modernc.org/sqlite dependency-version: 1.39.1 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: production-dependencies ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps golang from 1.25.1 to 1.25.3. --- updated-dependencies: - dependency-name: golang dependency-version: 1.25.3 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Create the sqlfile.Split method that splits a .sql file into multiple statements.
…qlc-dev#4137) Bumps the production-dependencies group in /docs with 1 update: [idna](https://github.com/kjd/idna). Updates `idna` from 3.10 to 3.11 - [Release notes](https://github.com/kjd/idna/releases) - [Changelog](https://github.com/kjd/idna/blob/master/HISTORY.rst) - [Commits](kjd/idna@v3.10...v3.11) --- updated-dependencies: - dependency-name: idna dependency-version: '3.11' dependency-type: direct:production update-type: version-update:semver-minor dependency-group: production-dependencies ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Author
|
@andrewmbenton please review |
This was referenced Nov 10, 2025
|
Thanks @ignat980 this looks a much more complete solution than I was expecting |
Collaborator
|
@ignat980 if you open this against main I can get this merged. Just make sure to includes Andrew's commits. |
kyleconroy
approved these changes
Dec 10, 2025
Replace naive line-based removal with a single-pass state machine that correctly distinguishes psql meta-commands from backslashes in SQL code, literals, and comments.
The previous implementation would incorrectly strip any line starting with a backslash, breaking valid SQL containing:
- Backslashes in string literals (E'\\n', escape sequences)
- Meta-command text in comments or documentation
- Dollar-quoted function bodies with backslash content
Changes:
- Track parsing state for single quotes, dollar quotes, and block comments
- Only remove backslash commands at true line starts outside any literal context
- Properly handle escaped quotes (''), nested block comments (/* /* */ */)
- Support dollar-quoted tags with identifiers ($tag$...$tag$)
- Add comprehensive test suite covering:
* All documented psql meta-commands (\connect, \set, \d*, etc.)
* String literals with backslashes and nested quotes
* Dollar-quoted blocks with various tag formats
* Nested block comments containing meta-command text
* Edge cases: empty input, whitespace-only, missing newlines
Performance improvements:
- Pre-allocate output buffer with strings.Builder.Grow()
- Single pass eliminates redundant string operations
- Reduces allocations by avoiding intermediate line slice
3436dde to
60585cd
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
fix(compiler): robustly strip psql meta commands without breaking SQL
Replace naive line-based removal with a single-pass state machine that correctly distinguishes psql meta-commands from backslashes in SQL code, literals, and comments.
The previous implementation would incorrectly strip any line starting with a backslash, breaking valid SQL containing:
E'\\n', escape sequences)Changes:
''), nested block comments (/* /* */ */)$tag$...$tag$)\connect,\set,\d*, etc.) See PostgreSQLpsqldocsPerformance improvements:
strings.Builder.Grow()Testing
go test ./internal/compilerremovePsqlMetaCommands()Addresses gbarr's comment in #4082 which closes #4065