Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
128 changes: 128 additions & 0 deletions NOTICE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,128 @@
# Third-Party Notices

## ANTLR grammars-v4 PostgreSQL grammar

The PostgreSQL grammar and its Java support sources under
`src/main/antlr4/ai/singlr/postgresql/parser/` and
`src/main/java/ai/singlr/postgresql/parser/` are vendored from the
[ANTLR grammars-v4](https://github.com/antlr/grammars-v4) project.

| | |
|---|---|
| Upstream repository | `https://github.com/antlr/grammars-v4` |
| Upstream path | `sql/postgresql` |
| Pinned commit | `76093c04af6a51f38a67d14f7e71ff0a9b4400da` (2026-06-20) |

Vendored files:

- `PostgreSQLLexer.g4` (from `sql/postgresql/PostgreSQLLexer.g4`)
- `PostgreSQLParser.g4` (from `sql/postgresql/PostgreSQLParser.g4`)
- `COPYRIGHT` (from `sql/postgresql/COPYRIGHT`, verbatim)
- `PostgreSQLLexerBase.java`, `PostgreSQLParserBase.java`,
`LexerDispatchingErrorListener.java`, `ParserDispatchingErrorListener.java`
(from `sql/postgresql/Java/`)

The test corpus under `src/test/resources/postgresql-corpus/` is vendored from
`sql/postgresql/examples/` at the same commit. Generated lexer/parser sources are
build output under `target/generated-sources/` and are never hand-edited.

### Local modifications

Every deviation from upstream is listed here and marked with a
`scim-sql local modification` comment at the change site:

1. `PostgreSQLParser.g4` — added `plsqlvariablename` as a labeled `c_expr`
alternative (`# c_expr_namedparam`) so named parameters (`:start_at`,
`:user_id`) are first-class expression values.
2. `PostgreSQLParser.g4` — removed `PLSQLVARIABLENAME` from the `identifier`
rule so `:name` can never be an identifier, alias, or relation name.
3. `PostgreSQLParser.g4` — split `createfunctionstmt` into a header plus
either the upstream AS-string option list or the PostgreSQL 14+ unquoted
SQL body (`RETURN expr` / `BEGIN ATOMIC stmt; ... END`), which upstream
does not parse.
4. `PostgreSQLParser.g4` — closed modern-PostgreSQL gaps against `gram.y`
(PostgreSQL 14 through 18), each also marked in place:
`json_aggregate_func` was defined upstream but never referenced (wired
into `func_expr` / `func_expr_windowless`, and its `json_returning_clause`
made optional as in `gram.y`); MERGE gained the PostgreSQL 17 `RETURNING`
clause and `WHEN NOT MATCHED BY SOURCE | BY TARGET`; `group_clause`
accepts the PostgreSQL 14 `ALL | DISTINCT` set quantifier;
`common_table_expr` gained the PostgreSQL 14 `SEARCH` and `CYCLE`
clauses; the PostgreSQL 16 `IS [NOT] JSON` predicate was added to
`a_expr_is_not`; the PostgreSQL 17 `JSON_TABLE` table function was added
and wired into `table_ref`; `UNIQUE`/`PRIMARY KEY` constraints accept the
PostgreSQL 18 `WITHOUT OVERLAPS` marker; `xmltable_column_option_el`
accepts `PATH` explicitly (the lexer keyword cannot match the generic
identifier option); `json_format_clause` uses the real `FORMAT` token —
upstream's `FORMAT_LA` literal is Bison lookahead-token residue that
never occurs in SQL text, so `FORMAT JSON` could never parse.
5. `PostgreSQLLexer.g4` — fixed the upstream `BREADTH: 'BREATH'` typo that
made `SEARCH BREADTH FIRST` unparseable.
6. `PostgreSQLLexerBase.java` — `nextToken()` splits a greedy
`PLSQLVARIABLENAME` match into `COLON` plus the re-lexed name whenever the
previous default-channel token can end an expression, so JSON `key:value`
separators and array-slice bounds (`arr[lo:hi]`) written without
whitespace stay operators instead of becoming named parameters.
7. `PostgreSQLLexerBase.java` — nested block comments are lexed iteratively
with a depth counter instead of the upstream recursive rule, which was
quadratic-time and could overflow the stack on adversarial nesting.
8. `*.java` support files — added a
`package ai.singlr.postgresql.parser;` declaration (upstream files have no
package). The files are excluded from code formatting to keep them
diffable against upstream.

Known consequences of the `:name` named-parameter extension:

- A colon directly after `[` binds to the parameter extension, so
`arr[:name]` is a subscript by named parameter, not a slice with an
omitted lower bound. Slices with an expression lower bound (`arr[lo:hi]`,
`arr[1:2]`) are unaffected.
- The `:"identifier"` PL/SQL form is rejected.

### Expected grammar warnings

Upstream ships two benign ANTLR warning-146 lexer rules
(`AfterEscapeStringConstantMode_NotContinued` and
`AfterEscapeStringConstantWithNewlineMode_NotContinued`). These are pinned by
`GrammarWarningsTest`; the build fails on any new grammar warning or error.

### Upgrading the grammar

1. Pick a new upstream commit and re-copy the files listed above.
2. Re-apply the local modifications (search for `scim-sql local modification`).
3. Update the pinned commit here and in `UpstreamCorpusTest`, refresh the
corpus files, and run `mvn clean verify`. Review any change in
`GrammarWarningsTest` expectations deliberately.

### Licenses

The grammar files carry the MIT license of their authors (Tunnel Vision
Laboratories, LLC and Oleksii Kovalov) in their headers, preserved verbatim.
The upstream `COPYRIGHT` file is preserved verbatim next to the grammars and
reproduced here as required:

```
PostgreSQL Database Management System
(formerly known as Postgres, then as Postgres95)

Portions Copyright (c) 1996-2020, PostgreSQL Global Development Group

Portions Copyright (c) 1994, The Regents of the University of California

Permission to use, copy, modify, and distribute this software and its
documentation for any purpose, without fee, and without a written agreement
is hereby granted, provided that the above copyright notice and this
paragraph and the following two paragraphs appear in all copies.

IN NO EVENT SHALL THE UNIVERSITY OF CALIFORNIA BE LIABLE TO ANY PARTY FOR
DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES, INCLUDING
LOST PROFITS, ARISING OUT OF THE USE OF THIS SOFTWARE AND ITS
DOCUMENTATION, EVEN IF THE UNIVERSITY OF CALIFORNIA HAS BEEN ADVISED OF THE
POSSIBILITY OF SUCH DAMAGE.

THE UNIVERSITY OF CALIFORNIA SPECIFICALLY DISCLAIMS ANY WARRANTIES,
INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY
AND FITNESS FOR A PARTICULAR PURPOSE. THE SOFTWARE PROVIDED HEREUNDER IS
ON AN "AS IS" BASIS, AND THE UNIVERSITY OF CALIFORNIA HAS NO OBLIGATIONS TO
PROVIDE MAINTENANCE, SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS.
```
33 changes: 32 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
# scim-sql

SCIM filter expression to parameterized SQL converter for **PostgreSQL**. Parses [RFC 7644](https://datatracker.ietf.org/doc/html/rfc7644#section-3.4.2.2) filter strings and produces SQL WHERE clauses with named parameters — safe from injection by design.
Two independent capabilities for **PostgreSQL** in one small library:

1. **SCIM filter → SQL** (`ai.singlr.scimsql`): parses [RFC 7644](https://datatracker.ietf.org/doc/html/rfc7644#section-3.4.2.2) filter strings and produces SQL WHERE clauses with named parameters — safe from injection by design. It parses SCIM filter expressions only; it does not validate full SQL.
2. **PostgreSQL query analysis** (`ai.singlr.postgresql`): parses complete SQL statements and reports structural facts — statement kind and count, relations, columns, functions, named parameters, and policy-relevant features — without executing SQL or resolving catalog objects. See [PostgreSQL Query Analysis](#postgresql-query-analysis).

The generated SQL uses PostgreSQL-specific syntax for typed values (`CAST(… AS UUID)`, `CAST(… AS timestamptz)`, `CAST(… AS jsonb)`, `@>` for JSON containment). Standard comparisons (`=`, `!=`, `LIKE`, `IN`, `IS NOT NULL`) are portable across databases. The `compareFilterBuilder` extension point allows overriding SQL generation for other databases.

Expand Down Expand Up @@ -121,6 +124,34 @@ var filter = engine.parseFilter(

This lets you intercept `ComparisonFilter` instances and return a subclass with custom `toClause()` or `paramKey()` behavior.

## PostgreSQL Query Analysis

`PostgresQueryAnalyzer.analyze(String)` parses a complete PostgreSQL statement (or script) through EOF and returns an immutable `QueryAnalysis`:

```java
var analysis = PostgresQueryAnalyzer.analyze(
"SELECT u.id, count(*) FROM users u WHERE u.created_at >= :start_at GROUP BY u.id");

analysis.statementKind(); // SELECT
analysis.statementCount(); // 1
analysis.relations(); // [RelationReference[schema=null, name=users, alias=u, kind=PHYSICAL]]
analysis.columns(); // [u.id, u.created_at, u.id]
analysis.functions(); // [count at 1:17]
analysis.parameters(); // [start_at]
analysis.features(); // []
analysis.normalizedSql(); // deterministic form for hashing/audit
```

Key properties:

- **Named parameters** (`:start_at`, `:user_id`) are first-class expression values via a deliberate, documented grammar extension. Names are reported exactly; values are never bound or substituted.
- **Statement kinds**: `SELECT`, `INSERT`, `UPDATE`, `DELETE`, `MERGE`, `DDL`, `UTILITY`, `UNKNOWN`. Prohibited-but-valid statements analyze successfully so callers can raise precise policy errors.
- **Features** flag CTEs (plain/recursive/writable), subqueries, set operations, window functions, `SELECT INTO`, row locks, `LATERAL`, function/VALUES relations, star projections, and multiple statements — at any nesting depth.
- **Relations** distinguish physical tables, CTE references, and function relations, preserving aliases and schema qualification. Classification is syntactic; the analyzer does not resolve catalog objects, authorize access, execute SQL, or decide policy.
- **Safety**: input is bounded (length, tokens, nesting depth), errors carry only a stable reason plus line/column, and SQL text is never logged or echoed.

The grammar is the [ANTLR grammars-v4](https://github.com/antlr/grammars-v4) PostgreSQL grammar, vendored at a pinned commit — see [NOTICE.md](NOTICE.md) for provenance, licenses, and the exact local modifications.

## Building

Requires JDK 25+ and Maven.
Expand Down
32 changes: 30 additions & 2 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,11 @@

<groupId>ai.singlr</groupId>
<artifactId>scim-sql</artifactId>
<version>1.0.1-SNAPSHOT</version>
<version>1.1.0-SNAPSHOT</version>
<packaging>jar</packaging>

<name>scim-sql</name>
<description>SCIM filter expression to parameterized SQL converter</description>
<description>SCIM filter expression to parameterized SQL converter and PostgreSQL query analyzer</description>
<url>https://github.com/singlr-ai/scim-sql</url>

<licenses>
Expand Down Expand Up @@ -69,9 +69,32 @@
<artifactId>junit-jupiter</artifactId>
<scope>test</scope>
</dependency>

<dependency>
<groupId>org.antlr</groupId>
<artifactId>antlr4</artifactId>
<version>${antlr4-version}</version>
<scope>test</scope>
</dependency>
</dependencies>

<build>
<resources>
<resource>
<directory>${project.basedir}</directory>
<includes>
<include>NOTICE.md</include>
</includes>
<targetPath>META-INF</targetPath>
</resource>
<resource>
<directory>src/main/antlr4/ai/singlr/postgresql/parser</directory>
<includes>
<include>COPYRIGHT</include>
</includes>
<targetPath>META-INF</targetPath>
</resource>
</resources>
<plugins>
<plugin>
<groupId>org.antlr</groupId>
Expand All @@ -80,6 +103,7 @@
<configuration>
<visitor>true</visitor>
<listener>false</listener>
<treatWarningsAsErrors>false</treatWarningsAsErrors>
</configuration>
<executions>
<execution>
Expand Down Expand Up @@ -109,6 +133,9 @@
<version>3.2.1</version>
<configuration>
<java>
<excludes>
<exclude>src/main/java/ai/singlr/postgresql/parser/**</exclude>
</excludes>
<googleJavaFormat>
<version>1.27.0</version>
<style>GOOGLE</style>
Expand Down Expand Up @@ -143,6 +170,7 @@
<exclude>ai/singlr/scimsql/ScimVisitor*</exclude>
<exclude>ai/singlr/scimsql/ScimListener*</exclude>
<exclude>ai/singlr/scimsql/ScimBaseListener*</exclude>
<exclude>ai/singlr/postgresql/parser/**</exclude>
</excludes>
</configuration>
<executions>
Expand Down
23 changes: 23 additions & 0 deletions src/main/antlr4/ai/singlr/postgresql/parser/COPYRIGHT
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
PostgreSQL Database Management System
(formerly known as Postgres, then as Postgres95)

Portions Copyright (c) 1996-2020, PostgreSQL Global Development Group

Portions Copyright (c) 1994, The Regents of the University of California

Permission to use, copy, modify, and distribute this software and its
documentation for any purpose, without fee, and without a written agreement
is hereby granted, provided that the above copyright notice and this
paragraph and the following two paragraphs appear in all copies.

IN NO EVENT SHALL THE UNIVERSITY OF CALIFORNIA BE LIABLE TO ANY PARTY FOR
DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES, INCLUDING
LOST PROFITS, ARISING OUT OF THE USE OF THIS SOFTWARE AND ITS
DOCUMENTATION, EVEN IF THE UNIVERSITY OF CALIFORNIA HAS BEEN ADVISED OF THE
POSSIBILITY OF SUCH DAMAGE.

THE UNIVERSITY OF CALIFORNIA SPECIFICALLY DISCLAIMS ANY WARRANTIES,
INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY
AND FITNESS FOR A PARTICULAR PURPOSE. THE SOFTWARE PROVIDED HEREUNDER IS
ON AN "AS IS" BASIS, AND THE UNIVERSITY OF CALIFORNIA HAS NO OBLIGATIONS TO
PROVIDE MAINTENANCE, SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS.
Loading
Loading