Skip to content

Add opt-in psql parsing - #143

Open
arnodirlam wants to merge 2 commits into
nene:masterfrom
arnodirlam:arno/psql-grammar-poc
Open

arnodirlam wants to merge 2 commits into
nene:masterfrom
arnodirlam:arno/psql-grammar-poc

Conversation

@arnodirlam

@arnodirlam arnodirlam commented Oct 31, 2026 •

Copy link
Copy Markdown

Hi Nene,

This lets callers parse PostgreSQL scripts that mix SQL with standalone psql commands and variable references. The psql mode is opt-in, so existing parsing stays unchanged by default. It follows your suggestion to put this support in the parser and relates to prettier-plugin-sql-cst#83.

I added a prepare build so the branch can be consumed as a Git dependency while the formatter integration is reviewed. That follow-up is in prettier-plugin-sql-cst#99.

I left COPY FROM STDIN for a separate parser change.

If this isn’t the direction you had in mind, feel free to close the PR. I’d be happy to adjust it around what you’d prefer.

@nene nene left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the pull request.

I wrote bunch of comments below, but these are sort of secondary in nature... First off I'd like to understand your approach to this psql parsing in general.

For example, how does your implementation compare to how the parsing of these psql commands works in the actual psql implementation? What I personally don't really understand is whether the psql tool actually parses the SQL by itself (like your implementation seems to suggest), or whether it works more akin to a templating system, where it just e.g. reads the text until the next semicolon and sends it to server. I think this core understanding should be also guiding us in how we should fundamentally implement this psql support.

Comment thread README.md
Parses SQL string and returns the CST tree. Takes the following options:

- **dialect**: `'sqlite' | 'bigquery' | 'mysql' | 'mariadb' | 'postgresql' | 'plpgsql'` The SQL dialect to parse **(required)**.
- **psql**: `boolean` Enables parsing standalone psql backslash commands and variable references for PostgreSQL. Command text is preserved.

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think a better approach would be to treat psql as a separate SQL dialect, not as a flag that's only applicable to PostgreSQL.

Comment thread test/psql.test.ts
"\\echo `printf hi`\n",
"\\unknown value\n",
"SELECT :id, :'name', :\"table\", :{?enabled};",
])("round-trips %s", (sql) => {

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test looks kinda horrendous. All these escape codes make my head hurt. Plus, it looks like we're testing all kinds of different things in this single it.each() block.

I think we should split this up to several separate tests, each with a clear description of what exactly we're testing.

Maybe we could use String.raw to avoid writing all these escapes. Or maybe we can adopt the dedent helper to aid us. Not entirely sure, but in the current form, this code is pretty hard to understand.


export const transformMap: FullTransformMap<string> = {
psql_command: (node) => node.text,
psql_query: (node) => show(node.query) + node.terminator,

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not good. This file should only import various -Map objects and combine them to one large transformMap.

Comment thread src/cst/Expr.ts
Comment on lines -439 to 444
export interface Identifier extends BaseNode {
export type Identifier = PlainIdentifier | PsqlIdentifier;

export interface PlainIdentifier extends BaseNode {
type: "identifier";

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't good. We should keep the type field in sync with type name. Like, when type is called PlainIdentifier, then I'd expect it to have type: "plain_identifier".

In here, we likely don't want to change the type: "identifier" at all.

I'm unsure about the best direction where we should go. I guess the main intention in here is to allow Psql identifiers in all places where previously only SQL identifiers were allowed. But frankly, I'm not really even sure if we should allow that in the first place.

@arnodirlam

Copy link
Copy Markdown
Author

Thanks for raising this. I looked at psql’s scanner and documentation. It scans SQL lexically, tracking quotes, comments, and statement boundaries while building a query buffer. Full SQL parsing happens on the server. Backslash commands can interrupt unfinished SQL, and unquoted variable substitutions can supply arbitrary SQL text.

My implementation assumes complete SQL statements with psql constructs added in selected positions. That covers a subset, but doesn’t capture the more general processing model.

I’d propose exposing psql as a separate dialect, as you suggested, with a script layer inside this parser. It would preserve command text, source order, and query boundaries, then delegate suitable SQL regions to the existing PostgreSQL parser. Keeping boundaries explicit matters for cases like \gset, where inserting a semicolon changes behavior.

For a first step, I’d focus on standalone commands and ordinary SQL, leaving Identifier unchanged. Structural support for variable interpolation could follow separately: :'name' and :"name" have constrained roles, while raw :name can change the SQL structure entirely.

Would you prefer that this dialect preserve unsupported SQL regions as opaque source text, or reject them explicitly? Opaque regions would let the formatter handle surrounding SQL, but callers would receive a partially parsed tree. Strict rejection would keep the parser contract simpler, with less formatting coverage.

I lean toward preserving opaque regions where their boundaries are reliable. Where boundaries themselves are uncertain, the formatter would leave the whole script unchanged rather than guessing. Does that direction fit what you had in mind?

@nene

nene commented Nov 12, 2026

Copy link
Copy Markdown
Owner

Why do I have the feeling that I'm reading a response from an LLM, not the words of Arno Dirlam?

If I wanted to talk with an LLM, I could do it the whole day... but where's the fun in that?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants