Skip to content

fix(semantics): bind join conditions, merged keys and named windows as each engine does - #270

Closed
kvlonge wants to merge 12 commits into
mainfrom
fix/join-and-window-binding
Closed

kvlonge wants to merge 12 commits into
mainfrom
fix/join-and-window-binding

Conversation

@kvlonge

@kvlonge kvlonge commented Nov 19, 2026

Copy link
Copy Markdown
Contributor

Why

Semantics accepted SQL its engines refuse, and refused some they accept:

  • An ON condition read every FROM item. PostgreSQL lets it read only its own comma group up to the join; DuckDB, every earlier item.
  • An unqualified USING or NATURAL key beside another item's column of that name is ambiguous on PostgreSQL and SQLite. A NATURAL join of sides sharing no column is an error on DuckDB.
  • Named windows: a case-folded name gave a false E232, an unknown base window and unknown frame offsets were missed, and window keys read output aliases where SQLite and BigQuery refuse them. SQLite's select list also read other projections' aliases.
  • The shared parser dropped a window name's quotes. Databricks lost raw strings in path keys and resources, and the spacing of RETURNS TABLE (...) reached the tree.

Changes

  • New dialect rules: which items a join condition reaches, whether a merged key hides other items, whether the select list reads outputs, and whether window keys do. Each matches the engine's binding.
  • A join condition sees each merged key as it stood at that join.
  • A named window is resolved per reading projection, with its bases and frame offsets, and found with the profile's identifier rule.
  • Window names keep quoted; Databricks keeps RawString path keys and resources and normalises RETURNS TABLE spacing. No public type changes.

Verification

  • Engine-checked cases for each rule against PostgreSQL 18, DuckDB 2.0 and SQLite 3.40, each failing on main.
  • One review and one follow-up check of its fixes. The two false negatives the follow-up found are fixed and covered.
  • make check-ci on the head. Analysis instruction count +1.93%, within the 2% tolerance; parse −0.03%.

A raw string in a semi-structured path key and in a resource gave the
same tree as the plain string: SELECT c1 : [ r'k' ] FROM t1 and
SELECT c1 : [ 'k' ] FROM t1, and ADD FILE r'v1' and ADD FILE 'v1' (the
same for ADD JAR and LIST FILE).

A path key now keeps r'k' as a raw string literal. A resource unquotes
only a plain string; a raw one is kept as written, as Spark's resource
commands unquote only text that begins with a quote.
…cing

A function's RETURNS TABLE ( ... ) kept its column list as the source
text, whitespace and comments included, so reformatting the statement
changed its tree and the formatter round trip failed:
RETURNS TABLE ( c1 INT , c2 STRING ) and RETURNS TABLE(c1 INT, c2 STRING)
gave two trees.

returns_table_body now holds the list's tokens separated by single
spaces, so spacing and comments no longer reach the tree while every
name and type still does.
An OVER window name was looked up among the WINDOW clause's names by its
exact key, so SELECT sum(b) OVER W AS x FROM t WINDOW w AS (...) gave a
false E232 "Unknown named window 'W'" in DuckDB and SQLite, which
resolve W to w. The check for which windows an OVER reaches already
compared names the profile's way.

The lookup now compares names with the profile's identifier rule too. A
DuckDB case covers a window named in another case.
A named window can extend another, as WINDOW w2 AS (zz ORDER BY b). When
an OVER read w2 and zz named no window, no finding was reported, while
PostgreSQL, DuckDB and SQLite refuse the query. The window an OVER
reads is now followed along its bases, and a base no WINDOW clause names
gives E232 "Unknown named window" at the base's name. As for a window's
keys, a window no OVER reaches is not checked.

DuckDB cases cover a direct and a two-step unknown base and an unused
window.
The shared parser built window names without their quoting: the name
after OVER, a base named inside OVER ( ... ) or a WINDOW definition, and
the WINDOW clause's own names all came out unquoted, and a quoted name
after WINDOW did not parse, because WINDOW followed by a quoted name read
as an alias. Semantics parses every dialect with this parser, so in
PostgreSQL SELECT sum(x) OVER "W" ... WINDOW w AS (...) resolved "W" to
w instead of reporting E232, and WINDOW "w" AS (...) was a parse error.

Each of these names now keeps whether it was quoted, and WINDOW followed
by a quoted name starts the clause. Formatting reads tokens and is
unchanged. A parser case covers each name position, and PostgreSQL cases
cover a quoted name that does not match, one that does, and an
unquoted name that folds to a quoted lower-case window.
…nces

A named window's PARTITION BY and ORDER BY keys were resolved, but not
the offsets of its frame, so WINDOW w AS (ORDER BY id ROWS BETWEEN nope
PRECEDING AND CURRENT ROW) reported nothing while the same frame written
inline in OVER gave E201.

The offsets of a used window's frame bounds now resolve with its keys,
in the same scope. DuckDB cases cover an unknown column in a start
offset, the inline form, and an unknown qualifier in an end offset.
…t reads

Where a SELECT's outputs can be read anywhere, an inline OVER key that
reads its own projection's alias gave E201, but the same key in a named
window did not: SELECT sum(amount) OVER w AS x FROM orders WINDOW w AS
(ORDER BY x) reported nothing, while DuckDB refuses both forms.

A used window's keys are now read once for each projection whose OVER
reaches it, with that projection's own alias hidden, as for an inline
window; a window only other clauses reach is read as before. A finding a
window shared by several projections gives is reported once. DuckDB
cases cover the own alias, a shared window ordered by one reader's
alias, and an unknown column in a shared window.
…put aliases

SQLite and BigQuery let a SELECT's clauses read its output aliases, and
the window keys of the select list were read the same way, so
SELECT quantity AS q, sum(quantity) OVER (PARTITION BY q) AS x FROM orders
reported nothing. SQLite 3.40 refuses it with "no such column: q",
inline or through a named window, and BigQuery's OVER cannot read an
output alias either.

A dialect now says whether the keys of a window the select list reads
may read output aliases. For SQLite and BigQuery they read only the
input columns; a window in ORDER BY still reads the aliases, as SQLite
accepts. Other dialects are unchanged. SQLite cases cover an inline and
a named window that refuse the alias and two ORDER BY windows that
accept it; a BigQuery case covers an inline window.
SQLite resolves output aliases in WHERE, GROUP BY, HAVING and ORDER BY,
and in subqueries read there, but not in the select list itself: both
`SELECT quantity AS q, q + 1 FROM orders` and a scalar subquery in the
select list reading `q` fail with "no such column". A new dialect rule,
select_list_reads_outputs, is false for SQLite only, so its select list
reads input columns and reports E201 for an output alias.
DuckDB refuses a NATURAL join whose sides have no column name in common
("No columns found to join on in NATURAL JOIN"), where PostgreSQL and
SQLite treat it as a cross join. On DuckDB, a NATURAL join of closed
sides that share no name now reports E201 at the joined item. Only the
join's own left side counts, not a comma neighbour, matching DuckDB 2.0.
…column on PostgreSQL and SQLite

PostgreSQL and SQLite read an unqualified USING or NATURAL key as the
merged column only when no FROM item outside that join offers the name:
`a JOIN b USING (id), c` and `a JOIN b USING (id) JOIN c ON ...` with a
`c.id` are "ambiguous" there, while DuckDB binds the merged key. A new
dialect rule, merged_keys_hide_other_items, is false for PostgreSQL and
SQLite, which now report E221 when a merged key and an uncovered item
both offer the name. A later USING or NATURAL join merging the key again
still reads it once.
… it see

An ON condition was checked against every FROM item of its level. In
PostgreSQL it sees only its own comma group, up to the join: `a, b JOIN
c ON a.id = c.id` is an "invalid reference to FROM-clause entry", also
from a subquery or a parenthesised join, and an ON condition reading a
later comma group or a later item of its own join chain is a "missing
FROM-clause entry". DuckDB lets it read every earlier item and rejects
only later ones. A new dialect rule, join_conditions_read, gives each
join condition the items it reaches: its own group on PostgreSQL, every
earlier item on DuckDB, and every item elsewhere, as before. The
existing name checks then report E222 or E201.
@kvlonge
kvlonge added this pull request to the merge queue Nov 19, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Nov 19, 2026

kvlonge commented Nov 19, 2026

Copy link
Copy Markdown
Contributor Author

Superseded by #271, which carries this PR's 12 commits unchanged together with the analysis performance work. On its own, this PR failed the merge queue's instruction-count check (analysis +3.38% against a 2% tolerance); combined in #271 analysis is 24% lower, with byte-identical lineage and findings. The branch is kept.

@kvlonge kvlonge closed this Nov 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant