Repository navigation
Conversation
- Closes apache#26462. `SELECT lower(e.email) AS email FROM employees e ORDER BY e.email` failed to plan: Schema error: Schema contains qualified field name e.email and unqualified field name email which would be ambiguous The sort key is not in the select list, so the planner carries `e.email` into the projection to sort on it. That projection then holds the qualified field `e.email` and the unqualified field `email` in one schema, which `DFSchema::check_names` rejects, even though the carried column is internal and is projected away above the sort. PostgreSQL plans the same query and sorts on the input column. `LogicalPlanBuilder::sort` (`datafusion/expr/src/logical_plan/builder.rs`) now carries a missing sort column under an alias when the projection it is added to already outputs an unqualified field with the same name. The alias is the column's flat name (`e.email`), so the carried column cannot collide with the projection's other fields, and the sorts are rebound to it. The projection above the sort still strips the extra column, so the output schema of the query is unchanged. `datafusion/sqllogictest/test_files/order.slt` gets the query from the issue and its `ORDER BY email` counterpart, which pins the two resolutions apart: `ORDER BY e.email` sorts on the input column (`b@x`, `a@y`), and `ORDER BY email` sorts on the output column (`a@y`, `b@x`), as in PostgreSQL. A unit test in `datafusion/expr/src/logical_plan/builder.rs` covers the same path through `LogicalPlanBuilder` and checks the resulting plan. Validation, on 61c3472: - `cargo test -p datafusion-expr` passed: 285 unit tests, 63 documentation tests. - `cargo test -p datafusion-sql` passed: 95, 604 and 12 tests. - `cargo test -p datafusion-optimizer` passed: 941, 29 and 5 tests. - `cargo test -p datafusion --lib` passed: 468 tests. - `cargo test --profile ci -p datafusion-sqllogictest --test sqllogictests` passed: 534 of 534 files. - `cargo clippy -p datafusion-expr --all-targets --all-features -- -D warnings` and `cargo fmt --all -- --check` passed. The `order.slt` case was also run against the unpatched code, where it fails with the schema error quoted above. None beyond the query now planning. The result and the schema of the query are the ones PostgreSQL returns; plans that already planned are unchanged.
6ae3a07 to
6edc624
Compare
|
Rebased onto One conflict, in Re-run after the rebase:
|
|
Thanks for the contribution! I am closing this under DataFusion's AI Dump policy. This has the hallmarks of AI code that was not reviewed by the author: a commit message that is the PR description verbatim, per-crate test counts as validation, and a defensive loop for a case that does not arise. If you would like to continue, please open a focused PR in your own words that you have reviewed and understand, and we are happy to help with review. |
Which issue does this PR close?
ORDER BYon a source column fails when a select item is aliased to that column's name #26462.Rationale for this change
SELECT lower(e.email) AS email FROM employees e ORDER BY e.emailfailed toplan:
The sort key is not in the select list, so the planner carries
e.emailintothe projection to sort on it. That projection then holds the qualified field
e.emailand the unqualified fieldemailin one schema, whichDFSchema::check_namesrejects, even though the carried column is internal andis projected away above the sort. PostgreSQL plans the same query and sorts on
the input column.
What changes are included in this PR?
LogicalPlanBuilder::sort(datafusion/expr/src/logical_plan/builder.rs) nowcarries a missing sort column under an alias when the projection it is added to
already outputs an unqualified field with the same name. The alias is the
column's flat name (
e.email), so the carried column cannot collide with theprojection's other fields, and the sorts are rebound to it. The projection
above the sort still strips the extra column, so the output schema of the query
is unchanged.
What is the testing strategy for this PR?
datafusion/sqllogictest/test_files/order.sltgets the query from the issue andits
ORDER BY emailcounterpart, which pins the two resolutions apart:ORDER BY e.emailsorts on the input column (b@x,a@y), andORDER BY emailsorts on the output column (a@y,b@x), as in PostgreSQL.A unit test in
datafusion/expr/src/logical_plan/builder.rscovers the samepath through
LogicalPlanBuilderand checks the resulting plan.Validation, on 61c3472:
cargo test -p datafusion-exprpassed: 285 unit tests, 63 documentation tests.cargo test -p datafusion-sqlpassed: 95, 604 and 12 tests.cargo test -p datafusion-optimizerpassed: 941, 29 and 5 tests.cargo test -p datafusion --libpassed: 468 tests.cargo test --profile ci -p datafusion-sqllogictest --test sqllogictestspassed: 534 of 534 files.
cargo clippy -p datafusion-expr --all-targets --all-features -- -D warningsand
cargo fmt --all -- --checkpassed.The
order.sltcase was also run against the unpatched code, where it failswith the schema error quoted above.
Are there any user-facing changes?
None beyond the query now planning. The result and the schema of the query are
the ones PostgreSQL returns; plans that already planned are unchanged.