Skip to content

fix: plan ORDER BY on a source column hidden by a select alias - #26477

Closed
paoValle wants to merge 1 commit into
apache:mainfrom
paoValle:fix/order-by-source-column-alias
Closed

paoValle wants to merge 1 commit into
apache:mainfrom
paoValle:fix/order-by-source-column-alias

Conversation

@paoValle

Copy link
Copy Markdown

Which issue does this PR close?

Rationale for this change

SELECT lower(e.email) AS email FROM employees e ORDER BY e.email failed to
plan:

Schema error: Schema contains qualified field name e.email and unqualified
field name email which would be ambiguous

The sort key is not in the select list, so the planner carries e.email into
the projection to sort on it. That projection then holds the qualified field
e.email and the unqualified field email in one schema, which
DFSchema::check_names rejects, even though the carried column is internal and
is projected away above the sort. PostgreSQL plans the same query and sorts on
the input column.

What changes are included in this PR?

LogicalPlanBuilder::sort (datafusion/expr/src/logical_plan/builder.rs) now
carries a missing sort column under an alias when the projection it is added to
already outputs an unqualified field with the same name. The alias is the
column's flat name (e.email), so the carried column cannot collide with the
projection's other fields, and the sorts are rebound to it. The projection
above the sort still strips the extra column, so the output schema of the query
is unchanged.

What is the testing strategy for this PR?

datafusion/sqllogictest/test_files/order.slt gets the query from the issue and
its ORDER BY email counterpart, which pins the two resolutions apart:
ORDER BY e.email sorts on the input column (b@x, a@y), and
ORDER BY email sorts on the output column (a@y, b@x), as in PostgreSQL.
A unit test in datafusion/expr/src/logical_plan/builder.rs covers the same
path through LogicalPlanBuilder and checks the resulting plan.

Validation, on 61c3472:

  • cargo test -p datafusion-expr passed: 285 unit tests, 63 documentation tests.
  • cargo test -p datafusion-sql passed: 95, 604 and 12 tests.
  • cargo test -p datafusion-optimizer passed: 941, 29 and 5 tests.
  • cargo test -p datafusion --lib passed: 468 tests.
  • cargo test --profile ci -p datafusion-sqllogictest --test sqllogictests
    passed: 534 of 534 files.
  • cargo clippy -p datafusion-expr --all-targets --all-features -- -D warnings
    and cargo fmt --all -- --check passed.

The order.slt case was also run against the unpatched code, where it fails
with the schema error quoted above.

Are there any user-facing changes?

None beyond the query now planning. The result and the schema of the query are
the ones PostgreSQL returns; plans that already planned are unchanged.

@github-actions github-actions Bot added logical-expr Logical plan and expressions sqllogictest SQL Logic Tests (.slt) labels Oct 18, 2026
@alamb alamb added the Prolific Author author opened PRs or issues in more than 20 repositories in the last 30 days label Oct 22, 2026
- Closes apache#26462.

`SELECT lower(e.email) AS email FROM employees e ORDER BY e.email` failed to
plan:

    Schema error: Schema contains qualified field name e.email and unqualified
    field name email which would be ambiguous

The sort key is not in the select list, so the planner carries `e.email` into
the projection to sort on it. That projection then holds the qualified field
`e.email` and the unqualified field `email` in one schema, which
`DFSchema::check_names` rejects, even though the carried column is internal and
is projected away above the sort. PostgreSQL plans the same query and sorts on
the input column.

`LogicalPlanBuilder::sort` (`datafusion/expr/src/logical_plan/builder.rs`) now
carries a missing sort column under an alias when the projection it is added to
already outputs an unqualified field with the same name. The alias is the
column's flat name (`e.email`), so the carried column cannot collide with the
projection's other fields, and the sorts are rebound to it. The projection
above the sort still strips the extra column, so the output schema of the query
is unchanged.

`datafusion/sqllogictest/test_files/order.slt` gets the query from the issue and
its `ORDER BY email` counterpart, which pins the two resolutions apart:
`ORDER BY e.email` sorts on the input column (`b@x`, `a@y`), and
`ORDER BY email` sorts on the output column (`a@y`, `b@x`), as in PostgreSQL.
A unit test in `datafusion/expr/src/logical_plan/builder.rs` covers the same
path through `LogicalPlanBuilder` and checks the resulting plan.

Validation, on 61c3472:

- `cargo test -p datafusion-expr` passed: 285 unit tests, 63 documentation tests.
- `cargo test -p datafusion-sql` passed: 95, 604 and 12 tests.
- `cargo test -p datafusion-optimizer` passed: 941, 29 and 5 tests.
- `cargo test -p datafusion --lib` passed: 468 tests.
- `cargo test --profile ci -p datafusion-sqllogictest --test sqllogictests`
  passed: 534 of 534 files.
- `cargo clippy -p datafusion-expr --all-targets --all-features -- -D warnings`
  and `cargo fmt --all -- --check` passed.

The `order.slt` case was also run against the unpatched code, where it fails
with the schema error quoted above.

None beyond the query now planning. The result and the schema of the query are
the ones PostgreSQL returns; plans that already planned are unchanged.
@paoValle
paoValle force-pushed the fix/order-by-source-column-alias branch from 6ae3a07 to 6edc624 Compare October 24, 2026 17:15
@paoValle

Copy link
Copy Markdown
Author

Rebased onto main cd571e51 (6ae3a073 → 6edc6246); the branch had gone conflicting.

One conflict, in datafusion/sqllogictest/test_files/order.slt, where both sides appended cases at the end of the file. Kept both: the SortPreservingMergeExec / probe_constant cases that landed on main and this PR's two employees cases. Nothing else conflicted; datafusion/expr/src/logical_plan/builder.rs auto-merged.

Re-run after the rebase:

  • cargo test -p datafusion-expr — 294 unit tests and 64 documentation tests passed;
  • cargo test -p datafusion-sqllogictest --test sqllogictests -- order — 5 of 5 files completed, exit 0, including the two cases this PR adds;
  • cargo fmt --all --check clean.

@alamb

alamb commented Oct 29, 2026

Copy link
Copy Markdown
Contributor

Thanks for the contribution! I am closing this under DataFusion's AI Dump policy.

https://datafusion.apache.org/contributor-guide/ai-policy.html#what-is-an-ai-dump-and-why-it-is-not-helpful

This has the hallmarks of AI code that was not reviewed by the author: a commit message that is the PR description verbatim, per-crate test counts as validation, and a defensive loop for a case that does not arise.

If you would like to continue, please open a focused PR in your own words that you have reviewed and understand, and we are happy to help with review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

logical-expr Logical plan and expressions Prolific Author author opened PRs or issues in more than 20 repositories in the last 30 days sqllogictest SQL Logic Tests (.slt)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ORDER BY on a source column fails when a select item is aliased to that column's name

2 participants