Skip to content

approx_distinct rejects floats, but accepts containers of them #26985

Description

@mkleen

Describe the bug

approx_distinct gives an estimate of how many different values a column holds. It doesn't work on floating-point numbers. It returns a "not implemented" error. If you put the same numbers inside a list, or inside a struct or map, approx_distinct accepts them and returns a result.

To Reproduce

SELECT approx_distinct(x) FROM (VALUES (1.5), (2.5)) t(x);
-- Today:    2
-- Expected: Error: not implemented for Float64

SELECT approx_distinct(arrow_cast(x, 'Dictionary(Int32, Float64)')) FROM (VALUES (1.5), (2.5)) t(x);
-- Today:    Error: not implemented for Dictionary(Int32, Float64)
-- Expected: Error: not implemented for Dictionary(Int32, Float64)

SELECT approx_distinct(l) FROM (VALUES ([1.5]), ([2.5])) t(l);
-- Today:    2
-- Expected: Error: not implemented for List(Float64)

SELECT approx_distinct(s) FROM (VALUES (named_struct('a', 1.5)), (named_struct('a', 2.5))) t(s);
-- Today:    2
-- Expected: Error: not implemented for Struct(a Float64)

SELECT approx_distinct(m) FROM (VALUES (map(['k'], [1.5])), (map(['k'], [2.5]))) t(m);
-- Today:    2
-- Expected: Error: not implemented for Map(Utf8, Float64)

SELECT g, approx_distinct(l) FROM (VALUES (1, [1.5]), (1, [2.5]), (2, [1.5])) t(g, l) GROUP BY g ORDER BY g;
-- Today:    2 for g = 1, 1 for g = 2
-- Expected: Error: not implemented for List(Float64)

This happens because the support check only looks at the outer type. For a list, it sees "list" and lets it through without checking what's inside. For dictionary-encoded and run-end-encoded columns it does look inside, which is why those are rejected correctly.

This came up in the review of #25951: #25951 (review)

Expected behavior

Floats should be handled the same way whether they are on their own or inside a list, struct or map. Either they work in both cases, or they fail in both cases. Therefore make the check look inside lists, structs and maps, and reject them if they contain decimal numbers.

Additional context

  • The check is in datafusion/functions-aggregate/src/approx_distinct.rs (is_supported_type and is_hll_groups_type).
  • Existing tests that expect the error are in datafusion/sqllogictest/test_files/aggregate.slt, around lines 1896–1903.

Activity

  1. added theissue type on Nov 1, 2026
  2. changed the title [-]`approx_distinct` rejects floats, but accepts lists of them[/-] [+]`approx_distinct` rejects floats, but accepts containers of them[/+] on Nov 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions