[Regression] Query using ARRAY_AGG(DISTINCT) causes panic about arrow-datafusion HOT 5 CLOSED

I was playing around with this issue and found a more minimal example:

WITH A AS (
    SELECT
        1 AS id, 1 AS foo
    UNION ALL
    SELECT
        1, 5
)
SELECT
    ARRAY_AGG(DISTINCT a.foo),
    SUM(DISTINCT 1) 
FROM
    A a
GROUP BY
    a.id;

It is interesting though, using SUM(1) instead of SUM(DISTINCT 1) did not panic

from arrow-datafusion.

@jayzhan211 I appreciate your concern regarding the complex example case; I attempted to create a simpler contrived example, but was unable to trigger the panic doing so. I don't currently know the exact conditions that are triggering the problem.

from arrow-datafusion.

I added the assertion because I don't know if there is any case that has len > 1.

It would be nice if you had a simpler example to add to the test!

from arrow-datafusion.

@jayzhan211 I appreciate your concern regarding the complex example case; I attempted to create a simpler contrived example, but was unable to trigger the panic doing so. I don't currently know the exact conditions that are triggering the problem.

That's fine. I think that is the reason why I can't easily find the test case that trigger this panic, probably only the complex aggregate query that has multiple states meet the requirement.

I think we can add your example as a test case. And, find out whether we should fix the multi states issue in merge_batch or elsewhere before entering merge_batch.

from arrow-datafusion.

Added to #10517

from arrow-datafusion.