Schema incorrect after select over aggregate function that returns a different type than the input about arrow-datafusion HOT 5 CLOSED

Commenting here for myself to continue looking further, but I suspect this is happening in exprlist_to_fields in datafusion/expr/src/utils.rs where we look towards the input fields of aggregates. Because this type of aggregate does not produce the same type as the input field is likely the culprit.

from arrow-datafusion.

Quickly took a look at another aggregate function that can return a different type. By calling count_distinct instead of array_agg and updating the input file to have strings instead of int64 values I verified it is also outputting the incorrect schema after doing the select operation.

from arrow-datafusion.

It should be caused by exprlist_to_fields_aggregate.

There are two columns named 'c', one from the aggregated input and the other from the output. exprlist_to_fields_aggregate forcibly uses the column 'c' from the input, which is of type Int64.

from arrow-datafusion.

exprlist_to_fields_aggregate was introduced by #2486, I doubt whether we still need it now because removing it won't break any tests.

from arrow-datafusion.

take

from arrow-datafusion.