groupby.median() will return floats dataframe in most

Pandas groupby.median casts to int when possible about modin HOT 4 CLOSED

modin-project commented on May 10, 2024

Pandas groupby.median casts to int when possible

from modin.

Comments (4)

simon-mo commented on May 10, 2024

We will skip the test for this at this moment.

from modin.

kunalgosar commented on May 10, 2024

I am reopening this issue as it needs a permanent fix

from modin.

dchigarev commented on May 10, 2024

It seems that this problem is already fixed, at least this reproduced works as expected in the current master:

reproducer

if __name__ == "__main__":
    import pandas
    import modin.pandas as pd

    future_int = {
        "A": ["a", "b", "c", "d"],
        "B": [0, 1, 2, 3],
        "C": [4, 5, 6, 7],
        "D": [8, 9, 10, 11],
    }
    md_df, pd_df = pd.DataFrame(future_int), pandas.DataFrame(future_int)

    print("pd dtypes:\n", pd_df.groupby("A").median().dtypes, sep="")
    print("md dtypes:\n", md_df.groupby("A").median().dtypes, sep="")

    future_float = {
        "A": ["a", "b", "b", "d"],
        "B": [0, 1, 2, 3],
        "C": [4, 5, 6, 7],
        "D": [8, 9, 10, 11],
    }
    md_df, pd_df = pd.DataFrame(future_float), pandas.DataFrame(future_float)
    print("\n\npd dtypes:\n", pd_df.groupby("A").median().dtypes, sep="")
    print("md dtypes:\n", md_df.groupby("A").median().dtypes, sep="")

Output:

pd dtypes:
B    int64
C    int64
D    int64
dtype: object
md dtypes:
B    int64
C    int64
D    int64
dtype: object


pd dtypes:
B    float64
C    float64
D    float64
dtype: object
md dtypes:
B    float64
C    float64
D    float64
dtype: object

Since for groupby aggregation we use pandas version under the hood and it encapsulates all logic about downcasting dtypes in it, we'll always gonna get same dtypes as pandas gives.

So @devin-petersohn what do you think about closing that issue?

from modin.