ValueError: The first argument of bincount must be non-negative about ydata-profiling HOT 1 CLOSED

Awesome you found a bug!

After investigation it is a bug (feature) in Pandas that calls NumPy with illegal arguments, caused by the infinities in your data.

As a workaround you can replace the infinities with missing values like so:

df.replace(to_replace=np.inf, value=np.NaN, inplace=True)

Depending on what you are trying to do, NaN might actually be better than infinities. Are the infinities real, mathematical infinities? If they are right-censored, for example because your sensor can only read to a certain upper limit, than replacing it with the right-censored value might be more appropriate.

In any case, a lot of calculated statistics will be invalid. For example, when there is a positive infinity, the calculated mean should be positive infinity as well, but the value will now be ignored, skewing the results.

I will leave this bug open for now to think about how we should deal with infinities:

• Replace them with NaN's by default
• Group them in separate bins and plot them separately
• Reject the variable since algorithms won't fit anyway

Would love input on this as well.

To reproduce:

import pandas as pd
import numpy as np
series=pd.Series([2.307568, 59.642170, np.inf, 1241.660000, np.inf])
plot = series.plot(kind='hist', bins=10)

from ydata-profiling.