Compatibility with scikit-learn 0.24.0 about ktboost HOT 4 CLOSED

Thanks for letting me know!

This should be fixed now with the new version 0.1.20 of KTBoost.

from ktboost.

Wow that's quick! I have explored this library a bit and found this approach provides better performance than other famous counterparts like XGBoost.

Having said that, I suspect I could have more performance gain by playing with the loss function. My target variable is following a zero-inflated log-normal / zero-inflated gamma, so I think using a tweedie deviance as loss function should be more appropriate. I tried Tobit loss but my zeros are in very high fraction (>.8) so the truncated normality assumption does not work well.

I am looking at how to define a custom loss, but it seems that apart from defining the loss, I would also need to work with the _update_terminal_region, which is not very familiar to me.

Would you have time to look into this? Or is there any doc that would be useful for this?

Thanks!

from ktboost.

Thanks for your feedback. I agree with your assessment that other discrete-continuous losses, apart from the Tobit loss, can lead to better performances.

Yes, you are right about what needs to be done in terms of coding for implementing other losses. Unfortunately, I don't have any docs for this. But the structure is the same as in scikit-learn.

For what type of application do you plan to use this? We might collaborate on this. If this is an option, send me an e-mail.

from ktboost.

The application is to model claims, for a Kaggle-like competition.

After spending some time reading the code, I find the function update_terminal_region is most of the time making a single Newton-Raphson step only. I think the implementation of tweedie loss should be similar to poisson loss, I will give it a try and make a pull request later.

Thanks!

from ktboost.