I am sorry for the confusion.

Basically, Your understanding is totally correct. For this logistic regression 'objective=binary:logistic'. What we get is in preds not x_i, but a transformed version preds_i = 1 /(1+exp(-x_i)) . So taking first and second order derivatives to x_i gives you preds_i - y_i, and preds_i (1- preds_i).

A more understandable case, and I commented, is to leave objective as default, which will not have any transformation on prediction, you get x_i in preds, and the code should be modified to

prob = 1.0/(1+np.exp(-preds))
grads = prob - labels
hess = prob * (1-prob)

But you need to remember to transform prediction into prob if you want probability.

Hope this answers your question

from xgboost.

Thank you for the explanation!
So to summarize, if I would want to minimize my own loss function, I should set objective="binary:logitraw" in order not to transform the preds variable entering my custom loss function, and supply a function which computes dL/dx_i and d^2L/dx_i^2 as the obj parameter to xgboost.train?

from xgboost.

Yes, exactly

On Tuesday, July 15, 2014, rmldj [email protected] wrote:

Thank you for the explanation!
So to summarize, if I would want to minimize my own loss function, I
should set objective="binary:logitraw" in order not to transform the
preds variable entering my custom loss function, and supply a function
which computes dL/dx_i and d^2L/dx_i^2 as the obj parameter to
xgboost.train?

—
Reply to this email directly or view it on GitHub
https://github.com/tqchen/xgboost/issues/15#issuecomment-49014367.

Sincerely,

Tianqi Chen
Computer Science & Engineering, University of Washington

from xgboost.

so then the objective function is to left as "binary:logitraw", and then where do we pass in the custom loss function to the xgb model? I thought the custom loss function was to be set to the obj parameter?

from xgboost.

I have a question about this code too. when I run the code on my computer, the python shows the error: Kernel died, restarting, do you know how can i fix it?

user define objective function, given prediction, return gradient and second order gradient

def logregobj( preds, dtrain ):
labels = dtrain.get_label()
grad = preds - labels
hess = preds * (1.0-preds)
return grad, hess

from xgboost.