questions about softmax + cross_entropy v.s. softmax_cross_entropy_with_logits about tensorflow-examples HOT 3 CLOSED

I think reason is that 'softmax_cross_entropy_with_logits' doesn't exactly apply cross entropy as defined in cost =tf.reduce_mean(-tf.reduce_sum(y*tf.log(pred),reduction_indices=1)). But this function is not so well documented, so you may need to find in the C++ sources how they compute it.

from tensorflow-examples.

I find the error yesterday, maybe it's caused by cross_entropy = tf.nn.softmax_cross_entropy_with_logits(tf.nn.softmax(tf.add(tf.matmul(x,W),b)),y). I should not add another softmax in the softmax. I don't know the detail reason, but now the result is stable.

from tensorflow-examples.

Yes, this is the mistake, I didn't notice it.

from tensorflow-examples.