nn.Linear outputs differ on the same input tensor about pytorch HOT 4 CLOSED

@zhuhaozhe please help fix this one !

from pytorch.

This sounds like expected behavior. Please see https://pytorch.org/docs/stable/notes/numerical_accuracy.html#batched-computations-or-slice-computations for details on this behavior.
Do re-open the issue if you think this doesn't match what is described there.

from pytorch.

This sounds like expected behavior. Please see https://pytorch.org/docs/stable/notes/numerical_accuracy.html#batched-computations-or-slice-computations for details on this behavior. Do re-open the issue if you think this doesn't match what is described there.

Thank you for reply. The issue might not be due to the mentioned link, and the situation is more serious than expected.

1. The batch calculation values have issues in the second linear layer but are accurate in the first one.
2. I verified that x1_fc2 and x2_fc2 are exactly the same on the GPU, but there are differences on the CPU. This causes the model to perform differently on different devices.
3. The numerical differences are actually quite large. For example,

x1_fc2[0, 5].item(), x2_fc2[0, 5].item()
(0.027112668380141258, 0.02711271122097969)

from pytorch.

I can not find the button to re-open this issue.

from pytorch.