use bfloat16 on nvidia V100 GPU about pytorch HOT 2 CLOSED

I think distinction here is "supported by software"(i.e. emulation) vs "supported by hardware". torch.cuda.is_bf16_supported() tells that your GPU hardware does not have native bf16 instructions, but software can easily emulate some bf16 operations by shifting input values to the left and then running computation in float32, but it will be slower

from pytorch.

I think distinction here is "supported by software"(i.e. emulation) vs "supported by hardware". torch.cuda.is_bf16_supported() tells that your GPU hardware does not have native bf16 instructions, but software can easily emulate some bf16 operations by shifting input values to the left and then running computation in float32, but it will be slower

Thanks for you answer! I have tried to use a bfloat16 mixed precision training on a V100 GPU, which shows the time cost is almost the same as full fp32 training(even a little slower).

from pytorch.