allow z.backward() even for non-scalar outputs about pytorch HOT 3 CLOSED

I actually don't think it's a good idea. All optimization we do makes sense only for scalar valued functions, and implicit initialization of a scalar "grad output" to 1 is sensible in this case, because it gives us correct gradient values. We also allow starting backward from an arbitrary location in the graph, as long as you provide gradient w.r.t. each element of the tensor, because it's also perfectly valid mathematically, but I'm not sure why should we interpret "backward from a tensor" as "backward from a sum for tensor elements".

@rbgirshick when do you think it would be useful?

from pytorch.

It's really not important. In the past I've occasionally debugged gradient computations given a known incoming gradient. Choosing all ones is usually a good choice for debugging since it's easy to understand. So this change would only save one line and an extra arg in a niche use case. Feel free to close without implementing.

from pytorch.

ok, closing this, as it'll be a niche use case, and adding a torch.ones(x.size()) wont be that bad and be more explicit.

from pytorch.