joelgrus / autograd Goto Github PK

How can we implement the ReLu function?

Hi Joel,

I think there's an error in your implementation of _slice. Currently, lines 273-287 of tensor.py read:

def _slice(t: Tensor, idxs) -> Tensor:
    data = t.data[idxs]
    requires_grad = t.requires_grad

    if requires_grad:
        def grad_fn(grad: np.ndarray) -> np.ndarray:
            bigger_grad = np.zeros_like(data)
            bigger_grad[idxs] = grad
            return bigger_grad

        depends_on = Dependency(t, grad_fn)
    else:
        depends_on = []

    return Tensor(data, requires_grad, depends_on)

However, line 279 bigger_grad = np.zeros_like(data) is incorrect, since data has already been indexed by idxs and is not the same size as our original tensor. We can easily fix this by changing the line to bigger_grad = np.zeros_like(t.data).

Also, and this is minor, you should wrap the dependency in a list.

All in all, nothing that unit tests wouldn't have caught!

Thanks for your work,
Clément