dlsyscourse / hw1 Goto Github PK

def test_transpose_forward():
    np.testing.assert_allclose(ndl.transpose(ndl.Tensor([[[1.95]],
       [[2.7 ]],
       [[3.75]]]), axes=(1, 2)).numpy(), np.array([[[1.95]],
       [[2.7 ]],
       [[3.75]]]))

It should be 0 1 2 , 3d tensor, Why it is out of index?

I think there is a bug/typo in https://github.com/dlsyscourse/hw1/blob/master/python/needle/autograd.py#L110:

• It is supposed to be `return Tensor.make_const(self.realize_cached_data())` not `return Tuple.make_const(self.realize_cached_data())`
  

I see the same line also in lecture 4.

It better aligns with pytorch interface without adding much implementation complexity. It also makes users' life easier because they could avoid the need to calculate the dimensions and reshape() before a broadcast.

Many random-float64-promotion complaints from hw2 can be traced to problematic ops implementation in hw1. Given that autograd didn't provide interface to cast Tensor's dtype, the dtype must be consistent from the beginning. Tests for gradient() methods should check that grad.dtype is "smaller" (won't cause promotion) than inputs[i].dtype

I find that in the codebase:

class Tensor(Value):
    __rsub__ = __sub__

Here is a test case:

import needle as ndl
1 - ndl.Tensor([0.5, 2.0, 3.0])   # get needle.Tensor([-0.5  1.   2. ]), which is wrong

Hello!

A implementation like this may pass the original test data.

def gradient(self, out_grad, node):
        return Tensor(array_api.ones_like(out_grad))

But when given 2 matrix , it fail.

So I sincerely propose to add this sample to function summation_backward() in test_autograd_hw.py of hw1

gradient_check(lambda A,C : ndl.summation(ndl.reshape(A,shape=(10,5))+C, axes=None),
                   ndl.Tensor(np.random.randn(50)),
                   ndl.Tensor(np.random.randn(10,5)), backward=True)

this originates from the last sample of function test_compute_gradient() with other operations removed.

I hope this may discover problems earlier to save time.
Thanks for your attention.

test_autograd_hw.py doesn't have any tests for both compute and gradient methods of PowerScalar class. Same for Exp class. Adding tests will verify that our implementation is correct.

I have a problem about test code of HW1 question2. In gradient_check() function, out.backward() needs to call compute_gradient_of_variables() function which is implemented in question4. How to verify code in question2 without implementation of compute_gradient_of_variables() ? I am confused.

np.testing.assert_allclose(ndl.transpose(ndl.Tensor([[[2.45]],
   [[3.5 ]],
   [[0.9 ]]])).numpy(), np.array([[[2.45],
    [3.5 ],
    [0.9 ]]]))

Aims to transpose (3, 1, 1) => (1, 3, 1), but not give axes = (0, 1). The default axes are the last two axes(1, 2).

In here: https://github.com/dlsyscourse/hw1/blob/master/python/needle/autograd.py#L133

self.cached_data

It seems like this code is not necessary