hw1's People
Forkers
ghostlikei guan-t7 yofufufufu abasrith95 lordidiot cmu-dl-systems reblexis dlcourse-learning vguerra soumitrapandit nasen23 a1194266996 yuki-younai myidis12138 hdksg10 lucas66zhang ncy-cohw1's Issues
test_transpose_forward() out of index
Typo/Bug in detach function
I think there is a bug/typo in https://github.com/dlsyscourse/hw1/blob/master/python/needle/autograd.py#L110:
-
It is supposed to be `return Tensor.make_const(self.realize_cached_data())` not `return Tuple.make_const(self.realize_cached_data())`
I see the same line also in lecture 4.
Support sum(keepdims=False)
It better aligns with pytorch interface without adding much implementation complexity. It also makes users' life easier because they could avoid the need to calculate the dimensions and reshape() before a broadcast.
Tests should check that grad has correct dtype
Many random-float64-promotion complaints from hw2 can be traced to problematic ops implementation in hw1. Given that autograd didn't provide interface to cast Tensor's dtype, the dtype must be consistent from the beginning. Tests for gradient() methods should check that grad.dtype is "smaller" (won't cause promotion) than inputs[i].dtype
`__rsub__=__sub__` is a problem?
I find that in the codebase:
class Tensor(Value):
__rsub__ = __sub__
Here is a test case:
import needle as ndl
1 - ndl.Tensor([0.5, 2.0, 3.0]) # get needle.Tensor([-0.5 1. 2. ]), which is wrong
Propose a hack data of Summation operation
Hello!
A implementation like this may pass the original test data.
def gradient(self, out_grad, node):
return Tensor(array_api.ones_like(out_grad))
But when given 2 matrix , it fail.
So I sincerely propose to add this sample to function summation_backward()
in test_autograd_hw.py
of hw1
gradient_check(lambda A,C : ndl.summation(ndl.reshape(A,shape=(10,5))+C, axes=None),
ndl.Tensor(np.random.randn(50)),
ndl.Tensor(np.random.randn(10,5)), backward=True)
this originates from the last sample of function test_compute_gradient()
with other operations removed.
I hope this may discover problems earlier to save time.
Thanks for your attention.
Add tests for PowerScalar and Exp TensorOps
test_autograd_hw.py
doesn't have any tests for both compute
and gradient
methods of PowerScalar
class. Same for Exp
class. Adding tests will verify that our implementation is correct.
Hw1 question 2 issue
I have a problem about test code of HW1 question2. In gradient_check()
function, out.backward()
needs to call compute_gradient_of_variables()
function which is implemented in question4. How to verify code in question2 without implementation of compute_gradient_of_variables()
? I am confused.
I think there is a problem at line 205 to line 209 of test_autograd_hw.py
np.testing.assert_allclose(ndl.transpose(ndl.Tensor([[[2.45]],
[[3.5 ]],
[[0.9 ]]])).numpy(), np.array([[[2.45],
[3.5 ],
[0.9 ]]]))
Aims to transpose (3, 1, 1) => (1, 3, 1), but not give axes = (0, 1). The default axes are the last two axes(1, 2).
Maybe there is a typo in autograd.py ?
In here: https://github.com/dlsyscourse/hw1/blob/master/python/needle/autograd.py#L133
self.cached_data
It seems like this code is not necessary
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ๐๐๐
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google โค๏ธ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.