Comments (8)
Your input,output,weights matrices aren't KnetArrays; calculations are being done on the CPU.
from knet.jl.
That means that whole package is optimized for GPU and slow on CPU?
from knet.jl.
Yes. GPUs are better at these type of calculations. Although I haven't tested your code, it should run much faster on a GPU.
from knet.jl.
from knet.jl.
Hi Dennis,
thanks for the answer. I think that the core issue is that weights of the model is vector of Any, which causes Julia to make ineffective code. I have copied that from example in ReadMe, but it is not something I am quite content. I have tried to change it to tuple, which should give it a rigid definition and julia can better optimize. Though I run into this error:
no method matching axpy!(::Float64, ::Tuple{Array{Float64,2},Array{Float64,1},Array{Float64,2},Array{Float64,1},Array{Float64,2},Float64}, ::Tuple{Array{Float64,2},Array{Float64,1},Array{Float64,2},Array{Float64,1},Array{Float64,2},Float64})
which I think is caused by missing overloaded function. But I was not able to find, where do you overload axpy! for lists of matrices or how to add handling of tuples such that it will work with update!
from knet.jl.
I have modified the code by replacing the array by tuple (which made the compiler aware of types) and removed the update!, because of the problem mentioned in my previous comment and it flies really nicely.
That said, the update! should be modified to handle tuples, especially if it is encouraged in the documentation to do so.
Tomas
from knet.jl.
Hi,
You can use a tuple to hold your weights, but update! accepts only one weight array. You should iterate over your weights to update them. Your initial examples works since julia can perform the minus operation on list of arrays (e.g. [rand(4,2), zeros(2)] - 0.1 * [rand(4,2), zeros(2)]). Optimization parameters should also be a list or tuple (if you do not use Sgd). Because, the optimization methods other than Sgd need to store some additional arrays to perform the update rule.
You can check this example .
from knet.jl.
Hi,
thanks for the answer. I was thinking about solution like that, but before writing that, I wanted to know if there is some machinery inside that makes this explicit.
Thank you very much for the answer.
I really appreciate it.
Tomas
from knet.jl.
Related Issues (20)
- dropout seeds the global RNG
- MethodError: no method matching LinearIndices HOT 2
- ERROR: LoadError: KeyError: key :nw not found
- missing docstring error for "File I/O" section
- Making a new CUDA 3 compatible release? HOT 7
- Create a DeepMap.jl package
- Knet 1.4.7: libknet8 library not found HOT 4
- LoadError: Failed to precompile Knet [1902f260-5fb4-5aff-8c31-6271790ab950] to my julia directory HOT 3
- TagBot trigger issue HOT 2
- load/save problem with CuArrays in tutorial/60.rnn.ipynb HOT 2
- Derivative of a Function That Includes @diff Macro HOT 4
- R1 Regularization HOT 9
- type DataType has no field mutable HOT 2
- Quick Start tutorial notebook is broken HOT 1
- Cannot locate artifact 'libknet8' for aarch64-linux-gnu-libgfortran5-cxx11-libstdcxx29-julia_version+1.7.1 in Docker Container on Apple Silicon HOT 9
- Fix deprecations from MLDatasets in tutorial notebooks
- Problem with conv4 on gpu HOT 4
- UndefVarError: accuracy not defined in tutorial 30.lin.ipynb HOT 1
- CuArray{Float32} in ytype raised error HOT 4
- how to use the resnet modules in example folder
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from knet.jl.