andersbll / deeppy Goto Github PK

Using deeppy 0.1 version.

I run run() from 'examples/mlp_mnist.py' and get this:

TypeError                                 Traceback (most recent call last)
<ipython-input-5-157c9bda2cd6> in <module>()
----> 1 run()

<ipython-input-4-87bc676759c6> in run()
     24             dp.FullyConnected(
     25                 n_output=800,
---> 26                 weights=dp.Parameter(dp.AutoFiller(), weight_decay=0.0001),
     27             ),
     28             dp.Activation('relu'),

TypeError: __init__() got an unexpected keyword argument 'weight_decay'

removing weight_decay option from dp.Parameter solved an issue.

In deeppy/feedforward/convnet_layers.py, the bprop for the LRN layer should not be the direct copy from the previous layer.

Reference: decaf
https://github.com/UCB-ICSI-Vision-Group/decaf-release/blob/master/decaf/layers/cpp/local_response_normalization.cpp

It would be nice to have a tool to import pre-trained models from Caffe. A few questions that could help with manual conversion:

1. Does deeppy implement convolution in the same way as Caffe (i. e. correlation)?
2. Does it use the same layout for filter weights and incoming tensors?

Is there an expected (ballpark, rough estimate) runtime for convnet_mnist.py? I ran mlp_mnist.py and the script finished extremely quickly. But for convnet_mnist.py, I've been sitting at the same output for over 30 minutes, which seems extremely high given that the Caffe MNIST examples finishes in a couple minutes:

INFO SGD: Model contains 127242 parameters.
INFO SGD: 469 mini-batch gradient updates per epoch.
(no extra output after this)

Hi Anders,

Sorry for bothering, I am relatively new to Python. I have a problem: after I do changes in the code, I want to recursively reload all deeppy module, so I call "dreload(deeppy)". However, even if I don't make any changes, reload causes the next run to fail with the following error. Could you please advise anything about it?

%run "/home/sdemyanov/Server/deeppy/examples/mlp_mnist.py"

AttributeError Traceback (most recent call last)
/home/sdemyanov/Server/deeppy/examples/mlp_mnist.py in ()
62 learn_rule=dp.Momentum(learn_rate=learn_rate/10**i, momentum=0.94),
63 )
---> 64 trainer.train(net, train_input)
65
66 # Evaluate on test data

/home/sdemyanov/Server/deeppy/deeppy/train/sgd.pyc in train(self, model, input, val_error_fun)
19 def train(self, model, input, val_error_fun=None):
20 input = Input.from_any(input)
---> 21 model._setup(**input.shapes)
22 params = model._params
23 self.learn_rule.learn_rate /= input.batch_size

/home/sdemyanov/Server/deeppy/deeppy/feedforward/neural_network.pyc in _setup(self, x_shape, y_shape)
20 return
21 for layer in self.layers:
---> 22 layer._setup(x_shape)
23 x_shape = layer.y_shape(x_shape)
24 self.loss._setup(x_shape, y_shape)

/home/sdemyanov/Server/deeppy/deeppy/feedforward/layers.pyc in _setup(self, x_shape)
39 if not self.weights.name:
40 self.weights.name = self.name + '_w'
---> 41 self.bias._setup(self.n_out)
42 if not self.bias.name:
43 self.bias.name = self.name + '_b'

/home/sdemyanov/Server/deeppy/deeppy/parameter.pyc in _setup(self, shape)
31 def _setup(self, shape):
32 if self._array is None:
---> 33 self._array = self.filler.array(shape)
34 else:
35 if isinstance(shape, int):

/home/sdemyanov/Server/deeppy/deeppy/filler.pyc in array(self, shape)
32
33 def array(self, shape):
---> 34 return ca.ones(shape)*self.value
35
36

/home/sdemyanov/anaconda/lib/python2.7/site-packages/cudarray-0.1.dev0-py2.7-linux-x86_64.egg/cudarray/cudarray.pyc in mul(self, other)
101
102 def mul(self, other):
--> 103 return elementwise.multiply(self, other)
104
105 def rmul(self, other):

AttributeError: 'module' object has no attribute 'multiply'

the LFW landmark points link denseLFW_v2.tar.gz link is down.
is there any other source for it

This is a fantastic library! I would love to use it, and I'm sure others would as well. Could this be added to the Python Package Index? The site makes it look like its simple to add to the list. I wouldn't want to take credit for work I didn't do by adding it to the list myself. Could you submit this to be included?

Hello, are you sure about the sign in these lines ?

if penalty is not None:
            step -= penalty

You multiply by -learn_rate next to make it -grad then at the snippet above step should be grad and penalty should be added with + sign. Is there is something I'm missing ? I've tried to set weight_decay to something below zero and this actually makes the weights shrink. If I set weight_decay to a large value > 0 then the weights explode.

celebA Dropbox links (downloaded in /deeppy/dataset/celeba.py), have been down for quite some time.

New links are available @ Google Drive:
https://drive.google.com/drive/folders/0B7EVK8r0v71pWEZsZE9oNnFzTm8

(source: http://mmlab.ie.cuhk.edu.hk/projects/CelebA.html)

installed cudarray, added /usr/local/lib/ to LD_LIBRARY_PATH, and cuda (/usr/local/cuda-6.5/lib64/) to PATH, but when i run this in python:

import os
os.environ['CUDARRAY_BACKEND'] = 'cuda'
import deeppy

i get this traceback.

CUDArray: Failed to load CUDA back-end.
Traceback (most recent call last):
File "", line 1, in
File "build/bdist.linux-x86_64/egg/deeppy/init.py", line 9, in
File "build/bdist.linux-x86_64/egg/deeppy/dataset/init.py", line 2, in
File "build/bdist.linux-x86_64/egg/deeppy/dataset/infimnist.py", line 6, in
File "build/bdist.linux-x86_64/egg/deeppy/base.py", line 1, in
File "/usr/local/lib/python2.7/dist-packages/cudarray-0.1.dev-py2.7-linux-x86_64.egg/cudarray/_
init_.py", line 29, in
wrap.cudart.initialize(_gpu_id)
File "cudarray/wrap/cudart.pyx", line 4, in cudarray.wrap.cudart.initialize (./cudarray/wrap/cu
dart.cpp:671)
File "cudarray/wrap/cudart.pyx", line 5, in cudarray.wrap.cudart.initialize (./cudarray/wrap/cu
dart.cpp:600)
File "cudarray/wrap/cudart.pyx", line 12, in cudarray.wrap.cudart.cudaCheck (./cudarray/wrap/cu
dart.cpp:736)
ValueError: no CUDA-capable device is detected

this worked a few months ago -- could there be a problem between recent commits of deeppy/cudarray and cuda-6.5? not exactly sure where this problem is coming from.

thanks!

If I have more than one GPU, is there a way to select the one I want to use?

Missing an 'else' condition in fprop to handle passing 0.0 into dp.Dropout() to 'deactivate' a dropout layer.

Obviously, one can just comment out a dropout layer in the network definition, but probably better to allow '0' to be passed in to turn off Dropout when experimenting with network structures programatically.

def conv_layer(n_filters):
    return dp.Convolution(
        n_filters=32,
        filter_shape=(5, 5),
        border_mode='full',
        weights=dp.Parameter(dp.AutoFiller(gain=1.25), weight_decay=0.003),
    )

shouldn't 'n_filters=n_filters' ?? Otherwise it seems like conv_layer is always fixed to 32 filters.

UPDATE: looks like a few of the other examples also are hard-coded to 32 filters as well.

Hi, 'import deeppy' displayed this error msg:

File "", line 1, in
import deeppy

File "C:\Anaconda3\lib\site-packages\deeppy-0.1.dev0-py3.5.egg\deeppy__init__.py", line 2, in
from . import expr

File "C:\Anaconda3\lib\site-packages\deeppy-0.1.dev0-py3.5.egg\deeppy\expr__init__.py", line 18, in
import nnet

ImportError: No module named 'nnet'

Where is this nnet module ?

Hi, just installed this a few minutes ago and ended up using the example code on the preliminary site (rather than the included code). Looks like the old activation methods are still being used which causes an error with the current version of deeppy.

For example:

dp.Activation('relu')

http://andersbll.github.io/deeppy-website/examples/index.html

May want to update code on site in case anybody else runs across this.

(The example code on GitHub does work)

After installing CUDA I wanted to install deeppy but have following error:

Installed /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/deeppy-0.1.dev0-py2.7.egg
Processing dependencies for deeppy==0.1.dev0
Searching for cudarray>=0.1.dev
Reading https://pypi.python.org/simple/cudarray/
Couldn't find index page for 'cudarray' (maybe misspelled?)
Scanning index of all packages (this may take a while)
Reading https://pypi.python.org/simple/
No local packages or download links found for cudarray>=0.1.dev
error: Could not find suitable distribution for Requirement.parse('cudarray>=0.1.dev')

libcudarray.ca is already placed in /usr/local/lib

Thanks.

Hi,
I have searched a little bit but I did not encounter memory allocation option like Theano's cnmem. You can find details here: cnmem

Is there any option like that we can use in deeppy?

This is just a feature request for a future updates to make single image prediction work easy for the user, not a bug.

Let input to NeuralNetwork.predict in

deeppy/deeppy/feed_forward/neural_network.py

be a single image (I'm testing with MNIST). Currently it requires a 4-dimensional array, I have to reshape a single image from the data set from a [1,28,28] to a [1,1,28,28] to execute.

Hey.
Is there an easy way to use this without installing CUDA?

Thanks :)

Hi Anders,

Could you also add an option to view the probabilities in 'predict'?
I tried to do it by myself:

1. In class SoftmaxCrossEntropy I changed the function 'fprop' on
   def fprop(self, x):
   sm = self._softmax(x)
   return ca.nnet.one_hot_decode(sm), sm

2. in class NeuralNetwork I change the function 'predict' to return both predictions and probabilities

3. in 'test_error()' I read two values:
   y, yp = net.predict(test_input)

However, I always get an error 'too many values to unpack' for the 'test_error" function, which means that I do not reload it properly. Could you also explain how to do it correctly? Surprisingly I find two sources: in deeppy/deeppy and deeppy/build/lib/deeppy. Moreover, the upstack error appears in '/anaconda/lib/python2.7/site-packages/deeppy-0.1.dev0-py2.7.egg/deeppy/train/sgd.py', in the function 'val_error = val_error_fun()'. Could you clarify these details please?

Regards,
Sergey.

I don't have an issue, I just thought that was fun!
And thanks for the neural style implementation that does not blow up the memory of a K20!!!
I don't know what is wrong with the other implementations out there (or with me), but yeah, tkx!
High 5 and close this as soon as you read it!

Hey there.

This issue started recently when importing deeppy in a python shell (just checking to see that all works properly at the time)

"ImportError: libcudart.so.7.0: wrong ELF class: ELFCLASS32"

Not sure what to do.

export PATH=$PATH:/usr/local/cuda-7.0/bin
export LD_LIBRARY_PATH=:/usr/local/cuda-7.0/lib64
export INSTALL_PREFIX="/usr/local"
export CUDA_PREFIX="/usr/local/cuda"
export CUDNN_ENABLED=1
export CUDARRAY_BACKEND=numpy
export LD_LIBRARY_PATH=/usr/local/cuda/lib

are my env variables. Ubuntu 14.04, CUDA 7.

EDIT: I just discovered it's because several things broke in the 7.5 CUDA version for Ubuntu 14.04. As a result I did 7.0 which doesn't have a x64 binary. This means I have to try the NVIDIA abstract documentation for graphics installation on linux to figure out what goes wrong during the install of 7.5 on Ubuntu 14.04.

posting all of this in case someone runs into this.

sorry can you give your email address or where is it convenient for you to chat? I wanted to ask something about the framework and skills.

Hi,
I'm trying to run the convnet_mnist.py example and I getting an error:

INFO     SGD: Model contains 127242 parameters.
INFO     SGD: 469 mini-batch gradient updates per epoch.
terminate called after throwing an instance of 'std::runtime_error'
  what():  src/image/img2win.cu:56: invalid device function 
[1]    10517 abort (core dumped)  python convnet_mnist.py

I'm not sure this is a problem with deeppy itselft. It's probably a problem with my cuda installation since I had similar problems trying to run caffe. But as I have no experience with cuda at all, do you guys have any pointers of what might be the issue?

I'll totally understand if you just say "it's a cuda problem, solve it with the cuda guys" , but it also might be something you encountered so...

I could not for the life of me figure out how to install cudarray with cuda (was getting a no file/directory error on make for nvcc), so have decided to move on and try it without CUDA.

I have a project almost identical to your cifar example with a different set of test images. I've been running your example to study the output before changing to my data, and it's been running for around an hour. Do you know how long it takes roughly without CUDA?