matthieucourbariaux / binaryconnect Goto Github PK

Hi Matthieu,

thanks for your BinaryConnect paper and implementation of it here, which is really inspiring and helpful. I got a concern about updating the trained weights. Empirically, you know, for each backpropagation, parameter changes after gradient descent are tiny, which is also illustrated in your paper. with these tiny changes on weights, after binarization of each epoch, it is possible that these binarized weights remain unchanged, for example, it is hard to change from 1 to -1 due to the tiny changes on weights. And based on that, after the forward pass, each epoch of result may be similar, which in turn result in more tiny changes on weights. Hince, after several epochs of training, the weights are hardly updated and it is still far away from optimization. How do you solve this issue?

another question is:
In your figure, after training, the distribution of weights are around -1 and 1. I don't know why but my training weights seems like a little bit random? do you know why?

thanks.

I am trying to implement BinaryConnect for Tensorflow, I have been unsuccessful thus far. Will the authors of the paper extend support for BinaryConnect to Tensorflow anytime soon?

Do you will include code also for http://arxiv.org/abs/1510.03009?

Does anyone know the version of python,theano and so on?Thank you !

Hi,

I am reading you code and try to understand any single piece. For backward, I got confused on calculating the gradient respect to binary weight. What is the definition of gradient to the discrete value? And also, does theano do this automaticly?

Best regards,

Tong

See main issue here --> lisa-lab/pylearn2#1585

I am having a hard time generating the .h5 datasets needed to run the SVHN example in Binary Connect. Has anyone tried to do this recently that can guide me to success?

'h5/splitted_train_32x32.h5'
'h5/valid_32x32.h5'
'h5/test_32x32.h5'

Hello Sir !
I am a student from a university in China and I am trying to reproduce your experimental results on Windows10
with the newest version of theano. I came with several errors and solved them. Maybe there are several changes which should be made
to run BinaryConnect with the newest version of theano ?
[1] As the newest version of theano changed "signal.downsample.max_pool_2d", line 415 in layer.py should be changed to
"z = T.signal.pool.pool_2d(input=z, ds=self.pool_shape, st=self.pool_stride, mode='max')"
...I hope I am right...
[2]I came with a problem between "float32" and "float64" and this finally worked
Chnage line 272 to 276 in trainer.py to:
self.shared_x.set_value(float32(set.X[start:(size+start)]))
self.shared_y.set_value(float32(set.y[start:(size+start)]))
Maybe adding float32 to it can me the program stronger?

Building the CNN...
/home/eli/anaconda2/lib/python2.7/site-packages/lasagne/layers/conv.py:489: UserWarning: The image_shape keyword argument to tensor.nnet.conv2d is deprecated, it has been renamed to input_shape.
border_mode=border_mode)
Traceback (most recent call last):
File "cifar10.py", line 301, in
W_grads = binary_connect.compute_grads(loss,cnn)
File "/home/eli/work/b2/BinaryConnect/binary_connect.py", line 163, in compute_grads
grads.append(theano.grad(loss, wrt=layer.Wb))
AttributeError: 'Conv2DLayer' object has no attribute 'Wb'