RBM Weight Updates Issue about deeplearning HOT 5 OPEN

@Darwin2011 I think (ph_mean[i] * input[j] - nh_means[i] * nv_samples[j]) is right. The weights gradient should be data-model。

from deeplearning.

@liulhdarks ,thanks
The following code shows that the different weight update method. Could you check the following link?
https://github.com/echen/restricted-boltzmann-machines/blob/master/rbm.py

from deeplearning.

@Darwin2011 https://github.com/echen/restricted-boltzmann-machines/blob/master/rbm.py doesn't use the visible layer after sampling when updating weights. You can refer to the follow ML frameworks.

• [Deeplearning4j] https://github.com/agibsonccc/java-deeplearning/blob/master/deeplearning4j-core/src/main/java/org/deeplearning4j/models/featuredetectors/rbm/RBM.java

INDArray wGradient = input.transpose().mmul(probHidden.getSecond()).sub(
nvSamples.transpose().mmul(nhMeans));

• [darks-learning] https://github.com/liulhdarks/darks-learning/blob/master/src/main/java/darks/learning/neuron/rbm/RBM.java

DoubleMatrix wGradient = input.transpose().mmul(hProp1.prob)
.sub(vPropn.sample.transpose().mmul(hPropn.prob));

Hope to help you!

from deeplearning.

@liulhdarks thanks for your kind help. I will follow your guide.

Best Regards

from deeplearning.

@liulhdarks @yusugomori
I think the update for W , a , b should as follows:
self.W += lr * (numpy.dot(self.input.T, ph_mean - numpy.dot(nv_samples.T, nh_means))
self.vbias += lr * numpy.mean(self.input - nv_samples, axis=0)
self.hbias += lr * numpy.mean(ph_mean - nh_means, axis=0)

In origin python RBM.py. , self.W and self.hbias is not correct. This problem is also in C/C++ files, but the W has been fixed.
If I am not right, please inform me.
Thanks.

from deeplearning.