yusugomori / deeplearning Goto Github PK
View Code? Open in Web Editor NEWDeep Learning (Python, C, C++, Java, Scala, Go)
Home Page: https://yusugomori.com
License: MIT License
Deep Learning (Python, C, C++, Java, Scala, Go)
Home Page: https://yusugomori.com
License: MIT License
I am using scala version of the code, but i dont know how the fine tune of stacked autoencoder is working out. What i can only see is that the logistic regression is made to learn the last layer patter,
please clarify
In the finetune phase, the whole dataset is used which will cause over fitting for the model...
So, I think this can be solved by:
do you have a license for this?
RBM.cpp
for(int i=0; i<n_hidden; i++)
{
for(int j=0; j<n_visible; j++)
{
W[i][j] += lr * (ph_mean[i] * input[j] - nh_means[i] * nv_samples[j]) / N;
}
// hbias[i] += lr * (ph_sample[i] - nh_means[i]) / N; //maybe error
hbias[i] += lr * (ph_mean[i] - nh_means[i]) / N; // correct
}
How to evaluate the DBN (Scala) model? -i.e. I don't see any way to compute performance metrics such as accuracy, precision, recall, f1 measure etc.
I want to create an Android application using your library, how could I do this? Would you give me some help? Thank you.
Such Issues are in your RBM different implementation.
[python version]
self.W += lr * (numpy.dot(self.input.T, ph_sample)
- numpy.dot(nv_samples.T, nh_means))
[C++ version]
W[i][j] += lr * (ph_mean[i] * input[j] - nh_means[i] * nv_samples[j]) / N;
For there two versions, the weight update methods are incosistent. And Actually I think the right version should be
self.W += lr * (numpy.dot(self.input.T, ph_means)
- numpy.dot(nv_means.T, nh_means))
Could you please help me confirm such issues? I am not quite sure that whether it's issue or not. And I am just a freshman for deep learning.
Best Regards
just A suggestion
what about the license?
DBN::DBN(int size, int n_i, int *hls, int n_o, int n_l) {
// construct rbm_layer
// DBN-RBM deconstructer bug fixed
rbm_layers[i] = new RBM(N, input_size, hidden_layer_sizes[i],
NULL, NULL, NULL);
//rbm_layers[i] = new RBM(N, input_size, hidden_layer_sizes[i],
// sigmoid_layers[i]->W, sigmoid_layers[i]->b, NULL);
}
}
sigmoid_layers[i]->W, sigmoid_layers[i]->b should be set NULL in DBN constructer
if not, RBM's W member will be sigmoid_layers[i]->W
when RBM deconstruction, there will have issue.
While changing from debug mode to release in Visual Studio makes the correct result. So where is the problem?
can I save the model of DBN?
can I save it using pickle?
I am testing the code using as a X input a matrix of values between -3 to 3, (23 columns) and the pre training is generating W matrix for the RBM layer with nan.
Reading the docs from DeepLearning (http://deeplearning.net/tutorial/rbm.html#rbm) the following: "Note that we also return the pre-sigmoid activation. To understand why this is so you need to understand a bit about how Theano works. Whenever you compile a Theano function, the computational graph that you pass as input gets optimized for speed and stability. This is done by changing several parts of the subgraphs with others. One such optimization expresses terms of the form log(sigmoid(x)) in terms of softplus. We need this optimization for the cross-entropy since sigmoid of numbers larger than 30. (or even less then that) turn to 1. and numbers smaller than -30. turn to 0 which in terms will force theano to compute log(0) and therefore we will get either -inf or NaN as cost. If the value is expressed in terms of softplus we do not get this undesirable behaviour. This optimization usually works fine, but here we have a special case. The sigmoid is applied inside the scan op, while the log is outside. Therefore Theano will only see log(scan(..)) instead of log(sigmoid(..)) and will not apply the wanted optimization. We can not go and replace the sigmoid in scan with something else also, because this only needs to be done on the last step. Therefore the easiest and more efficient way is to get also the pre-sigmoid activation as an output of scan, and apply both the log and sigmoid outside scan such that Theano can catch and optimize the expression"
However within this code the implementation has some differences. Any idea to overcome this issue?
Thank you
I converted the C code for DBN to Component Pascal and ran its analyzer over the code. It found in DBN_finetune that the variable "layer_input" was used before any values were set for it. Going back to the C code I see that is indeed true. In fact the array is accessed even before it is allocated.
That has got to be a bug.
void DBN_finetune(DBN* this, int *input, int *label, double lr, int epochs) {
int i, j, m, n, epoch;
int *layer_input;
// int prev_layer_input_size;
int *prev_layer_input;
int *train_X = (int *)malloc(sizeof(int) * this->n_ins);
int *train_Y = (int *)malloc(sizeof(int) * this->n_outs);
for(epoch=0; epoch<epochs; epoch++) {
for(n=0; nN; n++) { // input x1...xN
// initial input
for(m=0; mn_ins; m++) train_X[m] = input[n * this->n_ins + m];
for(m=0; mn_outs; m++) train_Y[m] = label[n * this->n_outs + m];
// layer input
for(i=0; i<this->n_layers; i++) {
if(i == 0) {
prev_layer_input = (int *)malloc(sizeof(int) * this->n_ins);
for(j=0; j<this->n_ins; j++) prev_layer_input[j] = train_X[j];
} else {
prev_layer_input = (int *)malloc(sizeof(int) * this->hidden_layer_sizes[i-1]);
for(j=0; j<this->hidden_layer_sizes[i-1]; j++) prev_layer_input[j] = layer_input[j];
free(layer_input);
}
layer_input = (int *)malloc(sizeof(int) * this->hidden_layer_sizes[i]);
HiddenLayer_sample_h_given_v(&(this->sigmoid_layers[i]), \
prev_layer_input, layer_input);
free(prev_layer_input);
}
LogisticRegression_train(&(this->log_layer), layer_input, train_Y, lr);
}
// lr *= 0.95;
}
free(layer_input);
free(train_X);
free(train_Y);
}
Dear Yusuke Sugomori,
In the C version of your deep learning program, I found a minor bug in DBN_predict (Line 193-200 of the DBN.c file). The initialization of " linear_output" should be moved into the k-loop as follows:
for(k=0; k<this->sigmoid_layers[i].n_out; k++) {
linear_output = 0.0;
for(j=0; j<this->sigmoid_layers[i].n_in; j++) {
linear_output += this->sigmoid_layers[i].W[k][j] * prev_layer_input[j];
}
linear_output += this->sigmoid_layers[i].b[k];
layer_input[k] = sigmoid(linear_output);
}
I guess the same problem may appear in the other versions.
Actually, I want to communicate with you for more details about the deep learning algorithm for specifc applications. May I know your email address?
peghoty
Recently, I am learning your RBM python code. All codes are OK except the updating formula of parameters used in your contrastive_divergence function. I found that the bias vectors are updated using the mean gradient of a batch training examples, while the weight matrix is not such the case. Hence, can kindly you give some explanation about the equation (Line 61-62) in details. I am looking forward your early replay.
Is there this type of deep learning model?
There are two labeled folders for binary classification.
ex) men and women, cats oand dogs, etc.
And then inserting the images to each folders as training data.
And then just run a simple command to train.
That’s all.
I need these simple training network model. Is there any?
I applied your dbn.cpp code . But for your all test data, the prediction results are equal. It is an unexpected situation (if we give a training sample as a train sample it can not find the true class, too). What is the problem? I expect your help.
Im trying to use python for SdA,
I dont know where I need to get LogisticRegression.
I have anaconda and Theano but still whenever I write:
from LogisticRegression import LogisticRegression
I receive an error that:
ImportError Traceback (most recent call last)
in ()
----> 1 from logistic_sgd import LogisticRegression, load_data
ImportError: No module named logistic_sgd
self.W += lr * ((numpy.dot(self.input.T, ph_sample) - numpy.dot(nv_samples.T, nh_means)))
I think this code should be self.W += lr * ((numpy.dot(self.input.T, ph_sample) - numpy.dot(nv_samples.T, nh_means)) / len(self.input))
I'm using the Java version of the code. Why the current functionality of the finetune
function is used for finetuning of the network? The finetune
function does not update the weights of the entire network (except for the last layer, i.e. log_layer
).
The other versions of the code written in different languages are using the same trick. Anyone can help me understand why this kind of fine-tuning works? Any references?
Thanks.
Hi,
I am trying to feed in a dataset with about 8000 instances (8391x8), but it gets an exception in LogisticRegression.train function computing d_y = self.y - p_y_given_x;
the error is
ValueError: operands could not be broadcast together with shapes (8391,2) (8,2)
I suppose using CDBN/CRBM classes we can do a prediction like regression but I don't know why the p_y_given_x size is not matching self.y which suppose to do.
Also I am using python.
thanks in advance
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.