tariqdaouda / mariana Goto Github PK
View Code? Open in Web Editor NEWThe Cutest Deep Learning Framework which is also a wonderful Declarative Language
Home Page: http://www.tariqdaouda.com
License: Apache License 2.0
The Cutest Deep Learning Framework which is also a wonderful Declarative Language
Home Page: http://www.tariqdaouda.com
License: Apache License 2.0
Training crashes with TypeError when using Binomial Dropout.
File "mlp.py", line 94, in train
res_train = mlp.train(o, inp=list(train_set['expressions']), target=train_set['class'])
File "build/bdist.linux-x86_64/egg/Mariana/network.py", line 202, in __getattribute__
File "build/bdist.linux-x86_64/egg/Mariana/network.py", line 129, in init
File "build/bdist.linux-x86_64/egg/Mariana/layers.py", line 384, in _setTheanoFunctions
File "build/bdist.linux-x86_64/egg/Mariana/wrappers.py", line 51, in __init__
File "/u/seguinj/repos/gvhd-ml/lib/python2.7/site-packages/Theano-0.7.0-py2.7.egg/theano/compile/function.py", line 310, in function
output_keys=output_keys)
File "/u/seguinj/repos/gvhd-ml/lib/python2.7/site-packages/Theano-0.7.0-py2.7.egg/theano/compile/pfunc.py", line 499, in pfunc
no_default_updates=no_default_updates)
File "/u/seguinj/repos/gvhd-ml/lib/python2.7/site-packages/Theano-0.7.0-py2.7.egg/theano/compile/pfunc.py", line 215, in rebuild_collect_shared
raise TypeError(err_msg, err_sug)
TypeError: ('An update must have the same type as the original shared variable (shared_var=<CudaNdarrayType(float32, matrix)>, shared_var.type=CudaNdarrayType(float32, matrix), update_val=Softmax.0, update_val.type=TensorType(float64, matrix)).', 'If the difference is related to the broadcast pattern, you can call the tensor.unbroadcast(var, axis_to_unbroadcast[, ...]) function to remove broadcastable dimensions.')
Decorators may not be applied to layers. This issue has been fixed in dev branch and will be part of the next merge.
The current version shows layer connectivity but not the hyper-parameters values. An improved visualization that contains information about hyper-parameters would allow the sharing of an entire model through a visual and small HTML file.
Downsampling has changed place in theano. Mariana will be updated very soon
Epoch 0
Train score: 0.735989272594
Validation score: 0.718354940414 +best+
Epoch 0
Train score: 0.915873587132
Validation score: 0.704464316368 +best+
Epoch 0
Train score: 2.71569609642
Validation score: 0.699838221073 +best+
Epoch 0
Train score: 20.8013763428
Validation score: 0.702481269836 +best+
Hello again @tariqdaouda,
Looks like I am facing a challenge here hehe.
What I am trying to do is actually use the already trained model in my system.
So I trained my model with the parameters I experimented to get me the best results and your trainer saved the parameters into the pkl file.
To load the model I used the code in the documentation:
`import cPickle
mlp = cPickle.load(open("myMLP.mariana.pkl"))
But mlp is a dict object, don't have the propagate function.
So I looked it up in the code and found the LoadModel method. I tried to load the model with the following code:
mlp_state = MN.loadModel("mlp_state/state-config")
Worked! Now I have the method propagate. To get the output layer result, I used this code (from test.py in the repository):
o = mlp_state.outputs.values()[0] mlp_state.classify(o, inp = numpy.asarray(inputSet))
But first I was getting an error that the dimensions of the input are wrong:
TypeError: ('Bad input argument to theano function with name "/media/guilhermevrs/Data/Documentos/Academico/TCC-code/Mariana/Mariana/wrappers.py:58" at index 1(0-based)', 'Wrong number of dimensions: expected 2, got 1 with shape (42,).')
My input is of 42 nodes. I even trained the model with 42.
So I placed my input data as the first element of an array. Now the error I am getting this one:
TypeError: Tried to provide value for implicit input: <TensorType(float64, matrix)>
Can you give me a light here, please?
ubgpu@ubgpu:~/github/Mariana$ sudo python Mariana/examples/mnist_mlp.py
Using gpu device 0: GeForce GTX 970
|/| /-\ |-> | /-\ || /->
I will use the [-GPU-] to run function 'train' of layer 'out'!
Cheers :),
Mariana
I will use the [-GPU-] to run function 'test' of layer 'out'!
Cheers :),
Mariana
I will use the [-GPU-] to run function 'propagate' of layer 'out'!
Cheers :),
Mariana
ERROR (theano.gof.opt): Optimization failure due to: local_argmax_pushdown
ERROR (theano.gof.opt): TRACEBACK:
ERROR (theano.gof.opt): Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/theano/gof/opt.py", line 1493, in process_node
replacements = lopt.transform(node)
File "/usr/local/lib/python2.7/dist-packages/theano/tensor/nnet/nnet.py", line 1448, in local_argmax_pushdown
return tensor._max_and_argmax(pre_x, axis)
File "/usr/local/lib/python2.7/dist-packages/theano/gof/op.py", line 507, in call
node = self.make_node(_inputs, *_kwargs)
File "/usr/local/lib/python2.7/dist-packages/theano/tensor/basic.py", line 1252, in make_node
raise TypeError("MaxAndArgmax needs a constant axis")
TypeError: MaxAndArgmax needs a constant axis
ERROR (theano.gof.opt): Optimization failure due to: local_argmax_pushdown
ERROR (theano.gof.opt): TRACEBACK:
ERROR (theano.gof.opt): Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/theano/gof/opt.py", line 1493, in process_node
replacements = lopt.transform(node)
File "/usr/local/lib/python2.7/dist-packages/theano/tensor/nnet/nnet.py", line 1454, in local_argmax_pushdown
('x', 0))(pre_bias), axis)
File "/usr/local/lib/python2.7/dist-packages/theano/gof/op.py", line 507, in call
node = self.make_node(_inputs, *_kwargs)
File "/usr/local/lib/python2.7/dist-packages/theano/tensor/basic.py", line 1252, in make_node
raise TypeError("MaxAndArgmax needs a constant axis")
TypeError: MaxAndArgmax needs a constant axis
==>rec: ggplot2, commit 1, pid: 14344:
|-test set
|->out: 1.62696 +best+
|-train set
|->out: 1.96462 +best+
|-validation set
|->out: 1.61709 +best+
==>rec: ggplot2, commit 2, pid: 14344:
|-test set
|->out: 1.48826 +best+
|-train set
|->out: 1.5747 +best+
|-validation set
|->out: 1.48133 +best+
==>rec: ggplot2, commit 3, pid: 14344:
|-test set
|->out: 1.38383 +best+
==>rec: ggplot2, commit 1001, pid: 14344:
|-test set
|->out: 0.119731 +best+
|-train set
|->out: 0.0630581 +best+
|-validation set
|->out: 0.124834 (best: 0.124825 @ commit: 974)
Dying gracefully from NameError, and saving myself to:
Traceback (most recent call last):
File "Mariana/examples/mnist_mlp.py", line 74, in
trainer.start("MLP", mlp)
File "/home/ubgpu/github/Mariana/Mariana/training/trainers.py", line 195, in start
Trainer_ABC.start( self, runName, model, recorder, trainingOrder, shuffle, datasetName )
File "/home/ubgpu/github/Mariana/Mariana/training/trainers.py", line 103, in start
return self.run(runName, model, recorder, _args, *_kwargs)
File "/home/ubgpu/github/Mariana/Mariana/training/trainers.py", line 340, in run
raise EndOfTraining(crit)
NameError: global name 'EndOfTraining' is not defined
ubgpu@ubgpu:~/github/Mariana$ git remote -v
Hello,
I am facing a problem when I start my trainer and I can't figure out the cause.
My input data is of dimension 42 and my output should be one value out of 4.
This is the shape of my training and test set:
Training set:
input = (1152, 42) target = (1152,)
Training set: input = (1152, 42) target = (1152,)
Test set: input = (384, 42) target = (384,)
This is the construction of my network:
ls = MS.GradientDescent(lr=0.01)
cost = MC.CrossEntropy()
i = ML.Input(42, name='inp')
h = ML.Hidden(23, activation=MA.Sigmoid(), initializations=[MI.GlorotTanhInit()], name="hid")
o = ML.SoftmaxClassifier(4, learningScenario=ls, costObject=cost, name="out")
mlp = i > h > o
And this is the construction of the datasets, trainers and recorders:
trainData = MDM.RandomSeries(distances = train_set[0], next_state = train_set[1])
trainMaps = MDM.DatasetMapper()
trainMaps.mapInput(i, trainData.distances)
trainMaps.mapOutput(o, trainData.next_state)
testData = MDM.RandomSeries(distances = test_set[0], next_state = test_set[1])
testMaps = MDM.DatasetMapper()
testMaps.mapInput(i, testData.distances)
testMaps.mapOutput(o, testData.next_state)
earlyStop = MSTOP.GeometricEarlyStopping(testMaps, patience=100, patienceIncreaseFactor=1.1, significantImprovement=0.00001, outputFunction="score", outputLayer=o)
epochWall = MSTOP.EpochWall(1000)
trainer = MT.DefaultTrainer(
trainMaps=trainMaps,
testMaps=testMaps,
validationMaps=None,
stopCriteria=[earlyStop, epochWall],
testFunctionName="testAndAccuracy",
trainMiniBatchSize=MT.DefaultTrainer.ALL_SET,
saveIfMurdered=False
)
recorder = MREC.GGPlot2("MLP", whenToSave = [MREC.SaveMin("test", o.name, "score")], printRate=1, writeRate=1)
trainer.start("MLP", mlp, recorder = recorder)
But the following error is being produced:
Traceback (most recent call last):
File "nn-mariana.py", line 82, in <module>
trainer.start("MLP", mlp, recorder = recorder)
File "SUPRESSED/Mariana/Mariana/training/trainers.py", line 226, in start
Trainer_ABC.start( self, runName, model, recorder, trainingOrder, moreHyperParameters )
File "SUPRESSED/Mariana/Mariana/training/trainers.py", line 110, in start
return self.run(runName, model, recorder, *args, **kwargs)
File "SUPRESSED/Mariana/Mariana/training/trainers.py", line 410, in run
outputLayers
File "SUPRESSED/Mariana/Mariana/training/trainers.py", line 269, in _trainTest
res = modelFct(output, **kwargs)
File "SUPRESSED/Mariana/Mariana/network.py", line 47, in __call__
return self.callTheanoFct(outputLayer, **kwargs)
File "SUPRESSED/Mariana/Mariana/network.py", line 44, in callTheanoFct
return self.outputFcts[ol](**kwargs)
File "SUPRESSED/Mariana/Mariana/wrappers.py", line 110, in __call__
return self.run(**kwargs)
File "SUPRESSED/Mariana/Mariana/wrappers.py", line 102, in run
fres = iter(self.theano_fct(*self.fctInputs.values()))
File "SUPRESSED/Theano/theano/compile/function_module.py", line 871, in __call__
storage_map=getattr(self.fn, 'storage_map', None))
File "SUPRESSED/Theano/theano/gof/link.py", line 314, in raise_with_op
reraise(exc_type, exc_value, exc_trace)
File "SUPRESSED/Theano/theano/compile/function_module.py", line 859, in __call__
outputs = self.fn()
ValueError: Input dimension mis-match. (input[0].shape[1] = 1152, input[1].shape[1] = 4)
Apply node that caused the error: Elemwise{Composite{((i0 * i1) + (i2 * log(i3)))}}[(0, 1)](InplaceDimShuffle{x,0}.0, LogSoftmax.0, Elemwise{sub,no_inplace}.0, Elemwise{sub,no_inplace}.0)
Toposort index: 18
Inputs types: [TensorType(int32, row), TensorType(float64, matrix), TensorType(int32, row), TensorType(float64, matrix)]
Inputs shapes: [(1, 1152), (1152, 4), (1, 1152), (1152, 4)]
Inputs strides: [(4608, 4), (32, 8), (4608, 4), (32, 8)]
Inputs values: ['not shown', 'not shown', 'not shown', 'not shown']
Outputs clients: [[Sum{axis=[1], acc_dtype=float64}(Elemwise{Composite{((i0 * i1) + (i2 * log(i3)))}}[(0, 1)].0)]]
Versions:
Mariana (1.0.1rc1, /media/guilhermevrs/Data/Documentos/Academico/TCC-code/Mariana)
Theano (0.8.0.dev0, SUPRESSED/Theano)
This code was produced having as base the tutorial code from the mnist example.
Could you please help me to figure out what's going on?
Thank you in advance
The gradient returned by the cost is 3D (because of the convolution), but the parameters of the embedding layer are 2D. This causes theano to crash.
The next update will have an Embedding layer for conv nets.
Composite layer is broken and may disappear in the future to be replaced by an automatic layer concatenation.
Hello @tariqdaouda,
First of all, thank you for this framework. It works very well!
I was wondering how does exactly work every observation made during the training phase.
I mean, supposing I am interested in getting the percentage error of my network on a given test set. What data should I look in this case?
Thank you in advance
Make Mariana compatible with python 3.
Add support for tensor flow.
Using save() on models with many layers may crash due to pickle limitations. This is also true for the auto-saves performed by the trainer and the recorder. It will be fixed in the upcoming version.
Hi,
I hope your well.
I was looking for the above file.
It comes with an error when i run the examples.
Many thanks,
Best,
Andrew
I am working on it. But the rest of Mariana is fine
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.