tflearn / tflearn Goto Github PK

View Code? Open in Web Editor NEW

9.6K 457.0 2.4K 1.55 MB

Deep learning library featuring a higher-level API for TensorFlow.

Home Page: http://tflearn.org

License: Other

Python 100.00%

tflearn tensorflow neural-network deep-learning machine-learning data-science

tflearn's Introduction

TFLearn: Deep learning library featuring a higher-level API for TensorFlow.

TFlearn is a modular and transparent deep learning library built on top of Tensorflow. It was designed to provide a higher-level API to TensorFlow in order to facilitate and speed-up experimentations, while remaining fully transparent and compatible with it.

TFLearn features include:

Easy-to-use and understand high-level API for implementing deep neural networks, with tutorial and examples.
Fast prototyping through highly modular built-in neural network layers, regularizers, optimizers, metrics...
Full transparency over Tensorflow. All functions are built over tensors and can be used independently of TFLearn.
Powerful helper functions to train any TensorFlow graph, with support of multiple inputs, outputs and optimizers.
Easy and beautiful graph visualization, with details about weights, gradients, activations and more...
Effortless device placement for using multiple CPU/GPU.

The high-level API currently supports most of recent deep learning models, such as Convolutions, LSTM, BiRNN, BatchNorm, PReLU, Residual networks, Generative networks... In the future, TFLearn is also intended to stay up-to-date with latest deep learning techniques.

Note: Latest TFLearn (v0.5) is only compatible with TensorFlow v2.0 and over.

Overview

# Classification
tflearn.init_graph(num_cores=8, gpu_memory_fraction=0.5)

net = tflearn.input_data(shape=[None, 784])
net = tflearn.fully_connected(net, 64)
net = tflearn.dropout(net, 0.5)
net = tflearn.fully_connected(net, 10, activation='softmax')
net = tflearn.regression(net, optimizer='adam', loss='categorical_crossentropy')

model = tflearn.DNN(net)
model.fit(X, Y)

# Sequence Generation
net = tflearn.input_data(shape=[None, 100, 5000])
net = tflearn.lstm(net, 64)
net = tflearn.dropout(net, 0.5)
net = tflearn.fully_connected(net, 5000, activation='softmax')
net = tflearn.regression(net, optimizer='adam', loss='categorical_crossentropy')

model = tflearn.SequenceGenerator(net, dictionary=idx, seq_maxlen=100)
model.fit(X, Y)
model.generate(50, temperature=1.0)

There are many more examples available here.

Compatibility

TFLearn is based on the original tensorflow v1 graph API. When using TFLearn, make sure to import tensorflow that way:

import tflearn
import tensorflow.compat.v1 as tf

Installation

TensorFlow Installation

TFLearn requires Tensorflow (version 2.0+) to be installed.

To install TensorFlow, simply run:

pip install tensorflow

or, with GPU-support:

pip install tensorflow-gpu

For more details see TensorFlow installation instructions

TFLearn Installation

To install TFLearn, the easiest way is to run

For the bleeding edge version (recommended):

pip install git+https://github.com/tflearn/tflearn.git

For the latest stable version:

pip install tflearn

Otherwise, you can also install from source by running (from source folder):

python setup.py install

For more details, please see the Installation Guide.

Getting Started

See Getting Started with TFLearn to learn about TFLearn basic functionalities or start browsing TFLearn Tutorials.

Examples

There are many neural network implementation available, see Examples.

Documentation

http://tflearn.org/doc_index

Model Visualization

Graph

Loss & Accuracy (multiple runs)

Layers

Contributions

This is the first release of TFLearn, if you find any bug, please report it in the GitHub issues section.

Improvements and requests for new features are more than welcome! Do not hesitate to twist and tweak TFLearn, and send pull-requests.

For more info: Contribute to TFLearn.

License

MIT License

tflearn's People

Contributors

Stargazers

Watchers

Forkers

codeaudit joshloyal laisun wavelets averroes directorscut82 leonardofed kfph bitcity snormore saifrahmed ghurkan lvtn protez faisal-w containerhouse ml-lab deenadayalancs lynsei firearasi tpxtask nimmen kursh fowum pkthebud clcarwin rubinovitz ahojnnes bnaul ml-ai-nlp-ir pseudoaj zgolkhou roth1002 chetkhatri parksanghyoun gpremsankar philipz zorodong gopal-m nkhuyu adrianlyjak snownus snazz2001 zihua rlugojr allenwoods debbierr wtest arunkgupta nickfyson dwyatte qidouhai xkxdhy philipbmw huleg hyungwonchoi fresiy8361 babooppa6 dwiel sejas snufkin sthomp debraid qgzang sebasstrogg mrn333 srikalyan daviddao loganding ewdave little1tow alex04309 tuyendothanh zhengyangcs tomlepaine xingyuxie zluo-miovision wanjinchang ipv1337 leo23 shendd 541435721 chrisemoulton berianjames heshizhu robustfengbin parthamaji jjsong beld qdrk kurtpan94 zbxzc35 lixiaosi33 290390573 gzzgz benjamesbabala liuhengli a75c6 ternence chizhizhen

tflearn's Issues

Embedding layer implementation

Hi,

Yesterday I happened to read the code for embedding layer:

inference = tf.cast(incoming, tf.int32)
inference = tf.nn.embedding_lookup(W, inference)
inference = tf.transpose(inference, [1, 0, 2])
inference = tf.reshape(inference, shape=[-1, output_dim])
inference = tf.split(0, n_inputs, inference)

However, I am not sure what function the last 3 lines serves. Could it be simplified as the following code?

inference = tf.cast(incoming, tf.int32)
inference = tf.nn.embedding_lookup(W, inference)

Thanks a lot !

[Request a new feature] support python3

Dear,

Do have a plan to support python3?

lstm

lstm example

conv_1d: TypeError: Expected list for attr squeeze_dims

in the func conv_1d:
343 inference = tf.nn.conv2d(inference, W, strides, padding) 344 if b: inference = tf.nn.bias_add(inference, b) --> 345 inference = tf.squeeze(inference, 3) 346 inference = activations.get(activation)(inference)
but tf.squeeze seems to accept only list type for the second parameter

Add more docs

There is only one page to teach how to use TFLearn (get_started.md), some doc should be added:

Layers (intro + usage)
Models (intro + usage)
Built-in ops (cf activations, losses, etc...) (intro + usage)
Helpers (intro + usage)
Create some custom layers
Tutorials

Any suggestion is welcome :)

lstm value error of different shape

I tried to modify imdb example to my dataset, which is given below
3 3 373 27 9 615 9 16 10 34 0 8 0 199 65917 1319 122 402 319 183
3 3 77 12 4 66 4 3 0 5 0 14 3 50 106 139 38 164 53 109
3 3 86 6 2 6 2 0 0 1 0 25 0 4 284 77888 19 66 11 25
3 3 469 21 7 291 7 43 15 82 0 207 0 181 115646 59073 294 928 112 675
3 3 2090 21 7 4035 7 17 8 40 0 317 10 717 1033 25661 142 2054 1795 1023
3 3 691 18 6 597 6 30 16 61 0 245 18 273 719 2352305 213 1106 324 719
6 6 229 0 8 526 0 11 1 13 0 6 5 101 7246 2082 120 141 288 1570
3 3 1158 9 3 649 3 16 6 17 1 247 38 477 592 987626 82 1305 653 707
4 4 211 0 10 429 0 16 9 20 0 3 0 106 42725 27302 4280 133 477 1567

The first column is the target which has 9 classes and around 1803 features

from future import print_function
import numpy as np
from sklearn.cross_validation import train_test_split
import tflearn
import pandas as pd
from tflearn.data_utils import to_categorical, pad_sequences

print("Loading")
data = pd.read_csv('Train.csv')

X = data.iloc[:,1:1805]
y = data.iloc[:,0]

X_train, X_test, y_train, y_test = train_test_split(X, y,test_size=0.2,random_state=42)

print(X_train.shape)
print(X_test.shape)
print(y_train.shape)
print(y_test.shape)

print("Preprocessing")
X_train1 = X_train.values.T.tolist()
X_test1 = X_test.values.tolist()
y_train1 = y_train.values.T.tolist()
y_test1 = y_test.values.tolist()

Data preprocessing

Sequence padding

trainX = pad_sequences(X_train1, maxlen=200, value=0.)
testX = pad_sequences(X_test1, maxlen=200, value=0.)

Converting labels to binary vectors

trainY = to_categorical(y_train, nb_classes=0)
testY = to_categorical(y_test, nb_classes=0)

Network building

net = tflearn.input_data([None, 200])
net = tflearn.embedding(net, input_dim=20000, output_dim=128)
net = tflearn.lstm(net, 128)
net = tflearn.dropout(net, 0.5)
net = tflearn.fully_connected(net, 2, activation='softmax')
net = tflearn.regression(net, optimizer='adam',loss='categorical_crossentropy')

Training

model = tflearn.DNN(net, clip_gradients=0., tensorboard_verbose=0)
model.fit(trainX, trainY, validation_set=(testX, testY), show_metric=True, batch_size=128)

Deep residual block always uses a projection instead of identity

Refering to this line: https://github.com/tflearn/tflearn/blob/master/tflearn/layers/conv.py#L617.
Because it is connected to the two high-dimensional ends, it doubles model size and the time to train a residual net according to He et al. .

I guess you want to do this instead:

# Projection to new dimension
                if in_channels != out_channels:
                    identity = conv_2d(identity, out_channels, 1, 1, 'same',
                                       'linear', bias, weights_init,
                                       bias_init, regularizer, weight_decay,
                                       trainable, restore)

Or an additional parameter called projection=False in the function arguments. I can submit a PR if you want.

After installing tflearn, the tensorflow have a numpy error

import tensorflow as tf
RuntimeError: module compiled against API version a but this version of numpy is 9
...
_pywrap_tensorflow = swig_import_helper()
File "/home/mvg/tensorflow/_python_build/tensorflow/python/pywrap_tensorflow.py", line 24, in swig_import_helper
_mod = imp.load_module('_pywrap_tensorflow', fp, pathname, description)
ImportError: numpy.core.multiarray failed to import

Support execution in Jupyter notebook

Trying to run the contents of examples/basics/linear_regression.py (or anything, really) within a Jupyter notebook gives the following error:

---------------------------------
Run id: M51N6E
Log directory: /tmp/tflearn_logs/
---------------------------------------------------------------------------
UnsupportedOperation                      Traceback (most recent call last)
<ipython-input-1-99453409cb2b> in <module>()
     15                                 metric='R2', learning_rate=0.01)
     16 m = tflearn.DNN(regression)
---> 17 m.fit(X, Y, n_epoch=1000, show_metric=True, snapshot_epoch=False)
     18 
     19 print("\nRegression result:")

/Users/brettnaul/miniconda3/envs/deep/lib/python3.5/site-packages/tflearn/models/dnn.py in fit(self, X_inputs, Y_targets, n_epoch, validation_set, show_metric, batch_size, shuffle, snapshot_epoch, snapshot_step, run_id)
    159                          snapshot_epoch=snapshot_epoch,
    160                          shuffle_all=shuffle,
--> 161                          run_id=run_id)
    162 
    163     def predict(self, X):

/Users/brettnaul/miniconda3/envs/deep/lib/python3.5/site-packages/tflearn/helpers/trainer.py in fit(self, feed_dicts, n_epoch, val_feed_dicts, show_metric, snapshot_step, snapshot_epoch, shuffle_all, run_id)
    188             validation_split(val_feed_dicts, feed_dicts)
    189 
--> 190             termlogger = callbacks.TermLogger(self.training_step)
    191             modelsaver = callbacks.ModelSaver(self.save,
    192                                               self.training_step,

/Users/brettnaul/miniconda3/envs/deep/lib/python3.5/site-packages/tflearn/callbacks.py in __init__(self, training_step)
     51         self.global_data_size = 0
     52         self.global_val_data_size = 0
---> 53         curses.setupterm()
     54         sys.stdout.write(curses.tigetstr('civis').decode())
     55 

/Users/brettnaul/miniconda3/envs/deep/lib/python3.5/site-packages/ipykernel/iostream.py in fileno(self)
    304 
    305     def fileno(self):
--> 306         raise UnsupportedOperation("IOStream has no fileno.")
    307 
    308     def write(self, string):

UnsupportedOperation: IOStream has no fileno.

I think this could be as simple as adding a flag to skip the TermLogger step, but maybe there's a solution that would look nice in the notebook and fall back to the curses version in the terminal?

Plain text progress bar that works in both: https://github.com/noamraph/tqdm
Fancy progress bar but not clear how hard it is to use outside of dask/distributed: http://matthewrocklin.com/blog/work/2016/02/26/dask-distributed-part-3#normalize-data

Implement dynamic length RNNs

Currently the use of RNNs in tflearn is quite limited by the fact that they can only operate on fixed length sequences. This limits real world applicability quite a bit since a lot of real world data doesn't have a fixed length, or has a length that exceeds the number of steps that can be unrolled on a normal GPU.

Also, when training a network on a continuous sequence like in the shakespeare example this makes the training a lot slower and the training set a lot bigger than it needs to be since overlapping sequences need to be fed into the network on every step, instead of just continuing from the state where the network was at before. Of course the also severely limits the attention span of the RNN.

I realize that until a short while ago this was a limitation of tensorflow itself, but they recently implemented support for dynamic RNNs and the option to iterate over variable length data. More info can be found here: tensorflow/tensorflow#208

I would really appreciate this feature since tflearn has been the first machine learning library I've actually been able to wrap my head around and easily get good results with.

Finetuning

How can we finetune a model?

NLP examples broken?

All the examples in the nlp folder are returning an error of the form:
1524438 inputs feeded, but expected: 2

Any idea why?

extract feature

How to extract extract features in a particular layer with a pre-trained model via tflearn ?

WARNING:Passing a `GraphDef` to the SummaryWriter is deprecated. Pass a `Graph` object instead, such as `sess.graph`.

When running an lstm_generator*.py and TensorFlow 0.8.

Guessing this an 0.8 issue and will go away once tflearn is upgraded but thought I would post it in case not.

Errors in running linear regression on Mac with python 3.5.1 (Anaconda)

I just did a fresh installation on my Mac 10.11.4 that has python 3.5.1 (Anaconda and Scikit-learn). The tensorflow installation was successful, as I was to verify it with their installation test.

However, I am running into many issues with the first example linear regress from here , I got many complaints about the python 2-style print statements (no paranthesis) and the following issues:

Run id: P97QY2
Log directory: /tmp/tflearn_logs/
Traceback (most recent call last):
  File "/Users/xxx/Desktop/Code/Python/Tensorflow/TFLearn/linear_regression/example1.py", line 15, in <module>
    m.fit(X, Y, n_epoch=1000, show_metric=True, snapshot_epoch=False)
  File "/Users/xxx/anaconda/lib/python3.5/site-packages/tflearn/models/dnn.py", line 161, in fit
    run_id=run_id)
  File "/Users/xxx/anaconda/lib/python3.5/site-packages/tflearn/helpers/trainer.py", line 190, in fit
    termlogger = callbacks.TermLogger(self.training_step)
  File "/Users/xxx/anaconda/lib/python3.5/site-packages/tflearn/callbacks.py", line 53, in __init__
    curses.setupterm()
_curses.error: setupterm: could not find terminfo database

Any idea what might be wrong? Thanks for making the module available!

.predict() doesn't work after loading a model

Hi,
I was playing with lstm.py example and noticed that it's crashing when it tries to predict from restored model. Mayby i'm doing something wrong, but you can reproduce this using this gist:
https://gist.github.com/morus12/846eedb8c90423ecd8b28c2440be6d6d

Output:

Run id: 2I82V8
Log directory: /tmp/tflearn_logs/
---------------------------------
Training samples: 10373
Validation samples: 25000
--
Training Step: 82  | total loss: 0.48628
| Adam | epoch: 001 | loss: 0.48628 - acc: 0.7767 | val_loss: 0.48750 - val_acc: 0.7694 -- iter: 00000/10373
--
[[0.06058039888739586, 0.9394195675849915]]
WARNING:tensorflow:Error encountered when serializing layer_variables/LSTM/.
Type is unsupported, or the types of the items don't match field type in CollectionDef.
'NoneType' object has no attribute 'name'
W tensorflow/core/common_runtime/executor.cc:1102] 0x7fc637a34520 Compute status: Failed precondition: Attempting to use uninitialized value Dense/W
     [[Node: Dense/W/read = Identity[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/cpu:0"](Dense/W)]]
     [[Node: _recv_InputData/X_0 = _Recv[client_terminated=true, recv_device="/job:localhost/replica:0/task:0/cpu:0", send_device="/job:localhost/replica:0/task:0/cpu:0", send_device_incarnation=-9003797786948543025, tensor_name="InputData/X:0", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/cpu:0"]()]]
Traceback (most recent call last):
  File "lstm.py", line 60, in <module>
    print(model.predict(pad_sequences([[1,2,3]], maxlen=50, value=0.)))
  File "/usr/local/lib/python2.7/site-packages/tflearn/models/dnn.py", line 177, in predict
    return self.predictor.predict(feed_dict)
  File "/usr/local/lib/python2.7/site-packages/tflearn/helpers/evaluator.py", line 53, in predict
    o_pred = self.session.run(output, feed_dict=feed_dict).tolist()
  File "/usr/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 315, in run
    return self._run(None, fetches, feed_dict)
  File "/usr/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 511, in _run
    feed_dict_string)
  File "/usr/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 564, in _do_run
    target_list)
  File "/usr/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 586, in _do_call
    e.code)
tensorflow.python.framework.errors.FailedPreconditionError: Attempting to use uninitialized value Dense/W
     [[Node: Dense/W/read = Identity[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/cpu:0"](Dense/W)]]
Caused by op u'Dense/W/read', defined at:
  File "lstm.py", line 46, in <module>
    net = tflearn.fully_connected(net, 2, activation='softmax')
  File "/usr/local/lib/python2.7/site-packages/tflearn/layers/core.py", line 116, in fully_connected
    trainable=trainable, restore=restore)
  File "/usr/local/lib/python2.7/site-packages/tflearn/variables.py", line 54, in variable
    collections=collections)
  File "/usr/local/lib/python2.7/site-packages/tensorflow/python/ops/variable_scope.py", line 256, in get_variable
    trainable, collections)
  File "/usr/local/lib/python2.7/site-packages/tensorflow/python/ops/variable_scope.py", line 188, in get_variable
    self.reuse, trainable, collections)
  File "/usr/local/lib/python2.7/site-packages/tensorflow/python/ops/variable_scope.py", line 132, in get_variable
    collections=collections)
  File "/usr/local/lib/python2.7/site-packages/tensorflow/python/ops/variables.py", line 195, in __init__
    name=name)
  File "/usr/local/lib/python2.7/site-packages/tensorflow/python/ops/variables.py", line 242, in _init_from_args
    self._snapshot = array_ops.identity(self._variable, name="read")
  File "/usr/local/lib/python2.7/site-packages/tensorflow/python/ops/gen_array_ops.py", line 436, in identity
    return _op_def_lib.apply_op("Identity", input=input, name=name)
  File "/usr/local/lib/python2.7/site-packages/tensorflow/python/ops/op_def_library.py", line 655, in apply_op
    op_def=op_def)
  File "/usr/local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2040, in create_op
    original_op=self._default_original_op, op_def=op_def)
  File "/usr/local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1087, in __init__
    self._traceback = _extract_stack()

MNIST example doesnt work using kaggle notebook

I am trying to run the mnist example code on kaggle using their python notebook and below is the error
I get. I think it is worth trying exactly what the error warning says.

Using a `tf.Tensor` as a Python `bool` is not allowed. Use `if t is not None:` instead of `if t:` to test if a tensor is defined, and use the logical TensorFlow ops to test the value of a tensor.
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-15-cd5f9ee44c89> in <module>()
     25 
     26 # Training
---> 27 model = tflearn.DNN(network, tensorboard_verbose=3)
     28 model.fit({'input': train}, {'target': target}, n_epoch=20,
     29           # validation_set=({'input': testX}, {'target': testY}),

....................

/opt/conda/lib/python3.5/site-packages/tensorflow/python/framework/ops.py in __bool__(self)
    460       `TypeError`.
    461     """
--> 462     raise TypeError("Using a `tf.Tensor` as a Python `bool` is not allowed. "
    463                     "Use `if t is not None:` instead of `if t:` to test if a "
    464                     "tensor is defined, and use the logical TensorFlow ops "

TypeError: Using a `tf.Tensor` as a Python `bool` is not allowed. Use `if t is not None:` instead of `if t:` to test if a tensor is defined, and use the logical TensorFlow ops to test the value of a tensor.

ImportError: cannot import name tensor_array_ops

This is with tensorflow-0.7.1-cp27-none-linux_x86_64.whl installed on Arch Linux. tensorflow itself works fine.

reed@4k ~ % python2
Python 2.7.11 (default, Mar 31 2016, 06:18:34) 
[GCC 5.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import tflearn
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/site-packages/tflearn/__init__.py", line 44, in <module>
    from . import layers
  File "/usr/lib/python2.7/site-packages/tflearn/layers/__init__.py", line 8, in <module>
    from .recurrent import lstm, gru, simple_rnn, bidirectional_rnn, dynamic_rnn, RNNCell, BasicLSTMCell, GRUCell, BasicRNNCell
  File "/usr/lib/python2.7/site-packages/tflearn/layers/recurrent.py", line 14, in <module>
    from tensorflow.python.ops import tensor_array_ops
ImportError: cannot import name tensor_array_ops

Something after april-24-day commit breaks dnn.py

The same code running in top of commit 19746fc was okey, bit now (on master) I got that:

              File "/usr/lib/python3.4/unittest/case.py", line 58, in testPartExecutor
    yield
  File "/usr/lib/python3.4/unittest/case.py", line 577, in run
    testMethod()
  File "/root/src/bitbucket.org/nexusedge/ai-nlp-tc/tests/test_classifier.py", line 21, in test_classification
    tfclassifier = classifier.TfClassifier(dataset, max_precision=α)
  File "/root/src/bitbucket.org/nexusedge/ai-nlp-tc/tc/classifier.py", line 52, in __init__
    self.precision = self._pre_training()
  File "/root/src/bitbucket.org/nexusedge/ai-nlp-tc/tc/classifier.py", line 125, in _pre_training
    self.train(X, Y)
  File "/root/src/bitbucket.org/nexusedge/ai-nlp-tc/tc/classifier.py", line 58, in train
    self.classifier.fit(X, Y, validation_set=(X, Y))
  File "/root/venv/3.4/src/tflearn/tflearn/models/dnn.py", line 168, in fit
    if dprep_collection[i] is not None:
list index out of range

What could be?

tflearn has dependency with PIL

Hello.
tflearn is cool!

I want to report a bug
I try vgg_network.py with my own dataset(street snap image)

...
import tflearn
from tflearn.layers.core import input_data, dropout, fully_connected
from tflearn.layers.conv import conv_2d, max_pool_2d
from tflearn.layers.estimator import regression

import input_data
dataset = input_data.load_fashion_dataset()
...

then it can't load numpy array data.
i use this function for converting image file to numpy array.

from PIL import Image
import numpy as np
...

def img2numpy_arr(img_path):
    return np.array(Image.open(img_path))

so i check this function and find ' Image.open(img_path) ' doesn't work.

now, i try vgg_network.py like this:

...
import input_data
dataset = input_data.load_fashion_dataset()
...
import tflearn
from tflearn.layers.core import input_data, dropout, fully_connected
from tflearn.layers.conv import conv_2d, max_pool_2d
from tflearn.layers.estimator import regression

AlexNet 17 flowers example - No change in loss/accuracy while training

Hi!
At first I want to say, I was very excited when I read about tflearn. I am currently writing a paper about image recognition with tensorflow and tflearn seems to be a great library for providing some examples easily.

First I tried some examples, the cifar10 worked very well but I got stuck in the AlexNet (tflearn/examples/images/alexnet.py) example. After solving some problems with Pillow the network loaded and got trained. For training I am using a GTX 750 ti. After loading the images I got a couple of the following warning:

W tensorflow/core/common_runtime/bfc_allocator.cc:213] Ran out of memory trying to allocate 
342.38MiB. The caller indicates that this is not a failure, but may mean that there could be 
performance gains if more memory is available.

But the network was trained afterwards. So I let it train for about 12 hours (last night) and was excited about the results in the morning. Unfortunately there was no big change in loss and accuracy. Loss is always somewhere around 2.8 and accuracy somewhere between 0.04 and 0.07.

Unfortunately I couldn't find any mistakes in building the net or loading the data, what led me to open an issue here. I didn't change any of the example code, do you have any idea why it isn't working?

Many thanks in advance,
Arno

Assign requires shapes of both tensors to match, restore=False

I am attempting to "finetune" a model, and alter the last three layers of the network. The network is essentially VGG

Initially I had trained with these last three layers:

network = fully_connected(network, 1024, activation='relu')
network = dropout(network, 0.5)
network = fully_connected(network, 1024, activation='relu')
network = dropout(network, 0.5)
network = fully_connected(network, 256, activation='relu')

I then attempted to fine tune, with these layers:

network = fully_connected(network, 4096, activation='relu', restore=False)
network = dropout(network, 0.5)
network = fully_connected(network, 4096, activation='relu', restore=False)
network = dropout(network, 0.5)
network = fully_connected(network, 256, activation='relu', restore=False)

I get this error:

tensorflow.python.framework.errors.InvalidArgumentError: Assign requires shapes of both tensors to match. lhs shape= [3136,4096] rhs shape= [3136,1024]

It makes sense that the loaded model weights can't be restored onto these newly shaped layers, but I expect that was what restore=False would do. Thank you in advance for the great project and whatever help you can offer.

Distributed TensorFlow

Hi Aymeric,

First off I really love this API, and it came out at just the right time for me. I was looking at TF for an upcoming research project and was a bit intimidated, but now I'm able to whip up a really nice convnet super quick, so thanks!

Secondly, based on what I read, TFLearn abstracts away device placement, right? So if I just toss my code onto a machine with two GPUs it will automatically put things in place, or is that handled entirely by the TensorFlow back-end? (The section on their website about this was a tad confusing)

Finally, with the recent release of tf version 0.8, how reasonable (or possible) would it be to implement a further abstraction for specifying distributed training? In their tutorial, it seems you need to specify a device for each variable. it would be really awesome if I could just provide some minimal configuration information and train the same network on a ton of machines without changing too much code.

Using pre-trained word vectors in embedding layer

Hi, thank you for creating such a wonderful project.

I am now wondering if there is an embedding layer using pre-trained weights of word vectors? Thank you!

HDF5 library version mismatched error - TFLearn on AWS EC2 Ubuntu 14.04

I'm trying to install TFLearn on AWS EC2 Ubuntu 14.04. I have done sudo apt-get install libhdf5-dev but when I import tflearn...

Warning! _HDF5 library version mismatched error_
Headers are 1.8.11, library is 1.8.15

TFLearn v0.1 & future development

Thanks to everyone who contributed! Many bugs have already been corrected.
It is planned to open a new branch "0.1" to keep a stable version of tflearn. This version will be used for pypi, so that future development will not affect it. "master" branch will be used instead as the bleeding edge version of TFLearn.

Note: Only critical issues may still be fix in stable version, but no new features will be added.

can not "predict" in other python file by loading saved model

Hi,

I have trained a model using the attached train.py (github does not allow to load .py, so I changed it to .txt). After that, I am going to test the saved model on some new data (by using predict.py). But I got some problems here, you can see the errors in the error_output.txt.
Did I miss sth about checkpoint file? I really don't know how to fix it.
Thanks.

error_ouput.txt
predict.py.txt
train.py.txt

Feature Request: Port into actual tensorflow

Similar to the skflow project. https://github.com/tensorflow/skflow

Potentially using https://git-scm.com/book/en/v2/Git-Tools-Submodules

How to save model and retrain it?

Hi,
Is there some APIs to be used to save model and restore it for retraining?

Loading saved model and model-meta doesn't work

Hi! Unfortunately, I still couldn't solve a big problem. It's very important to get a network with all its weights saved and then loaded from another program, so that its possible to train once and make predictions independently whenever needed. Saving the model with meta seems to work properly, the two files are created. Loading the model again won't work. I even tried to save and load model in the same program after my training has finished with the following code:

from __future__ import division, print_function, absolute_import

import tflearn
from tflearn.layers.core import input_data, dropout, fully_connected
from tflearn.layers.conv import conv_2d, max_pool_2d
from tflearn.layers.normalization import local_response_normalization
from tflearn.layers.estimator import regression

import image_loader
X, Y = image_loader.load_data(one_hot=True, shuffle=False)

print(X.shape)

# Build network
input_layer = input_data(shape=[None, 224, 224, 3])

conv1 = conv_2d(input_layer, 96, 11, strides=4, activation='relu')
pool1 = max_pool_2d(conv1, 3, strides=2)
network = local_response_normalization(pool1)

conv2 = conv_2d(network, 256, 5, activation='relu')
pool2 = max_pool_2d(conv2, 3, strides=2)
network = local_response_normalization(pool2)

conv3 = conv_2d(network, 384, 3, activation='relu')
conv4 = conv_2d(conv3, 384, 3, activation='relu')
conv5 = conv_2d(conv4, 256, 3, activation='relu')
pool3 = max_pool_2d(conv5, 3, strides=2)
network = local_response_normalization(pool3)

fc1 = fully_connected(network, 4096, activation='tanh')
dropout1 = dropout(fc1, 0.5)

fc2 = fully_connected(dropout1, 4096, activation='tanh')
dropout2 = dropout(fc2, 0.5)

fc3 = fully_connected(dropout2, 2, activation='softmax')
network = regression(fc3, optimizer='momentum',
                     loss='categorical_crossentropy',
                     learning_rate=0.01)

# Training
model = tflearn.DNN(network, checkpoint_path='model_ba.ckpt',
                    max_checkpoints=1, tensorboard_verbose=0)
# n_epoch von 2000 auf 1 gesetzt
model.fit(X, Y, n_epoch=1, validation_set=0.1, shuffle=True,
          show_metric=True, batch_size=32, snapshot_step=400,
          snapshot_epoch=False, run_id='ba_soccer_network')


# Save the model
model.save('my_model')

# Load weights from saved model
model.load('my_model')

Running the program, it always crashes with the error message (and a whole bunch of memory allocation warnings)

tensorflow.python.framework.errors.ResourceExhaustedError: OOM when allocating tensor 
with shape[12544,4096]

Can't explain that behaviour because it worked to train the network of the same size before without getting any error.

I also tried to load the model from another source-code where it didn't train before, so I just built up the same network without model.fit() and tried model.load() with the same error but a smaller shape:

W tensorflow/core/framework/op_kernel.cc:900] Resource exhausted: OOM when allocating 
tensor with shape[4096,4096]

What is it I don't get? Or is there an issue in model.load() maybe?
Thanks in advance

Installing from source on Ubuntu: "ValueError: jpeg is required unless explicitly disabled using --disable-jpeg, aborting"

Hi,

I'm attempting to install from the source on Ubuntu.

However I'm getting an error. I get the same error when installing using pip install git+https://github.com/tflearn/tflearn.git

Any idea how to fix?

Here is the output of the terminal:

  julien@julien-Linux:~/Desktop/AI/tensorFlow/examples/tflearn-master$ sudo python setup.py install
  running install
  running bdist_egg
  running egg_info
  writing requirements to TFLearn.egg-info/requires.txt
  writing TFLearn.egg-info/PKG-INFO
  writing top-level names to TFLearn.egg-info/top_level.txt
  writing dependency_links to TFLearn.egg-info/dependency_links.txt
  reading manifest file 'TFLearn.egg-info/SOURCES.txt'
  writing manifest file 'TFLearn.egg-info/SOURCES.txt'
  installing library code to build/bdist.linux-x86_64/egg
  running install_lib
  running build_py
  creating build/bdist.linux-x86_64/egg
  creating build/bdist.linux-x86_64/egg/tflearn
  copying build/lib.linux-x86_64-2.7/tflearn/metrics.py -> build/bdist.linux-x86_64/egg/tflearn
  creating build/bdist.linux-x86_64/egg/tflearn/datasets
  copying build/lib.linux-x86_64-2.7/tflearn/datasets/mnist.py -> build/bdist.linux-x86_64/egg/tflearn/datasets
  copying build/lib.linux-x86_64-2.7/tflearn/datasets/oxflower17.py -> build/bdist.linux-x86_64/egg/tflearn/datasets
  copying build/lib.linux-x86_64-2.7/tflearn/datasets/cifar10.py -> build/bdist.linux-x86_64/egg/tflearn/datasets
  copying build/lib.linux-x86_64-2.7/tflearn/datasets/imdb.py -> build/bdist.linux-x86_64/egg/tflearn/datasets
  copying build/lib.linux-x86_64-2.7/tflearn/datasets/__init__.py -> build/bdist.linux-x86_64/egg/tflearn/datasets
  copying build/lib.linux-x86_64-2.7/tflearn/collections.py -> build/bdist.linux-x86_64/egg/tflearn
  copying build/lib.linux-x86_64-2.7/tflearn/data_preprocessing.py -> build/bdist.linux-x86_64/egg/tflearn
  copying build/lib.linux-x86_64-2.7/tflearn/initializations.py -> build/bdist.linux-x86_64/egg/tflearn
  copying build/lib.linux-x86_64-2.7/tflearn/summaries.py -> build/bdist.linux-x86_64/egg/tflearn
  copying build/lib.linux-x86_64-2.7/tflearn/losses.py -> build/bdist.linux-x86_64/egg/tflearn
  copying build/lib.linux-x86_64-2.7/tflearn/data_augmentation.py -> build/bdist.linux-x86_64/egg/tflearn
  copying build/lib.linux-x86_64-2.7/tflearn/utils.py -> build/bdist.linux-x86_64/egg/tflearn
  copying build/lib.linux-x86_64-2.7/tflearn/config.py -> build/bdist.linux-x86_64/egg/tflearn
  copying build/lib.linux-x86_64-2.7/tflearn/optimizers.py -> build/bdist.linux-x86_64/egg/tflearn
  copying build/lib.linux-x86_64-2.7/tflearn/data_flow.py -> build/bdist.linux-x86_64/egg/tflearn
  copying build/lib.linux-x86_64-2.7/tflearn/callbacks.py -> build/bdist.linux-x86_64/egg/tflearn
  creating build/bdist.linux-x86_64/egg/tflearn/models
  copying build/lib.linux-x86_64-2.7/tflearn/models/generator.py -> build/bdist.linux-x86_64/egg/tflearn/models
  copying build/lib.linux-x86_64-2.7/tflearn/models/dnn.py -> build/bdist.linux-x86_64/egg/tflearn/models
  copying build/lib.linux-x86_64-2.7/tflearn/models/__init__.py -> build/bdist.linux-x86_64/egg/tflearn/models
  copying build/lib.linux-x86_64-2.7/tflearn/objectives.py -> build/bdist.linux-x86_64/egg/tflearn
  copying build/lib.linux-x86_64-2.7/tflearn/data_utils.py -> build/bdist.linux-x86_64/egg/tflearn
  copying build/lib.linux-x86_64-2.7/tflearn/activations.py -> build/bdist.linux-x86_64/egg/tflearn
  copying build/lib.linux-x86_64-2.7/tflearn/__init__.py -> build/bdist.linux-x86_64/egg/tflearn
  creating build/bdist.linux-x86_64/egg/tflearn/helpers
  copying build/lib.linux-x86_64-2.7/tflearn/helpers/generator.py -> build/bdist.linux-x86_64/egg/tflearn/helpers
  copying build/lib.linux-x86_64-2.7/tflearn/helpers/evaluator.py -> build/bdist.linux-x86_64/egg/tflearn/helpers
  copying build/lib.linux-x86_64-2.7/tflearn/helpers/regularizer.py -> build/bdist.linux-x86_64/egg/tflearn/helpers
  copying build/lib.linux-x86_64-2.7/tflearn/helpers/trainer.py -> build/bdist.linux-x86_64/egg/tflearn/helpers
  copying build/lib.linux-x86_64-2.7/tflearn/helpers/__init__.py -> build/bdist.linux-x86_64/egg/tflearn/helpers
  copying build/lib.linux-x86_64-2.7/tflearn/helpers/summarizer.py -> build/bdist.linux-x86_64/egg/tflearn/helpers
  creating build/bdist.linux-x86_64/egg/tflearn/layers
  copying build/lib.linux-x86_64-2.7/tflearn/layers/embedding_ops.py -> build/bdist.linux-x86_64/egg/tflearn/layers
  copying build/lib.linux-x86_64-2.7/tflearn/layers/normalization.py -> build/bdist.linux-x86_64/egg/tflearn/layers
  copying build/lib.linux-x86_64-2.7/tflearn/layers/conv.py -> build/bdist.linux-x86_64/egg/tflearn/layers
  copying build/lib.linux-x86_64-2.7/tflearn/layers/__init__.py -> build/bdist.linux-x86_64/egg/tflearn/layers
  copying build/lib.linux-x86_64-2.7/tflearn/layers/recurrent.py -> build/bdist.linux-x86_64/egg/tflearn/layers
  copying build/lib.linux-x86_64-2.7/tflearn/layers/estimator.py -> build/bdist.linux-x86_64/egg/tflearn/layers
  copying build/lib.linux-x86_64-2.7/tflearn/layers/core.py -> build/bdist.linux-x86_64/egg/tflearn/layers
  copying build/lib.linux-x86_64-2.7/tflearn/layers/merge_ops.py -> build/bdist.linux-x86_64/egg/tflearn/layers
  copying build/lib.linux-x86_64-2.7/tflearn/variables.py -> build/bdist.linux-x86_64/egg/tflearn
  byte-compiling build/bdist.linux-x86_64/egg/tflearn/metrics.py to metrics.pyc
  byte-compiling build/bdist.linux-x86_64/egg/tflearn/datasets/mnist.py to mnist.pyc
  byte-compiling build/bdist.linux-x86_64/egg/tflearn/datasets/oxflower17.py to oxflower17.pyc
  byte-compiling build/bdist.linux-x86_64/egg/tflearn/datasets/cifar10.py to cifar10.pyc
  byte-compiling build/bdist.linux-x86_64/egg/tflearn/datasets/imdb.py to imdb.pyc
  byte-compiling build/bdist.linux-x86_64/egg/tflearn/datasets/__init__.py to __init__.pyc
  byte-compiling build/bdist.linux-x86_64/egg/tflearn/collections.py to collections.pyc
  byte-compiling build/bdist.linux-x86_64/egg/tflearn/data_preprocessing.py to data_preprocessing.pyc
  byte-compiling build/bdist.linux-x86_64/egg/tflearn/initializations.py to initializations.pyc
  byte-compiling build/bdist.linux-x86_64/egg/tflearn/summaries.py to summaries.pyc
  byte-compiling build/bdist.linux-x86_64/egg/tflearn/losses.py to losses.pyc
  byte-compiling build/bdist.linux-x86_64/egg/tflearn/data_augmentation.py to data_augmentation.pyc
  byte-compiling build/bdist.linux-x86_64/egg/tflearn/utils.py to utils.pyc
  byte-compiling build/bdist.linux-x86_64/egg/tflearn/config.py to config.pyc
  byte-compiling build/bdist.linux-x86_64/egg/tflearn/optimizers.py to optimizers.pyc
  byte-compiling build/bdist.linux-x86_64/egg/tflearn/data_flow.py to data_flow.pyc
  byte-compiling build/bdist.linux-x86_64/egg/tflearn/callbacks.py to callbacks.pyc
  byte-compiling build/bdist.linux-x86_64/egg/tflearn/models/generator.py to generator.pyc
  byte-compiling build/bdist.linux-x86_64/egg/tflearn/models/dnn.py to dnn.pyc
  byte-compiling build/bdist.linux-x86_64/egg/tflearn/models/__init__.py to __init__.pyc
  byte-compiling build/bdist.linux-x86_64/egg/tflearn/objectives.py to objectives.pyc
  byte-compiling build/bdist.linux-x86_64/egg/tflearn/data_utils.py to data_utils.pyc
  byte-compiling build/bdist.linux-x86_64/egg/tflearn/activations.py to activations.pyc
  byte-compiling build/bdist.linux-x86_64/egg/tflearn/__init__.py to __init__.pyc
  byte-compiling build/bdist.linux-x86_64/egg/tflearn/helpers/generator.py to generator.pyc
  byte-compiling build/bdist.linux-x86_64/egg/tflearn/helpers/evaluator.py to evaluator.pyc
  byte-compiling build/bdist.linux-x86_64/egg/tflearn/helpers/regularizer.py to regularizer.pyc
  byte-compiling build/bdist.linux-x86_64/egg/tflearn/helpers/trainer.py to trainer.pyc
  byte-compiling build/bdist.linux-x86_64/egg/tflearn/helpers/__init__.py to __init__.pyc
  byte-compiling build/bdist.linux-x86_64/egg/tflearn/helpers/summarizer.py to summarizer.pyc
  byte-compiling build/bdist.linux-x86_64/egg/tflearn/layers/embedding_ops.py to embedding_ops.pyc
  byte-compiling build/bdist.linux-x86_64/egg/tflearn/layers/normalization.py to normalization.pyc
  byte-compiling build/bdist.linux-x86_64/egg/tflearn/layers/conv.py to conv.pyc
  byte-compiling build/bdist.linux-x86_64/egg/tflearn/layers/__init__.py to __init__.pyc
  byte-compiling build/bdist.linux-x86_64/egg/tflearn/layers/recurrent.py to recurrent.pyc
  byte-compiling build/bdist.linux-x86_64/egg/tflearn/layers/estimator.py to estimator.pyc
  byte-compiling build/bdist.linux-x86_64/egg/tflearn/layers/core.py to core.pyc
  byte-compiling build/bdist.linux-x86_64/egg/tflearn/layers/merge_ops.py to merge_ops.pyc
  byte-compiling build/bdist.linux-x86_64/egg/tflearn/variables.py to variables.pyc
  creating build/bdist.linux-x86_64/egg/EGG-INFO
  copying TFLearn.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
  copying TFLearn.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
  copying TFLearn.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
  copying TFLearn.egg-info/requires.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
  copying TFLearn.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
  zip_safe flag not set; analyzing archive contents...
  creating 'dist/TFLearn-0.1-py2.7.egg' and adding 'build/bdist.linux-x86_64/egg' to it
  removing 'build/bdist.linux-x86_64/egg' (and everything under it)
  Processing TFLearn-0.1-py2.7.egg
  Removing /usr/local/lib/python2.7/dist-packages/TFLearn-0.1-py2.7.egg
  Copying TFLearn-0.1-py2.7.egg to /usr/local/lib/python2.7/dist-packages
  TFLearn 0.1 is already the active version in easy-install.pth

  Installed /usr/local/lib/python2.7/dist-packages/TFLearn-0.1-py2.7.egg
  Processing dependencies for TFLearn==0.1
  Searching for Pillow
  Reading https://pypi.python.org/simple/Pillow/
  Best match: Pillow 3.2.0
  Downloading https://pypi.python.org/packages/e2/af/0a3981fffc5cd43078eb8b1057702e0dd2d5771e5aaa36cbd140e32f8473/Pillow-3.2.0.tar.gz#md5=7cfd093c11205d9e2ebe3c51dfcad510
  Processing Pillow-3.2.0.tar.gz
  Writing /tmp/easy_install-uMqe0s/Pillow-3.2.0/setup.cfg
  Running Pillow-3.2.0/setup.py -q bdist_egg --dist-dir /tmp/easy_install-uMqe0s/Pillow-3.2.0/egg-dist-tmp-PNQLRN
  warning: no files found matching '*.sh'
  no previously-included directories found matching 'docs/_static'
  warning: no previously-included files found matching '.coveragerc'
  warning: no previously-included files found matching '.editorconfig'
  warning: no previously-included files found matching '.landscape.yaml'
  warning: no previously-included files found matching 'appveyor.yml'
  warning: no previously-included files found matching 'build_children.sh'
  warning: no previously-included files found matching 'tox.ini'
  warning: no previously-included files matching '.git*' found anywhere in distribution
  warning: no previously-included files matching '*.pyc' found anywhere in distribution
  warning: no previously-included files matching '*.so' found anywhere in distribution
  Traceback (most recent call last):
    File "setup.py", line 32, in <module>
      'AI'
    File "/usr/lib/python2.7/distutils/core.py", line 151, in setup
      dist.run_commands()
    File "/usr/lib/python2.7/distutils/dist.py", line 953, in run_commands
      self.run_command(cmd)
    File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
      cmd_obj.run()
    File "/usr/local/lib/python2.7/dist-packages/setuptools/command/install.py", line 67, in run
      self.do_egg_install()
    File "/usr/local/lib/python2.7/dist-packages/setuptools/command/install.py", line 117, in do_egg_install
      cmd.run()
    File "/usr/local/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 391, in run
      self.easy_install(spec, not self.no_deps)
    File "/usr/local/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 621, in easy_install
      return self.install_item(None, spec, tmpdir, deps, True)
    File "/usr/local/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 672, in install_item
      self.process_distribution(spec, dist, deps)
    File "/usr/local/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 717, in process_distribution
      [requirement], self.local_index, self.easy_install
    File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 826, in resolve
      dist = best[req.key] = env.best_match(req, ws, installer)
    File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 1092, in best_match
      return self.obtain(req, installer)
    File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 1104, in obtain
      return installer(requirement)
    File "/usr/local/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 640, in easy_install
      return self.install_item(spec, dist.location, tmpdir, deps)
    File "/usr/local/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 670, in install_item
      dists = self.install_eggs(spec, download, tmpdir)
    File "/usr/local/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 850, in install_eggs
      return self.build_and_install(setup_script, setup_base)
    File "/usr/local/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 1078, in build_and_install
      self.run_setup(setup_script, setup_base, args)
    File "/usr/local/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 1064, in run_setup
      run_setup(setup_script, args)
    File "/usr/local/lib/python2.7/dist-packages/setuptools/sandbox.py", line 246, in run_setup
      raise
    File "/usr/lib/python2.7/contextlib.py", line 35, in __exit__
      self.gen.throw(type, value, traceback)
    File "/usr/local/lib/python2.7/dist-packages/setuptools/sandbox.py", line 195, in setup_context
      yield
    File "/usr/lib/python2.7/contextlib.py", line 35, in __exit__
      self.gen.throw(type, value, traceback)
    File "/usr/local/lib/python2.7/dist-packages/setuptools/sandbox.py", line 166, in save_modules
      saved_exc.resume()
    File "/usr/local/lib/python2.7/dist-packages/setuptools/sandbox.py", line 141, in resume
      six.reraise(type, exc, self._tb)
    File "/usr/local/lib/python2.7/dist-packages/setuptools/sandbox.py", line 154, in save_modules
      yield saved
    File "/usr/local/lib/python2.7/dist-packages/setuptools/sandbox.py", line 195, in setup_context
      yield
    File "/usr/local/lib/python2.7/dist-packages/setuptools/sandbox.py", line 243, in run_setup
      DirectorySandbox(setup_dir).run(runner)
    File "/usr/local/lib/python2.7/dist-packages/setuptools/sandbox.py", line 273, in run
      return func()
    File "/usr/local/lib/python2.7/dist-packages/setuptools/sandbox.py", line 242, in runner
      _execfile(setup_script, ns)
    File "/usr/local/lib/python2.7/dist-packages/setuptools/sandbox.py", line 46, in _execfile
      exec(code, globals, locals)
    File "/tmp/easy_install-uMqe0s/Pillow-3.2.0/setup.py", line 767, in <module>

    File "/usr/lib/python2.7/distutils/core.py", line 151, in setup
      dist.run_commands()
    File "/usr/lib/python2.7/distutils/dist.py", line 953, in run_commands
      self.run_command(cmd)
    File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
      cmd_obj.run()
    File "/usr/local/lib/python2.7/dist-packages/setuptools/command/bdist_egg.py", line 161, in run
      cmd = self.call_command('install_lib', warn_dir=0)
    File "/usr/local/lib/python2.7/dist-packages/setuptools/command/bdist_egg.py", line 147, in call_command
      self.run_command(cmdname)

    File "/usr/lib/python2.7/distutils/cmd.py", line 326, in run_command
      self.distribution.run_command(command)
    File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
      cmd_obj.run()
    File "/usr/local/lib/python2.7/dist-packages/setuptools/command/install_lib.py", line 10, in run
      self.build()
    File "/usr/lib/python2.7/distutils/command/install_lib.py", line 111, in build
      self.run_command('build_ext')
    File "/usr/lib/python2.7/distutils/cmd.py", line 326, in run_command
      self.distribution.run_command(command)
    File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
      cmd_obj.run()
    File "/usr/lib/python2.7/distutils/command/build_ext.py", line 339, in run
      self.build_extensions()
    File "/tmp/easy_install-uMqe0s/Pillow-3.2.0/setup.py", line 512, in build_extensions

  ValueError: jpeg is required unless explicitly disabled using --disable-jpeg, aborting

Ubuntu: recommend install libhdf5

I had trouble doing pip install on tflearn, until I found this recommendation for installing on Ubuntu:

sudo apt-get install libhdf5-dev

Please add to the installation instructions

Edit: on AWS GPU units, also:

sudo apt-get install cuda-ld-conf-7-5

residual network example on cifar10 not converging

Ran residual_network_cifar10.py in example/image folder.
At epoch 1 val_acc: 0.3040, after that is oscillates between 0.09xx and 0.10xx.

Another example from the same folder, network_in_network.py gives accuracy on validation set upto .81.

Data Preprocessing and Augmentation

Master branch has been updated and models can now handle real-time data preprocessing and augmentation. A 'DataFlow' class has also been created to easily handle data stream, and use some pipelining to speed-up computation when training on GPU. The 'FeedDictFlow' subclass handles a stream of feed_dict generated from any numpy array or hdf5 dataset.

Current progress:
DataFlow:

Base class
FeedDictFlow
TFRecordsFlow (Moved to v0.3)

DataPreprocessing:

Base class
Zero Center / STD norm (samplewise and featurewise)
ZCA Whitening

DataAugmentation

Docs

Add docs
Add usage example in 'Getting Started'

Contributions are welcome, if you feel that some new data augmentation or preprocessing are needed :)

Cannot Load Saved Shakespeare Model

Running the lstm_generator_shakespeare.py example from a saved model is giving me the following stacktrace:

  File "lstm_generator_shakespeare.py", line 37, in <module>
    m.load('model_shakespeare-5')
  File "/Users/adrianlyjak/dev/futz/tflearn/examples/nlp/lib/python2.7/site-packages/tflearn/models/generator.py", line 226, in load
    self.predictor = Evaluator(self.net, model=model_file)
  File "/Users/adrianlyjak/dev/futz/tflearn/examples/nlp/lib/python2.7/site-packages/tflearn/helpers/evaluator.py", line 26, in __init__
    self.graph = tensors[0].graph
  File "/Users/adrianlyjak/dev/futz/tflearn/examples/nlp/lib/python2.7/site-packages/tensorflow/python/ops/array_ops.py", line 154, in _SliceHelper
    sliced = slice(tensor, indices, sizes)
  File "/Users/adrianlyjak/dev/futz/tflearn/examples/nlp/lib/python2.7/site-packages/tensorflow/python/ops/array_ops.py", line 204, in slice
    return gen_array_ops._slice(input_, begin, size, name=name)
  File "/Users/adrianlyjak/dev/futz/tflearn/examples/nlp/lib/python2.7/site-packages/tensorflow/python/ops/gen_array_ops.py", line 908, in _slice
    name=name)
  File "/Users/adrianlyjak/dev/futz/tflearn/examples/nlp/lib/python2.7/site-packages/tensorflow/python/ops/op_def_library.py", line 655, in apply_op
    op_def=op_def)
  File "/Users/adrianlyjak/dev/futz/tflearn/examples/nlp/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2042, in create_op
    set_shapes_for_outputs(ret)
  File "/Users/adrianlyjak/dev/futz/tflearn/examples/nlp/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1528, in set_shapes_for_outputs
    shapes = shape_func(op)
  File "/Users/adrianlyjak/dev/futz/tflearn/examples/nlp/lib/python2.7/site-packages/tensorflow/python/ops/array_ops.py", line 778, in _SliceShape
    input_shape.assert_has_rank(ndims)
  File "/Users/adrianlyjak/dev/futz/tflearn/examples/nlp/lib/python2.7/site-packages/tensorflow/python/framework/tensor_shape.py", line 596, in assert_has_rank
    raise ValueError("Shape %s must have rank %d" % (self, rank))
ValueError: Shape (?, 67) must have rank 1

It seems to be coming from line 26 in helpers/evaluator.py Evaluator.__init__

 self.graph = tensors[0].graph

Here, tensors is already a a Tensor, so changing the line to self.graph = tensors.graph fixes the slice error. Is tensors ever another type?

Understanding Tensorflow/Tflearn LSTM input?

I have some trouble understanding LSTM . For simplicity lets consider the Example program.

I use the tflearn as a wrapper as it does all the initialization and other higher level stuff automatically. Thank you for that. :-)

Till line number 42 net = tflearn.input_data([None, 200]) it's pretty clear what happens. You load a data-set into variables and make it of a standard lengths in this case 200. for both the input variables and also the the 2 classes present in this case are converted to one hot vectors.

What I would like to know here is how the LSTM takes the input and across how many samples does it it predict the output ? here the n_words=20000 & net = tflearn.embedding(net, input_dim=20000, output_dim=128) What do these parameters indicate?

My goal is to replicate the Activity recognition data set in the paper.

For example I would like to input a fc6 4096 vector as input to the lstm and the idea is to take 16 of such vectors and then produce the classification result.

I think the code would look like this but I dont know how the input to the LSTM should be given.
Could any one help me understand how I input data to the LSTM.

from __future__ import division, print_function, absolute_import

import tflearn
from tflearn.data_utils import to_categorical, pad_sequences
from tflearn.datasets import imdb

train, val = something.load_data()
trainX, trainY = train #each X sample is a (16,4096) nd float64 
valX, valY = val #each Y is a one hot vector of 101 classes.

net = tflearn.input_data([None, 16,4096])
net = tflearn.embedding(net, input_dim=4096, output_dim=256)
net = tflearn.lstm(net, 256)
net = tflearn.dropout(net, 0.5)
net = tflearn.lstm(net, 256)
net = tflearn.dropout(net, 0.5)
net = tflearn.fully_connected(net, 101, activation='softmax')
net = tflearn.regression(net, optimizer='adam',
                         loss='categorical_crossentropy')

model = tflearn.DNN(net, clip_gradients=0., tensorboard_verbose=3)
model.fit(trainX, trainY, validation_set=(testX, testY), show_metric=True,
          batch_size=128,n_epoch=2,snapshot_epoch=True)

Extremely slow with hdf5 dataset, consuming all memory

My problem:

I have a huge hdf5 dataset stored in 'data.h5'. The shape of X is (2000000,40,40,3). The following code consumes all my memory. It seems the program loads all the data into memory. What is the problem?

Thank you.

My code:

Load hdf5 dataset

h5f = h5py.File('data/data.h5', 'r')
X = h5f['fre_ori_train']
Y = h5f['targets_train']
X_test = h5f['fre_ori_val'][:10000,:,:,:]
Y_test = h5f['targets_val'][:10000,:]

print(X.shape)
print(Y.shape)

Build network

network = input_data(shape=[None, 40, 40, 3], dtype=tf.float32)
network = conv_2d(network, 32, 3, activation='relu')
network = max_pool_2d(network, 2)
network = conv_2d(network, 64, 3, activation='relu')
network = conv_2d(network, 64, 3, activation='relu')
network = max_pool_2d(network, 2)
network = fully_connected(network, 512, activation='relu')
network = dropout(network, 0.5)
network = fully_connected(network, 5, activation='linear')
network = regression(network, optimizer='adam',
loss='mean_square',
learning_rate=0.001)

Training

model = tflearn.DNN(network, tensorboard_verbose=0)
model.fit(X, Y, n_epoch=50, shuffle=False, validation_set=(X_test, Y_test),
show_metric=True, batch_size=32, run_id='fre_ori_cnn')

h5f.close()

Input shape for an RNN

I've looked at the LSTM example, and other examples I could find, but I'm still unclear how to shape the data for my example. My training data set comprises a number of sequences of 30 float values; the target value for each sequence is a single float value. Given a sequence of 30 values, I would like to use the model to predict the target value.

What would you suggest for the input shape?

Installation issues

I've followed the instructions in the installation guide, but no matter what I do, when I run >>>import tflearn I get:

ValueError: numpy.dtype has the wrong size, try recompiling. Expected 88, got 96

Here is the full error for reference:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "tflearn/__init__.py", line 8, in <module>
    from . import models
  File "tflearn/models/__init__.py", line 2, in <module>
    from .dnn import DNN
  File "tflearn/models/dnn.py", line 5, in <module>
    from ..helpers.trainer import Trainer
  File "tflearn/helpers/__init__.py", line 2, in <module>
    from .evaluator import Evaluator
  File "tflearn/helpers/evaluator.py", line 6, in <module>
    from ..utils import to_list
  File "tflearn/utils.py", line 7, in <module>
    import h5py
  File "/usr/local/lib/python2.7/dist-packages/h5py-2.6.0-py2.7-linux-x86_64.egg/h5py/__init__.py", line 34, in <module>
    from ._conv import register_converters as _register_converters
  File "h5py/h5t.pxd", line 14, in init h5py._conv (/tmp/easy_install-w7IxB0/h5py-2.6.0/h5py/_conv.c:7357)
  File "h5py/numpy.pxd", line 66, in init h5py.h5t (/tmp/easy_install-w7IxB0/h5py-2.6.0/h5py/h5t.c:20344)
ValueError: numpy.dtype has the wrong size, try recompiling. Expected 88, got 96

define a customized objective function

hi there
how can I define my own loss function in tf-learn, thanks!

TFLearn v0.2

Here are some development directions for the next version of TLearn:

More docs (see #41)
Add data preprocessing and augmentation (see #66)
Data Flow with Pipelining for faster data processing ~~and TF Records support~~ (see #66) (TFRecords moved to v0.3)
Layers can accept functions as argument (c.f. activation=custom_activ_fn, loss=custom_loss_fn, etc...)
Add more exceptions (See #51 for notebook related errors)
Finish tests (+ Travis integration)
Add attention module (See below)

Any other ideas are welcome :)

Call the pre-trained model from C++

Hi Aymeric,

First of all, thank you for creating such an amazing project, I really love this API, it makes my research projects much easier, many thanks!

I'd like to call the pre-trained (and saved on the disk) model to predict new Y from C++ (that means I will load the saved model and pass X to the model in C++), I am just wondering which is the best way to do this? Thanks.

Exception when running in Jupyter Notebook

Looks like there's still some compatibility issues with Jupyter Notebooks. Running the example https://github.com/tflearn/tflearn/blob/master/examples/images/dnn.py gives: Exception: 55000 inputs feeded, but expected: 11. The example works in the same environment outside a notebook.

Full error:

Exception                                 Traceback (most recent call last)
<ipython-input-20-895c23faf0cc> in <module>()
     26 model = tflearn.DNN(net, tensorboard_verbose=3)
     27 model.fit(X, Y, n_epoch=20, validation_set=(testX, testY),
---> 28           show_metric=True, run_id="dense_model")

/home/john/Desktop/capstone/env/lib/python2.7/site-packages/tflearn/models/dnn.pyc in fit(self, X_inputs, Y_targets, n_epoch, validation_set, show_metric, batch_size, shuffle, snapshot_epoch, snapshot_step, run_id)
    151         # TODO: check memory impact for large data and multiple optimizers
    152         feed_dict = feed_dict_builder(X_inputs, Y_targets, self.inputs,
--> 153                                       self.targets)
    154         feed_dicts = [feed_dict for i in self.train_ops]
    155         val_feed_dicts = None

/home/john/Desktop/capstone/env/lib/python2.7/site-packages/tflearn/utils.pyc in feed_dict_builder(X, Y, net_inputs, net_targets)
    245                 elif len(X) != len(net_inputs):
    246                     raise Exception(str(len(X)) + " inputs feeded, "
--> 247                                     "but expected: " + str(len(net_inputs)))
    248             else:
    249                 X = [X]

Exception: 55000 inputs feeded, but expected: 11

Classification: `X` input feeded, but expected: `Y`

I'm used a derivative example of your classification to try training my own classifier. My train-set data is little small, I know (129~). But I only wants to running that.

My input/output vectors are word2vec embeddings with 200 dimensional size and dtype=float32.

The slice of code using tflearn is:

import tflearn

def get_data():
    return tuple(zip(*[(G.node[m]['vector'], G.node[s]['vector'], G.node[i]['vector'])
                        for m, s, i in zip(dataset.message, dataset.sentiment, dataset.idea)
                        if G.node[m]['vector'] is not np.NaN]))

data = get_data()

def _deep_classifier(features=200):
    net = tflearn.input_data(shape=[None, features])
    net = tflearn.fully_connected(net, 64)
    net = tflearn.dropout(net, 0.5)
    net = tflearn.fully_connected(net, 10, activation='softmax')
    net = tflearn.regression(net, optimizer='adam', loss='categorical_crossentropy')
    return tflearn.DNN(net,  tensorboard_verbose=3)

def deep_sentiment():
    global data
    model = _deep_classifier()
    X, Y, _ = data
    model.fit(X, Y, show_metric=True, run_id="dense_model")
    return model

def deep_idea():
    global data 
    model = _deep_classifier()
    X, _, Y = data
    model.fit(X, Y, validation_set=(X, Y))
    return model

sentiment_classifier = deep_sentiment()

Here as dump of my data variable: http://sprunge.us/dLeY. Are a tuple of 3 lists of word2vec embeddings (numpy arrays) with 129 examples each of them.

Trying running the could below, the traceback is always some like that:

---------------------------------------------------------------------------
Exception                                 Traceback (most recent call last)
<ipython-input-49-466468d5c3b9> in <module>()
----> 1 sentiment_classifer = deep_sentiment()

<ipython-input-48-939247752b0a> in deep_sentiment()
     25     model = _deep_classifier()
     26     X, Y, _ = data
---> 27     model.fit(X, Y, show_metric=True, run_id="dense_model")
     28     return model
     29 

/usr/local/lib/python3.4/dist-packages/tflearn/models/dnn.py in fit(self, X_inputs, Y_targets, n_epoch, validation_set, show_metric, batch_size, shuffle, snapshot_epoch, snapshot_step, run_id)
    151         # TODO: check memory impact for large data and multiple optimizers
    152         feed_dict = feed_dict_builder(X_inputs, Y_targets, self.inputs,
--> 153                                       self.targets)
    154         feed_dicts = [feed_dict for i in self.train_ops]
    155         val_feed_dicts = None

/usr/local/lib/python3.4/dist-packages/tflearn/utils.py in feed_dict_builder(X, Y, net_inputs, net_targets)
    245                 elif len(X) != len(net_inputs):
    246                     raise Exception(str(len(X)) + " inputs feeded, "
--> 247                                     "but expected: " + str(len(net_inputs)))
    248             else:
    249                 X = [X]

Exception: 129 inputs feeded, but expected: 17

I think this is really weird. I tried read in the docs the standard for input set, but I didn't found. I don't know what I missing.

By the way, always I try running this cell at Jupyter Notebook again the expected value increase! D:

How to initializer the weights by using a tensor

    self.Fea_In_C3 = tflearn.conv_2d(vgg.pool5, 256, 11,
                                     weights_init= tf.random_normal_initializer(stddev=0.0001),
                                     padding='valid',
                                     activation= 'relu',
                                     name='Fea_In_C3')

I want to initializer the weights by using a small deviation, and I use the code showing above, but it gives error.

File "/home/zluo/PycharmProjects/DeepActiveContour/deepnet/Model.py", line 25, in build_model
name='Fea_In_C3')
File "/usr/local/lib/python2.7/dist-packages/tflearn/layers/conv.py", line 64, in conv_2d
W_init = initializations.get(weights_init)()
TypeError: _initializer() takes at least 1 argument (0 given)

Uninitialized session in autoencoder.py example

On trying out tflearn for the first time, I encountered a very misleading error in the autoencoder.py example:

tflearn/examples/images/autoencoder.py

Line 46 in 1420986

encoding_model = tflearn.DNN(encoder)

a new tflearn.DNN is initialized with the encoder tensor. Unfortunately, it seems that this creates a new Session, where the weights of the tensor are not set to the values in the session where it was just trained.

This can be verified by adding the following debug statements after the predict():

print("bias in model:", model.get_weights(encoder.b))
print("bias in encoding_model:", encoding_model.get_weights(encoder.b))

The biases in the model are non-zero, whereas the biases in the encoding_model are zeroed.

I'd suggest replacing the DNN by an Evaluator sharing the session of the model, maybe:

ev = tflearn.Evaluator([encoder], session=model.session)
print(ev.predict({model.inputs[0]: [X[0]]}))

I am not 100% sure this is correct, since I just started out using this library.

Python3 UnicodeDecodeError in cifar10.load_data()

Running python3 convnet_cifar10.py gives this error:

File "/usr/local/lib/python3.4/dist-packages/tflearn/datasets/cifar10.py", line 55, in load_batch
    d = pickle.load(f)
UnicodeDecodeError: 'ascii' codec can't decode byte 0x8b in position 6: ordinal not in range(128)

Thanks to SO I got the tip to add an encoding that resolves the problem:
d = pickle.load(f, encoding='latin1')

Maybe wrong when computing batch_normalization?

Hi,
in function batch_normalization, there has a line which update the mean and var or not according to in training or evaluating mode, but seems like both return the same variables.
... mean, var = tf.python.control_flow_ops.cond( is_training, update_mean_var, lambda: (ema_mean, ema_var)) ...

AlexNet 17 flowers - Evaluating not possible / Easier evaluating method?

Hi!
Unfortunately I got stuck in the next problem. Now that the AlexNet example makes a good training process and I get about 97% accuracy after a while, I wanted to evaluate what was learned. Ideally I need a procedure, that is very resource-inexpensive, because I want to train on my GPU but apply what was learned on a less powerful machine. Loading the model shall take a longer time, but evaluating image by image should take very short there.

I was now trying your example described in #39 (https://github.com/tflearn/tflearn/blob/master/examples/basics/weights_persistence.py). My modified code therefore is very simple:

network = ...

# Training
model = tflearn.DNN(network, checkpoint_path='model_alexnet',
                max_checkpoints=1, tensorboard_verbose=0)
model.fit(X, Y, n_epoch=3, validation_set=0.1, shuffle=True,
      show_metric=True, batch_size=32, snapshot_step=400,
      snapshot_epoch=False, run_id='alexnet_oxflowers17')

# Load a model from auto-generated checkpoint
model.load("model_alexnet-800")

# Resume training
model.fit(X, Y, n_epoch=1,
      validation_set=(testX, testY),
      show_metric=True,
      snapshot_epoch=True,
      run_id='model_and_weights')`

But I get a lot (!) of messages like:

I tensorflow/core/common_runtime/bfc_allocator.cc:635] Bin (268435456): Total Chunks: 0, 
Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 
0B client-requested in use in bin.

And in the end:

tensorflow.python.framework.errors.ResourceExhaustedError: OOM when allocating tensor with 
shape[12544,4096]

What am I doing wrong here? Or is there possibly a much simpler method to evaluate single images resource-inexpensively?

Thanks in advance,
Arno

Add tests to TFLearn

In the future, we should also have some tests, to be sure that new pull requests are fully compatible.
This thread will stay open until it is done.

tflearn / tflearn Goto Github PK

tflearn's Introduction

TFLearn: Deep learning library featuring a higher-level API for TensorFlow.

Overview

Compatibility

Installation

Getting Started

Examples

Documentation

Model Visualization

Contributions

License

tflearn's People

Contributors

Stargazers

Watchers

Forkers

tflearn's Issues

My problem:

My code:

Load hdf5 dataset

Build network

Training

Recommend Projects

Recommend Topics

Recommend Org