Y [batch, 20] => Y [batch, 10]

• Thanks Martin for your efforts very much .
• I'm so curious about how to create this type of visualization (with Accuracy Scores and Cost Function metrics) for different/generic networks? I think that is so cool!
• If there is a tutorial ,I will be much honored.
  Thanks

Hi, I need to download the slides? How can I download them?

there is no error but my accuracy is always 0, How should I solve it ?

First of all, thank you very much for this excellente tutorial.

Unfortunately I always get empty plots when I run the code. I've updated matplotlib, but nothing changed.
I use Python 3.5 and Spyder3.1.
Do you have any idea what can be the problem?

Thanks in advance!

I run this file on my labtop: windows 10 + Anaconda python3.5.2

When run mnist_3.0_convolutional.py , the Anaconda will break down.
When I drag the file into the cmd window, it shows:
FileNotFoundError: [Errno 2] No such file or directory: 'tensorflowvisu.mplstyle'
OSError: 'tensorflowvisu.mplstyle' not found in the style library and input is not a valid URL or path. See style.available for list of available styles.

If you can tell me what's wrong wth it? And how can I deal with it?
Thank you very much!

Most Ubuntu installations still come with Python 3.4 and Matplotlib 1.3
The visualisation code has a problem in these conditions as reported here b010098

Hi, I try your method in issue 33, but I cannot download the slide for recurrent network.

I started the training . it ran ok for a short time. When train loop reached to epoch 4，It hang! and never recover.

the snapshot is below:

The environment is as same as issue 10。

I met a problem when running the training. My environment is below:

ubuntu 16.04.1
git version 2.7.4
python3-matplotlib  :1.5.1-1ubuntu1
python3-pip: 8.1.1-2ubuntu0.4

$ pip3 install --upgrade tensorflow
Collecting tensorflow
  Downloading tensorflow-0.12.1-cp35-cp35m-manylinux1_x86_64.whl (43.1MB)
    100% |████████████████████████████████| 43.1MB 46kB/s
Collecting numpy>=1.11.0 (from tensorflow)
  Downloading numpy-1.12.0-cp35-cp35m-manylinux1_x86_64.whl (16.8MB)
    100% |████████████████████████████████| 16.8MB 111kB/s
Collecting six>=1.10.0 (from tensorflow)
  Downloading six-1.10.0-py2.py3-none-any.whl
Collecting wheel>=0.26 (from tensorflow)
  Downloading wheel-0.29.0-py2.py3-none-any.whl (66kB)
    100% |████████████████████████████████| 71kB 1.3MB/s
Collecting protobuf>=3.1.0 (from tensorflow)
  Downloading protobuf-3.1.0.post1-py2.py3-none-any.whl (347kB)
    100% |████████████████████████████████| 348kB 1.1MB/s
Collecting setuptools (from protobuf>=3.1.0->tensorflow)
  Downloading setuptools-33.1.1-py2.py3-none-any.whl (472kB)
    100% |████████████████████████████████| 481kB 1.1MB/s
Installing collected packages: numpy, six, wheel, setuptools, protobuf, tensorflow
Successfully installed numpy-1.11.0 protobuf setuptools-20.7.0 six-1.10.0 tensorflow wheel-0.29.0

the latest commit of my local copy for this tutorial is “commit a9eb2bf”.

The error log is ：

$ python3 mnist_1.0_softmax.py
/usr/lib/python3/dist-packages/matplotlib/font_manager.py:273: UserWarning: Matplotlib is building the font cache using fc-list. This may take a moment.
  warnings.warn('Matplotlib is building the font cache using fc-list. This may take a moment.')
/usr/lib/python3/dist-packages/matplotlib/font_manager.py:273: UserWarning: Matplotlib is building the font cache using fc-list. This may take a moment.
  warnings.warn('Matplotlib is building the font cache using fc-list. This may take a moment.')


Successfully downloaded train-images-idx3-ubyte.gz 9912422 bytes.
Extracting data/train-images-idx3-ubyte.gz
Successfully downloaded train-labels-idx1-ubyte.gz 28881 bytes.
Extracting data/train-labels-idx1-ubyte.gz
Successfully downloaded t10k-images-idx3-ubyte.gz 1648877 bytes.
Extracting data/t10k-images-idx3-ubyte.gz
Successfully downloaded t10k-labels-idx1-ubyte.gz 4542 bytes.
Extracting data/t10k-labels-idx1-ubyte.gz
Traceback (most recent call last):
  File "mnist_1.0_softmax.py", line 80, in <module>
    datavis = tensorflowvisu.MnistDataVis()
  File "/home/tonybai/test/tensorflow/tensorflow-mnist-tutorial/tensorflowvisu.py", line 166, in __init__
    self._color4 = self.__get_histogram_cyclecolor(histogram4colornum)
  File "/home/tonybai/test/tensorflow/tensorflow-mnist-tutorial/tensorflowvisu.py", line 160, in __get_histogram_cyclecolor
    colors = clist.by_key()['color']
AttributeError: 'Cycler' object has no attribute 'by_key'

The optional argument validation_size is not supported by Tensorflow 0.10.
mnist = read_data_sets("data", one_hot=True, reshape=False, validation_size=0)

Just in case someone wants to try this on older version.

First of all, thanks for this helpful tutorial.
I tried to run minst_1.0_softmax.py on multi-threads in a CPU by changing the code to:

# placehoder and variable definations
# def training_step function
with tf.Session() as sess:
    with tf.device("/cpu:0"):
        sess.run(init)
        datavis.animate(...)

But it didn't work and the CPU utilization was still around 25%.
Could you please tell me how to do?
Thanks!

Hi Martin,
In mnist_1.0_softmax.py you have these lines to describe and calculate cross-entropy:

log takes the log of each element, * multiplies the tensors element by element
reduce_mean will add all the components in the tensor
so here we end up with the total cross-entropy for all images in the batch
cross_entropy = -tf.reduce_mean(Y_ * tf.log(Y)) * 1000.0 # normalized for batches of 100 images,
*10 because "mean" included an unwanted division by 10

Could you please explain a bit more what are these numbers mean?

1. Why are we multiplying the final value by 1000.0 if the batch is of size 100? I thought maybe it was because of 1000 iterations, but in the code we have 2000 iterations.
2. "*10 because "mean" included an unwanted division by 10" - what does this phrase relate to, I don't see any multiplication by 10.
   Thanks

I was trying run this on a Ubuntu machine on Google cloud and it got stuck at below issue:

Tensorflow version 1.2.1
Extracting data/train-images-idx3-ubyte.gz
Extracting data/train-labels-idx1-ubyte.gz
Extracting data/t10k-images-idx3-ubyte.gz
Extracting data/t10k-labels-idx1-ubyte.gz
Traceback (most recent call last):
File "mnist_1.0_softmax.py", line 85, in
datavis = tensorflowvisu.MnistDataVis()
File "/home/vcohen/tensorflow-mnist-tutorial/tensorflowvisu.py", line 168, in init
fig = plt.figure(figsize=(19.20,10.80), dpi=dpi)
File "/usr/local/lib/python3.4/dist-packages/matplotlib/pyplot.py", line 535, in figure
**kwargs)
File "/usr/local/lib/python3.4/dist-packages/matplotlib/backends/backend_tkagg.py", line 81, in new_figure_manager
return new_figure_manager_given_figure(num, figure)
File "/usr/local/lib/python3.4/dist-packages/matplotlib/backends/backend_tkagg.py", line 89, in new_figure_manager_given_figure
window = Tk.Tk()
File "/usr/lib/python3.4/tkinter/init.py", line 1854, in init
self.tk = _tkinter.create(screenName, baseName, className, interactive, wantobjects, useTk, sync, use)
_tkinter.TclError: no display name and no $DISPLAY environment variable

Could you make a step by step guide/video on how to install Tensorflow. There is always a new problem that pops out...

After training and testing it makes common sense to add prediction code.
Prediction is really helpful to understand how one can use the results.
Thanks you.

I installed tensorflow ( python3 ) successfully
and run test ( hello tensorflow ) OK.
but when I run

python3 mnist_1.0_softmax.py

got the error:

Traceback (most recent call last):
File "mnist_1.0_softmax.py", line 17, in
import tensorflowvisu
File "/home/minh/git/tensorflow-mnist-tutorial/tensorflowvisu.py", line 19, in
plt.style.use(["ggplot", "tensorflowvisu.mplstyle"])
AttributeError: 'module' object has no attribute 'style'

Mac
Python 3.5.2

Hi! First of all thanks for this tutorial.

I am trying to run the program but a timed out exception stops the program execution. I am using a Mac, so I followed the corresponding steps stated in the INSTALL.txt file. To test the installation, I ran python3 mnist_1.0_softmax.py and the following error raised.

$ python3 mnist_1.0_softmax.py
Traceback (most recent call last):
  File "/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/lib/python3.5/urllib/request.py", line 1254, in do_open
    h.request(req.get_method(), req.selector, req.data, headers)
  File "/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py", line 1106, in request
    self._send_request(method, url, body, headers)
  File "/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py", line 1151, in _send_request
    self.endheaders(body)
  File "/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py", line 1102, in endheaders
    self._send_output(message_body)
  File "/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py", line 934, in _send_output
    self.send(msg)
  File "/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py", line 877, in send
    self.connect()
  File "/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py", line 849, in connect
    (self.host,self.port), self.timeout, self.source_address)
  File "/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/lib/python3.5/socket.py", line 711, in create_connection
    raise err
  File "/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/lib/python3.5/socket.py", line 702, in create_connection
    sock.connect(sa)
TimeoutError: [Errno 60] Operation timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "mnist_1.0_softmax.py", line 39, in <module>
    mnist = read_data_sets("data", one_hot=True, reshape=False, validation_size=0)
  File "/Users/lucasrodes/.env/dl/lib/python3.5/site-packages/tensorflow/contrib/learn/python/learn/datasets/mnist.py", line 211, in read_data_sets
    SOURCE_URL + TRAIN_IMAGES)
  File "/Users/lucasrodes/.env/dl/lib/python3.5/site-packages/tensorflow/contrib/learn/python/learn/datasets/base.py", line 208, in maybe_download
    temp_file_name, _ = urlretrieve_with_retry(source_url)
  File "/Users/lucasrodes/.env/dl/lib/python3.5/site-packages/tensorflow/contrib/learn/python/learn/datasets/base.py", line 165, in wrapped_fn
    return fn(*args, **kwargs)
  File "/Users/lucasrodes/.env/dl/lib/python3.5/site-packages/tensorflow/contrib/learn/python/learn/datasets/base.py", line 190, in urlretrieve_with_retry
    return urllib.request.urlretrieve(url, filename)
  File "/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/lib/python3.5/urllib/request.py", line 188, in urlretrieve
    with contextlib.closing(urlopen(url, data)) as fp:
  File "/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/lib/python3.5/urllib/request.py", line 163, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/lib/python3.5/urllib/request.py", line 466, in open
    response = self._open(req, data)
  File "/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/lib/python3.5/urllib/request.py", line 484, in _open
    '_open', req)
  File "/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/lib/python3.5/urllib/request.py", line 444, in _call_chain
    result = func(*args)
  File "/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/lib/python3.5/urllib/request.py", line 1282, in http_open
    return self.do_open(http.client.HTTPConnection, req)
  File "/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/lib/python3.5/urllib/request.py", line 1256, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 60] Operation timed out>

I am using a virtual environment using Python 3.5.2. Do you have any idea why this is happening?

Thanks in advance!

I came across this error when i tried this the second time
ymd@ymd:~/tensorflow-mnist-tutorial$ python mnist_1.0_softmax.py
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcublas.so.8.0 locally
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcudnn.so.5 locally
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcufft.so.8.0 locally
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcurand.so.8.0 locally
Traceback (most recent call last):
File "mnist_1.0_softmax.py", line 39, in
mnist = read_data_sets("data", one_hot=True, reshape=False, validation_size=0)
File "/home/ymd/anaconda2/lib/python2.7/site-packages/tensorflow/contrib/learn/python/learn/datasets/mnist.py", line 211, in read_data_sets
SOURCE_URL + TRAIN_IMAGES)
File "/home/ymd/anaconda2/lib/python2.7/site-packages/tensorflow/contrib/learn/python/learn/datasets/base.py", line 208, in maybe_download
temp_file_name, _ = urlretrieve_with_retry(source_url)
File "/home/ymd/anaconda2/lib/python2.7/site-packages/tensorflow/contrib/learn/python/learn/datasets/base.py", line 165, in wrapped_fn
return fn(*args, **kwargs)
File "/home/ymd/anaconda2/lib/python2.7/site-packages/tensorflow/contrib/learn/python/learn/datasets/base.py", line 190, in urlretrieve_with_retry
return urllib.request.urlretrieve(url, filename)
File "/home/ymd/anaconda2/lib/python2.7/urllib.py", line 98, in urlretrieve
return opener.retrieve(url, filename, reporthook, data)
File "/home/ymd/anaconda2/lib/python2.7/urllib.py", line 245, in retrieve
fp = self.open(url, data)
File "/home/ymd/anaconda2/lib/python2.7/urllib.py", line 213, in open
return getattr(self, name)(url)
File "/home/ymd/anaconda2/lib/python2.7/urllib.py", line 350, in open_http
h.endheaders(data)
File "/home/ymd/anaconda2/lib/python2.7/httplib.py", line 1038, in endheaders
self._send_output(message_body)
File "/home/ymd/anaconda2/lib/python2.7/httplib.py", line 882, in _send_output
self.send(msg)
File "/home/ymd/anaconda2/lib/python2.7/httplib.py", line 844, in send
self.connect()
File "/home/ymd/anaconda2/lib/python2.7/httplib.py", line 821, in connect
self.timeout, self.source_address)
File "/home/ymd/anaconda2/lib/python2.7/socket.py", line 575, in create_connection
raise err
IOError: [Errno socket error] [Errno 110] Connection timed out

i'm a fresh man on python and tensorflow,and i will be so pleased if you could tell me how to make it right.

after installed successfully and run the first example

python3 mnist_1.0_softmax.py

Traceback (most recent call last):
File "/usr/local/lib/python3.4/dist-packages/matplotlib/backends/backend_gtk3agg.py", line 69, in on_draw_event
buf, cairo.FORMAT_ARGB32, width, height)
NotImplementedError: Surface.create_for_data: Not Implemented yet.

As I don't have any GPU available, I decided to try the examples on AWS. They are running very slow on AWS 'p2-xlarge' instance. Much slower than on the video and even slower than on my desktop which doesn't have any GPU (2-3 slower--my visual estimation)

My config: instance p2-xlarge Ubuntu 16.04.2, tensorflow 1.2, python3, CUDA 8.0, cudaDNN 6 (installed as .deb from NVIDIA)

What I have tested so far:

1. CPU-GPU Tensorflow test from here: http://learningtensorflow.com/lesson10/ shows performance improvement of 11x when running matrix multiplication on GPU
2. When I am running mnist_1.0 example and run nvidia-smi command in another window, it shows that the GPU is busy and it's running that same python process ID that is running the example.
3. When the example starts it shows that it has found the GPU and is using it:

2017-06-18 18:35:07.317169: I tensorflow/core/common_runtime/gpu/gpu_device.cc:940] Found device 0 with properties: 
name: Tesla K80
major: 3 minor: 7 memoryClockRate (GHz) 0.8235
pciBusID 0000:00:1e.0
Total memory: 11.17GiB
Free memory: 11.11GiB
2017-06-18 18:35:07.317192: I tensorflow/core/common_runtime/gpu/gpu_device.cc:961] DMA: 0 
2017-06-18 18:35:07.317200: I tensorflow/core/common_runtime/gpu/gpu_device.cc:971] 0:   Y 
2017-06-18 18:35:07.317209: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1030] Creating TensorFlow device (/gpu:0) -> (device: 0, name: Tesla K80, pci bus id: 0000:00:1e.0)

All signs confirm that Tensorflow is using the GPU, but why it's so slow? It seems I'm missing something, but I can't find it.
Thanks,
Pavel

I am getting this error message on Mac OSX when following the installation instructions:

from matplotlib.backends import _macosx
RuntimeError: Python is not installed as a framework. The Mac OS X backend will not be able to function correctly if Python is not installed as a framework. See the Python documentation for more information on installing Python as a framework on Mac OS X. Please either reinstall Python as a framework, or try one of the other backends. If you are using (Ana)Conda please install python.app and replace the use of 'python' with 'pythonw'. See 'Working with Matplotlib on OSX' in the Matplotlib FAQ for more information.

I am getting this error, Here is a list of my pip libraries

I have installed tensorflow ver 1.2.0rc2
numpy 1.13.0 with all the dependencies

I tried to run it from Anaconda Shell it gave me the same error. I am new to this sorry.

Failed with the error:

'pandas' has no attribute 'computation'

Had to downgrade to pandas 0.20.0 like that:

pip install 'pandas==0.20.' --force-reinstall

Hope that helps someone

• Env:

  • Python 3.5.3
  • Linux Mint 18 Cinnamon 64-bit
  • matplotlib 1.5.0
  • tensorflow 1.0.0

• Installation:

I have installed both matplotlib and tensorflow with pip.

• The bug:

$ python mnist_1.0_softmax.py
Extracting data/train-images-idx3-ubyte.gz
Extracting data/train-labels-idx1-ubyte.gz
Extracting data/t10k-images-idx3-ubyte.gz
Extracting data/t10k-labels-idx1-ubyte.gz
Traceback (most recent call last):
  File "mnist_1.0_softmax.py", line 78, in <module>
    I = tensorflowvisu.tf_format_mnist_images(X, Y, Y_)  # assembles 10x10 images by default
  File "/home/gabriel/Work/tensorflow-mnist-tutorial/tensorflowvisu.py", line 42, in tf_format_mnist_images
    everything_incorrect_first = tf.concat(0, [incorrectly_recognised_indices, correctly_recognised_indices]) # images reordered with indeces of unrecognised images first
  File "/home/gabriel/.pyenv/versions/3.5.3/lib/python3.5/site-packages/tensorflow/python/ops/array_ops.py", line 1047, in concat
    dtype=dtypes.int32).get_shape(
  File "/home/gabriel/.pyenv/versions/3.5.3/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 651, in convert_to_tensor
    as_ref=False)
  File "/home/gabriel/.pyenv/versions/3.5.3/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 716, in internal_convert_to_tensor
    ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
  File "/home/gabriel/.pyenv/versions/3.5.3/lib/python3.5/site-packages/tensorflow/python/framework/constant_op.py", line 176, in _constant_tensor_conversion_function
    return constant(v, dtype=dtype, name=name)
  File "/home/gabriel/.pyenv/versions/3.5.3/lib/python3.5/site-packages/tensorflow/python/framework/constant_op.py", line 165, in constant
    tensor_util.make_tensor_proto(value, dtype=dtype, shape=shape, verify_shape=verify_shape))
  File "/home/gabriel/.pyenv/versions/3.5.3/lib/python3.5/site-packages/tensorflow/python/framework/tensor_util.py", line 367, in make_tensor_proto
    _AssertCompatible(values, dtype)
  File "/home/gabriel/.pyenv/versions/3.5.3/lib/python3.5/site-packages/tensorflow/python/framework/tensor_util.py", line 302, in _AssertCompatible
    (dtype.name, repr(mismatch), type(mismatch).__name__))
TypeError: Expected int32, got list containing Tensors of type '_Message' instead.

• How did this occurred:

I just cloned the repo, installed the packages and tried to run the very first file in the codelab instructions.

fix division issue in learning rate decay computation on Python 2

Hello,
Thanks for your tutorial, it helps me a lot to clarify the concept and usage of batch norm in TF. However, as you said,

In the present tf.layers API (TF1.3), there is no one-line syntax for a dense layer with batch norm and relu.

Have you ever considered tf.contrib.layers.batch_norm? It seems that this API can do all thing as a whole. But I am puzzled that there are not many people using this API...

InvalidArgumentError: You must feed a value for placeholder tensor 'Placeholder_3' with dtype float
[[Node: Placeholder_3 = Placeholderdtype=DT_FLOAT, shape=, _device="/job:localhost/replica:0/task:0/cpu:0"]]

Great tutorial. Would love to use this in a class. Could not figure out how to run this within a Jupyter notebook so that the visualization works. Alternatively, would you have some instructions on how to run this in a Docker container on Mac/Windows so that the Display attaches properly?

I've just used pip3 to upgrade tensorflow on my windows computer. The tensorflow has been download successfully, but something is wrong when I try importing tensorflow in python. My python's version is 3.5 and is 64-bits, and my pip's version is 8.1.1.
The error report is as below.


I really can't figure out what happened. Can anyone help me? Thanks.

Could you make a step by step guide/video on how to install Tensorflow and use it in a ide. There is always a new problem that pops out... Several of my friends have problems here and its super frustrating.

cross_entropy = tf.nn.softmax_cross_entropy_with_logits(logits=Ylogits, labels=Y_)
cross_entropy = tf.reduce_mean(cross_entropy)*100

in the second line, why there have to * 100? Anyone who can tell me that? Thanks.

Demo is successfully working when using local but when I try to execute remotely (to take advantage of GPU)

tf.Session("grpc://HOSTNAME:2222")

I have the following error when running mnist_2.0_five_layers_sigmoid.py

Caused by op 'Variable_1/Assign', defined at:
  File "mnist_2.0_five_layers_sigmoid.py", line 51, in <module>
    B1 = tf.Variable(tf.zeros([L]))
  File "/Volumes/Users/<USERNAME>/.tensorflow/lib/python3.6/site-packages/tensorflow/python/ops/variables.py", line 226, in __init__
    expected_shape=expected_shape)
  File "/Volumes/Users/<USERNAME>/.tensorflow/lib/python3.6/site-packages/tensorflow/python/ops/variables.py", line 334, in _init_from_args
    validate_shape=validate_shape).op
  File "/Volumes/Users/<USERNAME>/.tensorflow/lib/python3.6/site-packages/tensorflow/python/ops/gen_state_ops.py", line 47, in assign
    use_locking=use_locking, name=name)
  File "/Volumes/Users/<USERNAME>/.tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 763, in apply_op
    op_def=op_def)
  File "/Volumes/Users/<USERNAME>/.tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 2395, in create_op
    original_op=self._default_original_op, op_def=op_def)
  File "/Volumes/Users/<USERNAME>/.tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1264, in __init__
    self._traceback = _extract_stack()

InvalidArgumentError (see above for traceback): Assign requires shapes of both tensors to match. lhs shape= [10] rhs shape= [200]
	 [[Node: Variable_1/Assign = Assign[T=DT_FLOAT, _class=["loc:@Variable_1"], use_locking=true, validate_shape=true, _device="/job:worker/replica:0/task:0/gpu:0"](Variable_1, zeros)]]

Update I fixed InvalidArgumentError from mnist_1.0_softmax.py by upgrading server python version tensorflow/tensorflow:latest-gpu -> tensorflow/tensorflow:latest-gpu-py3

Hello, I have the question related to the compatible_convolutional_noise_shape function which is used in the dropout in the convolution layer. The problem is if I create weights as below and use it as input of the function

W1 = tf.Variable(tf.truncated_normal([3, 3, 3, 64], mean = 0.0, stddev = 0.1), name = 'W1')

The output will be [3 1 1 64] which is not the same shape as W1 that is [3 3 3 64]. The question is why we create the noise shape that is not equal to the shape of input of the function?

I am working with Windows 7/64, Phyton 3.6.0 Anaconda 4.3.0 (64bit), TF1.0.
I have just cloned the updated example from Github (fresh code from today morning).
I tried to run the example mnist_1.0_softmax.py . The code has already the change +init = tf.global_variables_initializer()) from the commit.

It get the error, the same as a few people already mentioned :
TypeError: Expected int32, got list containing Tensors of type '_Message' instead.

The listing is shown here:

C:\Users\Public\Documents\Python Scripts\tensorflow-mnist-tutorial-master>python mnist_1.0_softmax.py
I d:\build\tensorflow\tensorflow_gpu-r0.12\tensorflow\stream_executor\dso_loader.cc:128] successfully opened CUDA librar
y cublas64_80.dll locally
I d:\build\tensorflow\tensorflow_gpu-r0.12\tensorflow\stream_executor\dso_loader.cc:128] successfully opened CUDA librar
y cudnn64_5.dll locally
I d:\build\tensorflow\tensorflow_gpu-r0.12\tensorflow\stream_executor\dso_loader.cc:128] successfully opened CUDA librar
y cufft64_80.dll locally
I d:\build\tensorflow\tensorflow_gpu-r0.12\tensorflow\stream_executor\dso_loader.cc:128] successfully opened CUDA librar
y nvcuda.dll locally
I d:\build\tensorflow\tensorflow_gpu-r0.12\tensorflow\stream_executor\dso_loader.cc:128] successfully opened CUDA librar
y curand64_80.dll locally
Extracting data\train-images-idx3-ubyte.gz
Extracting data\train-labels-idx1-ubyte.gz
Extracting data\t10k-images-idx3-ubyte.gz
Extracting data\t10k-labels-idx1-ubyte.gz
Traceback (most recent call last):
File "mnist_1.0_softmax.py", line 78, in
I = tensorflowvisu.tf_format_mnist_images(X, Y, Y_) # assembles 10x10 images by default
File "C:\Users\Public\Documents\Python Scripts\tensorflow-mnist-tutorial-master\tensorflowvisu.py", line 42, in tf_for
mat_mnist_images
everything_incorrect_first = tf.concat([incorrectly_recognised_indices, correctly_recognised_indices], 0) # images r
eordered with indeces of unrecognised images first
File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\ops\array_ops.py", line 1075, in concat
dtype=dtypes.int32).get_shape(
File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\framework\ops.py", line 669, in convert_to_tensor
ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\framework\constant_op.py", line 176, in _constant_t
ensor_conversion_function
return constant(v, dtype=dtype, name=name)
File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\framework\constant_op.py", line 165, in constant
tensor_util.make_tensor_proto(value, dtype=dtype, shape=shape, verify_shape=verify_shape))
File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\framework\tensor_util.py", line 367, in make_tensor
_proto
_AssertCompatible(values, dtype)
File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\framework\tensor_util.py", line 302, in _AssertComp
atible
(dtype.name, repr(mismatch), type(mismatch).name))
TypeError: Expected int32, got list containing Tensors of type '_Message' instead.

I have an error when I run mnist_1.0_softmax.py with tensorflow 1.0.

$ python3 mnist_1.0_softmax.py
Successfully downloaded train-images-idx3-ubyte.gz 9912422 bytes.
Extracting data/train-images-idx3-ubyte.gz
Successfully downloaded train-labels-idx1-ubyte.gz 28881 bytes.
Extracting data/train-labels-idx1-ubyte.gz
Successfully downloaded t10k-images-idx3-ubyte.gz 1648877 bytes.
Extracting data/t10k-images-idx3-ubyte.gz
Successfully downloaded t10k-labels-idx1-ubyte.gz 4542 bytes.
Extracting data/t10k-labels-idx1-ubyte.gz
Traceback (most recent call last):
  File "mnist_1.0_softmax.py", line 78, in <module>
    I = tensorflowvisu.tf_format_mnist_images(X, Y, Y_)  # assembles 10x10 images by default
  File "/Users/guillaumegirou/dev/tensorflow/tensorflow-mnist-tutorial/tensorflowvisu.py", line 42, in tf_format_mnist_images
    everything_incorrect_first = tf.concat(0, [incorrectly_recognised_indices, correctly_recognised_indices]) # images reordered with indeces of unrecognised images first
  File "/usr/local/lib/python3.6/site-packages/tensorflow/python/ops/array_ops.py", line 1047, in concat
    dtype=dtypes.int32).get_shape(
  File "/usr/local/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 651, in convert_to_tensor
    as_ref=False)
  File "/usr/local/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 716, in internal_convert_to_tensor
    ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
  File "/usr/local/lib/python3.6/site-packages/tensorflow/python/framework/constant_op.py", line 176, in _constant_tensor_conversion_function
    return constant(v, dtype=dtype, name=name)
  File "/usr/local/lib/python3.6/site-packages/tensorflow/python/framework/constant_op.py", line 165, in constant
    tensor_util.make_tensor_proto(value, dtype=dtype, shape=shape, verify_shape=verify_shape))
  File "/usr/local/lib/python3.6/site-packages/tensorflow/python/framework/tensor_util.py", line 367, in make_tensor_proto
    _AssertCompatible(values, dtype)
  File "/usr/local/lib/python3.6/site-packages/tensorflow/python/framework/tensor_util.py", line 302, in _AssertCompatible
    (dtype.name, repr(mismatch), type(mismatch).__name__))
TypeError: Expected int32, got list containing Tensors of type '_Message' instead.

I needed to rollback to tensorflow 0.12.1

pip3 install 'tensorflow==0.12.1' --force-reinstall

Hi!
In example mnist_4.2_batchnorm_convolutional.py we have:

    # compute training values for visualisation
    if update_train_data:
        a, c, im, ca, da = sess.run([accuracy, cross_entropy, I, conv_activations, dense_activations], {X: batch_X, Y_: batch_Y, tst: False, pkeep: 1.0, pkeep_conv: 1.0})
        print(str(i) + ": accuracy:" + str(a) + " loss: " + str(c) + " (lr:" + str(learning_rate) + ")")
        datavis.append_training_curves_data(i, a, c)
        datavis.update_image1(im)
datavis.append_data_histograms(i, ca, da)

But shouldn't tst: False be actually True there?

hello,
When I launch: sudo -H pip3 install --upgrade https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.9.0rc0-cp35-cp35m-linux_x86_64.whl
I get on Ubuntu 64bit:
tensorflow-0.9.0rc0-cp35-cp35m-linux_x86_64.whl is not a supported wheel on this platform.
Thank you

Windows batch file for installing TensorFlow and Python 3.5 on Windows. The issue is that as of this date, TensorFlow is not updated to Python 3.6+ and will not install. Additionally, many systems have an incompatible version of Python. See REM comments for assumptions.

REM download Anaconda3-4.2.0-Windows-x86_64.exe (contains python 3.5) from https://repo.continuum.io/archive/index.html
REM Assumes download is in %USERPROFILE%\Downloads
%USERPROFILE%\Downloads\Anaconda3-4.2.0-Windows-x86_64.exe

REM change path to use Anaconda3 (python 3.5)
PATH %USERPROFILE%\Anaconda3;%USERPROFILE%\Anaconda3\Scripts;%USERPROFILE%\Anaconda3\Library\bin;%PATH%

REM update pip to 9.0 or later (mandatory)
python -m pip install --upgrade pip

REM tell conda where to load tensorflow
conda config --add channels conda-forge

REM elevate command (mandatory) and install tensorflow - use explicit path to conda "%USERPROFILE%\Anaconda3\scripts\conda install tensorflow"
powershell.exe -Command start-process -verb runas cmd {/K "%USERPROFILE%\Anaconda3\scripts\conda install tensorflow"}

`

I'm new to TensoFlow, and I really appreciated your tutorial. I'm pretty sure running the MNIST example on a GPU wouldn't really speed up the overall process, but I'm curious anyway: How would you convert the mnist_3.0_convolutional.py example to run on a GPU? Is there a flag to set somewhere or do you have to change/add some method calls or???

Much appreciated!

Hi, thank you for very nice tutorials!!

I get the following when run the code:
runfile('C:/Users/natlun/Documents/Python Scripts/Recognition/tensorflow-mnist-tutorial-master/mnist_4.2_batchnorm_convolutional.py', wdir='C:/Users/natlun/Documents/Python Scripts/Recognition/tensorflow-mnist-tutorial-master')
Reloaded modules: tensorflowvisu_digits, tensorflowvisu
Extracting data\train-images-idx3-ubyte.gz
Extracting data\train-labels-idx1-ubyte.gz
Extracting data\t10k-images-idx3-ubyte.gz
Extracting data\t10k-labels-idx1-ubyte.gz


max test accuracy: 0

How to fix that issue??

it's strange that the first time i tried your code that was all right but today i tried again the max test accuracy is 0.
both time i didn't change anything in your code ,and i used python2.7 ,and there was no error reported.
looking for help anxiously,thanks!

Ubuntu 16.04
Python 3.5.2
Tensorflow 0.12.1 rc

Hello, I worked through Google Lab and on step 2 I got error:

sundisktop@sun:~/tensorflow-mnist-tutorial$ python3 mnist_1.0_softmax.py
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcublas.so locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcudnn.so locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcufft.so locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcurand.so locally
Successfully downloaded train-images-idx3-ubyte.gz 9912422 bytes.
Extracting data/train-images-idx3-ubyte.gz
Successfully downloaded train-labels-idx1-ubyte.gz 28881 bytes.
Extracting data/train-labels-idx1-ubyte.gz
Successfully downloaded t10k-images-idx3-ubyte.gz 1648877 bytes.
Extracting data/t10k-images-idx3-ubyte.gz
Successfully downloaded t10k-labels-idx1-ubyte.gz 4542 bytes.
Extracting data/t10k-labels-idx1-ubyte.gz
Traceback (most recent call last):
File "mnist_1.0_softmax.py", line 78, in
I = tensorflowvisu.tf_format_mnist_images(X, Y, Y_) # assembles 10x10 images by default
File "/home/sundisktop/tensorflow-mnist-tutorial/tensorflowvisu.py", line 42, in tf_format_mnist_images
everything_incorrect_first = tf.concat([incorrectly_recognised_indices, correctly_recognised_indices], 0) # images reordered with indeces of unrecognised images first
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/array_ops.py", line 1075, in concat
dtype=dtypes.int32).get_shape(
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 669, in convert_to_tensor
ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/constant_op.py", line 176, in _constant_tensor_conversion_function
return constant(v, dtype=dtype, name=name)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/constant_op.py", line 165, in constant
tensor_util.make_tensor_proto(value, dtype=dtype, shape=shape, verify_shape=verify_shape))
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/tensor_util.py", line 367, in make_tensor_proto
_AssertCompatible(values, dtype)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/tensor_util.py", line 302, in _AssertCompatible
(dtype.name, repr(mismatch), type(mismatch).name))
TypeError: Expected int32, got list containing Tensors of type '_Message' instead.

I have installed the Anaconda and tensorflow 1.0. and then I started Spyder ,ran the file 'mnist_1.0_softmax.py',and a folder named 'data' appeared..Next ,I only saw a graph without any data.

$ python3 mnist_1.0_softmax.py
Tensorflow version 0.12.1
Extracting data/train-images-idx3-ubyte.gz
Extracting data/train-labels-idx1-ubyte.gz
Extracting data/t10k-images-idx3-ubyte.gz
Extracting data/t10k-labels-idx1-ubyte.gz
Traceback (most recent call last):
File "mnist_1.0_softmax.py", line 79, in
I = tensorflowvisu.tf_format_mnist_images(X, Y, Y_) # assembles 10x10 images by default
File "/home/raveen/tensorflow-mnist-tutorial/tensorflowvisu.py", line 42, in tf_format_mnist_images
everything_incorrect_first = tf.concat([incorrectly_recognised_indices, correctly_recognised_indices], 0) # images reordered with indeces of unrecognised images first
File "/home/raveen/.local/lib/python3.5/site-packages/tensorflow/python/ops/array_ops.py", line 1075, in concat
dtype=dtypes.int32).get_shape(
File "/home/raveen/.local/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 669, in convert_to_tensor
ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
File "/home/raveen/.local/lib/python3.5/site-packages/tensorflow/python/framework/constant_op.py", line 176, in _constant_tensor_conversion_function
return constant(v, dtype=dtype, name=name)
File "/home/raveen/.local/lib/python3.5/site-packages/tensorflow/python/framework/constant_op.py", line 165, in constant
tensor_util.make_tensor_proto(value, dtype=dtype, shape=shape, verify_shape=verify_shape))
File "/home/raveen/.local/lib/python3.5/site-packages/tensorflow/python/framework/tensor_util.py", line 367, in make_tensor_proto
_AssertCompatible(values, dtype)
File "/home/raveen/.local/lib/python3.5/site-packages/tensorflow/python/framework/tensor_util.py", line 302, in _AssertCompatible
(dtype.name, repr(mismatch), type(mismatch).name))
TypeError: Expected int32, got list containing Tensors of type '_Message' instead.

Hi, after a fresh install of the tensorflow-1.0.1 on Linux through virtual env and pip3 I'm getting error when trying to import tensorflowvisu.

ImportError: No module named 'tensorflowvisu'

import tensorflow works ok, but the tensorflowvisu does not.

Installation went fine until I ran this .py file: python mnist_1.0_softmax.py

Successfully downloaded train-images-idx3-ubyte.gz 9912422 bytes.
Extracting data\train-images-idx3-ubyte.gz
Successfully downloaded train-labels-idx1-ubyte.gz 28881 bytes.
Extracting data\train-labels-idx1-ubyte.gz
Successfully downloaded t10k-images-idx3-ubyte.gz 1648877 bytes.
Extracting data\t10k-images-idx3-ubyte.gz
Successfully downloaded t10k-labels-idx1-ubyte.gz 4542 bytes.
Extracting data\t10k-labels-idx1-ubyte.gz
Traceback (most recent call last):
  File "mnist_1.0_softmax.py", line 78, in <module>
    I = tensorflowvisu.tf_format_mnist_images(X, Y, Y_)  # assembles 10x10 images by default
  File "D:\ProgramData\tensorflow-mnist-tutorial\tensorflowvisu.py", line 42, in tf_format_mnist_images
    everything_incorrect_first = tf.concat([incorrectly_recognised_indices, correctly_recognised_indices], 0) # images reordered with indeces of unrecognised images first
  File "D:\ProgramData\Miniconda3\envs\carnd-term1\lib\site-packages\tensorflow\python\ops\array_ops.py", line 1075, in concat
    dtype=dtypes.int32).get_shape(
  File "D:\ProgramData\Miniconda3\envs\carnd-term1\lib\site-packages\tensorflow\python\framework\ops.py", line 669, in convert_to_tensor
    ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
  File "D:\ProgramData\Miniconda3\envs\carnd-term1\lib\site-packages\tensorflow\python\framework\constant_op.py", line 176, in _constant_tensor_conversion_function
    return constant(v, dtype=dtype, name=name)
  File "D:\ProgramData\Miniconda3\envs\carnd-term1\lib\site-packages\tensorflow\python\framework\constant_op.py", line 165, in constant
    tensor_util.make_tensor_proto(value, dtype=dtype, shape=shape, verify_shape=verify_shape))
  File "D:\ProgramData\Miniconda3\envs\carnd-term1\lib\site-packages\tensorflow\python\framework\tensor_util.py", line 367, in make_tensor_proto
    _AssertCompatible(values, dtype)
  File "D:\ProgramData\Miniconda3\envs\carnd-term1\lib\site-packages\tensorflow\python\framework\tensor_util.py", line 302, in _AssertCompatible
    (dtype.name, repr(mismatch), type(mismatch).__name__))
TypeError: Expected int32, got list containing Tensors of type '_Message' instead.

What am I missing? Thanks.

Hii,

I am trying to train the points(using Line eqn and sine eqn) as in the .csv file attached.I am trying to classify using the code as below
Labels=[]
data=pd.read_csv('Line_SineDataNew.csv')

dataArray=data.values
np.random.shuffle(dataArray)
print(dataArray)
data_Label=dataArray[:, [2]]
print(data_Label)
for i in range (0,20000):
#print(dataArray[i])
#print (data_Label[i])
if (data_Label[i]==0):
Labels.append([1,0])
else:
Labels.append([0,1])

x = tf.placeholder(tf.float32, [None,2])
W = tf.Variable(tf.zeros([2, 2]))
b = tf.Variable(tf.zeros([2]))
y = tf.nn.relu(tf.matmul(x, W) + b)
#y = tf.nn.softmax(tf.matmul(x, W) + b)
y_ = tf.placeholder(tf.float32, [None, 2])
#cross_entropy = -tf.reduce_sum(y_ * tf.log(y))
cross_entropy = tf.reduce_mean(-tf.reduce_sum(y_ * tf.log(y), reduction_indices=[1]))
is_correct=tf.equal(tf.arg_max(y,1),tf.arg_max(y_,1))
accuracy=(tf.cast(is_correct,tf.float32))
train_step = tf.train.GradientDescentOptimizer(0.00030).minimize(cross_entropy)

init = tf.initialize_all_variables()

sess = tf.Session()
sess.run(init)

for i in range(20000):
#batch_X=dataArray[:,[0,1]]
#batch_Ylabel=Labels
batch_xs=dataArray[:,[0,1]]
#print(batch_xs)
batch_ys=Labels
#batch_xs=batch_X.next_batch(10)

batch_xs, batch_y

# s = mnist.train.next_batch(100)
sess.run(train_step, feed_dict= {x: batch_xs, y_: batch_ys})
a, iscorrect =sess.run([accuracy,is_correct],feed_dict= {x: batch_xs, y_: batch_ys})

b=np.mean(a)
print("accuracy=", b)

print("done with training")

But the accuracy is not increasing. I get only 50%.Can you advise me what could be issue
The below is the data