soloice / mnist-bn Goto Github PK
View Code? Open in Web Editor NEWUsing slim to perform batch normalization
mnist-bn's People
Contributors
Stargazers
Watchers
Forkers
libardo1 duyanggithub deeplearningzhy infinitas-loop zhianyang darktemplar2015 azuredsky bigdataha liveopp convmech youngfrank meldong mmasterliu sxsxsx shawngao94 csldali makelei-2019 chenjunjie1994mnist-bn's Issues
The result is different from your description! Help! Thanks!
when I set the parameter of istraining in batchnormalization as False, I get low acuracy about 0.1, however, if i set it as true, the accuracy is 0.99+,why?
The GraphKeys.GLOBAL_VARIABLES collection lacks some variables?
Hello! Thank you for the code which is very helpful to me.
But I observed something weird. After the graph is build, I add variable_to_restore = slim.get_model_variables() to get the model's variables, and this is what I get:
I wonder why the biases for convolutional&fc layers and γ for batch normalization are nowhere to be found? By the way, if I don't use batch normalization, that is, comment out the normalization_fn part in arg_scope, the biases will appear in the collection.
Thanks!
Updating operations w/ slim.learning.create_train_op
I am confused about the manual dependency established between the op "cross_entropy" and "update_op". I thought "slim.learning.create_train_op" would automatically gather the variables and update them with the training operation.
In practice, I'm asking: why is this piece of code necessary?
update_ops = tf.get_collection(tf.GraphKeys.UPDATE_OPS)
if update_ops:
updates = tf.group(*update_ops)
cross_entropy = control_flow_ops.with_dependencies([updates], cross_entropy)
I thought the conclusion of the issue you opened on this topic was that it's not necessary when using "slim.learning.create_train_op" .
Testing with batch_size = 1?
Hello.
Thanks for your code. It provides a lot of clarity among the midst of batch normalisation in TF. :)
Have you tried doing inference with one data point before?
The reason I am asking is that in general, the model does SIGNIFICANTLY worse with 1 data point (as I experience with my model, different from yours, but also use batch norm). Therefore, I just want to pick your brain on this :)
Thanks:)
using `slim.batch_norm` get low accuracy about 0.1
hello, when I use slim.batch_norm as your mnist_bn.py did, I get low accuracy about 0.1. But when I use tf.layers.batch_normalization like:
with slim.arg_scope([slim.conv2d, slim.fully_connected],
activation_fn=tf.nn.relu,
normalizer_fn=tf.layers.batch_normalization,
normalizer_params={'training': is_training, 'momentum': 0.95}):
the accuracy is about 0.98..
can you explain this situation？
Two problems
Invalid argument: Nan in summary histogram for: conv2/BatchNorm/moving_variance_0
hello, when I run your mnist_bn.py, it shows error:
Train step 0.0: entropy 0.779536366463: accuracy 0.959999978542
***** Valid step 0.0: entropy 1.93715846539: accuracy 0.620000004768 *****
2018-12-04 14:07:54.700603: W tensorflow/core/framework/op_kernel.cc:1192] Invalid argument: Nan in summary histogram for: conv2/BatchNorm/moving_variance_0
[[Node: conv2/BatchNorm/moving_variance_0 = HistogramSummary[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"](conv2/BatchNorm/moving_variance_0/tag, conv2/BatchNorm/AssignMovingAvg_1/read/_41)]]
2018-12-04 14:07:54.701488: W tensorflow/core/framework/op_kernel.cc:1192] Invalid argument: Nan in summary histogram for: conv2/BatchNorm/moving_variance_0
[[Node: conv2/BatchNorm/moving_variance_0 = HistogramSummary[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"](conv2/BatchNorm/moving_variance_0/tag, conv2/BatchNorm/AssignMovingAvg_1/read/_41)]]
2018-12-04 14:07:54.701654: W tensorflow/core/framework/op_kernel.cc:1192] Invalid argument: Nan in summary histogram for: conv2/BatchNorm/moving_variance_0
[[Node: conv2/BatchNorm/moving_variance_0 = HistogramSummary[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"](conv2/BatchNorm/moving_variance_0/tag, conv2/BatchNorm/AssignMovingAvg_1/read/_41)]]
Traceback (most recent call last):
File "mnist_bn.py", line 188, in <module>
tf.app.run(main=main, argv=[sys.argv[0]] + unparsed)
File "/xxx/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 48, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "mnist_bn.py", line 168, in main
train()
File "mnist_bn.py", line 105, in train
feed_dict=train_dict)
File "/xxx/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 889, in run
run_metadata_ptr)
File "/xxx/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1120, in _run
feed_dict_tensor, options, run_metadata)
File "/xxx/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1317, in _do_run
options, run_metadata)
File "/xxx/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1336, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Nan in summary histogram for: conv2/BatchNorm/moving_variance_0
[[Node: conv2/BatchNorm/moving_variance_0 = HistogramSummary[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"](conv2/BatchNorm/moving_variance_0/tag, conv2/BatchNorm/AssignMovingAvg_1/read/_41)]]
how to fix it?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.