Comments (7)
Hi,
sorry, I am not correct, confused by the use of the class tf.train.ExponentialMovingAverage.
Another question, where does this op 'ema_apply_op' runs during training?
from tflearn.
Hi,
By returning same variables, do you mean same value? or is it another issue?
This operation returns a single Tensor, whatever "is_training" value. But during computation, the calculation graph will be altered according to "is_training" value, so that if is_training is True, the first operation (mean/variance update) will be performed, or the second one otherwise.
If you are using that operation without TFLearn trainer class, you need to specify a training mode:
# To set training mode:
tflearn.is_training(True)
# To remove training mode:
tflearn.is_training(False)
(There is little typo in the doc hat can be confusing; this operation actually 'set' the training mode, and not 'check' it. I will update it)
from tflearn.
Hi,
During training, function update_mean_var seems just not update the mean/variance and just return a identity of it.
` def update_mean_var():
with tf.control_dependencies([ema_apply_op]):
return tf.identity(ema_mean), tf.identity(ema_var)
is_training = tflearn.get_training_mode()
mean, var = tf.python.control_flow_ops.cond(
is_training, update_mean_var, lambda: (ema_mean, ema_var))`
from tflearn.
tf.control_dependencies is an operation used to ensure that ema_apply_op
is calculated before return tf.identity(ema_mean), tf.identity(ema_var)
, so the ema_apply_op
is actually computed (updating the mean/variance).
from tflearn.
ema_apply_op
runs at training time only, everytime a data batch 'enter' a batch_normalization layer, it will be applied (So the mean/variance is updated at every batch).
from tflearn.
I find a example code use the very similar code in tensorflow/tensorflow#1724 as in batch_normalization:
`import tensorflow as tf
import numpy as np
x = tf.Variable(1.)
update_x = tf.assign_add(x, 1.0)
do_update = tf.placeholder(tf.bool)
ema = tf.train.ExponentialMovingAverage(.9)
ema_assign = ema.apply([x])
avg_without_update = ema.average(x)
with tf.control_dependencies([ema_assign]):
avg_with_update = tf.identity(avg_without_update)
avg = tf.python.control_flow_ops.cond(do_update, lambda: avg_with_update, lambda: avg_without_update)
with tf.Session() as sess:
sess.run(tf.initialize_all_variables())
print(sess.run([update_x, avg], {do_update: False}))
print(sess.run([update_x, avg], {do_update: False}))
print(sess.run([update_x, avg], {do_update: False}))
print(sess.run([update_x, avg], {do_update: False}))
print(sess.run([update_x, avg], {do_update: False}))`
here do_update is false, but the average is updated, could you tell me where goes wrong?
from tflearn.
It seems there is a bug here. Referring to the issue you sent, it seems that conditions (fn1 & fn2) dependencies are always executed, no matter the condition of tf.cond(), thus the result... I can try to see if there is a workaround, or wait for TensorFlow to patch it.
from tflearn.
Related Issues (20)
- HELLO? ANYBODY HOME? HOT 1
- Syntax warnings due to comparison of literals using is
- Error in TF2.0... Any ideas? HOT 3
- #001 'unicodeescape' code can't decode bytes in position 2-3: truncated \UXXXXXXXX escape HOT 1
- Update required HOT 1
- Not working with tensorflow 2.3.1 HOT 5
- Import error
- ValueError: Cannot feed value of shape (61,) for Tensor 'InputData/X:0', which has shape '(?, 61)' HOT 1
- Examples
- Xception Example model
- def variance_scaling in initializations.py tries to call deprecated class
- LSTM stateful?
- why i got same number with Alexnet ? HOT 1
- Why tflearn disables executing eagerly ? HOT 2
- OSS License compatibility question
- tflearn import issue HOT 5
- Pillow 10 breaks tflearn import HOT 2
- Requesting Review for pull request #1176
- Use github.com/apssouza22/chatflow as a conversational layer. It would enable actual API requests to be carried out from natural language inputs.
- tflearn HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from tflearn.