TrainTest fails sometimes about alf HOT 11 CLOSED

Random seed is not fixed for environment , It is a possible reason that train test fails sometimes.
Any other case fails except TrainTest ?

from alf.

And there are other reasons:

1. multiple threads are used for parallelism between independent operations in tf
2. the order of running op are uncertain (when they have no dependencies)

perharps we should set tf.config.threading.set_inter_op_parallelism_threads(1) for unittest

from alf.

Random seed is not fixed for environment , It is a possible reason that train test fails sometimes.
Any other case fails except TrainTest ?

Sometimes the SAC case will also fail.

from alf.

And there are other reasons:

1. multiple threads are used for parallelism between independent operations in tf
2. the order of running op are uncertain (when they have no dependencies)

perharps we should set tf.config.threading.set_inter_op_parallelism_threads(1) for unittest

I think for unittest, to avoid stochasticity introduced by parallelism, we can set num_envs=1 and not use async-off policy training.

from alf.

It's hard to make the training have deterministic result, i did some experiments
with fixed seed for tf and environments and set inter_op_parallelism_threads to 1 , the result shows it still has the probability of getting different results

Personally think, it's ok when some unittest fails

from alf.

It's hard to make the training have deterministic result, i did some experiments
with fixed seed for tf and environments and set inter_op_parallelism_threads to 1 , the result shows it still has the probability of getting different results

Personally think, it's ok when some unittest fails

OK, I thought the reason why we changed unittest to tf.unittest is because of the determinism it provides. Then maybe next time the test threshold should be less strict.

from alf.

So if everything has fixed random seeds, then the only stochasticity is from CPU scheduling for parallelism, right? What about we use eager mode for unittests?

from alf.

So if everything has fixed random seeds, then the only stochasticity is from CPU scheduling for parallelism, right? What about we use eager mode for unittests?

Yes , the only stochasticity is from CPU scheduling for parallelism, it affect the generation of random numbers. I have tried using eager mode for train test with only 1 thread, but it does not make deterministic result (still do not know the reason)

from alf.

So if everything has fixed random seeds, then the only stochasticity is from CPU scheduling for parallelism, right? What about we use eager mode for unittests?

Yes , the only stochasticity is from CPU scheduling for parallelism, it affect the generation of random numbers. I have tried using eager mode for train test with only 1 thread, but it does not make deterministic result (still do not know the reason)

Hmm.. Interesting. @emailweixu Do you have any insight into this?

from alf.

Using eager mode will make the test much longer.
Perhaps the game itself has some randomness inside?

from alf.

Using eager mode will make the test much longer.
Perhaps the game itself has some randomness inside?

I think @witwolf tried setting the seeds of environments deterministically. Even so, the results are nondeterministic.

from alf.