A3C Doom: What is the number of trainable variables? about deeprl-agents HOT 4 OPEN

Hi Ibrahim,

I am unsure of what you mean by the shared network being counted twice. If you are using A3C with one worker, then there should be two network initialized, one global network and one worker network.

from deeprl-agents.

Hi Arthur

I want to know exactly the number of trainable parameters ... I used this code

total_parameters = 0
for variable in tf.trainable_variables():
    shape = variable.get_shape()
    print(shape)
    variable_parametes = 1
    for dim in shape:
        variable_parametes *= dim.value
    total_parameters += variable_parametes
print(total_parameters)

and got this result above where this part is repeated twice:

(8, 8, 1, 16)
(16,)
(4, 4, 16, 32)
(32,)

My question is, how many trainable parameters are there in A3C code?

Thank you

from deeprl-agents.

The answer above accurately describes the total number of trainable parameters. But you need to keep into account that it is doubled the number of parameters actually ever being updated (or used for value and policy calculation).

from deeprl-agents.

Thanks @awjuliani

Could you please comment on this elaboration to ensure correct understanding?

I understand that the following:
1- each worker get a copy of the global network
2- each worker interacts with its environment and calculates loss and gradients
3- each worker updates the global network using gradients (not its local network)
4- go to step 1

So, if we have only one worker, we will have 2 identical networks:
1- global (where actual parameters updates take place)
2- local (worker) where environment interactions, loss and gradients are calculated (no parameter update for local networks)

This is why number of parameters increases when number of workers increases (because each worker has local copy of the network)

And the actual number of parameters that are actually updates is 2398816 / 2 (Number of parameters when using one worker divided by 2)

I hope this is correct

Finally, I would like to thank you for your excellent article

from deeprl-agents.