shawk91 / evolutionary-reinforcement-learning Goto Github PK

Thanks for the very sophisticated code.
I applied the command(python main.py --env MountainCar-v0) to run the code. The following figure shows part of the result. Apparently, the performance of evolutionary algorithm is much better than gradient descent method(DDQN).

ERL, as the paper said, retains the experiences from the entire evolutionary population in the replay buffer and uses them to update DDQN parameters by gradient descent. Are there big differences between the performance of evolutionary algorithm and DDQN, or am I misunderstanding the code?

 def train(self):
        self.gen_frames = 0

        ####################### EVOLUTION #####################
        all_fitness = []
        #Evaluate genomes/individuals
        for net in self.pop:
            fitness = 0.0
            for eval in range(self.args.num_evals): fitness += self.evaluate(net, is_render=False, is_action_noise=False)
            all_fitness.append(fitness/self.args.num_evals)

should we store transaction when eval the net from pop ?
if we do store，the

if store_transition: self.num_games += 1

in the func def evaluate(self, net, is_action_noise=False, store_transition=True):

is the num_games counter right?

thanks

If I use your env
env = utils.NormalizedActions(gym.make(env_tag))
I can not step an action, it says not implement error,

In the old code, network computing (including forward propagation and backward propagation) is on the GPU. But in the new version of the parallelized code, except for the PG part, all network calculations (population, rollout) are on the CPU. Why is that?
Is it a compromise of parallelization, or am I misunderstanding the code?