patscott / diver Goto Github PK

Parallellisation in Diver is very simple (which is why it is generally very fast -- it has very little MPI overhead). The current strategy is simply to make NP an even multiple of the number of MPI processes, and give every MPI process NP/nprocs individuals to evolve every generation.

However, for target functions with highly variable evaluation times, this can be very inefficient: if one process gets a couple of particularly slow individuals, it holds up the entire generation. It may be more efficient in such cases to use a queuing strategy, where each MPI process takes one or more individuals to evolve from the head of a queue, and returns to the queue to take more if and when it is finished with its current individuals.

Diver therefore needs an option to operate in queue mode, and to allow the number of individuals taken from the head of the queue in each 'bite' to also be set as an option.

At the moment, you can tell Diver to delete duplicate members of the population in a given generation, but it does this after evaluating the target function -- so it wastes time re-evaluating the target for the duplicate points. The deletion of duplicates needs to be done before the objective function is evaluated.