Monte Carlo paralellization at the photon level about vts HOT 4 CLOSED

There is renewed interest. In the link above, 2 methods are described: 1) Threadpool, 2) Parallel.For. I tried the Parallel.For because it is easy to add around the MonteCarloSimulation.cs for loop that loops over each photon.

I first tried on a linux 18 core, 2 threads per core machine. Without any code modifications N=1e6 photons took 1/2 hour to run. This machine utilizes the threads on its own and typically I see 8-12 threads being used during a simulation. With the code fix the code took 1.7 hours! Not sure why. I thought possibly the threading management was conflicting with the parallelized code somehow.

So I tried the same on a linux computer with 4 cores, no threads. Without the code fix N=1e6 took about an hour. With the code fix, the code compiles but the many unit tests fail.

The Threadpool requires a larger code modification. I will look into this method and continue searching for other options.

from vts.

Based on your last comment, it sounds like we can close this Item. As you discovered, I think the Task Parallel Library (TPL) probably isn't a great fit for "per-photon" parallelization, which is best suited for parallelizing "large" jobs. Seeking this level of performance from parallelization likely requires some significant work. Have we done any profiling of our code to identify the bottlenecks? Wonder if the biggest item is the RNG? Or the double-precision cosine calculations? Or perhaps the heap- and allocation-happy simulation structure? Lots to think about and explore!

from vts.

Yes, Lisa helping me with this task and we now have a command line option to run in parallel. Here is our current documentation:
https://github.com/VirtualPhotonics/Vts.MonteCarlo/wiki/MCCL-Parallel-Processing

from vts.

Neat! Thanks for sharing the link.

from vts.