Regression: Can't resume chains anymore about cobaya HOT 15 CLOSED

Hi Vivian,

I have been doing some tests but have had no problem resuming with MCMC or PolyChord.

The last line in out_... suggests that resuming is not possible because the previous run was send with a different number of MPI processes from the new one. Do you think that's the case? If it is, I am afraid, this is a limitation that is hard to tackle right now.

Also, I am a little surprised that the chains files are deleted. This should never happen! Does it delete the chains when it tries to resume but fails?

(Sorry for the delay with this! I have too many projects going on right now)

from cobaya.

from cobaya.

from cobaya.

You are right, sorry! It does in the case in which there is a different number of MPI processes. This should definitely not happen! Fixing it right now! I hope you haven't lost anything important :(

from cobaya.

from cobaya.

Wow, I am really very sorry!!!

from cobaya.

from cobaya.

Thanks! :)

Still debugging: it's a tricky one! But won't close for today until I get it done. Stay tuned!

from cobaya.

Done! It was a error in my MPI-wrapping of the output class. I am very grateful to you for suffering through all these bugs!

On Monday, I'll try to incorporate your fix for the new Planck likelihood code, so that you don't need to keep your own version.

from cobaya.

I am not suffering! I am enjoying your new very cool code! I am glad I can contribute in some small ways!

from cobaya.

Hi I still get problems. Now this

Traceback (most recent call last):
File "/scratch/midway2/viniciusvmb/krause/cosmo_cobaya/bin/cobaya-run", line 11, in
load_entry_point('cobaya', 'console_scripts', 'cobaya-run')()
File "/scratch/midway2/viniciusvmb/krause/cosmo_cobaya/cosmolike_core/cobaya/cobaya/run.py", line 126, in run_script
run(info)
File "/scratch/midway2/viniciusvmb/krause/cosmo_cobaya/cosmolike_core/cobaya/cobaya/run.py", line 45, in run
_force_reproducible, _force_reproducible_default))
File "/scratch/midway2/viniciusvmb/krause/cosmo_cobaya/cosmolike_core/cobaya/cobaya/output.py", line 208, in get_Output
return Output(*args, **kwargs)
File "/scratch/midway2/viniciusvmb/krause/cosmo_cobaya/cosmolike_core/cobaya/cobaya/output.py", line 191, in init
setattr(self, var, comm.bcast(getattr(self, var), root=0))
AttributeError: 'Output_MPI' object has no attribute 'resuming'

Best

from cobaya.

And if I run without mpirun I get the error

Traceback (most recent call last):
File "/scratch/midway2/viniciusvmb/krause/cosmo_cobaya/bin/cobaya-run", line 11, in
load_entry_point('cobaya', 'console_scripts', 'cobaya-run')()
File "/scratch/midway2/viniciusvmb/krause/cosmo_cobaya/cosmolike_core/cobaya/cobaya/run.py", line 126, in run_script
run(info)
File "/scratch/midway2/viniciusvmb/krause/cosmo_cobaya/cosmolike_core/cobaya/cobaya/run.py", line 66, in run
modules=info.get(_path_install)) as sampler:
File "/scratch/midway2/viniciusvmb/krause/cosmo_cobaya/cosmolike_core/cobaya/cobaya/sampler.py", line 202, in get_sampler
info_sampler[name], posterior, output_file, resume=resume, modules=modules)
File "/scratch/midway2/viniciusvmb/krause/cosmo_cobaya/cosmolike_core/cobaya/cobaya/sampler.py", line 161, in init
self.initialize()
File "/scratch/midway2/viniciusvmb/krause/cosmo_cobaya/cosmolike_core/cobaya/cobaya/samplers/mcmc/mcmc.py", line 65, in initialize
get_mpi_comm().barrier()
AttributeError: 'NoneType' object has no attribute 'barrier'

So I can't use Cobaya at all now

from cobaya.

I was able to create the following workaround on output.py

from cobaya.

Thanks for the fix, and sorry! Nothing goods come out of a last-minute Friday-evening attempt of a bugfix.

from cobaya.

from cobaya.