Duration of iteration increases as the number of iterations increase, with distributed optimization. about optuna HOT 14 CLOSED

The orange line below shows the duration per trial for a single worker, it seems that time per trial is independent of number of trials completed:

from optuna.

Thank you for reporting the issue and sharing your code. Given the complexity of the provided code, we are unable to execute it on our end. Could you perhaps provide a minimalistic version of the code that only focuses on and can reproduce this specific issue?

from optuna.

Thanks for your reply. I found out what the issue was, If you load a study using "optuna.study.load_study" the settings for the study are not loaded and they revert to default. Which in this case meant the sampler was changing from NSGAIII to the BaseSampler which then has increasing suggestion time dependant on the number of previous samples. To get around this "optuna.study.create_study" with" load_if_exists = True" or "optuna.study.load_study" can be used however all study settings must be replicated.

from optuna.

As explained in https://optuna.readthedocs.io/en/stable/reference/samplers/index.html, evaluating the objective function takes more time on the number of trials for almost all samplers. So I'm not sure this is a bug.

from optuna.

Thanks for your reply, I am using the NSGAIII sampler which is not shown in that table but I assume it would have the same relationship as NSGAII. NSGAII time per trial should be independent of the number of trials completed. As mentioned above, if I use a single worker I do not get the same issue.

from optuna.

Thanks. Right, but in _collect_parent_population of NSGAIII has for loop over trials, so I'm not sure this sampler is completely independent of the number of finished trials. Sorry I cannot answer clearly because I'm not familiar with this sampler...

from optuna.

Thanks! Could you share the minimal reproducible code with us?

from optuna.

I have sent an invite to collaborate to you.

from optuna.

Hi thank you for inviting me to your repo, but I believe the core-dev team also needs to access the repo if they investigate the issue. So I would appreciate it if you could create a public repo.

from optuna.

ok, I have made it public--> https://github.com/jt269/Optuna_share

I am currently testing with a single worker created through a subprocess and that also seems to have the same issue.

from optuna.

Hello, I have uploaded a minimalistic version. Thanks.

from optuna.

Hello, did you manage to reproduce this issue? Thanks

from optuna.

Sorry for my late response and thank you for sharing the minimum reproducible code.
I tried running your code, but unfortunately I couldn't reproduce the slowdown in my environment.
Could you share your results for the minimum reproducible code?

from optuna.

Thank you for sharing. To restore the Study state with load_study, you need to specify NSGAIIISampler in the sampler argument.

If the problem is resolved, I would like to close the issue, what do you think?

from optuna.