Optimal tuning of parameters for estimation about sccoda HOT 3 CLOSED

Hello!

An acceptance rate of 57.8% is in line with what we usually see in a converged chain that explores the parameter space well. From our experience, a non-converged chain usually has an acceptance rate of less than 35%. Keep in mind that this is only a heuristic, but it is usually enough.

You can also visually assess the inference quality by the traceplots, as shown in our advanced tutorial, section "Diagnostics and plotting". Also, the result object you receive after running sample_hmc supports all other functionalities from arviz, which you may find useful.

from sccoda.

Thanks for your reply. Do you think future versions of scCODA will provide additional diagnostics like posterior predictive checks, or more elaborate tutorials on diagnostics to make sure that the chains have converged, and the estimation has to be trusted?
I think that it might help since some of the people that will use it might not fully understand the details and implications of this model. In particular, I noticed in a previous run (with the same parameters but different chain length) I get slightly different results regarding a credible effect for a specific cell type (I haven't been able to reproduce it consistently, unfortunately).
It's totally fine if you don't plan to, but in any case, thank you for this work!

from sccoda.

Thanks for these suggestions!

We plan to continuously develop and improve scCODA even after publication, and model diagnostics are one point that we want to address in the future. Most diagnostic tools (like R-hat) are only really meaningful when running multiple MCMC chains, so we will look into it again, once we get around to this.

from sccoda.