Comments (9)
My notebook with some plots is also attached here.
mcmc_BAT_Julia.pdf
from bat.jl.
Hi @atasattari, BAT.jl has trouble running with only one chain, could you run with two chains at least? It's on the to-do list, we just haven't gotten around to fixing that yet (since people typically run with 4 or 8 chains).
Also I recommend using BAT.jl#main currently instead of the last 2.x release, which is quite old. We're preparing a BAT.jl v3.0 but there's some changes to be made first.
from bat.jl.
I updated the package and increased the number of chains to 4. It took almost 45 minutes to get to Begin tuning of 4 MCMC chain(s)
. Does that make any sense?
Would you happen to notice an obvious issue in my cost function or Julia code? Is there a way to get a progress bar while running the sampler?
Just to compare, I implemented a similar cost function in Python optimized with Numba and used emcee
for sampling. With 30 walkers ('emcee' language), the computation with the same number of steps takes less than a minute which is confusing.
from bat.jl.
One important difference is that emcee can work in parallel (we should probably add an emcee-like sampler to BAT) and the many walkers share information about the posterior, so they "help each other" converges. While Metropolis or HMC single walker chains can converge more slowly, since each walker has to find the "way" to the typical set by itself. We have some code for improved burn in based on the "Pathfinder" algorithm and the Robust adaptive Metropolis
Algorithm, but it's not integrated in BAT.jl yet. The plan is to add this and some other shiny new features until summer. We'll also add progress bars, so one can see how sampling progresses - BAT.jl can nest algorithms, so we need nested progress display, which makes it a bit more tricky than just using ProgressMeter.
In case it's a performance issue with you model, could you run BenchmarkTools.@benchmark logdensityof($your_posterior, rand($prior))
or so, and maybe check your likelihood implementation with @code_warntype
?
from bat.jl.
Thanks so much for the hints. I performed a basic comparison between the log-likelihood call times in Julia and Python. The Python one is ~20-30% faster. I'm not a Julia expert, but there seem to be some tweaks to boost the performance. At least running in parallel seems to be much simpler in Julia. 'emcee' has issues pickling some of the Numba functions which prevent it to run in parallel.
I'm not sure how many events were thrown away before convergence but it took BAT almost 8 hours to produce a sample of 10^3 events for my model. Apart from the performance, other features look really cool. I'll update you if I figure out something about the performance. Also, please let me know if you guys have a tutorial or a Jupyter Notebook that shows some performance optimization or fancier tricks.
from bat.jl.
If you can send me your complete notebook (with necessary data files) I can take a look at potential performance issues.
from bat.jl.
@atasattari is this still an active issue?
from bat.jl.
@oschulz The issue is resolved. It was mainly slowness and a lack of log reports for debugging. Turned out to be repetitive integral calculations to find the PDF for custom models in the likelihood. This situation should be common for unbinned likelihood studies. Would it be reasonable to add a similar example to the tutorial? I can provide my code for reference if needed.
from bat.jl.
Would it be reasonable to add a similar example to the tutorial? I can provide my code for reference if needed.
Thanks you, yes. We we thinking about adding more examples/tutorials anyway.
from bat.jl.
Related Issues (20)
- Type instability in bat_sample HOT 1
- Support pyhf HOT 27
- Doesn't seem to work with Julia 1.6.2? HOT 3
- How does BAT differ from Turing? HOT 10
- Pullback of DistributionTransform HOT 7
- Sampling with a single chain
- Bridge sampling with non-integer weights
- Handling periodic parameters HOT 3
- Use histogram as both prior and input data? HOT 1
- MLE best-fit parameter disagreeing with `pyhf` HOT 15
- BAT dependencies, package load time(s) and modularization HOT 10
- Order of Samples not preserved in IO via HDF5.jl
- Custom CIs and/or printing all CI values via bat_report HOT 8
- Multil Template Fitter planned? HOT 6
- too strong version req. of dependencies HOT 2
- `HamiltonianMC()` is failed on the tutorial model HOT 4
- MCMC diagnostic plots HOT 1
- Add CodeMeta file HOT 3
- `bat_report()` error: Tuple field type cannot be `Union{}` HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from bat.jl.