hyosubkim / bayes-toolbox Goto Github PK

Add a locall testing framework. I suggest pytest, its what most of the big packages use.

@hyosubkim any strong preference? Once we know which one to use we can get it implemented

Theres a section in the README but as more and more stuff gets added such as testing and linting this will expand. Itll be good to preemptively create one

https://github.com/hyosubkim/bayesian-statistics-toolbox#how-to-contribute-to-bst

Add a documentation framework. The two come to mind are mkdocs and sphinx. Sphinx is the most used but most heavy weight. Ive heard good things about mkdocs

https://www.sphinx-doc.org/en/master/
https://www.mkdocs.org/

I have created a tool for evaluating the statistical significance/predictive reliability of ML/AI analysis and inference. My use case assumes that an analyst has identified an estimator and parameter values that he/she believes to be superior. My tool (working title mlcompare) then applies one or more “meta-tests” to confirm (or not) if the estimator’s claimed superiority is truly meaningful, that is, statistically significant.

The initial version of mlcompare used frequentist (aka p-val) methods to evaluate statistical significance, but I have since decided to replace them with Bayesian methods and, ultimately, causal analysis.

I am interested in experimenting with your package in hopes of collaborating with you to blend our packages (or at least elements of them) to create a tool that integrates the best of Bayesian inference with statistically based model/estimator comparisons.

I have attached the latest version of my “statement of need,” which term I noticed in your upcoming paper introducing bayes-toolbox. This document was created before my decision to move to a Bayesian foundation, but it presents the overall use case.

Thanks, and I look forward to hearing from you.

David L. Wilt
[email protected]
1-540-420-0844

PDF_V5.5.pdf

After tests are implemented itll be easy to upgrade without worrying about regressions in expected behavior

Once local tests are created adding a CI runner for pull requests will make the contribution pipeline smoother for yourself and contributors

Easiest way to do this will be to copy the one from bambi, or PyMC and modify it for use here
https://github.com/bambinos/bambi/blob/main/.github/workflows/test.yml

DS_store, ipynb checkpoints, egg-info, should be removed from git, and added to the gitignore. Examples from PyMC gitignore are below.

Note. This may require the uuse of a repo cleaner

https://github.com/hyosubkim/bayesian-statistics-toolbox/tree/main/.ipynb_checkpoints
https://github.com/hyosubkim/bayesian-statistics-toolbox/tree/main/src.egg-info
https://github.com/pymc-devs/pymc/blob/main/.gitignore#L12

Note. This may require the use of a repo cleaner due to the way git works once file are committed
https://rtyley.github.io/bfg-repo-cleaner/

I have tried a number of different ways to install your package in a virtual environment, both conda and poetry, but all have failed.

I have set the requirements to meet those associated with your latest version, 0.1.3. When I try to install with pip, I am advised, for example, that there is no version available, so it can't meet confirm requirements for any of your versions. Cloning the latest development version results in the same problem.

I am running on a macAIr M2, 16GB, macOS Sonoma 14.1.2

Thanks.

David Wilt

P. S. I am interested in contributing to your project and will add another issue for that purpose.