I suggest we make an official release (v0.1 perhaps?) to bookmark the state of this repository for our initial submission, and then also for each subsequent revision + the final accepted (🤞 ) version.

To keep the annotations consistent across all methods, please add links to the papers. See ISC annotations as a sample format to follow.

Hi folks -

One of the cool things we've tried pushing with brainiak.matnormal is a prototyping tool people can use to try structured-covariance models more easily. With this in mind, it'd be awesome if someone wanted to kick the tires and see how easy it is to set up their favorite models in separable-structured-cov flavor. In the paper we had full examples with SRM and RSA, and in the notebook I've added matrix-normal factor analysis (basically slight extension of PPCA) just to show something new, but having someone else actually try it would be great. I think @jeremymanning and I discussed doing the same with TFA forever ago at SFN, and in my very naive understanding the existence of MatnormalRegression.calibrate makes that model a simple instance of IEM. Basically any (multi)linear gaussian regression or factor model should hopefully be not very much code to try.

To keep the annotations consistent across all methods, please add links to the papers. See ISC annotations as a sample format to follow.

To keep the annotations consistent across all methods, please add links to the papers. See ISC annotations as a sample format to follow.

I couldn't figure out how to post my review on the OSF Preprint page so I am putting it here. Hopefully that will not cause any issues.

The article “BrainIAK: The Brain Imaging Analysis Kit” describes a software package for performing several different advanced analyses of functional neuroimaging data and resources for learning how to use it. Overall the paper is well written and provides detailed and informative descriptions of each of the included methods. The notebooks are well done and provide helpful demos to help users more quickly learn how to use the system. By optimizing the software to run efficiently on HPC systems, the developers have further empowered users to scale up their analysis. Overall, nicely done!

I enjoyed the section on optimization and was glad to see that a notebook with scaling advice is provided. I would prefer to see some of this information in the paper, perhaps each method could be accompanied by a brief section that provides this advice? In particular details about what dimension of the data is most crucial for an algorithm, and how to set bounds to avoid overwhelming the system.

I couldn’t find information about how Brainiak handles scheduling itself to run on clusters. What types of clusters and cluster software does it support? Or does it leave this detail to the user?

It would be nice to include some more information about how the software is developed. It appears to be primarily writing in Python with at least some of the more computationally intensive code written in C++. Also, please provide some information about how testing is performed at various levels. Are their datasets that the users can use to verify or benchmark their installations? Are their CI/CD systems that can automatically test pull requests?

It is nice to see the section on how to contribute to Brainiak. It would be useful to also include code standard requirements, any form of autolinting that will be performed on PRs, and details about how PRs are reviewed prior to merging.

In section 2.5 it might be useful to mention the similarity of the implemented methods to methods developed for dynamic functional connectivity analyses e.g., PMID 19629982 and PMC3920766. Is it possible that the described HMM approach could be used for dynamic FC with some modification?

In the introduction to real time fMRI section, the statement “Existing software frameworks have helped researchers to better implement RT-fMRI studies.” appears to be missing citations.

The optimization section mentions searchlight analysis, but I couldn’t find it mentioned in the earlier descriptions of methods. Are all of the methods in the toolbox described here? If not, it would be good to include a table of all of the methods included in the toolbox along with citations.

I was thinking of creating a new repo for these examples, once everything is in its final state in the next day or so. We can call it brainiak-new-examples and then change it to brainiak-aperture-examples (or some variant) if the paper goes through review successfully at Aperture.

Do you'll think this is a good idea? Alternatively, we can make this repo public access, and rename the repo to brainiak-new-examples (or something else that we agree on).