Build a transformer that performs censorship and return the censored dataset. (Power et al 2012)

When we set save_partial = True the software stores the partial accuracies map but then it doesn't save the entire map, and doesn't delete partial files as well.

Using dask branch

Build a reader to get files in BIDS and use pymvpa ds.

When we use AnalysisIterator(kind='configuration') it may happen that sample_slicer__attribute is not overwrited, sometimes it is ok since we want to slice different attributes but sometimes not.

One solution could be to use a dictionary instead of different sample_slicer__attribute keys.

New in version 0.4: sample_indices_ used instead of return_indices=True.

Build a Transformer that performs Regression / GLM (see nistats)

Cross-decoding should be performed with an ad-hoc split of examples during cross-validation:

Things to do:

• Build a custom partitioner that separates labels from an experiment to train and other to test.

https://realpython.com/python-memory-management/

https://www.codementor.io/satwikkansal/python-practices-for-efficient-code-performance-memory-and-usability-aze6oiq65

https://chase-seibert.github.io/blog/2013/08/03/diagnosing-memory-leaks-python.html#

How to manage

From @robbisg on June 13, 2018 12:38

• PermutatorTransformer
• Save function for Transformer
• Cross-decoding How to
• Within/Across Searchlight

Copied from original issue: robbisg/mvpa_itab_wu#24

This is to uniform and avoid error when loading when the dir is full of different files.

Use glob and regular expression.

From @robbisg on June 13, 2018 12:11

Implement states.

• Build StateAnalyzer

Copied from original issue: robbisg/mvpa_itab_wu#22

data_path composition is delegated to load_ds, but to check load_subjects or maybe other functions, maybe is better to build a function build_data_path from configuration file to help doing this!

This is used to setup _default_options and _default_config when we iterate.
And this is also to setup CrossValidation and Analyziz parameter.

List of all possible analysis.
List of parameters.

From @robbisg on April 12, 2018 10:3

Copied from original issue: robbisg/mvpa_itab_wu#16

File "/home/robbis/git/joblib/joblib/externals/loky/process_executor.py", line 418, in _process_worker
r = call_item()
File "/home/robbis/git/joblib/joblib/externals/loky/process_executor.py", line 272, in call
return self.fn(*self.args, **self.kwargs)
File "/home/robbis/git/joblib/joblib/_parallel_backends.py", line 567, in call
return self.func(*args, **kwargs)
File "/home/robbis/git/joblib/joblib/parallel.py", line 225, in call
for func, args, kwargs in self.items]
File "/home/robbis/git/joblib/joblib/parallel.py", line 225, in
for func, args, kwargs in self.items]
File "/home/robbis/git/pyitab/pyitab/analysis/results/base.py", line 23, in get_values
with open(conf_fname) as f:
FileNotFoundError: [Errno 2] No such file or directory: configuration.json'

In get_values and get_*_results we have configuration.json that is hard-coded, we need to first generate a BIDS-valid json and then obtain it from the informations.

From @robbisg on June 5, 2018 13:14

Document balancer.utils

Copied from original issue: robbisg/mvpa_itab_wu#19

In utils.atlas, the developed functions import data from a local directory.

Build a transformer that filters typical frequencies for fMRI.

The Analyzer class should generate an id to be used in conjunction with analysis.name.

If we use the iterator, the responsibility of the id generation should be on the shoulders of the iterator?

analysis

configurator.py

• test _get_fname_info
• test _get_kwargs (maybe deprecated)
• test save

pipeline.py

• Test fit
• test lines 53/54
• test save

iterator.py

• Test iterator
• test combination, list and dict types

base.py

• test save

results/base.py

• test get_values
• test get_results
• test filter_dataframe
• test df_fx_over_keys

io

subjects.py

• Test with selected_subjects

configuration.py

• Test save_configuration

Loading package

• Test fetch with n_subjects=1
• Test load_ds with subjects without data (e.g. monks data)

ValueError: Classification metrics can't handle a mix of binary and unknown targets

If scoring is set!

Working around with setting self.scoring = None in temporal_decoding.py at line 100.

From @robbisg on June 5, 2018 10:23

I need to write functions for writing results.

Questions:

• Where use path?
• Use a different saver for the transformers/analyzers?

Searchlight results

• Maps for each cv
• Average map if a cv within subject is performed
• Merged map for a statistical test
• Files / something for AFNI test
• Permutation maps ?

Decoding

• Weight map
• Feature selection
• Cross validation indices
• Accuracies / Avg. Accuracy
• Permutation values

Connectivity

States

Cross-decoding

Copied from original issue: robbisg/mvpa_itab_wu#18

Permutation

• Is it better to use a transformer? #82

Cross-decoding

• Build a custom CrossValidator

Transformers

• #85

Connectivity

See #60

States

• Build StateAnalyzer
• Build Transformers for states
• Import metrics from mvpa_itab

Results

• Refactor results using BIDS format of derivatives #27
• Cope with subject-wise results.

Configurator, Pipelines and Iterator

• Create a Configurator class that is general. #30
• Create a DecodingConfigurator class that inherits from Configurator. #30
• Add Dataset Loading into Configurator #31
• Study the possibility to use iterator to run subject-wise analysis.

Deprecated

• Evaluate load_spatiotemporal_dataset
• Evaluate read_configuration_json
• Evaluate read_remote_configuration

Implement a Balancer for TimeGeneralizationDecoding.

We need a transformer that using fa attributes of rois or something else transforms data by averaging/ getting PCs or by MVPC.

Substitute underscore with plus or point

ValueError: Cannot detrend the dataset, since it neither provides location information of its samples in the space spanned by the polynomials, nor does it match the number of samples this this mapper has been trained on. (got: 360 and was trained on 240).

Solve this issue in propagating name field!

pyitab/analysis/searchlight/init.py in _split_name(self, X, y, cv, groups)
171 X = X[..., 0]
172
-->173 split = [np.unique(groups[:, 1][test])[0] for train, test in cv.split(X, y=y, groups=groups)]
174 return split

It depends on group attribute which sometimes could be a list, in that case it should be used a different solution to be thinked.

Remove num from the dir created and add sub folder to that analysis.

find if sample_slicer__subject is in iterator's default_options.keys()

Build an Analyzer that performs correlation or coherence analysis!

Take care of also MEG data!

How to manage this?

For BIDS and for other ds.

If we need to change the target and slice the dataset according to target we need to implement a method to cope with this.

e.g. {'sample_slicer__%s':['L', 'F']} where? I don't know!

How to cope with this??

1. Use the iterator, by iterating the fetch (#31) ➡️ maybe problems in saving (#27) (but we can use the AnalysisPipeline save subdir function!)
2. Maybe is better to develop #27
3. For some analyses (connectivity) we need to use the iterating stuff.

This class is responsible of:

• Loading dataset loader
• Save configuration
• Prepare all of the important objects for AnalysisPipeline class

Specific analysis configuration should inherit from this class!

From @robbisg on June 13, 2018 12:25

Build TemporalGeneralizatorTransformer and Analyzer

Copied from original issue: robbisg/mvpa_itab_wu#23

If ds.sa has an attribute name it will raise this error!

Problem

Use BIDS specs to create dirs and so on!

Every analysis must be included in derivatives folder
the structure must be
<ds-root>/derivatives/<pipeline-name>/<subj-dir>

in the subj dir the files must be named as:
<source_keyword>[_keyword-value]_<suffix>.<ext>
in addition a dataset-description.json must be included.

1. The best solution is that <pipeline-name> is the on provided by AnalysisPipeline or by Analyzer
2. dataset-description.json must include PipelineDescription.Name
3. Maybe we can use keyword-value in file to specify different analyses.
   • A possibility is to use <pipeline-name>-<variant>
4. Each subdir file should be described by a filename.json file with:

TODO

• Every analysis must have a pipeline name
• We must check if dataset is composed by a single subject or by several subjects
• The derivatives/<subdir> must be [pipeline-<pipeline_name>]_analysis-<searchlight | decoding | connectivity>_[<analysis-specific-key>-<value]
• Analyzer must implement _get_pipeline_info to build dir name
• For searchlight analyses we must have derivatives/analsysis-searchlight_radius-0.3
• For decoding analyses we must have derivatives/analysis-roi_decoding_area-brain
• Obtain <source_keywords>
• Analyzer must implement _get_fname_info in order to build filename appropriately.
• Filenames for searchlight analyses must be <source_keywords>_target-<values>_task-<value>_date-<datetime>_num-<number>_<keyf>-<value>_<avg | cv>.nii.gz
• Filenames for decoding analyses must be <source_keywords>_target-<values>_task-<task>_mask-<mask>_value-<roi_value>_date-<datetime>_num-<num>_<key>-<value>_data.mat
• Each filename is paired with filename.json which is our configuration file! #43
• We can split information about dataset in dataset-description.json at the top of the pipeline dir #43

Fixes

• Remove underscore from dir and files names #41
• Analyses with sample_slicer__subjects must share same derivatives dir #42
• Results reader
• Remove hard-coded configuration.json #43

From @robbisg on June 5, 2018 16:2

The problem is that I need to do within-subject searchlight using the AnalysisScript and save results.

Possible solutions:

• Use script iterator that generates a unique id, then collect results using this id.
• Insert a name in the ScriptConfigurator or AnalysisIterator that should be used by AnalysisScript to
  generate a folder to store data.

Copied from original issue: robbisg/mvpa_itab_wu#20

This lets the BIDS package to index the files 💪

From @robbisg on June 13, 2018 9:37

I need to implement several things to perform a connectivity analysis

Transformers

• Averager across ROI
• PCA
• Multivariate distance
• ...

Analysis / Measures (look nilearn)

• Seed based Correlation
• ICA

Preprocessing / Trasformers?

• GLM regressor

Copied from original issue: robbisg/mvpa_itab_wu#21

The problem with searchlight is that some results may be lost if machines go down.
Maybe joblib Memory utility is useful to store partial results.

As the mapper for pymvpa dasets.