Take [0,0,0] in MNI (that should be roughly the middle voxel of the MNI brain)and use that to determine where the center of the brain is in the data (that should be an inverse transform away, but is it? ...)

Any BIDS-compliant directory should be a possible input, not only a directory organized the way that our AFQ.data module does it.

Test to see if things work better with an initial affine. Another issue we're having is that the ROIs end up with holes in them when xformed to the individual brain.

In particular, need to rotate the b vectors according to the affine transformations derived from the registration.

We'd want to cache something to file in-between to save time

See discussion in #94

We want conda/pip requirements properly encoded so that we can easily install from them

1. Is the python version has the same effects with the Matlab code? I notice the Matlab code is still under the older version which needs the Matlab 2012, and will you give up updating the matlab code and maintain the python version from now on?
2. I also notice that you decide to use dipy to do the tractography in this year. What is the difference between the dipy and the code you wrote before for the tractography?
3. I check the code on the github website and find you do not update the streamline.py in pyAFQ/AFQ/utils which means the code I download from the website still does not work. In that case, I check the github and you update pyAFQ/AFQ/tests, _fixes.py and segmentation.py, so I run the pyAFQ/examples again, and got the error like this in the attached figure. Could you please help me check it and tell me which example code I should use? Btw, I use the python 3.6 under the Mac environment.

That reports on the versions of different dependencies (e.g., DIPY) and the version of the code that is being run. To better support users.

Seems like it would be have performance advantages. But might also change things. Possibly a flag that you use to configure things, but probably not.

We would like to be able to "reanimate" these objects, so we want to save some kind of a config to disk that can be used to store the key-word arguments. Probably as a json file.

1. Should the function _brain_extract (and others like it) take rows or should the row grabbing happen in a higher function
2. Do we want a general apply function. In other words, how much can we make general across all specific computations.

This currently doesn't work raising something like:

MNI_T2_img = afd.s3fs_nifti_read('hcp.recobundles/mni_icbm152_t2_tal_nlin_asym_09a.nii')
File "/usr/local/lib/python3.7/site-packages/AFQ/data.py", line 547, in s3fs_nifti_read
rr = zz.read()
File "/usr/local/lib/python3.7/gzip.py", line 276, in read
return self._buffer.read(size)
File "/usr/local/lib/python3.7/gzip.py", line 463, in read
if not self._read_gzip_header():
File "/usr/local/lib/python3.7/gzip.py", line 411, in _read_gzip_header
raise OSError('Not a gzipped file (%r)' % magic)
OSError: Not a gzipped file (b'\\\x01')

I tried to make it compatible here, but that didn't work (I think -- raising the same error). It's not central to what I am doing right now, so I am just leaving this note here, so we remember to come back to it at some point.

See: yeatmanlab/AFQ-Browser#19

See what the Matlab implementation did for this.

Hi!

I've been working on implementing pyAFQ in a pipeline, and have encountered a couple odd issues. Firstly, I was unable to install the released version using macOSX, though I was able to install it through pip on a Linux machine. On the Mac, I just got the generic pip error "ERROR: Could not find a version that satisfies the requirement AFQ (from versions: none) ERROR: No matching distribution found for AFQ". I went ahead and installed the development version, but have since been running into an error with convex_hull.py returning ValueError 'Input must be a 2D image', 'occurred at index 0'. The same data ran without a problem on the Linux machine using the released version of the software. Thank you so much for any insight into what I’m doing wrong here :)

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-8-e771d817d53c> in <module>
      7 myafq.set_dti_pdd()
      8 myafq.set_template_xform()
----> 9 myafq.export_rois()
     10 myafq.export_bundles()
     11 myafq.combine_profiles()

/usr/local/anaconda3/lib/python3.6/site-packages/AFQ/api.py in export_rois(self)
   1072                               args=[self.bundle_dict,
   1073                                     self.reg_template],
-> 1074                               axis=1)
   1075 
   1076     def export_bundles(self):

/usr/local/anaconda3/lib/python3.6/site-packages/pandas/core/frame.py in apply(self, func, axis, broadcast, raw, reduce, result_type, args, **kwds)
   6485                          args=args,
   6486                          kwds=kwds)
-> 6487         return op.get_result()
   6488 
   6489     def applymap(self, func):

/usr/local/anaconda3/lib/python3.6/site-packages/pandas/core/apply.py in get_result(self)
    149             return self.apply_raw()
    150 
--> 151         return self.apply_standard()
    152 
    153     def apply_empty_result(self):

/usr/local/anaconda3/lib/python3.6/site-packages/pandas/core/apply.py in apply_standard(self)
    255 
    256         # compute the result using the series generator
--> 257         self.apply_series_generator()
    258 
    259         # wrap results

/usr/local/anaconda3/lib/python3.6/site-packages/pandas/core/apply.py in apply_series_generator(self)
    284             try:
    285                 for i, v in enumerate(series_gen):
--> 286                     results[i] = self.f(v)
    287                     keys.append(v.name)
    288             except Exception as e:

/usr/local/anaconda3/lib/python3.6/site-packages/pandas/core/apply.py in f(x)
     76 
     77             def f(x):
---> 78                 return func(x, *args, **kwds)
     79         else:
     80             f = func

/usr/local/anaconda3/lib/python3.6/site-packages/AFQ/api.py in _export_rois(row, bundle_dict, reg_template, force_recompute)
    575                 (mapping.transform_inverse(
    576                     roi.get_data(),
--> 577                     interpolation='linear')) > 0).astype(int)
    578             # Cast to float32, so that it can be read in by MI-Brain:
    579             nib.save(nib.Nifti1Image(warped_roi.astype(np.float32),

/usr/local/anaconda3/lib/python3.6/site-packages/AFQ/utils/volume.py in patch_up_roi(roi, sigma, truncate)
     27 
     28     return convex_hull_image(gaussian(ndim.binary_fill_holes(roi),
---> 29                                       sigma=sigma, truncate=truncate) > 0)

/usr/local/anaconda3/lib/python3.6/site-packages/skimage/morphology/convex_hull.py in convex_hull_image(image)
     36     """
     37     if image.ndim > 2:
---> 38         raise ValueError("Input must be a 2D image")
     39 
     40     if Delaunay is None:

ValueError: ('Input must be a 2D image', 'occurred at index 0')

This block:

could be executed post-segmentation using the code that is now in https://github.com/nipy/dipy/blob/master/dipy/tracking/streamline.py#L691

Might incur a tiny performance cost (I need to think about it), but will simplify the code here.

Here:

http://scikit-learn.org/stable/faq.html#why-do-i-sometime-get-a-crash-freeze-with-n-jobs-1-under-osx-or-linux

Right now, if we are using the reco as segmentation algorithm, we can't call export_rois, because there are no way-point ROIs in this method. But -- we could export the ROIs that overlap with the location of the atlas streamlines in the subject space.

Rotate the b-vectors in this function

Use: https://github.com/nipy/dipy/blob/master/dipy/core/gradients.py#L449

To get only the central part of the streamline in each individual. See dipy/dipy#1617.

Alternatively, think of a way to align the nodes across individuals (@jyeatman has ideas how to do this, I believe).

We need to go through our various scripts, figure out what still works, and fix bugs
In other words: Identify the problem and fix it

Probably makes more sense to use map here than to use dask.delayed?

Might be because of:

https://github.com/yeatmanlab/pyAFQ/blob/master/AFQ/api.py#L18

Maybe defer this to first use? Should be resolved in #33

Should be computed this way:

https://github.com/vistalab/vistasoft/blob/master/mrDiffusion/fiber/stats/dtiFiberGroupPropertyWeightedAverage.m

We should keep the Mahalanobis distance calculation, but use it for cleaning, not for weighting

So that we can get fine-grained information about versions (in support of #148)

https://github.com/warner/python-versioneer

And represent the entire tractogram as an array right after tracking?

Hi guys,
First of all, thanks for your work!

I am trying to run the "plot_tract_profile.py" example script using my data.

However, I get this error: 'tuple' object has no attribute 'forward', which is related to the lines:
reg.write_mapping(mapping, hardi_out_dir + '/mapping.nii.gz')
and
mapping_data = np.array([mapping.forward.T, mapping.backward.T]).T

any idea?

I have other issues, for example, the tract "ARC" is not recognized:

bundle_names = ["SLF", "ILF", "UNC", "ARC"]

...

  7     for hemi in ['_R', '_L']:
  8         bundles[name + hemi] = {

----> 9 'ROIs': [templates[name + '_roi1' + hemi],
10 templates[name + '_roi2' + hemi]],
11 'rules': [True, True],

KeyError: 'ARC_roi1_R'

I checked and despite "ARC_L_prob_map" and "ARC_R_prob_map" are in the templates folder, their corresponding "roi" images are not...

Thanks!
Regards,
Erick

See yeatmanlab/AFQ-Browser#100

Would be good to know whether/how much results here differ from the Matlab results

So we can avoid warnings like this one: https://travis-ci.org/yeatmanlab/pyAFQ/jobs/553808933#L65

We can use functions from here: https://github.com/nipy/dipy/blob/master/dipy/io/streamline.py to replace utils.streamlinse.read_trk and utils.streamlinse.save_trk

Issue

I think we're missing some dependencies. When I try to install pyAFQ in a Docker container, import AFQ fails because it lacks the pandas and toolz dependencies. Here's a minimal Dockerfile that recreates the problem.

FROM python:3

# Install git so that we can get pyAFQ from its github repo
RUN apt-get update
RUN apt-get install -y git

# Install pyAFQ dependencies
RUN pip install dask
RUN pip install dipy
RUN pip install nibabel
RUN pip install boto3
RUN pip install cloudpickle

# Install pyAFQ from its github repo
RUN pip install git+git://github.com/yeatmanlab/pyAFQ.git@master#egg=pyAFQ

If you create an image from this docker file using docker build -t pyafq . and then run using docker run -it pyafq python, you can replicate the issue by attempting import AFQ.

Solution

One solution is to use pip install dask[complete] to pick up pandas and toolz. I recommend we change the requirements from dask to dask[complete].

I'm happy to submit a PR for this if you agree.

https://github.com/yeatmanlab/pyAFQ/blob/master/AFQ/utils/models.py#L23

As suggested by @akeshavan in the AFQ-Browser meeting on 26 Mar 2018.

Follow up on #136

Hi @arokem ! So excited about pyAFQ. Wondering if there is a way to feed in multishell ODF data processed using mrtrix (CSD; iFOD2 algorithm) for automated tract segmentation and quantification using the AFQ algorithm? Would be amazing to be able to do so, and apologies if I missed this in the documentation. Thanks for your time!

Traceback (most recent call last):
File "bin/pyAFQ_dki", line 5, in
from AFQ import dki
File "/Users/Adam/code/projects/pyAFQ/AFQ/init.py", line 1, in
from .api import * # noqa
File "/Users/Adam/code/projects/pyAFQ/AFQ/api.py", line 15, in
import AFQ.data as afd
File "/Users/Adam/code/projects/pyAFQ/AFQ/data.py", line 5, in
import boto3
ImportError: No module named 'boto3'

conda install boto3 fixed it so I recommend adding boto3 to the requirements.txt

The plotting tract profiles example works well, but when I run it with my own data, it shows MemoryError when tracking:

Dataset is already in place. If you want to fetch it again please first remove the folder /home/z/.dipy/stanford_hardi
Calculating DTI...
Tracking...

---

MemoryError Traceback (most recent call last)
in ()
35 print("Tracking...")
36 if not op.exists('dti_streamlines.trk'):
---> 37 streamlines = list(aft.track(dti_params['params']))
38 aus.write_trk('./dti_streamlines.trk', streamlines, affine=img.affine)
39 else:

~/anaconda3/lib/python3.5/site-packages/AFQ/tractography.py in track(params_file, directions, max_angle, sphere, seed_mask, n_seeds, random_seeds, stop_mask, stop_threshold, step_size, min_length, max_length)
107 evals = model_params[..., :3]
108 evecs = model_params[..., 3:12].reshape(params_img.shape[:3] + (3, 3))
--> 109 odf = tensor_odf(evals, evecs, sphere)
110 dg = dg.from_pmf(odf, max_angle=max_angle, sphere=sphere)
111

~/anaconda3/lib/python3.5/site-packages/AFQ/dti.py in tensor_odf(evals, evecs, sphere)
137
138 lower = 4 * np.pi * np.sqrt(np.prod(evals[mask], -1))
--> 139 projection = np.dot(sphere.vertices, evecs[mask])
140 projection /= np.sqrt(evals[mask])
141 odf[mask] = ((vector_norm(projection) ** -3) / lower).T

MemoryError:

Thank you.

To follow the BIDS derivatives spec as described here: https://docs.google.com/document/d/17ebopupQxuRwp7U7TFvS6BH03ALJOgGHufxK8ToAvyI/edit#heading=h.9vm2vlfgdks8

Here: #33