implement support for scans.tsv about bids-matlab HOT 10 CLOSED

You are right: so far, all the sub-<participant_label>/[ses-<session_label>/]sub-<participant_label>_scans.tsv files are ignored.
How would you envisage the query syntax to look like?

Querying participants.tsv was discussed here: https://www.jiscmail.ac.uk/cgi-bin/webadmin?A2=SPM;fadb5af3.1902

from bids-matlab.

With FieldTrip data2bids I have consistently been trying to always write the scans.tsv. See the examples on http://www.fieldtriptoolbox.org/example/bids/, and especially http://www.fieldtriptoolbox.org/workshop/madrid2019/bids_sedation/ (data is available from ftp), where the scans.tsv contains something relevant besides the name of the dataset.

What I'd like with this data is to query it like this

indx = BIDS.subjects(1).scans.reactiontime > 800;
all = bids.query(BIDS, 'data', 'sub', '01');
slow = all(indx);

from bids-matlab.

Looking at `bids-examples/synthetic', which has sessions, I see

>> BIDS = bids.layout('.')
BIDS = 
  struct with fields:

             dir: '/Users/roboos/tmp/bids-examples/synthetic'
     description: [1×1 struct]
        sessions: {}
           scans: [0×0 struct]
            sess: [0×0 struct]
    participants: [1×1 struct]
        subjects: [1×10 struct]

It is not clear to me what the sessions, scans, and sess fields at the top level are meant to contain. Both scans information and sessions information are at the subject level, hence I would expect in this case that BIDS.subject(1).sessions would contain tabular information that can be queried for that specific subject.

from bids-matlab.

hmm, it appears that multiple sessions are not yet supported. The data on disk has two sessions, the BIDS layout not. @gllmflndn can you confirm this limitation?

>> ls sub-01/
ses-01			ses-02			sub-01_sessions.tsv

>> BIDS.subjects(1)
ans = 
  struct with fields:

       name: 'sub-01'
       path: '/Users/roboos/tmp/bids-examples/synthetic/sub-01/ses-01'
    session: 'ses-01'
       anat: [1×1 struct]
       func: [8×1 struct]
       fmap: [0×0 struct]
        beh: [0×0 struct]
        dwi: [0×0 struct]
        eeg: [0×0 struct]
        meg: [0×0 struct]
       ieeg: [0×0 struct]
        pet: [0×0 struct]

from bids-matlab.

Hi @robertoostenveld ,

• Support of multiple sessions: I think it does. The hierarchical nature of the data (with sessions) is flattened when stored into the BIDS structure (into subjects(i).session) but you should still be able to query session-specific data.
• You are right that some of the fields in the BIDS structure are unused. Some could be removed and some are a reminder of further work to take place. I started writing this library several years ago when the initial BIDS specifications were still work in progress. This is why the BIDS structure should only be accessed through queries and not directly, so that even if its internals are completely changed, the user-level interface will still work.
• To make queries according to scans.tsv, I am keen to add an interface similar to the one with participants:

 subjects = spm_BIDS(BIDS,'subjects','participants',...
   struct('sex','male', 'age',@(x) x > 25));
 spm_BIDS(BIDS,'data','sub',subjects,'type','T1w')

I will have a look at your exemplar dataset and comment further here.

from bids-matlab.

Thanks for the reply and sharing the design ideas. I am at the MNE-Python code sprint (secretly still working on MATLAB code...) and hence now have time to look into this.

from bids-matlab.

confirmed, sessions are stored. In a single subject dataset with three sessions I see this

>> cd ~/tmp/bids-examples/ieeg_epilepsy_ecog/
>> BIDS = bids.layout('.')
BIDS = 
  struct with fields:

             dir: '/Users/roboos/tmp/bids-examples/ieeg_epilepsy_ecog'
     description: [1×1 struct]
        sessions: {}
           scans: [0×0 struct]
            sess: [0×0 struct]
    participants: [1×1 struct]
        subjects: [1×3 struct]

However, it is not clear with this representation where information from sessions.tsv should be stored. The 7t_trt example would be good to use here. However, for now I will restrict myself to the scans.tsv

I have now added the scans.tsv to the subject field like this

>> BIDS.subjects(1)
ans = 
  struct with fields:

       name: 'sub-ecog01'
       path: '/Users/roboos/tmp/bids-examples/ieeg_epilepsy_ecog/sub-ecog01/ses-postimp'
    session: 'ses-postimp'
       anat: [1×1 struct]
       func: [0×0 struct]
       fmap: [0×0 struct]
        beh: [0×0 struct]
        dwi: [0×0 struct]
        eeg: [0×0 struct]
        meg: [0×0 struct]
       ieeg: [1×1 struct]
        pet: [0×0 struct]
      scans: [1×1 struct]
>> BIDS.subjects(1).scans
ans = 
  struct with fields:

    filename: {2×1 cell}
    acq_time: {2×1 cell}

from bids-matlab.

Looking into this reminded me that I had wondered whether scans.tsv were of any use anyway. They contain file-specific metadata that could be stored in JSON sidecar files

This is a side point because they are described in the specs so their support has to be implemented here but part of me wonders whether these files are inherited from the file organisation BIDS was built upon "by accident".
https://bids-specification.readthedocs.io/en/stable/03-modality-agnostic-files.html#scans-file
This is also mentioned here:
https://docs.google.com/document/d/1LEgsMiisGDe1Gv-hBp1EcLmoz7AlKj6VYULUgDD3Zdw/edit

from bids-matlab.

scans.tsv is important to relate simultaneously acquired data types to each other. See http://www.fieldtriptoolbox.org/example/bids_pom/#final-_scanstsv-file as an example of simultaneous fMRI, EMG and eye tracker.

'scans.tsv` is also to be used for "Additional fields can include external behavioral measures relevant to the scan. For example vigilance questionnaire score administered after a resting state scan." link. That is what I use on http://www.fieldtriptoolbox.org/workshop/madrid2019/bids_sedation/ (see the data on the ftp server) where the scans file contains details on sedation.

EDIT: ... oh, I already have this example before.

from bids-matlab.

withdrawing PR due to lack of progress

from bids-matlab.