incremental mode of operation about mrqa HOT 5 OPEN

hi Yarik, that is exactly what had been working on now for the past few months and we call it monitoring and it's on the branch https://github.com/Open-Minds-Lab/mrQA/tree/mrqa_monitor that will soon merge to main. Do you wanna take a look and test it? Thank you.

@sinhaharsh

from mrqa.

So the current design focusses on simply updating the existing compliance report, but we could also produce deltas in terms of what subjects and sessions have changed, and what in the report has been changed. I personally don't think the Delta in the report is not very useful, but you may have other uses for it, and it's rather not that difficult to produce. Let me know. Thanks.

from mrqa.

docs: https://open-minds-lab.github.io/mrQA/examples.html#monitoring-use-case

from mrqa.

Great to hear that you are working on pretty much what we need! ;) One tricky aspect is that, according to the doc you point out to, you depend on a singular --data-source /path/to/dataset . In our case we have accessions across all studies falling under some common /path/to/dataset and we would like to give specific new (can do with old ones if needed) subdirectories for monitoring/update. E.g. it would be smth like we have D1, D2, D3, etc folders but want to include in the QA only D1, D3, D10 ... which we figure out what to include based on some metadata within them. Do you think it would be possible or complicate situation too much?

BTW -- can it be .tgz archives with dicoms? then we could point to sourcedata/ of the converted BIDS dataset (e.g. http://datasets.datalad.org/?dir=/dbic/QA/sourcedata) but it would require first running conversion and I wanted to add mrQA as early in the pipeline as "process all dicoms received today and adjust reports for each of the corresponding studies", possibly before we get to BIDS conversion. Well, thinking about it -- would lead to situation where study decides to drop specific accession etc, which would get reflected while converting to BIDS but might not be known at the "DICOMs level".. we will see...

from mrqa.

1. there is no specific constraints on /path/to/dataset - it could be anything you want as long as there are valid DICOM files inside. You can setup a cron job on top of this however you like (exposing to different datasets as is convenient for you)

2. we chatted about this in another issue in - see #22 (comment)
   we are yet to work on it, and happy to take your help.

from mrqa.