Currently, both ITK5 and ANTs are compiled fully from source, only for the standalone N4BiasFieldCorrection executable to be the only file remaining after pruning. It would be preferable, if possible, to only compile that one command, as this would greatly reduce container build time.

Unlike other more general neuroimaging containers, which may include full versions of multiple relevant software packages, in the case of this particular container the ANTs and FSL packages are only installed due to the dependence of specific MRtrix3 scripts on their availability.

As such, instead of installing and retaining the full gamut of those software packages, one option would be to strip those external dependencies of all content save for the specific functionalities required, potentially greatly reducing the size of the container.

The disadvantage of such is that it may remove commands from those packages that plug current gaps in MRtrix3 capabilities, e.g. inter-modal registration. On the other hand, the goal of this container should really be only to provide MRtrix3 functionalities; commands from other packages can be executed using those packages directly (or indeed dedicated containers for those packages), and there are alternative projects for providing containers encapsulating multiple neuroimaging software packages, e.g. VNM.

A valid FREESURFER_HOME envvar and FreeSurferColorLUT.txt file are necessary for a couple of scripts. Don't particularly want to be downloading a 4GB FreeSurfer tarball within the recipe only to extract one text file and throw out the rest; but conversely not sure about just hosting a duplicate copy of a file from some other software.

The current pruning process implicitly assumes that the example data on which the external software scripts are run are sufficient to explore the full set of commands that may be invoked by those scripts. If however there is any branching logic within those scripts that results in a specific command being invoked only in certain circumstances, then it is possible that a command may be pruned that is in fact requisite for certain types of data input.

While such absences should be reported by users over time, it would be preferable to do an initial interrogation of the code within these scripts, and cross-reference with the results of the minify pruning process. If such code branches can be found, then the test data should be augmented to ensure that such commands are utilised at least once during the battery of tests.

With the setup I'm currently working on, I was unable to get mrview working through Docker. However I don't know whether or not it's due to using Arch and having a dual-GPU laptop that's not properly set up for using such.

What's needed here before an official 3.0.2 MRtrix3 Docker container can be pushed is:

• Confirm that the existing README instructions for Linux do work.

• Add instructions for Mac and Windows, if they are different / possible.

While the set of commands from non-MRtrix3 neuroimaging software packages that will be present in the container has been reduced, they will still nevertheless be within PATH, and hence one could do e.g. MRtrix3.sif N4BiasFieldCorrection ...

What might be interesting is to have some form of gateway command / mechanism that only inserts the non-MRtrix3 contents of /opt/ into PATH following resolution of the requested command, such that the above example would not work but dwibiascorrect ants would be able to find that requested command.

It might be the case that this is fairly trivial to do for command invocations via the entrypoint but breaks down if SSHing into the container, but that would probably be an acceptable limitation.

Some users / sysadmins may prefer to build a Singularity image directly from a recipe file, rather than converting from Docker. With the external dependencies tarballed, it should not be too difficult to specify a Singularity file that does essentially the equivalent of Dockerfile.

This is utilised by 5ttgen hsvs when available, for trying to modify the 5TT image to permit streamlines through the anterior commissure. Unfortunately it seems to only be available through NITRC, which requires entering credentials on the website to access, and therefore can't be installed in a container recipe.

Before releasing / announcing, make sure that the container builds without issue on Singularity and that everything works as expected. Some HPC sysadmins have ended up building their own MRtrix3 container recipes in order to provide the software to their users, it would be preferable if they could instead just use ours.

Given the complexity of the similarities and differences between my #4 and @kaczmarj's #11, I thought I'd create a new dedicated thread where we can disentangle them as best we can and figure out the right strategy for merge resolution. It might also make things cleaner for anyone else wanting to bring themselves up to date and take part in the conversation.

Admittedly I ended up doing a lot more work here than I'd intended given @kaczmarj's prior offer for contribution: I'm eager to sort out comprehensive CI testing of MRtrix3 Python scripts (MRtrix3/mrtrix3#2143) and it's my intention for the solution here to be applicable there also. I've also ended up adding some other bits and bobs I needed along the way, which aren't strictly tied to the title of #4 but I merged to that branch nevertheless to prevent divergence in my own work.

So here we go:

1. Intermediate build:

   • #4 uploads a container with minified dependencies to DockerHub, to be used as the base for build of the actual containers intended for use (the MRtrix3 Docker container for users, and the container used by GitHub Actions for CI testing of MRtrix3 Python scripts);

   • #11 generates multiple tarballs of the minified dependencies and hosts them online, to be directly downloaded in the construction of those containers.

   Overall here, I think I'm in favour of the first approach: the base container can follow its own version tagging, it's transparent what versions of the different software dependencies are embedded within it, and I think it should be easier long-term to manage within DockerHub two containers instead of one, rather than having external downloads managed through some other means.

2. Parallel build:

   I wasn't aware of this functionality until now. It's not super urgent given that the base container should only need to be built very irregularly, but if the capability is there and working it probably makes sense to make use of it. Having said that, I don't think that utilisation of that capability is predicated on the totality of #11: it should be possible to patch that specific capability into the contents of #4?

3. Dockerfile envvars:

   I think I prefer having the dependencies of different software packages specified as individual environment variables as in #4, just to partition them out and better manage them over time.

4. Extra dependencies:

   #4 adds ACPCdetect via #10, and FreeSurferColorLUT.txt via #7; #11 also includes the latter. Need to make sure that these are not lost during the resolution process.

5. FSL installation:

   #4 installs FSL via the installation script; #11 does a direct tarball download & extraction. The "general recommendation" from FMRIB seems to be to use the installation script, but I don't know how necessary it is to follow that advice in the context of container construction.

   There's also some added trickery included in master that I devised from MRtrix3_connectome where sometimes the Python components of FSL don't work after the installation script, and so a second installer needs to be run (FMRIB instructions); I don't know whether / how installing via a tarball gets around this. But I have also found that more recently that script branch is no longer necessary; I don't know whether there were changes to the FSL installer script or to FSL itself that have rectified this.

6. FSL configuration:

   #4 runs the FSL configuration script as part of the entrypoint, whereas #11 seems to avoid it and instead manually sets the requisite environment variables itself. I feel as though the former is a bit fragile, e.g. if someone overrides the entrypoint when running the container the config script won't be run. So if the latter works I'm happy to go with that approach.

7. mrview capability:

   #11 additionally installs the dependencies for running GUI applications, and instructions for how to execute such using Docker. These should however be relatively easy to add as a standalone update.

So reflecting on it all, I think the approach that I want to take is to merge #4, and then create individual PRs for the individual additions that #11 provides over and above #4 that we want to utilise, being:

1. Parallelisation of base container build

2. Getting mrview working

3. Setting multiple envvars in single Docker instructions to reduce the number of layers

4. Possibly avoiding the FSL setup script at container entry

In parallel to this, I'm continuing to work on MRtrix3/mrtrix3#2143, but I'll be able to verify my progress on that a bit better if it's agreed to use the approach of a base container on DockerHub.

But all of the above is open to alternative proposals. While I've used Docker a bit, it's not an area of expertise for me, so if there's good justification for doing anything differently I'm happy to hear it.

Hey there,

I tried to reproduce the steps described here in order to adapt it to my needs by adding a command I need in docker (dtifit) so that I can use it in one image together with mrtrix3.

Unfortunately just cloning and reproducing the steps described in the README gave me the following error:

[NEURODOCKER 2021-10-18 16:36:33,328 INFO]: Configuration file written in /tmp/neurodocker-reprozip-trace/config.yml
Edit that file then run the packer -- use 'reprozip pack -h' for help
[NEURODOCKER 2021-10-18 16:36:33,328 INFO]: Uploading usage statistics is currently disabled
Please help us by providing anonymous usage statistics; you can enable this
by running:
    reprozip usage_report --enable
If you do not want to see this message again, you can run:
    reprozip usage_report --disable
Nothing will be uploaded before you opt in.
Traceback (most recent call last):
  File "/Users/nudelsalat/opt/anaconda3/bin/neurodocker-minify", line 8, in <module>
    sys.exit(main())
  File "/Users/nudelsalat/opt/anaconda3/lib/python3.8/site-packages/neurodocker/reprozip/gentle/trace.py", line 169, in main
    trace_and_prune(container=args.container, commands=args.commands, directories_to_prune=args.dirs_to_prune)
  File "/Users/nudelsalat/opt/anaconda3/lib/python3.8/site-packages/neurodocker/reprozip/gentle/trace.py", line 96, in trace_and_prune
    raise RuntimeError("Failed: {}".format(result))
RuntimeError: Failed: Traceback (most recent call last):
  File "/tmp/_prune.py", line 69, in <module>
    main(yaml_file=args.config_file, directories_to_prune=args.dirs_to_prune)
  File "/tmp/_prune.py", line 26, in main
    config = yaml.load(f)
TypeError: load() missing 1 required positional argument: 'Loader'

It looks like something related to the specific python version (2 vs 3) or a deprecated method in the yaml package. Have you had this issue before or know any way to solve it?