Super Resolution 4D Flow MRI using Residual Neural Network

This is an implementation of the paper 4DFlowNet: Super-Resolution 4D Flow MRI Using Deep Learning and Computational Fluid Dynamics using Tensorflow 2.0 with Keras. Previous implementation on Tensorflow 1.8 can be found in the tf1.8 branch.

Please find the pre-trained networks weights (for both Tensorflow versions) here:

4DFlowNet pre-trained weights

If you are using later Tensorflow 1.x version that is not compatible with this 1.8 version, please refer to Tensorflow backwards compatibility (tf.compat module).

Below are the example prediction results from an actual 4D Flow MRI of a bifurcation phantom dataset.

LowRes input (voxel size 4mm)

High Res Ground Truth vs noise-free Super Resolution (2mm)

High Res Ground Truth vs noise-free Super Resolution (1mm)

To prepare training or validation dataset, we assume a High resolution CFD dataset is available. As an example we have provided this under /data/example_data_HR.h5

How to prepare training/validation dataset.

1. Generate lowres dataset
    >> Configure the datapath and filenames in prepare_lowres_dataset.py
    >> Run prepare_lowres_dataset.py
    >> This will generate a separate HDF5 file for the low resolution velocity data.
2. Generate random patches from the LR-HR dataset pairs.
    >> Configure the datapath and filenames in prepare_patches.py
    >> Configure patch_size, rotation option, and number of patches per frame
    >> Run prepare_patches.py
    >> This will generate a csv file that contains the patch information.

The training accepts csv files for training and validation set. A benchmark set is used to keep prediction progress everytime a model is being saved as checkpoint. Example csv files are provided in the /data folder.

To run a training for 4DFlowNet:

1. Put all data files (HDF5) and CSV patch index files in the same directory (e.g. /data)
2. Open trainer.py and configure the data_dir and the csv filenames
3. Adjust hyperparameters. The default values from the paper are already provided in the code.
4. Run trainer.py

Adjustable parameters:

To prepare 4D Flow MRI data to HDF5, go to the prepare_data/ directory and run the following script:

>> python prepare_data.py --input-dir [4DFlowMRI_CASE_DIRECTORY]

>> usage: prepare_mri_data.py [-h] --input-dir INPUT_DIR
                       [--output-dir OUTPUT_DIR]
                       [--output-filename OUTPUT_FILENAME]
                       [--phase-pattern PHASE_PATTERN]
                       [--mag-pattern MAG_PATTERN] [--fh-mul FH_MUL]
                       [--rl-mul RL_MUL] [--in-mul IN_MUL]

Notes:

• The directory must contains the following structure: [CASE_NAME]/[Magnitude_or_Phase]/[TriggerTime]
• There must be exactly 3 Phase and 3 Magnitude directories
• To get the required directory structure, DicomSort is recommended. Sort by SeriesDescription -> TriggerTime.
• In our case, VENC and velocity direction is read from the SequenceName DICOM HEADER. Code might need to be adjusted if the criteria is different.

To run the prediction, download first the pre-trained weights. We have provided an example dataset under the data/ folder.

1. Create a directory named models/
2. Put the downloaded 4DFlowNet folder under models/ 
3. Put your dataset under the data/ folder
4. Go to src/ and open predictor.py and configure the input_filename and output_filename if necessary
5. Run predictor.py

Adjustable parameters:

If you encounter any problems in using the code, please open an issue in this repository or feel free to contact me by email.

Author: Edward Ferdian ([email protected]).