Git Product home page Git Product logo

comp2comp's Introduction

Comp2Comp

License: Apache 2.0 GitHub Workflow Status Documentation Status

Paper | Installation | Basic Usage | Inference Pipelines | Contribute | Citation

Comp2Comp is a library for extracting clinical insights from computed tomography scans.

Installation

git clone https://github.com/StanfordMIMI/Comp2Comp/

# Install script requires Anaconda/Miniconda.
cd Comp2Comp && bin/install.sh

Alternatively, Comp2Comp can be installed with pip:

git clone https://github.com/StanfordMIMI/Comp2Comp/
cd Comp2Comp
conda create -n c2c_env python=3.9
conda activate c2c_env
pip install -e .

For installing on the Apple M1 chip, see these instructions.

Basic Usage

bin/C2C <pipeline_name> -i <path/to/input/folder>

For running on slurm, modify the above commands as follow:

bin/C2C-slurm <pipeline_name> -i <path/to/input/folder>

Inference Pipelines

We have designed Comp2Comp to be highly extensible and to enable the development of complex clinically-relevant applications. We observed that many clinical applications require chaining several machine learning or other computational modules together to generate complex insights. The inference pipeline system is designed to make this easy. Furthermore, we seek to make the code readable and modular, so that the community can easily contribute to the project.

The InferencePipeline class is used to create inference pipelines, which are made up of a sequence of InferenceClass objects. When the InferencePipeline object is called, it sequentially calls the InferenceClasses that were provided to the constructor.

The first argument of the __call__ function of InferenceClass must be the InferencePipeline object. This allows each InferenceClass object to access or set attributes of the InferencePipeline object that can be accessed by the subsequent InferenceClass objects in the pipeline. Each InferenceClass object should return a dictionary where the keys of the dictionary should match the keyword arguments of the subsequent InferenceClass's __call__ function. If an InferenceClass object only sets attributes of the InferencePipeline object but does not return any value, an empty dictionary can be returned.

Below are the inference pipelines currently supported by Comp2Comp.

End-to-End Spine, Muscle, and Adipose Tissue Analysis at T12-L5

Usage

bin/C2C spine_muscle_adipose_tissue -i <path/to/input/folder>
  • input_path should contain a DICOM series or subfolders that contain DICOM series.

Example Output Image

Spine Bone Mineral Density from 3D Trabecular Bone Regions at T12-L5

Usage

bin/C2C spine -i <path/to/input/folder>
  • input_path should contain a DICOM series or subfolders that contain DICOM series.

Example Output Image

Abdominal Aortic Calcification Segmentation

Usage

bin/C2C aortic_calcium -i <path/to/input/folder> -o <path/to/input/folder> --threshold
  • The input path should contain a DICOM series or subfolders that contain DICOM series, or a nifti file.
  • The threshold can be controlled with --threshold and be either an integer HU threshold, "adataptive" or "agatson".
    • If "agatson" is used, agatson score is calculated and the a threshold of 130 HU is used
  • Aortic calcifications are divided into abdominal and thoracic at the T12 level
  • Segmentation masks for aortic calcium, the dilated aorta mask and the T12 seperation plane are saved in ./segmentation_masks/
  • Metrics on an aggregated and individual level for the calcifications are written to .csv files in ./metrics/

Example Output

Statistics on aortic calcifications:
Abdominal:
Total number:            10
Total volume (cm³):      0.161
Mean HU:                 383.3+/-66.4
Median HU:               366.5+/-62.9
Max HU:                  571.9+/-190.6
Mean volume (cm³):       0.016+/-0.020
Median volume (cm³):     0.011
Max volume (cm³):        0.074
Min volume (cm³):        0.002
Threshold (HU):          266.000


Thoracic:
Total number:            1
Total volume (cm³):      0.030
Mean HU:                 378.1+/-0.0
Median HU:               376.0+/-0.0
Max HU:                  538.0+/-0.0
Mean volume (cm³):       0.030+/-0.000
Median volume (cm³):     0.030
Max volume (cm³):        0.030
Min volume (cm³):        0.030
Threshold (HU):          266.000

AAA Segmentation and Maximum Diameter Measurement

Usage

bin/C2C aaa -i <path/to/input/folder>
  • input_path should contain a DICOM series or subfolders that contain DICOM series.

Example Output Image (slice with largest diameter)

Example Output Video Example Output Graph

Contrast Phase Detection

Usage

bin/C2C contrast_phase -i <path/to/input/folder>
  • input_path should contain a DICOM series or subfolders that contain DICOM series.
  • This package has extra dependencies. To install those, run:
cd Comp2Comp
pip install -e '.[contrast_phase]'

3D Analysis of Liver, Spleen, and Pancreas

Usage

bin/C2C liver_spleen_pancreas -i <path/to/input/folder>
  • input_path should contain a DICOM series or subfolders that contain DICOM series.

Example Output Image

Contribute

We welcome all pull requests. If you have any issues, suggestions, or feedback, please open a new issue.

Citation

@article{blankemeier2023comp2comp,
  title={Comp2Comp: Open-Source Body Composition Assessment on Computed Tomography},
  author={Blankemeier, Louis and Desai, Arjun and Chaves, Juan Manuel Zambrano and Wentland, Andrew and Yao, Sally and Reis, Eduardo and Jensen, Malte and Bahl, Bhanushree and Arora, Khushboo and Patel, Bhavik N and others},
  journal={arXiv preprint arXiv:2302.06568},
  year={2023}
}

In addition to Comp2Comp, please consider citing TotalSegmentator:

@article{wasserthal2022totalsegmentator,
  title={TotalSegmentator: robust segmentation of 104 anatomical structures in CT images},
  author={Wasserthal, Jakob and Meyer, Manfred and Breit, Hanns-Christian and Cyriac, Joshy and Yang, Shan and Segeroth, Martin},
  journal={arXiv preprint arXiv:2208.05868},
  year={2022}
}

comp2comp's People

Contributors

ad12 avatar adritrao avatar asch99 avatar edreismd avatar fohofmann avatar louisblankemeier avatar malteekj avatar sssomani avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

comp2comp's Issues

Local Implementation @ AppleSilicon M1

First of all, Louis, Jakob and all, thank you for your efforts implementing an easy-to-use pipeline for body composition analysis. For reference, and for future versions, a documentation of the issues I ran into during setup and implementation at a local machine w/ M1 Max, and its solution:

Install

Comp2Comp requires tensorflow, TotalSegmentator PyTorch. The standard installation will not work on a M1/M2 machine. Solution:

Install tensorflow

conda install -c apple tensorflow-deps=2.9.0 -y
python -m pip install tensorflow-macos==2.9
python -m pip install tensorflow-metal==0.5.0

Install PyTorch

conda install pytorch torchvision torchaudio -c pytorch

Install Numpy, scikit-learn and scipy manually

conda install -c conda-forge numpy scikit-learn scipy plotly -y

Install TotalSegmentor

 python -m pip install git+https://github.com/wasserth/TotalSegmentator.git

Install Comp2Comp.

It is important not to use the installation bash, as some of the predefined requirements won't work.Thus:

  1. Clone Comp2Comp git clone https://github.com/StanfordMIMI/Comp2Comp.git in the respective directory
  2. Remove the following requirements from setup.py
"numpy==1.23.5",
"tensorflow>=2.0.0"
'totalsegmentator @ git+https://github.com/StanfordMIMI/TotalSegmentator.git'
  1. install comp2comp using python -m pip install -e . in the respective directory

Pipeline 2 (body composition segmentation) should work now on a M1/M2 taking 0.5-2sec per image. For pipeline 1 (additional segmentation of spine using TotalSegmentator) additional adaptions are necessary:

dl_utils.get_available_gpus using nvidia-smi

The authors check free memory on gpus using nvidia-smi. This will not work on a M1/M2 machine, and TotalSegmentator works only with CUDA compatible GPUs (!="mps"). I am not sure, about torch.device("mps") in the future, see also wasserth/TotalSegmentator#39. A simple workaround is to use only the CPU, therefore avoid the nvidia-smi-call by setting num_gpus = 0; gpus = None manually and remove or comment out all dl_utils.get_available_gpus-calls in C2C and cli.py.

get_dicom_paths_and_num

For Pipeline 1, comp2comp searches for dicom files in the input directory. The get_dicom_paths_and_num-function does not return anything if any other files than *.dcm (e.g. hidden files such as .DS_Store) are included in the directory. Adapt the function accordingly, or make sure that only *.dcm are included in the directory.

to be continued / remaining issues:

At a local machine the integration and use --fast and --body_seg for TotalSegmentator might be preferable.

A problem I could not solve so far is, that TotalSegmentator using Pipeline1 breaks down with

"File .../lib/python3.9/site-packages/nibabel/loadsave.py", line 104, in load raise ImageFileError(f'Cannot work out file type of "{filename}"').

I think the underlying problem is, that TotalSegmentator usually takes *.nii.gz instead of a whole folder containing *.dcm. However, I am not sure about it (as it seems to be working at the authors' machine). Maybe the implementation of NIFTI support in general #19 might be a solution to get rid of the dicom_paths_and_num-issue (see above) and this at the same time?!

load model correction, SpineComputeROIs inconsistent error

Hi Louis,

thank you for cleaning up :) Two issues I encountered with the latest version of your work:

Downloading model from hugging face each run

The pipeline loads muscle/fat model from hugging face each run, regardless whether the model already exists in models/ or not.

Downloading muscle/fat model from hugging face
100% [......................................................................] 138570720 / 138570720

Although this might drive your download rates at hugging face :), it's unnecessary 140mb traffic per segmentation. Thus, I changed models.py line L71-72 to:

try:
   filename = Models.find_model_weights(self.model_name, model_dir)

Spine compute ROIs

In some dcm, the computation of the ROIs does not work.

Finished SpineSegmentation with output keys dict_keys(['segmentation', 'medical_volume'])
Running SpineReorient with input keys odict_keys(['inference_pipeline', 'segmentation', 'medical_volume'])
Finished SpineReorient with output keys dict_keys([])
Running SpineComputeROIs with input keys odict_keys(['inference_pipeline'])
ERROR PROCESSING inputs/batch/27
Traceback (most recent call last):
File "...Comp2Comp/bin/C2C", line 122, in spine_muscle_adipose_tissue pipeline()
File "...Comp2Comp/comp2comp/inference_pipeline.py", line 48, in call output = inference_class(self, **output)
File "...Comp2Comp/comp2comp/spine/spine.py", line 143, in call (spine_hus, rois, centroids_3d) = spine_utils.compute_rois(
File "...Comp2Comp/comp2comp/spine/spine_utils.py", line 229, in compute_rois two_largest = keep_two_largest_connected_components(slice)
File "...Comp2Comp/comp2comp/spine/spine_utils.py", line 262, in keep_two_largest_connected_components mask[labels == sorted_indices[i] + 1] = 1
IndexError: index 1 is out of bounds for axis 0 with size 1

The strange thing is, that this error does not occur consistently. Instead, if I rerun the pipeline using exactly the sam dcm, in sometimes everything works fine. Any ideas where to start debugging? I assume that the problem is not keep_two_largest_connected_components itself but the result of connectedComponentsWithStats?!

Best, Felix

wrong weights url

The system tries to download the weights file from the wrong url, so it fails there and stops.
Furthermore, as it is right now the system redownloads the weights for every new run, which makes it slow and results in redundant copies of the same file.

Both are easily solved. in models.py/load_model():

    try:
        filename = Models.find_model_weights()
    except:
        #weights_file_name = wget.download("https://huggingface.co/stanfordmimi/stanford_abct_v0.0.1/blob/main/stanford_v0.0.1.h5", out=PREFERENCES.MODELS_DIR)
        weights_file_name = wget.download("https://huggingface.co/stanfordmimi/stanford_abct_v0.0.1/resolve/main/stanford_v0.0.1.h5", out=PREFERENCES.MODELS_DIR)

        logger.info("Downloading muscle/fat model from hugging face")
        # filename = os.path.join(
        #    PREFERENCES.MODELS_DIR, "{}.h5".format(self.model_name)
        # )
        filename = Models.find_model_weights()
    logger.info("Loading muscle/fat model from {}".format(filename))
    return load_model(filename)

Cannot slice image objects; consider using img.slicer[slice] to generate a sliced image (see documentation for caveats) or slicing image array data with img.dataobj[slice] or img.get_fdata()[slice]

Hi. I'm currently using the the muscle_adipose_tissue pipeline and it works perfectly.
However, I run into some issues when trying to use the spine_muscle_adipose_tissue and liver_spleen_pancreas pipelines, they all give the same type error:

force_separate_z: None interpolation order: 0
Traceback (most recent call last):
  File ".../.Trash/Comp2Comp/comp2comp/spine/spine.py", line 135, in spine_seg
    img, seg = nnUNet_predict_image(
  File "....../lib/python3.9/site-packages/nibabel/spatialimages.py", line 645, in __getitem__
    raise TypeError(
TypeError: Cannot slice image objects; consider using `img.slicer[slice]` to generate a sliced image (see documentation for caveats) or slicing image array data with `img.dataobj[slice]` or `img.get_fdata()[slice]`

Are there any specific modifications needed for the DICOM files to run these pipelines?

dicom_to_nifti orientation issue

dicom_to_nifti triggers an error if it detects that different slices have different orientations. Seems that using SimpleITK solves this.

3D method does not accept dicom folder

The 2D and 3D method expect the same input: a folder with dicom files (or rather files ending in .dcm). The 2D method works fine with .dcm files, but the 3D method crashes. The reason being that the 3D method uses the TotalSegmentator net, which needs a nifty file as input instead of a folder (with dicom files).

A kludge to on the fly convert the folder with dicom files to a nii.gz file is possible, but still fails.

A solution might be to either use TotalSegmentator directly; TotalSegmentator does work with the nii.gz file above.

Manual adjustment of segmentation output

Thank you very much for this repository.
If there are some failed segmentation, is it possible to adjust it manually? as you can see in this example the psoas muscles are not fully segmented at the level of L5

spine_muscle_adipose_tissue_report

Dicom outputs

Pack image outputs into Dicom files for Dicom input / output

Issues Running Comp2Comp on Mac OS Apple Silicon

Hello,

Im having issues running C2C. Im getting this error:
zsh: parse error near `\n'

This is the command Im using: bin/C2C spine_muscle_adipose_tissue -i </Users/JohnDoe/Downloads/AX Thicks/DICOM>

I followed all the instructions as outlined for Apple silicone. When I double click on C2C files in the bin folder, I get this error:

/Users/JohnDoe/Comp2Comp/bin/C2C ; exit;
(base) JohnDoe@JohnDoe-MacBook-Pro ~ % /Users/JohnDoe/Comp2Comp/bin/C2C ; exit;
Traceback (most recent call last):
File "/Users/JohnDoe/Comp2Comp/bin/C2C", line 8, in
from comp2comp.aaa import aaa
ModuleNotFoundError: No module named 'comp2comp'

What am I doing wrong?
Thank you

issue in processing result C2C spine_muscle_adipose_tissue -i dir

Processing: dir with 529 slices

Inference pipeline:
(1) InferencePipeline
(2) SpineFindDicoms
(3) InferencePipeline
(4) SpineMuscleAdiposeTissueReport

Starting inference pipeline for:

Running InferencePipeline with input keys odict_keys(['inference_pipeline', 'kwargs'])

Inference pipeline:
(1) DicomToNifti
(2) SpineSegmentation
(3) ToCanonical
(4) SpineComputeROIs
(5) SpineMetricsSaver
(6) SpineCoronalSagittalVisualizer
(7) SpineReport

Starting inference pipeline for:

Running DicomToNifti with input keys odict_keys(['inference_pipeline'])
Finished DicomToNifti with output keys dict_keys([])

Running SpineSegmentation with input keys odict_keys(['inference_pipeline'])

If you use this tool please cite: https://pubs.rsna.org/doi/10.1148/ryai.230024

Resampling...
Resampled in 5.60s
Predicting part 1 of 1 ...
100%|███████████████████████████████████████████████████████████████████████████████████| 27/27 [00:11<00:00, 2.45it/s]
Predicted in 90.40s
Resampling...
Finished SpineSegmentation with output keys dict_keys([])

Running ToCanonical with input keys odict_keys(['inference_pipeline'])
Finished ToCanonical with output keys dict_keys([])

Running SpineComputeROIs with input keys odict_keys(['inference_pipeline'])
RuntimeWarning: invalid value encountered in divide
normalized_sums = sums / np.sum(sums)
WARNING:root:Label L5 not found in segmentation volume.
WARNING:root:Label L4 not found in segmentation volume.
WARNING:root:Label L3 not found in segmentation volume.
Computing ROI with centroid 255.000, 198.800, 17.864 and pixel spacing 0.709mm, 0.709mm, 0.625mm...
Elapsed time for erosion operation: 1.412116527557373 seconds
Elapsed time for full ROI computation: 1.7626237869262695 seconds
Computing ROI with centroid 257.000, 196.715, 64.390 and pixel spacing 0.709mm, 0.709mm, 0.625mm...
Killed I have gpu machine in system torch.gpu.is_available() return true
How to handle this situation i need some help here

Possible to work on google colab?

Hi,

is it possible to run comp2comp on google colab?
and do you consider to integrate nifti format for input and output images?

Best,
Aymen

Per-pipeline installation

There should be a way to install dependencies for only the pipelines that you are interested in running.

Master pipeline

TODO: Implement a master pipeline that includes all currently implemented pipelines.

Todos

  • Make the spine, muscle, adipose tissue pipeline work when a subset of T12 - L5 is visible
  • Dataset integrity check (check for missing slices, nonuniform SI spacing)
  • Add support for NIfTIs

Tensorflow version

Tensorflow version 2.12.0 ran into the following error
image
when running the inference pipeline.
Tested that Tensorflow version 2.11.0 was working correctly with the pipeline.

Request for working docker image

This is an amazing resource and tool, thank you for developing and sharing as open source.

It looks like several dependencies have been updated and we're having difficulties with building a working container from the dockerfile in the repo. As a consequence, the installation instructions don't seem to work.

Could you please release a working docker image?

ImportError: libXrender.so

Screenshot 2023-03-20 at 4 43 48 PM

If you receive an error like this, running the following in the c2c_env conda environment solves the issue:

conda install -c conda-forge xorg-libxrender

Nested pipelines

Use nested pipelines to reduce code duplication - pipeline as an InferenceClass object

get_dicom_or_nifti_paths_and_num naming

Hi,
I just tried to run Comp2Comp and I experienced the following error:

from comp2comp.io.io_utils import get_dicom_nifti_paths_and_num
ImportError: cannot import name 'get_dicom_nifti_paths_and_num' from 'comp2comp.io.io_utils' (/home/jakob/dev/Comp2Comp/comp2comp/io/io_utils.py)

I fixed it by renaming get_dicom_nifti_paths_and_num to get_dicom_or_nifti_paths_and_num.

todos

  • Make the package pip installable for those that don't want to run the installation script
  • Asserts for the dimensionality of the input CTs (512x512 and heuristics for covering T12-L5)
  • Add NIFTI support
  • Improve logging
  • Add Stanford spine model

Todos

  • Add an info file that includes all of the completed scans, runtime, error status, etc.
  • Potentially get rid of spine seg borders so can see cortical bone easier
  • Curved planar reconstructions
  • Spherical ROIs with median as opposed to mean
  • Put T12 - L5 in corner of image
  • Fix reverse ordering of metrics with spine level
  • Validate IMAT
  • Decrease image saturation
  • Add Stanford spine model

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.