Official repository for the paper: "Development and Validation of an Automated Image-Based Deep Learning Platform for Sarcopenia Assessment in Head and Neck Cancer", published in JAMA Network Open 2023;6(8),e2328280.

Keywords: Sarcopenia, Head and Neck Cancer, Deep Learning, Convolutional Neural Network, CT

• DenseNet-163
• 3D U-Net

U-Net Segmentation Model Performance

• Python 3.8
• TensorFlow 2.4
• Keras

This code was developed and tested using Python 3.8.5.

For the code to run as intended, all the packages under requirements.txt should be installed. In order not to break previous installations and ensure full compatibility, it's highly recommended to create a virtual environment to run the DeepContrast pipeline in. Here follows an example of set-up using python virtualenv:

• install python's virtualenv

sudo pip install virtualenv

• parse the path to the python3 interpreter

export PY2PATH=$(which python3)

• create a virtualenv with such python3 interpreter named "venv" (common name, already found in .gitignore)

virtualenv -p $PY2PATH venv 

• activate the virtualenv

source venv/bin/activate

At this point, (venv) should be displayed at the start of each bash line. Furthermore, the command which python3 should return a path similar to /path/to/folder/venv/bin/python3. Once the virtual environment is activated:

• once the virtualenv is activated, install the dependencies

pip install -r requirements.txt

At this stage, everything should be ready for the data to be processed by the DeepContrast pipeline. Additional details can be found in the markdown file under src.

The virtual environment can be deactivated by running:

deactivate

The codebase for auto-segmenting the C3 skeletal muscle mass and for calculating muscle cross-sectional area (CSA), and Skeletal Mass Index (SMI) from 3D CT images.

Consists of two deep-learning models: Slice Selection Model and Segmentation Model. The Slice Selection Model is adapted from DenseNet and the Segmentation model is adapted from UNet.

Model Weights are available in the "model/test" folder.

1. Download the model weights from the google drive link provided above, unzip the files, and save them in the 'model' subfolder. Please note that the CV folder within the zip file contains five-fold cross-validation sub-models for segmentation.

2. Make sure you set the env properities (pass as command line arguments to main.py script) <proj_dir> - Project Directories Path where this code is copied - instuition name of curated dataset e.g. BWH , default value is "test"

3. The input files are stored in the following folder

   Raw scans - /data//img/

4. Following are default directories/paths that get created to stroe the pre-processed files, slice selection output as segmentation output. If you prefer, you can also set them using env variables <pre_process_dir>, <slice_csv_path> and <output_dir>. Passing them to main.py as command line arguments is also accepted.

   Pre-processed Scans - /data//preprocessed Slice Selection Output CSV - /data//_C3_top_slice_pre.csv Segmentation Model Output - /data//output_segmentations

Main script executes pre-processing, slice selction and segmentation sequentially

STEP argument can be used to execute each of the above steps individually. The arguments takes values ALL, SLICE, PREPROCESS, SEGMENT

Other scripts provided here OPTIONAL, used to generate the needed CSA information, generates statistics, visualizations and final clinical file needed combining patient meta information with image segmenation data

The Main script step can be run by executing:

main.py

This script generates the Cross-Sectional Area (CSA) of the C3 Skeletal Muscle Mass from the auto-segmented masks.

Set the proj_dir, raw_img_dir, raw_seg_dir, Slice Prediction CSV, and output_dir directories before executing the code.

1. Input#1: Auto segmentation files
2. Input#2: Manual segmentation files of test data
3. Input#3: C3_Top_Slice_Prediction.csv
4. Output: CSA in a CSV file

The Calculate C3 Cross Sectional Area step can be run by executing:

get_CSA.py

This script generates the masks with contours for easy visualization.

Set the proj_dir, raw_img_dir, raw_seg_dir, Slice Prediction CSV, and output_dir directories before executing the code.

1. Input#1: Auto segmentation files
2. Input#3: C3_Top_Slice_Prediction.csv
3. Output: Segmentation files with contours for visualizing the masks

The Visualize C3 Segmentation function can be run by executing:

visualize.py

This script generates the C3 Skeletal Muscle Index (SMI) and translates that to the corresponding L3 Skeletal Muscle Index (SMI).

Set the proj_dir, raw_img_dir, raw_seg_dir, Slice Prediction CSV, Clinical Meta-data files and output_dir directories before executing the code.

1. Input#1: Auto segmentation files
2. Input#2: Cross-Sectional Area CSV file
3. Input#3: C3_Top_Slice_Prediction.csv
4. Input#4: Patient Clinical meta information in a CSV file
5. Output: SMI data in a CSV file

The Calculate C3 Skeletal Muscle Index (SMI) step can be run by executing:

clinical.py

Please cite the following article if you use this code or pre-trained models:

@article{Ye2023DeepSarcopenia,
  title = {Development and Validation of an Automated Image-Based Deep Learning Platform for Sarcopenia Assessment in Head and Neck Cancer}, 
  author = {Z Ye, A Saraf, Y Ravipati, F Hoebers, P Catalano, Y Zha, A Zapaishchykova, J Likitlersuang, C Guthier, R Tishler, J Schoenfeld, D Margalit, R Haddad, R Mak, M Naser, K Wahid, J Sahlsten, J Jaskari, K Kaski, A Mäkitie, C Fuller, H Aerts, and B Kann},
  journal = {JAMA Network Open 6 (8), e2328280},
  year = {2023}, 
}

Code development, testing, refactoring and documentation: YR and ZY.

The code and data of this repository are provided to promote reproducible research. They are not intended for clinical care or commercial use.

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

The software is provided "as is", without warranty of any kind, express or implied, including but not limited to the warranties of merchantability, fitness for a particular purpose and noninfringement. In no event shall the authors or copyright holders be liable for any claim, damages or other liability, whether in an action of contract, tort or otherwise, arising from, out of or in connection with the software or the use or other dealings in the software.