Seize Disease

A project around histology with state of the art machine learning algorithms.

Seize Disease includes many models stretching out into the field of computer vision so that histology is more easily accessible all around the world. Altering some of these models has shown to be beneficial in terms of accuracy and reliability. On top of that, the Seize Disease Website includes a full analyzing tool in which one can upload and alter an histology image. In addition to that, the use and introduction of the OpenFlexure microscope into the project is planned.

The Website can be used to analyze and alter an image the following ways:

Tumor Detection
Tissue Detection
Location Detection
Analyzing nuclei:
- Semantic Segmentation (Multi Class)
- Semantic Segmentation (Binary)
- Border Segmentation
  - Nuclei counting
Microscopy Image Restoration (empathy on histology)
Microscopy Super Resolution (empathy on histology)
Glandular Morphology within Colon Histopathology Images (Binary)
Glandular Morphology within Colon Histopathology Images (Object Detection)

Get Started:

Local:

First, create a Conda or Pip environment for installing the necessary requirements. Once that's done, simply download the datasets and prepare them if necessary. You can find the links to the datasets in the references section.

# Using Pip
pip install -r requirements.txt

# Using Conda
conda create --name <env_name> --file requirements.txt

Colab:

If you're using Google Colab, open any project file and make sure you have the requirements.txt file in it. Then, run the following command:

!pip install -r requirements.txt

Results (evaluation: 30.09.2023):

first row → models shipped with this project
*changed/altered models
Custom Model X are new architectures, HistoSeg DP is also new, however, based on the HistoSeg Network

Nuclei Segmentation:

Multi-Class:

Dataset	Model	loss	dice score	pixel error	recall	precision	f1	iou	jaccard index	year
Pannuke	HrNetV2 + OCR*	1.2188	0.8814	0.0270	0.9047	0.9302	0.9172	0.7107	0.7892	/
...	SONNET	/	0.824	/	/	/	/	/	0.686	2022

Binary:

Dataset	Model	loss	dice score	pixel error	recall	precision	f1	iou	jaccard index
Pannuke	HrNetV2 + OCR*	0.2217	0.8558	0.07797	0.9117	0.8098	0.8573	0.7510	0.7484

Edge:

Dataset	Model	loss	dice score	pixel error	recall	precision	f1	iou	jaccard index
Pannuke	HrNetV2 + OCR*	1.0987	0.9388	0.0141	0.9578	0.9578	0.9578	0.8612	0.8850

Location Detection:

Dataset	Model	loss	accuracy	f1	precision	recall	specificity at sensitivity
Pannuke	TinyVit	0.0281	0.9955	0.9948	0.9966	0.9930	1.0

Tumor/Tissue Detection (Best Model only x/y):

Tumor + Tissue

Dataset	Model	loss	accuracy	f1	precision	recall	specificity at sensitivity	additional models	year
NCT-CRC-HE-100K	EfficientNetV2B2 (1/3)	0.0096	0.9975	0.9975	0.9979	0.9972	1.0	EfficientNetV2B1, EfficientNetV2B3	2021
...	Efficientnet-b0	/	0.9559	0.9748	0.9989	/	0.9945	/	2019
...	ResNeXt-50-32x4d	/	0.9546	0.9746	0.9991	/	0.9943	/	2021

Dataset	Model	loss	accuracy	f1	precision	recall	auc	specificity at sensitivity	additional models
Kather	ResAttInceptionV4* (1/2)	0.9002	0.9271	0.9180	0.9449	0.8963	0.9970	0.9999	ResAttInceptionV4* (smaller)

Tumor

Dataset	Model	loss	accuracy	f1	precision	recall	specificity at sensitivity	additonal models	Year
ICIAR2018_BACH_Challenge	TinyVit (1/3)	0.5071	0.9000	0.9274	0.9583	0.8984	1.0	EfficientNetV2B1, EffiencientNetV2B2	2022
...	Pretrained Resnet-101; Densenet-161	/	0.87	/	/	/	/	/	2018

Microscopy Image Restoration:

Dataset	Model	loss	PSNR	SSIM	Model Config
Custom (Mixed Microscopy Images)	NafNet (256x256)	0.0716	28.4709	0.837	filters = 16, middle_block_num = 2, encoder_block_nums = (1,1,1,28), decoder_block_nums= = (1,1,1,1), block_type = NAFBLOCK, drop_out_rate = 0.0
... sligtly worse quality images	NafNet (128x128)	0.0986	25.9457	0.7728	filters = 32, middle_block_num = 1, encoder_block_nums = (1,1,1,7), decoder_block_nums= = (1,1,1,1), block_type = NAFBLOCK, drop_out_rate = 0.05

Dataset	Model	loss	PSNR	SSIM	Model Config
Custom (Histology Images)	NafNet (256x256)	0.0647	30.9962	0.8412	filters = 16, middle_block_num = 2, encoder_block_nums = (1,1,1,28), decoder_block_nums= = (1,1,1,1), block_type = NAFBLOCK, drop_out_rate = 0.0
...	NafNet (128x128)	0.0587	29.622	0.8703	filters = 32, middle_block_num = 1, encoder_block_nums = (1,1,1,7), decoder_block_nums= = (1,1,1,1), block_type = NAFBLOCK, drop_out_rate = 0.05

Microscopy Image Super Resolution

Dataset	Model	loss	PSNR	SSIM	Additions	Resolution	Additional Models
Custom (Mixed Microscopy Images)	HAT (small)	0.0382	29.6236	0.8663	Images were also slightly blurred, compressed and noised	128 -> 256	HAT (Mid)
...	HAT (small)	0.0272	26.7771	0.9039	/	64 -> 128	/
...	HAT (small)	0.0623	23.7196	0.7859	/	64 -> 256	/

Glandular Morphology within Colon Histopathology Images

Binary Segmentation

Dataset	Model	loss	Dice/F1	Recall	Precison	Additional Models	Model Config
Colorectal Adenocarcinoma Gland (CRAG)	HistoSeg*	0.1567	0.8433	0.8018	0.8922	VitaeV2 + OCR*	backbone = "xception"
...	Custom Model L*	0.2397	0.9033	0.8895	0.9176	/	/
...	Custom Model S/M*	0.2724	0.854	0.8275	0.8849	/	/

...	HistoSeg DP*	0.6872	0.7306	0.6843	0.7836	/	based on HistoSeg mobilenetv2
...	HistoSeg*	0.5615	0.701	0.6589	0.7514	/	backbone = "mobilenetv2"

Binary Object Detection

Dataset	Model	loss
Colorectal Adenocarcinoma Gland (CRAG)	FasterRCNN RegNet Y 400MF	0.8131

Updates:

26.10.2023: New tiling feature for the website

References:

Datasets used:

Kather Texture 2016 Image Tiles: Download here
ICIAR 2018 BACH: Download here
PanNuke: Download here, Paper
NCT-CRC-HE-100K: Download here
CPM 15, CPM 17: Download here
TNBC: Download here
Kumar: Download here
CoNSeP: Download here, Paper
"Potato Tuber" included in Super Resolution/Restoration Dataset only : Download here
Malaria Bounding Boxes included in Super Resolution/Restoration Dataset only: Download here
Restoration/Super Resolution Dataset (self-made out of all of the above datasets)

The Custom datasets can be acquired from my drive:

Code used:

HRNetV2 Backbone: https://github.com/noelcodella/HRNetV2_keras_tensorflow_semisupervised @Noel Codella
OCR: https://github.com/Burf/HRNetV2-OCR-Tensorflow2 @Burf
Efficient NetV2: https://github.com/leondgarse/keras_efficientnet_v2 @leondgarse
Data Preparation for Merged Dataset: https://github.com/nauyan/NucleiSegmentation @ nauyan
Data Preparation for PanNuke: https://github.com/Mr-TalhaIlyas/Prerpcessing-PanNuke-Nuclei-Instance-Segmentation-Dataset @Mr-TalhaIlyas
Mish Activation Function: https://www.kaggle.com/code/imokuri/mish-activation-function/notebook @SUGIYAMA Yoshio
Segment Anything: https://segment-anything.com/ @Meta AI, Github
Tiny Vit: https://github.com/taki0112/vit-tensorflow/tree/main @taki0112
StarDist: https://github.com/stardist/stardist @StarDist
NafNet: https://github.com/dslisleedh/NAFNet-tensorflow2 @dslisleedh
NafNet/Restorer and SR Losses: https://github.com/TrystAI/restorers @TrystAI
HAT: https://github.com/XPixelGroup/HAT @ XPixelGroup
FasterRCNN: https://github.com/sovit-123/fasterrcnn-pytorch-training-pipeline/tree/main @sovit123
VitaeV2: https://github.com/ViTAE-Transformer/ViTAE-Transformer/tree/main
HistoSeg: https://github.com/saadwazir/HistoSeg @saadwazir
Efficient Attention: https://github.com/cmsflash/efficient-attention @Shen Zhuoran

Comparisions:

Pannuke Dataset Comparisions: https://paperswithcode.com/dataset/pannuke | SONNET
NCT-CRC-HE-100K Dataset Comparisions: https://paperswithcode.com/sota/medical-image-classification-on-nct-crc-he | EfficientNet
ICIAR 2018 BACH Dataset Comparisions: https://figshare.com/articles/dataset/Results_on_the_ICIAR_2018_challenge_dataset_/19676718/1; https://arxiv.org/pdf/1808.04277.pdf

lorenzrutkevich / seize-disease Goto Github PK

seize-disease's Introduction

Seize Disease

A project around histology with state of the art machine learning algorithms.

The Website can be used to analyze and alter an image the following ways:

Tumor Detection

Tissue Detection

Location Detection

Analyzing nuclei:

Semantic Segmentation (Multi Class)

Semantic Segmentation (Binary)

Border Segmentation

Nuclei counting

Microscopy Image Restoration (empathy on histology)

Microscopy Super Resolution (empathy on histology)

Glandular Morphology within Colon Histopathology Images (Binary)

Glandular Morphology within Colon Histopathology Images (Object Detection)

Get Started:

Local:

First, create a Conda or Pip environment for installing the necessary requirements. Once that's done, simply download the datasets and prepare them if necessary. You can find the links to the datasets in the references section.

Colab:

If you're using Google Colab, open any project file and make sure you have the requirements.txt file in it. Then, run the following command:

Results (evaluation: 30.09.2023):

Nuclei Segmentation:

Location Detection:

Tumor/Tissue Detection (Best Model only x/y):

Microscopy Image Restoration:

Microscopy Image Super Resolution

Glandular Morphology within Colon Histopathology Images

Updates:

26.10.2023: New tiling feature for the website

References:

Datasets used:

Kather Texture 2016 Image Tiles: Download here

ICIAR 2018 BACH: Download here

PanNuke: Download here, Paper

NCT-CRC-HE-100K: Download here

CPM 15, CPM 17: Download here

TNBC: Download here

Kumar: Download here

CoNSeP: Download here, Paper

"Potato Tuber" included in Super Resolution/Restoration Dataset only : Download here

Malaria Bounding Boxes included in Super Resolution/Restoration Dataset only: Download here

Restoration/Super Resolution Dataset (self-made out of all of the above datasets)

The Custom datasets can be acquired from my drive:

Code used:

HRNetV2 Backbone: https://github.com/noelcodella/HRNetV2_keras_tensorflow_semisupervised @Noel Codella

OCR: https://github.com/Burf/HRNetV2-OCR-Tensorflow2 @Burf

Efficient NetV2: https://github.com/leondgarse/keras_efficientnet_v2 @leondgarse

Data Preparation for Merged Dataset: https://github.com/nauyan/NucleiSegmentation @ nauyan

Data Preparation for PanNuke: https://github.com/Mr-TalhaIlyas/Prerpcessing-PanNuke-Nuclei-Instance-Segmentation-Dataset @Mr-TalhaIlyas

Mish Activation Function: https://www.kaggle.com/code/imokuri/mish-activation-function/notebook @SUGIYAMA Yoshio

Segment Anything: https://segment-anything.com/ @Meta AI, Github

Tiny Vit: https://github.com/taki0112/vit-tensorflow/tree/main @taki0112

StarDist: https://github.com/stardist/stardist @StarDist

NafNet: https://github.com/dslisleedh/NAFNet-tensorflow2 @dslisleedh

NafNet/Restorer and SR Losses: https://github.com/TrystAI/restorers @TrystAI

HAT: https://github.com/XPixelGroup/HAT @ XPixelGroup

FasterRCNN: https://github.com/sovit-123/fasterrcnn-pytorch-training-pipeline/tree/main @sovit123

VitaeV2: https://github.com/ViTAE-Transformer/ViTAE-Transformer/tree/main

HistoSeg: https://github.com/saadwazir/HistoSeg @saadwazir

Efficient Attention: https://github.com/cmsflash/efficient-attention @Shen Zhuoran

Comparisions:

Pannuke Dataset Comparisions: https://paperswithcode.com/dataset/pannuke | SONNET

NCT-CRC-HE-100K Dataset Comparisions: https://paperswithcode.com/sota/medical-image-classification-on-nct-crc-he | EfficientNet

ICIAR 2018 BACH Dataset Comparisions: https://figshare.com/articles/dataset/Results_on_the_ICIAR_2018_challenge_dataset_/19676718/1; https://arxiv.org/pdf/1808.04277.pdf

Further Sources and Citations can be found in the code itself.

seize-disease's People

Contributors

Stargazers

Watchers

Recommend Projects

Recommend Topics

Recommend Org