zfturbo / segmentation_models_3d Goto Github PK

View Code? Open in Web Editor NEW

115.0 4.0 26.0 66 KB

Set of models for segmentation of 3D volumes

License: MIT License

Python 100.00%

3d-segmentation 3d-semantic-segmentation keras segmentation

segmentation_models_3d's Introduction

Segmentation models 3D Zoo for Keras 3

The repository contains 3D variants of popular models for segmentation like FPN, Unet, Linknet and PSPNet.

This repository is based on great segmentation_models repo by @qubvel

Available architectures:

Installation

pip install segmentation-models-3D

Examples

Loading model:

import segmentation_models_3D as sm

model1 = sm.Unet(
    'resnet34', 
    encoder_weights='imagenet'
)

# binary segmentation (these parameters are default when you call Unet('resnet34')
model2 = sm.FPN(
    'densenet121', 
    classes=1, 
    activation='sigmoid'
)

# multiclass segmentation with non overlapping class masks (your classes + background)
model3 = sm.Linknet(
    'resnet34', 
    classes=3, 
    activation='softmax'
)

# multiclass segmentation with independent overlapping/non-overlapping class masks
model4 = sm.PSPNet(
    'resnet34', 
    classes=3,
    activation='sigmoid'
)

# If you need to specify non-standard input shape
model5 = sm.Unet(
    'resnet50', 
    input_shape=(96, 128, 128, 6), 
    encoder_weights=None
)

All possible backbones: 'resnet18, 'resnet34', 'resnet50', 'resnet101', 'resnet152', 'seresnet18', 'seresnet34', 'seresnet50', 'seresnet101', 'seresnet152', 'seresnext50', 'seresnext101', 'senet154', 'resnext50', 'resnext101', 'vgg16', 'vgg19', 'densenet121', 'densenet169', 'densenet201', 'inceptionresnetv2', 'inceptionv3', 'mobilenet', 'mobilenetv2', 'efficientnetb0', 'efficientnetb1', 'efficientnetb2', 'efficientnetb3', 'efficientnetb4', 'efficientnetb5', 'efficientnetb6', 'efficientnetb7', 'efficientnetv2-b1', 'efficientnetv2-b2', 'efficientnetv2-b3', 'efficientnetv2-s', 'efficientnetv2-m', 'efficientnetv2-l'

More examples can be found in:

Training model:

There is training example in training_example_tensorflow.py

I tried to keep code as simple as possible
I couldn't find good dataset for 3D segmentation task. So I randomly generate 3D volumes with dark background with light figures (spheres and cuboids) and model tries to segment these figures independetly. 1st mask for circles and 2nd mask for cuboids.

To Do List

Add stride_size parameter for better control of models

Related repositories

https://github.com/qubvel/classification_models - original classification 2D repo
https://github.com/qubvel/segmentation_models - original segmentation 2D repo
classification_models_3D - models for classification in 3D
segmentation_models_pytorch_3d - models for segmentation in 3D for Pytorch
volumentations - 3D augmentations

Unresolved problems

There is no 'bilinear' interpolation for UpSample3D layer, so it uses Nearest Neighbour upsampling.

Older versions

Last version which supports Keras 2 is 1.0.7

pip install segmentation-models-3D==1.0.7

Citation

For more details, please refer to the publication: https://doi.org/10.1016/j.compbiomed.2021.105089

If you find this code useful, please cite it as:

@article{solovyev20223d,
  title={3D convolutional neural networks for stalled brain capillary detection},
  author={Solovyev, Roman and Kalinin, Alexandr A and Gabruseva, Tatiana},
  journal={Computers in Biology and Medicine},
  volume={141},
  pages={105089},
  year={2022},
  publisher={Elsevier},
  doi={10.1016/j.compbiomed.2021.105089}
}

segmentation_models_3d's People

Contributors

Stargazers

Watchers

segmentation_models_3d's Issues

not working for tensorflow 2x

not able to install for TensorFlow 2x. please let us know the requirements with versions

input dimension

my input data has the shape of (118, 224, 224, 35), 118 3D nii images, width: 224, height: 224 and depth (#of slices) 35. My question is what would be the correct order of input shape? is the order of (#images, width, height, #slices) correct? Some examples use the shape of (#images, width, height, #slices, 1). Should I set the last dimension to 1 as channel dimension?

different # of slices in volume

Is it necessary for volumes to be in the same size. Is it possible to use these models for a datasets that volumes have different number of slices? I am working on a 3d liver volume segmentation and my volumes have fixed width and heights but different number of slices in z axis. for example first volume is in shape 512, 512, 26 and the second one is 512, 512, 32. So, my question is, should I fixed the number of slices for the third dimension for the input_shape?

Unable to load mobilenetv2 version

Hello, when I try to load a model which used a mobilenetv2 backbone:

model = tensorflow.keras.models.load_model(MobileNetModel.h5',compile=False)

I get this issue:

ValueError: Unknown layer: DepthwiseConv3D. Please ensure this object is passed to the custom_objects argument. See https://www.tensorflow.org/guide/keras/save_and_serialize#registering_the_custom_object for details.

Any ideas on how to fix this?

Index Error

IndexError Traceback (most recent call last)
in <cell line: 65>()
63
64 # Initialize Model, Loss Function, and Optimizer
---> 65 model = smp.Unet(
66 'resnet34',
67 encoder_weights=None,

7 frames
/usr/local/lib/python3.10/dist-packages/keras/src/layers/normalization/batch_normalization.py in build(self, input_shape)
168
169 def build(self, input_shape):
--> 170 shape = (input_shape[self.axis],)
171 if self.scale:
172 self.gamma = self.add_weight(

IndexError: tuple index out of range. Please help me out for this error, My image is of shape (256,128,128)

import error

Create EfficientNetV2 as bacbone for Unet

I find creating EfficientNetV2 versions as bacbone for Unet, if possible could please help me out in this situation

No module named 'keras.engine'.

im getting this error while importing 'import segmentation_models_3D as sm'

tensorflow2.12 version

Hello, the tensorflow-gpu2.8 version has been installed in my virtual environment. During the installation of your segmentation-models-3d, the tensorflow2.12 version will be downloaded to me automatically. tensorflow2.12 does not support gpu under windows. How to solve it?

I am getting error on importing this library segmentation_models_3D

I even installed the library 'segmentation_models_3D' successfully.

import segmentation_models_3D as sm
On importing above i get something like this error
cannot import name 'deserialize_keras_object' from 'keras.utils.generic_utils'

Does anyone have a solution please help me?

activations

Hi, can I change the activations "relu" to "prelu" for example?

Thanks

Unable to create basic model

Hello, I have the following environments:

tensorflow==1.15.0 and keras==2.3.1 and python==3.6

When I try to run the basic line:

model1 = sm.Unet('resnet34', encoder_weights='imagenet')

I get this error:

File "", line 1, in
File "/Users/karl.landheer/opt/anaconda3/envs/Seg3D/lib/python3.6/site-packages/segmentation_models_3D/init.py", line 34, in wrapper
return func(*args, **kwargs)
File "/Users/karl.landheer/opt/anaconda3/envs/Seg3D/lib/python3.6/site-packages/segmentation_models_3D/models/unet.py", line 231, in Unet
**kwargs,
File "/Users/karl.landheer/opt/anaconda3/envs/Seg3D/lib/python3.6/site-packages/segmentation_models_3D/backbones/backbones_factory.py", line 102, in get_backbone
model = model_fn(*args, **kwargs)
File "/Users/karl.landheer/opt/anaconda3/envs/Seg3D/lib/python3.6/site-packages/classification_models_3D/models_factory.py", line 74, in wrapper
return func(*args, **new_kwargs)
File "/Users/karl.landheer/opt/anaconda3/envs/Seg3D/lib/python3.6/site-packages/classification_models_3D/models/resnet.py", line 321, in ResNet34
**kwargs
File "/Users/karl.landheer/opt/anaconda3/envs/Seg3D/lib/python3.6/site-packages/classification_models_3D/models/resnet.py", line 231, in ResNet
x = layers.BatchNormalization(name='bn_data', **no_scale_bn_params)(img_input)
File "/Users/karl.landheer/opt/anaconda3/envs/Seg3D/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/base_layer.py", line 824, in call
self._maybe_build(inputs)
File "/Users/karl.landheer/opt/anaconda3/envs/Seg3D/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/base_layer.py", line 2146, in _maybe_build
self.build(input_shapes)
File "/Users/karl.landheer/opt/anaconda3/envs/Seg3D/lib/python3.6/site-packages/tensorflow_core/python/keras/layers/normalization.py", line 289, in build
raise ValueError('Invalid axis: %d' % x)

I have tried many permutations of different versions of keras and tensorflow, but to no avail. Any ideas?

Baseline models

Hello,

It's a great piece of work that you implement various of 3d segmentation models with different backbones.

I'm thinking if you use a naked segmentation model without any backbone as a baseline would be a great perfection.

Just a small suggestion.

Have a nice day!

efficientnetb3 cannot run

i tried using efficientnetb3 with unet, it was created but when i try to either fit or predict, it just do not response and code stop there without doing anything at all.

i am using NVIDIA RTX 3070 8G GPU

Unable to create model with shape(240,240,155,3)

pleeeease help !!!!

Error during .fit in latest version.

Greetings.

I tried using your library to perform a 3D semantic segmentation. I attempted to follow Sreeni's (https://www.youtube.com/watch?v=Dt73QWZQck4) tutorial, but I kept running into this error during the fit.

Exception has occurred: AttributeError
module 'keras.api.ops' has no attribute 'pow'
File "C:\Users\Pineirin\Desktop\3DCNN\3D_UNET.py", line 72, in
model.fit(
AttributeError: module 'keras.api.ops' has no attribute 'pow'

I'm using your lates version, after installing the tensorflow, keras and classification_models_3D version defined in your requirements.txt.

I also attempted to use your library myself, as i guess the problem is with the updates, using a chatbot to aid me. But I ran into exactly the same error. I'm using Windows 10 and 11.

Sorry if I'm bothering with an issue that is due to my error, but I feel that if I reach the same error from two different paths it may be an error of the library.

Please let me know if you need any additional information.

3D Instance segmentation

Hi,
is it possible to modify this repo for instance segmentation? I would like to segment single class and differentiate objects in slices of 3D volume. please advise

thank you

Correct Tensorflow/Keras Versions

Hello!
Love the library, it's been super useful for a segmentation project I've been working on. I am currently trying to run the model on CUDA 12, and I saw that in the latest version, 1.0.6, you guys are able to support higher versions of tensorflow (2.13 or above). Currently, when I try to install it with pip, however, I get the following dependency error:
segmentation-models-3d 1.0.6 requires keras==2.8.0, but you have keras 2.13.1 which is incompatible.
segmentation-models-3d 1.0.6 requires tensorflow==2.8.0, but you have tensorflow 2.13.1 which is incompatible.

I can't downgrade my CUDA version for my GPU to make it compatible with tensorflow==2.8.0. Would it be possible to have some help?
Thank you so much!

Unequal Dataset Dimension

I have a spectral cube dataset of shape (100,256,256,50).
There are 100 images of 256 by 256 of 50 dimensions. I want to use such kind of data on the 3D network but according to my understanding the size of height, width and dimensions should be equal otherwise the model won't compile. Please guide what to do in this scenario. Thank you.

Haussdorf Distance percentile 95

Hi ZFTurbo,

Do you have plains of implement HDF95 distance for evaluate models? Or, do you know some repo that has it implemented? I need it but I don't have time to think about how to implement it myself :( If you know, please help!

Thank you in advance!

About the final prediction head

Hi , when reading your code about the model head (define number of output classes) ,I have some question about this. In 2D segmentation task , we usually use Conv2D with kernel size 1 to get the final segmentaion mask. However, in this library, I notice that Conv3D with kernel size 3 has been used to get the final prediction rather than kernel size 1. I'm just wondering is there difference between Conv3D with kernel size 3 and 1 to get the prediction in 3D segmentation task?

"URL Fetch Failure" on executing training_example.py

Hey, I'm trying to execute the program but as the program executes, it fails and i get this exception

Exception: URL fetch failure on https://github.com/ZFTurbo/classification_models_3D/releases/download/v1.0.4/resnet18_inp_channel_3_tch_0_top_False.h5: None -- [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:992)

How do i resolve it? Cheers.

ImportError

Hi, I work in google colab, try to import segmentation_models_3D as sm, but I get an ImportError: cannot import name 'base' from 'segmentation_models_3D'. I installed keras >= 2.2.0, tensorflow >= 1.13.1, keras_applications >= 1.0.7, classification_models_3D, efficientnet_3D, segmentation_models_3D. What should I do to solve my problem?

how does the 3d cnn pretrained on ImageNet?

Hello,

As we know that many 2d CNN architectures have pretrained weights thanks to ImageNet or other huge datasets.

I just wonder does the 3d CNN architecture's backbone in this repo have pretrained weights? If it does, how it comes?

Thank you in advance!

Jing

Manual Input Shape Error

Every time i use inceptionresnetv2, inceptionv3
with input_shape = [4, 256, 256, 1]
it give error

Update 1.0.7 is not suitable for windows

The requirement is 'tensorflow>=2.13.0, <=2.15.0'.
However, for windows the GPU is not available for tensorflow>=2.10.0

Propose README

Hi ZFTurbo,

It is very important to realize that backbones expects inputs between 0, 255. So I think that is a good idea to specify it in the README.md file, for avoid unexpected behaviour.

I noticed this because if I didn't use Batch Normalization and I used RESNET, when my data was in the range 0, 1, as RESNET subtracts the average (which is around 120), all the data stayed at -120.something and the network didn't learn anything. If you use batchnormalization this is solved because it re-normalizes the batch, but if you don't use it the nets won't train.

ValueError: axes don't match array

Hi, help please! :(

I try to create a simple model like

import segmentation_models_3D as sm
import os
sm.set_framework('tf.keras')
os.environ["KERAS_BACKEND"] = "tensorflow"
model1 = sm.Unet(backbone_name="resnet50", input_shape=(32, 96, 96, 1), encoder_weights="imagenet")

but it says ValueError: axes don't match array.
I tried #8
to make os.environ["KERAS_BACKEND"] = "tensorflow" as above shows and nothing!

The tst_keras.py file works for me, I don't understand nothing! (I'm trying to run the code at the beggining in Visual Studio Code, .ipynb file).

Please :(

Import Error for segmentation_models_3d as sm (TF version: 2.12.0)

I have TensorFlow version 2.12.0 and since Keras has been now a part of tensor flow, I am not installing Keras explicitly. The pip install command is running pretty well but on importing the segmentations_models_3d, I am receiving error saying Keras.engine module not found. I tried troubleshooting here and there and also tried with different versions of TF and Keras but of no success. Can anyone help?