xtudbxk / sec-tensorflow Goto Github PK

a tensorflow version for SEC approach in the paper "seed, expand and constrain: three principles for weakly-supervised image segmentation".

Python 100.00%

tensorflow sec semantic-segmentation weakly-supervised seed-expand-constraints

sec-tensorflow's Introduction

#SEC-tensorflow version

Introduction

This is a project which just move the SEC-caffe to SEC-tensorflow. The SEC is referring to the approach for weakly-supervised semantic segmentation in the paper "seed, expand and constrain: three principles for weakly-supervised image segmentation". And here, I just use the tensorflow to implement the approach with the help of the SEC-caffe project.

Citing this repository

If you find this code useful in your research, please consider citing them:

@inproceedings{kolesnikov2016seed,

title={Seed, Expand and Constrain: Three Principles for Weakly-Supervised Image Segmentation},

author={Kolesnikov, Alexander and Lampert, Christoph H.},

booktitle={European Conference on Computer Vision ({ECCV})},

year={2016},

organization={Springer}

}

Preparation

for using this code, you have to do something else:

1. Install pydensecrf

For using the densecrf in python, we turn to the project pydensecrf. And you just using the following code to install it.

pip install pydensecrf

note: from the page of pydensecrf, maybe you should upgrade the cython to a newer version befor install it.

2. Download the data and model

for pascal data, please referring to its official website and to the augmental SBD data. Just download it and extract it in the data/, then 'cd data' and run convert.py with 'python convert.py'.
for localization_cues.pickle, please referring to SEC-caffe. And download it and extract in the data/ (don't forget to uncompress the data by "gzip -kd localization_cues.pickle.gz").
for the init.npy, I upload a converted file in google driver, just download it and put it in the model/ . And those weights in the file is exactly the same with the vgg16_20M.caffemodel in SEC-caffe.

For more details, you can referring to the correspond code files or leave a message in the issue.

3. Be careful about the versions of python and tensorflow

We just only test on python3.7 and tensorflow 1.14.0.

Training

then, you just input the following sentence to train it.

python SEC.py <gpu_id>

Evaluation

I just release a project to provide the code for evaluation.

sec-tensorflow's People

Contributors

Stargazers

Watchers

Forkers

johndpope bityangke sfd198977 hust-wayne mathpopo xbutterflyx rosanajurdi cyberspacefighter ws-liu

sec-tensorflow's Issues

Tensorflow version

Have you tried to generate the localization cues by yourself

Have you tried to generate the localization cues by yourself ? I mean, to train a multi label classification network and generate the initial seeds using CAM

Error while loading localisation_cues.pickle

Hi, While loading cues file as instructed, i am facing following issue---->
_pickle.UnpicklingError: invalid load key, '\x0a'.

Test Code

I am a beginner, can you provide the test code?For example, input an image, load the trained model, and then output the segmented image.

Problem about the final result

Hi,xtudbxk:

I trained a model by using the training code, and evaluated on https://github.com/xtudbxk/semantic-segmentation-metrics. But finally, I get mIoU=0.471 which is lower than 0.502. In training code, i didn't change any thing, the final model is "norm-77999". In evaluation code, I changed the network

def create_network(self):
    if "init_model_path" in self.config:
        self.load_init_model()
    with tf.name_scope("vgg") as scope:
        # build block
        block = self.build_block("input",["conv1_1","relu1_1","conv1_2","relu1_2","pool1"])
        block = self.build_block(block,["conv2_1","relu2_1","conv2_2","relu2_2","pool2"])
        block = self.build_block(block,["conv3_1","relu3_1","conv3_2","relu3_2","conv3_3","relu3_3","pool3"])
        block = self.build_block(block,["conv4_1","relu4_1","conv4_2","relu4_2","conv4_3","relu4_3","pool4"])
        block = self.build_block(block,["conv5_1","relu5_1","conv5_2","relu5_2","conv5_3","relu5_3","pool5","pool5a"])
        fc1 = self.build_fc(block,["fc6_1","relu6_1","drop6_1","fc7_1","relu7_1","drop7_1","fc8_1"], dilate_rate=6)
        fc2 = self.build_fc(block,["fc6_2","relu6_2","drop6_2","fc7_2","relu7_2","drop7_2","fc8_2"], dilate_rate=12)
        fc3 = self.build_fc(block,["fc6_3","relu6_3","drop6_3","fc7_3","relu7_3","drop7_3","fc8_3"], dilate_rate=18)
        fc4 = self.build_fc(block,["fc6_4","relu6_4","drop6_4","fc7_4","relu7_4","drop7_4","fc8_4"], dilate_rate=24)
        #self.net["fc8"] = (self.net[fc1]+self.net[fc2]+self.net[fc3]+self.net[fc4])/4.0
        self.net["fc8"] = self.net[fc1]+self.net[fc2]+self.net[fc3]+self.net[fc4]
        return self.net["fc8"] # note that the output is a log-number

to

def create_network(self):
    if "init_model_path" in self.config:
        self.load_init_model()
    with tf.name_scope("vgg") as scope:
        # build block
        block = self.build_block("input",["conv1_1","relu1_1","conv1_2","relu1_2","pool1"])
        block = self.build_block(block,["conv2_1","relu2_1","conv2_2","relu2_2","pool2"])
        block = self.build_block(block,["conv3_1","relu3_1","conv3_2","relu3_2","conv3_3","relu3_3","pool3"])
        block = self.build_block(block,["conv4_1","relu4_1","conv4_2","relu4_2","conv4_3","relu4_3","pool4"])
        block = self.build_block(block,["conv5_1","relu5_1","conv5_2","relu5_2","conv5_3","relu5_3","pool5","pool5a"])
        fc = self.build_fc(block, ["fc6", "relu6", "drop6", "fc7", "relu7", "drop7", "fc8"])
        return self.net[fc] # note that the output is a log-number

and I also change build_fc

def build_fc(self,last_layer, layer_lists,dilate_rate):
    for layer in layer_lists:
        if layer.startswith("fc"):
            with tf.name_scope(layer) as scope:
                weights,bias = self.get_weights_and_bias(layer)
                if layer.startswith("fc6"):
                    self.net[layer] = tf.nn.atrous_conv2d( self.net[last_layer], weights, rate=dilate_rate, padding="SAME", name="conv")

                else:
                    self.net[layer] = tf.nn.conv2d( self.net[last_layer], weights, strides = [1,1,1,1], padding="SAME", name="conv")
                self.net[layer] = tf.nn.bias_add( self.net[layer], bias, name="bias")
                last_layer = layer
        if layer.startswith("batch_norm"):
            with tf.name_scope(layer) as scope:
                self.net[layer] = tf.contrib.layers.batch_norm(self.net[last_layer])
                last_layer = layer
        if layer.startswith("relu"):
            with tf.name_scope(layer) as scope:
                self.net[layer] = tf.nn.relu( self.net[last_layer])
                last_layer = layer
        if layer.startswith("drop"):
            with tf.name_scope(layer) as scope:
                self.net[layer] = tf.nn.dropout( self.net[last_layer],self.net["drop_prob"])
                last_layer = layer

    return last_layer

to

def build_fc(self, last_layer, layer_lists):
    for layer in layer_lists:
        if layer.startswith("fc"):
            with tf.name_scope(layer) as scope:
                weights,bias = self.get_weights_and_bias(layer)
                if layer.startswith("fc6"):
                    self.net[layer] = tf.nn.atrous_conv2d( self.net[last_layer], weights, rate=12, padding="SAME", name="conv")

                else:
                    self.net[layer] = tf.nn.conv2d( self.net[last_layer], weights, strides = [1,1,1,1], padding="SAME", name="conv")
                self.net[layer] = tf.nn.bias_add( self.net[layer], bias, name="bias")
                last_layer = layer
        if layer.startswith("batch_norm"):
            with tf.name_scope(layer) as scope:
                self.net[layer] = tf.contrib.layers.batch_norm(self.net[last_layer])
                last_layer = layer
        if layer.startswith("relu"):
            with tf.name_scope(layer) as scope:
                self.net[layer] = tf.nn.relu( self.net[last_layer])
                last_layer = layer
        if layer.startswith("drop"):
            with tf.name_scope(layer) as scope:
                self.net[layer] = tf.nn.dropout( self.net[last_layer],self.net["drop_prob"])
                last_layer = layer
    return last_layer

Is there angthing wrong? Thank you.

Trained Model, Please

Thanks for the tensorflow of SEC! I did the training, but the predicted image were all black images. I guess there might be something wrong of my dataset. Therefore, it would be helpful if you could provide the trained model also?

the Training Net work and the Test Network

Are the Training Net work and the Test Network different? I can't restore the norm-38999

Segmentation fault (core dumped) on self.sess.run(self.net["accum_gradient_accum"], feed_dict=params)

Thanks for the wonderful work.

I'm trying to execute the Python script, but I am getting the following error:

$ python SEC.py 1
len:{'train': 10582}
name: Tesla K80 major: 3 minor: 7 memoryClockRate(GHz): 0.8235
pciBusID: 0000:84:00.0
totalMemory: 11.17GiB freeMemory: 11.10GiB
load init model success: ./model/init.npy
start_time: 1543842539.624039
config -- lr:0.001000 weight_decay:0.000050 momentum:0.900000 batch_size:4.000000 epoches:30.000000
Segmentation fault (core dumped)

I'm using python 2.7.13 and TensorFlow 1.4.1.

Can someone tell me why this happens, and how can I solve?

"model_path":"./20180622-3-4/final-0",

I didn't find the model in your Evaluation project.

How to produce my own localization_cues.pickle?

I need to use weak supervised learning to segment my own medical image data. How do I produce my own localization_cues.pickle? Can you provide the code?

Segmentation fault with pydensecrf

Hi,

I am trying to install the pydensecrf module with pip install. The library seemed to be installed successfully since it can import normally. My tensorflow version is gpu-1.5. But when I tried to run "python SEC.py 0", it get a segmentation fault. I used "gdb python" to locate error is in "pydensecrf/densecrf/src/permutohedral.cpp: 255 : No such files".

I would really appreciate if anyone could help me with this.