xtudbxk / dsrg-tensorflow Goto Github PK
View Code? Open in Web Editor NEWa tensorflow version for DSRG (Weakly-Supervised Semantic Segmentation Network with Deep Seeded Region Growing)
a tensorflow version for DSRG (Weakly-Supervised Semantic Segmentation Network with Deep Seeded Region Growing)
Hi, thank you so much for your efforts to release such a beneficial repo.
DSRG-deeplab-aspp-vgg16 is workable. I tried to replace the vgg16 in your code with ResNet101 following https://github.com/zhengyang-wang/Deeplab-v2--ResNet-101--Tensorflow/blob/master/model.py, but I failed. The total loss seemed to be unchanged and the constrain loss was near 0 from the beginning. I couldn't figure out the problem. I wonder if you have a plan to release a DSRG-deeplab-aspp-resnet101 tf-version? Or maybe could you please kindly give me some tips how to adapt deeplab-aspp-resnet101 to your current framework? I'm looking forward to your reply. Many thanks!!!
training log: (I have edited: unary = -np.log(feat + 1e-5))
Restored model parameters from C:/Tensorflow/DSRG-tensorflow/resnet_v1_101.ckpt
start_time: 1565056349.789119
config -- lr:0.001000 weight_decay:0.000500 momentum:0.900000 batch_size:4.000000 epoches:32.000000
epoch:0.000000, iteration:0.000000, total_loss:27.615696
seed_loss:17.612474,constrain_loss:0.002580
epoch:0.003781, iteration:10.000000, total_loss:27.492531
seed_loss:18.424877,constrain_loss:0.000213
epoch:0.007561, iteration:20.000000, total_loss:27.569458
seed_loss:18.424877,constrain_loss:0.000341
epoch:0.011342, iteration:30.000000, total_loss:27.570873
seed_loss:18.424877,constrain_loss:-0.000000
epoch:0.015123, iteration:40.000000, total_loss:27.570740
seed_loss:18.424877,constrain_loss:0.000137
epoch:0.018904, iteration:50.000000, total_loss:22.964874
seed_loss:18.424877,constrain_loss:0.000025
epoch:0.022684, iteration:60.000000, total_loss:27.569254
seed_loss:18.424877,constrain_loss:-0.000000
epoch:0.026465, iteration:70.000000, total_loss:25.265747
seed_loss:16.122267,constrain_loss:-0.000000
epoch:0.030246, iteration:80.000000, total_loss:25.264816
seed_loss:18.424877,constrain_loss:-0.000000
epoch:0.034026, iteration:90.000000, total_loss:27.566475
seed_loss:18.424877,constrain_loss:-0.000000
epoch:0.037807, iteration:100.000000, total_loss:25.635326
seed_loss:18.424877,constrain_loss:-0.000000
epoch:0.041588, iteration:110.000000, total_loss:27.564554
seed_loss:18.424877,constrain_loss:-0.000000
epoch:0.045369, iteration:120.000000, total_loss:27.563591
seed_loss:18.424877,constrain_loss:-0.000000
epoch:0.049149, iteration:130.000000, total_loss:27.562630
seed_loss:13.819657,constrain_loss:-0.000000
epoch:0.052930, iteration:140.000000, total_loss:26.324894
seed_loss:18.424877,constrain_loss:-0.000000
epoch:0.056711, iteration:150.000000, total_loss:27.560703
seed_loss:18.424877,constrain_loss:-0.000000
epoch:0.060491, iteration:160.000000, total_loss:27.559740
seed_loss:18.424877,constrain_loss:-0.000000
epoch:0.064272, iteration:170.000000, total_loss:27.558777
seed_loss:18.424877,constrain_loss:-0.000000
epoch:0.068053, iteration:180.000000, total_loss:25.255207
seed_loss:18.424877,constrain_loss:-0.000000
hi @xtudbxk , thanks for your code,I have a question that how can I test my own image after train?
when I run the project something wrong happened, information as follows:
`tensorflow.python.framework.errors_impl.InvalidArgumentError: Cannot add tensor to the batch: number of elements does not match. Shapes are: [tensor]: [321,321,3], [batch]: [321,321,1]
[[node IteratorGetNext (defined at /workbench/DSRG-tensorflow-master/pythonlib/dataset_DSRG.py:109) ]]
Errors may have originated from an input operation.
Input Source operations connected to node IteratorGetNext:
IteratorV2 (defined at /workbench/DSRG-tensorflow-master/pythonlib/dataset_DSRG.py:108)
Original stack trace for 'IteratorGetNext':
File "/workbench/DSRG-tensorflow-master/DSRG.py", line 409, in
dsrg.train(base_lr=lr,weight_decay=5e-4,momentum=0.9,batch_size=batch_size,epoches=epoches)
File "/workbench/DSRG-tensorflow-master/DSRG.py", line 349, in train
data_x,data_y,data_tag,data_cues,id_of_image,iterator_train = self.data.next_batch(category="train",batch_size=batch_size,epoches=-1)
File "/workbench/DSRG-tensorflow-master/pythonlib/dataset_DSRG.py", line 109, in next_batch
img,gt,tag,cues,id_ = iterator.get_next()
File "/anaconda3/lib/python3.7/site-packages/tensorflow/python/data/ops/iterator_ops.py", line 426, in get_next
output_shapes=self._structure._flat_shapes, name=name)
File "/anaconda3/lib/python3.7/site-packages/tensorflow/python/ops/gen_dataset_ops.py", line 1947, in iterator_get_next
output_shapes=output_shapes, name=name)
File "/anaconda3/lib/python3.7/site-packages/tensorflow/python/framework/op_def_library.py", line 788, in _apply_op_helper
op_def=op_def)
File "/anaconda3/lib/python3.7/site-packages/tensorflow/python/util/deprecation.py", line 507, in new_func
return func(*args, **kwargs)
File "/anaconda3/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 3616, in create_op
op_def=op_def)
File "/anaconda3/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 2005, in init
self._traceback = tf_stack.extract_stack()
`
what should i do?
thank u
How could I get localization_cues-cal.pickle and init.model? @xtudbxk
I am trying to reproduce the results but I am getting the below error while training the model.
Traceback (most recent call last):
File "/home/ssindhu/deeplab_env/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1334, in _do_call
return fn(*args)
File "/home/ssindhu/deeplab_env/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1319, in _run_fn
options, feed_dict, fetch_list, target_list, run_metadata)
File "/home/ssindhu/deeplab_env/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1407, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Cannot batch tensors with different shapes in component 1. First element had shape [321,321,1] and element 2 had shape [321,321,3].
[[{{node IteratorGetNext}} = IteratorGetNext[output_shapes=[[?,321,321,3], [?,321,321,1], [?,21], [?,41,41,21], [?]], output_types=[DT_FLOAT, DT_UINT8, DT_FLOAT, DT_FLOAT, DT_STRING], _device="/job:localhost/replica:0/task:0/device:CPU:0"](IteratorV2)]]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "DSRG.py", line 407, in <module>
dsrg.train(base_lr=lr,weight_decay=5e-4,momentum=0.9,batch_size=batch_size,epoches=epoches)
File "DSRG.py", line 377, in train
self.sess.run(self.net["accum_gradient_accum"])
File "/home/ssindhu/deeplab_env/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 929, in run
run_metadata_ptr)
File "/home/ssindhu/deeplab_env/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1152, in _run
feed_dict_tensor, options, run_metadata)
File "/home/ssindhu/deeplab_env/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1328, in _do_run
run_metadata)
File "/home/ssindhu/deeplab_env/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1348, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Cannot batch tensors with different shapes in component 1. First element had shape [321,321,1] and element 2 had shape [321,321,3].
[[node IteratorGetNext (defined at DSRG.py:347) = IteratorGetNext[output_shapes=[[?,321,321,3], [?,321,321,1], [?,21], [?,41,41,21], [?]], output_types=[DT_FLOAT, DT_UINT8, DT_FLOAT, DT_FLOAT, DT_STRING], _device="/job:localhost/replica:0/task:0/device:CPU:0"](IteratorV2)]]
Caused by op 'IteratorGetNext', defined at:
File "DSRG.py", line 407, in <module>
dsrg.train(base_lr=lr,weight_decay=5e-4,momentum=0.9,batch_size=batch_size,epoches=epoches)
File "DSRG.py", line 347, in train
data_x,data_y,data_tag,data_cues,id_of_image,iterator_train = self.data.next_batch(category="train",batch_size=batch_size,epoches=-1)
File "/home/ssindhu/winter2020/DSRG-tensorflow/pythonlib/dataset_DSRG.py", line 109, in next_batch
img,gt,tag,cues,id_ = iterator.get_next()
File "/home/ssindhu/deeplab_env/lib/python3.6/site-packages/tensorflow/python/data/ops/iterator_ops.py", line 421, in get_next
name=name)), self._output_types,
File "/home/ssindhu/deeplab_env/lib/python3.6/site-packages/tensorflow/python/ops/gen_dataset_ops.py", line 2069, in iterator_get_next
output_shapes=output_shapes, name=name)
File "/home/ssindhu/deeplab_env/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "/home/ssindhu/deeplab_env/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 488, in new_func
return func(*args, **kwargs)
File "/home/ssindhu/deeplab_env/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3274, in create_op
op_def=op_def)
File "/home/ssindhu/deeplab_env/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1770, in __init__
self._traceback = tf_stack.extract_stack()
InvalidArgumentError (see above for traceback): Cannot batch tensors with different shapes in component 1. First element had shape [321,321,1] and element 2 had shape [321,321,3].
[[node IteratorGetNext (defined at DSRG.py:347) = IteratorGetNext[output_shapes=[[?,321,321,3], [?,321,321,1], [?,21], [?,41,41,21], [?]], output_types=[DT_FLOAT, DT_UINT8, DT_FLOAT, DT_FLOAT, DT_STRING], _device="/job:localhost/replica:0/task:0/device:CPU:0"](IteratorV2)]]
How can I download it ? Does it mean download CAM seed and put it in training/localization_cues folder?
After training 250 iterations, I meet a problem of RuntimeWarning: divide by zero encountered in log
unary = -np.log(feat)
after that, constrain_loss=0.000000, how could I colve this problem?
您好,我想问下您在readme文档中附带的trained model是指已经完全训练到达到DSRG原论文中一样的效果吗??因为我用您给的trained model和提供的用于评估的程序,跑出来的图分割效果跟DSRG论文中的分割效果不一样,所以想问一下是模型问题还是??
We are using Segmentation classes of Pascal VOC to generate Augmented images which inturn is used totrain the model which means that we are using fully supervised data isn't it??So how is the model weakly supervised?Please let me know
Hello! Your work is amazing! However, the checkpoint file is not included in the trained_model_for_dsrg_tf.tar.gz. Without it, I can not restore Tensorflow model. Could you please upload it? Thank you very much!
What are the initial weights "vgg16_deeplab_aspp.npy" from? Why does the state_dict contain fc8-SEC_1, fc8_SEC_2, fc8-SEC_3, fc8-SEC_4? Is that correct the initial weights initialise layers from Conv1 to fc7, and fc8_1, fc8_2, fc8_3 and fc8_4 are trained from scratch?
@xtudbxk .The model is for 21 categories.Can u please suggest methods to convert it to a 2 category model?
hi @xtudbxk , could you provide some detailed infomation about the loss curve of your result which is 0.564?
when I ran the code, the loss of epoch 0 is 1.0 and then it increased to 12, after that it decreased gradually and finally vibrated around 1.3, and MIOU on validation set is only 52.90(use single scale). Could you provide some suggestions? thanks~
1.想问vgg16_deeplab_aspp.npy是从 http://liangchiehchen.com/projects/DeepLabv2_vgg.html 这个链接里面的init.caffemodel转化得到的吗,这个init.caffemodel是只使用imagenet训过的吗,因为最后一层是21分类,有一些怀疑是训练了分割任务得到的模型权重
2.想问您使用过 https://github.com/speedinghzl/DSRG 里面给的vgg16_20M_mc.caffemodel进行初始化吗
谢谢
Hello, for the init.model vgg16_deeplab_aspp.npy
, is it converted from this link http://liangchiehchen.com/projects/DeepLabv2_vgg.html? I noticed that the miou of deeplab v2 can reach above 0.6 in the paper of deeplab v2 in the PASCAL VOC 2012. However, after trained using the DSRG model, the miou decrease to about 0.57. Why did the miou decrease using the DSRG model? Is there something wrong with my understandings? Thank you very much!
Hi,
First of all, thank you very much for providing the tf implementation.
I am trying to recreate the results in PyTorch, but I have very slow training time due to the consistency loss computation, which uses pydensecrf
, and it takes a lot of time for a moderately sized images (~ 356).
Do you have any recommendations on how to speed things up, and can you please provide the time it took to train the network for in your case.
Thank you very much.
Thanks very much for your code.
The script accepts arguments like 0,1,2,3 to specify multiple gpus.
However, only 1 gpu is actually used for computation.
Could you explain how to train with multiple gpus?
Thanks!
Hi, Thank you for preparing the tensorflow DSRG model. In your mark down, it was mentioned that there is the pretrained model which can be downloaded and loaded into the DSRG model. However, I am wondering is this model pretrained by imagenet (for normal classification task) or by Voc2012 (for semantic segmentation). I am not quite sure about the supervision, as the DSRG is the model designed for weakly supervised segmentation, I just want to make sure how much supervision provided by the pretrained weight. . Thanks.
Hi, @xtudbxk. Could you provide the evaluation code, thanks~
Hi,
Have you tried the retrain step, which can boost the performance by around 1.5%, as mentioned in the original DSRG paper?
I tried to implement the retrain step based on your code, but I got worse results compared to those without retrain. Sad..............................................................................................................
I tried to implement in pytorch , but during training , i met a problem that the BalancedSeedLoss will make softmax probability of one in all position , and it does not work in fg area ,and the loss will keep 8~9 . Have you met this problem?
how could I solve this problem? thx.
Hi,I encountered difficulty in download this localization_cues-cal.pickle file .I can't open the CAM url in DSRG[https://github.com/speedinghzl/DSRG], because I'm in China and I can't visit Google...
Could you provide me this file maybe in the form of BaiduNetDisk?
Thank you very much!!!
THX for UR work!
but I got an error
Traceback (most recent call last):
File "DSRG.py", line 411, in
dsrg.train(base_lr=lr,weight_decay=5e-4,momentum=0.9,batch_size=batch_size,epoches=epoches)
File "DSRG.py", line 371, in train
self.saver["lr"].save(self.sess,os.path.join(self.config.get("saver_path","saver"),"lr-%f" % base_lr),global_step=i)
AttributeError: 'DSRG' object has no attribute 'saver'
Hi, there some questions I'd like to ask you
hi
the download rate of data ' localization_cues-cal.pickle' in BaiduNetdisk is too low, so may I request some other way like qq mail to get that. My email: [email protected]. Thx.
Thank u for your work so much
It helps me a lot,but i have encountered an error
W tensorflow/core/framework/op_kernel.cc:1275] OP_REQUIRES failed at whole_file_read_ops.cc:114 : Not found: /data/VOCdevkit/VOC2012/JPEGImages/2008_002175.jpg; No such file or directory
W tensorflow/core/framework/op_kernel.cc:1275] OP_REQUIRES failed at whole_file_read_ops.cc:114 : Not found: /data/VOCdevkit/VOC2012/SegmentationClassAug/2008_002175.png; No such file or directory
I don't know why the error occurs
I want to use my own dataset to generate the corresponding pickle file, but I don't know how to do it. Can you help me to solve it
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.