zju3dv / snake Goto Github PK
View Code? Open in Web Editor NEWCode for "Deep Snake for Real-Time Instance Segmentation" CVPR 2020 oral
License: Other
Code for "Deep Snake for Real-Time Instance Segmentation" CVPR 2020 oral
License: Other
Can I ask from more function or explain?
I have read the paper for many times.
But I still can't understand how to turn contour(blue) into Object shape(green).
Thanks
@pengsida
Iam having problem while compiling the library.
Have you met these errors when compiling ?
I`m interested in this project. And tried to deploy it, but visualisation failing every time in the same point.
This error could be healed by downgrading tqdm to 4.19 instead ~4.25 or upgrading to 4.40+. But there are strange another error in same place
Do you know anything about this?
I use nvidia tesla, cuda-10, i`ve compiled apex and _ext from install.md instruction.
And pytorch is convinient to cuda version: https://download.pytorch.org/whl/cu100/torch-1.0.0-cp36-cp36m-linux_x86_64.whl
When opening the source code
Thanks for your contribution !
How to change the code about the number of dataset.
And could you tell me the design mode. I 'm missing in same code.
@pengsida
Hi there,
it would be very interesting testing the benefits of SNAKE in text detection.
Have you ever considered that.
运行测试指令出现问题,好像是说模型损坏?
vignette: “Figure 7 shows some qualitative results on Kins. ” in page 7 should be
“Figure 6 (row 3) shows some qualitative results on Kins. ”
I am confused
how to understand amodal and inmodal or modal in accurate chinese?
感谢作者优秀的工作!
请问,能否写个输入是boxes 和 image时的demo?
是否仅需要修改ct_snake.py的forward函数的
ct, detection = self.decode_detection(output, cnn_feature.size(2), cnn_feature.size(3))
这条语句,由外部输入的boxes来生成ct。
谢谢!
Excellent and inspiring work it is!
But I wander how to train and apply the algorithm on
my own instance segmentation dataset ?
I want to use this algorithm for cells and nucleus instance segmentation. The instance is annotated
with .png
format or the widely used .json
.
I have download and carefully reviewed your code and project structure but it is too complicated and there is little comment about how the code works or the return result of the function.
Can you give some instructions or tutorials like the popular used repo: https://github.com/matterport/Mask_RCNN
I think it will be a nice work and inspre more wonderful ideas. Thank for your contribution to the community again.
I only find component detector in training code for cityscapes dataset.
Windows10
Python3.7
cd dcn_v2
python setup.py build_ext --inplace
cd ../extreme_utils
python setup.py build_ext --inplace
when I enter
python setup.py build_ext --inplace
I got diffirent error from each of them
cd dcn_v2:
dcn_v2_cuda.obj : error LNK2001: 無法解析的外部符號 "struct THCState * state" (?state@@3PEAUTHCState@@ea)
build\lib.win-amd64-3.7_ext.cp37-win_amd64.pyd : fatal error LNK1120: 1 個無法解析的外部符號
error: command 'C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\VC\Tools\MSVC\14.16.27023\bin\HostX86\x64\link.exe' failed with exit status 1120
cd extreme_utils:
6 errors detected in the compilation of "C:/Users/evans/AppData/Local/Temp/tmpxft_000000e8_00000000-8_utils.cpp4.ii".
error: command 'C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.0\bin\nvcc.exe' failed with exit status 2
您好!
我认真阅读了您的论文,但是有些疑惑
section 3.2 Initial contour proposal.
“the diamond contour is uniformly upsampled to 40 points, and deep snake correspondingly outputs 40 offsets.”
为什么要对菱形轮廓采40个样? 后面的loss function 不是只用到了4个点吗?能否解答一下,感激不尽!谢谢!
Hi Authors!
Thanks for the nice work! Waiting for the code to play with it~
Just a quick question: did you try the model on COCO dataset and whether it can perform well on that much more challenging dataset as well?
I have read your code and found there is no offset, but only 'py_pred'.
作者您好,非常感谢能开源这么优秀的代码。我在训练自己的数据。网络输入尺寸在代码里固定为896*384,我想知道这个值是比较特殊的值吗?我如果改成更大或者更小一点,对训练结果有精度影响吗?
笔者你好,今天将代码在Kins上运行,出现如下错误,不知道问题出在哪
index created!
/home/mil/ruan/.pyenv/versions/anaconda3-2019.07/envs/snake1/lib/python3.7/site-packages/torch/nn/parallel/_functions.py:61: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.
warnings.warn('Was asked to gather along dimension 0, but all '
Traceback (most recent call last):
File "train_net.py", line 54, in
main()
File "train_net.py", line 50, in main
train(cfg, network)
File "train_net.py", line 25, in train
trainer.train(epoch, train_loader, optimizer, recorder)
File "/home/mil/ruan/home/mil/ruan/snake/snake-master/lib/train/trainers/trainer.py", line 43, in train
loss.backward()
File "/home/mil/ruan/.pyenv/versions/anaconda3-2019.07/envs/snake1/lib/python3.7/site-packages/torch/tensor.py", line 102, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph)
File "/home/mil/ruan/.pyenv/versions/anaconda3-2019.07/envs/snake1/lib/python3.7/site-packages/torch/autograd/init.py", line 90, in backward
allow_unreachable=True) # allow_unreachable flag
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation
Compile the dcn_v2 some error as follow, is there any idea ?
required from here /usr/local/include/c++/6.1.0/tuple:495:244: error: wrong number of template arguments (5, should be 2)
Why don't you show results on coco dataset?
Good job, but what’s the performance on the coco benchmark?
Why was this happend ?
Traceback (most recent call last):
File "train_net.py", line 54, in <module>
main()
File "train_net.py", line 50, in main
train(cfg, network)
File "train_net.py", line 25, in train
trainer.train(epoch, train_loader, optimizer, recorder)
File "/ldap_shared/home/s_zyt/inseg/code/snake/lib/train/trainers/trainer.py", line 38, in train
output, loss, loss_stats, image_stats = self.network(batch)
File "/ldap_shared/home/s_zyt/.pyenv/versions/anaconda3-2019.07/envs/snake/lib/python3.7/site-packages/torch/nn/modules/module.py", line 547, in __call__
result = self.forward(*input, **kwargs)
File "/ldap_shared/home/s_zyt/.pyenv/versions/anaconda3-2019.07/envs/snake/lib/python3.7/site-packages/torch/nn/parallel/data_parallel.py", line 152, in forward
outputs = self.parallel_apply(replicas, inputs, kwargs)
File "/ldap_shared/home/s_zyt/.pyenv/versions/anaconda3-2019.07/envs/snake/lib/python3.7/site-packages/torch/nn/parallel/data_parallel.py", line 162, in parallel_apply
return parallel_apply(replicas, inputs, kwargs, self.device_ids[:len(replicas)])
File "/ldap_shared/home/s_zyt/.pyenv/versions/anaconda3-2019.07/envs/snake/lib/python3.7/site-packages/torch/nn/parallel/parallel_apply.py", line 85, in parallel_apply
output.reraise()
File "/ldap_shared/home/s_zyt/.pyenv/versions/anaconda3-2019.07/envs/snake/lib/python3.7/site-packages/torch/_utils.py", line 369, in reraise
raise self.exc_type(msg)
RuntimeError: Caught RuntimeError in replica 0 on device 0.
Original Traceback (most recent call last):
File "/ldap_shared/home/s_zyt/.pyenv/versions/anaconda3-2019.07/envs/snake/lib/python3.7/site-packages/torch/nn/parallel/parallel_apply.py", line 60, in _worker
output = module(*input, **kwargs)
File "/ldap_shared/home/s_zyt/.pyenv/versions/anaconda3-2019.07/envs/snake/lib/python3.7/site-packages/torch/nn/modules/module.py", line 547, in __call__
result = self.forward(*input, **kwargs)
File "lib/train/trainers/snake.py", line 24, in forward
ct_loss = self.ct_crit(net_utils.sigmoid(output['ct_hm']), batch['ct_hm'])
File "/ldap_shared/home/s_zyt/.pyenv/versions/anaconda3-2019.07/envs/snake/lib/python3.7/site-packages/torch/nn/modules/module.py", line 547, in __call__
result = self.forward(*input, **kwargs)
File "/ldap_shared/home/s_zyt/inseg/code/snake/lib/utils/net_utils.py", line 50, in forward
return self.neg_loss(out, target)
File "/ldap_shared/home/s_zyt/inseg/code/snake/lib/utils/net_utils.py", line 29, in _neg_loss
pos_loss = torch.log(pred) * torch.pow(1 - pred, 2) * pos_inds
RuntimeError: The size of tensor a (20) must match the size of tensor b (80) at non-singleton dimension 1
How to reproduce:
I used the COCO format dataset. And run:
python train_net.py --cfg_file configs/sbd_snake.yaml model custom_model train.dataset CocoTrain test.dataset CocoVal
At first glance, you have chosen a traditional strategy known as "detect and segment" widely used in previous methods. I was wondering why didn't you choose FPN to acquire multiple-level information or follow the style of FCOS?
Hi. Thank you very much for your contribution.
I have a problem while running the testing code below:
python run.py --type evaluate --cfg_file configs/city_rcnn_snake.yaml
Below is the error prompted:
ImportError: /home/tunghandsome/Tung/snake/snake/lib/csrc/dcn_v2/_ext.cpython-37m-x86_64-linux-gnu.so: undefined symbol: _ZN6caffe26detail36_typeMetaDataInstance_preallocated_7E
Does anyone know how to fix this error?
Traceback (most recent call last):
File "run.py", line 99, in <module>
globals()['run_'+args.type]()
File "run.py", line 51, in run_evaluate
network = make_network(cfg).cuda()
File "/home/tunghandsome/Tung/snake/snake/lib/networks/make_network.py", line 23, in make_network
return imp.load_source(module, path).get_network(cfg)
File "/home/tunghandsome/snake/lib/python3.7/imp.py", line 171, in load_source
module = _load(spec)
File "<frozen importlib._bootstrap>", line 696, in _load
File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 728, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "lib/networks/rcnn_snake/__init__.py", line 1, in <module>
from .ct_rcnn_snake import get_network as get_rcnn
File "lib/networks/rcnn_snake/ct_rcnn_snake.py", line 2, in <module>
from .dla import DLASeg
File "lib/networks/rcnn_snake/dla.py", line 16, in <module>
from lib.networks.dcn_v2 import DCN
File "/home/tunghandsome/Tung/snake/snake/lib/networks/dcn_v2.py", line 13, in <module>
from lib.csrc.dcn_v2 import _ext as _backend
ImportError: /home/tunghandsome/Tung/snake/snake/lib/csrc/dcn_v2/_ext.cpython-37m-x86_64-linux-gnu.so: undefined symbol: _ZN6caffe26detail36_typeMetaDataInstance_preallocated_7E
RT, thanks very much.
Hi, I have a little question about the contour sampling. In the Section 3.2 / Contour deformation of your paper, you mentioned that
We first uniformly sample the octagon contour N points along its edges and let it start from the top extreme points x^ex_1.
However, I found it hard to achieve "uniformly sampling" while trying to implement deep snake. If I start from one extreme point and sample 1 point for every perimeter / N
distance around the octagon, the other 7 vertices (or 11?) of the octagon may not be in the sampled point set at all. And if I sample (N - 12) / 4
points uniformly along each of the 4 slopes, it doesn't look reasonable because their lengths are different.
Could you explain more about your sampling strategy? Thank you!
any paper link or code info before it release?
作者你好,请问如何用您的代码和模型,直接分割一张已经得到boundingbox的图片?
Thanks for sharing the paper. We would like to do a research on a V100 GPU and 2080TI. Does the code support multi-GPU?
It seems the pre-trained detection model for CityScapes is not provided?
Sincerely thank you!
感谢作者优秀的工作!
最近受到deep snake的启发,想要将其应用到医学数据集中,用于血管的boundary detection。 然而由于数据集结构的不同,我的数据集并没有对应的json文件,并且没有对应的bbox的label。我的数据集所拥有的只有原CTimage 与 所对应的血管内外壁 boundary 的mask。 自己想通过修改作者的代码把数据集移植进去但遇到了困难。 我观察到作者似乎是通过tools.convert_sbd 来对数据集进行处理的。
我想首先听听作者的意见,以及如果可以的话能否share一份处理不同数据集的demo呢?
感激不尽!
It's a nice idea, but I can't install this project in Windows10, and every "requirement" are satisfied. Follow the INSTALL.md, “install apex" ” have error " C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.0\include\crt/host_config.h(133): fatal error C1189: #error: -- unsupported Microsoft Visual Studio version! Only the versions 2012, 2013, 2015 and 2017 are supported!
error: command 'C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.0\bin\nvcc.exe' failed with exit status 2". I have installed VS2017 and VS2019. Do you know what's the problem? Or how to change to VS2017
How to train, test, and visualize on our coco-type dataset?
I insert of speed test code in the function run_visualize
:
def run_visualize():
from lib.networks import make_network
from lib.datasets import make_data_loader
from lib.utils.net_utils import load_network
import tqdm
import torch
from lib.visualizers import make_visualizer
network = make_network(cfg).cuda()
load_network(network, cfg.model_dir, resume=cfg.resume, epoch=cfg.test.epoch)
network.eval()
data_loader = make_data_loader(cfg, is_train=False)
visualizer = make_visualizer(cfg)
time_use = 0
for batch in tqdm.tqdm(data_loader):
for k in batch:
if k != 'meta':
batch[k] = batch[k].cuda()
with torch.no_grad():
start = time()
output = network(batch['inp'], batch)
end = time()
time_use += (end - start)
visualizer.visualize(output, batch)
print("FPS: ", len(data_loader) / time_use)
And I got FPS:
FPS: 5.974452101070888
It seems the speed is far slower from real time. Is there any problem in my test code.
The size of my evaluation data is mostly (544, 768).
Can you explain what act_hm, ct_hm and wh in yaml ?
Thanks
I have trained your model successfully yesterday, Now I'm working on testing my obtained model on cityscapes. However, No matter how I tried, a CUDA error always exists. I'm not familiar with CUDA programming, so I have to open an issue to trouble you @pengsida Have you ever encounter this kind of problem? thank you in advance.
loading annotations into memory...
Done (t=0.17s)
creating index...
index created!
loading annotations into memory...
Done (t=0.14s)
creating index...
index created!
0%| | 0/492 [00:00<?, ?it/s]THCudaCheck FAIL file=/home/fei/snake/lib/csrc/extreme_utils/src/utils.cu line=299 error=9 : invalid configuration argument
Traceback (most recent call last):
File "run.py", line 89, in <module>
globals()['run_'+args.type]()
File "run.py", line 60, in run_evaluate
output = network(inp)
File "/home/fei/anaconda3/envs/snake/lib/python3.7/site-packages/torch/nn/modules/module.py", line 493, in __call__
result = self.forward(*input, **kwargs)
File "lib/networks/rcnn_snake/ct_rcnn_snake.py", line 42, in forward
output = self.gcn(output, cnn_feature, batch)
File "/home/fei/anaconda3/envs/snake/lib/python3.7/site-packages/torch/nn/modules/module.py", line 493, in __call__
result = self.forward(*input, **kwargs)
File "lib/networks/rcnn_snake/evolve.py", line 111, in forward
init = self.prepare_testing_init(output)
File "lib/networks/rcnn_snake/evolve.py", line 41, in prepare_testing_init
i_it_4py = snake_gcn_utils.uniform_upsample(i_it_4py, snake_config.init_poly_num)
File "/home/fei/snake/lib/utils/snake/snake_gcn_utils.py", line 226, in uniform_upsample
extreme_utils.calculate_edge_num(edge_num, edge_num_sum, edge_idx_sort, p_num)
RuntimeError: cuda runtime error (9) : invalid configuration argument at /home/fei/snake/lib/csrc/extreme_utils/src/utils.cu:299
I constantly get this error in almost every 4-5 epoch training.
Traceback (most recent call last):
File "train_net.py", line 57, in <module>
main()
File "train_net.py", line 53, in main
train(cfg, network)
File "train_net.py", line 35, in train
trainer.val(epoch, val_loader, evaluator, recorder)
File "/ldap_shared/home/s_zyt/inseg/code/snake/lib/train/trainers/trainer.py", line 76, in val
for batch in tqdm.tqdm(data_loader):
File "/ldap_shared/home/s_zyt/.pyenv/versions/anaconda3-2019.07/envs/snake/lib/python3.7/site-packages/tqdm/_tqdm.py", line 979, in __iter__
for obj in iterable:
File "/ldap_shared/home/s_zyt/.pyenv/versions/anaconda3-2019.07/envs/snake/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 346, in __next__
data = self.dataset_fetcher.fetch(index) # may raise StopIteration
File "/ldap_shared/home/s_zyt/.pyenv/versions/anaconda3-2019.07/envs/snake/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 47, in fetch
return self.collate_fn(data)
File "/ldap_shared/home/s_zyt/inseg/code/snake/lib/datasets/collate_batch.py", line 7, in snake_collator
ret = {'inp': default_collate([b['inp'] for b in batch])}
File "/ldap_shared/home/s_zyt/.pyenv/versions/anaconda3-2019.07/envs/snake/lib/python3.7/site-packages/torch/utils/data/_utils/collate.py", line 65, in default_collate
return default_collate([torch.as_tensor(b) for b in batch])
File "/ldap_shared/home/s_zyt/.pyenv/versions/anaconda3-2019.07/envs/snake/lib/python3.7/site-packages/torch/utils/data/_utils/collate.py", line 56, in default_collate
return torch.stack(batch, 0, out=out)
RuntimeError: invalid argument 0: Sizes of tensors must match except in dimension 0. Got 544 and 480 in dimension 2 at /pytorch/aten/src/TH/generic/THTensor.cpp:689
Should the validation dataloader worker number be set as 0?
Previously, I got this error in train dataloader, so I set the num_worker=0, it works.
Configuration as follow:
train:
optim: 'adam'
lr: 1e-4
milestones: (80, 120, 150, 170)
gamma: 0.5
batch_size: 16
dataset: 'CocoTrain'
num_workers: 0
epoch: 150
test:
# dataset: 'SbdMini'
dataset: 'CocoVal'
batch_size: 16
num_workers: 0
My operating system is Ubuntu16.04.
Thank you!
Hello, I have converted my own data set to the coco data set format and modified the data path, but during the training process, the loss is changing, but all ap, ar values are almost 0 ,the result of visualization is equivalent to the original image, without any detection frame and segmentation effect. I think the data set is not really trained, have I missed any parameter adjustment?
After I finished the setting from your INSTALL.md and when I ran the testing code:
python run.py --type evaluate --cfg_file configs/city_rcnn_snake.yaml test.dataset CityscapesVal
I saw these errors:
WARNING: NO MODEL LOADED !!!
load model: data/model/rcnn_snake/long_rcnn/197.pth
0%| | 0/492 [00:00<?, ?it/s]torch_shm_manager: error while loading shared libraries: libcudart.so.10.1: cannot open shared object file: No such file or directory
torch_shm_manager: error while loading shared libraries: libcudart.so.10.1: cannot open shared object file: No such file or directory
torch_shm_manager: error while loading shared libraries: libcudart.so.10.1: cannot open shared object file: No such file or directory
torch_shm_manager: error while loading shared libraries: libcudart.so.10.1: cannot open shared object file: No such file or directory
torch_shm_manager: error while loading shared libraries: libcudart.so.10.1: cannot open shared object file: No such file or directory
Traceback (most recent call last):
File "run.py", line 99, in
globals()'run_'+args.type
File "run.py", line 57, in run_evaluate
for batch in tqdm.tqdm(data_loader):
File "/home/andrew/anaconda3/lib/python3.7/site-packages/tqdm/std.py", line 1119, in iter
for obj in iterable:
File "/home/andrew/anaconda3/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 345, in next
data = self._next_data()
File "/home/andrew/anaconda3/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 856, in _next_data
return self._process_data(data)
File "/home/andrew/anaconda3/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 881, in _process_data
data.reraise()
File "/home/andrew/anaconda3/lib/python3.7/site-packages/torch/_utils.py", line 394, in reraise
raise self.exc_type(msg)
RuntimeError: Caught RuntimeError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "/home/andrew/anaconda3/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 178, in _worker_loop
data = fetcher.fetch(index)
File "/home/andrew/anaconda3/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 47, in fetch
return self.collate_fn(data)
File "/home/andrew/Documents/snake-master/lib/datasets/collate_batch.py", line 94, in rcnn_snake_collator
ret = {'inp': default_collate([b['inp'] for b in batch])}
File "/home/andrew/anaconda3/lib/python3.7/site-packages/torch/utils/data/_utils/collate.py", line 64, in default_collate
return default_collate([torch.as_tensor(b) for b in batch])
File "/home/andrew/anaconda3/lib/python3.7/site-packages/torch/utils/data/_utils/collate.py", line 53, in default_collate
storage = elem.storage()._new_shared(numel)
File "/home/andrew/anaconda3/lib/python3.7/site-packages/torch/storage.py", line 128, in _new_shared
return cls._new_using_filename(size)
RuntimeError: error executing torch_shm_manager at "/home/andrew/anaconda3/lib/python3.7/site-packages/torch/bin/torch_shm_manager" at /pytorch/torch/lib/libshm/core.cpp:99
torch_shm_manager: error while loading shared libraries: libcudart.so.10.1: cannot open shared object file: No such file or directory
torch_shm_manager: error while loading shared libraries: libcudart.so.10.1: cannot open shared object file: No such file or directory
torch_shm_manager: error while loading shared libraries: libcudart.so.10.1: cannot open shared object file: No such file or directory
torch_shm_manager: error while loading shared libraries: libcudart.so.10.1: cannot open shared object file: No such file or directory
torch_shm_manager: error while loading shared libraries: libcudart.so.10.1: cannot open shared object file: No such file or directory
torch_shm_manager: error while loading shared libraries: libcudart.so.10.1: cannot open shared object file: No such file or directory
torch_shm_managertorch_shm_manager: error while loading shared libraries: libcudart.so.10.1: cannot open shared object file: No such file or directory
: error while loading shared libraries: libcudart.so.10.1: cannot open shared object file: No such file or directory
0%| | 0/492 [00:01<?, ?it/s]
My environment are as fellows:
Pytorch: 1.4.0
Torchvision: 0.5.0
Ubuntu: 18.04.2
CUDA version(from "torch.version.cuda"): 10.1
And I have already checked the path of your pretrained model, which is $ROOT/data/model/rcnn_snake/long_rcnn/197.pth
So could you please help me to figure out what is going on? I would be very honored if you can help me!
Thanks!
In case wanna reproduce paper results, when will code release?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.