Git Product home page Git Product logo

ttfnet's Introduction

Training-Time-Friendly Network for Real-Time Object Detection

The code for implementing the TTFNet (Accepted to AAAI2020).

image-20190807160835333

Highlights

  • Simple: Anchor-free, single-stage, light-head, no time-consuming post-processing. TTFNet only requires two detection heads for object localization and size regression, respectively.
  • Training Time Friendly: Our TTFNet outperforms a range of real-time detectors while suppressing them in training time. Moreover, super-fast TTFNet-18 and TTFNet-53 can reach 25.9 AP / 112 FPS only after 2 hours and 32.9 AP / 55 FPS after about 3 hours on the MS COCO dataset using 8 GTX 1080Ti.
  • Fast and Precise: Our TTFNet-18/34/53 can achieve 28.1AP / 112FPS, 31.3AP / 87FPS, and 35.1AP / 54 FPS on 1 GTX 1080Ti.

Performances

Table

Table

TT stands for training time. * indicates that the result is not presented in the original paper. fast stands for the super-fast version and 10x stands for the long-training version.

All the training time is measured on 8 GTX 1080Ti, and all the inference speed is measured using converged models on 1 GTX 1080Ti. Note that the training time does not include the time consumed by evaluation.

Installation

Our TTFNet is based on mmdetection. Please check INSTALL.md for installation instructions, and you may want to see the original README.md. We will submit a pull request soon.

Note that the darknet part was transplanted (i.e., MXNet => Pytorch) from another toolbox Gluoncv. In addition, portions of the code are borrowed from CornerNet and CenterNet. Thanks for their work !

Inference

We provide the following converged models.

Model Training Hours FPS AP(minival) Link
TTFNet-18 (1x) 1.8 112.2 25.9 Download
TTFNet-18 (2x) 3.6 112.3 28.1 Download
TTFNet-18 (no-pretrain 10x) - 121.0 30.3 Download
TTFNet-18 (10x) - 113.6 31.8 Download
TTFNet-34 (2x) 4.1 86.6 31.3 Download
TTFNet-34 (no-pretrain 10x) - 89.2 33.2 Download
TTFNet-34 (10x) - 88.4 35.3 Download
TTFNet-53 (1x) 3.1 54.8 32.9 Download
TTFNet-53 (2x) 6.1 54.4 35.1 Download
TTFNet-53 (no-pretrain 10x) - 57.2 36.2 Download
TTFNet-53 (10x) 30.6 57.0 39.3 Download

We also provide the pretrained Darknet53 and DLA-34 here.

The following command will evaluate converged TTFNet-53 on 8 GPUs:

./tools/dist_test.sh configs/ttfnet/ttfnet_d53_2x.py /path/to/the/checkpoint 8

Training

The following commands will train TTFNet-18 on 8 GPUs for 24 epochs and TTFNet-53 on 8 GPUs for 12 epochs:

./tools/dist_train.sh configs/ttfnet/ttfnet_r18_2x.py 8
./tools/dist_train.sh configs/ttfnet/ttfnet_d53_1x.py 8

Citations

Please consider citing our paper in your publications if the project helps your research. BibTeX reference is as follows.

@article{liu2019training,
  title   = {Training-Time-Friendly Network for Real-Time Object Detection},
  author  = {Zili Liu, Tu Zheng, Guodong Xu, Zheng Yang, Haifeng Liu, Deng Cai},
  journal = {arXiv preprint arXiv:1909.00700},
  year    = {2019}
}

ttfnet's People

Contributors

ancientmooner avatar borda avatar dhananjaisharma10 avatar dmarnerides avatar donnyyou avatar erotemic avatar eugenelawrence avatar gfjiangly avatar hellock avatar innerlee avatar korabelnikov avatar lindahua avatar liushuchun avatar liuzili97 avatar luxiin avatar michaelisc avatar myownskyw7 avatar oceanpang avatar patrick-llgc avatar sovrasov avatar ternaus avatar thangvubk avatar turoad avatar wondervictor avatar wswday avatar xvjiarui avatar yhcao6 avatar youkaichao avatar zhihuagao avatar zwwwayne avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

ttfnet's Issues

ImportError: cannot import name 'get_dist_info'

I get this error message 'ImportError: cannot import name 'get_dist_info' when I run './tools/dist_test.sh configs/ttfnet/ttfnet_d53_2x.py /path/to/the/checkpoint 8' command.
As I see from issues of open-mmlab/mmdetection repository, I upgraded mmcv and mmdet with 'pip install --upgrade mmdet' and launched 'python setup.py develop' again but it doesn't work. Also I
tried using open-mmlab/mmdetection repository, but it doesn't contain ttfnet config file. So, I copied the ttfnet config file from ttfnet repo to mmdetection config files, and tried to test webcam demo. However, I got this error message: 'KeyError: 'TTFNet is not in the detector registry'. Can anyone help about that? Thanks.

用了可行变卷积,但是论文没说

作者你好,代码里面我看到你们用了DCN,这个肯定对最终性能帮助很大,但是论文里面我没看到有提到这个地方,不知道是不是我疏忽了,你们有对比不用DCN只采用普通卷积的性能么。

What's meaning of wh_offset_base ?

It seems that the model will converge much more slower if wh_offset_base =1, especially the wh_loss. Besides, compared with centernet, the loss weight of wh_loss is much more larger than hm_loss(centernet: wh_loss_weight=0.1, hm_loss=1),why?

How to install ttfnet on cuda11.1?

你好,因为RTX30系显卡只支持cuda11的版本,所以想请问一下怎么在cuda11.1安装ttfnet,按照文档中的方法安装报错了。

Loss curves

Can you please post a plot of loss curves (hm and reg) over epochs?
Thanks

关于椭圆高斯

通过draw_truncate_gaussian画出的椭圆高斯相对原版的高斯多了宽高比例信息理论上应该是更优,这个对涨点有实验数据提供吗?

question about 'wh_offset_base'

Hi, thanks for your great work.
I am confused that 'wh_offset_base' =16 in ttf_head.py, but I don't know why you need to set this parameter.
image

Why do you need to multiply it in the regression head? Thanks.

Training a model with an own dataset

Hi,

First of all, tks for this great job. I'd like to know if is there a tutorial step-by-step (or something like this) helping with how to train a TTFNet model with our own dataset? Tks.

wired output of wh

the wl of wh ouput are always 0, they are negative before relu

tensor([[[ 0.0000, 116.5211, 42.4937, 122.5416],
[ 0.0000, 107.6900, 41.2038, 117.2426],
[ 0.0000, 125.4157, 44.4884, 128.1640],
[ 0.0000, 118.4363, 38.8104, 120.8187],
[ 0.0000, 144.1779, 51.2308, 147.0248],
[ 0.0000, 123.4721, 42.6045, 126.1252],
[ 0.0000, 115.3537, 38.5883, 124.3103],
[ 0.0000, 95.5488, 33.9789, 121.5930],
[ 0.0000, 137.6849, 48.2651, 129.0714],
[ 0.0000, 153.8599, 44.6644, 129.4741],
[ 0.0000, 79.8924, 34.0436, 128.6441],
[ 0.0000, 43.4521, 19.8980, 50.6122],
[ 0.0000, 114.4678, 53.5928, 117.2749],
[ 0.0000, 50.5073, 21.2092, 52.7150],
[ 0.0000, 37.3370, 20.9332, 60.4037],
[ 0.0000, 58.5186, 24.4326, 60.5577],
[ 0.0000, 56.8286, 23.6414, 60.3964],
[ 0.0000, 20.9488, 12.8595, 23.5858],
[ 0.0000, 20.5648, 14.1972, 24.3704],
[ 0.0000, 21.1813, 14.0686, 24.6013],
[ 0.0000, 25.0874, 15.2591, 25.0824],
[ 0.0000, 58.8945, 7.9307, 51.6445],
[ 0.0000, 20.0220, 14.1485, 22.9746],
[ 0.0000, 23.1689, 14.9487, 26.1784],
[ 0.0000, 105.1940, 18.1960, 107.0099],
[ 0.0000, 93.0207, 22.4406, 81.9989],
[ 0.0000, 161.1775, 69.2532, 120.0340],
[ 0.0000, 152.0186, 59.0491, 113.5726],
[ 0.0000, 30.4867, 19.4364, 45.0907],
[ 0.0000, 35.7395, 70.6150, 40.3779],
[ 0.0000, 19.9065, 13.7627, 22.8954],
[ 0.0000, 87.0246, 91.8417, 110.4563],
[ 0.0000, 47.9610, 25.7173, 49.1239],
[ 0.0000, 124.6373, 26.2414, 132.3073],
[ 0.0000, 42.9990, 52.1530, 58.9938],
[ 0.0000, 109.3388, 92.1436, 129.7543],
[ 0.0000, 43.4066, 26.2116, 48.3573],
[ 0.0000, 19.8315, 13.0908, 21.9701],
[ 0.0000, 136.0730, 28.4794, 128.4158],
[ 0.0000, 19.9120, 13.3052, 21.5078],
[ 0.0000, 20.3097, 13.3365, 23.5736],
[ 0.0000, 20.0541, 13.0240, 23.1604],
[ 0.0000, 86.9304, 11.6221, 84.4194],
[ 0.0000, 108.4659, 18.4569, 93.0499],
[ 0.0000, 19.3067, 13.9523, 19.6273],
[ 0.0000, 123.7198, 23.3926, 114.5532],
[ 0.0000, 23.0683, 15.0457, 27.2681],
[ 0.0000, 11.1033, 5.4479, 18.0002],
[ 0.0000, 31.8376, 16.7024, 35.1585],
[ 0.0000, 65.6375, 32.1086, 78.4625],
[ 0.0000, 126.2547, 46.1084, 141.0742],
[ 0.0000, 49.8713, 16.5768, 61.0893],
[ 0.0000, 97.8429, 104.7201, 100.0325],
[ 0.0000, 59.1779, 41.5078, 135.6670],
[ 0.0000, 76.3698, 10.7484, 72.4081],
[ 0.0000, 74.9829, 46.3468, 155.4478],
[ 0.0000, 119.8739, 24.9125, 133.0377],
[ 0.0000, 10.6555, 4.1952, 17.6679],
[ 0.0000, 63.5594, 30.0634, 88.3806],
[ 0.0000, 88.6980, 93.6552, 72.6249],
[ 0.0000, 19.6395, 13.0398, 20.9386],
[ 0.0000, 128.0675, 22.5303, 117.6851],
[ 0.0000, 63.3495, 29.9718, 67.9388],
[ 0.0000, 18.8939, 13.5181, 18.6566],
[ 0.0000, 7.2600, 6.0671, 11.0599],
[ 0.0000, 7.3105, 6.0417, 11.0529],
[ 0.0000, 43.4574, 15.4914, 53.1781],
[ 0.0000, 7.0489, 5.8090, 10.0173],
[ 0.0000, 7.1726, 5.9593, 10.8656],
[ 0.0000, 62.8165, 34.6642, 66.0653],
[ 0.0000, 51.7567, 10.6115, 50.0044],
[ 0.0000, 92.7709, 105.0076, 102.7054],
[ 0.0000, 39.5032, 23.0723, 42.0505],
[ 0.0000, 47.5203, 42.7517, 108.0348],
[ 0.0000, 6.9966, 5.8431, 10.0747],
[ 0.0000, 113.8149, 107.7129, 119.9966],
[ 0.0000, 120.9095, 43.7954, 145.0118],
[ 0.0000, 7.0511, 5.7632, 9.8832],
[ 0.0000, 19.7299, 12.0109, 17.0321],
[ 0.0000, 7.3372, 5.9645, 10.9145],
[ 0.0000, 90.5423, 101.5567, 97.5997],
[ 0.0000, 7.0606, 5.7998, 10.0662],
[ 0.0000, 117.0862, 46.2676, 152.8364],
[ 0.0000, 144.9170, 44.8524, 92.6930],
[ 0.0000, 105.6485, 17.5557, 119.2687],
[ 0.0000, 105.2603, 105.4551, 105.4100],
[ 0.0000, 7.4934, 6.0036, 11.0666],
[ 0.0000, 78.2761, 101.2695, 83.9413],
[ 0.0000, 34.0802, 21.5975, 41.9197],
[ 0.0000, 37.6092, 22.0470, 35.8518],
[ 0.0000, 52.5578, 38.3820, 74.1862],
[ 0.0000, 79.0693, 61.6724, 153.6753],
[ 0.0000, 10.1691, 5.8722, 13.3266],
[ 0.0000, 73.8878, 53.4783, 138.8358],
[ 0.0000, 7.8271, 6.2989, 11.5233],
[ 0.0000, 62.8140, 23.9927, 66.1418],
[ 0.0000, 7.9598, 6.2933, 11.6692],
[ 0.0000, 111.2614, 103.2351, 134.6110],
[ 0.0000, 69.5796, 87.4113, 71.6074],
[ 0.0000, 54.6593, 26.5077, 60.6459]]], device='cuda:0')

this is the detection result
test

你好,我使用的是ttfnet_d53_1x.py的配置文件,但是当使用darknet53.pth作为预训练模型训练时,已经训练了7个epoch,但是mAP还是很低(和不用预训练模型差不多),请问这是什么原因?

Thanks for your error report and we appreciate it a lot.

Checklist

  1. I have searched related issues but cannot get the expected help.
  2. The bug has not been fixed in the latest version.

Describe the bug
A clear and concise description of what the bug is.

Reproduction

  1. What command or script did you run?
A placeholder for the command.
  1. Did you make any modifications on the code or config? Did you understand what you have modified?
  2. What dataset did you use?

Environment

  • OS: [e.g., Ubuntu 16.04.6]
  • GCC [e.g., 5.4.0]
  • PyTorch version [e.g., 1.1.0]
  • How you installed PyTorch [e.g., pip, conda, source]
  • GPU model [e.g., 1080Ti, V100]
  • CUDA and CUDNN version
  • [optional] Other information that may be related (such as $PATH, $LD_LIBRARY_PATH, $PYTHONPATH, etc.)

Error traceback
If applicable, paste the error trackback here.

A placeholder for trackback.

Bug fix
If you have already identified the reason, you can provide the information here. If you are willing to create a PR to fix it, please also leave a comment here and that would be much appreciated!

关于Size Regression标签的一些困惑

你好,感谢你的工作!关于Gaussian Kernels for Training, Size Regression部分,不是很明白里面的scalar s的作用,加上s后不是网络预测的 w_l, h_l, w_r, h_r值反而会更小更不利于网络预测吗?

Can you add Dockerfile?

I have some dependency problems. So I want to use docker. If you don't mind, can you upload requirements.txt or Dockerfile? Thank you.

Darknet pretrained model?

Hello,

Thanks for sharing nice work and codes.
Could you please provide darknet pretrained model?

Thanks,

About the stride in your paper

HI, in your paper, what is the parameter r? And you use xˆ1=ir−wˆls, yˆ1=jr−hˆts, to recover the bbox and enlarge the predicted results, means that your parameter r is big than 16?

about loss convergence

Can you provide the train log file of COCO dataset? I want to know if I converge correctly when I train my model.

dla 34 model get key error

when I run dla34 trained model published from this website,
image

it seems keys mismatched from the trained model.

Have you tried large-scale input?

I tried to use 1333x800 and keep_ratio, the backbone is darknet53, other parameters have not changed, the result is not good, map = 30.6; have you tried? Why the effect is so bad?

If set wh_gaussian=False, the wh_loss is nan

Thanks for sharing your work.
I have one question: in tab 1 of the paper, if not use wh_gaussian, the accuracy reduce from 32.9 to 32. I want to reproduce the result of 32. If set wh_gaussian=False, the wh_loss is nan. What other setting I should change.

cost more training time per epoch than centernet

Using single GPU, same network architecture, I found the training time per epoch of ttfnet is about 1.6 times than centernet.
I'm not sure whether I did the comparison right , has anyone made such comparison?

question realted to aspect ratio of the box

In order to eliminate the need for the NMS, we adopt a
similar strategy for center localization. Specifically, we further
consider the aspect ratio of the box in the Gaussian kernel
since the strategy that does not consider it in CenterNet
is obviously sub-optimal.

为什么是centernet中不考虑长宽比的方法是sub-optimal的呢,能否给个原因,感觉ttfnet里面这样做意义不大,因为后面做max pool的卷积核也是3*3的,是正方形的

Can't get the same map as paper

Hello! I like your project. I tried to train ttfnet_r18_2x on 2 TITAN V and 3 P100. I used 13 hours for 24 epoch, and get map of 0.9%. I don't know why I used so much time and got so low map. Do you know what's wrong with my training?

FPS

Hi,is the FPS computed by mmdetection as follow?
./tools/dist_test.sh configs/ttfnet/ttfnet_18_1x.py /path/to/the/checkpoint 1
My gpu is 2080ti, but I can only get inferred speeds of 90FPS (90 task/s) for TTFnet18.
What does ''using converged models'' mean?
thanks

python setup.py develop 报错 RuntimeError: Error compiling objects for extension

当我在搭建ttfnet环境时,执行setup.py develop 报错如下:
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
[1/1] c++ -MMD -MF /home/user/yzw/ttfnet-master/build/temp.linux-x86_64-3.7/mmdet/ops/nms/src/nms_cuda.o.d -pthread -B /home/user/.conda/envs/pytorch_cop/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/user/.conda/envs/pytorch_cop/lib/python3.7/site-packages/torch/include -I/home/user/.conda/envs/pytorch_cop/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/home/user/.conda/envs/pytorch_cop/lib/python3.7/site-packages/torch/include/TH -I/home/user/.conda/envs/pytorch_cop/lib/python3.7/site-packages/torch/include/THC -I/usr/local/cuda-11.1/include -I/home/user/.conda/envs/pytorch_cop/include/python3.7m -c -c /home/user/yzw/ttfnet-master/mmdet/ops/nms/src/nms_cuda.cpp -o /home/user/yzw/ttfnet-master/build/temp.linux-x86_64-3.7/mmdet/ops/nms/src/nms_cuda.o -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=nms_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
FAILED: /home/user/yzw/ttfnet-master/build/temp.linux-x86_64-3.7/mmdet/ops/nms/src/nms_cuda.o
c++ -MMD -MF /home/user/yzw/ttfnet-master/build/temp.linux-x86_64-3.7/mmdet/ops/nms/src/nms_cuda.o.d -pthread -B /home/user/.conda/envs/pytorch_cop/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/user/.conda/envs/pytorch_cop/lib/python3.7/site-packages/torch/include -I/home/user/.conda/envs/pytorch_cop/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/home/user/.conda/envs/pytorch_cop/lib/python3.7/site-packages/torch/include/TH -I/home/user/.conda/envs/pytorch_cop/lib/python3.7/site-packages/torch/include/THC -I/usr/local/cuda-11.1/include -I/home/user/.conda/envs/pytorch_cop/include/python3.7m -c -c /home/user/yzw/ttfnet-master/mmdet/ops/nms/src/nms_cuda.cpp -o /home/user/yzw/ttfnet-master/build/temp.linux-x86_64-3.7/mmdet/ops/nms/src/nms_cuda.o -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=nms_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
In file included from /home/user/.conda/envs/pytorch_cop/lib/python3.7/site-packages/torch/include/ATen/Parallel.h:149:0,
from /home/user/.conda/envs/pytorch_cop/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/utils.h:3,
from /home/user/.conda/envs/pytorch_cop/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:5,
from /home/user/.conda/envs/pytorch_cop/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/nn.h:3,
from /home/user/.conda/envs/pytorch_cop/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/all.h:12,
from /home/user/.conda/envs/pytorch_cop/lib/python3.7/site-packages/torch/include/torch/extension.h:4,
from /home/user/yzw/ttfnet-master/mmdet/ops/nms/src/nms_cuda.cpp:2:
/home/user/.conda/envs/pytorch_cop/lib/python3.7/site-packages/torch/include/ATen/ParallelOpenMP.h:84:0: warning: ignoring #pragma omp parallel [-Wunknown-pragmas]
#pragma omp parallel for if ((end - begin) >= grain_size)

/home/user/yzw/ttfnet-master/mmdet/ops/nms/src/nms_cuda.cpp: In function ‘at::Tensor nms(const at::Tensor&, float)’:
/home/user/yzw/ttfnet-master/mmdet/ops/nms/src/nms_cuda.cpp:4:39: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
#define CHECK_CUDA(x) AT_CHECK(x.type().is_cuda(), #x, " must be a CUDAtensor ")
^
/home/user/yzw/ttfnet-master/mmdet/ops/nms/src/nms_cuda.cpp:9:3: note: in expansion of macro ‘CHECK_CUDA’
CHECK_CUDA(dets);
^~~~~~~~~~
In file included from /home/user/.conda/envs/pytorch_cop/lib/python3.7/site-packages/torch/include/ATen/Tensor.h:3:0,
from /home/user/.conda/envs/pytorch_cop/lib/python3.7/site-packages/torch/include/ATen/Context.h:4,
from /home/user/.conda/envs/pytorch_cop/lib/python3.7/site-packages/torch/include/ATen/ATen.h:9,
from /home/user/.conda/envs/pytorch_cop/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
from /home/user/.conda/envs/pytorch_cop/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
from /home/user/.conda/envs/pytorch_cop/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
from /home/user/.conda/envs/pytorch_cop/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
from /home/user/.conda/envs/pytorch_cop/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
from /home/user/.conda/envs/pytorch_cop/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
from /home/user/.conda/envs/pytorch_cop/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/all.h:8,
from /home/user/.conda/envs/pytorch_cop/lib/python3.7/site-packages/torch/include/torch/extension.h:4,
from /home/user/yzw/ttfnet-master/mmdet/ops/nms/src/nms_cuda.cpp:2:
/home/user/.conda/envs/pytorch_cop/lib/python3.7/site-packages/torch/include/ATen/core/TensorBody.h:277:30: note: declared here
DeprecatedTypeProperties & type() const {
^~~~
/home/user/yzw/ttfnet-master/mmdet/ops/nms/src/nms_cuda.cpp:4:23: error: ‘AT_CHECK’ was not declared in this scope
#define CHECK_CUDA(x) AT_CHECK(x.type().is_cuda(), #x, " must be a CUDAtensor ")
^
/home/user/yzw/ttfnet-master/mmdet/ops/nms/src/nms_cuda.cpp:9:3: note: in expansion of macro ‘CHECK_CUDA’
CHECK_CUDA(dets);
^~~~~~~~~~
/home/user/yzw/ttfnet-master/mmdet/ops/nms/src/nms_cuda.cpp:4:23: note: suggested alternative: ‘DCHECK’
#define CHECK_CUDA(x) AT_CHECK(x.type().is_cuda(), #x, " must be a CUDAtensor ")
^
/home/user/yzw/ttfnet-master/mmdet/ops/nms/src/nms_cuda.cpp:9:3: note: in expansion of macro ‘CHECK_CUDA’
CHECK_CUDA(dets);
^~~~~~~~~~
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
File "/home/user/.conda/envs/pytorch_cop/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1539, in _run_ninja_build
env=env)
File "/home/user/.conda/envs/pytorch_cop/lib/python3.7/subprocess.py", line 468, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "setup.py", line 208, in
zip_safe=False)
File "/home/user/.conda/envs/pytorch_cop/lib/python3.7/site-packages/setuptools/init.py", line 153, in setup
return distutils.core.setup(**attrs)
File "/home/user/.conda/envs/pytorch_cop/lib/python3.7/distutils/core.py", line 148, in setup
dist.run_commands()
File "/home/user/.conda/envs/pytorch_cop/lib/python3.7/distutils/dist.py", line 966, in run_commands
self.run_command(cmd)
File "/home/user/.conda/envs/pytorch_cop/lib/python3.7/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/home/user/.conda/envs/pytorch_cop/lib/python3.7/site-packages/setuptools/command/develop.py", line 34, in run
self.install_for_development()
File "/home/user/.conda/envs/pytorch_cop/lib/python3.7/site-packages/setuptools/command/develop.py", line 136, in install_for_development
self.run_command('build_ext')
File "/home/user/.conda/envs/pytorch_cop/lib/python3.7/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/home/user/.conda/envs/pytorch_cop/lib/python3.7/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/home/user/.conda/envs/pytorch_cop/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 79, in run
_build_ext.run(self)
File "/home/user/.conda/envs/pytorch_cop/lib/python3.7/site-packages/Cython/Distutils/old_build_ext.py", line 186, in run
_build_ext.build_ext.run(self)
File "/home/user/.conda/envs/pytorch_cop/lib/python3.7/distutils/command/build_ext.py", line 339, in run
self.build_extensions()
File "/home/user/.conda/envs/pytorch_cop/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 670, in build_extensions
build_ext.build_extensions(self)
File "/home/user/.conda/envs/pytorch_cop/lib/python3.7/site-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
_build_ext.build_ext.build_extensions(self)
File "/home/user/.conda/envs/pytorch_cop/lib/python3.7/distutils/command/build_ext.py", line 448, in build_extensions
self._build_extensions_serial()
File "/home/user/.conda/envs/pytorch_cop/lib/python3.7/distutils/command/build_ext.py", line 473, in _build_extensions_serial
self.build_extension(ext)
File "/home/user/.conda/envs/pytorch_cop/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 196, in build_extension
_build_ext.build_extension(self, ext)
File "/home/user/.conda/envs/pytorch_cop/lib/python3.7/distutils/command/build_ext.py", line 533, in build_extension
depends=ext.depends)
File "/home/user/.conda/envs/pytorch_cop/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 500, in unix_wrap_ninja_compile
with_cuda=with_cuda)
File "/home/user/.conda/envs/pytorch_cop/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1255, in _write_ninja_file_and_compile_objects
error_prefix='Error compiling objects for extension')
File "/home/user/.conda/envs/pytorch_cop/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1555, in _run_ninja_build
raise RuntimeError(message) from e
RuntimeError: Error compiling objects for extension
我该怎么解决这个问题

question about loss

In this paper, loss is composed of hm loss and wh loss, why did you delete reg loss used in CenterNet?

Questions regarding giou loss.

Isn't the output of weight, bboxes1, and bboxes2 after slicing with pos_mask a 5-dim tensor? After reshape, it can't fit into memory.

weight = weight[pos_mask].float() bboxes1 = pred[pos_mask].view(-1, 4) bboxes2 = target[pos_mask].view(-1, 4)

与其他anchor free方法比较,如FCOS?

不知道我理解的对不对,感觉类似centernet与FCOS的结合改进。
1.centernet方法改进来获得中心点,与centernet相比使用了椭圆区域的更多像素作为正样本中心点,这一点与FCOS使用box内的所有像素很类似,但正样本比FCOS的正样本更有效,因为更接近中心。
2.尺寸预测中使用了另一个高斯核,同样,高斯核内比centernet拥有了更多的正样本。而高斯核外尺寸回归的准确率较低故作为负样本。这里的高斯核又很类似于FCOS中的center-ness的作用,将远离中心的尺寸回归置信度降低。直接使用高斯核是否比FCOS去预测center-ness效率要高很多?
不知道理解对不对,感谢指正。

How to get FPS?

HI! Thank you for your excellent work. I want to konw how can I get FPS of TTFNet?

Segmentation fault

I'm working on object detection using TTFnet but I'm stucked on this issue:
after running the following command:

./tools/dist_test.sh configs/ttfnet/ttfnet_r18_1x.py 8
I'm getting this:
./tools/dist_train.sh: line 10: 3398 Segmentation fault (core dumped) $PYTHON -m torch.distributed.launch --nproc_per_node=$GPUS --master_port=$((RANDOM + 10000)) $(dirname "$0")/train.py $CONFIG --validate --launcher pytorch ${@:3} --autoscale-lr --gpus=$GPUS

Please suggest me how to solve this.
Thank You !

Update the codes to latest mmdetection.

@liuzili97 @Turoad

Motivation
Update current codes into the latest mmdetection is helpful.

Additional context
By simply copying the model and config files and modifying import settings, my implementation fails to converge during the training. The mAP is in fact near 0. Could you give some suggestions that transfer your codes into the latest version?

Gaussian area calculation

Thanks for the excellent work.

I could not find how to calculate the Gaussian area(Fig 3. (d)) in your paper. But the code here gives me some clues. Maybe you would like to add the calculation formula in your paper.

你好,那个训练好的模型无法获取了,感谢

Thanks for your error report and we appreciate it a lot.

Checklist

  1. I have searched related issues but cannot get the expected help.
  2. The bug has not been fixed in the latest version.

Describe the bug
A clear and concise description of what the bug is.

Reproduction

  1. What command or script did you run?
A placeholder for the command.
  1. Did you make any modifications on the code or config? Did you understand what you have modified?
  2. What dataset did you use?

Environment

  • OS: [e.g., Ubuntu 16.04.6]
  • GCC [e.g., 5.4.0]
  • PyTorch version [e.g., 1.1.0]
  • How you installed PyTorch [e.g., pip, conda, source]
  • GPU model [e.g., 1080Ti, V100]
  • CUDA and CUDNN version
  • [optional] Other information that may be related (such as $PATH, $LD_LIBRARY_PATH, $PYTHONPATH, etc.)

Error traceback
If applicable, paste the error trackback here.

A placeholder for trackback.

Bug fix
If you have already identified the reason, you can provide the information here. If you are willing to create a PR to fix it, please also leave a comment here and that would be much appreciated!

Parameter alpha (0.54) multiplied in code but division in paper?

In the paper the formula for the gaussian is given:
σ = w / 6α

In the code alpha is being multiplied to the radius like so (brackets added to emphasise order of operations)

        h_radiuses_alpha = ((feat_hs / 2.) * self.alpha).int()
        w_radiuses_alpha = ((feat_ws / 2.) * self.alpha).int()

This seems to me that alpha is 1/alpha... excuse me if I'm mistaken!

performance for resnet50

did anyone tried resnet50?

i simply set
bbox_head=dict( type='TTFHead', inplanes=(256, 512, 1024, 2048),
and change the backbone to resnet50, the performace on my own dataset is worser than darknet53, about 6 points lower in AP

is the ttfhead not suitable for inplanes=(256, 512, 1024, 2048)

RuntimeError: CUDA error: unspecified launch failure

I run ./tools/dist_train.sh configs/ttfnet/ttfnet_r18_1x.py 4, there is an error:

2019-10-04 15:27:50,440 - INFO - Epoch [5][400/1284]    lr: 0.00800, eta: 1:54:10, time: 0.666, data_time: 0.033, memory: 9804, losses/ttfnet_loss_heatmap: 2.0091, losses/ttfnet_loss_wh: 3.2484, loss: 5.2576
Traceback (most recent call last):
  File "./tools/train.py", line 106, in <module>
    main()
  File "./tools/train.py", line 102, in main
    logger=logger)
  File "/media/hdd/han/detections/ttfnet/mmdet/apis/train.py", line 58, in train_detector
    _dist_train(model, dataset, cfg, validate=validate)
  File "/media/hdd/han/detections/ttfnet/mmdet/apis/train.py", line 187, in _dist_train
    runner.run(data_loaders, cfg.workflow, cfg.total_epochs)
  File "/home/han/anaconda3/envs/maskrcnn2/lib/python3.6/site-packages/mmcv-0.2.13-py3.6-linux-x86_64.egg/mmcv/runner/runner.py", line 358, in run
    epoch_runner(data_loaders[i], **kwargs)
  File "/home/han/anaconda3/envs/maskrcnn2/lib/python3.6/site-packages/mmcv-0.2.13-py3.6-linux-x86_64.egg/mmcv/runner/runner.py", line 271, in train
    self.call_hook('after_train_iter')
  File "/home/han/anaconda3/envs/maskrcnn2/lib/python3.6/site-packages/mmcv-0.2.13-py3.6-linux-x86_64.egg/mmcv/runner/runner.py", line 229, in call_hook
    getattr(hook, fn_name)(self)
  File "/media/hdd/han/detections/ttfnet/mmdet/core/utils/dist_utils.py", line 57, in after_train_iter
    self.clip_grads(runner.model.parameters())
  File "/home/han/anaconda3/envs/maskrcnn2/lib/python3.6/site-packages/mmcv-0.2.13-py3.6-linux-x86_64.egg/mmcv/runner/hooks/optimizer.py", line 13, in clip_grads
    filter(lambda p: p.requires_grad, params), **self.grad_clip)
  File "/home/han/anaconda3/envs/maskrcnn2/lib/python3.6/site-packages/torch/nn/utils/clip_grad.py", line 33, in clip_grad_norm_
    total_norm += param_norm.item() ** norm_type
RuntimeError: CUDA error: unspecified launch failure
terminate called without an active exception

Environment

  • OS: [e.g., Ubuntu 16.04.6]
  • GCC [e.g., 5.4.0]
  • PyTorch version [e.g., 1.0.0]
  • How you installed PyTorch [e.g., conda]
  • GPU model [e.g., 1080Ti]
  • CUDA and CUDNN version
  • [optional] Other information that may be related (such as $PATH, $LD_LIBRARY_PATH, $PYTHONPATH, etc.)
cudatoolkit               9.0                  h13b8566_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
pytorch-nightly           1.0.0.dev20190318 py3.6_cuda9.0.176_cudnn7.4.2_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/pytorch

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.