Comments (15)
Thank you for your suggestions. It will be great to output features of selector as (n_obj, 520). It does save time for training. However, as far as I know, the input size of FCN cannot get changed. If not, could you provide me with some examples?
from sst.
Thanks for your reply. Actually I am not that familiar with Pytorch. I will try and report the results.
from sst.
Look forward to your good results :) .
from sst.
Thank you for your suggestions. It will be great to output features of selector as (n_obj, 520). It does save time for training. However, as far as I know, the input size of FCN cannot get changed. If not, could you provide me with some examples?
In semantic segmentation, it seems that the FCN network could deal with any size input image.
from sst.
Thank you for your reply, @SpyderXu . Now, I understand that. @nejyeah , Your idea is really great. I think this network can be greatly improved.
from sst.
Actually, the feature extraction process can be put in the detector (such as SSD), which will also improve this framework. I need to do that.
from sst.
Hi, the problem of varying size seems to locate at the trainning batch. In a Batch, we need to make the features to have the same size. The training loss may be modified to discard the padding value when using a fixed input size. And then the inference can take advantage of the varying size for just one frame each time.
from sst.
In the training process, the row and col of the labels have been shuffled so as the input feature(1040 x 80 x 80) before feeding into the compression network. Because the compression network just consists of 1x1 conv, the shuffle of the order(80 x 80) seems to be useless.
I am a beginner to the tracking. So I am grateful to you for sharing the nice project. I have learned a lot from it.
from sst.
Actually, the feature extraction process can be put in the detector (such as SSD), which will also improve this framework. I need to do that.
Good idea! For practical, the detect net and MOT net should be merged into one.
from sst.
In the training process, the row and col of the labels have been shuffled so as the input feature(1040 x 80 x 80) before feeding into the compression network. Because the compression network just consists of 1x1 conv, the shuffle of the order(80 x 80) seems to be useless.
Yes. as you say, the shuffle of the order is useless. @nejyeah . Thank you for your appreciation of my work. It's also my first work on tracking. Thank you for your suggestions again.
from sst.
Hi, the problem of varying size seems to locate at the trainning batch. In a Batch, we need to make the features to have the same size. The training loss may be modified to discard the padding value when using a fixed input size. And then the inference can take advantage of the varying size for just one frame each time.
I add dummy box into the MOTTrainDataset. So it seems that you need start from here and it should be lots of work. The training loss uses mask in order to remove the negative affect of dummy box. All the work is based on my first thinking which is not that perfect.
from sst.
I have tested the varying size for inference on GTX1080Ti for MOT17-01-FRCNN. For the original version, the inference time is about 0.266 s/frame. For the varying size, the inference time is about 0.243. And It saves 20 ms. And the results seems to make little difference from eyes based on the output vedio.
from sst.
That's great. By the way, different input boxes may take different time. Is this the average time. If possible, could you push a branch to this repository?
from sst.
Yes, it is the average time in your code for tracking test.
I just modified two functions in your code.
Modify layers/sst.py
126 def forward_stacker_features(self, xp, xn, fill_up_column=True):
127 pre_rest_num = self.max_object - xp.shape[1]
128 next_rest_num = self.max_object - xn.shape[1]
129 pre_num = xp.shape[1]
130 next_num = xn.shape[1]
131 ### x = self.forward_stacker2(
132 ### self.resize_dim(xp, pre_rest_num, dim=1),
133 ### self.resize_dim(xn, next_rest_num, dim=1)
134 ### )
x = self.forward_stacker2_inference(xp, xn)
135
136 x = self.final_dp(x)
137 # [B, N, N, 1]
138 x = self.forward_final(x, self.final_net)
139 x = x.contiguous()
140 # add zero
141 ### if next_num < self.max_object:
142 ### x[0, 0, :, next_num:] = 0
143 ### if pre_num < self.max_object:
144 ### x[0, 0, pre_num:, :] = 0
145 x = x[0, 0, :]
146 # add false unmatched row and column
147 x = self.resize_dim(x, 1, dim=0, constant=self.false_constant)
148 x = self.resize_dim(x, 1, dim=1, constant=self.false_constant)
149
150 x_f = F.softmax(x, dim=1)
151 x_t = F.softmax(x, dim=0)
152 # slice
153 last_row, last_col = x_f.shape
154 row_slice = list(range(pre_num)) + [last_row-1]
155 col_slice = list(range(next_num)) + [last_col-1]
156 ### x_f = x_f[row_slice, :]
157 ### x_f = x_f[:, col_slice]
158 ### x_t = x_t[row_slice, :]
159 ### x_t = x_t[:, col_slice]
Add function forward_stacker2_inference(), copy and revised from forward_statcker2 for not affecting training
223 def forward_stacker2_inference(self, stacker1_pre_output, stacker1_next_output):
224 ### stacker1_pre_output = stacker1_pre_output.unsqueeze(2).repeat(1, 1, self.max_object, 1).permute(0, 3, 1, 2)
225 ### stacker1_next_output = stacker1_next_output.unsqueeze(1).repeat(1, self.max_object, 1, 1).permute(0, 3, 1, 2)
n_pre = stacker1_pre_output.shape[1]
n_next = stacker1_next_output.shape[1]
stacker1_pre_output = stacker1_pre_output.unsqueeze(2).repeat(1, 1, n_next, 1).permute(0, 3, 1, 2)
stacker1_next_output = stacker1_next_output.unsqueeze(1).repeat(1, n_pre, 1, 1).permute(0, 3, 1, 2)
226
227 stacker1_pre_output = self.stacker2_bn(stacker1_pre_output.contiguous() )
228 stacker1_next_output = self.stacker2_bn(stacker1_next_output.contiguous ())
229
230 output = torch.cat(
231 [stacker1_pre_output, stacker1_next_output],
232 1
233 )
234
235 return output
from sst.
Great, Thank you.
from sst.
Related Issues (20)
- Issues on Track age
- All_similarity and all_iou
- ToPercentCoords
- Input questions about the Network Diagram
- Question about fps HOT 1
- Issues with training HOT 1
- I'm trying to understand the code
- UserWarning HOT 3
- Base network selection HOT 3
- Apply for License HOT 2
- The final affinity matrix HOT 1
- Incorrect ID assignments in the project
- Why the accuracy and speed of retrained model is not as good as the sst300_0712_83000.pth which is given by author?
- About FPS HOT 3
- Two important parameters
- Why DAN get high IDSW score?
- Get Bad results when retraining the DAN on MOT17?
- There is always "None processing" while training on MOT15 dataset HOT 1
- 运行到一半出错
- Understanding the Assembly Loss in the loss function
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from sst.