z-x-yang / aot Goto Github PK
View Code? Open in Web Editor NEWAssociating Objects with Transformers for Video Object Segmentation
License: BSD 3-Clause "New" or "Revised" License
Associating Objects with Transformers for Video Object Segmentation
License: BSD 3-Clause "New" or "Revised" License
Hi,
I have read some of your great papers for VOS.
I noticed that models are usually first pre-trained on image datasets (COCO, VOC, etc.) and then trained on video datasets (DAVIS, YTB). May I know some experience or results about the performance gap that training with/without image datasets?
In addition, does the MobileNetV2 in this work pre-trained on ImageNet or trained from scratch?
Many thanks in advance.
Hi, love your AOT/AOST. Excellent scalability. The open-source code is another big plus.
I was wondering what counts as "all frames" in Table 1 of AOST? Most papers don't mention whether they use 6FPS/30FPS (and they also don't release the code!) The authors of STM did say they use all the frames: seoungwugoh/STM#3 (comment). HMMN uses a similar evaluation structure so it's likely also the case; KMN has no code but their baseline score matches STM so it is also likely that they use the 30 FPS version. Am I missing something here?
Cheers.
May I ask the number of params of the LSTT block?
Except the feature map output by LSTT, do you also directly add the output feature of the backbone to the input of the encoder? And do you add features with different scale from backbone to the decoder?
Do you use dropout in Long-Term Attention and Short-Term Attention?
Hi,
I haven't explored your algorithm yet but I was wondering if it could be useful to me.
I wanted to know if this solution can also be used in a context where there are several images of the same object from multiple angles (not necessarily in order) and therefore not a video of the same.
Are there any theoretical bases that motivate this or some experiments done?
Thanks
Gianluca
When open the source code ?
Do you realized key_padding_mask
(a parameter in torch.nn.multiheadattention
) in the long-term attention and short-term Attention? Or you just do not need this in your network?
Hello,
Is it possible to provide an example video or two which demonstrate the results of the method?
Thanks,
Chris
Hi Zongxin,
May I know more details about the training strategies?
Thank you.
Hi,
Do you consider the background like an object when embedding identification or you aggregate the background after decoding?
Thanks.
Hi, Zongxin,
Thank you for the nice work. I am concerned about the training detail.
Hi Zongxin,
Congrats on a series of great works as well as Segment-and-Track Everything Project!
Now I am working on testing the Track Everything project on a multi-view microscopic dataset where the severe occlusions are likely happening and my target is tracking many small "black circles" like only 20 x 20 pixel size in a 1024 x 1024 full image while may have more than hundreds in a single image.
I am a bit surprised to see after some efforts, your model gave us a pretty nice result but still sometimes wrongly tracking multiple circles as the same or the mask could be kind of rough.
I guess there may be some additional finetuning work awaiting for us, though I am not the expert in this area. My current efforts is more like exploring recent SOTA works and preparing datasets for them. Unfortunately, I can't find some way to optimize your model for tracking part (I have already finetuned SAM in our task so every frame can produce pretty nice mask but I cannot track them correctly).
Could you please give me some advice? And any plan to release code for customized dataset finetuning? I am very happy to help to test & discuss if you are interested in how well your model can work with medical & microscopic data like CT, MRI or Cryo-EM, Cryo-ET after finetuning.
Thanks in advance!
May I ask the number of params of AOT-T, AOT-S, AOT-B, AOT-L?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.