Git Product home page Git Product logo

Comments (18)

devyhia avatar devyhia commented on August 21, 2024 3

Dear @amoudgl,

I tried to reproduce a working GoTurn model from this project. However, I think there is a problem with the data generator as it does not converge beyond a certain loss (for me that loss is 6.0). It begins at loss=12.xx and converges to 6.0xx. However, I believe it should converge beyond that as well. I think it might be an issue with the dataset generator. For example, in the original C++ implementation, I see that the bounding boxes are squeezed into (0, 1) and then scaled by a factor of 10 to be within a (0, 10) range. However, you don't do that kind of normalization. You do the context scaling (i.e. doubling the size of the box by kContextFactor=2, then re-center the bounding box). However, the final bounding box is not a ratio of the original image dimension.

I have been tinkering around with your implementation for a couple of days and trying to get it work. What really confuses me is the bounding box pre-processing. Did you get it to work on your end?

Best,
Yehya

from pygoturn.

amoudgl avatar amoudgl commented on August 21, 2024 2

Hi @devyhia,

I do normalize the values of bounding boxes in my training code here. However, your observation is correct that loss doesn't converge beyond a certain value.

I believe that it is happening due to huge difference between the model capacity and dataset size on which we are training. We are using AlexNet for training on ALOV dataset (just 16k images), which is actually an overkill. However, an argument can still be given that if training size is small, model should overfit and loss should still converge to really low values which is clearly not happening here. I experimented a little and found out that it is happening due to stochastic behaviour of dropout in the model. If you try removing the dropout layers, you should get the desired results (but model would still be overfitting). Correct me if I am wrong somewhere.

Also, I recently updated the code to test GOTURN on OTB formatted sequences. The basic framework of training and testing is ready, only ImageNet training is left. Let me know if you face any more issues.

Thanks for your review.

from pygoturn.

devyhia avatar devyhia commented on August 21, 2024 1

@amoudgl The dropout is a true observation as well. I could make the model converge with a dropout of 0.1 (rather than the previous p=0.5). By the way, I forked this project and added few nice things for the training. For examples:

  1. Ability to resume the training
  2. Ability to visualize the training using visdom.
  3. The p=0.1 dropout.
  4. The ability to resume training from a specific epoch (i.e. to utilize your learning scheduler-- to begin with a smaller learning rate from later epochs).

I got the model to converge somewhere around loss=1.xx. Also, the visualization show clearly that it is learning something meaningful. By the way, I also tried to overfit the model using both SGD and Adam optimizer. Adam converges much faster (i.e. in terms of number of epochs) than SGD— which is a bit expected. However, it is slower. On a GPU (and batch size=1), SGD yields 100 iterations in 5 seconds where Adam yields the same 100 iterations in 12 seconds.

By the way, if you are trying to re-produce a working version of Held's paper, we could collaborate on it. I already have a fork of your repository. You could check it out here: https://github.com/devyhia/pygoturn/

from pygoturn.

amoudgl avatar amoudgl commented on August 21, 2024 1

Hi! I fixed training and testing scripts recently. Training is still going on and I'm doing a few experiments to further fix errors (if any). I tested some of the intermediate training checkpoints on OTB sequences and they look fine. I'd try to release the pretrained model as soon as possible.

To estimate, it may take a week to release the final trained model. Thanks for your comment!

from pygoturn.

fzyzcjy avatar fzyzcjy commented on August 21, 2024 1

@amoudgl Add oil! 😄

from pygoturn.

amoudgl avatar amoudgl commented on August 21, 2024 1

Hi! I finished everything, finally. Please have a look at the updated README. Thank you all!

from pygoturn.

amoudgl avatar amoudgl commented on August 21, 2024

Hi! Thanks for showing your interest.

I got busy with some other urgent research commitments while implementing this, so the project is still incomplete. Would surely update it with the proper README when it's done (hopefully by the end of November).

from pygoturn.

wangxiao5791509 avatar wangxiao5791509 commented on August 21, 2024

@amoudgl Cool. Looking forward to this project. Thanks for your kind share.

from pygoturn.

amoudgl avatar amoudgl commented on August 21, 2024

@devyhia Thanks for pointing out the scaling issue. Just fixed it.

from pygoturn.

amoudgl avatar amoudgl commented on August 21, 2024

@devyhia Great effort, indeed. Feel free to send pull requests for these additional functionalities, just make sure that your commit names are neat so that we can have a clean history of commits for the project.

Yes, we can surely collaborate. I have sent you an invite on Slack (using your gmail address), let's move forward with the detailed discussion there.

from pygoturn.

sydney0zq avatar sydney0zq commented on August 21, 2024

@amoudgl Could I add your team? Recently I also reimplement this project and have a lot of interests. My account is [email protected] on slack.

Thank you.

from pygoturn.

amoudgl avatar amoudgl commented on August 21, 2024

@sydney0zq Have you worked on implementing motion smoothness model as mentioned in the GOTURN paper? If no, can you work on it? Because apart from this part, we have already divided the work and started working on it.

from pygoturn.

sydney0zq avatar sydney0zq commented on August 21, 2024

@amoudgl I am not familiar with the smooth motion model, but I am current doing some research on GOTURN to improve its performance(The result is not good on VOT/OTB). So maybe I have no time for it.

Have you prepared code for testing on benchmarks such as OTB or VOT? If not, maybe I could make some commits.

from pygoturn.

amoudgl avatar amoudgl commented on August 21, 2024

No problem @sydney0zq. We are currently working on completing the training part first. The testing code works for the OTB-formatted sequences but we don't have any proper pretrained model yet. Maybe, after we are done with the training and all, we would let you know and then, you can work with us on benchmarking and integrating our PyTorch pretrained model into VOT and OTB.

from pygoturn.

devyhia avatar devyhia commented on August 21, 2024

@amoudgl @sydney0zq I have a fairly working model. However, it is not as robust as Held's original model (or, those trained on Caffe). It works for some VOT videos (like the ball video in VOT 2014). @sydney0zq If you need this, I could send it to you.

from pygoturn.

sydney0zq avatar sydney0zq commented on August 21, 2024

@devyhia Yep, could you please share it with me, along with your code? I am still training on just ALOV. I will mention you as soon as I get a reasonable result.

from pygoturn.

IceClear avatar IceClear commented on August 21, 2024

@devyhia Hi, I am now working with a pytorch project that may need a tracking model. So would you send a working model of GUTURN to me ? Just email me. That may be very helpful, thanks a lot!

from pygoturn.

fzyzcjy avatar fzyzcjy commented on August 21, 2024

@amoudgl Hi! Thank you for your wonderful pygoturn! I wonder whether it is finished now?

from pygoturn.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.