Git Product home page Git Product logo

posenet-pytorch's Introduction

PoseNet-Pytorch

This is the implementation of PoseNet, Bayesian PoseNet, PoseNet with Geometric Loss. For PoseNet with Geometric Loss, we only implemented homoscedastic uncertainty based automatic weight scaling on loss function. We selected ResNet34 as base architecture, and it resulted in the better performance than the original papers. The code is inspired by DejavuGAN and MapNet.

TODO

  • Minimal implementation of PoseNet
  • Minimal implementation of Bayesian PoseNet
  • Minimal implementation of PoseNet with Geometric loss
  • Complete Readme
  • Automatic save and load the configuration
  • Visualization of Saliency

Prerequisites (Tested environment)

  • Ubuntu 16.04
  • Python 3.5
  • Pytorch 0.4.x
  • Tensorboard (optional for visualization)
  • TensorboardX (optional for visualization)

Data

The dataset loader is based on dataset of original PoseNet. Please download on of the dataset in PoseNet Dataset. Also, you can use other datasets your own. Or, you can download by running the following script (KingsCollege Dataset from PoseNet).

wget https://www.repository.cam.ac.uk/bitstream/handle/1810/251342/KingsCollege.zip?sequence=4&isAllowed=y

Training

  • Train a model

    python3 train.py --image_path ./posenet/KingsCollege --metadata_path ./posenet/KingsCollege
    
    • Trained models are saved in ./models_{Last Folder name of image path} and the summaries (optional) are save in ./summaries_{Last Folder name of image path}.
    • (Optinal) To visualize the training procedure, run tensorboard --logdir ./summaries_{Last Folder name of image path} and check the URL http://localhost:6006.
  • Continue training

    python3 train.py --image_path ./posenet/KingsCollege --metadata_path ./posenet/KingsCollege --pretrained_model {desired epoch}
    

Optimization Scheme

Optimization Scheme

  • The above figure represent the example of optimziation scheme of training. The loss function is based on Adam optimizaer with rotation weigh parameter (beta) 500.
  • Currently, we add homoscedastic uncertainty based automatic weight scaling used in PoseNet with Geometric Loss (Maybe not same the original paper). Thus, you can check the change of s_x and s_q on training.

Test

  • Test a model with best model from validation

    python3 test.py
    
  • Test a model with the selected epoch

    python3 test.py --test_model {Desired Epoch}
    
  • Test a model as Baysian Inference (Dropout Inference)

    python3 test.py --baysian True
    

Related Project

  • We also try to implement other learning based pose regression methods

posenet-pytorch's People

Contributors

youngguncho avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

posenet-pytorch's Issues

pre-trained model parameters

Thank you for your work, I would like to ask if you have pre-trained model parameters here, and if so, can you make them public? I will be very grateful.

How is the PoseLoss loss function derived?

Great implementation of PoseNet! However, the loss doesn't seem to converge to minima when I start learning sx and sq. Any help with recreating the results in the Geometric Pose Loss paper would be a big help!

How can 'sx' and 'sq' capture the variance of position and orientation noise?

Hi,

As the implementation of the loss
loss = torch.exp(-self.sx)*loss_x + self.sx + torch.exp(-self.sq)*loss_q + self.sq
is equivalent to eq.4 in Kendall et al. paper, I still not understand how can the trainable parameters 'sx' and 'sq' can capture the variance of position and orientation noise from the model output? I mean, how come you implement the loss this way?

Thank you very much for your time. :)

Does sx mean...

Hi, I have a question!

Does sx mean the standard deviation(variance) of error : x-x' ?

I mean.. if we set batch size to 30,

then Sx = stard deviation of 30 x-x'? (Because x has three values (x, y, z), Sx = std of 90 numbers?)

negative loss while training on kings college dataset

20th train Loss: total loss -1.273 / pos loss 1.371 / ori loss 0.018
21th train Loss: total loss -1.584 / pos loss 1.084 / ori loss 0.016
22th train Loss: total loss -1.778 / pos loss 0.875 / ori loss 0.017
23th train Loss: total loss -1.884 / pos loss 0.764 / ori loss 0.018
24th train Loss: total loss -1.651 / pos loss 1.008 / ori loss 0.017
25th train Loss: total loss -1.831 / pos loss 0.801 / ori loss 0.018
26th train Loss: total loss -1.562 / pos loss 1.067 / ori loss 0.018
27th train Loss: total loss -1.713 / pos loss 0.965 / ori loss 0.016
28th train Loss: total loss -1.338 / pos loss 1.275 / ori loss 0.019
29th train Loss: total loss -1.722 / pos loss 0.964 / ori loss 0.016
30th train Loss: total loss -1.551 / pos loss 1.094 / ori loss 0.018
31th train Loss: total loss -1.603 / pos loss 1.084 / ori loss 0.016
32th train Loss: total loss -1.714 / pos loss 0.947 / ori loss 0.017
33th train Loss: total loss -1.465 / pos loss 1.029 / ori loss 0.025
34th train Loss: total loss -1.740 / pos loss 0.927 / ori loss 0.017
35th train Loss: total loss -1.579 / pos loss 0.987 / ori loss 0.022
36th train Loss: total loss -1.537 / pos loss 1.074 / ori loss 0.019
37th train Loss: total loss -1.515 / pos loss 1.123 / ori loss 0.018
38th train Loss: total loss -1.181 / pos loss 1.409 / ori loss 0.020
0th val Loss: total loss -2.384 / pos loss 0.434 / ori loss 0.009
1th val Loss: total loss -2.452 / pos loss 0.358 / ori loss 0.009
2th val Loss: total loss -2.407 / pos loss 0.375 / ori loss 0.011
3th val Loss: total loss -2.356 / pos loss 0.417 / ori loss 0.011
Train and Validaion error -1.696101188659668 / -2.395916223526001

========================================
Epoch 225/399

0th train Loss: total loss -1.572 / pos loss 1.093 / ori loss 0.017
1th train Loss: total loss -1.546 / pos loss 1.108 / ori loss 0.017
2th train Loss: total loss -1.847 / pos loss 0.873 / ori loss 0.014
3th train Loss: total loss -1.953 / pos loss 0.679 / ori loss 0.018
4th train Loss: total loss -1.753 / pos loss 0.908 / ori loss 0.017
5th train Loss: total loss -1.600 / pos loss 1.083 / ori loss 0.016
6th train Loss: total loss -1.617 / pos loss 1.014 / ori loss 0.018
7th train Loss: total loss -1.760 / pos loss 0.888 / ori loss 0.018
8th train Loss: total loss -1.264 / pos loss 1.346 / ori loss 0.019
9th train Loss: total loss -1.606 / pos loss 1.040 / ori loss 0.018
10th train Loss: total loss -1.619 / pos loss 0.982 / ori loss 0.020
11th train Loss: total loss -1.708 / pos loss 0.935 / ori loss 0.018
12th train Loss: total loss -1.645 / pos loss 1.058 / ori loss 0.015
13th train Loss: total loss -1.703 / pos loss 0.959 / ori loss 0.017
14th train Loss: total loss -1.768 / pos loss 0.860 / ori loss 0.019
15th train Loss: total loss -1.705 / pos loss 0.931 / ori loss 0.018
16th train Loss: total loss -1.712 / pos loss 0.958 / ori loss 0.016
17th train Loss: total loss -1.656 / pos loss 0.974 / ori loss 0.018
18th train Loss: total loss -1.686 / pos loss 0.984 / ori loss 0.016
19th train Loss: total loss -1.657 / pos loss 0.989 / ori loss 0.018

pos out [ 60.13612 -37.06963 1.4968748]
ori_out [ 1.43032163 0.86646476 -0.1398881 ]
pos true [ 58.309566 -36.34202 1.250287]
ori true [ 1.38532974 0.91265223 -0.18131388]
[ 60.13612 -37.06963 1.4968748]
[ 58.309566 -36.34202 1.250287]
1219th Error: pos error 1.982 / ori error 0.077

Overall median pose errer 0.973 / 0.054
Overall average pose errer 1.162 / 0.069

Pre-trained models?

Hi,
Great work on the repo!
I couldn't find any links/reference to any pretrained model files for a quick test.

Thanks

Bayesian Posenet

In the bayesian posenet article "Modelling Uncertainty in Deep Learning for Camera Relocalization", it is given that dropout layers are added after each convolutional layer, not at the last layer. Please fix that.

Can't download the King's College Dataset?

I tried both the wget command, and going to the link that directed me to the University of Cambridge site, but I got a proxy error for both attempts.

Is there any way to download the dataset?

Test.py

Hi!!! Thank you for your contribution.
The freshmen have some problems.
What does the test generated file mean?
What do each column of parameters mean? 6 parameters.
Looking forward to your reply.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.