honline-cpp-public's Introduction

Online Generative Model Personalization for Hand Tracking

Video: http://lgg.epfl.ch/publications/2017/HOnline/video.mp4
Paper PDF: http://lgg.epfl.ch/publications/2017/HOnline/paper.pdf
Dataset: http://lgg.epfl.ch/publications/2017/HOnline/guess-who.zip

Disclaimer

To obtain the results shown in the video proper hardware is necessary:

Windows 10
Intel Core i7 @4GhZ
CUDA Graphic card (NVIDIA GTX1080 used in our demo)
Inter RealSense depth camera

Other notes:

note the software must be compiled in 64bits
Wristband color calibration (make sure the wristband is detected robustly otherwise the tracking might not perform as effectively, you can check this by enabling "show wband" in the hmodel_atb application)

BibTex

@article{tkach2017online, title={Online generative model personalization for hand tracking}, author={Tkach, Anastasia and Tagliasacchi, Andrea and Remelli, Edoardo and Pauly, Mark and Fitzgibbon, Andrew}, journal={ACM Transactions on Graphics (TOG)}, volume={36}, number={6}, pages={243}, year={2017}, publisher={ACM} }

Running "Teaser" Sequence

in honline/apps/honline_atb/main.cpp

change 90 | worker.settings->sequence_path to the path to teaser dataset in you machine
change 91 | worker.settings->data_path to the path to honline/data/ folder on your machine
set 98| bool benchmarkto true

Running Live

in honline/apps/honline_atb/main.cpp

set 98 | bool benchmark to false
wear a blue wristband and make sure that the wristband is always in the view of the sensor

Tracking

Tracking on a fast machine

worker.settings->calibration_type = NONE
worker.E_fitting.settings->fit2D_outline_enable = true
worker.settings->termination_max_iters = 8

Tracking on a slow machine

worker.settings->calibration_type = NONE
worker.E_fitting.settings->fit2D_outline_enable = false
worker.settings->termination_max_iters = 6

Calibration

Independent

worker.settings->calibration_type = FULL;

Standard Kalman Filter

worker.settings->calibration_type = FULL;
worker.settings->run_kalman_filter = true;
worker.settings->kalman_filter_type = STANDARD;
worker.settings->display_estimated_certainty = true;

Extended Kalman Filter

worker.settings->calibration_type = FULL;
worker.settings->run_kalman_filter = true;
worker.settings->kalman_filter_type = EXTENDED;
worker.settings->display_estimated_certainty = true;

Batch

To add every 30th frame to batch solve

worker.settings->frames_interval_between_measurements = 30;

To add the current frame to batch solve

press "A"

To start batch solve

press "B"

To run more iterations of batch solve

press "B"

To stop batch solve and resume tracking (the system sometimes crashes at this point)

press "Shift + B"

Batch Online

worker.settings->calibration_type = FULL;
worker.settings->run_batch_solver = true;
worker.settings->use_online_betas_for_batch_solver = true;

Batch Offline

worker.settings->calibration_type = NONE;
worker.settings->run_batch_solver = true;
worker.settings->use_online_betas_for_batch_solver = false;

Manual Uniform Scaling

Press "1" to make the model bigger
Press "2" to make the model smaller
Press "3" to make the model wider
Press "4" to make the model narrower
Press "5" to make the model thicker
Press "6" to make the model thinner

honline-cpp-public's People

Stargazers

Watchers

honline-cpp-public's Issues

Scaling Issues

I'm using a RealSense SR300, and after fighting some build issues and getting things to stream properly, I'm noticing the size of the built-in hand model seems to be much much smaller than what is rendering via the streaming point cloud.

Here's a picture showing the scale:

And a picture showing what happens when it somehow starts tracking and locks on. It seems to lock into a region of interest and stick around there flopping around as it goes in and out of losing tracking.

I've messed with the clipping planes, and among many other things have edited (and switched to using) the "INTEL" camera mode to have the following parameters:

           _width = 640/2;
           _height = 480/2;
           px = 301.461975/2; //have commented this one out since it doesn't seem to be used.
	   py = 239.074234/2; // ^^^
           _focal_length_x = 611.828552/2;
           _focal_length_y = 611.828674/2;
           _zFar = 5000;
           _zNear = 150;

Thanks!

Problems about Running "Teaser" Sequence

First i build your project successfully, My vs2015 did not report any errors.

Then:
I follow your "README" , and set
worker.settings->sequence_path = "C:/Users/78513/Downloads/"; (changed)
worker.settings->data_path = "O:/githubProjects/honline-cpp-public/data/"; (changed)
worker.settings->calibrated_model_path = "C:/Data/honline-results/calibrated-models/experiments/"; (not change)
worker.settings->sequence_name = "teaser"; (changed)
worker.settings->logs_path = "C:/Users/tkach/Desktop/Test/"; (not change)

AND

   benchmark = true;

BUT i get an breakdown. I did not know how to deal with this problem, or your README may have some wrong?

Recommend Projects

anastasia-tkach / honline-cpp-public Goto Github PK