Git Product home page Git Product logo

hfl-net's Introduction

Harmonious Feature Learning for Interactive Hand-Object Pose Estimation

Directory

${ROOT}  
|-- data  
|   |-- HO3D
|   |   |-- data
|   |   |   |-- train
|   |   |   |   |-- ABF10
|   |   |   |   |-- ......
|   |   |   |-- evaluation
|   |   |   |-- train_segLable
|   |   |   |-- ho3d_train_data.json
|   |-- DEX_YCB
|   |   |-- data
|   |   |   |-- 20200709-subject-01
|   |   |   |-- ......
|   |   |   |-- object_render
|   |   |   |-- dex_ycb_s0_train_data.json
|   |   |   |-- dex_ycb_s0_test_data.json

Data

You need to follow directory structure of the data as below.

  • Download HO3D(version 2) data data
  • Download DexYCB data data
  • Download the process data data

Pytorch MANO layer

  • For the MANO layer, I used manopth. The repo is already included in manopth.
  • Download MANO_RIGHT.pkl and MANO_LEFT.pkl from here and place at assets/mano_models.

Train

HO3d

sh sh/train_ho3d.sh

Dex-ycb

sh sh/train_dex-ycb.sh

Test

HO3d

sh sh/train_ho3d_test.sh

Dex-ycb

sh sh/train_dex-ycb_test.sh

Acknowledgments

We thank:

hfl-net's People

Contributors

lzfff12 avatar

Stargazers

avatar avatar avatar avatar avatar Daniel Wild avatar benh_g avatar Xiaoge Cao avatar avatar Jianxff avatar Jun Zhou avatar Ge Yuting avatar Karim Abou Zeid avatar Arno.Meng avatar TIen avatar JunyuZhang avatar Sinkers Lan avatar gusang avatar HaoXu avatar 一尊还酹江月丶 avatar Junya Morioka avatar MY avatar Shuqiang Cao avatar YinqiaoWang avatar BAI Yuan avatar SeanD avatar avatar

Watchers

JJMa-coder avatar avatar

Forkers

my-code-1981 scanood goldonkey dqj5182

hfl-net's Issues

About specific training setting and result

First thanks for your nice work. Personally, I have tried retraining it myself and got results, but this result differs especially on dexycb from the pretrained checkpoint you provided (approximately 13.0 vs 12.5 difference for hand mpjpe). So I wonder if there is a difference between the specific setup of your training and the code in this github repository. Or some tricks when you're training. Or maybe it's simply me doing something wrong? I need your opinion.
Also, if possible, please tell me your training speed, because I feel that my training speed is a bit slow, no noticeable improvement using multiple GPUs :(

about result

hello! I reproduced the results according to your paper code, and the difference between my results and the original result in your paper is significant. Can you provide the checkpoint. pth file?

About the re-trained results on HO3D dataset

Hello, @lzfff12 . Thanks for your excellent work.

I tried to retrain the model on the HO3Dv2 dataset using your default settings for three times. However, the best result I obtained is only 9.38 on the public leaderboard.
Therefore, I wonder if you have some other tricks applied during training to boost the performance?

The following are the parameter output in option.txt and the trained losses for reference.

====== Options ======
HO3D_root: /media/data1/Dataset/HO3D_v2/Data
blocks: 1
channels: 256
dex_ycb_root: /media/data1/Dataset/data_for_alignsdf/dexycb/dex-ycb
epochs: 70
evaluate: False
host_folder: experiments/ho3d
inp_res: 256
lambda_joints2d: 100.0
lambda_objects: 500.0
lr: 0.0001
lr_decay_gamma: 0.7
lr_decay_step: 10
mano_lambda_joints3d: 10000.0
mano_lambda_manopose: 10
mano_lambda_manoshape: 0.1
mano_lambda_regulpose: 0.0
mano_lambda_regulshape: 0.0
mano_lambda_verts3d: 10000.0
mano_neurons: [1024, 512]
mano_root: assets/mano_models
manual_seed: 0
momentum: 0.9
network: honet_transformer
obj_model_root: assets/object_models
resume: None
save_results: True
snapshot: 10
stacks: 1
test_batch: 64
test_freq: 10
train_batch: 64
transformer_depth: 1
transformer_head: 4
use_cuda: 1
use_ho3d: True
weight_decay: 0.0005
workers: 16
=====================
launched traineval.py at 2023-10-06 23:15:23.422155

==== log  at Fri Oct  6 23:15:31 2023 ====
(epoch: 1, time: Fri Oct  6 23:42:02 2023),mano_mesh3d_loss:5.545252323150635,mano_joints3d_loss:5.8843488693237305,manopose_loss:1.0071043968200684,manoshape_loss:0.3506675064563751,mano_total_loss:12.787386894226074,hm_joints2d_loss:3.3286962509155273,obj_reg_loss:95.60641479492188,obj_conf_loss:10.81941032409668,total_loss:122.54190063476562
(epoch: 2, time: Sat Oct  7 00:01:25 2023),mano_mesh3d_loss:1.6909847259521484,mano_joints3d_loss:1.8189302682876587,manopose_loss:0.3632090091705322,manoshape_loss:0.2927539646625519,mano_total_loss:4.165874004364014,hm_joints2d_loss:1.0852590799331665,obj_reg_loss:74.8833999633789,obj_conf_loss:6.6124725341796875,total_loss:86.74691009521484
(epoch: 3, time: Sat Oct  7 00:20:30 2023),mano_mesh3d_loss:1.320427417755127,mano_joints3d_loss:1.418541669845581,manopose_loss:0.33809858560562134,manoshape_loss:0.2805778682231903,mano_total_loss:3.3576478958129883,hm_joints2d_loss:0.7900612354278564,obj_reg_loss:68.6631088256836,obj_conf_loss:6.496152877807617,total_loss:79.30706024169922
(epoch: 4, time: Sat Oct  7 00:39:24 2023),mano_mesh3d_loss:1.0319397449493408,mano_joints3d_loss:1.1067063808441162,manopose_loss:0.31386926770210266,manoshape_loss:0.27310410141944885,mano_total_loss:2.725619316101074,hm_joints2d_loss:0.626470148563385,obj_reg_loss:50.83426284790039,obj_conf_loss:6.533202648162842,total_loss:60.71949768066406
(epoch: 5, time: Sat Oct  7 00:58:18 2023),mano_mesh3d_loss:1.109512209892273,mano_joints3d_loss:1.1911731958389282,manopose_loss:0.3025909662246704,manoshape_loss:0.26847589015960693,mano_total_loss:2.8717548847198486,hm_joints2d_loss:0.6564722061157227,obj_reg_loss:32.07004165649414,obj_conf_loss:6.203098773956299,total_loss:41.8013801574707
(epoch: 6, time: Sat Oct  7 01:17:10 2023),mano_mesh3d_loss:0.8024893999099731,mano_joints3d_loss:0.8596831560134888,manopose_loss:0.2736554443836212,manoshape_loss:0.2501206696033478,mano_total_loss:2.1859476566314697,hm_joints2d_loss:0.49380725622177124,obj_reg_loss:24.4095458984375,obj_conf_loss:5.715297698974609,total_loss:32.8045654296875
(epoch: 7, time: Sat Oct  7 01:36:03 2023),mano_mesh3d_loss:0.7018308043479919,mano_joints3d_loss:0.7514442801475525,manopose_loss:0.2519541382789612,manoshape_loss:0.23022988438606262,mano_total_loss:1.9354584217071533,hm_joints2d_loss:0.4389413297176361,obj_reg_loss:21.660181045532227,obj_conf_loss:5.430682182312012,total_loss:29.465261459350586
(epoch: 8, time: Sat Oct  7 01:54:57 2023),mano_mesh3d_loss:0.7097055912017822,mano_joints3d_loss:0.760745108127594,manopose_loss:0.237700954079628,manoshape_loss:0.21521404385566711,mano_total_loss:1.9233664274215698,hm_joints2d_loss:0.4387645721435547,obj_reg_loss:20.013832092285156,obj_conf_loss:5.234278202056885,total_loss:27.61023712158203
(epoch: 9, time: Sat Oct  7 02:13:49 2023),mano_mesh3d_loss:0.6313846707344055,mano_joints3d_loss:0.6764567494392395,manopose_loss:0.21927033364772797,manoshape_loss:0.19187459349632263,mano_total_loss:1.7189861536026,hm_joints2d_loss:0.4031234681606293,obj_reg_loss:18.915592193603516,obj_conf_loss:5.095491886138916,total_loss:26.133188247680664
(epoch: 10, time: Sat Oct  7 02:32:41 2023),mano_mesh3d_loss:0.523613452911377,mano_joints3d_loss:0.5597004890441895,manopose_loss:0.20173479616641998,manoshape_loss:0.1715564727783203,mano_total_loss:1.4566054344177246,hm_joints2d_loss:0.34694531559944153,obj_reg_loss:18.07473373413086,obj_conf_loss:4.9923834800720215,total_loss:24.870689392089844
(epoch: 11, time: Sat Oct  7 02:54:18 2023),mano_mesh3d_loss:0.4169944226741791,mano_joints3d_loss:0.4443188011646271,manopose_loss:0.18351642787456512,manoshape_loss:0.15521594882011414,mano_total_loss:1.2000457048416138,hm_joints2d_loss:0.2933536171913147,obj_reg_loss:16.09233856201172,obj_conf_loss:4.738110542297363,total_loss:22.323837280273438
(epoch: 12, time: Sat Oct  7 03:13:10 2023),mano_mesh3d_loss:0.4134601652622223,mano_joints3d_loss:0.44073936343193054,manopose_loss:0.17495545744895935,manoshape_loss:0.14630506932735443,mano_total_loss:1.175460696220398,hm_joints2d_loss:0.28917309641838074,obj_reg_loss:15.431581497192383,obj_conf_loss:4.642738342285156,total_loss:21.538957595825195
(epoch: 13, time: Sat Oct  7 03:32:06 2023),mano_mesh3d_loss:0.38888412714004517,mano_joints3d_loss:0.41429993510246277,manopose_loss:0.16559498012065887,manoshape_loss:0.13578476011753082,mano_total_loss:1.1045626401901245,hm_joints2d_loss:0.27534645795822144,obj_reg_loss:15.078113555908203,obj_conf_loss:4.583775043487549,total_loss:21.04180145263672
(epoch: 14, time: Sat Oct  7 03:51:07 2023),mano_mesh3d_loss:0.3549215495586395,mano_joints3d_loss:0.37789350748062134,manopose_loss:0.15566569566726685,manoshape_loss:0.12407279014587402,mano_total_loss:1.0125529766082764,hm_joints2d_loss:0.25565218925476074,obj_reg_loss:14.624402046203613,obj_conf_loss:4.510421276092529,total_loss:20.403018951416016
(epoch: 15, time: Sat Oct  7 04:10:10 2023),mano_mesh3d_loss:0.33881744742393494,mano_joints3d_loss:0.3610582649707794,manopose_loss:0.14748990535736084,manoshape_loss:0.11132899671792984,mano_total_loss:0.9586951732635498,hm_joints2d_loss:0.24588024616241455,obj_reg_loss:14.220447540283203,obj_conf_loss:4.439467906951904,total_loss:19.864479064941406
(epoch: 16, time: Sat Oct  7 04:29:12 2023),mano_mesh3d_loss:0.344839483499527,mano_joints3d_loss:0.36794713139533997,manopose_loss:0.1418749988079071,manoshape_loss:0.10055044293403625,mano_total_loss:0.9552118182182312,hm_joints2d_loss:0.2514665722846985,obj_reg_loss:14.002887725830078,obj_conf_loss:4.404102802276611,total_loss:19.613676071166992
(epoch: 17, time: Sat Oct  7 04:48:17 2023),mano_mesh3d_loss:0.31140848994255066,mano_joints3d_loss:0.33215394616127014,manopose_loss:0.13374680280685425,manoshape_loss:0.09187189489603043,mano_total_loss:0.8691810369491577,hm_joints2d_loss:0.23249368369579315,obj_reg_loss:13.677811622619629,obj_conf_loss:4.357553958892822,total_loss:19.13705062866211
(epoch: 18, time: Sat Oct  7 05:07:23 2023),mano_mesh3d_loss:0.3076970875263214,mano_joints3d_loss:0.32838159799575806,manopose_loss:0.12850818037986755,manoshape_loss:0.08531295508146286,mano_total_loss:0.8499000072479248,hm_joints2d_loss:0.23005546629428864,obj_reg_loss:13.268312454223633,obj_conf_loss:4.292179107666016,total_loss:18.640451431274414
(epoch: 19, time: Sat Oct  7 05:26:21 2023),mano_mesh3d_loss:0.28561967611312866,mano_joints3d_loss:0.3048730790615082,manopose_loss:0.12182731181383133,manoshape_loss:0.0790138989686966,mano_total_loss:0.7913343906402588,hm_joints2d_loss:0.21642200648784637,obj_reg_loss:13.026782989501953,obj_conf_loss:4.262256622314453,total_loss:18.296777725219727
(epoch: 20, time: Sat Oct  7 05:45:18 2023),mano_mesh3d_loss:0.277950257062912,mano_joints3d_loss:0.2968460023403168,manopose_loss:0.11732558161020279,manoshape_loss:0.07458312064409256,mano_total_loss:0.7667051553726196,hm_joints2d_loss:0.21429479122161865,obj_reg_loss:12.710926055908203,obj_conf_loss:4.2091264724731445,total_loss:17.901058197021484
(epoch: 21, time: Sat Oct  7 06:06:40 2023),mano_mesh3d_loss:0.22853213548660278,mano_joints3d_loss:0.24356263875961304,manopose_loss:0.10862389206886292,manoshape_loss:0.06617280840873718,mano_total_loss:0.6468917727470398,hm_joints2d_loss:0.19150859117507935,obj_reg_loss:11.651315689086914,obj_conf_loss:4.061398506164551,total_loss:16.55109977722168
(epoch: 22, time: Sat Oct  7 06:25:32 2023),mano_mesh3d_loss:0.22958622872829437,mano_joints3d_loss:0.24495862424373627,manopose_loss:0.10522033274173737,manoshape_loss:0.06243627518415451,mano_total_loss:0.6422010064125061,hm_joints2d_loss:0.1889542043209076,obj_reg_loss:11.409490585327148,obj_conf_loss:4.019417762756348,total_loss:16.260072708129883
(epoch: 23, time: Sat Oct  7 06:44:26 2023),mano_mesh3d_loss:0.2153177112340927,mano_joints3d_loss:0.22962281107902527,manopose_loss:0.10147976875305176,manoshape_loss:0.058818519115448,mano_total_loss:0.6052390336990356,hm_joints2d_loss:0.18056973814964294,obj_reg_loss:11.198007583618164,obj_conf_loss:3.98140025138855,total_loss:15.965216636657715
(epoch: 24, time: Sat Oct  7 07:03:19 2023),mano_mesh3d_loss:0.20651397109031677,mano_joints3d_loss:0.22022728621959686,manopose_loss:0.09777483344078064,manoshape_loss:0.05607279762625694,mano_total_loss:0.580588698387146,hm_joints2d_loss:0.1739843785762787,obj_reg_loss:11.012195587158203,obj_conf_loss:3.9543087482452393,total_loss:15.721080780029297
(epoch: 25, time: Sat Oct  7 07:22:12 2023),mano_mesh3d_loss:0.20141790807247162,mano_joints3d_loss:0.21492783725261688,manopose_loss:0.09459319710731506,manoshape_loss:0.0531650073826313,mano_total_loss:0.5641037821769714,hm_joints2d_loss:0.17087452113628387,obj_reg_loss:10.864304542541504,obj_conf_loss:3.9264392852783203,total_loss:15.52572250366211
(epoch: 26, time: Sat Oct  7 07:41:09 2023),mano_mesh3d_loss:0.201676607131958,mano_joints3d_loss:0.21520575881004333,manopose_loss:0.09257832169532776,manoshape_loss:0.0514155849814415,mano_total_loss:0.5608764886856079,hm_joints2d_loss:0.17022444307804108,obj_reg_loss:10.622261047363281,obj_conf_loss:3.8847124576568604,total_loss:15.238078117370605
(epoch: 27, time: Sat Oct  7 08:00:07 2023),mano_mesh3d_loss:0.19071389734745026,mano_joints3d_loss:0.20348554849624634,manopose_loss:0.08914336562156677,manoshape_loss:0.04889301583170891,mano_total_loss:0.5322357416152954,hm_joints2d_loss:0.16337093710899353,obj_reg_loss:10.549095153808594,obj_conf_loss:3.870743751525879,total_loss:15.115458488464355
(epoch: 28, time: Sat Oct  7 08:19:07 2023),mano_mesh3d_loss:0.23688818514347076,mano_joints3d_loss:0.2534085214138031,manopose_loss:0.0935741439461708,manoshape_loss:0.05119090899825096,mano_total_loss:0.6350612640380859,hm_joints2d_loss:0.19653542339801788,obj_reg_loss:10.403693199157715,obj_conf_loss:3.8390674591064453,total_loss:15.074353218078613
(epoch: 29, time: Sat Oct  7 08:38:06 2023),mano_mesh3d_loss:0.19114631414413452,mano_joints3d_loss:0.20406465232372284,manopose_loss:0.08648949861526489,manoshape_loss:0.04697412624955177,mano_total_loss:0.5286747813224792,hm_joints2d_loss:0.16974963247776031,obj_reg_loss:10.26810073852539,obj_conf_loss:3.820260763168335,total_loss:14.786787033081055
(epoch: 30, time: Sat Oct  7 08:57:06 2023),mano_mesh3d_loss:0.17334526777267456,mano_joints3d_loss:0.1849597543478012,manopose_loss:0.08274952322244644,manoshape_loss:0.04422345012426376,mano_total_loss:0.48527833819389343,hm_joints2d_loss:0.1554250419139862,obj_reg_loss:10.121633529663086,obj_conf_loss:3.7856688499450684,total_loss:14.548008918762207
(epoch: 31, time: Sat Oct  7 09:18:31 2023),mano_mesh3d_loss:0.16478119790554047,mano_joints3d_loss:0.17587342858314514,manopose_loss:0.07944251596927643,manoshape_loss:0.041589926928281784,mano_total_loss:0.4616870880126953,hm_joints2d_loss:0.15273596346378326,obj_reg_loss:9.506138801574707,obj_conf_loss:3.6882781982421875,total_loss:13.80883502960205
(epoch: 32, time: Sat Oct  7 09:37:30 2023),mano_mesh3d_loss:0.15073515474796295,mano_joints3d_loss:0.16063547134399414,manopose_loss:0.07667186111211777,manoshape_loss:0.04007638618350029,mano_total_loss:0.4281182885169983,hm_joints2d_loss:0.14260604977607727,obj_reg_loss:9.351908683776855,obj_conf_loss:3.6598691940307617,total_loss:13.582501411437988
(epoch: 33, time: Sat Oct  7 09:56:27 2023),mano_mesh3d_loss:0.14841628074645996,mano_joints3d_loss:0.15824133157730103,manopose_loss:0.07510595768690109,manoshape_loss:0.0386144183576107,mano_total_loss:0.4203777015209198,hm_joints2d_loss:0.1396702080965042,obj_reg_loss:9.239740371704102,obj_conf_loss:3.633779287338257,total_loss:13.433578491210938
(epoch: 34, time: Sat Oct  7 10:15:24 2023),mano_mesh3d_loss:0.14759379625320435,mano_joints3d_loss:0.1574031263589859,manopose_loss:0.07368716597557068,manoshape_loss:0.03823874145746231,mano_total_loss:0.4169226884841919,hm_joints2d_loss:0.13818441331386566,obj_reg_loss:9.119954109191895,obj_conf_loss:3.6089816093444824,total_loss:13.284035682678223
(epoch: 35, time: Sat Oct  7 10:34:22 2023),mano_mesh3d_loss:0.14904358983039856,mano_joints3d_loss:0.15902993083000183,manopose_loss:0.07242788374423981,manoshape_loss:0.037397585809230804,mano_total_loss:0.4178985357284546,hm_joints2d_loss:0.1392756700515747,obj_reg_loss:9.069344520568848,obj_conf_loss:3.5924813747406006,total_loss:13.219003677368164
(epoch: 36, time: Sat Oct  7 10:53:18 2023),mano_mesh3d_loss:0.14350050687789917,mano_joints3d_loss:0.15311022102832794,manopose_loss:0.0709749087691307,manoshape_loss:0.036276381462812424,mano_total_loss:0.4038615822792053,hm_joints2d_loss:0.13455429673194885,obj_reg_loss:9.051795959472656,obj_conf_loss:3.595832347869873,total_loss:13.186050415039062
(epoch: 37, time: Sat Oct  7 11:12:13 2023),mano_mesh3d_loss:0.14273126423358917,mano_joints3d_loss:0.15233394503593445,manopose_loss:0.06969793140888214,manoshape_loss:0.03588210791349411,mano_total_loss:0.40064552426338196,hm_joints2d_loss:0.13360971212387085,obj_reg_loss:8.948972702026367,obj_conf_loss:3.57106614112854,total_loss:13.054300308227539
(epoch: 38, time: Sat Oct  7 11:31:07 2023),mano_mesh3d_loss:0.13943752646446228,mano_joints3d_loss:0.14888498187065125,manopose_loss:0.0683208778500557,manoshape_loss:0.034927383065223694,mano_total_loss:0.3915705680847168,hm_joints2d_loss:0.1317022144794464,obj_reg_loss:8.835185050964355,obj_conf_loss:3.544879674911499,total_loss:12.903342247009277
(epoch: 39, time: Sat Oct  7 11:50:01 2023),mano_mesh3d_loss:0.13801789283752441,mano_joints3d_loss:0.14734631776809692,manopose_loss:0.06706735491752625,manoshape_loss:0.03384441137313843,mano_total_loss:0.38627585768699646,hm_joints2d_loss:0.13088363409042358,obj_reg_loss:8.772802352905273,obj_conf_loss:3.527719736099243,total_loss:12.817673683166504
(epoch: 40, time: Sat Oct  7 12:08:56 2023),mano_mesh3d_loss:0.13565415143966675,mano_joints3d_loss:0.14483524858951569,manopose_loss:0.06606493145227432,manoshape_loss:0.03335905075073242,mano_total_loss:0.3799135088920593,hm_joints2d_loss:0.12853878736495972,obj_reg_loss:8.71053695678711,obj_conf_loss:3.5165371894836426,total_loss:12.735530853271484
(epoch: 41, time: Sat Oct  7 12:30:16 2023),mano_mesh3d_loss:0.12016043812036514,mano_joints3d_loss:0.12817123532295227,manopose_loss:0.06273456662893295,manoshape_loss:0.031158236786723137,mano_total_loss:0.3422246277332306,hm_joints2d_loss:0.11996433883905411,obj_reg_loss:8.308180809020996,obj_conf_loss:3.449866771697998,total_loss:12.220237731933594
(epoch: 42, time: Sat Oct  7 12:49:10 2023),mano_mesh3d_loss:0.11684741824865341,mano_joints3d_loss:0.1246257871389389,manopose_loss:0.06163860112428665,manoshape_loss:0.030793393030762672,mano_total_loss:0.33390501141548157,hm_joints2d_loss:0.11763712018728256,obj_reg_loss:8.220166206359863,obj_conf_loss:3.4317102432250977,total_loss:12.103408813476562
(epoch: 43, time: Sat Oct  7 13:08:03 2023),mano_mesh3d_loss:0.11615670472383499,mano_joints3d_loss:0.12390806525945663,manopose_loss:0.06090841442346573,manoshape_loss:0.030320031568408012,mano_total_loss:0.3312932252883911,hm_joints2d_loss:0.11643796414136887,obj_reg_loss:8.138731002807617,obj_conf_loss:3.419668436050415,total_loss:12.006118774414062
(epoch: 44, time: Sat Oct  7 13:26:57 2023),mano_mesh3d_loss:0.11400722712278366,mano_joints3d_loss:0.12164226174354553,manopose_loss:0.05996820703148842,manoshape_loss:0.029928108677268028,mano_total_loss:0.32554587721824646,hm_joints2d_loss:0.1151140034198761,obj_reg_loss:8.098417282104492,obj_conf_loss:3.4054768085479736,total_loss:11.944546699523926
(epoch: 45, time: Sat Oct  7 13:45:51 2023),mano_mesh3d_loss:0.11454638093709946,mano_joints3d_loss:0.12228084355592728,manopose_loss:0.05945529043674469,manoshape_loss:0.029216475784778595,mano_total_loss:0.3254990875720978,hm_joints2d_loss:0.11491260677576065,obj_reg_loss:7.9944000244140625,obj_conf_loss:3.383631706237793,total_loss:11.818445205688477
(epoch: 46, time: Sat Oct  7 14:04:47 2023),mano_mesh3d_loss:0.11160285025835037,mano_joints3d_loss:0.1191403791308403,manopose_loss:0.05832337215542793,manoshape_loss:0.028604691848158836,mano_total_loss:0.31767117977142334,hm_joints2d_loss:0.11264491826295853,obj_reg_loss:7.98276424407959,obj_conf_loss:3.376598358154297,total_loss:11.789679527282715
(epoch: 47, time: Sat Oct  7 14:23:42 2023),mano_mesh3d_loss:0.11110903322696686,mano_joints3d_loss:0.11865051835775375,manopose_loss:0.05760474130511284,manoshape_loss:0.028183262795209885,mano_total_loss:0.3155476152896881,hm_joints2d_loss:0.11159872263669968,obj_reg_loss:7.9566850662231445,obj_conf_loss:3.363412380218506,total_loss:11.747257232666016
(epoch: 48, time: Sat Oct  7 14:42:38 2023),mano_mesh3d_loss:0.1091856062412262,mano_joints3d_loss:0.11657179147005081,manopose_loss:0.056735437363386154,manoshape_loss:0.027795489877462387,mano_total_loss:0.3102882504463196,hm_joints2d_loss:0.1106155663728714,obj_reg_loss:7.884998321533203,obj_conf_loss:3.3498823642730713,total_loss:11.655786514282227
(epoch: 49, time: Sat Oct  7 15:01:35 2023),mano_mesh3d_loss:0.1097884252667427,mano_joints3d_loss:0.11723995953798294,manopose_loss:0.05634310096502304,manoshape_loss:0.027560019865632057,mano_total_loss:0.31093165278434753,hm_joints2d_loss:0.11030231416225433,obj_reg_loss:7.853546142578125,obj_conf_loss:3.3441338539123535,total_loss:11.618903160095215
(epoch: 50, time: Sat Oct  7 15:20:32 2023),mano_mesh3d_loss:0.10706812143325806,mano_joints3d_loss:0.11436346173286438,manopose_loss:0.05535992607474327,manoshape_loss:0.026760032400488853,mano_total_loss:0.303551584482193,hm_joints2d_loss:0.1082940325140953,obj_reg_loss:7.794037818908691,obj_conf_loss:3.329094171524048,total_loss:11.534963607788086
(epoch: 51, time: Sat Oct  7 15:41:52 2023),mano_mesh3d_loss:0.09684311598539352,mano_joints3d_loss:0.10332230478525162,manopose_loss:0.053307268768548965,manoshape_loss:0.02556159719824791,mano_total_loss:0.2790343165397644,hm_joints2d_loss:0.1032257229089737,obj_reg_loss:7.533874988555908,obj_conf_loss:3.2844197750091553,total_loss:11.200546264648438
(epoch: 52, time: Sat Oct  7 16:00:49 2023),mano_mesh3d_loss:0.09450680762529373,mano_joints3d_loss:0.10085099190473557,manopose_loss:0.052514947950839996,manoshape_loss:0.025269990786910057,mano_total_loss:0.27314284443855286,hm_joints2d_loss:0.10182569175958633,obj_reg_loss:7.460644245147705,obj_conf_loss:3.267666816711426,total_loss:11.103293418884277
(epoch: 53, time: Sat Oct  7 16:19:46 2023),mano_mesh3d_loss:0.09501060098409653,mano_joints3d_loss:0.1014629378914833,manopose_loss:0.05211701616644859,manoshape_loss:0.025020809844136238,mano_total_loss:0.27361124753952026,hm_joints2d_loss:0.10079546272754669,obj_reg_loss:7.461483001708984,obj_conf_loss:3.261809825897217,total_loss:11.097702980041504
(epoch: 54, time: Sat Oct  7 16:38:42 2023),mano_mesh3d_loss:0.09330987930297852,mano_joints3d_loss:0.09960994124412537,manopose_loss:0.05142093822360039,manoshape_loss:0.024672970175743103,mano_total_loss:0.2690138518810272,hm_joints2d_loss:0.09964569658041,obj_reg_loss:7.3937153816223145,obj_conf_loss:3.2502307891845703,total_loss:11.012606620788574
(epoch: 55, time: Sat Oct  7 16:57:39 2023),mano_mesh3d_loss:0.09286456555128098,mano_joints3d_loss:0.09916429966688156,manopose_loss:0.050903432071208954,manoshape_loss:0.024266956374049187,mano_total_loss:0.2671990692615509,hm_joints2d_loss:0.0990472212433815,obj_reg_loss:7.3620781898498535,obj_conf_loss:3.24031138420105,total_loss:10.968634605407715
(epoch: 56, time: Sat Oct  7 17:16:35 2023),mano_mesh3d_loss:0.09175398200750351,mano_joints3d_loss:0.09801234304904938,manopose_loss:0.05042383447289467,manoshape_loss:0.02417638897895813,mano_total_loss:0.26436638832092285,hm_joints2d_loss:0.09816519170999527,obj_reg_loss:7.323878765106201,obj_conf_loss:3.2299399375915527,total_loss:10.916350364685059
(epoch: 57, time: Sat Oct  7 17:35:32 2023),mano_mesh3d_loss:0.0910763218998909,mano_joints3d_loss:0.09726162254810333,manopose_loss:0.04974101856350899,manoshape_loss:0.023786157369613647,mano_total_loss:0.26186510920524597,hm_joints2d_loss:0.09725120663642883,obj_reg_loss:7.318131446838379,obj_conf_loss:3.2261650562286377,total_loss:10.903412818908691
(epoch: 58, time: Sat Oct  7 17:54:29 2023),mano_mesh3d_loss:0.09058976918458939,mano_joints3d_loss:0.09672299027442932,manopose_loss:0.04962611198425293,manoshape_loss:0.02374994568526745,mano_total_loss:0.2606890797615051,hm_joints2d_loss:0.09684713929891586,obj_reg_loss:7.309973239898682,obj_conf_loss:3.224032402038574,total_loss:10.8915433883667
(epoch: 59, time: Sat Oct  7 18:13:25 2023),mano_mesh3d_loss:0.08985183387994766,mano_joints3d_loss:0.09598299115896225,manopose_loss:0.04902930185198784,manoshape_loss:0.023459285497665405,mano_total_loss:0.25832319259643555,hm_joints2d_loss:0.09613147377967834,obj_reg_loss:7.263236045837402,obj_conf_loss:3.2125003337860107,total_loss:10.830181121826172
(epoch: 60, time: Sat Oct  7 18:32:21 2023),mano_mesh3d_loss:0.08880581706762314,mano_joints3d_loss:0.0948566421866417,manopose_loss:0.04857998341321945,manoshape_loss:0.023149723187088966,mano_total_loss:0.2553921937942505,hm_joints2d_loss:0.09555651247501373,obj_reg_loss:7.233233451843262,obj_conf_loss:3.2053377628326416,total_loss:10.789507865905762
(epoch: 61, time: Sat Oct  7 18:53:40 2023),mano_mesh3d_loss:0.08232354372739792,mano_joints3d_loss:0.08788643777370453,manopose_loss:0.04713120684027672,manoshape_loss:0.022293152287602425,mano_total_loss:0.23963432013988495,hm_joints2d_loss:0.09172297269105911,obj_reg_loss:7.020287036895752,obj_conf_loss:3.166713237762451,total_loss:10.518362045288086
(epoch: 62, time: Sat Oct  7 19:12:36 2023),mano_mesh3d_loss:0.08107398450374603,mano_joints3d_loss:0.0865740180015564,manopose_loss:0.04653767868876457,manoshape_loss:0.022064145654439926,mano_total_loss:0.2362498790025711,hm_joints2d_loss:0.09076698869466782,obj_reg_loss:7.007259368896484,obj_conf_loss:3.1664960384368896,total_loss:10.500771522521973
(epoch: 63, time: Sat Oct  7 19:31:33 2023),mano_mesh3d_loss:0.08061462640762329,mano_joints3d_loss:0.08610554784536362,manopose_loss:0.046216391026973724,manoshape_loss:0.02184167318046093,mano_total_loss:0.23477840423583984,hm_joints2d_loss:0.09029495716094971,obj_reg_loss:6.9472975730896,obj_conf_loss:3.147662878036499,total_loss:10.420034408569336
(epoch: 64, time: Sat Oct  7 19:50:27 2023),mano_mesh3d_loss:0.08001669496297836,mano_joints3d_loss:0.08543546497821808,manopose_loss:0.04600432887673378,manoshape_loss:0.021648744121193886,mano_total_loss:0.23310525715351105,hm_joints2d_loss:0.08920629322528839,obj_reg_loss:6.9754958152771,obj_conf_loss:3.1554527282714844,total_loss:10.453252792358398
(epoch: 65, time: Sat Oct  7 20:09:23 2023),mano_mesh3d_loss:0.07977167516946793,mano_joints3d_loss:0.08519323915243149,manopose_loss:0.045651257038116455,manoshape_loss:0.02133576199412346,mano_total_loss:0.23195204138755798,hm_joints2d_loss:0.08907648921012878,obj_reg_loss:6.944581508636475,obj_conf_loss:3.1451847553253174,total_loss:10.41077995300293
(epoch: 66, time: Sat Oct  7 20:28:18 2023),mano_mesh3d_loss:0.07970717549324036,mano_joints3d_loss:0.08516678214073181,manopose_loss:0.045242391526699066,manoshape_loss:0.021310513839125633,mano_total_loss:0.23142695426940918,hm_joints2d_loss:0.08866064250469208,obj_reg_loss:6.913706302642822,obj_conf_loss:3.1371090412139893,total_loss:10.370906829833984
(epoch: 67, time: Sat Oct  7 20:47:14 2023),mano_mesh3d_loss:0.07800474762916565,mano_joints3d_loss:0.08331823348999023,manopose_loss:0.04487057030200958,manoshape_loss:0.02107924036681652,mano_total_loss:0.22727279365062714,hm_joints2d_loss:0.08763626962900162,obj_reg_loss:6.888713359832764,obj_conf_loss:3.130798578262329,total_loss:10.334418296813965
(epoch: 68, time: Sat Oct  7 21:06:10 2023),mano_mesh3d_loss:0.07723458856344223,mano_joints3d_loss:0.08247040957212448,manopose_loss:0.044599227607250214,manoshape_loss:0.0208301842212677,mano_total_loss:0.22513432800769806,hm_joints2d_loss:0.08709744364023209,obj_reg_loss:6.8570237159729,obj_conf_loss:3.1245005130767822,total_loss:10.293752670288086
(epoch: 69, time: Sat Oct  7 21:25:05 2023),mano_mesh3d_loss:0.07718764990568161,mano_joints3d_loss:0.08243744820356369,manopose_loss:0.04429706931114197,manoshape_loss:0.02068469673395157,mano_total_loss:0.22460657358169556,hm_joints2d_loss:0.08711066842079163,obj_reg_loss:6.865562438964844,obj_conf_loss:3.12186861038208,total_loss:10.299142837524414
(epoch: 70, time: Sat Oct  7 21:44:01 2023),mano_mesh3d_loss:0.07669926434755325,mano_joints3d_loss:0.08193569630384445,manopose_loss:0.04408050328493118,manoshape_loss:0.020810119807720184,mano_total_loss:0.22352568805217743,hm_joints2d_loss:0.08646570146083832,obj_reg_loss:6.827320575714111,obj_conf_loss:3.114396572113037,total_loss:10.251701354980469

I am looking forward to your reply. And thanks in advance for your help!

about training setting and result

First thanks for your nice work. Personally, I have tried retraining it myself and got results, but this result differs especially on dexycb from the pretrained checkpoint you provided . So I wonder if there is a difference between the specific setup of your training and the code in this github repository. Or some tricks when you're training. Or maybe it's simply me doing something wrong? I need your opinion.
Also, if possible, please tell me your training speed, because I feel that my training speed is a bit slow, no noticeable improvement using multiple GPUs

Looking forward to your reply

3D translation of the hands

Thanks for sharing such excellent work.
However, after reading the code, I still have some questions regarding the 3D translation of the hands. I'm still not sure about how the 3D translation of the hands were produced.
I would greatly appreciate it if you could provide more details to help me better understand how the 3D translation of the hands was predicted.

Thank you for your sharing

Hello,
Thank you for your nice works!
I am very interested in your proposed network structure and will the code of this article be updated?

你好，我复现不出原论文的效果

在HO3D数据集上，按照你的论文中的参数设置，原模原样的代码任何配置都没有修改，测试集上的结果如下：
Evaluation 3D KP results:
auc=0.047, mean_kp3d_avg=10638.99 cm
Evaluation 3D KP PROCRUSTES ALIGNED results:
auc=0.775, mean_kp3d_avg=1.13 cm
Evaluation 3D KP SCALE-TRANSLATION ALIGNED results:
auc=0.407, mean_kp3d_avg=3.92 cm

Evaluation 3D MESH results:
auc=0.000, mean_kp3d_avg=10192.54 cm
Evaluation 3D MESH ALIGNED results:
auc=0.778, mean_kp3d_avg=1.11 cm

F-scores
[email protected] = 0.000 [email protected] = 0.453
[email protected] = 0.000 [email protected] = 0.929

各个评价指标都与原论文所宣称的差距较大，请问这是为什么？

Object-to-Hand Enhancement.

Hello, I would like to ask you why you are utilizing the splicing information of the hand object features when it comes to hand augmentation and not doing the same thing as hand to object augmentation.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.