melonwan / densereg Goto Github PK
View Code? Open in Web Editor NEW3D hand pose estimation via dense regression
3D hand pose estimation via dense regression
python model/hourglass_um_crop_tiny.py --dataset 'msra' --pid 0 --batch_size 3 --num_stack 2 --num_fea 128 --debug_level 2 --is_train False
finally into the long long training loop
Traceback (most recent call last):
File "model/hourglass_um_crop_tiny.py", line 932, in <module>
run_train(dataset, val_dataset)
File "model/hourglass_um_crop_tiny.py", line 889, in run_train
train(model, restore_step)
File "/home/denseReg/model/train_single_gpu.py", line 149, in train
assert not np.isnan(loss_value), 'Model diverged with loss = NaN'
AssertionError: Model diverged with loss = NaN
Hi,
When run the script, the file info.txt can not been found. Can you give some more information on that?
Thanks!
gpu_lock_path = '/tmp/lock-gpu*/info.txt'
I found you use nyu_bbx when testing on NYU dataset, which means you use 'ground truth' hand location data when running test. I think it should be predict by network itself instead of given by you.
So how did you get nyu_bbx.pkl in your experiment? Is it computed from ground truth xyz labels or predicted by another method which can compute bbx of hand location?
hi,
I am now setup the DenseReg environment in local server, but encounter issue when executing the training command below,
python model/hourglass_um_crop_tiny.py --dataset 'nyu' --batch_size 3 --num_stack 2 --num_fea 128 --debug_level 2 --is_train True
It seems the code doesn't find some file, but finally it enters the training log, can you help me to clarify what cause the issue? from my analysis, it seems
there is error happening in the below function .
135 #TODO: change to tf.train.SummaryWriter()
136 summary_writer = tf.summary.FileWriter(
137 model.summary_dir,
138 graph=sess.graph)
ERROR:tensorflow:Exception in QueueRunner: ./exp/data/nyu/tf_train/training-47-of-300; No such file or directory
[[Node: batch_processing/ReaderReadV2 = ReaderReadV2[_device="/job:localhost/replica:0/task:0/device:CPU:0"](batch_processing/TFRecordReaderV2, batch_processing/input_producer)]]
Exception in thread QueueRunnerThread-batch_processing/random_shuffle_queue-batch_processing/random_shuffle_queue_enqueue:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/usr/lib/python2.7/threading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/queue_runner_impl.py", line 252, in _run
enqueue_callable()
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1205, in _single_operation_run
self._call_tf_sessionrun(None, {}, [], target_list, None)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1350, in _call_tf_sessionrun
run_metadata)
NotFoundError: ./exp/data/nyu/tf_train/training-47-of-300; No such file or directory
[[Node: batch_processing/ReaderReadV2 = ReaderReadV2[_device="/job:localhost/replica:0/task:0/device:CPU:0"](batch_processing/TFRecordReaderV2, batch_processing/input_producer)]]
ERROR:tensorflow:Exception in QueueRunner: ./exp/data/nyu/tf_train/training-4-of-300; No such file or directory
[[Node: batch_processing/ReaderReadV2_1 = ReaderReadV2[_device="/job:localhost/replica:0/task:0/device:CPU:0"](batch_processing/TFRecordReaderV2_1, batch_processing/input_producer)]]
Exception in thread QueueRunnerThread-batch_processing/random_shuffle_queue-batch_processing/random_shuffle_queue_enqueue_1:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/usr/lib/python2.7/threading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/queue_runner_impl.py", line 252, in _run
enqueue_callable()
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1205, in _single_operation_run
self._call_tf_sessionrun(None, {}, [], target_list, None)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1350, in _call_tf_sessionrun
run_metadata)
NotFoundError: ./exp/data/nyu/tf_train/training-4-of-300; No such file or directory
[[Node: batch_processing/ReaderReadV2_1 = ReaderReadV2[_device="/job:localhost/replica:0/task:0/device:CPU:0"](batch_processing/TFRecordReaderV2_1, batch_processing/input_producer)]]
finally into the long long training loop
When I run python data/icvl.py:
it reports the error:
Traceback (most recent call last):
File "data/icvl.py", line 271, in
run_preprocess()
File "data/icvl.py", line 215, in run_preprocess
preprocess_op=dataset.preprocess_op(128, 128))
File "/fast/junyan/HandPose/denseReg/data/dataset_base.py", line 199, in get_batch_op
result = preprocess_op(dm, pose, self.cfg)
File "data/icvl.py", line 154, in preprocess_op
dm, pose, cfg = crop_from_xyz_pose(dm, pose, cfg, input_width, input_height)
File "/fast/junyan/HandPose/denseReg/data/preprocess.py", line 64, in crop_from_xyz_pose
if FLAGS.dataset == 'icvl':
File "/root/anaconda3/lib/python3.6/site-packages/tensorflow/python/platform/flags.py", line 85, in getattr
return wrapped.getattr(name)
File "/root/anaconda3/lib/python3.6/site-packages/absl/flags/_flagvalues.py", line 472, in getattr
raise AttributeError(name)
AttributeError: dataset
I create the soft link in the exp/data/icvl.
The icvl data directory hierarchy looks like this:
**>Training
Depth(With many images in different directories)
labels.txt
Testing
Depth(With many images in different directories)
labels.txt**
I am not sure how can I resolve this problem.
Hi, thank you very much for sharing your code. I am little confused that about the network depicted in the paper and code.
um_in = tf.concat([hg_outs, hm_out, hm3_out], axis=-1)
um_in = _residual(_residual(um_in, 256))
um_in_mask = tf.concat([hg_outs, hm_out, hm3_out], axis=-1)
mask = tf.tile(tf.less(tiny_dm, -0.9), (1,1,1,um_in_mask.get_shape()[-1].value))
um_in_mask = tf.where(mask, tf.zeros_like(um_in_mask), um_in_mask)
um_in_mask = _residual(_residual(um_in_mask, 256))
There is two consecutive residual model flowing um_in
and um_in_mask
, respectively. However, I cannot find in the figure 1 of the paper.
Thank you!
I find that you use all three views of depth images for training in the \denseReg-master\data\nyu.py .
def loadAnnotation(self, is_trun=False):
'''is_trun:
True: to load 14 joints from self.keep_list
False: to load all joints
'''
t1 = time.time()
path = os.path.join(self.src_dir, 'joint_data.mat')
mat = sio.loadmat(path)
camera_num = 1 if self.subset=='testing' else 3
joints = [mat['joint_xyz'][idx] for idx in range(camera_num)]
names = [['depth_{}_{:07d}.png'.format(camera_idx+1, idx+1) for idx in range(len(joints[camera_idx]))] for camera_idx in range(camera_num)]
But for fair comparison only view 1 images should be used for training.
I am wondering why data.preprocess.pose_sync not found in the data/preprocess.py file ?
After download nyu dataset and your pretrained model, I run the command python model/hourglass_um_crop_tiny.py --dataset 'nyu' --batch_size 3 --num_stack 2 --num_fea 128 --debug_level 2
with is_train equals False in the code. But the results is not good as it should be:
[2018-04-19 16:06:11.933652]: 2727/393226 computed, with 0.10s
finish test
10mm percentage: 0.012239
20mm percentage: 0.057683
30mm percentage: 0.072225
40mm percentage: 0.095613
This is what i see in tensorboard:
And this is the generating result files:
testing-2018-04-19_15_59_22.194820-result.txt
testing-2018-04-19_15_59_22.194820-result_error.txt
What's the problem? Am I wrong with the code?
My environment:
Hi,
The download link in the script seems not working ant more, when I run the script, I got the following errors:
downloading the pretrained model(566M)... --2021-03-01 15:21:48-- https://polybox.ethz.ch/index.php/s/B2W1ngyUAitsv2e/download Resolving polybox.ethz.ch (polybox.ethz.ch)... 129.132.71.243 Connecting to polybox.ethz.ch (polybox.ethz.ch)|129.132.71.243|:443... connected. HTTP request sent, awaiting response... 503 Service Unavailable 2021-03-01 15:21:49 ERROR 503: Service Unavailable.
Is it possible to update the link?
Thanks
Hi, thanks to your great work!
If it's to your convenience, could you share the experimental data of ploting success rate on MSRA dataset? Namely, the fig. 6 on your paper. Because we are doing the research on depth hand pose, need to make a comparison. I would much appreciate it if you can share it with me!
if n > 1:
lower2 = _hourglass(lower1, n-1)
else:
lower2 = lower1
I think lower2 should also be add to residual module:
if n > 1:
lower2 = _hourglass(lower1, n-1)
else:
lower2 = lower1
lower2 = _residual(lower2)
half_num_in
instead of half_num_out
Can be this method used to estimate 3D human pose? please. @melonwan
Hi,
The first method is coordinate regression method, whether the label is pixel coordinate or world coordinate for the loss calculation.
What is the next two line code meaning? Could you give more information in details? Thank you!
norm_xyz_pose = tf.divide(xyz_pose - tf.tile(com,[jnt_num]), POSE_NORM_RATIO) gt_oms = tf.reshape(gt_normed_poses, (-1,1,1,3*self._jnt_num)) - xyzs
hms, dcs = data.preprocess.pose_sync(norm_poses, normed_dms, cfgs, coms, 128, 128)
msra.py
if __name__ == '__main__':
saveTFRecord()
# run_check_record()
# run_preprocess()
it means that i must uncomment the last two command of msra.py ?
But when i uncomment the last two command of msra.py, it always lead to error.
Can you help give me a little hint about how i can solve the problem ?
Hi, I have tried the inference code in NYU dataset. However, I can't achieve "real-time" performance as mentioned in your paper.
For batch size=1: frame rate is 12
For batch size=3: frame rate is 17
Both are not good for real time(less than 24).
I wonder why the inference speed isn't good. Did you do infernece in TWO Titans?
Thank you very much.
Can I test the pretrained model with webcam?
or should I prepare depth cam too?
When using
python model/hourglass_um_crop_tiny.py --dataset 'icvl' --batch_size 3 --num_stack 2 --num_fea 128 --debug_level 2 --is_train False
it seems to be an import error
from data.preprocess import crop_from_xyz_pose, crop_from_uvd_pose, crop_from_bbx, center_of_mass
ImportError: cannot import name crop_from_uvd_pose
And there is no crop_from_uvd_pose in data.preprocess indeed. It is in file 'data/icvl.py'.
How to solve it
Hello,
While i try to run the model in test mode using the pre-trained model i always get an error at some point during testing (with num_shards=16 for test, this error happens after treating the first 400 points). The i receive this error:
flat indices[34, :] = [32, 26, 6] does not index into param (shape: [32,32,14]).
[[Node: map_3/while/GatherNd = GatherNd[Tindices=DT_INT32, Tparams=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"](map_3/while/TensorArrayReadV3_3, map_3/while/stack_1)]]
I tried to debug this with not much success, would you happen to know what is the problem?
hello!Thank you for your generous sharing!There is some wrong when I train the NYU dataset.How to solve it?
AttributeError: 'FigureCanvasBase' object has no attribute 'tostring_argb'
[[node pt_summary_1/Plot_1 (defined at D:\ProgramData\Anaconda3\envs\tfenv\lib\site-packages\tfplot\ops.py:114) ]]
Caused by op 'pt_summary_1/Plot_1', defined at:
File "hourglass_um_crop_tiny.py", line 918, in
run_train(dataset, val_dataset)
File "hourglass_um_crop_tiny.py", line 882, in run_train
train(model, restore_step)
File "..\model\train_single_gpu.py", line 59, in train
loss = model.loss(*batches)
File "hourglass_um_crop_tiny.py", line 419, in loss
tf.reshape(hm_uvd_pts, (3,-1,3))))
File "D:\ProgramData\Anaconda3\envs\tfenv\lib\site-packages\tfplot\wrapper.py", line 87, in wrapped_fn
**merge_kwargs(kwargs, kwargs_call))
File "D:\ProgramData\Anaconda3\envs\tfenv\lib\site-packages\tfplot\ops.py", line 167, in plot_many
im = plot(plot_func, arg, name=('Plot%d' % k), **kwargs)
File "D:\ProgramData\Anaconda3\envs\tfenv\lib\site-packages\tfplot\ops.py", line 114, in plot
name=name)
File "D:\ProgramData\Anaconda3\envs\tfenv\lib\site-packages\tensorflow\python\util\deprecation.py", line 324, in new_func
return func(*args, **kwargs)
File "D:\ProgramData\Anaconda3\envs\tfenv\lib\site-packages\tensorflow\python\ops\script_ops.py", line 468, in py_func
func=func, inp=inp, Tout=Tout, stateful=stateful, eager=False, name=name)
File "D:\ProgramData\Anaconda3\envs\tfenv\lib\site-packages\tensorflow\python\ops\script_ops.py", line 282, in _internal_py_func
input=inp, token=token, Tout=Tout, name=name)
File "D:\ProgramData\Anaconda3\envs\tfenv\lib\site-packages\tensorflow\python\ops\gen_script_ops.py", line 154, in py_func
"PyFunc", input=input, token=token, Tout=Tout, name=name)
File "D:\ProgramData\Anaconda3\envs\tfenv\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 788, in _apply_op_helper
op_def=op_def)
File "D:\ProgramData\Anaconda3\envs\tfenv\lib\site-packages\tensorflow\python\util\deprecation.py", line 507, in new_func
return func(*args, **kwargs)
File "D:\ProgramData\Anaconda3\envs\tfenv\lib\site-packages\tensorflow\python\framework\ops.py", line 3300, in create_op
op_def=op_def)
File "D:\ProgramData\Anaconda3\envs\tfenv\lib\site-packages\tensorflow\python\framework\ops.py", line 1801, in init
self._traceback = tf_stack.extract_stack()
UnknownError (see above for traceback): AttributeError: 'FigureCanvasBase' object has no attribute 'tostring_argb'
Traceback (most recent call last):
File "D:\ProgramData\Anaconda3\envs\tfenv\lib\site-packages\tensorflow\python\ops\script_ops.py", line 207, in call
ret = func(*args)
File "D:\ProgramData\Anaconda3\envs\tfenv\lib\site-packages\tfplot\ops.py", line 101, in _render_image
image = figure.to_array(fig)
File "D:\ProgramData\Anaconda3\envs\tfenv\lib\site-packages\tfplot\figure.py", line 90, in to_array
img = np.frombuffer(fig.canvas.tostring_argb(), dtype=np.uint8)
AttributeError: 'FigureCanvasBase' object has no attribute 'tostring_argb'
[[node pt_summary_1/Plot_1 (defined at D:\ProgramData\Anaconda3\envs\tfenv\lib\site-packages\tfplot\ops.py:114) ]]
Hi:
thanks for sharing your work!
I try to run the code following your instructions. and when i run the python script ./data/nyu.py.it takes 30 hours and still runing. Could you tell me is this normal or I did something wrong?
thank you.
Hi, thank you very for your help. Have you ever run the project on multiple GPUs to speed up the trainding process? Thank you!
Can I estimate 3D hand pose (uvd) in ASL Finger Spelling Dataset using the pretrained models?
Note that the resolution of the given depth images is different and there is no camera configuration.
Thanks in advance~
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.