stefanopini / simple-hrnet Goto Github PK

@stefanopini hi, thanks so much for your nice work. I test your scripts and I faced following issues.

I read and applied arguments as in (live-demo.py --help) but below are some problems.

Single person with MPII joints not working but single person with coco joints are working but results are not too good as for multi-person, I also changed the pre-trained model from coco to mpii but there is error and also I changed arguments for single person.

How to solve it?
Thanks for your support.

Hi, and thank you for making this code available.

I am running it on a single GTX 1080, and the inference time is around 200ms per frame.

Using pose_hrnet_w48_384x288.pth and yolov3.weights.

When I try to use pose_hrnet_w32_256x192.pth

I get errors:

size mismatch for final_layer.weight: copying a param with shape torch.Size([17, 32, 1, 1]) from checkpoint, the shape in current model is torch.Size([17, 48, 1, 1]).

What else do I need to change? And how much of a speed increase can I expect? Can I also switch to yolo-tiny weights to improve speed? Where would I make this change?

I am trying to get to < 50ms per frame. Is this a reasonable goal?

Thanks!

Hello
I am having troubles in executing , the live-demo script. I get the following , issue . I am using the following pre trained weight pose_hrnet_w48_384x288.pth
I have not come across this issue before

python3.6 scripts/live-demo.py --filename /home/test/simple-HRNet/test.mp4 
cuda:0
Traceback (most recent call last):
  File "scripts/live-demo.py", line 99, in <module>
    main(**args.__dict__)
  File "scripts/live-demo.py", line 57, in main
    pts = model.predict(frame)
  File "/home/test/simple-HRNet/SimpleHRNet.py", line 124, in predict
    return self._predict_single(image)
  File "/home/test/simple-HRNet/SimpleHRNet.py", line 184, in _predict_single
    shape=(self.resolution[0] // 4, self.resolution[1] // 4))
TypeError: Required argument 'dims' (pos 2) not found

Also tried to use the run hrnet on image as shown , with a file test.py in the folder scripts

import os
import sys
import cv2
import torch
sys.path.insert(1, os.getcwd())
import cv2
from SimpleHRNet import *

model = SimpleHRNet(48, 17, "./weights/pose_hrnet_w48_384x288.pth")
image = cv2.imread("/home/test/simple-HRNet/nadal.jpg", cv2.IMREAD_COLOR)
joints = hrnet.predict(image)

It seems to have , problems in importing , perhaps the python script should be in some other directory ?

Traceback (most recent call last): File "scripts/test.py", line 12, in <module> joints = hrnet.predict(image) NameError: name 'hrnet' is not defined

I was playing around with your code and noticed that the loss and backprop is doing something strange. I didn't change anything in your code - ran validation before training the (trained) model for 1 epoch. That 1 epoch of training seemed to drag the results to almost 0. Any idea why that is happening?

`
Loading train and validation datasets...
loading annotations into memory...
Done (t=8.38s)
creating index...
index created!
100%|███████████████████████████████████████████████████████████████████████████| 118287/118287 [00:42<00:00, 2776.12it/s]

COCO dataset loaded!
loading annotations into memory...
Done (t=0.37s)
creating index...
index created!
100%|███████████████████████████████████████████████████████████████████████████████| 5000/5000 [00:02<00:00, 2203.10it/s]

COCO dataset loaded!
cuda:0

Training started @ 2019-10-13 14:48:47

Epoch 1 of 200 @ 2019-10-13 14:48:47
Validating: 100%|███████████████████████████████████████████████████████████████████████| 397/397 [12:05<00:00, 1.26s/it]

Val AP/AR
WARNING:root:json-tricks: numpy scalar serialization is experimental and may work differently in future versions
Loading and preparing results...
DONE (t=0.70s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type keypoints
DONE (t=7.33s).
Accumulating evaluation results...
DONE (t=0.13s).
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets= 20 ] = 0.772
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets= 20 ] = 0.936
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets= 20 ] = 0.847
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets= 20 ] = 0.739
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets= 20 ] = 0.828
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 20 ] = 0.800
Average Recall (AR) @[ IoU=0.50 | area= all | maxDets= 20 ] = 0.945
Average Recall (AR) @[ IoU=0.75 | area= all | maxDets= 20 ] = 0.864
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets= 20 ] = 0.762
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets= 20 ] = 0.859
Training: 100%|█████████████████████████████████████████████████████████████████████| 9363/9363 [8:32:08<00:00, 2.00s/it]

Train AP/AR
Loading and preparing results...
DONE (t=17.94s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type keypoints
DONE (t=136.31s).
Accumulating evaluation results...
DONE (t=3.30s).
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets= 20 ] = 0.011
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets= 20 ] = 0.034
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets= 20 ] = 0.004
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets= 20 ] = 0.011
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets= 20 ] = 0.011
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 20 ] = 0.037
Average Recall (AR) @[ IoU=0.50 | area= all | maxDets= 20 ] = 0.105
Average Recall (AR) @[ IoU=0.75 | area= all | maxDets= 20 ] = 0.020
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets= 20 ] = 0.034
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets= 20 ] = 0.040
Validating: 100%|███████████████████████████████████████████████████████████████████████| 397/397 [08:43<00:00, 1.27s/it]

Val AP/AR
Loading and preparing results...
DONE (t=1.84s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type keypoints
DONE (t=5.02s).
Accumulating evaluation results...
DONE (t=0.16s).
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets= 20 ] = 0.020
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets= 20 ] = 0.089
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets= 20 ] = 0.002
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets= 20 ] = 0.018
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets= 20 ] = 0.026
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 20 ] = 0.040
Average Recall (AR) @[ IoU=0.50 | area= all | maxDets= 20 ] = 0.147
Average Recall (AR) @[ IoU=0.75 | area= all | maxDets= 20 ] = 0.010
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets= 20 ] = 0.030
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets= 20 ] = 0.054
best_loss 0.001153 at epoch 1
best_acc 0.232520 at epoch 1
best_mAP 0.020019 at epoch 1
`

@stefanopini
Thanks for making available this user-friendly version of deep HRNet!

I am trying to retrain a pre-trained model (the idea is to augment to coco dataset with my own data) and run into this problem:

`(myenv) gilles@gilles-N55SF:~/mywork/gilles_hrnet$ python3 scripts/train_coco.py -p ./weights/pose_hrnet_w48_384x288.pth -e 200

cuda:0
Starting experiment 20191107_1858 @ 2019-11-07 18:58:36
Loading train and validation datasets...
loading annotations into memory...
Done (t=4.22s)
creating index...
index created!
100%|█████████████████████████████████| 118287/118287 [00:43<00:00, 2717.40it/s]

COCO dataset loaded!
loading annotations into memory...
Done (t=0.13s)
creating index...
index created!
100%|█████████████████████████████████████| 5000/5000 [00:01<00:00, 2704.04it/s]

COCO dataset loaded!
cuda:0
Traceback (most recent call last):
File "scripts/train_coco.py", line 153, in
main(**args.dict)
File "scripts/train_coco.py", line 108, in main
device=device
File "/home/gilles/mywork/gilles_hrnet/training/COCO.py", line 119, in init
device=device
File "/home/gilles/mywork/gilles_hrnet/training/Train.py", line 201, in init
last_epoch=self.starting_epoch)
File "/home/gilles/myenv/lib/python3.6/site-packages/torch/optim/lr_scheduler.py", line 245, in init
super(MultiStepLR, self).init(optimizer, last_epoch)
File "/home/gilles/myenv/lib/python3.6/site-packages/torch/optim/lr_scheduler.py", line 26, in init
"in param_groups[{}] when resuming an optimizer".format(i))
KeyError: "param 'initial_lr' is not specified in param_groups[0] when resuming an optimizer"
`
I understand that the training script is expecting an initial_lr, but I'd appreciate if you could help me find the best way to pass it.

Hi ,
I was wondering how can we access the x and y co-ordinates of certain keypoints ?? For example right ankle_x and right ankle _y ??
My aim is to calculate the angles between the joints and if possible show it overlaid on the video sequence ?

how conda install requirement?????????????????

Hi since , Hrnet has Yolov3 , enabled by default , is it possible to implement a tracking functionality as well ?

I'm wondering if I can run this code on machines without a gpu and just with the cpu version of pytorch.

The prediction of YOLOv3 usually a rectangle and I see that you use transforms method to resize the image before put it into HRNet. It stretching the image and make pose estimation network performs badly.
I try to make YOLOv3 prediction regions have the same h/w ratio as the input of HRNet. For example, HRNet requires image_resolution (256, 256)，but YOLOv3 gives a rectangle region. What we should do is calculate the center of YOLO prediction region and expand it as a square.

In SimpleHRNet._predict_single I modify it by adding:

squareLen = max(x2 - x1, y2 - y1) // 2
centerX = x1 + (x2 - x1) // 2
centerY = y1 + (y2 - y1) // 2
x1 = max(0, centerX - squareLen)
x2 = min(len(image[1]), centerX + squareLen)
y1 = max(0, centerY - squareLen)
y2 = min(len(image[0]), centerY + squareLen)

And it works on pose_hrnet_w32_256x256 with mpii annotation.
Maybe you can add it on your proj :D

Hi ,
Is it possible to display the overlaid image on still images using the class usage example explained in the readme file . If so how can we do it ??

Hi, bother again!

I'm a little confused about training data code here. I find that you return the ori_img with corresponding keypoints annotations together (maybe there are multi persons' annotation). So, as hrnet is run as a single pose network, how can you training network using the ori image instead of the crop image based the human box annotation? Is there somewhere you have done this before?

For more, clearly, i think we can train hrnet ( a single pose network) using MSCOCO dataset is that we need to crop out multi/single input image from original image because maybe one image could have multi human annotations, just like in live-demo.py you have done.

So, could you tell me what mine thought is right or not, and to be honest, i'm also confused about how to train a single person pose network with using MSCOCO dataset?

Thanks a lot.

I want to train pose-hrnet from imagenet pretrained model, but l only have one Titan XP GUP , so I want to know how much time it will consume ? Thanks！

Hi , i see the overlaid images are saved in the following
https://github.com/stefanopini/simple-HRNet/blob/master/scripts/live-demo.py#L77

I was wondering if it is possile to save the overlaid images as a video stream as output ?

Hi!

Is this also compatible with the Simple Baselines models provided in the original HRNet repository? (for example, pose_resnet_50_256x192.pth)

Thank you

Hi, higher hrnet has been released code. Do you have a plan to do with it and make a Simple Higher-HRNet just like this repo. Thanks~

(simpleHR) scout@scout-Aspire-GX-781:/media/scout/26869a43-e371-4f24-97af-3bbd01078e2d/scout/Documents/simple-HRNet-master$ python scripts/train_coco.py
cuda:0

Starting experiment 20190828_0938 @ 2019-08-28 09:38:20

Loading train and validation datasets...
loading annotations into memory...
Done (t=7.03s)
creating index...
index created!
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 118287/118287 [00:19<00:00, 5930.78it/s]

COCO dataset loaded!
loading annotations into memory...
Done (t=0.26s)
creating index...
index created!
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 5000/5000 [00:00<00:00, 6104.34it/s]

COCO dataset loaded!
cuda:0

Training started @ 2019-08-28 09:38:56

Epoch 0 of 200 @ 2019-08-28 09:38:56
Training: 0%| | 0/149813 [00:00<?, ?it/s]Traceback (most recent call last):
File "scripts/train_coco.py", line 153, in
main(**args.dict)
File "scripts/train_coco.py", line 111, in main
train.run()
File "/media/scout/26869a43-e371-4f24-97af-3bbd01078e2d/scout/Documents/simple-HRNet-master/training/Train.py", line 348, in run
self._train()
File "/media/scout/26869a43-e371-4f24-97af-3bbd01078e2d/scout/Documents/simple-HRNet-master/training/COCO.py", line 131, in _train
for step, (image, target, target_weight, joints_data) in enumerate(tqdm(self.dl_train, desc='Training')):
File "/home/scout/anaconda3/lib/python3.7/site-packages/tqdm/_tqdm.py", line 1005, in iter
for obj in iterable:
File "/home/scout/anaconda3/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 278, in iter
return _MultiProcessingDataLoaderIter(self)
File "/home/scout/anaconda3/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 682, in init
w.start()
File "/home/scout/anaconda3/envs/simpleHR/lib/python3.7/multiprocessing/process.py", line 112, in start
self._popen = self._Popen(self)
File "/home/scout/anaconda3/envs/simpleHR/lib/python3.7/multiprocessing/context.py", line 223, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "/home/scout/anaconda3/envs/simpleHR/lib/python3.7/multiprocessing/context.py", line 277, in _Popen
return Popen(process_obj)
File "/home/scout/anaconda3/envs/simpleHR/lib/python3.7/multiprocessing/popen_fork.py", line 20, in init
self._launch(process_obj)
File "/home/scout/anaconda3/envs/simpleHR/lib/python3.7/multiprocessing/popen_fork.py", line 70, in _launch
self.pid = os.fork()
OSError: [Errno 12] Cannot allocate memory

hello, thanks for your work. when i run train_coco.py, i get the model 'checkpoint.pkl'. how to use this model ?Is it the same as using the pre-training model in folder 'weights'?

@stefanopini Hi, how we can add more than one camera (2 or 3) in this project? No matter how many output screens (2 or 3) or no any need of calibration. Just simple input 2 or 3 cameras and then estimate poses on cameras separately. Simple OpenCV code is working fine for 2 cameras but I want to apply using threading as given below.
I want to add this code in your scripts/live-demo.py camera reading code

import cv2
import threading
class camThread(threading.Thread):
    def __init__(self, previewName, camID):
        threading.Thread.__init__(self)
        self.previewName = previewName
        self.camID = camID
    def run(self):
        print ("Starting " + self.previewName)
        camPreview(self.previewName, self.camID)
def camPreview(previewName, camID):
    cv2.namedWindow(previewName)
    cam = cv2.VideoCapture(camID)
    if cam.isOpened():
        rval, frame = cam.read()
    else:
        rval = False
    while rval:
        cv2.imshow(previewName, frame)
        rval, frame = cam.read()
        key = cv2.waitKey(20)
        if key == 27:  
            break
    cv2.destroyWindow(previewName)
thread1 = camThread("Camera 1", 0)
thread2 = camThread("Camera 2", 1)
thread1.start()
thread2.start()

Or you can add your own simple code for dual camera.
Looking for your kind suggestions.
Thanks

Thanks for your excellent work. If I want to train on my own dataset, how can I do? Thanks.

Hello
I was playing around with different videos and clips to check performance of how pretrained models work for both Hrnet and Openpose. I seem to notice the open pose seem to have a better accuracy ? Have you tried it , what is your opinion about it ?

Hi, i'm tring to using hrnet with different input size, but there is wrong when I set input size is 192x144, seems like this model can only handle with input 256x192 and 384x288. Is there anyway should to make this work done? Thanks~

When i ran datasets/COCO.py,this error appears.Any suggestions?

Hi i am trying to do normalization of keypoints for post processing. On looking through the code it seems like one needs to first extract x,y cordinates . For a single person it seems to work ok .
Pseudo code

vec = np.empty(pts.shape[1] * 2)
vec[0::2] = pts[:,:,0]
vec[1::2] = pts[:,:,1]
print(vec)

norm = np.linalg.norm(vec)

Could this be the right way to do it ? This method only works for single person detection . Do you perhaps have a better suggesiton ?

Hi,
I find somewhere maybe cause error in repo, here, sometimes this will cause error like that:

AttributeError: 'cv2.UMat' object has no attribute 'shape'

Maybe do not return after cv2.line is better. This error did not shown as before but occured today when i use it, so please check it.

Hi while working on one of the videos, i realized that the input videos was upside down and thus it failed to detect any 2D keypoints. Is there a way in opencv or otherwise to add an check for it ? Is it even possible

Hi,
I tried to replace yolov3.weights and yolov3.cfg with new yolov4.weights and yolov4.cfg but it took forever to predict with yolo detection. Actually I am on colab and it terminated the runtime because of memory, so no prediction occurred.

Has anyone else tried it?

Hi,

Thank you for your excellent work which is a huge help for beginners. I have a question about how to find the official pre-trained weights from the link you gave. Looking forward to your reply. : )

Hi,

Thank you for the easier to use implementation.
I am currently trying to train a pre-trained weight with leeds extended sports data. I have converted keypoint annotations to COCO format, but head points do not match, so I assume they are not available. Also leeds has points either visible or not, so no visibile==1 (COCO).
Long story short, training actually declined the accuracy. So, do you think lack of data compatibility is an issue?

Hi Stefano,
After being able to run , simpleHRnet , i was quite happy and played around a bit, trying with different weights and configurations. I have a few collective questions on how to make , the pose estimation more robust .

a) Apparently the pretrained weights 'pose_hrnet_w48_256x192.pth' seems to be a good trade off between speed and estimation
b) Is there a way to increase the speed of inference ? (Frame rate )
c) Will we get a better performance if we resize the image sizes to '384x288'
d) The default max_batch_size is set to 32 , can we change it to other vaiue and will it increase the accuracy ?
e) The parameters in 'non_max_supression' conf_thres and nms_thres . can i guess effect the estimation . Lower the conf_thres i guess the higher the joints wil be predicted?
f) For multiperson , if i want to change the pretrained Yolo weights , I guess i have to define them in the Yolov3.py file ?

These are some questions that pop up initially. I will try to play around with it .

A quick heads up , there seems to be problem with the new Pillow version. Alternate solution is here

pytorch/vision#1712

Thank you very much

Hi! First, I love the work you have been doing. I'm working on implementing a pipeline for these heatmap-based pose problems in the fast.ai framework at the moment and I had a quick question! So I understand we are encoding our y's into n heatmaps that have a size of 1/2 our image size. How do we then grab the coordinates from these generated heatmaps? I'm a bit lost on the math from this :/ I think we need the bounding boxes for this? I'm having a little difficulty putting it all together. Any help would be greatly appreciated!

Thank you!

Zach

Just a followup from the last issue("Loss Implementation") which was closed. I just checked the saved weights of the HR Net models but it turns out that they have the batchnorm parameters as well (running_mean, etc). Also, the adam parameters are available in the yaml files on the actual repo (the lr's 1e-5, etc). But, continuing retraining from the saved model still seems to drop the accuracy to 0. Any idea why that may be happening?

Hi Stefano , thanks for the ease of use code , is it possible to run the code in windows ? And i guess it requires CUDA toolkit to run ? I tried a to port a similar hrnet implementation to run in windows but i could not manage it as such . Sadly wsl does not really support CUDA access in windows

I tried continuing the training for a couple of epochs starting from official weights pose_hrnet_w32_256x192.pth. But the average precision and recall is so low：

Train AP/AR
WARNING:root:json-tricks: numpy scalar serialization is experimental and may work differently in future versions
Loading and preparing results...
DONE (t=8.20s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type keypoints
DONE (t=78.57s).
Accumulating evaluation results...
DONE (t=2.16s).
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets= 20 ] = 0.018
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets= 20 ] = 0.041
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets= 20 ] = 0.012
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets= 20 ] = 0.015
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets= 20 ] = 0.017
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 20 ] = 0.037
Average Recall (AR) @[ IoU=0.50 | area= all | maxDets= 20 ] = 0.106
Average Recall (AR) @[ IoU=0.75 | area= all | maxDets= 20 ] = 0.020
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets= 20 ] = 0.035
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets= 20 ] = 0.039
Validating: 100%|███████████████████████████████████████████████████████████████████| 397/397 [03:31<00:00, 1.90it/s]

Val AP/AR
Loading and preparing results...
DONE (t=0.33s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type keypoints
DONE (t=3.29s).
Accumulating evaluation results...
DONE (t=0.08s).
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets= 20 ] = 0.067
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets= 20 ] = 0.259
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets= 20 ] = 0.010
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets= 20 ] = 0.075
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets= 20 ] = 0.064
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 20 ] = 0.117
Average Recall (AR) @[ IoU=0.50 | area= all | maxDets= 20 ] = 0.360
Average Recall (AR) @[ IoU=0.75 | area= all | maxDets= 20 ] = 0.046
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets= 20 ] = 0.118
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets= 20 ] = 0.116
best_loss 0.001845 at epoch 10
best_acc 0.363634 at epoch 10
best_mAP 0.067391 at epoch 10

I continue the training with coco dataset, and load the weights pose_hrnet_w32_256x192.pth.
I'm confused , is this result normal ? ?

how to solve this quetion?

from vidgear.gears import CamGear
ModuleNotFoundError: No module named 'vidgear'

Is there an easy way to batch images to be predicted on? As of now it looks like the predict function only applies to a single image.

Is it possible to export keypoints of the joints into a csv /excel file to do post processing ?

Hi , I was wondering , if one can make a GUI , perhaps for example , to show the angles of rotation between the joints. For now one can show it using open CV cv2.put2text format. I was wondering what packages to use ?

Second , is it possible to have different color codes of the lines joining the keypoints ? If so how and where can one make such changes ?

Thanks in advance

Hi,

I have used hrnet model on real-live video and find that the keypoint of same person jitter seriously. The alphapose paper have said that they add STN network before pose-estimation network and SDTN network after pose-estimation network, the reason that they do this is they found that slightly difference of one same person box can make great influence on pose-estimation. So, how do you think of this? Could make the pose result more stable by adding STN and SDTN as alphapose does?

Thanks.

Hi,

I just wanted to check with you if you have or plan to retrain a new HRNet model for the pose keypoint detection using the YOLOv3 detection boxes as the ground truth instead of the COCO annotated ground truth boxes used to train the original model. I feel like the model might learn to handle keypoint detection in noisy/wrong YOLO boxes to a greater extent as compared to the current model. Do you believe there is any scope for this and it may help the accuracy of the pose points that are detected ?

How about the inference speed on cpu, please?

Hi @stefanopini i seem to unable to import the simple Hrnet module . I have the following error
I was wondering , if i have missed some steps or additional confguration . I am running on a virtual machine with ubuntu 16.04

Traceback (most recent call last):
  File "scripts/live-demo.py", line 6, in <module>
    from SimpleHRNet import SimpleHRNet
ImportError: No module named SimpleHRNet

My directory structure is similar to yours

── datasets
│   ├── COCO.py
│   └── LiveCamera.py
├── LICENSE
├── misc
│   └── utils.py
├── models
│   ├── detectors
│   │   ├── yolo
│   │   │   ├── assets
│   │   │   │   ├── dog.png
│   │   │   │   ├── giraffe.png
│   │   │   │   ├── messi.png
│   │   │   │   └── traffic.png
│   │   │   ├── config
│   │   │   │   ├── coco.data
│   │   │   │   ├── create_custom_model.sh
│   │   │   │   ├── custom.data
│   │   │   │   ├── yolov3.cfg
│   │   │   │   └── yolov3-tiny.cfg
│   │   │   ├── data
│   │   │   │   ├── coco.names
│   │   │   │   ├── custom
│   │   │   │   │   ├── classes.names
│   │   │   │   │   ├── images
│   │   │   │   │   │   └── train.jpg
│   │   │   │   │   ├── labels
│   │   │   │   │   │   └── train.txt
│   │   │   │   │   ├── train.txt
│   │   │   │   │   └── valid.txt
│   │   │   │   ├── get_coco_dataset.sh
│   │   │   │   └── samples
│   │   │   │       ├── dog.jpg
│   │   │   │       ├── eagle.jpg
│   │   │   │       ├── field.jpg
│   │   │   │       ├── giraffe.jpg
│   │   │   │       ├── herd_of_horses.jpg
│   │   │   │       ├── messi.jpg
│   │   │   │       ├── person.jpg
│   │   │   │       ├── room.jpg
│   │   │   │       └── street.jpg
│   │   │   ├── detect.py
│   │   │   ├── LICENSE
│   │   │   ├── models.py
│   │   │   ├── README.md
│   │   │   ├── requirements.txt
│   │   │   ├── test.py
│   │   │   ├── train.py
│   │   │   ├── utils
│   │   │   │   ├── augmentations.py
│   │   │   │   ├── datasets.py
│   │   │   │   ├── __init__.py
│   │   │   │   ├── logger.py
│   │   │   │   ├── parse_config.py
│   │   │   │   └── utils.py
│   │   │   └── weights
│   │   │       ├── darknet53.conv.74
│   │   │       ├── download_weights.sh
│   │   │       ├── yolov3-tiny.weights
│   │   │       └── yolov3.weights
│   │   └── YOLOv3.py
│   ├── hrnet.py
│   └── modules.py
├── README.md
├── requirements.txt
├── scripts
│   └── live-demo.py
├── SimpleHRNet.py
├── SimpleHRNet.pyc
├── weights
│   └── pose_hrnet_w48_384x288.pth
└── workout_mod.mp4

Hi,

I met a problem when I run 'python scripts/live-demo.py --camera_id 0', it said that 'ModuleNotFoundError: No module named 'models.detectors.yolo.models'. Does anyone have idea how to solve this problem?

Btw, I have installed all those things in the requirments.txt and my system is Win10

Many thanks in advance!

hi, thanks for your works. The problem I met is that when I run the following in command line, it could run successfully.

python ./tools/live_demo.py --camera 0

However, when I run in pycharm, i appear the following errors:
Traceback (most recent call last):
File "/mnt/simple-HRNet/scripts/sit_ups.py", line 12, in
from SimpleHRNet import SimpleHRNet
File "/mnt/simple-HRNet/SimpleHRNet.py", line 7, in
from models.detectors.YOLOv3 import YOLOv3
File "/mnt/simple-HRNet/models/detectors/YOLOv3.py", line 12, in
from .yolo.models import Darknet
File "/mnt/simple-HRNet/models/detectors/yolo/models.py", line 10, in
from utils.parse_config import *
ModuleNotFoundError: No module named 'utils'

could you tell me how to solve this problem, thanks a lot.

Hi Stefano
I am currently getting the following error while running the live demo script for a saved video. I was wondering if you had faced such an issue ? python3.6

ImportError: numpy.core.multiarray failed to import
Error in sys.excepthook:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/apport_python_hook.py", line 63, in apport_excepthook
    from apport.fileutils import likely_packaged, get_recent_crashes
  File "/usr/lib/python3/dist-packages/apport/__init__.py", line 5, in <module>
    from apport.report import Report
  File "/usr/lib/python3/dist-packages/apport/report.py", line 12, in <module>
    import subprocess, tempfile, os.path, re, pwd, grp, os, time
  File "/usr/lib/python3.6/tempfile.py", line 184, in <module>
    from random import Random as _Random
ImportError: cannot import name 'Random'

Original exception was:
ImportError: numpy.core.multiarray failed to import
Traceback (most recent call last):
  File "scripts/live-demo.py", line 5, in <module>
    import cv2
  File "/home/timtensor/.local/lib/python3.6/site-packages/cv2/__init__.py", line 3, in <module>
    from .cv2 import *
ImportError: numpy.core.multiarray failed to import
Error in sys.excepthook:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/apport_python_hook.py", line 63, in apport_excepthook
    from apport.fileutils import likely_packaged, get_recent_crashes
  File "/usr/lib/python3/dist-packages/apport/__init__.py", line 5, in <module>
    from apport.report import Report
  File "/usr/lib/python3/dist-packages/apport/report.py", line 12, in <module>
    import subprocess, tempfile, os.path, re, pwd, grp, os, time
  File "/usr/lib/python3.6/tempfile.py", line 184, in <module>
    from random import Random as _Random
ImportError: cannot import name 'Random'

Original exception was:
Traceback (most recent call last):
  File "scripts/live-demo.py", line 5, in <module>
    import cv2
  File "/home/timtensor/.local/lib/python3.6/site-packages/cv2/__init__.py", line 3, in <module>
    from .cv2 import *
ImportError: numpy.core.multiarray failed to import