Git Product home page Git Product logo

prnet_pytorch's Introduction

PRNet PyTorch 1.1.0

Github Github Github

This is an unofficial pytorch implementation of PRNet since there is not a complete generating and training code of 300WLP dataset.

  • Author: Samuel Ko, mjanddy.

Update Log

@date: 2019.11.13

@notice: An important bug has been fixed by mj in loading uv map. The original uv_map.jpg is flipped, so *.npy is used here to redress this problem. Thanks to mjanddy!

@date: 2019.11.14

@notice: Inference Stage Uploaded, pretrain model available in results/latest.pth. Thanks to mjanddy!


Noitce

Since replacing the default PIL.Imgae by cv2.imread in image reader, you need do a little revise on your tensorboard package in your_python_path/site-packages/torch/utils/tensorboard/summary.py

What you should do is add tensor = tensor[:, :, ::-1] before image = Image.fromarray(tensor) in function make_image(...).

...
def make_image(tensor, rescale=1, rois=None):
    """Convert an numpy representation image to Image protobuf"""
    from PIL import Image
    height, width, channel = tensor.shape
    scaled_height = int(height * rescale)
    scaled_width = int(width * rescale)

    tensor = tensor[:, :, ::-1]
    image = Image.fromarray(tensor)
    ...
...

① Pre-Requirements

Before we start generat uv position map and train it. The first step is generate BFM.mat according to Basel Face Model. For simplicity, The corresponding BFM.mat has been provided here.

After download it successfully, you need to move BFM.mat to utils/.

Besides, the essential python packages were listed in requirements.txt.

② Generate uv_pos_map

YadiraF/face3d have provide scripts for generating uv_pos_map, here i wrap it for Batch processing.

You can use utils/generate_posmap_300WLP.py as:

python3 generate_posmap_300WLP.py --input_dir ./dataset/300WLP/IBUG/ --save_dir ./300WLP_IBUG/

Then 300WLP_IBUG dataset is the proper structure for training PRNet:

- 300WLP_IBUG
 - 0/
  - IBUG_image_xxx.npy
  - original.jpg (original RGB)
  - uv_posmap.jpg (corresponding UV Position Map)
 - 1/
 - **...**
 - 100/ 

Except from download from 300WLP, I provide processed original--uv_posmap pair of IBUG here.

③ Training

After finish the above two step, you can train your own PRNet as:

python3 train.py --train_dir ./300WLP_IBUG

You can use tensorboard to visualize the intermediate output in localhost:6006:

tensorboard --logdir=absolute_path_of_prnet_runs/

Tensorboard example

The following image is used to judge the effectiveness of PRNet to unknown data.

(Original, UV_MAP_gt, UV_MAP_predicted) Test Data

④ Inference

You can use following instruction to do your prnet inference. The detail about parameters you can find in inference.py.

python3 inference.py -i input_dir(default is TestImages) -o output_dir(default is TestImages/results) --model model_path(default is results/latest.pth) --gpu 0 (-1 denotes cpu)

Test Data


Citation

If you use this code, please consider citing:

@inProceedings{feng2018prn,
  title     = {Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network},
  author    = {Yao Feng and Fan Wu and Xiaohu Shao and Yanfeng Wang and Xi Zhou},
  booktitle = {ECCV},
  year      = {2018}
}

prnet_pytorch's People

Contributors

tomguluson92 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

prnet_pytorch's Issues

About forward

I have trained a model in Pytorch with your code. But I wonder how to transform the Pytorch model to tensorflow one. Cause your code don't have forward code.
I hope you can help me. Thanks a lot.

请教一下训练策略

batch,epoch,lr衰减等等怎么设置,我发现batch越大越难收敛,但怎么都收敛不到原论文的精度,请问下你是怎么复现的

[FIXED] TypeError: cannot unpack non-iterable NoneType object

Solution

Original Code

Fixed version

if img_id is not None:
    ...

or simply just remove if img_id:

why

The TypeError: cannot unpack non-iterable NoneType object occurs when the function get_img_path in PRNetDataset tries to load img_id == 0.

When the generate_posmap_300WLP.py is executed according to README.md, each folder is generated sequentially numbered from folder 0 as described.

However, in python, 0 would be same with False as below:

img_id = 0
if img_id: 
    print("true")
else:
    print("false")

>>> false

Therefore, the function does not return a value, causing the data loader to bug out.

someproblem about inference

when i run the inference code it shows errors like below
RuntimeError: Error(s) in loading state_dict for ResFCN256:
Missing key(s) in state_dict: "block0.weight", "block1.shortcut_conv.weight"
hope you replay! thanks

poor inference.

Hello,

I am trying to run the landmark detector on a webcam video stream and plot the landmarks but the landmarks are very unstable and almost always far off from what is correct. This is the piece of code that i am running for plotting the landmarks. How can i fix this?

transform_img = transforms.Compose([
        transforms.ToTensor(),
        transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
    ])
prn = PRN("results/latest.pth")
for frame from cam:  #psuedo code
    frame_temp = cv2.resize(frame[int(bbox[1]*0.7):int(bbox[3]*1.3),int(bbox[0]*0.7):int(bbox[2]*1.3)],     (256, 256)) #sending only the face portion of the video stream
    image_t = transform_img(frame_temp)
    image_t = image_t.unsqueeze(0)
    pos = prn.net_forward(image_t.cuda())


    out = pos.cpu().detach().numpy()
    pos = np.squeeze(out)
    cropped_pos = pos * 255
    pos = cropped_pos.transpose(1, 2, 0)
    kpt = prn.get_landmarks(pos)
    cv2.imshow('sparse alignment', plot_kpt(frame_temp, kpt))

cv2.waitKey(10)




I am running this repeatedly over each frame from the web cam.

About training code

Hi, I am on reproducing your training code, however I find the model is pretty small(2 MB). I wonder if it's my training data fault or what.
My email : [email protected]

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.