eladrich / pix2vertex.pytorch Goto Github PK

after the cropping process i am getting traceback errors. ( i am using myBinder link )

NameError Traceback (most recent call last)
in
----> 1 net_res = reconstructor.run_net(img_crop)
2 p2v.vis_net_result(img_crop,net_res)
3 final_res = reconstructor.post_process(net_res)

NameError: name 'reconstructor' is not defined

Thank you for the awesome work and making the pytorch port available.

Is there a way to visualize the intermediate "registered template". Correct me if I am wrong, "registered template" is the projection to a template face before fine detail reconstruction?

I see that the matplotlib plot has colors along with a mesh.
Currently the mesh generated doesnt have any color/texture with it. How do I export a mesh with a texture?

So I ran this piece of code and it gave me visualization of that 3D face mesh but how do I get vertices, colors and triangles out of this object:

import pix2vertex as p2v
from imageio import imread
image = imread("trump.jpg")

detector = p2v.Detector()
reconstructor = p2v.Reconstructor(detector=detector)
img_crop = detector.detect_and_crop(image)
net_res = reconstructor.run_net(img_crop)
p2v.vis_net_result(img_crop,net_res)
final_res = reconstructor.post_process(net_res)

Everything work just fine but the last step am getting this error..... module 'pix2vertex' has no attribute 'save2stl'... How can I fix it ..Highly appreciate your efforts

How do I run it on Windows in detail? I am new to these things. I hope you explain, thank you

The thing panic after "p2v.reconstruct()"

OS: High Sierra
CPU: AMD Ryzen 5 1500x
Python 3.7.0
Preinstalled python: 2.7.16

`
Python 3.7.0 (v3.7.0:1bf9cc5093, Jun 26 2018, 23:26:24)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

import pix2vertex as p2v
from imageio import imread
image = imread("Desktop/lebedev.jpg")
result, crop = p2v.reconstruct(image)
Loading default detector weights from /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pix2vertex/../weights/shape_predictor_68_face_landmarks.dat
loading default reconstructor weights from /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pix2vertex/../weights/faces_hybrid_and_rotated_2.pth
Detection 0: Left: 297 Top: 297 Right: 759 Bottom: 759
Segmentation fault: 11
`

The released network was trained on a combination of synthetic images and unlabelled real images for some extra robustness

As this uses the pix2pix pipeline, I'm curious to how unlabelled real images were incorporated during training. Did you fit the synthetic dataset to some real life face dataset like CelebA as done by Sengupta et al. in SfSNet ?

Edit: It also looks like your new model is more robust to occlusions than the previous one.

Hi, Thank you for releasing Pytorch version. I was hoping to have some fun but ran into some plotting issues.

I was wondering if you can add the code to output it as obj to actually preview the mesh in 3D packages ?

Hi, I run the model and get some results, which is not good comparised with those in the paper.
The difference is huge, I want to know where I might have done wrong.

The results in the paper:

The results I get:

Hello thank you for this amazing repository I want to know if it's possible to save the MTL file to use it in other programs such as Blender, thank you very much again.

After executing the reconstruct_pipeline.ipynb notebook both locally (after git cloning the repo) and on myBinder (with the link you provide), the resulting faces_hybrid_and_rotated_2.pth file is just 2.2kB and contains only HTML code which (obviously) triggers an error when trying to unpickle it.

Steps to reproduce the error on myBinder:

1. run the notebook on myBinder
   --> results in the error: UnpicklingError: invalid load key, '<'.
   (Note: of course, since it tries to unpickle some HTML code)

2. add a line with: !cat weights/faces_hybrid_and_rotated_2.pth

Which results in displaying the content of faces_hybrid_and_rotated_2.pth:

<!DOCTYPE html><html><head><title>Google Drive - Virus scan warning</title><meta http-equiv="content-type" content="text/html; charset=utf-8"/><style nonce="WemtlDkUYYfsC7fLz9V91w">/* Copyright 2022 Google Inc. All Rights Reserved. */
.goog-inline-block{position:relative;display:-moz-inline-box;display:inline-block}* html .goog-inline-block{display:inline}*:first-child+html .goog-inline-block{display:inline}.goog-link-button{position:relative;color:#15c;text-decoration:underline;cursor:pointer}.goog-link-button-disabled{color:#ccc;text-decoration:none;cursor:default}body{color:#222;font:normal 13px/1.4 arial,sans-serif;margin:0}.grecaptcha-badge{visibility:hidden}.uc-main{padding-top:50px;text-align:center}#uc-dl-icon{display:inline-block;margin-top:16px;padding-right:1em;vertical-align:top}#uc-text{display:inline-block;max-width:68ex;text-align:left}.uc-error-caption,.uc-warning-caption{color:#222;font-size:16px}#uc-download-link{text-decoration:none}.uc-name-size a{color:#15c;text-decoration:none}.uc-name-size a:visited{color:#61c;text-decoration:none}.uc-name-size a:active{color:#d14836;text-decoration:none}.uc-footer{color:#777;font-size:11px;padding-bottom:5ex;padding-top:5ex;text-align:center}.uc-footer a{color:#15c}.uc-footer a:visited{color:#61c}.uc-footer a:active{color:#d14836}.uc-footer-divider{color:#ccc;width:100%}</style><link rel="icon" href="null"/></head><body><div class="uc-main"><div id="uc-dl-icon" class="image-container"><div class="drive-sprite-aux-download-file"></div></div><div id="uc-text"><p class="uc-warning-caption">Google Drive can't scan this file for viruses.</p><p class="uc-warning-subcaption"><span class="uc-name-size"><a href="/open?id=1op5_zyH4CWm_JFDdCUPZM4X-A045ETex">faces_hybrid_and_rotated_2.pth</a> (208M)</span> is too large for Google to scan for viruses. Would you still like to download this file?</p><form id="downloadForm" action="https://docs.google.com/uc?export=download&amp;id=1op5_zyH4CWm_JFDdCUPZM4X-A045ETex&amp;confirm=t" method="post"><input type="submit" id="uc-download-link" class="goog-inline-block jfk-button jfk-button-action" value="Download anyway"/></form></div></div><div class="uc-footer"><hr class="uc-footer-divider"></div></body></html>

Note: Manually opening this file in a browser indeed leads to the good link. After manual download, and moving the file to weights/. I could use your program.

Otherwise, your program is great! Congratulations and thank you!

Hi, this is a great implementation! not only the code is complete from model to plotting phase, but the 3D plot is detailed as well!

As a new deep learning enthusiast I have learned alot from your code, but I still don't understand about the pix2pix output shape : (1, 7, 512, 512)

from reconstructor.py I just noticed that the 7 is separated into two variables: im_pncc and im_depth,
but I have difficulties understanding what im_pncc is and why in the following code:
mask = np.any(im_depth, axis=2) * np.all(im_pncc, axis=2)

is not just:
mask = np.any(im_depth, axis=2)
since im_depth across each of 512x512 already shows the location of the face, therefore there is no need for im_pncc (?)

and lastly, is there any way to train the pix2pix model for full-body 3D?

Thank you !