fengju514 / expression-net Goto Github PK

I don't know if there is any # 3D facial expression model that can be provided。Thank you very much.

Thanks for your code
Can you tell me which face detector you use

hello，How to train your network with the data sets you provide？thank you.

Need to update the README to include missing dependency scipy.

Traceback (most recent call last):
File "main_ExpShapePoseNet.py", line 5, in
import scipy.io as sio
ImportError: No module named scipy.io

Solution: $pip install scipy

hello,
thanks for your great works! I found the PoseNet just estimate the Rx,Ry,Rz,Tx,Ty,Tz parameters but lack of scale parameters. I found the project of Face-Pose-Net predict 6DoF head pose (scale, pitch, yaw, roll, translation_x, translation_y). how can i got the parameter of scale?

Could you please tell me which version of tensorflow do you used?
Could you please tell me which version of openCV for python do you used?
Without those infomation I can't run your project in my computer!
Thank you!

Current build will not correctly process facial hair so individuals with beards will not have correct mouth placement in the mesh (.ply) file.

Is it possible to tune the net so that it is able to deal with facial hair?

Note that the images in the examples folder include no individuals with beards or facial hair.

Suggestion: include this note in the readme.

@fengju514 Great job! I have appreciated your brilliant paper that can output 29D expression coefficients. But I wonder if these expression coefficents could be used for expression retargeting purpose. I have a doubt because these coefficients are for the first 29 principal components of BFM expression space, which dont have a meaning mapping to man-made expression blendshapes, such as left-eye wink, wide-open mouth. Am I right? If I am wrong, could you please tell me how the 29 expression blendshapes should be created? for example, which coeficient should be mapped to left-eye wink?
Any explanation from anyone would be appreciated. Thanks in advance.
Best regards,
Sean

Trying to run the program via the directions returns error:

$ python main_ExpShapePoseNet.py /home/johnh/gits/Expression-Net/

make dir
Traceback (most recent call last):
File "main_ExpShapePoseNet.py", line 280, in
tf.app.run()
File "/home/johnh/gits/Expression-Net/venv/local/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 125, in run
_sys.exit(main(argv))
File "main_ExpShapePoseNet.py", line 276, in main
extract_PSE_feats()
File "main_ExpShapePoseNet.py", line 73, in extract_PSE_feats
data_dict = myparse.parse_input(inputlist) # please see input.csv for the input format
File "/home/johnh/gits/Expression-Net/myparse.py", line 5, in parse_input
reader = csv.DictReader(open(input_file,'r'))

Suggest modifying program to search for input.csv file and/or changing quickstart directions to include path-to-file and file-name. Modifying data_dict = myparse.parse_input(inputlist) to simply grab whatever .csv file is found in directory might also work but I'm not knowledgeable enough python user to implement.

After fixing some stupid issue on the setting on my machine, the code finally run. The result is some how working fine on pose and expression. However, the shape and color of the results are not coming out as expected. I guess they were some color/material from the database. For the shape part, its working well for moderate figure/face while there is still room on the details of shape in the future. Here are 2 questions would like to ask.
1.Would it be possible to have the color extracted from the input images and where should the code be amended?
2.Would it be possible to have texture coordinates on the mesh rather than the vertices colors.

Would like to help on the project as well, say implement the features that I have asked under your kindly guidance. Thanks

Traceback (most recent call last):
File "main_ExpShapePoseNet.py", line 50, in
file = np.load("./fpn_new_model/perturb_Oxford_train_imgs_mean.npz")
File "/home/zhangyongle/.conda/envs/tensorflow_expression-net/lib/python2.7/site-packages/numpy/lib/npyio.py", line 443, in load
"Failed to interpret file %s as a pickle" % repr(file))
IOError: Failed to interpret file './fpn_new_model/perturb_Oxford_train_imgs_mean.npz' as a pickle

The current repo does not contain a license, which means the following according to GitHub's licensing help page

You're under no obligation to choose a license. It's your right not to include one with your code or project, but please be aware of the implications. Generally speaking, the absence of a license means that the default copyright laws apply. This means that you retain all rights to your source code and that nobody else may reproduce, distribute, or create derivative works from your work. This might not be what you intend.

Even if this is what you intend, if you publish your source code in a public repository on GitHub, you have accepted the Terms of Service which do allow other GitHub users some rights. Specifically, you allow others to view and fork your repository.

If you want to share your work with others, we strongly encourage you to include an open source license.

I really want to use your code. Can you confirm that you want to retain all rights?

I downloaded the model according to your steps and I get an error while running。

hello and thanks for the code!
I have some questions regarding the image format
-your input images are all in 720x576 format . is this mandatory ?
-in your readme you say : <FACE_X, FACE_y, FACE_WIDTH, FACE_HEIGHT> is the face bounding box information, either obtained manually or by the face detector.
which face detector ?
can I use dlib face detector or do you recommend another one ?
-your tmp (cropped) images are all 227x227 , is this mandatory ?

thanks in advance

When I run the code, an error occurred regarding the caffe.py file in Kaffe folder. But actually there was no caffe.py file in Kaffe. What should I do?

I am receiving constantly an annoying error when I am trying to run the approach. More specifically,

net_data = np.load("fpn_new_model/PAM_frontal_ALexNet.npy").item()
(Pdb) c
Traceback (most recent call last):
File "/opt/libraries/anaconda3/envs//lib/python3.6/site-packages/numpy/lib/format.py", line 650, in read_array
array = pickle.load(fp, **pickle_kwargs)
UnicodeDecodeError: 'ascii' codec can't decode byte 0x8f in position 0: ordinal not in range(128)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "main_ExpShapePoseNet.py", line 292, in
tf.app.run()
File "/opt/libraries/anaconda3/envs//lib/python3.6/site-packages/tensorflow/python/platform/app.py", line 125, in run
_sys.exit(main(argv))
File "main_ExpShapePoseNet.py", line 288, in main
extract_PSE_feats()
File "main_ExpShapePoseNet.py", line 100, in extract_PSE_feats
net_data = np.load("fpn_new_model/PAM_frontal_ALexNet.npy").item()
File "/opt/libraries/anaconda3/envs//lib/python3.6/site-packages/numpy/lib/npyio.py", line 421, in load
pickle_kwargs=pickle_kwargs)
File "/opt/libraries/anaconda3/envs//lib/python3.6/site-packages/numpy/lib/format.py", line 656, in read_array
"to numpy.load" % (err,))

I am not sure what is happening but it seems that i has to do with loading the AlexNet file. What is strange is that 2 days before i run the code without issue with the same files. What can be wrong here?

Hey @fengju514! I'm a high school student interested in TensorFlow. I love this project, and have an idea for taking it further. Would you be interested in working together on an implementation for this using TensorFlow Graphics? I think it would make a great addition to the TF Graphics Tutorial Repository, and it will also further the project by further allowing the 3D outputs to be visualized in TensorBoard with threejs instead of MeshLab. Please let me know if you want to work on this, because while I'm willing to start a PR for this, I don't think I can do it without guidance and assistance.

When the input picture is closed, the fitting effect is open mouth. How can I solve it? @fengju514

Sir,i have read the paper and went through this Expression-Net with a Lenovo docker Container.And now,i want to train my own model,but i have no idea about it. could u help me ?

I downloaded the model according to your steps and I get an error while running。

When I run the code there was an error "ModuleNotFoundError: No module named 'kaffe.caffe'" and actually there was no caffe.py file in the Kaffe folder. Do you have any idea about this?

Now, python 3.10.6 has been published and tensorflow 2.10.1 is the latest.
But it completely cannot support this code.
Should I download your version to run the code or there are some other methods to solve this problem?

why?

hello and thanks for the code!
I want to know the means of 'FACE_X' ,'FACE_y' ,'WIDTH' and 'HEIGHT'