yuvalnirkin / face_segmentation Goto Github PK
View Code? Open in Web Editor NEWDeep face segmentation in extremely hard conditions
License: Apache License 2.0
Deep face segmentation in extremely hard conditions
License: Apache License 2.0
Hi,
Thanks for the great work, many years later it is in fact still difficult to find this particular functionality and quality elsewhere.
Unfortunately, Caffe
was never a tame dependency and this is not improving, to the point that it is harder to get it to work nicely with other dependencies as it is to rewrite full setups from scratch.
Fast forward to this moment, I've ported the model to PyTorch. Hopefully it can be helpful for others:
https://github.com/andres-fr/face_segmentation_pytorch
I'm happy to receive any feedback. Cheers!
Andres
Hello, thanks for your wonderfull work! What is the license for the pre-trained model? And could we use it for a commercial application?
Hi,
When trying to run tour code, I got the following error:
F0613 23:16:31.623750 22135 common.cpp:66] Cannot use GPU in CPU-only Caffe: check mode.
Is it possible to use this code without the GPU?
Best,
Michal
I tried the python caffe inference. It took 4 seconds to detect an image using CPU.
Will the C++ inference be much faster than caffe?
Thanks
could this output an alpha mask?
Greetings,
This may be off topic, but I'm just trying to find some help.
I'm trying to get 300 FCN model to run in my xCode project. I'm converting .coffemodel to .mlmodel with coremltools:
coreml_model = coremltools.converters.caffe.convert(caffe_model, image_input_names='data', is_bgr = True, red_bias = -104, blue_bias = -123, green_bias = -117, image_scale = 1)
As far as I understand the input image is in BGR color space, with above mentioned biases.
After conversions when I read the model description with coremltools:
input { name: "data" type { imageType { width: 300 height: 300 colorSpace: BGR } } } output { name: "score" type { multiArrayType { dataType: DOUBLE } } } metadata { userDefined { key: "coremltoolsVersion" value: "3.3" } }
The output has no shapes.
When I add the model to Xcode project, I run the model by passing CVPixelBuffer as input
let input = buffer(from: userSelectedImage_UI) guard let prediction = try? model.prediction(data: input!) else { return }
the output of the model is MultyArray.
let output = prediction.score
How can I convert it to CVPixelBuffer, if there're no shapes.
I've tried using MultiArray converters to no avail, the output is just black image
I've tried this and this methods.
If anybody knows how to get this working in CoreML I'd really appreciate it
I coded the model in lasagne
and changed the weights of the model accordingly but I am not getting good results. There is too much of noise in the results. Below is an example showing the original image and the corresponding segmentation mask.
Any idea why is this happening and where the issue might be?
I had a hard time getting Caffe installed, so I figured I'd try out your model using openCV's dnn.readNetFromCaffe() along with your .caffemodel and .prototxt.
The output of net.forward() when using this method is a 1x21x300x300 matrix which can be squeezed to 21x300x300. Each of the 21 300x300 arrays, when normalized, seem to constitute a type of heat map. Some of these can be seen below.
My question is, how would I combine these to get the actual face segmentation? I tried to parse your code to see if I could figure it out but fell short of understanding. Thanks!
Installation is not clear. Could you update the README with more thorough installation instructions?
I'm trying to understand the output of the caffe network and where argmax
prediction is coming from in the python interface: out = net.blobs['score'].data[0].argmax(axis=0)
The output: net.blobs['score'].data
has 21 channels. What are the additional 20 channels and how do we know that if a face pixel is present, the maximum value will be in the second channel?
Thanks
How to organize data
When compile this the cmake file found caffe folder properly, but shows below error.
Does it only supports caffe without gpu?
device_alternate.hpp:34:23: fatal error: cublas_v2.h: No such file or directory
Hi, I got a error when I try to compile.
/home/migudm/caffe/build/include/caffe/proto/caffe.pb.h:17:2: error: #error This file was generated by an older version of protoc which is incompatible with your Protocol Buffer headers. Please regenerate this file with a newer version of protoc.
There are two versions of protobuf(2.5 and 3.5.1) in my Ubuntu14. caffe is complied by protoc 2.5
I try to change the default protoc to version2.5. but it failed again.
shall I recomplie the Caffe with protoc 3.5.1 ?
I have looked for many similar problems, but still not solved
Can u release Unet model,thanks。
I come across the following question. Anyone can help me.
//usr/lib/libblas.so.3: undefined reference to `gotoblas'
collect2: error: ld returned 1 exit status
face_seg_image/CMakeFiles/face_seg_image.dir/build.make:151: recipe for target 'face_seg_image/face_seg_image' failed
make[2]: *** [face_seg_image/face_seg_image] Error 1
CMakeFiles/Makefile2:140: recipe for target 'face_seg_image/CMakeFiles/face_seg_image.dir/all' failed
make[1]: *** [face_seg_image/CMakeFiles/face_seg_image.dir/all] Error 2
Makefile:127: recipe for target 'all' failed
make: *** [all] Error 2
hi,in your face_seg.py ,u write :in_ -= np.array((104.00698793,116.66876762,122.67891434))
can u explain the meaning of three numbers in the brackets
thanks a lot~
Hi, Thanks for this project. It works good for the front faces. However, for some scenarios results are not so good.
Few suggestions or update to this paper will be great help. Thanks!
Hello, I am interested in your work and want to train my own face segmentation network.
Could you release your database for face segmentation?
Thank you very mach!
Hi,
Thanks for the software. When running "face_seg_image", here is an error:
Is there an option in the compiling stage to specify to use Caffe CPU?
Thanks!
Hi,
I met this error when running the demo
I0921 13:13:41.511903 1711 net.cpp:242] This network produces output score
I0921 13:13:41.511961 1711 net.cpp:255] Network initialization done.
I0921 13:13:43.045377 1711 net.cpp:744] Ignoring source layer data
I0921 13:13:43.045660 1711 net.cpp:744] Ignoring source layer data_data_0_split
I0921 13:13:43.154325 1711 net.cpp:744] Ignoring source layer loss
Segmentation fault (core dumped)
I run in CPU mode, using a computer with 32 GB RAM. I think that is related to Caffe.
Did someone meet this error.
Thank you!
Great work. I've enjoyed the results and experimenting with your concept. Appreciate the effort you made towards sharing your code and pipeline.
I saw a reference to a new and improved Unet version in PyTorch? How goes the experiments with the better version?
Hi
Is it possible to have your training dataset?
I would like to train from scratch for a new model.
Best regards,
Hello Yuval,
Thank you for your project. I am also working on face swap projects. Now I am trying to tackle the problem of face occlusions.
I find the performance on the README page really amazing. However, after installation I find that the performance on my face-swap training images not as good as your demo image. The model is unable to segment eye-glasses.
Here are some examples:
I ran the model by this command:
face_seg_image my_image.jpg -o . -m ../data/face_seg_fcn8s.caffemodel -d ../data/face_seg_fcn8s_deploy.prototxt
where my_image.jpg is a 256x256 image in which the face is aligned via 5 landmarks (eyes, nose and mouth).
Do you have any ideas on this? Or is there anything wrong with my input format?
Thank you in advance.
when I run
python face_seg.py
I get an error as No module surgery
line number 4 in face_seg.py is
import surgery
But there is no module as surgery any where on my system
from where can i get the surgery module
I am getting the following error while trying to install face-segmentation.
face_segmentation/build$ make -j8
Scanning dependencies of target face_seg
[ 14%] Building CXX object face_seg/CMakeFiles/face_seg.dir/face_seg.cpp.o
[ 28%] Building CXX object face_seg/CMakeFiles/face_seg.dir/utilities.cpp.o
In file included from /usr/include/cublas_v2.h:65:0,
from /usr/include/caffe/util/device_alternate.hpp:34,
from /usr/include/caffe/common.hpp:22,
from /usr/include/caffe/blob.hpp:8,
from /usr/include/caffe/caffe.hpp:7,
from /home/deyrahul/Research/3DMM/face_segmentation/face_seg/face_seg/face_seg.h:11,
from /home/deyrahul/Research/3DMM/face_segmentation/face_seg/face_seg.cpp:1:
/usr/include/cublas_api.h:72:10: fatal error: driver_types.h: No such file or directory
#include "driver_types.h"
^~~~~~~~~~~~~~~~
compilation terminated.
face_seg/CMakeFiles/face_seg.dir/build.make:82: recipe for target 'face_seg/CMakeFiles/face_seg.dir/face_seg.cpp.o' failed
make[2]: *** [face_seg/CMakeFiles/face_seg.dir/face_seg.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
CMakeFiles/Makefile2:197: recipe for target 'face_seg/CMakeFiles/face_seg.dir/all' failed
make[1]: *** [face_seg/CMakeFiles/face_seg.dir/all] Error 2
Makefile:149: recipe for target 'all' failed
make: *** [all] Error 2
Ubuntu: 18.04
CUDA: 10.1
Caffe: 1.0
Boost: 1.72
OpenCV: 4.2
My CMake-GUI 3.17.1 produces the following configuration for it:
Any help would be appreciated! Thanks.
where is your U-Net models? and the train and inference code of Pytorch?
Hello. I want to use your model to segment my own face. But i have no idea about the pricinple of face bbox cropping. I failed to segment face when i used face bbox which is to tight. So i expand the face by some ratios. However, the result is not good enough as well.
Can u tell me how to choose the most suitable face to achieve your performance? Thank you.
I am interested in your work and want to train my own face segmentation network. Could you release your database for face segmentation? Thanks a lot!
Hi, I tried your system - and maybe i am doing something wrong, but this system seems to give rather poor results. How were you able to get the results you describe?
I am using your implementation directly, so it should be same code i am running as you.
See example of poor result here:
Maybe you used some special model which you have not shared?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.