Head horizontal rotation. about jeelizfacefilter HOT 10 CLOSED

There are 3 rotation axis, X,Y and Z, so what do you mean by rotated horizontally ?
Do you have try to change the neural net ? There is one model which is specialized for wide angles (read the readme.md). The link to the boilerplate is here: https://jeeliz.com/demos/faceFilter/demos/threejs/cubeNNCwideAngles/

The rotation is an output from the neural net, then it is stabilized. There is no "rotation detection step".

from jeelizfacefilter.

Hi @xavierjs,
What I mean is that the when I look into the camera and turn my head ~90 degrees, the detection is not working anymore.

BZW: Do you have something similar to: https://github.com/justadudewhohacks/face-api.js ?

from jeelizfacefilter.

Hi,

This is normal because the neural network is trained for clamped angles values.
For the default neural net (NNC.json), these values are:

40 degrees for rotX
60 for rotY (Y pointing up, like in OpenGL)
15 for rotZ

on the wideAngles NN, these values are:
45 degrees for rotX
64 for rotY (Y pointing up, like in OpenGL)
35 for rotZ

So 90° is too wide, it won't work. I regularly push the limits of these neural networks.

I don't have anything similar to https://github.com/justadudewhohacks/face-api.js but I don't really understand why you are submitting all the links to other face detection and tracking libraries. Nobody forces you to use FaceFilter if you think another one is better ;) .

from jeelizfacefilter.

Hi @xavierjs, please, don't get me wrong. I do like your solution and find the examples impressive. I also appreciate that you do open source, good work. I also find that you are quite skilled 👍.
I just want to make sure that I understand the solution as compared to another ones.
At the moment, I don't really know the status and maturity of the solution, although the examples are impresive :-). I still would like to try it and contribute, if possible. I hope that you'll take my comments as my best effort to help ;-).

from jeelizfacefilter.

Related to the question:
I was wandering if it is possible to improve the rotated face detection by using some kind of simple pre-processing, so that the neural network would see more or less aligned face image.

from jeelizfacefilter.

Hi @MariasStory MariasStory,

Yes, I would be possible to rotate the detection window according to the output of the neural network, that's a good idea, I will do some trials I add it to my pipeline (but I have quite a lot of work now so I cannot garantee I will do it soon).

I did not dig vey much the problem of face rotation along Z because I though it was not a real problem (when used with a desktop computer, or a laptop, or a static camera in a kiosk for example the head rotation aroung Z is always small). But this is true that with a mobile device we often have wide angles...

We are working on a landmark based library, which will be very similar to FaceFilter in term of interface and integration but which will detect landmarks on the face. We will release it on a separate github. It is not ready yet but the result are very encouraging. We will also release new and more optimized neuron network models for this library, so it is still improving :).

from jeelizfacefilter.

Hi,

I have added the feature to the lib.
You can do unlimited Z rotation now :).
You only have to set followZRot parameter to true when initializing FaceFilter API.
I have enabled it on the glasses VTO demo here:
https://github.com/jeeliz/jeelizFaceFilter/tree/master/demos/threejs/VTO

But for the detection the face still have to be aligned on the screen vertical axis (with a tolerance of +/- 30 degrees, depending on the neural net).
Then for the tracking the user can rotate its head around the depth axis as much as he wants.

from jeelizfacefilter.

@andrewdeutsch I think this improvement may interest you :)
You asked me, I remember about the limitation for the Z rotation. And also for a better face positionning.

from jeelizfacefilter.

from jeelizfacefilter.

I close the issue since the feature seems to work well. Don't mind to re-open it otherwise :)

from jeelizfacefilter.