Identify screen coordinates where user would be looking about gazeml HOT 4 CLOSED

Hi, I'd be happy to point you towards relevant work. Unfortunately, the functionality you look for isn't planned to be implemented in GazeML in the near future.

Broadly speaking there are 3 separate calibrations required for a working eye tracking system.

1. Camera intrinsic parameter calibration OpenCV tutorial
2. Extrinsic camera transformation calibration (can be done using a mirror)
3. User-specific parameter learning for either estimating gaze, or correcting estimated gaze direction

In addition, you need to estimate head pose, which you can do in a few different ways. One simple example uses a 3D head model and the PnP algorithm.

Once you have the head pose (rotation and translation) and gaze direction, it's a matter of some geometry. More specifically speaking, you should use the known (or generic) 3D model of the head to compute an estimated 3D eyeball center for each eye, then use that as the origin of the gaze ray. Intersecting the estimated gaze ray with the known screen plane and applying some scaling yields the on-screen coordinates.

You can skip Step 3 in implementing this pipeline, though with reduced PoR accuracy. Steps 1 and 2 cannot be skipped, nor the head pose estimation. One work which goes against what I stated is iTracker which directly regresses positions on the camera-plane. Since they achieve very impressive results, I would look into their work for a quick and effective solution.

Best of luck in your research.

from gazeml.

Hi Swook,

Thanks for your help with your previous answer!

We tried implementing the steps as suggested by you:

However, I am facing few challenges on this front, your help or guidance would be highly appreciated!!

1. We calculated the intrinsic matrix using the opencv calibration example. However, I am surprised to see my intrinsic matrix is changing every time I run this code on new images of the same chess board

2. We got the Extrinsic Parameters as well along with Head Pose (Rotation and Translation Vector).

3. Since we know, 2D iris center from the Hour-glass model, we converted that to a 3D point using Camera Matrix. and assumed that gaze vector will originate from this point.

Now, we are stuck at the following steps:

a. How do we define the screen plane with the same coordinates. For instance, if camera is located on top of screen just like we have it on a laptop and screen size if 42 inch by 40 inch.

b. How to find the intersection point of that plane and the gaze vector and how to convert that 3D point into 2D point on screen using any scaling function

Please let us know if you could help us/guide on this project

from gazeml.

Hi Subhash,

Apolgies for replying to late, I missed this issue. Unfortunately, I do not have the capacity to provide any guidance on your project.

Best of luck,
Seonwook

from gazeml.

https://www.youtube.com/watch?v=H_9viDBiwOE&list=PLLB6WOMcarJgAyGKsLUgqYD9eTb0GgX04&index=5

Can check the paper @SubhashPavan

from gazeml.