emilianavt / openseeface Goto Github PK

Hello,dear anchor.I‘m testing your project,and now have a problem.That is why detect time always show 0.00ms,test picture is following:

Next I found that parameter duration_fd mainly saved the detection time.and it's true.picture is following:

How should I modify to display the correct detection time?
Thanks you watching!

Application appears to be running as expected however VSeeface is unable to communicate with it.

I ran netstat -atulpn but do not see any ports associated with the application. What am I missing?

Any help would be appreciated.

Just to be clear, I am attempting to use Os on one machine Vs on another, if this is not possible please let me know.

Hello! Thanks for your work!
I have some problem when running model.py file

And what means "geffnet.mobilenetv3._gen_mobilenet_v3 needs to be patched to return the parameters instead of instantiating the network" ?

nice work, I dont comprehend heatmap offset format. such as x_offset= 223 * log(p/(1-p)) / 16.
can you provide some explanation? thank you.

Hi.excellent work.Thank you so much for making this amazing project available for others to use.
I'm trying to use the mouth_open and eye blink components separately and build two flask API out of it.one for mouth_open detection and one for eye_blink detection but I read the code so many times everything is so connected I can't seem to use only the parts I want.I want to load the model once and then only do prediction with every API request.how can I do that?
I'm afraid if I edit the code I may do something wrong.do you have any separate source code or anything?

Hey there,

First of all thanks for the awesome work!

I`m trying to grab the gaze estimation results but do not understand the format of the data.
I concluded that the second and third (indices 1 and 2) entries of the eye_state variable show the current gaze, but what exactly do the numbers mean?
Is the gaze relative to the head position or the camera?
Also what is the immediate output of the gaze model?

Can you maybe give me some clarification?

Thank you and all the best!

So i want to use Iruin Webcam for this tool and don't know how to switch the webcam its using and not making it auto use the inbuilt webcam in my laptop

When trying to log the data with the --log-data parameter, I get the complaint AttributeError: 'Namespace' object has no attribute 'fps', and it refers to line 296 in facetracker.py, where args.fps is used instead of fps. Changing this to fps seems to fix the problem. I don't know how to actually contribute this change so I'm creating this issue instead.

Hi! I am creating a rust crate for face alignment and was thinking if I could integrate this as one of the backends. I was wondering of the license terms of the models and if I could integrate them (with credit, of course).

and get this error
[ WARN:0] global /tmp/pip-req-build-99ib2vsi/opencv/modules/videoio/src/cap_v4l.cpp (893) open VIDEOIO(V4L2:/dev/video0): can't open camera by index
There was no valid input.
this command python facetracker.py --visualize 3 --pnp-points 1 --max-threads 4 -c video.mp4
There was no valid input
it was working earlier a few days ago but it is not working anymore
also

Hi.thanks for answering the last issue I posted and thanks again for this great repo. I almost figured the code out and tried to customize it for my own purpose of use the only problem here is that the frames I want to use are frames of faces and I don't want to use the detection model . I tested this approach and it decreased the model accuracy.now my question is:
could you please tell me exactly what kind of preprocessing has to be done on detected faces frames before feeding it to the landmark detector?or what could possibly be the reason for this decrease in model accuracy after deleting the detection part.

` def predict(self, frame, additional_faces=[]):
self.frame_count += 1
start = time.perf_counter()
im = frame

    duration_fd = 0.0
    duration_pp = 0.0
    duration_model = 0.0
    duration_pnp = 0.0

    #new_faces = []
    #new_faces.extend(self.faces)
    bonus_cutoff = len(self.faces)
    #new_faces.extend(additional_faces)
    #self.wait_count += 1
    #if self.detected == 0:
    #    start_fd = time.perf_counter()
    #    if self.use_retinaface > 0:
    #        retinaface_detections = self.retinaface.detect_retina(frame)
    #        new_faces.extend(retinaface_detections)
    #    else:
    #        new_faces.extend(self.detect_faces(frame))
    #    duration_fd = 1000 * (time.perf_counter() - start_fd)
    #    self.wait_count = 0
    #elif self.detected < self.max_faces:
    #    if self.use_retinaface > 0:
    #        new_faces.extend(self.retinaface_scan.get_results())
    #    if self.wait_count >= self.scan_every:
    #        if self.use_retinaface > 0:
    #            self.retinaface_scan.background_detect(frame)
    #        else:
    #            start_fd = time.perf_counter()
    #            new_faces.extend(self.detect_faces(frame))
    #            duration_fd = 1000 * (time.perf_counter() - start_fd)
    #            self.wait_count = 0
    #else:
    #    self.wait_count = 0

    #if len(new_faces) < 1:
    #    duration = (time.perf_counter() - start) * 1000
    #    if not self.silent:
    #        print(f"Took {duration:.2f}ms")
    #    return []

    crops = []
    crop_info = []
    num_crops = 1
    #for j, (x,y,w,h) in enumerate((0,0,self.width,self.height)):
    #SET THIS TO THIS BOUNDING BOX BEACUSE IT'S ALREADY A FRAME OF FACE
    (x,y,w,h) = (0,0,self.width,self.height)
    #(crop_x1,crop_y1,crop_x2,crop_y2) = (0,0,self.width,self.height)
    crop_x1 = x - int(w * 0.1)
    crop_y1 = y - int(h * 0.125)
    crop_x2 = x + w + int(w * 0.1)
    crop_y2 = y + h + int(h * 0.125)
    

    
    

    crop_x1, crop_y1 = clamp_to_im((crop_x1, crop_y1), self.width, self.height)
    crop_x2, crop_y2 = clamp_to_im((crop_x2, crop_y2), self.width, self.height)
    
    scale_x = float(crop_x2 - crop_x1) / self.res
    scale_y = float(crop_y2 - crop_y1) / self.res


    start_pp = time.perf_counter()
    cv2.imwrite('marzi'+str(start_pp)+'.jpg',im[crop_y1:crop_y2, crop_x1:crop_x2])
    #bounding_box = (0,0,self.width,self.height)
    crop = self.preprocess(im, (crop_x1, crop_y1, crop_x2,crop_y2))
    #crop = self.preprocess(im, bounding_box)
    duration_pp += 1000 * (time.perf_counter() - start_pp)
    crops.append(crop)
    #crop_info.append((crop_x1, crop_y1, scale_x, scale_y, 0.0 if j >= bonus_cutoff 0.1))
    crop_info.append((crop_x1, crop_y1, scale_x, scale_y,1))

    start_model = time.perf_counter()
    outputs = {}
    if num_crops == 1:
        output = self.session.run([], {self.input_name: crops[0]})[0]
        conf, lms = self.landmarks(output[0], crop_info[0])
        print(conf)
        if conf > self.threshold:
            try:
                eye_state = self.get_eye_state(frame, lms)
            except:
                eye_state = [(1.0, 0.0, 0.0, 0.0), (1.0, 0.0, 0.0, 0.0)]
            outputs[crop_info[0]] = (conf, (lms, eye_state), 0)
    else:
        started = 0
        results = queue.Queue()
        for i in range(min(num_crops, self.max_workers)):
            thread = threading.Thread(target=worker_thread, args=(self.sessions[started], frame, crops[started], crop_info[started], results, self.input_name, started, self))
            started += 1
            thread.start()
        returned = 0
        while returned < num_crops:
            result = results.get(True)
            if len(result) != 1:
                session, conf, lms, sample_crop_info, idx = result
                outputs[sample_crop_info] = (conf, lms, idx)
            else:
                session = result[0]
            returned += 1
            if started < num_crops:
                thread = threading.Thread(target=worker_thread, args=(session, frame, crops[started], crop_info[started], results, self.input_name, started, self))
                started += 1
                thread.start()

    actual_faces = []
    good_crops = []
    for crop in crop_info:
        if crop not in outputs:
            continue
        conf, lms, i = outputs[crop]
        x1, y1, _ = lms[0].min(0)
        x2, y2, _ = lms[0].max(0)
        bb = (x1, y1, x2 - x1, y2 - y1)
        outputs[crop] = (conf, lms, i, bb)
        actual_faces.append(bb)
        good_crops.append(crop)
    groups = group_rects(actual_faces)

    best_results = {}
    for crop in good_crops:
        conf, lms, i, bb = outputs[crop]
        if conf < self.threshold:
            continue;
        group_id = groups[str(bb)][0]
        if not group_id in best_results:
            best_results[group_id] = [-1, [], 0]
        if conf > self.threshold and best_results[group_id][0] < conf + crop[4]:
            best_results[group_id][0] = conf + crop[4]
            best_results[group_id][1] = lms
            best_results[group_id][2] = crop[4]

    sorted_results = sorted(best_results.values(), key=lambda x: x[0], reverse=True)[:self.max_faces]
    self.assign_face_info(sorted_results)
    duration_model = 1000 * (time.perf_counter() - start_model)

    results = []
    detected = []
    start_pnp = time.perf_counter()
    for face_info in self.face_info:
        if face_info.alive and face_info.conf > self.threshold:
            face_info.success, face_info.quaternion, face_info.euler, face_info.pnp_error, face_info.pts_3d, face_info.lms = self.estimate_depth(face_info)
            face_info.adjust_3d()
            lms = face_info.lms[:, 0:2]
            x1, y1 = tuple(lms[0:66].min(0))
            x2, y2 = tuple(lms[0:66].max(0))
            bbox = (y1, x1, y2 - y1, x2 - x1)
            face_info.bbox = bbox
            detected.append(bbox)
            results.append(face_info)
    duration_pnp += 1000 * (time.perf_counter() - start_pnp)

    if len(detected) > 0:
        self.detected = len(detected)
        self.faces = detected
        self.discard = 0
    else:
        self.detected = 0
        self.discard += 1
        if self.discard > self.discard_after:
            self.faces = []
        else:
            if self.bbox_growth > 0:
                faces = []
                for (x,y,w,h) in self.faces:
                    x -= w * self.bbox_growth
                    y -= h * self.bbox_growth
                    w += 2 * w * self.bbox_growth
                    h += 2 * h * self.bbox_growth
                    faces.append((x,y,w,h))
                self.faces = faces
    self.faces = [x for x in self.faces if not np.isnan(np.array(x)).any()]
    self.detected = len(self.faces)

    duration = (time.perf_counter() - start) * 1000
    if not self.silent:
        print(f"Took {duration:.2f}ms (detect: {duration_fd:.2f}ms, crop: {duration_pp:.2f}, track: {duration_model:.2f}ms, 3D points: {duration_pnp:.2f}ms)")

    results = sorted(results, key=lambda x: x.id)

    return results`

Thank you :)

Hi @emilianavt,
I'm just using opencv without onnxruntime and got the landmarking working using the unoptimized models you shared in #48, which is great.

I just tried the optimized detection model (mnv3_detection_opt.onnx) but not that surprisingly got the same layer error as with the optimized landmarking models.

It'd be fab if you could share the unoptimized detection model too.

I am currently in the process of writing a Flatpak for OpenSeeFace, and have already gotten vpuppr (formerly openseefacegd) to work as a flatpak.
I think it would be beneficial for OpenSeeFace to officially support being ran as a flatpak, as it allows for one command to install (literally just flatpak install) and it wouldn't break on system python upgrades.
This would probably only be beneficial to Linux, and maybe any other UNIX-like OSes.

I would have put up a PR, but I wanted to run this by you real quick because I'm not super familiar with C# and there is a really good chance I just don't have something setup correctly.

OpenSeeVRMDriver.cs seems to have a typing issue, lines below. nowT is a double, but the InterpolatedMap takes a float in that param. I'm more familiar with python, seems like this SHOULD implicitly cast, but Unity and Visual Studio both throw an error. If I cast it as a float inline, Unity seems happy like that.

Does this sound like an actual thing, or am I missing something dumb like upgrading some .net stuff?

Literally cant install this thing cause dependencies are hard to install, are outdated or are just an inconvenience to install.and cant find a proper guide for arch linux.

There is a renderer for OpenSeeFace data that Timothy wrote for the Godot engine.

https://github.com/you-win/openseeface-gd

When I searched this repo for 'godot', there was no results. For people using Godot, it would be nice to find this in the readme. Perhaps in a section titled "See Also"?

Hi emilianavt! Really impressed by the work done in the project. I was trying to use the project in a standalone mobile way, with Unity's Barracuda AI solution.

With importing the .onnx models you generated though, I believe it does not support some of the transform functions you used to create the .onnx model, the following and many similiar errors are thrown

...
Asset import failed, "Assets/models/lm_modelV_opt.onnx" > OnnxImportException: Unknown type FusedConv encountered while parsing layer 361.
...

Here are the transforms supported by Barracuda. Was wondering if you'd consider making the onnx model compatible with Barracuda! It would make it more usable by many Unity developers and bring more attention to it!

Disclosure-- I'm building an open-source SDK, native Unity for face-tracking to Avatars and I'd like to use the tech you've developed!

Hi, your project is very great !

I'm now working on head and eye estimation such as up-down, left-right directions problem. I've read your tracker.py file and I think you already have figured out this problem. Could you please guide which features or attributes I should use to solve this problem ?

Specifically this video https://cdn.discordapp.com/attachments/720652345540870237/1016320201807106118/OSFMediaPipe3DFR.mp4

This is due to Discord now requiring tracking URLs. You can get a one by simply posting the above link into discord, but it appears that the URLs do eventually expire. I suggest uploading to a dedicated video hosting service, like YouTube or Vimeo.

Hello emilianavt !!

Thank you very much for keeping up this great job! I have some doubts.

1 - Are facial marks obtained through an average during the video? LmX = (Lm1 + Lm2 + Lm3 ... LmN) / 2?

2 - When a transition between faces occurs, does the new detection have residual values of the old face?

3 - It is possible to reset the tracking status whenever a new person appears.

I would like to obtain landmarks as current as possible and without any trace of the old face.

Thank you very much.

I was writing a 3D model puppeteering program based on OpenSeeFace. It seems that the documentation of the UDP protocol is somewhat missing. I made my personal "summary" according to multiple code snippets I've found.

I've been thinking about opening a pull request for creating such a documentation.

The new dshowcapture DLLs in commit 89e1040 will open a different camera than the old DLLs in commit 92717a6.

For example, here's my camera list:

With --capture 1, the old DLLs open VTubeStudioCam:

Trying to open camera 1
Trying to open camera 1 with DShowCapture
Camera configuration: 640x360 333333 101
Final camera configuration: 640x360 30
Format: 0 Internal format: 101
Camera: "VTubeStudioCam" Capability ID: None Resolution: 640x360 Frame rate: 30 Colorspace: 101 Internal: 101 Flipped: False
Got frame
Got frame
Got frame
Got frame
Got frame
Got frame

With the new DLLs, 'WarudoCam' is opened instead:

Trying to open camera 1
Trying to open camera 1 with DShowCapture
Camera configuration: 640x360 333333 101
Final camera configuration: 640x360 30
Format: 0 Internal format: 101
Camera: "WarudoCam" Capability ID: None Resolution: 640x360 Frame rate: 30 Colorspace: 101 Internal: 101 Flipped: False
Got frame
Got frame
Got frame
Got frame
Got frame
Got frame

Is this because the new VTubeStudioCam has been blacklisted in the new dshowcapture DLLs, so that the IDs are off-by-one? Is it possible to blacklist the vcams without affecting their ID?

Thank you for this awesome library!

Hello Emiliana emilianavt:
Very nice work for this project! And i have some problems about landmark heatmap offset label for training:
1. How to generate groud truth?
2. What is the loss function for landmark heatmap offset?
I am looking forward to your reply.Thank you very much!

Thanks for this awesome repo. i had a difficuilty in loading the model in unity. it is giving some kind or few layer aren't supported.

I tried many arguments and rebuild few times, but it didn't solved.
When I turn on ir in obs that works correctly (not blacked out, exactly it repeatly blinked like incandescent light)
on OSF, not blinked at all and blacked out randomly.
idk what parts having trouble, but different of obs and osf can tell somehing I think.

under txt file is short log of excuting (no arguments applied without -c (grabbing ir cam))

lenovo500irlog.txt

Hello!
After digging a bit with the program a little bit, it seems the "sock.sendto(packet, (target_ip, target_port))" line on 369 in facetracker.py simply doesn't function through a VPN connection
Attached underneath is the error code produced

Extra info that may be useful for reproducing error and to find fix:
Windows 11 (64 bit, AMD CPU)
NordVPN Connection -> default protocol
Python 3.9 installed
Doesn't work with test using unbranded "USB Camera" as webcam, additionally with it processed through NVIDIA Broadcast's Virtual camera wouldn't work either

Hi, I'm trying to run models with onnxruntime-gpu with TensorRT/Cude executors, and it looks like they do not have FusedConv operator. Can you provide with lesser operator set? Also INT32 model would be nice to have. Thanks.

2021-12-31 23:40:05.374626878 [W:onnxruntime:Default, tensorrt_execution_provider.h:53 log] [2021-12-31 20:40:05 WARNING] /onnxruntime_src/cmake/external/onnx-tensorrt/onnx2trt_utils.cpp:362: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
2021-12-31 23:40:05.374878637 [E:onnxruntime:Default, tensorrt_execution_provider.h:51 log] [2021-12-31 20:40:05   ERROR] 3: getPluginCreator could not find plugin: FusedConv version: 1
/onnxruntime_src/cmake/external/onnx-tensorrt/onnx2trt_utils.cpp:362: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.

When using wink-optimized model, if a person winks, it causes the open eye to veer off to the side in gaze tracking. This results in a lazy eye effect.
This is in using version 1.20.4 under Ubuntu 23.10 in conjunction with VSeeFace. I verified that I was not squinting my open eye at the time and that it occurs with only my left eye open. It also appears to forward as having both eyes closed with my right eye is open regardless of tracking as winking in the terminal output. I have verified that VSeeFace can track my face properly via webcam, but that also brings about a separate issue that was to be resolved via external tracking. This is really one of the only options that I have for my system at the moment, otherwise I would be using other methods. But gaming is and has been shifting to the Linux platform for a little bit now and with good reason, so I'm hoping that I can it along.

Hi.
I'm trying to crop faces from original frame with quaternion points.
but there is a problem. The cropped image does not include the face and is only slightly smaller.
I think these points are not correct for cropping.
Do I using right point for cropping?
Do you have any idea for cropping frame and extract (save) face?

I would like to serve the tracking data using a socket.io server so that it's easy to access the data from a browser. I'm already working an a prototype but I would like to know if that is something that you would be interested in. If you are, I'd give it the appropriate polish and make a pull request.

Hi. Very nice work for this project!
I`m try to make an API with django and I have 2 question.

1. Could we use video file instead of webcam? if yes, how?

2. Is there any feature that extract (save) detected faces?

thank you.

Thanks for your contribution! And I will include your license and your repo link correctly.
I also have a question, estimate facial landmarks from 224x224 image is not neccassary sometime since my input image is near 100x100, is that possible to rescale the network?
I will also try this by my own.

I tried onnx-tf on the models in the models folder but got the error BackendIsNotSupposedToImplementIt: FusedConv is not implemented. Any ideas how this can be fixed? Thanks.

Great work. is the code in model.py used for training the onnx inference models? any chance to release the training codes?

comments are present often it seems; however, you're busy and could probs use a hand.
todo: update comment

Hi sorry if this isn't where I ask for help, I'm new to this.

When opening facetracker.exe , it shows this text, then closes after a second.

Any reason why this might be happening? I might be doing something wrong.

Sorry if this is vague, really dunno anything

EDIT:
sometimes it writes "No frame" briefly before closing. As closing is called before "No frame" shows, and it just depends on how long it takes to close

Trying the new landmark model, It seems that I get random landmarks. You mention that accuracy is very low, but I wonder if I'm doing some mistake! could you share some sample results of the new landmark model as well? thanks.

Also I think the reshaping in here

as stated above do you have any interest in supporting the oak-d line of spatial 3d cameras?, as with the kickstarter that is going around they are now at the same price as the leap controller though that price will rise. the products use a open python sdk called depthai.

Thank you for sharing such a great project. Yet I am working on some project with real time face tracking where I am using dlib 68_face_landmarks.dat but its not stable and accurate somehow I found this project, I saw some links and examples and guess what its amazing.

So how can I use this module in unity for real time face tracking. Or can you guide me which file should I use for real time face tracking. Or is there any chance we can create tracking module as .dat file formats.

Hi.I'm trying to retrieve the eye ratio.I uncommented this line of tracker.py and did it this way and it worked perfectly:

        f = np.clip((np.mean([f_pts[0,1], f_pts[1,1]]) - np.mean([f_pts[2,1], f_pts[3,1]])) / norm_distance_y, 0, None)
        #features["eye_r"] = self.eye_r.update(f, now)

        features["eye_r"] = f

I tested so many times using a live webcam but testing it on frames gotten from a mobile camera dose not work correctly.for example it used to detect a ratio of 0.15 when eyes are closed but for some of the frames it detects the ratio of 0.38 even though eyes are closed.I also saved the image with landmarks projected on it to see if it detects the landmarks correctly and it does.
what do you think the problem might be?
do you think I sould some sort of preprocessing or change the way I'm calculating the ratios?
I also tested it with f.eye_blink and it detected the eyes as open.returning 1 instead of 0
thank you
sorry for bothering you :)

Hello emilianavt, I'm trying to utilize the provided pytorch models (model.py) and weights to detect facial landmarks. However, I found its performance is worse than onnx version. An example is shown below (lm_model3 for both).

Do you have any suggestion? In my practice, I modify the model.py according to this issue, and I adapt the tracker.py by replacing the onnx models (self.session and self.detection) with pytorch models (OpenSeeFaceLandmarks and OpenSeeFaceDetect). I follow the image preprocessing codes and postprocessing codes in the original tracker.py. I'm wondering if you could provide a demo tracking code for pytorch models. Any advice is appreciated!

Hello. Probably, my question is quite stupid, but nvm:
Ur model should work in a detector fashion (fully convolutional). But ur model (at least the large one) requires a precise image shape (224x224). Can you briefly describe the architecture?

About 10 months ago, pull request #57 fixed a deprecation issue in model.py and tracker.py. Running facetracker.py works fine when using master, however if you try to run the latest release you still get an error:

AttributeError: module 'numpy' has no attribute 'float'.
`np.float` was a deprecated alias for the builtin `float`. To avoid this error in existing code, use `float` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.float64` here.
The aliases was originally deprecated in NumPy 1.20; for more details and guidance see the original release note at:
    https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
Traceback (most r....

Creating a new release with the latest code should fix the issue 😄

How about to add VMC Assistant support to OpenSeeFace?

I have read the docu of the protocol.

Currently iam writing an python pip package for vmc.
But i have some questions about bone transform:

/VMC/Ext/Bone/Pos <boneName> <p.x> <p.y> <p.z> <q.x> <q.y> <q.z> <q.w>

After running an OSC sniffer with VSeeFace in "transmitter mode" and covered webcam i get for each bone the T-Pose values of my model.
That's clear.
Now the question: How do you apply the tracking data to these T-Pose to get the correct position and quaternion?

In short:
So if i want to add VMC support to OpenSeeFace, then how to apply the tracking data from the webcam to the initial T-Pose of the model bones?

When a face is centered on camera the reported translation X and Y coordinates are not zeros. A possible reason for this is that the camera intrinsic matrix has offsets computed for the wrong axis when doing the position solving.

The camera matrix is

self.camera = np.array([[width, 0, width/2], [0, width, height/2], [0, 0, 1]], np.float32)

so that means c_x = width/2 and c_y = height/2 but then later image_pts that is passed to solvePnP() is a vector of (y,x) points so self.camera translates the Y coordinates with width/2 and vice-versa for the point X coordinates.

I'd submit a patch to fix this but the code is somewhat tricky so I'm not sure I'd get it right.

NumPy 1.24 appears to break pupil/gaze tracking, when running python facetracker.py --model 4 -c models/benchmark.bin --repeat-video 1 -v 1 there is no point in or near the pupil, but after downgrading to NumPy 1.23 the dot tracking the pupil appears again.

Below is the output with NumPy 1.23 and 1.24 for comparison:

I am using Python 3.10.10 on Arch Linux, and the specific NumPy versions I tried are 1.23.5 and 1.24.2.

Output of pip freeze in the virtual environment excluding numpy:

coloredlogs==15.0.1
flatbuffers==23.3.3
humanfriendly==10.0
mpmath==1.3.0
onnxruntime==1.14.1
opencv-python==4.7.0.72
packaging==23.0
Pillow==9.4.0
protobuf==4.22.1
sympy==1.11.1

P.S. onnxruntime works for Python 3.10 now so no need to use the nightly version.

How can I use this project if I use a macbook?

What does the single in get_eye_state mean?

I guess it's for detecting multiple people's eyes? Anyway, when num_crops == 1 (ie only 1 person), I think this argument needs to be set to true here:

Otherwise, it consumes too much cpu! Initially I found this because the built VSeeFace.exe consumes only ~10% cpu, but using this python script with lower fps consumes up to 20% cpu, so I was wondering what makes the difference. After setting single=True inside that code, now the cpu usage is about the same (10%).

I tried to download your program and it says Failed-forbidden and I tried everything so it must be on your side.

Hi @emilianavt
Thanks for the nice repo . While running the code , the tracker takes a long time after some frames , I have attached the screenshot here , can you please tell what may be the possible reason ? You can see that in the attached screenshot that , after 22 ms , the tracker takes a longer time upto 2398 ms which is almost 100 times more .
Thanks
Trilok