This code should use video scaler (video_scaler_t) from obs-studio so that any format can be converted.

Hello,

sorry if I ask a totally stupid question, but for the hell I cannot find the download files for obs-face-tracker. I wanted to install it on macOS according to the Wiki instructions, but I have no idea where to get the dmg from.

Could you please point out, where the files reside?

Thanks a lot.

Thinking the user experiences, this plugin should send commands to the camera through the other plugin obs-ptz so that a user can install both plugin and use at the same time. Especially VISCA over serial port cannot be controlled from individual program.
glikely/obs-ptz#17

A library openblas need to be added target_link_libraries.

Error

When face detection starts, I got this error.

obs: symbol lookup error: /usr/local//lib/obs-plugins/obs-face-tracker.so: undefined symbol: dgesvd_

Environment

OS: CentOS8

Solution

Add openblas to target_link_libraries(${CMAKE_PROJECT_NAME} ...).

Other comments

It would be better performance to enable openblas for Windows and MacOS build.
I don't know about Ubuntu behavior. Maybe it's better to add.

This is a feature request to have a preset of parameters.

Motivation

• Some parameters are difficult to adjust
• There is a need to change parameters to make a dynamic move

Consideration

• Maybe the preset can change track_x/y/z, scale_max, and PID control parameters at once. Or, should I separate PID?
• The preset can be added by user as well.
• Assigning to hotkeys would be useful.

Having these states and switch between them so that the behavior is closer to real camera operator

• PID control state; same as current tracking
  • If the move of the subject becomes slower, transit to Slow down state.
• Slow down state; gradually reduce gain of PID control
  • If the move of the subject becomes stable, transit to Freeze state.
  • If the move of the subject becomes faster, transit back to PID control state.
• Freeze state; don't move camera at all
  • If the subject goes to out of frame or the moving speed exceeds a threshold, transit back to PID control state.

When zooming from the most wide shot, it start to zoom toward the center at first, then gradually move to the face.
Expected behavior is zoom toward the face from the beginning.

I hope the face will be centered in the screen, even including the size after cropping.
Amazing OBS Plugin! Thank you for your development!

Conditions

• Duplicate Sources and Duplicate Scene are both enabled.
• Face Tracker is added to a Group, which contains a camera or video source.

Since the internal state of Face Tracker is not copied, Face Tracker start from the initial location.
I found this issue when looking into #14 but this is a different issue originally reported.

• Add face landmark detection.
• Put textures on the face optionally. And not render the original texture optionally.
• If this is accurate and fast, it can replace correlation-tracker and will detect lost face earlier.
  • No, face landmark detection requires the bounding box accurately.

Hi. I'm just a fan of obs-face-tracker.
Here is the most likely branch of OBS for Apple silicon and obs-face-tracker doesn't work because of architecture maybe.
obsproject/obs-studio#5155

I hope it work with this project someday.

The idea comes from a discussion on the forum.
https://obsproject.com/forum/threads/face-tracker.144419/post-543304

3. A way to adjust the video levels (gain, brightness, contrast) of the frame sent to the detector algorithm. This way you can lower the chance of a face being detected in dark grainy backgrounds.

May not be supported but I'm running Monterey on a MBP M1. Everything looks good as far as installation goes but the plugin doesn't show up in the filters list for my camera.

Hi,

Great plugin detecting the face, it would be nice if rather than a zoomed out and digital zoom to the area of interest that the software would control a PTZ Camera to track and optimize the image resolution.

That is, the PTZ control would move the camera left/right/up/down and zoom in or out to help maximize the persons face.

Certainly it would be nice to extend this to bodies as people might not be facing the camera and if there are multiple bodies to include as many as possible.

Thanks for the work on this!

If the face is lost, it should gradually zoom-out or go back to the home or the preset position.
Depends #52.

THX , great plugin! & sorry for abusing issue tracking for feature requests:

I am building a multicam virtual studio in OBS. After some tweaking the tracker works quite well on 4K
live input, yet - as expected - I run into problems as soon as there are multiple faces in the picture, and it uses about 50% CPU.
I wish I could run the tracker on separate layer with just the raw live input - possibly down scaled to save CPU -
then apply translation on the group that contains my complete scene the way the "Move Source" plugin that comes with
move transition works: https://obsproject.com/forum/resources/move-transition.913/
We would need X/Y scale and offset to make this work I on scenes, where live input is not same size as the scene itself.

This might be useful for additional smoothing:
https://cristal.univ-lille.fr/~casiez/1euro/

• Get the response of inquiry commands for current zoom, pan and tilt.

Switch scene by these triggers.

• A face is detected first
• A previously detected face is lost (and it takes several seconds but still the face is not found.)
  See also #53

Not just detecting faces, but also recognize faces.
It would be useful when there are a lot of people on the scene but want to track just main person.

This feature was requested by several users such as
https://obsproject.com/forum/threads/face-tracker.144419/post-541643

Hi,

Just a quick info: Apparently the obs-face-tracker-0.5.4-Windows.zip file is damaged. I can't unzip it on my Win10 computer.

Hi @norihiro ,

Thanks for the great plugin!
Played around with it for the first time today and it was a lot of fun and we will use it a lot in the future.

What I couldn't find is a basic explanation what each attribute or setting actually does?
A couple are explained in the wiki but not all of them.

The best way would be little hover infos next to the settings, but of course a basic web page would do the trick too.

Thanks again for your work!

Hello I have installed this plugin on both of my PCs but it will not show up in the filters list, Is there something I am missing?

Error log shows noting is wrong.

Windows 10

Latest OBS

I should suppress logs that is shown for each iteration

System: Win10x64 1809
OBS: 27.1.3
Face Tracker: 0.5.2

To avoid OBS Studio from potential crashing, main algorithm should be separated from the OBS Studio.
Drawback will be slow data transfer if using pipe, complicated implementation if using shared memory.

If my webcam is on its side, with the capture rotated 90°, the Face Tracker no longer works. Is it possible to make the tracker function if the webcam is rotated?

I'd like to try and compare with another face detection like using OpenCV.

Thanks for the author's update, I followed your updated compilation guide, generated the sln project under win10 64 bit, compile it, and encountered some errors. Please check it out!

Getting the following error with selfcompiled obs-face-tracker & obs-studio installed via flatpak

13:50:39.080: os_dlopen(/home/user/.var/app/com.obsproject.Studio/config/obs-studio/plugins/obs-face-tracker/bin/64bit/obs-face-tracker.so->/home/user/.var/app/com.obsproject.Studio/config/obs-studio/plugins/obs-face-tracker/bin/64bit/obs-face-tracker.so): libflexiblas.so.3: Kann die Shared-Object-Datei nicht öffnen: Datei oder Verzeichnis nicht gefunden
13:50:39.080: 
13:50:39.080: Module '/home/user/.var/app/com.obsproject.Studio/config/obs-studio/plugins/obs-face-tracker/bin/64bit/obs-face-tracker.so' not loaded

The face tracker is not working, I just compiled using the same instructions as the azure pipline for linux
(though I am using manjaro, not ubuntu)
The regular face tracking filter does not work, I tried the PTZ version for fun to just see what it does, and when enabling the detection it detects me nicely

I tried both latest main and v0.2.2

I am using a NDI capture source for my webcam

logs for the regular tracker

17:06:23.122: [obs-face-tracker] face_detector_base: starting the thread.
17:06:23.122: [obs-face-tracker] Cannot load '(null)' (face-tracker.effect)

logs for the ptz tracker

17:06:44.669: [obs-face-tracker] face_detector_base: starting the thread.
17:06:44.696: [obs-face-tracker] received frame format=5

I hope this is enough, If you need me to collect more information, please let me know

Is it possible to add swap x y z option, support camera vertical installation, or directly support face recognition after image rotation.

As the main idea, after pressing the RESET button, the KPI value is scaled back to its original size.
The idea is that if the face is not tracked, or if there is a mistake, it can be elegantly returned to the original size, instead of directly switching to the original size.

I think it might be possible to make a checkbox option so that "directly RESET" and "elegant RESET" can coexist.

My English is not very good, so if there is anything I can't understand, please talk to me.

Thanks to the developer for creating such a great plug-in!

Even if the face is lost from the frame, the tracker is still remaining for a long time.
It should be cleaned up in a reasonable time.

#55 could be one of the solutions.

I'd like to see the ability to use the tracker to apply an image or mask to the video source. The tracking is quite effective with appropriate lighting, and there is a lack of effective, straightforward plugins for OBS to use face coverings that track tightly.

If the resolution is too small to detect a face, the face tracker does not work.
I'd like to add an FAQ entry to describe how to apply face tracker filter for the low resolution sources.

1. Expand the source on a scene
2. Apply the filter to the scene

Screenshot would be helpful to understand how to operate on OBS Studio.

Export detection and/or tracking results so that a user can control something through Lua script.

This idea could be an alternative of #57.
It will lso resolve this use case: https://obsproject.com/forum/threads/face-tracker.144419/post-541686

Can you provide a smoother description of the win 10 compilation process? It seems a bit tricky. Thank you.

In recent update, the number of the tracker threads keeps increasing. I should check it with bisect.

Track object instead of automatically detected face.

• Implement dock widget to choose the object to track
  • Also provide face-detection range for the other new-feature proposal
• Provide choice of face-detection; dlib detection, manual detection

Related discussion

https://obsproject.com/forum/threads/face-tracker.144419/post-548452

These code should be configurable from a property so that user can adjust the trade-off between speed and accuracy.
Especially those who use HD or 4K, the scale factor should be larger to reduce CPU usage.

There may be some problem when showing or hiding switch in the group. The Preview screen will cause not to work. However, When restarting the OBS 27.01, The Program screen won't work in stead.

It just zooms in on where it last tracked my face and doesn't track when I come back. "Reset Tracking" doesn't work, the only solution is to delete filter and add it again, which resets the preferences set.

The idea comes from a discussion on the forum.
https://obsproject.com/forum/threads/face-tracker.144419/post-543304

2. Some sort if setting that when several faces are detected, more weight is given to the last face detected, and/or more weight is given to the face closest to the center of the frame, and/or if another face is detected, it won't be selected if it's more than a settable distance from the current face. These will keep the tracking from bouncing from one face to the next.

Hello there!

I am planning to use this plugin to convert the 16:9 live video to 9:16 for streaming to Instagram. (I am also streaming at 16:9 on another computer to other platforms.) So it would be great to add a 9:16 aspect ratio to the existing list of the final video dimensions. Now the closest aspect ratio to portrait is 4:3. It would be also great to add a possibility not to zoom at all (now the minium zoom is 0,10). With these 2 little improvements, I guess this plugin could serve very well on purposes when wanting to track a face in landscape video to convert to portrait video.

Thank you very much for developing this plugin!
Jeremias

It gives the option to create a facetracker ptz under the audio tab in filters but when ever you click on it nothing happens.

The idea comes from a discussion on the forum.
https://obsproject.com/forum/threads/face-tracker.144419/post-543304

1. Keep the face detector from looking in certain areas of the source frame by defining this area using left, right, top and bottom sliders to "crop" out the areas. When you check the "Stop tracking faces" box, you can see red crop lines, and they move as you adjust the sliders.

OBS 27.01 installed on Mint 20.01 (Mate)

deb install puts .so in: /usr/lib/x86_64-linux-gnu/obs-plugins/obs-face-tracker.so
Needed to add link to the rest of the plugins for it to appear in OBS.

sudo ln -s /usr/lib/x86_64-linux-gnu/obs-plugins/obs-face-tracker.so /usr/lib/obs-plugins/