airlegend / aitrack Goto Github PK
View Code? Open in Web Editor NEW6DoF Head tracking software
License: MIT License
6DoF Head tracking software
License: MIT License
Describe the bug
Two heads in the FOV of the camera and the tracking become erractic
To Reproduce
Just add another person in front of the camera.
Sometimes you see the camera passing from one face to the other, so my plane crashed :-)
Expected behavior
Just the more "centered" face should be tracked (?)
After Executing AITrack.exe the application crashes within 3 Seconds. I cant see a GUI.
Windows 10, generic HD Camera (60Hz)
Already tried:
log.txt contains:
[2020-09-25 21:06:39.323] [aitrack] [info] ---------- AITRACK LOG ----------
[2020-09-25 21:06:39.797] [aitrack] [info] Created/Found prefs.ini
[2020-09-25 21:06:39.797] [aitrack] [info] Searching for cameras...
To get the best performance with the logitech c920 (and any webcam) it reduces lag greatly if you can switch the camera from 30fps to 60fps. To do this with the Logitech c920 the only way is to install Logitech Capture and change the camera to 360p and 60fps (see screenshot).
You can then close Logitech Capture and the settings persist.
Describe the bug
I'm trying to open it in windows 10, but receiving the error bellow
To Reproduce
Steps to reproduce the behavior:
Environment (please complete the following information):
errore di sistema Su windows 10
"l esecuzione del codice non puo proseguire perche MSVCP140.dll non e stato trovato. Per risolvere il problema prova a reinstallare il programma" seguono altri 3 errori con la stessa dicitura ma diversi DLL
VCOMP140.DLL
VCRUNTIME140_1.dll
VCRUNTIME140.dll
Describe the bug
Crashes when I press "Start". I have tried this on 0.6, 0.5.1 and 0.5. I have to re download the app every time I use it.
To Reproduce
Steps to reproduce the behavior:
It does the same thing when I run as Admin
Expected behavior
Ability to use the app, close it and use it again with no issues of crashing.
Screenshots
If applicable, add screenshots to help explain your problem.
Environment (please complete the following information):
Additional context
Add any other context about the problem here.
Describe your idea
Currently, you need to keep a watch on latest releases and download and installation of AITrack is manual. It would be great if AITrack is added to Scoop.
Describe the bug
Crash when starting tracking
To Reproduce
Steps to reproduce the behavior:
Expected behavior
App tracks properly without crashing
Environment (please complete the following information):
Additional context
Using with OpenTrack 2.3.12
Right now, the default resolution for a generic camera is set to 640x480. Thanks to @rbroker, the ability to change the resolution via prefs.ini
is possible. However, an automatic approach is needed.
A system to check all the available camera resolutions should be implemented and called inside CameraFactory.cpp (https://github.com/AIRLegend/aitrack/blob/master/Client/src/camera/CameraFactory.cpp) in order to create an OCVCamera (https://github.com/AIRLegend/aitrack/blob/master/Client/src/camera/OCVCamera.h) with the lowest resolution compatible.
Describe the bug
Crash after launch the app
To Reproduce
Steps to reproduce the behavior:
Expected behavior
Ability to use the app, close it and use it again with no issues of crashing.
Screenshots
1s after launch can't screen
Environment (please complete the following information):
Additional context
It does the same thing when I run as Admin
Hi,
In FS2020, the image goes upside down when i'm using the yaw to the left or right. after a certain degree (close to end of the range) the image goes upside down
Hi,
great little tool...
It would be great to have some startup options like Minimize to Tray and Start Tracking on startup. An option for Autostart with Windows would be nice as well, but this could be solved with the WIndows Autostart folder.
And, is there a way to donate to this project?
Best regards
Oliver
Would you like to wrap any pointer data members with the class template โstd::unique_ptrโ?
On loading, it looks for camera index "selected_camera=" from prefs.ini. If that camera doesn't exist it just exits with no message.
To Reproduce
Steps to reproduce the behavior:
1- plug in two webcams.
B. Run aitrack and select the second camera.
3. Exit out to desktop.
4. Unplug the second camera.
E. Launch again, observe crash.
Expected behavior
Should gracefully handle a missing camera. If the selected one is not found, do something like reset index to 0 and use a messagebox to inform user of the situation.
Environment (please complete the following information):
It's picking up the cam on my Vive instead of webcam. Any to change this please without unplugging the Vive? Thanks
Describe the bug
Large yaw reading found when moving head without turning.
Camera mounted at top of the monitor.
When I move my head to left front, but keeping eyes looking towards the monitor (head not turning at all), the AI tracker would report large yaw reading.
Start:
--------C-------
..............................
..............................
..............H.............
End:
--------C-------
...H........................
..............................
To Reproduce
Steps to reproduce the behavior:
Expected behavior
Should expect Z+ and X- readings, without yaw.
Screenshots
Environment (please complete the following information):
Additional context
Root cause seems like to be that, camera is computing the head's angle related to the camera?
Errors is more prominent when head is closer to camera.
Maybe related to #42 ?
Trying 0.6.3 alpha, FOV value is not saved by the apply button.
Manually editing prefs.ini seems to work.
Aymeric
Describe the bug
When I try to use my PS3 eye camera, it either crashes instantly, or works for a while and then crashes.
To Reproduce
Steps to reproduce the behavior:
Environment (please complete the following information):
Additional context
Logs :
[2020-09-26 01:16:07.578] [aitrack] [info] ---------- AITRACK LOG ----------
[2020-09-26 01:16:07.974] [aitrack] [info] Created/Found prefs.ini
[2020-09-26 01:16:07.974] [aitrack] [info] Searching for cameras...
[2020-09-26 01:16:08.722] [aitrack] [info] Number of recognized cameras: 2
[2020-09-26 01:16:08.725] [aitrack] [info] UDP sender reinitialized. IP: 127.0.0.1 PORT: 4242
[2020-09-26 01:16:08.725] [aitrack] [info] Building Tracker with selected camera: 0
[2020-09-26 01:16:08.783] [aitrack] [info] Tracker initialized.
[2020-09-26 01:16:08.783] [aitrack] [info] Updated stabilizer.
[2020-09-26 01:16:08.783] [aitrack] [info] Updating camera parameters...
[2020-09-26 01:16:08.783] [aitrack] [info] Updated camera parameters. 640x480@60
[2020-09-26 01:16:08.783] [aitrack] [info] Checking for updates
[2020-09-26 01:16:08.808] [aitrack] [info] App initialized
[2020-09-26 01:16:10.180] [aitrack] [info] Starting camera 0 capture
[2020-09-26 01:16:10.193] [aitrack] [info] Camera 0 started capturing
When I open AiTrack it gives me a message saying no cameras detected.
I have verified my webcam is working with both CL Eye Test and Open Track point tracker.
After that message it also displays a message saying there was a problem initializing the tracker. Check the model folders and restart the program.
Any ideas what I might be doing wrong? I have opentrack configured based on the examples provided.
I've tried using both 0.4 and 0.5 and no matter what, every time I try and launch them the window (window only, no actual UI) shows up for a couple of seconds and crashes. I have a couple of error logs from Windows if that helps
Faulting module name: ntdll.dll, version: 10.0.18362.815, time stamp: 0xb29ecf52
Exception code: 0xc0000005
Fault offset: 0x000000000001b38d
Faulting process ID: 0x1520
Faulting application start time: 0x01d67a39d4603851
Faulting application path: C:\Users\knigh\Downloads\aitracker-v0.4\aitracker-v0.4\AITrack.exe
Faulting module path: C:\Windows\SYSTEM32\ntdll.dll
Report ID: 884315b6-c4cb-4221-a187-0c61f7e23e5f
Faulting package full name:
Faulting package-relative application ID:
Event Name: APPCRASH
Response: Not available
Cab Id: 0
Problem signature:
P1: AITrack.exe
P2: 0.0.0.0
P3: 5f36c416
P4: ntdll.dll
P5: 10.0.18362.815
P6: b29ecf52
P7: c0000005
P8: 000000000001b38d
P9:
P10:
Attached files:
\\?\C:\ProgramData\Microsoft\Windows\WER\Temp\WERFF09.tmp.dmp
\\?\C:\ProgramData\Microsoft\Windows\WER\Temp\WERFFB6.tmp.WERInternalMetadata.xml
\\?\C:\ProgramData\Microsoft\Windows\WER\Temp\WERFFD6.tmp.xml
\\?\C:\ProgramData\Microsoft\Windows\WER\Temp\WERFFF0.tmp.csv
\\?\C:\ProgramData\Microsoft\Windows\WER\Temp\WER20.tmp.txt
These files may be available here:
\\?\C:\ProgramData\Microsoft\Windows\WER\ReportArchive\AppCrash_AITrack.exe_fee5552e8d284e2019772265c7d020aba0cc2fb_80129c8b_03dc7bda-dbe4-4cf1-b5bd-058f9f7ace09
Analysis symbol:
Rechecking for solution: 0
Report Id: 884315b6-c4cb-4221-a187-0c61f7e23e5f
Report Status: 268435456
Hashed bucket: 048d30332d77101fab2fdbeb38f44338
Cab Guid: 0
Describe your idea
I'm always frustrated when moving in Z direction, or yaw left/right. Since the camera is mounted at the top of the screen, there's an offset: if I'm moving directly to the screen, instead of a +Z movement, the camera would detect a -Y.(very strong when moving close the the camera). This totally breaks the immersion.
Same when I look left/right. Since the camera is looking down at me, the left/right look would always induce a pitch movement. Instead of looking left/right in game, it would look left+down, or right+down.
In short: the current implementation reports the 6DOF movement relative to the CAMERA, not the MONITOR.(where our focus is at)
Describe the solution you'd like
Theoretically, we should be able to easily compensate for the camera's offset from the screen center, and calculate the 6DOF data relative to the screen center.
We may ask users to input their horizontal/vertical offset to the camera.
Currently we assume/require users to put camera at nose height. However that's really hard/impossible: there's the monitor at that place.
Describe alternatives you've considered
It's almost impossible to put camera at the center of the screen, so no very good way.
Additional context
I did tried other algorithms/apps for face tracking and some did pretty well on this. I simply my my head towards the screen, and the virtual camera would pan forward. Although I'm not moving towards the tracking camera.
Describe your idea
IR cameras offer a better precision and resilience to environmental light when it comes to tracking facial features. I see them being mentioned in the main readme file, however I'm not sure if they are supported or how to activate them
I have a Logitech Brio but it only works with the normal camera
Would it be possible to support them?
(First of all, a big thank you for your software !)
Describe the bug
I noticed poor tracking even after understanding the Wiki.
An i think i found a clue = it depends on the framerate (?!) => High framerates implies poor brightness and so poor tracking (?)
To Reproduce
Steps to reproduce the behavior:
(see the screenshots below)
Expected behavior
The same brightness as the CL-Eye Test to allow a good tracking
Screenshots
Screenshots which show clearly the issue
Environment :
Sorry for question but I have limited knowledge. My camera contrast is very bright and I'm thinking I may need the PS3 experimental driver mentioned. How do I get that and have it installed? Right now I'm using CL-Eye Test driver.
Thanks for all your hard work. I love the app!
Describe your idea
The app doesn't work too great with a headset on (Phone VR with Riftcat using Freetrack tracking)
I suppose it looks for the eyes which cannot be seen with a headset on.
Describe the solution you'd like
Rely just on head shape/mouth tracking, or add the ability for manual tracking points
Describe alternatives you've considered
I could stick some printed eyes on the headset, but that's creepy ๐คฃ
Additional context
Thanks!
Checkbox is also checked after unchecking if the user presses 'apply'
Can you add a option to show/display the output values (Y/P/R/X/Y/Z) at main window, maybe just bellow the preview video stream?
Describe the bug
When changing resolution to match the camera, the preview is not scaled, causing only a portion of the image to display. Tracking appears to still work.
To Reproduce
Steps to reproduce the behavior:
Expected behavior
Preview should show full frame at new resolution
Screenshots
Environment (please complete the following information):
Additional context
Using with OpenTrack 2.3.12
Apologies for putting this in the issue/feature tracker. I would have made the update myself but I didn't see a link where I could edit the wiki on the page.
For the common camera FOV settings page, the Logitech c270 has a FOV of 60.
It's not a great camera by any means, but it's worked well for me at 640x480 with aitrack. Good budget option if anyone wants to get started with aitrack.
Thanks for the amazing app.
I can't put the camera completely in front of me and it's a little above me. (About 10 degrees)
In this case, if I turn left and right, the output in the pitch direction will be added against my intention.
Would it be possible to have a choice which PS3 driver is used (CL eye already installed or your internal PS3 driver)?
I'm finding the driver you force is bad in low light even with exposure/gain raised a lot, and the landmarks even with stabilization are really twitchy.
Attack Surface Reduction rule c1db55ab-c21a-4637-bb3f-a12568109d35 (Use advanced protection against ransomware) is blocking AITrack.exe from launching.
Microsoft Defender Exploit Guard has blocked an operation that is not allowed by your IT administrator.
For more information please contact your IT administrator.
ID: C1DB55AB-C21A-4637-BB3F-A12568109D35
Detection time: 2020-08-12T16:49:31.248Z
User: ******\********
Path: C:\Users\*******\Downloads\aitracker-v0.3\AITrack.exe
Process Name: C:\Windows\explorer.exe
Security intelligence Version: 1.321.1239.0
Engine Version: 1.1.17300.4
Product Version: 4.18.2007.8
Windows 10 Pro for Workstations v2004, 19041.388.
Describe the bug
Video capture broken - seems as if multiple streams are combined horizontally and fed into tracking
To Reproduce
Steps to reproduce the behavior:
Open webcam at native resolution and frame rate - get three video streams combined horizontally which are fed into the tracker.
Expected behavior
One video stream at the set dimension and frame rate
Environment (please complete the following information):
Thank you very much for this awesome program! :-)
@AIRLegend
Thanks for your outstanding contribution of facial head tracker for gaming.
I am also developing an open-source non-commercial facial head tracker FlightAgentX which combines landmark and end-to-end methods for head pose estimation with Kalman filter. I found your project today at Bilibili, which performs excellent and your pre-train model from @emilianavt is much better than the one provided by dlib. So I would like to integrate the model to replace the dlib model. I used pieces of your code of few functions including proc_heatmaps, normalize and transpose and will include your LICENSE and a link point to your repo in my own repo in next few commits.
I am writing this just to let you know.
Describe your idea
I'd to be able to run OpenTrack remotely on my laptop that has a webcam. Then I'd like AITrack to be able to hook into the OpenTrack feed.
Describe the solution you'd like
It looks like network capabilities are already built into both (you have to specify an IP address and port), but I'm not sure if there are any other limiting factors.
Additional context
Currently when I open AITrack on my desktop it will error out saying that there's no webcam connected. And the options are greyed out to input my laptop's IP address (can't even tick the checkbox).
I already tried disabling webroot my antivirus and I also tried without windows defender, I am using this cam off of amazon (https://www.amazon.com/gp/product/B08FMCZCZY?pf_rd_r=3ST4YWE0NNZ3V4BFTGGH&pf_rd_p=edaba0ee-c2fe-4124-9f5d-b31d6b1bfbee&th=1) and the log is below
[2020-10-05 20:28:33.939] [aitrack] [info] ---------- AITRACK LOG ----------
[2020-10-05 20:28:34.167] [aitrack] [info] Created/Found prefs.ini
[2020-10-05 20:28:34.167] [aitrack] [info] Searching for cameras..
Latest Opentrack is running using Aruco and my camera works (and I see myself).
I start AITrack and it says No Video Input.
I click Start Tracking and get a black screen.
Using 0.5.1 (You really should put the ver number in the window title).
Is there a way to change the input source (camera)?
I have two camera's (the default camera and an external one), but I don't see a way to select the second camera in AITracker.
Roll value quickly swaps from 0 to 180. It happens more often when I'm looking straight forward.
To Reproduce
I'm not sure if it will reproduce on anyone else's setup. Linked is a video showing the result at the beginning and then all of the data glitching.
Expected behavior
Quick/huge changes should be ignored and not considered valid input/output.
Screenshots
https://youtu.be/mDlfIRWKJgQ
Environment (please complete the following information):
Additional context
Output Roll is also centered at 90. AFAIK it should be centered at 0.
(Also huge thanks for the software so far, it's awesome!)
Please add possibility to select camera Resolution and FPS for tracking
Describe the bug
A clear and concise description of what the bug is.
To Reproduce
Steps to reproduce the behavior:
Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
Environment (please complete the following information):
Additional context
Add any other context about the problem here.
Right now the inference is done on the CPU with the small-ish models. Latency is dramatically improved in many tasks, even at small batch sizes, by moving the network over to the GPU. Exploration of using https://developer.nvidia.com/tensorrt might reduce input lag and/or allow for more complex models to be used with similar latency.
Describe your idea
When the app crashes it would be useful to get the user's log to have more information about what happened.
Describe the solution you'd like
A lightweight logger should be included and especially called in presenter.cpp
. The log should include information from state
on each change (on initialization, when tracking starts, when tracking ends, on configuration applied...) and should be reset on each program startup.
When I start the app the camera starts and stops. I see light on camera come on and then immediately quit. Flight Sim seems to be connected for that second.
Log data
[2020-09-09 15:26:34.535] [aitrack] [info] ---------- AITRACK LOG ----------
[2020-09-09 15:26:34.831] [aitrack] [info] Created/Found prefs.ini
[2020-09-09 15:26:34.831] [aitrack] [info] Searching for cameras...
[2020-09-09 15:26:35.645] [aitrack] [info] Number of recognized cameras: 1
[2020-09-09 15:26:35.646] [aitrack] [info] UDP sender reinitialized. IP: 127.0.0.1 PORT: 4242
[2020-09-09 15:26:35.646] [aitrack] [info] Building Tracker with selected camera: 0
[2020-09-09 15:26:35.687] [aitrack] [info] Tracker initialized.
[2020-09-09 15:26:35.687] [aitrack] [info] Updated stabilizer.
[2020-09-09 15:26:35.687] [aitrack] [info] Updating camera parameters...
[2020-09-09 15:26:35.687] [aitrack] [info] Updated camera parameters. 640x480@0
[2020-09-09 15:26:35.687] [aitrack] [info] App initialized
[2020-09-09 15:26:38.885] [aitrack] [info] Starting camera 0 capture
[2020-09-09 15:26:39.119] [aitrack] [info] Camera 0 started capturing
Running Win 10 2004 and is up to date with patches
Microsoft Flight Sim 2020
Camera Logitech Quickcam Ultravision
Describe your idea
The app works great for 1x camera, but I want to move around the room or maybe just turn my head around to face another camera.
Describe the solution you'd like
This is obviously a much larger undertaking.
The ability to add in multiple cameras to track the user would potentially allow for full-room, or at least allow the user to turn their head to face another camera.
Each camera would have to have an option to specify where it is in the 3D space (offset x,y,z from perhaps a main camera?)
It would be cool if this could be calculated/calibrated automatically by tracking the face and its size as it moves around, maybe just then add a zoom factor and the ability to override this (This would also obviously work better if using multiple of the same camera (same zoom, fov)).
All of the values seem to change rapidly at random moments when I keep head steady.
Screenshots
https://www.youtube.com/watch?v=k6IYNwVJtt8
https://www.youtube.com/watch?v=jvwk6xHQHmM
Environment (please complete the following information):
Additional context
Seems to me like an autofocus problem, but from what I see the C270 doesn't have it.
Hello, first of all awesome program. It works flawlessly considering the limitations implied by the cameras used.
I have an issue with the integrated Ps3 driver your Aitrack is using, i think: I see that the exposure and gain seems fixed and not configurable, this causes the camera to be inoperable in dark environments.
Comparing the driver with the CL-Eye camera driver I see that in the CL-Eye Test app the camera is much more sensitive to light, allowing it to see my face in much darker environments, despite of obviously increased noise value, but this should not pose a problem to your tech. It is possible to bypass the integrated driver, or make the exposure and gain configurable by ini file in any way?
You should really open a donation something because you really deserve it.
Thanks.
Best regards
Silvio
Whenever you close AITracker, it stays open in the background. The only solution is by pressing 'Stop Tracking' rather than closing the application.
SOLUTION: Automatically stop tracking when the user exits the program.
Describe the bug
Application does not respond / crashed, in to 2 cases :
To Reproduce
Aitrack is crashing at start one time over 2
When successfull started, it is crashing everytime when launching the tracking
In any cases, application is never working as expected.
Screenshots
2 logs available :
Crash at start :
[2020-10-03 12:41:07.032] [aitrack] [info] ---------- AITRACK LOG ----------
[2020-10-03 12:41:07.265] [aitrack] [info] Created/Found prefs.ini
[2020-10-03 12:41:07.265] [aitrack] [info] Searching for cameras...
Crash at tracking start :
[2020-10-03 12:45:21.465] [aitrack] [info] ---------- AITRACK LOG ----------
[2020-10-03 12:45:21.714] [aitrack] [info] Created/Found prefs.ini
[2020-10-03 12:45:21.714] [aitrack] [info] Searching for cameras...
[2020-10-03 12:45:23.378] [aitrack] [info] Number of recognized cameras: 1
[2020-10-03 12:45:23.379] [aitrack] [info] UDP sender reinitialized. IP: 127.0.0.1 PORT: 4242
[2020-10-03 12:45:23.379] [aitrack] [info] Building Tracker with selected camera: 0
[2020-10-03 12:45:23.420] [aitrack] [info] Tracker initialized.
[2020-10-03 12:45:23.420] [aitrack] [info] Updated stabilizer.
[2020-10-03 12:45:23.420] [aitrack] [info] Updating camera parameters...
[2020-10-03 12:45:23.420] [aitrack] [info] Updated camera parameters. 1280x720@30
[2020-10-03 12:45:23.420] [aitrack] [info] Checking for updates
[2020-10-03 12:45:23.457] [aitrack] [info] App initialized
[2020-10-03 12:45:29.839] [aitrack] [info] Starting camera 0 capture
Environment
Additional context
Resolution method tried :
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.