Git Product home page Git Product logo

ms-teams-background-changer's Introduction

MS Teams background changer

Microsoft Teams on linux operating system does not have background effects like background blur or custom background image. This utility application enables these missing features to blur and add custom background image. It works on other video conference platform as well like Zoom, Google Meet and Skype.

We have used AI based image segmentation model DeepLab-V3 to segment person as foreground and the rest of the objects as a background and apply filters to it.

architecture

Installation

Install dependencies

sudo apt-get install v4l2loopback-dkms v4l-utils

v4l2loopback module is used to create virtual video devices. Therefore, all v4l2 applications will read the newly created virtual video device as a physical video device. This allows us to push modified video stream to the virtual camera and use it as a normal video feed coming from a physical camera device.

Install requirements

  • Python 3.x

Install python dependencies using

pip3 install -r requirements.txt

Usage

1.Create Virtual Camera

sudo modprobe v4l2loopback video_nr=20 card_name="Virtual Camera 1"

This will create a virtual camera with a device ID 20 and with a name Virtual Camera 1. If it's taken, change to different ID. Recommended to give higher number therefore there won't be a clash with other devices.

If you get this error

modprobe: ERROR: could not insert 'v4l2loopback': Operation not permitted

Try disablling "Secure Boot" option in BIOS settings. And if still doesn't work try installing v4l2loopback from source. Refer here for step by step process.

You can verify that by running

ls -1 /sys/devices/virtual/video4linux

and this should output something like video20.

2.Verify Virtual camera

Debug the list of physical and virtual camera using the command

v4l2-ctl --list-devices

And that will pop out information about all the integrated, connected(usb) and virtual camera along will associated device ID currently connected to the PC.

Virtual Camera 1 (platform:v4l2loopback-000):
	/dev/video20

Lenovo FHD Webcam Audio: Lenovo (usb-0000:00:14.0-4):
	/dev/video2
	/dev/video3

Integrated Camera: Integrated C (usb-0000:00:14.0-7):
	/dev/video0
	/dev/video1

3.Update Configurations

Edit configuration data in config.yaml file.

[V4L2]
# add virtual camera device ID
virtualDeviceID = /dev/video20 

Edit the input cameraID from which the realtime video stream should be fetched.

[stream]
# change input camera device ID 
CameraID = /dev/video2

edit the background settings in the configuration file.

[background]
# if blur = yes, only background will be blurred, background image is not added
# if blur = no, custom background image is added, blur effect is not applied
# blur_intensity should always be an odd number
blur = yes
blur_intensity = 11
image = images/workspace.jpg

4.Run application

Now we are ready to run the application,

python3 main.py

This should start to read the video feed from the input video camera, apply effects and push it to the Virtual camera. You can press ctrl+c to stop the application.

ms-teams-background-changer's People

Contributors

prabhakar-sivanesan avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

ms-teams-background-changer's Issues

can't open camera by index

2022-10-13 11:22:52.632939: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-10-13 11:22:52.831699: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /home/perrella/vscodecamera/MS-Teams-background-changer/venv/lib/python3.10/site-packages/cv2/../../lib64:/opt/oracle/instantclient_21_4
2022-10-13 11:22:52.831731: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
2022-10-13 11:22:52.869068: E tensorflow/stream_executor/cuda/cuda_blas.cc:2981] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2022-10-13 11:22:53.715209: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /home/perrella/vscodecamera/MS-Teams-background-changer/venv/lib/python3.10/site-packages/cv2/../../lib64:/opt/oracle/instantclient_21_4
2022-10-13 11:22:53.715306: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /home/perrella/vscodecamera/MS-Teams-background-changer/venv/lib/python3.10/site-packages/cv2/../../lib64:/opt/oracle/instantclient_21_4
2022-10-13 11:22:53.715317: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
set format result (0 is good):0
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
[ WARN:[email protected]] global /io/opencv/modules/videoio/src/cap_v4l.cpp (889) open VIDEOIO(V4L2:/dev/video2): can't open camera by index

Error starting virtual camera

When running:

sudo modprobe v4l2loopback video_nr=20 card_name="Virtual Camera 1" exclusive_caps=1

I'm getting:
modprobe: ERROR: could not insert 'v4l2loopback': Operation not permitted

Background overflows with face

When using this background changer (installed flawlessly on pop OS 22.04) I am getting a weird overflow of the background on my face. Is there anything I can tweak to make it less overflowing?

image

Unable to run - Out of range

Hello, I was trying to run this and I am unable to get it to run. I followed the instructions and I had an initial no cv2 module. So I installed cv2-python through pip3. Then I got no v4l2 module and installed that through pip3 also. Now I am getting the following error:

sudo python3 main.py         
Traceback (most recent call last):
  File "/opt/MS-Teams-background-changer/main.py", line 11, in <module>
    import v4l2
  File "/usr/local/lib/python3.10/dist-packages/v4l2.py", line 197, in <module>
    ) = range(1, 9) + [0x80]
TypeError: unsupported operand type(s) for +: 'range' and 'list'

Dummy camera not listed in MS Teams under Linux

Unfortunately there's not much more to report. The dummy camera simply doesn't show in MS Teams.
If I try to view the output through guvcview it does list the camera, but can't show a stream.
The source webcam is active after running main.py, and there are no console errors...
Are there any steps I can try to debug this or provide further information?

Error on running

Hello. Thanks for your help. Im running fedora 36 and first time script run ok and second time is showing this error. Any ideas?

2022-08-18 13:07:56.926703: I tensorflow/core/util/util.cc:169] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable TF_ENABLE_ONEDNN_OPTS=0.
2022-08-18 13:07:56.929525: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /home/federicobuffa/.local/lib/python3.10/site-packages/cv2/../../lib64:
2022-08-18 13:07:56.929544: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
Traceback (most recent call last):
File "/home/federicobuffa/MS-Teams-background-changer/main.py", line 62, in
print ("set format result (0 is good):{}".format(fcntl.ioctl(device, v4l2.VIDIOC_S_FMT, format)))
OSError: [Errno 22] Invalid argument

Glitched video output when using blur - crashes when using bg_image

When using the blur option, the output is, while kind of cool in its own way, but obviously not what I'm after.

glitch

The console looks normal with Processing frame: being output.

When attempting to use a bg_image however, it simply crashes and refuses to process any frames.
I've attached the output here.

output.txt

Any clues?
Cheers :)

PS - I also followed this guide to get the GPU recognised by TensorFlow

Physical camera not displaying video

OS
Ubuntu 20.04.4 LTS 64-bit

Graphics
Mesa Intel® Xe Graphics (TGL GT2)

Steps followed / commands executed
I followed steps from readme. Runnning v4l2-ctl --list-devices shows:

$ v4l2-ctl --list-devices
Dummy video device (0x0000) (platform:v4l2loopback-000):
	/dev/video20

Integrated Camera: Integrated C (usb-0000:00:14.0-8):
	/dev/video0
	/dev/video1

I updated the cameraID setting in ./config.yaml to just use the number as so:

# change input camera device ID (just add the ID number alone)
cameraID = 0

Running python3 main.py shows this output (with the Processing frame logs continuing until I stop the script):

2022-07-08 08:38:07.014005: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /home/manny/.local/lib/python3.8/site-packages/cv2/../../lib64:/usr/local/lib
2022-07-08 08:38:07.014018: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
set format result (0 is good):0
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
2022-07-08 08:38:08.138452: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /home/manny/.local/lib/python3.8/site-packages/cv2/../../lib64:/usr/local/lib
2022-07-08 08:38:08.138474: W tensorflow/stream_executor/cuda/cuda_driver.cc:269] failed call to cuInit: UNKNOWN ERROR (303)
2022-07-08 08:38:08.138486: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (manny-ThinkPad-E15-Gen-2): /proc/driver/nvidia/version does not exist
2022-07-08 08:38:08.138705: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F AVX512_VNNI FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
Processing frame: 0 at 21 fps. Press ctrl+q to stop
Processing frame: 1 at 33 fps. Press ctrl+q to stop
Processing frame: 2 at 32 fps. Press ctrl+q to stop
Processing frame: 3 at 30 fps. Press ctrl+q to stop
...

I then fire up a camera app (e.g., MS Teams) but the camera only displays a black image, no video and no virtual or blurred background.

teams_virtual_background_sad

If I have the physical camera in use before running the python3 main.py script, then I get the error:

[ WARN:[email protected]] global /io/opencv/modules/videoio/src/cap_v4l.cpp (889) open VIDEOIO(V4L2:/dev/video0): can't open camera by index

So I assume it is correct to run the script prior to starting the physical camera.

Am I missing a step or anything?

Edit: After doing some more digging, I realize perhaps this is due to my graphics card not being supported sufficiently for running TensorFlow? If that's the case, feel free to close this issue.

`libnvinfer.so.7` nout found

Hi everyone!

I'm on Pop!_02 22.04 (up to date), and while trying this, managed to have all ok, except that it fails at startup like this:

…/MS-Teams-background-changer on  main [!?] ❯ python3 main.py 
2023-01-09 18:51:24.021859: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-01-09 18:51:24.544095: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /home/dxps/.local/lib/python3.10/site-packages/cv2/../../lib64:
2023-01-09 18:51:24.544154: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /home/dxps/.local/lib/python3.10/site-packages/cv2/../../lib64:
2023-01-09 18:51:24.544162: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
Traceback (most recent call last):
  File "/tmp/MS-Teams-background-changer/main.py", line 36, in <module>
    background = config['background']['bg_image']
  File "/usr/lib/python3.10/configparser.py", line 1258, in __getitem__
    raise KeyError(key)
KeyError: 'bg_image'
…/MS-Teams-background-changer on  main [!?] took 1s❯ 

Do I need to install TensorRT

Thanks!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.