ecsplendid / rembg-greenscreen Goto Github PK

View Code? Open in Web Editor NEW

This project forked from danielgatis/rembg

253.0 11.0 30.0 18.44 MB

Rembg Video Virtual Green Screen Edition

License: MIT License

Python 83.16% PowerShell 16.84%

video greenscreen segmentation segmentation-models deeplearning

rembg-greenscreen's Introduction

Rembg Virtual Greenscreen Edition (Dr. Tim Scarfe)

Rembg Virtual Greenscreen Edition is a tool to create a green screen matte for videos

Video Virtual Green Screen Edition

15th Jan 2021 -- made a new YouTube explainer

Take any video file and convert it to an alpha matte to apply for a virtual green screen
It runs end-to-end non-interactively
You need ffmpeg installed and on your path
There is also a powershell script ./remove-bg.ps1 which will do the job in a manual way i.e. first create frames, then run the rembg -p ... command and then run ffmpeg to create the matte movie. This was my first approach to solve this problem but then I migrated onto just making a new version of rembg.

If you have any ideas for speeding this up further, please let us know. We have tried quite a few things at this stage and are a bit stuck on how to proceed from here. See some of the "evolution" in the Whimsical notes.

Usage;

pip install rembg-greenscreen

greenscreen -g "path/video.mp4"

Experimental parallel green screen version;

greenscreen --parallelgreenscreen "path/video.mp4" --workernodes 3 --gpubatchsize 5

The command above will produce a video.matte.mp4 in the same folder, also works with mov and avi extensions. Uses ffmpeg under the hood to stream and re-encode the frames into a grayscale matte video.

Be careful with the default parameters, my 11GB GPU is already pretty much maxed with 3 instances of the NN with 5 image gpu batches in forward pass.

You can see how much free GPU ram you have with

nvidia-smi

CLI interface

Important notes

Don't use VBR videos, it will run forever -- use Handbrake to convert them to CFR

References

https://arxiv.org/pdf/2005.09007.pdf
https://github.com/NathanUA/U-2-Net (thanks to these folks for making the semantic segmentation model and publishing online)

License

Licensed under MIT License

rembg-greenscreen's People

Contributors

Stargazers

Watchers

rembg-greenscreen's Issues

Output resolution does not match input resolution

Is there a reason we are multiplying 320 to the frame size here

rembg-greenscreen/src/rembg/multiprocessing.py

Lines 120 to 125 in b79648a

 '-y', 

 '-f', 'rawvideo', 

 '-vcodec', 'rawvideo', 

 '-s', F"{frame.shape[1]}x320", 

 '-pix_fmt', 'gray', 

 '-r', F"{framerate}",

about ffprobe

hello, I encountered an error while running，could you help me?
(torchenv) zjy@sugon-X795-G30:~/rembg-greenscreen-master$ greenscreen -pg "/public/zjy/rembg-greenscreen-master/vid/1.mp4"
/public/zjy/rembg-greenscreen-master/vid/1.mp4
Traceback (most recent call last):
File "/public/zjy/.local/bin/greenscreen", line 8, in
sys.exit(main())
File "/public/zjy/.local/lib/python3.8/site-packages/rembg/cmd/cli.py", line 77, in main
parallel_greenscreen(os.path.abspath(args.parallelgreenscreen),
File "/public/zjy/.local/lib/python3.8/site-packages/rembg/multiprocessing.py", line 75, in parallel_greenscreen
info = ffmpeg.probe(file_path)
File "/public/zjy/.local/lib/python3.8/site-packages/ffmpeg/_probe.py", line 20, in probe
p = subprocess.Popen(args, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
File "/public/lzh/anaconda3/envs/torchenv/lib/python3.8/subprocess.py", line 854, in init
self._execute_child(args, executable, preexec_fn, close_fds,
File "/public/lzh/anaconda3/envs/torchenv/lib/python3.8/subprocess.py", line 1702, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'ffprobe'

Can't install on Windows 10 + Anaconda - Running = Errors ??

I've created an environment called: "rembg-greenscreen" with python 3.9 and activated it
I tried to install using: pip install rembg-greenscreen

But I got some errors I'm not sure how to fix as I'm not a programmer,

Can someone please explain how to install it correct without errors?

Thanks ahead!

RuntimeError: Cannot re-initialize CUDA in forked subprocess. To use CUDA with multiprocessing, you must use the 'spawn' start method

Is there anyone knows how to set this up please?

about run error

/data/chengt1/project/demo-video/1003656657.mp4
FRAME RATE DETECTED: 30/1 (if this looks wrong, override the frame rate)
FRAME RATE: 30 TOTAL FRAMES: 417
WORKER FRAMERIPPER ONLINE
WORKER 0 ONLINE
WORKER 1 ONLINE
Process Process-3:
Traceback (most recent call last):
File "/data/chengt1/python-3.8.3/lib/python3.8/site-packages/urllib3/connection.py", line 169, in _new_conn
conn = connection.create_connection(
File "/data/chengt1/python-3.8.3/lib/python3.8/site-packages/urllib3/util/connection.py", line 96, in create_connection
raise err
File "/data/chengt1/python-3.8.3/lib/python3.8/site-packages/urllib3/util/connection.py", line 86, in create_connection
sock.connect(sa)
OSError: [Errno 101] Network is unreachable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/data/chengt1/python-3.8.3/lib/python3.8/site-packages/urllib3/connectionpool.py", line 699, in urlopen
httplib_response = self._make_request(
File "/data/chengt1/python-3.8.3/lib/python3.8/site-packages/urllib3/connectionpool.py", line 382, in _make_request
self._validate_conn(conn)
File "/data/chengt1/python-3.8.3/lib/python3.8/site-packages/urllib3/connectionpool.py", line 1010, in _validate_conn
conn.connect()
File "/data/chengt1/python-3.8.3/lib/python3.8/site-packages/urllib3/connection.py", line 353, in connect
conn = self._new_conn()
File "/data/chengt1/python-3.8.3/lib/python3.8/site-packages/urllib3/connection.py", line 181, in _new_conn
raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7f578247ca00>: Failed to establish a new connection: [Errno 101] Network is unreachable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/data/chengt1/python-3.8.3/lib/python3.8/site-packages/requests/adapters.py", line 439, in send
resp = conn.urlopen(
File "/data/chengt1/python-3.8.3/lib/python3.8/site-packages/urllib3/connectionpool.py", line 755, in urlopen
retries = retries.increment(
File "/data/chengt1/python-3.8.3/lib/python3.8/site-packages/urllib3/util/retry.py", line 574, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='docs.google.com', port=443): Max retries exceeded with url: /uc?export=download&id=1ao1ovG1Qtx4b7EoskHXmi2E9rp5CHLcZ (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f578247ca00>: Failed to establish a new connection: [Errno 101] Network is unreachable'))

During handling of the above exception, another exception occurred:

Failed to install scikit-image (macOS Big Sur 11.2.3)

System information

OS macOS Big Sur 11.2.3
Python version: 3.9.1
pip 21.0.1 from /usr/local/lib/python3.9/site-packages/pip (python 3.9)

Hi,

I'm having issues with the install and scikit-image throwing up errors which looks like a macOS specific issue. Using sudo -H and or --user still gives errors. All other requirements are satisfied. Any help resolving the install issue would be appreciated.

ERROR: Command errored out with exit status 1: /usr/local/opt/[email protected]/bin/python3.9 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/4j/z_6bp83j4ld54fhfh5kx789h0000gn/T/pip-install-2_1gjz34/scikit-image_7401d09bd4c442df88801aef1f118ada/setup.py'"'"'; file='"'"'/private/var/folders/4j/z_6bp83j4ld54fhfh5kx789h0000gn/T/pip-install-2_1gjz34/scikit-image_7401d09bd4c442df88801aef1f118ada/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record /private/var/folders/4j/z_6bp83j4ld54fhfh5kx789h0000gn/T/pip-record-b00qt4za/install-record.txt --single-version-externally-managed --compile --install-headers /usr/local/include/python3.9/scikit-image Check the logs for full command output.

Requirement divergence and Issue to run cuda in multiprocessing

System information
- Ubuntu 20.04
- Python version: 3.9.4
- pip 20.3.4
- Nvidia drivers 460.80
- Cuda Version 11.2

Hi,

I'm having issues by install and running rembg-greenscreen as you showwn in the YT Video.
Your requirements contains a requirement of numpy 1.19.4 But greenscreen requires 1.19.5
However, I tried both versions, and I run into the following execution error later:

bubu@desktop:~/share$ greenscreen -pg "WebcamFanny2015.mp4"
/home/munsch/share/WebcamFanny2015.mp4
FRAME RATE DETECTED: 25/1 (if this looks wrong, override the frame rate)
FRAME RATE: 25 TOTAL FRAMES: 428
WORKER FRAMERIPPER ONLINE
WORKER 0 ONLINE
Process Process-3:
Traceback (most recent call last):
File "/usr/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/usr/lib/python3.9/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/home/munsch/.local/lib/python3.9/site-packages/rembg/multiprocessing.py", line 24, in worker
net = Net(model_name)
File "/home/munsch/.local/lib/python3.9/site-packages/rembg/bg.py", line 74, in init
net.load_state_dict(torch.load(path, map_location=torch.device(DEVICE)))
File "/home/munsch/.local/lib/python3.9/site-packages/torch/serialization.py", line 593, in load
return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
File "/home/munsch/.local/lib/python3.9/site-packages/torch/serialization.py", line 772, in _legacy_load
result = unpickler.load()
File "/home/munsch/.local/lib/python3.9/site-packages/torch/serialization.py", line 728, in persistent_load
deserialized_objects[root_key] = restore_location(obj, location)
File "/home/munsch/.local/lib/python3.9/site-packages/torch/serialization.py", line 812, in restore_location
return default_restore_location(storage, str(map_location))
File "/home/munsch/.local/lib/python3.9/site-packages/torch/serialization.py", line 175, in default_restore_location
result = fn(storage, location)
File "/home/munsch/.local/lib/python3.9/site-packages/torch/serialization.py", line 154, in _cuda_deserialize
with torch.cuda.device(device):
File "/home/munsch/.local/lib/python3.9/site-packages/torch/cuda/init.py", line 223, in enter
self.prev_idx = torch._C._cuda_getDevice()
File "/home/munsch/.local/lib/python3.9/site-packages/torch/cuda/init.py", line 160, in _lazy_init
raise RuntimeError(
RuntimeError: Cannot re-initialize CUDA in forked subprocess. To use CUDA with multiprocessing, you must use the 'spawn' start method

Do you have any Tip or solution?
Regards
Rene´

Is there any way to use this just for images?

I just read the documentation, and the code. It seems you where able to speed up the rembg for video. But I'm unsure if it's possible to just use greenscreen for a image? Also is it possible to ouput the location or file name? ie, with -o

Broke rembg

Hoping to get some insight here - I had rembg running fine, I installed this and it broke rembg.

I have unistalled everything but simply can not get standard rembg back up and running.

Would love if I could get some insight into what may have happened?

what version of python should we use?

Hi there,

I tried both python 3.8 and 3.9 on Ubuntu-21.10 and failed at installing the requirements.txt.

Is this project tested under a specific version of python?

For example, scikit-image 0.17.2 does not support python 3.9 but it is specified in the requirements.txt.

Thanks

	'-y',
	'-f', 'rawvideo',
	'-vcodec', 'rawvideo',
	'-s', F"{frame.shape[1]}x320",
	'-pix_fmt', 'gray',
	'-r', F"{framerate}",

ecsplendid / rembg-greenscreen Goto Github PK

rembg-greenscreen's Introduction

Rembg Virtual Greenscreen Edition (Dr. Tim Scarfe)

Video Virtual Green Screen Edition

CLI interface

Important notes

References

License

rembg-greenscreen's People

Contributors

Stargazers

Watchers

Forkers

rembg-greenscreen's Issues

Recommend Projects

Recommend Topics

Recommend Org