iperov / deepfacelive Goto Github PK
View Code? Open in Web Editor NEWReal-time face swap for PC streaming or video calls
License: GNU General Public License v3.0
Real-time face swap for PC streaming or video calls
License: GNU General Public License v3.0
Heres the error , newest python instaled, gtx drivers 472.12
`Running DeepFaceLive.
FaceDetector error: [ONNXRuntimeError] : 1 : FAIL : Non-zero status code returned while running Sigmoid node. Name:'Sigmoid_1' Status Message: CUDA error cudaErrorNoKernelImageForDevice:no kernel image is available for execution on the device Traceback (most recent call last):
File "J:\SOFT\DeepFaceLive_NVIDIA_internal\DeepFaceLive\xlib\mp\csw\CSWBase.py", line 484, in _start_proc
self.on_tick()
File "J:\SOFT\DeepFaceLive_NVIDIA_internal\DeepFaceLive\apps\DeepFaceLive\backend\FaceDetector.py", line 234, in on_tick
rects = self.YoloV5Face.extract (frame_image, threshold=detector_state.threshold, fixed_window=detector_state.fixed_window_size)[0]
File "J:\SOFT\DeepFaceLive_NVIDIA_internal\DeepFaceLive\modelhub\onnx\YoloV5Face\YoloV5Face.py", line 74, in extract
preds = self._get_preds(ip.get_image('NCHW'))
File "J:\SOFT\DeepFaceLive_NVIDIA_internal\DeepFaceLive\modelhub\onnx\YoloV5Face\YoloV5Face.py", line 106, in _get_preds
preds = self._sess.run(None, {self._input_name: img})
File "J:\SOFT\DeepFaceLive_NVIDIA_internal\python\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 192, in run
return self._sess.run(output_names, input_feed, run_options)
onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Non-zero status code returned while running Sigmoid node. Name:'Sigmoid_1' Status Message: CUDA error cudaErrorNoKernelImageForDevice:no kernel image is available for execution on the device
FaceDetector error: [ONNXRuntimeError] : 1 : FAIL : Non-zero status code returned while running Sigmoid node. Name:'Sigmoid_1' Status Message: CUDA error cudaErrorNoKernelImageForDevice:no kernel image is available for execution on the device Traceback (most recent call last):
File "J:\SOFT\DeepFaceLive_NVIDIA_internal\DeepFaceLive\xlib\mp\csw\CSWBase.py", line 484, in _start_proc
self.on_tick()
File "J:\SOFT\DeepFaceLive_NVIDIA_internal\DeepFaceLive\apps\DeepFaceLive\backend\FaceDetector.py", line 234, in on_tick
rects = self.YoloV5Face.extract (frame_image, threshold=detector_state.threshold, fixed_window=detector_state.fixed_window_size)[0]
File "J:\SOFT\DeepFaceLive_NVIDIA_internal\DeepFaceLive\modelhub\onnx\YoloV5Face\YoloV5Face.py", line 74, in extract
preds = self._get_preds(ip.get_image('NCHW'))
File "J:\SOFT\DeepFaceLive_NVIDIA_internal\DeepFaceLive\modelhub\onnx\YoloV5Face\YoloV5Face.py", line 106, in _get_preds
preds = self._sess.run(None, {self._input_name: img})
File "J:\SOFT\DeepFaceLive_NVIDIA_internal\python\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 192, in run
return self._sess.run(output_names, input_feed, run_options)
onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Non-zero status code returned while running Sigmoid node. Name:'Sigmoid_1' Status Message: CUDA error cudaErrorNoKernelImageForDevice:no kernel image is available for execution on the device
FaceMarker error: [ONNXRuntimeError] : 1 : FAIL : Non-zero status code returned while running Transpose node. Name:'conv2d_1__50' Status Message: CUDA error cudaErrorNoKernelImageForDevice:no kernel image is available for execution on the device Traceback (most recent call last):
File "J:\SOFT\DeepFaceLive_NVIDIA_internal\DeepFaceLive\xlib\mp\csw\CSWBase.py", line 484, in _start_proc
self.on_tick()
File "J:\SOFT\DeepFaceLive_NVIDIA_internal\DeepFaceLive\apps\DeepFaceLive\backend\FaceMarker.py", line 173, in on_tick
lmrks = self.google_facemesh.extract(face_image)[0]
File "J:\SOFT\DeepFaceLive_NVIDIA_internal\DeepFaceLive\modelhub\onnx\FaceMesh\FaceMesh.py", line 57, in extract
lmrks = self._sess.run(None, {self._input_name: feed_img})[0]
File "J:\SOFT\DeepFaceLive_NVIDIA_internal\python\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 192, in run
return self._sess.run(output_names, input_feed, run_options)
onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Non-zero status code returned while running Transpose node. Name:'conv2d_1__50' Status Message: CUDA error cudaErrorNoKernelImageForDevice:no kernel image is available for execution on the device
FaceSwapper error: [ONNXRuntimeError] : 1 : FAIL : Non-zero status code returned while running Transpose node. Name:'transpose_21' Status Message: CUDA error cudaErrorNoKernelImageForDevice:no kernel image is available for execution on the device Traceback (most recent call last):
File "J:\SOFT\DeepFaceLive_NVIDIA_internal\DeepFaceLive\xlib\mp\csw\CSWBase.py", line 484, in _start_proc
self.on_tick()
File "J:\SOFT\DeepFaceLive_NVIDIA_internal\DeepFaceLive\apps\DeepFaceLive\backend\FaceSwapper.py", line 280, in on_tick
celeb_face, celeb_face_mask_img, face_align_mask_img = dfm_model.convert(face_align_image, morph_factor=model_state.morph_factor)
File "J:\SOFT\DeepFaceLive_NVIDIA_internal\DeepFaceLive\modelhub\DFLive\DFMModel.py", line 114, in convert
out_face_mask, out_celeb, out_celeb_mask = self._sess.run(None, {'in_face:0': img})
File "J:\SOFT\DeepFaceLive_NVIDIA_internal\python\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 192, in run
return self._sess.run(output_names, input_feed, run_options)
onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Non-zero status code returned while running Transpose node. Name:'transpose_21' Status Message: CUDA error cudaErrorNoKernelImageForDevice:no kernel image is available for execution on the device
FaceMarker error: [ONNXRuntimeError] : 1 : FAIL : Non-zero status code returned while running Transpose node. Name:'conv2d_1__50' Status Message: CUDA error cudaErrorNoKernelImageForDevice:no kernel image is available for execution on the device Traceback (most recent call last):
File "J:\SOFT\DeepFaceLive_NVIDIA_internal\DeepFaceLive\xlib\mp\csw\CSWBase.py", line 484, in _start_proc
self.on_tick()
File "J:\SOFT\DeepFaceLive_NVIDIA_internal\DeepFaceLive\apps\DeepFaceLive\backend\FaceMarker.py", line 173, in on_tick
lmrks = self.google_facemesh.extract(face_image)[0]
File "J:\SOFT\DeepFaceLive_NVIDIA_internal\DeepFaceLive\modelhub\onnx\FaceMesh\FaceMesh.py", line 57, in extract
lmrks = self._sess.run(None, {self._input_name: feed_img})[0]
File "J:\SOFT\DeepFaceLive_NVIDIA_internal\python\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 192, in run
return self._sess.run(output_names, input_feed, run_options)
onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Non-zero status code returned while running Transpose node. Name:'conv2d_1__50' Status Message: CUDA error cudaErrorNoKernelImageForDevice:no kernel image is available for execution on the device
FaceDetector error: [ONNXRuntimeError] : 1 : FAIL : Non-zero status code returned while running Sigmoid node. Name:'Sigmoid_1' Status Message: CUDA error cudaErrorNoKernelImageForDevice:no kernel image is available for execution on the device Traceback (most recent call last):
File "J:\SOFT\DeepFaceLive_NVIDIA_internal\DeepFaceLive\xlib\mp\csw\CSWBase.py", line 484, in _start_proc
self.on_tick()
File "J:\SOFT\DeepFaceLive_NVIDIA_internal\DeepFaceLive\apps\DeepFaceLive\backend\FaceDetector.py", line 234, in on_tick
rects = self.YoloV5Face.extract (frame_image, threshold=detector_state.threshold, fixed_window=detector_state.fixed_window_size)[0]
File "J:\SOFT\DeepFaceLive_NVIDIA_internal\DeepFaceLive\modelhub\onnx\YoloV5Face\YoloV5Face.py", line 74, in extract
preds = self._get_preds(ip.get_image('NCHW'))
File "J:\SOFT\DeepFaceLive_NVIDIA_internal\DeepFaceLive\modelhub\onnx\YoloV5Face\YoloV5Face.py", line 106, in _get_preds
preds = self._sess.run(None, {self._input_name: img})
File "J:\SOFT\DeepFaceLive_NVIDIA_internal\python\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 192, in run
return self._sess.run(output_names, input_feed, run_options)
onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Non-zero status code returned while running Sigmoid node. Name:'Sigmoid_1' Status Message: CUDA error cudaErrorNoKernelImageForDevice:no kernel image is available for execution on the device
Press any key to continue . . .`
Hello!
I am trying to run the demo program, following this tutorial, however at the point when I choose the device for the face detector, it goes red and the terminal has this output:
FaceDetector error: CUDAExecutionProvider is not avaiable in onnxruntime Traceback (most recent call last):
File "D:\Desktop\DeepFaceLive-master\xlib\mp\csw\CSWBase.py", line 472, in _start_proc
self.on_start(*worker_start_args, **worker_start_kwargs)
File "D:\Desktop\DeepFaceLive-master\apps\DeepFaceLive\backend\FaceDetector.py", line 81, in on_start
cs.detector_type.select(state.detector_type)
File "D:\Desktop\DeepFaceLive-master\xlib\mp\csw\DynamicSingleSwitch.py", line 99, in select
result = self._set_selected_idx(idx_or_choice)
File "D:\Desktop\DeepFaceLive-master\xlib\mp\csw\DynamicSingleSwitch.py", line 33, in _set_selected_idx
self._on_selected_evl.call(selected_idx, self.get_selected_choice() )
File "D:\Desktop\DeepFaceLive-master\xlib\python\EventListener.py", line 24, in call
func(*args, **kwargs)
File "D:\Desktop\DeepFaceLive-master\apps\DeepFaceLive\backend\FaceDetector.py", line 99, in on_cs_detector_type
cs.device.select(state.YoloV5_state.device)
File "D:\Desktop\DeepFaceLive-master\xlib\mp\csw\DynamicSingleSwitch.py", line 99, in select
result = self._set_selected_idx(idx_or_choice)
File "D:\Desktop\DeepFaceLive-master\xlib\mp\csw\DynamicSingleSwitch.py", line 33, in _set_selected_idx
self._on_selected_evl.call(selected_idx, self.get_selected_choice() )
File "D:\Desktop\DeepFaceLive-master\xlib\python\EventListener.py", line 24, in call
func(*args, **kwargs)
File "D:\Desktop\DeepFaceLive-master\apps\DeepFaceLive\backend\FaceDetector.py", line 144, in on_cs_devices
self.YoloV5Face = onnx_models.YoloV5Face(device)
File "D:\Desktop\DeepFaceLive-master\modelhub\onnx\YoloV5Face\YoloV5Face.py", line 34, in __init__
self._sess = sess = InferenceSession_with_device(str(path), device_info)
File "D:\Desktop\DeepFaceLive-master\xlib\onnxruntime\InferenceSession.py", line 23, in InferenceSession_with_device
raise Exception('CUDAExecutionProvider is not avaiable in onnxruntime')
Exception: CUDAExecutionProvider is not avaiable in onnxruntime
I have CUDA 11.4 installed with the cudnn support. My graphics card is GTX 1080 Ti.
Looking forward to hearing from you! Thanks.
Hi! When I try to change the face opacity from the frame merger and the selected device for it is the CPU, frame merger module stop to work and it raises an exception.
FaceMerger error: Iterator requested dtype could not be cast from dtype('float64') to dtype('float32'), the operand 0 dtype, according to the rule 'safe' Traceback (most recent call last):
File "U:\DeepFake_softwares\DeepFaceLive_NVIDIA\_internal\DeepFaceLive\xlib\mp\csw\CSWBase.py", line 484, in _start_proc
self.on_tick()
File "U:\DeepFake_softwares\DeepFaceLive_NVIDIA\_internal\DeepFaceLive\apps\DeepFaceLive\backend\FaceMerger.py", line 350, in on_tick
merged_frame = self._merge_on_cpu(out_merged_frame, frame_image, face_resolution, face_align_img, face_align_mask_img, face_align_lmrks_mask_img, face_swap_img, face_swap_mask_img, aligned_to_source_uni_mat, frame_width, frame_height )
File "U:\DeepFake_softwares\DeepFaceLive_NVIDIA\_internal\DeepFaceLive\apps\DeepFaceLive\backend\FaceMerger.py", line 252, in _merge_on_cpu
ne.evaluate('frame_image*(one_f-frame_face_mask) + frame_image*frame_face_mask*(one_f-opacity) + frame_face_swap_img*frame_face_mask*opacity', out=out_merged_frame)
File "U:\DeepFake_softwares\DeepFaceLive_NVIDIA\_internal\python\lib\site-packages\numexpr\necompiler.py", line 836, in evaluate
return compiled_ex(*arguments, **kwargs)
TypeError: Iterator requested dtype could not be cast from dtype('float64') to dtype('float32'), the operand 0 dtype, according to the rule 'safe'
With GPU is alright.
I've download some .npy models either from Internet and local DeepFaceLab, but they are not detected from DFL workscreen
Destination path can't have spaces, otherwise you get the error 'not recognized as an internal or external command'
How to use deepfacelive on linux
Is it possible to swap multiple faces?
Thanks for your sharing of your wonderful work!
But I get some trouble when using. Hope you can give some help. Thank you very much!
I cannot import trained dfl model by click the 'select' button in Face swapper.
Actually, I can find my model by the small eye on the right.
So I think mybe the structure of my model files is not correct. But I dont know how to fix it.
My folder is as following:
Hi,
first off, this is so cool, thanks for your marvellous work!
This is more of a suggestion than an issue.
When I saw your excellent pipeline, the extent on how this is all optimized to run in real time and the ease of configurability, I was wondering if you know of Spout.
It could save some resources to just output the final stream as a SpoutSender
. This could then easily be imported into OBS by the Spout Plugin.
The advantage would be that all the buffers already in the GPU would get directly transferred to OBS and not via a window capture.
can you plan to add xseg face mask ?
I tried to download the model from your codes, but it seems the url is not work:
dfm_models = [
#DFMModelInfo(name='Tom Cruise', model_path=models_path / f'Tom Cruise.dfm', url=rf'https://github.com/iperov/DeepFaceLive/releases/download/test/TOM_CRISE.onnx'),#TODO https://github.com/iperov/DeepFaceLive/releases/download/dfm/TOM_CRUISE.dfm'),
#DFMModelInfo(name='Vladimir Putin', model_path=models_path / f'Vladimir Putin.dfm', url=rf'https://github.com/iperov/DeepFaceLive/releases/download/dfm/VLADIMIR_PUTIN.dfm'),
]
Will the project provided a default dfm model (such as Tom_Cruise.dfm)?
Or a public model download link is OK.
Thanks a lot.
Hi !
I heard about your wonderful creation and I went to download it !
But I have a big problem, I can't find the file "DeepFaceLive.bat" or a similar file as you show in your setup tutorial.
I wonder if the software is still totally accessible or just the source code.
Thanks in advance !
Just a tip, that someone is monetizing your tech. They are charging $250 for a two minute video. https://theindiandeepfaker.com/pricing/
in apps/DeepFaceLive/DeepFaceLiveApp.py
line about 164.
menu_help_action_github.triggered.connect(lambda: qtx.QDesktopServices.openUrl(qtx.QUrl('https://github.com/iperov/DeepFaceLive' )))
you miss the code qtx.
so you change your code QUrl to qtx.QUrl
I use my phone's camera as a virtual camera for my PC via OBS using https://vdo.ninja/
Is it possible to use this virtual camera as a Camera Source?
Excuse me, do you need the linux networking version? Windows I don't think it can be better commercialized. I have changed the linux interfaceless version.
Previus versions with old my models works 3x faster, with new only Tom kruse is working ok and my own dfl old models works very slow, its very strange, how correct train and convert model? And where get other pretrained models like Tom Kruse and why its works on all faces how to traint it like that?
I training with RTM model for the first time
I'm wondering should the dst(rtm wf faceset in the torrent) be xseg and apply?
In the FAQ it only said that i should xseg and apply in src but don't known should it be apply in dst
Hello God, now the framework can only be used on the PC side to realize synchronous face change with streaming.
Can it be optimized on the VR glasses side and mobile phone side.
请问deepfacelive可以在换脸的同时将头发一起更换吗,请看这个例子 https://www.youtube.com/watch?v=xr5FHd0AdlQ
Hi
I'd like to know if there is a python script or example code that can invoke DeepFaceLive from the command line.
I Know that this functionality is available in deepfacelab but as far as I know deepfacelive has 2 main advantages:
1- It can work with any target video
2- It is real time fast (30 fps is possible)
So I'd like to know how to load dfm model and point deepfacelive to a video file to instantly convert it.
Is there any script (python of course) or example code that can do this
Thanks
This technology is only going to be used for evil purposes. To deceive people, steal their money, possessions, undercover operations. Nothing good. It is already being used for scams like the pig-butchering plate, in which hundreds of thousands of lives around the world are being left without money, causing a lot of suffering.
I've download some .npy models either from Internet and local DeepFaceLab, but they are not detected from DFL workscreen
All models from DFL are .npy!
can the project work on ubuntu/linux environment? if so, could you point me to the doc or how could i help to port.
Thanks for your great work.
Is it possible to run it in colab?
Or, do you have any plans to release colab demo?
Добрый день!
Спасибо за крутое приложение!
Возможно ли использовать в качестве источника камеры - виртуальную камеру OBS? Пытался транслировать, но при всех конфигурациях получал только черный экран. OBS virtualcam plugin не помог.
I can't open the faceset.pak file . i think it is corrupted
Awesome work! i throw modles into DFMfolder, the software can't detect it.
Hello, on Linux (ubuntu 20.04) I got this error:
[ WARN:[email protected]] global /io/opencv/modules/videoio/src/cap_v4l.cpp (889) open VIDEOIO(V4L2:/dev/video0): can't open camera by index
[ WARN:[email protected]] global /io/opencv/modules/videoio/src/cap_v4l.cpp (889) open VIDEOIO(V4L2:/dev/video1): can't open camera by index
I have test with one camera & one webcam:
ls /dev/video*
/dev/video0 /dev/video1
Hi Team,
The mega.nz download link isn't working (it times out). Has the installer been moved?
Автор, как изменить разрешение источника видео, хочу добиться характеристик мобильного телефона прямой трансляции, 720х1280
Hello,
Thank you for sharing this wonderful work with us.
I need to configure the project in PyCharm IDE using Anaconda.
My Basic question is
"Can I run the whole project by typing one complete python command with command-line arguments by providing the link to the image and video. This means can I run the project by calling the main.py file only and then generate the output in a separate folder."
OR
Do I need to run the DeepFaceLive.py file and then run the project from the user interface?
Thank You
How can I generate a dfm file?:)
What settings and resolution are the ready-to-use public face models created with?
Is the resolution better than the user-faq supplied settings "res:224, WF, archi:liae-udt, ae_dims:512, e_dims:64, d_dims:64, d_mask_dims:32, eyes_mouth_prio:Y, blur_out_mask:Y, uniform_yaw:Y, lr_dropout:Y, batch:8. Others by default"?
And whats the difference with the Google FaceMesh models, is there any code for that?
I have made a model and using xseg I have excluded the mouth when open and trained it, and it masks as desired. But when I export this model to dfm and use it in deepfacelive the mouth is not excluded. I can however use deepfacelabs merge SAEHD and set mask_mode to Xseg-dst. This gives the desired result.
Is there any way to achieve the same in deepfacelive?
Thanks a lot for your amazing work!
In China, we can't download software from Mega even if we use VPN QAQ
So can you share Google drive links about this release?
Thanks again!
Автор вроде бы знает русский, поэтому буду писать на русском)
Как я могу добавить свою модель? Как отсканировать свое лицо? Или как вообще можно расширить те 8 лиц, которые идут по умолчанию?
How can I add my face model? How to scan my face? Or how can you expand those 8 faces that come by default?
Running DeepFaceLive.
Traceback (most recent call last):
File "internal\DeepFaceLive\main.py", line 95, in
main()
File "internal\DeepFaceLive\main.py", line 88, in main
args.func(args)
File "internal\DeepFaceLive\main.py", line 30, in run_DeepFaceLive
from apps.DeepFaceLive.DeepFaceLiveApp import DeepFaceLiveApp
File "D:\DeepFaceLive_NVIDIA_internal\DeepFaceLive\apps\DeepFaceLive\DeepFaceLiveApp.py", line 14, in
from . import backend
File "D:\DeepFaceLive_NVIDIA_internal\DeepFaceLive\apps\DeepFaceLive\backend_init.py", line 1, in
from .BackendBase import (BackendConnection, BackendConnectionData, BackendDB,
File "D:\DeepFaceLive_NVIDIA_internal\DeepFaceLive\apps\DeepFaceLive\backend\BackendBase.py", line 7, in
from xlib import time as lib_time
File "D:\DeepFaceLive_NVIDIA_internal\DeepFaceLive\xlib\time_init.py", line 1, in
from .time import timeit, measure, FPSCounter, AverageMeasurer
File "D:\DeepFaceLive_NVIDIA_internal\DeepFaceLive\xlib\time\time_.py", line 11, in
if not kernel32.QueryPerformanceFrequency(_perf_freq):
File "D:\DeepFaceLive_NVIDIA_internal\DeepFaceLive\xlib\api\win32\wintypes\wintypes.py", line 32, in wrapper
raise RuntimeError(f'Unable to load {dll_name} library.')
RuntimeError: Unable to load kernel32 library.
Press any key to continue . . .
I've tried both versions. I have an rtx 2070 with 8gig of vram. All other programs work just fine. Deepfacelab works great to. This is the only program that won't run. I've tried everything. Even tried replacing the kernel32.dll with another windows 10 kernel.dll.
Does this have MacOs support?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.