seanseattle / mobilefaceswap Goto Github PK
View Code? Open in Web Editor NEWMobileFaceSwap: A Lightweight Framework for Video Face Swapping (AAAI 2022)
License: Apache License 2.0
MobileFaceSwap: A Lightweight Framework for Video Face Swapping (AAAI 2022)
License: Apache License 2.0
Hi,
I've just added some extra functions to your video swap example , eg. keep audio and different detection size to make it even faster.
Maybe stupid question, can you please give a hint how to expand the bbox of the face detection?
In some cases it cuts the forehead and chin region of the detected face.
Thanks
Thomas
hi, thank you for providing your paper & source code.
but there is no license anywhere.
could you specify the license?
or can i use your the code and pretrained model for commercially?
The comments say the 256 image_size will use FaceShifter as the model, but it appears that it always loads the 224 (SimSwap) weights. Can you add weights/code for the FaceShifter model so that the results can be compared?
Does there any pytorch implementation?
Please suggest how to improve source face features or weight in the output image.
Thanks
Hi, I'm experiencing the following error below.
I'm on MacOS 12.6
I've installed the following:
python 3.9
numpy==1.23.4
opencv-python==4.6.0.66
insightface==0.2.1
paddlepaddle==2.1.2
onnxruntime==1.13.1 // had to install because the image_test.py threw an error saying it was required
I've extracted checkpoints.tar
in the root dir of the repo.
And I'm getting the following error:
$ python3.9 ./image_test.py --target_img_path ./content/input/target1.png --source_img_path ./content/input/source1.png --output_dir ./content/output/
/usr/local/lib/python3.9/site-packages/paddle/vision/transforms/functional_pil.py:36: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.
'nearest': Image.NEAREST,
/usr/local/lib/python3.9/site-packages/paddle/vision/transforms/functional_pil.py:37: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
'bilinear': Image.BILINEAR,
/usr/local/lib/python3.9/site-packages/paddle/vision/transforms/functional_pil.py:38: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.
'bicubic': Image.BICUBIC,
/usr/local/lib/python3.9/site-packages/paddle/vision/transforms/functional_pil.py:39: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.
'box': Image.BOX,
/usr/local/lib/python3.9/site-packages/paddle/vision/transforms/functional_pil.py:40: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.
'lanczos': Image.LANCZOS,
/usr/local/lib/python3.9/site-packages/paddle/vision/transforms/functional_pil.py:41: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.
'hamming': Image.HAMMING
Traceback (most recent call last):
File "/Users/ME/MobileFaceSwap/./image_test.py", line 98, in <module>
landmarkModel = LandmarkModel(name='landmarks')
File "/Users/ME/MobileFaceSwap/utils/prepare_data.py", line 25, in __init__
assert 'detection' in self.models
AssertionError
Any help would be really appreciated.
Thank you.
Has anyone figured out how to get this to work on colab? Currently getting issues.
How to increase the resolution? as of now it is 224 and I would like to test on higher resolution
Thank you for your wonderful results.
You defined Q for Reweighting loss in your paper. You also described that you supervisioned with L2 loss using the ResNet-18 network to output the regression value.
If so, what data was used as the label of the teacher image?
What is expected is to be supervision only when the source and target are the same. However, in this case, if the source and target are different, quality check cannot be performed.
I am very curious and would appreciate it if you could give me an answer.
I run it on 3090 but the FPS can't reach 25fps
Am I missing some details?
Looking forward to your response
Has anyone been able to deploy this on mobile? the owner of this repo said to use paddle lite, but there are no details on model conversion or optimization for Paddle Lite. has anyone tried to convert it to onnx, or used Paddle Lite's Conversion Tools for this?
hello! Thank you for sharing this awesome implementation!
but, I cannot download pretrained weight from google drive, because of sharing setting...
can you fix the sharing setting?
would be amazing if this could be hosted on huggingface using gradio.
Thank you for this work, very interesting. When trying to run the test with a portrait image and a standard video, no error is raised but results
and merge_result
folders are not created. Do you have any idea why?
(mobilesim) user@MacBook-Pro-de-user MobileFaceSwap % python video_test.py --target_video_path dataB/test.mp4 --source_img_path dataB/ml.jpeg --output_path results --merge_result merge_result
sysctl: unknown oid 'machdep.cpu.leaf7_features'
input mean and std: 127.5 127.5
find model: ./checkpoints/landmarks/glintr100.onnx recognition
find model: ./checkpoints/landmarks/scrfd_10g_bnkps.onnx detection
set det-size: (640, 640)
/Users/user/Desktop/MobileFaceSwap/utils/align_face.py:30: FutureWarning: `rcond` parameter will change to the default of machine precision times ``max(M, N)`` where M and N are the input matrix dimensions.
To use the future default and silence this warning we advise to pass `rcond=None`, to keep using the old, explicitly pass `rcond=-1`.
A, res, rank, s = np.linalg.lstsq(src_pts_, dst_pts_)
/Users/user/miniconda3/envs/mobilesim/lib/python3.9/site-packages/paddle/tensor/creation.py:125: DeprecationWarning: `np.object` is a deprecated alias for the builtin `object`. To silence this warning, use `object` by itself. Doing this will not modify any behavior and is safe.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
if data.dtype == np.object:
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 45/45 [00:13<00:00, 3.40it/s]
(mobilesim) user@MacBook-Pro-de-user MobileFaceSwap %
hi thank you for porviding the code
Hello,
Thanks for the awesome work and release of the model and test code. Can you release the training code and suggest a dataset.
I want to train it for high res faces.
同一个视频,我发现外国人换上去要比亚洲人换上去的效果,好的多。
赞
Thx for your excellent works! I wonder when will the codes be released?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.