Git Product home page Git Product logo

enhancr's Introduction

heading-icon

enhancr is an elegant and easy to use GUI for Video Frame Interpolation and Video Upscaling which takes advantage of artificial intelligence - built using node.js and Electron. It was created to enhance the user experience for anyone interested in enhancing video footage using artificial intelligence. The GUI was designed to provide a stunning experience powered by state-of-the-art technologies without feeling clunky and outdated like other alternatives.

gui-preview-image

It features blazing-fast TensorRT inference by NVIDIA, which can speed up AI processes significantly. Pre-packaged, without the need to install Docker or WSL (Windows Subsystem for Linux) - and NCNN inference by Tencent which is lightweight and runs on NVIDIA, AMD and even Apple Silicon - in contrast to the mammoth of an inference PyTorch is, which only runs on NVIDIA GPUs.

Features

  • Encodes video on the fly and reads frames from source video, without the need of extracting frames or loading into memory
  • Queue for batch processing
  • Live Preview of output media
  • Allows chaining of interpolation, upscaling & restoration
  • Offers the possibility to trim videos before processing
  • Can load custom ESRGAN models in onnx & pth format and converts them automatically
  • Has Scene Detection built-in, to skip interpolation on scene change frames & mitigate artifacts
  • Color Themes for user customization
  • Discord Rich Presence, to show all your friends progress, current speed & what you're currently enhancing
  • ... and much more

Interpolation

RIFE (NCNN) - megvii-research/ECCV2022-RIFE - powered by styler00dollar/VapourSynth-RIFE-NCNN-Vulkan

RIFE (TensorRT) - megvii-research/ECCV2022-RIFE - powered by AmusementClub/vs-mlrt & styler00dollar/VSGAN-tensorrt-docker

GMFSS (PyTorch) - 98mxr/GMFupSS - styler00dollar/VSGAN-tensorrt-docker

GMFSS (TensorRT) - 98mxr/GMFSS_Union - powered by HolyWu/vs-gmfss_union

CAIN (NCNN) - myungsub/CAIN - powered by mafiosnik/vsynth-cain-NCNN-vulkan (unreleased)

CAIN (TensorRT) - myungsub/CAIN - powered by HubertSotnowski/cain-TensorRT

Thanks to HubertSontowski and styler00dollar for helping out with implementing CAIN.

Upscaling

waifu2x (NCNN) - nagadomi/waifu2x - powered by nlzy/vapoursynth-waifu2x-NCNN-vulkan

RealESRGAN (NCNN) - xinntao/Real-ESRGAN - powered by AmusementClub/vs-mlrt

RealESRGAN (TensorRT) - xinntao/Real-ESRGAN - powered by AmusementClub/vs-mlrt

Thanks to HubertSontowski for helping out with implementing AnimeSR

Restoration

DPIR (TensorRT) - cszn/DPIR - powered by AmusementClub/vs-mlrt

System Requirements

Minimum:

  • Dual Core CPU with Hyperthreading enabled
  • Vulkan-capable graphics processor
  • Windows 10

Recommended:

  • Quad Core Intel Kaby Lake/AMD Ryzen or newer with Hyperthreading enabled
  • 16 GB RAM
  • NVIDIA 1000 Series (Pascal) for TensorRT/NCNN or AMD Polaris for NCNN
  • Windows 11

Installation

To ensure that you have the most recent version of the software and all necessary dependencies, we recommend downloading the installer from Patreon. Please note that builds and an embeddable python environment are not provided through this repository.

installer

There probably will be free versions down the line, after a backlog of versions has released.

macOS and Linux Support

The GUI was created with cross-platform compatibility in mind and is compatible with both operating systems. Our primary focus at the moment is ensuring a stable and fully functioning solution for Windows users, but support for Linux and macOS will be made available in the near future.

enhancr-macos

Support for Apple Silicon is planned as well, but I currently only have an Intel Macbook Pro available for testing.

Benchmarks

Input size: 1920x1080 @ 2x

RTX 2060S 1 RTX 3070 2 RTX A4000 3 RTX 3090 Ti 4 RTX 4090 5
RIFE / rife-v4.6 (NCNN) 53.78 fps 64.08 fps 80.56 fps 86.24 fps 136.13 fps
RIFE / rife-v4.6 (TensorRT) 70.34 fps 94.63 fps 86.47 fps 122.68 fps 170.91 fps
CAIN / cvp-v6 (NCNN) 9.42 fps 10.56 fps 13.42 fps 17.36 fps 44.87 fps
CAIN / cvp-v6 (TensorRT) 45.41 fps 63.84 fps 81.23 fps 112.87 fps 183.46 fps
GMFSS / Up (PyTorch) - - 4.32 fps - 16.35 fps
GMFSS / Union (PyTorch) - - 3.68 fps - 13.93 fps
GMFSS / Union (TensorRT) - - 6.79 fps - -
waifu2x / anime_style_art_rgb (NCNN) 6.71 fps 9.36 fps 9.81 fps 15.48 fps 39.77 fps
RealESRGAN / animevideov3 (TensorRT) 7.64 fps 9.10 fps 8.49 fps 18.66 fps 38.67 fps
DPIR / Denoise (TensorRT) 4.38 fps 6.45 fps 5.39 fps 11.64 fps 27.41 fps

1 Ryzen 5 3600X - Gainward RTX 2060 Super @ Stock

2 Ryzen 7 3800X - Gigabyte RTX 3070 Eagle OC @ Stock

3 Ryzen 5 3600X - PNY RTX A4000 @ Stock

4 i9 12900KF - ASUS RTX 3090 Ti Strix OC @ ~2220MHz

5 Ryzen 9 5950X - ASUS RTX 4090 Strix OC - @ ~3100MHz with curve to achieve maximum performance

Troubleshooting and FAQ (Frequently Asked Questions)

This section has moved to the wiki: https://github.com/mafiosnik777/enhancr/wiki

Check it out to learn more about getting the most out of enhancr or how to fix various problems.

Inferences

TensorRT is a highly optimized AI inference runtime for NVIDIA GPUs. It uses benchmarking to find the optimal kernel to use for your specific GPU, and there is an extra step to build an engine on the machine you are going to run the AI on. However, the resulting performance is also typically much much better than any PyTorch or NCNN implementation.

NCNN is a high-performance neural network inference computing framework optimized for mobile platforms. NCNN does not have any third party dependencies. It is cross-platform, and runs faster than all known open source frameworks on most major platforms. It supports NVIDIA, AMD, Intel Graphics and even Apple Silicon. NCNN is currently being used in many Tencent applications, such as QQ, Qzone, WeChat, Pitu and so on.

Supporting this project

I would be grateful if you could show your support for this project by contributing on Patreon or through a donation on PayPal. Your support will help to accelerate development and bring more updates to the project. Additionally, if you have the skills, you can also contribute by opening a pull request. Regardless of the form of support you choose to give, know that it is greatly appreciated.

Plans for the future

I am continuously working to improve the codebase, including addressing any inconsistencies that may have arisen due to time constraints. Regular updates will be released, including new features, bug fixes, and the incorporation of new technologies and models as they become available. Thank you for your understanding and support.

Join the discord

To interact with the community, share your results or to get help when encountering any problems visit our discord. Previews of upcoming versions are gonna be showcased on there as well.

enhancr's People

Contributors

bubblemint864 avatar icedshake avatar korakoe avatar mafiosnik777 avatar woofinas avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.