Git Product home page Git Product logo

vcamsample's Introduction

VCamSample

This solution contains a Media Foundation Virtual Camera Sample. It works only on Windows 11 thanks to the MFCreateVirtualCamera API.

There are two projects in the solution:

  • VCamSampleSource: the Media Source that provides RGB32 and NV12 streaming samples.
  • VCamSample: the "driver" application that does very little but calls MFCreateVirtualCamera.

Note there's a VCamNetSample .NET C# port of this project available here : https://github.com/smourier/VCamNetSample

To test the virtual cam:

  • Build in debug or release
  • Go to the build output and register the media source (a COM object) with a command similar to this: regsvr32 VCamSampleSource.dll (you must run this as administrator, it' not possible to register a Virtual Camera media source in HKCU, only in HKLM since it will be loaded by multiple processes)
  • Run the VCamSample app.
  • Run for example the Windows Camera app or using a Web Browser ImageCapture API

You should now see something like this in the Windows Camera App

Screenshot 2024-01-22 131726

Something like this in Windows' Edge Web Browser, using this testing page: https://googlechrome.github.io/samples/image-capture/grab-frame-take-photo.html

Screenshot 2024-01-22 133220

Something like this in OBS (Video Capture Device):

image

Notes

  • The media source uses Direct2D and DirectWrite to create images. It will then create Media Foundation samples from these. To create MF samples, it can use:

    • The GPU, if a Direct3D manager has been provided by the environment. This is the case of the Windows 11 camera app.
    • The CPU, if no Direct3D environment has been provided. In this case, the media source uses a WIC bitmap as a render target and it then copies the bits over to an MF sample. The ImageCapture API code embedded in Chrome or Edge, Teams, etc. is an example of such a D3D-less environment.
    • If you want to force CPU usage at all times, you can change the code in MediaStream::SetD3DManager and put the lines there in comment.
  • The media source provides RGB32 and NV12 formats as most setups prefer the NV12 format. Samples are initially created as RGB32 (Direct2D) and converted to NV12. To convert the samples, the media source uses two ways:

    • The GPU, if a Direct3D manager has been provided, using Media Foundation's Video Processor MFT.
    • The CPU, if no Direct3D environment has been provided. In this case, the RGB to NV12 conversion is done in the code (so on the CPU).
    • If you want to force RGB32 mode, you can change the code in MediaStream::Initialize and set the media types array size to 1 (check comments in the code).
  • The code crrently has an issue where the virtual camera screen is shown in the preview window of apps such as Microsoft Teams, but it's not rendered to the communicating party. Not sure why it doesn't fully work yet, if you know, just ping me!

Tracing

The code output lots of interesting traces. It's quite important in this virtual camera environment because there's not just your process that's involved but at least 4: the VCamSample app, the Windows Frame Server, the Windows camera monitor, and the reader app (camera, etc.). They all load the media source COM object in-process.

Tracing here doesn't use OutputDebugString because it's 100% old, crappy, truncating text, slow, etc. Instead it uses Event Tracing for Windows ("ETW") in "string-only" mode (the mode where it's very simple and you don't have to register painfull traces records and use complex readers...).

So to read these ETW traces, use WpfTraceSpy you can download here https://github.com/smourier/TraceSpy. Configure an ETW Provider with the GUID set to 964d4572-adb9-4f3a-8170-fcbecec27467

vcamsample's People

Contributors

smourier avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

vcamsample's Issues

GPU & CPU

In Win11, the system camera uses a GPU
屏幕截图 2024-04-19 145515

OBS uses CPU
Screenshot 2024-04-19 14-56-16

modify 【_renderTarget->Clear(D2D1::ColorF(0, 255, 255, 0.5));】rgba->a 0.5
屏幕截图 2024-04-19 145651
Comparing two images, setting 【 rgba ->a 0.5 】 did not take effect in the CPU channel

unable to run

Screenshot 2024-04-14 132610

I tried on 2 different computer and it does the same problem. I registred the .dll with regsvr32 VCamSampleSource.dll as administrator

Unable to build VCamSampleSource on Windows 11

Hello!

I've tried to build VCamSampleSource but I am getting some errors while compiling.
Do I have a wrong library somewhere?

image

Neuerstellen gestartet...
1>------ Neues Erstellen gestartet: Projekt: VCamSampleSource, Konfiguration: Debug x64 ------
1>pch.cpp
1>Activator.cpp
1>C:\Users\lasse\Desktop\VCamSample-main\VCamSampleSource\Activator.cpp(14,2): error C2065: "MF_VIRTUALCAMERA_PROVIDE_ASSOCIATED_CAMERA_SOURCES": nichtdeklarierter Bezeichner
1>C:\Users\lasse\Desktop\VCamSample-main\VCamSampleSource\Activator.cpp(14,2): error C2672: "wil::verify_hresult": keine übereinstimmende überladene Funktion gefunden
1>C:\Users\lasse\Desktop\VCamSample-main\packages\Microsoft.Windows.ImplementationLibrary.1.0.240122.1\include\wil\common.h(755,54): message : kann "long wil::verify_hresult(T)" sein
1>C:\Users\lasse\Desktop\VCamSample-main\VCamSampleSource\Activator.cpp(14,2): error C2737: "__hrRet": Das Objekt "const" muss initialisiert werden.
1>C:\Users\lasse\Desktop\VCamSample-main\VCamSampleSource\Activator.cpp(14,2): error C3536: "__hrRet": Kann nicht verwendet werden, bevor es initialisiert wurde.
1>dllmain.cpp
1>EnumNames.cpp
1>FrameGenerator.cpp
1>MediaSource.cpp
1>MediaStream.cpp
1>MFTools.cpp
1>Tools.cpp
1>C:\Users\lasse\Desktop\VCamSample-main\VCamSampleSource\Tools.cpp(70,3): error C2065: "MF_VIRTUALCAMERA_PROVIDE_ASSOCIATED_CAMERA_SOURCES": nichtdeklarierter Bezeichner
1>C:\Users\lasse\Desktop\VCamSample-main\VCamSampleSource\Tools.cpp(72,3): error C2065: "MF_VIRTUALCAMERA_ASSOCIATED_CAMERA_SOURCES": nichtdeklarierter Bezeichner
1>WinTrace.cpp
1>Code wird generiert...
1>Die Erstellung des Projekts "VCamSampleSource.vcxproj" ist abgeschlossen -- FEHLER.
========== Alle neu erstellen: 0 erfolgreich, 1 fehlgeschlagen, 0 übersprungen ==========
========== Neu erstellen wurde am 16:26 gestartet und dauerte 14,447 Sekunden ==========

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.