Git Product home page Git Product logo

com.unity.demoteam.digital-human's Introduction

Package: com.unity.demoteam.digital-human

Library of tech features used to realize the digital human from The Heretic and Enemies.

Usage

Declare the package as a git dependency in Packages/manifest.json:

"dependencies": {
    "com.unity.demoteam.digital-human": "https://github.com/Unity-Technologies/com.unity.demoteam.digital-human.git",
    ...
}

Requirements

-Minimum Requirements - Unity 2020.3 + - HDRP 10.9.0 +

  • Requirements for Skin Deformation and Skin Attachment GPU Path

    • Unity 2021.2 +
  • Requirements for new Eye and Skin Shaders

    • Unity 2022.2.0a16 +
    • HDRP 14.0.3 +

Features

  • Facial animation systems

    • Tools for 4D clip import and processing. (When we say 4D, we mean a sequence of meshes captured over time.)
    • 4D clip rendering with timeline integration.
    • 4D frame fitting allowing detail injection from facial rig.
    • Integration of facial rig from Snappers.
    • Pose facial rig directly in Unity.
  • Skin attachment system

    • Drive meshes and transforms in relation to dynamically deforming skin.
    • Used to drive eyebrows, eyelashes, stubble and logical markers.
    • Accelerated by C# Job System and Burst Compiler.
  • Shaders and rendering

    • Full shader graphs for skin/eyes/teeth/hair as seen in The Heretic.
    • Custom pass for cross-material normal buffer blur (tearline).
    • Custom marker-driven occlusion for eyes and teeth.

New improvements

  • Added GPU path for skin deformation and skin attachment calculations.
    • When using GPU path for skin attachment target, EyeRenderer and TeethRenderer need a reference to the SkinAttachmentTarget driving the markers as the transforms aren't updated on CPU anymore
  • Added skin tension to apply wrinkle maps.
  • Custom pass for slight blurring around the eye lids.
  • New shader graphs for eyes and skin.
  • SnappersHeadRenderer now uses texture arrays. If migrating from older version of DHP, use "build texture arrays" to setup the textures properly.

See also

https://github.com/Unity-Technologies/com.unity.demoteam.digital-human.sample

Related links

Video: The Heretic - Unity Short Film

Video: Making of The Heretic (Digital Dragons 2019)

com.unity.demoteam.digital-human's People

Contributors

fuglsang avatar mikko-alaluusua-unity avatar patrickp-unity3d avatar robcupisz avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

com.unity.demoteam.digital-human's Issues

Unity.DemoTeam.DigitalHuman.Runtime Assembly Definition - auto referenced?

I'm working on a large project where I can't simply create Assembly Definitions and References as I'd like. I'm able to use the Demo Team Hair System and Mesh to SDF packages as their runtime .asmdef files have Auto Referenced = true. I manually changed this for the Hair System Runtime .asmdef, but of course this will get overwritten by the next package update. Is it possible the Hair System Runtime .asmdef could have Auto Referenced set to true to match the other two packages?

I'd really like to try this out but..

I created an empty project. Added the lines

"com.unity.demoteam.attributes": "https://github.com/Unity-Technologies/com.unity.demoteam.attributes.git", "com.unity.demoteam.digital-human": "https://github.com/Unity-Technologies/com.unity.demoteam.digital-human.git", "com.unity.demoteam.digital-human.sample": "https://github.com/Unity-Technologies/com.unity.demoteam.digital-human.sample.git",

into manifest.json

I also cloned the packages from the github manually into the assets folder.

Unity Package Manager Error:
Resolved version 1.2.13 does not satisfy requested version 1.2.14

Why does it always need to be issues and errors when trying examples from Unity-Technologies? Usually you need to spend some time debugging and investing time solving errors.

SkinTensionRenderer - errors when using

I'm trying to use the SkinTensionRenderer script in my project but am getting two errors. The first is "Metal: Vertex or Fragment Shader xxx requires a ComputeBuffer at index x to be bound, but none provided." I guess this is related to an ongoing Mac OS Metal bug in Unity, so I switched over to Windows. Unfortunately now I'm getting null reference exceptions and Jobs errors.

I've attached the SkinTensionRenderer script to my SkinnedMesh and assigned the mesh to TensionBasePose in the Editor. Is there anything else I need to set up to get this working? Also, what are the Unity requirements for SkinTensionRender? I thought it was version 2021.2 +.

Any help you can give is much appreciated.

Documentation

We need more documetations about it, detailed one and not brief explaination.

For example, how to save out own facial expression using only this package. Able to adjust the facial expression I want through the SNAPPERS:Head, but how to save the transform of it so that able to load it at runtime?

Using EyeRenderer with a SkinnedMeshRenderer

Hi,

I am trying to use the EyeRenderer with a SMR. With a normal MeshRenderer I don't have issues, but as soon as I am using a SMR it seems some offsets are not correctly calculated; I can use the Geometry Offsets to adjust the Gizmos without problems, but I don't get any refraction from the Cornea and no Pupil rendering. I guess it's a bug in either EyeRenderer.cs or EyeProperties.hlsl, but after some research in the code I was not able to find it.

Creating custom lips moving playables

Hi, I am wondering how I can create custom lips-moving playables. Seems that you store all key frames of SNAPPERS:Head in Gawain_OneShotAtThis.asset, but how did you generate those key frames? Is there an editor or just hard coding (which is quite impossible)?
Looking forward to your reply:)

SkinAttachmentTarget is cloning full mesh

Hi,

I saw in profiler that SkinAttachmentTarget is cloning and keeping the full mesh; this does not seem to be necessary I guess, as it occupies potentially a lot of memory on GPU.

Enemies?

Hey! I was just wondering when you would be releasing the Enemies demo to the public to see how you did stuff. I remember something about Q3 being the tentative date of release, but just wanted to check. Thanks!

Please allow external update of SkinAttachmentTarget

Hi, thanks for the great work!
Currently, SkinAttachmentTarget is being updated on BeginFrameRendering / EndFrameRendering. While this might work for many use cases, it breaks as soon as overriding a SkinnedMeshRenderer (e.g. for Softbody simulation or Dual Quaternion Skinning). In this case the mesh needs to be changed in BeginContextRendering, e.g. after the skin attachment has been done.
Best solution would be to offer an explicit update mode, where an outside script would trigger the AfterGpuSkinningCallback (or ResolveSubjects). This would not break anything, but would allow more elaborate use cases which go beyond using the outdated linear blend skinning of SMR.

Thanks!

Get IDs for vertices being used in a SkinAttachmentTarget

How would I get the Vertex ID for all vertices being used in a SkinAttachmentTarget? If I toggle 'Show Resolved' in the Editor I can see which parts of the mesh are being used to bind attachments. What I'd like to do is get the Vertex IDs for these and use them in my custom shader. This would allow me to hide the parts of the SkinAttachmentTarget mesh that are occluded by the attachment mesh.

Missing meta file for ShaderGraphs2022 folder

After I installed this 0.2.0 preview package in Editor ver. 2022.2.0.bxx (Editor version doesn't matter in this case. )
Then I got a error message : "Asset Packages/com.unity.demoteam.digital-human/ShaderLibrary/ShaderGraphs2022 has no meta file, but it's in an immutable folder. The asset will be ignored."

I saw the package has a folder call "ShaderGraphs2022" which is missing to load in project. I believe that content the newest shaders from ENEMIES's character.

ERRORS WHEN INSTALLING IN 2022

Library\PackageCache\[email protected]\Unity.Jobs\IJobParallelForDefer.cs(176,85): error CS8377: The type 'U' must be a non-nullable value type, along with all fields at any level of nesting, in order to use it as parameter 'T' in the generic type or method 'NativeList

Skin Attachment - how to control vertex attachment threshold

We're working on an application where we can afford a performance hit in return for increased visual quality, so skin attachment is useful for hair, clothing and more. It also simplifies our asset production pipeline. However, we're having issues when attaching meshes such as these. What would you recommend for situations like this? Is it possible to control the threshold for vertex attachment?

Screenshot 2023-04-20 at 5 09 19 pm

Mesh attachments working, but transform attachments don't if using GPU

We're using SkinAttachments with AttachmentType of Mesh without any problems, but SkinAttachments with AttachmentType of Transform simply won't work. We'd like to use these empty transforms to drive occlusion markers for eyes, as in the Enemies demo, but the attached transforms don't move. Are we missing something?

Mesh attachment updates with target:
Screenshot 2023-07-14 at 4 37 50 pm

Transform attachment does not update position with target:
Screenshot 2023-07-14 at 4 38 00 pm

Skin Attachment settings:
Screenshot 2023-07-14 at 4 38 20 pm

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.