Git Product home page Git Product logo

ossos's Introduction

ossos

npm twitter youtube Ko-Fi Patreon

Character Animation Library

This project is working toward a complete character skinning & animation library for the web. First most, this library is focused on being independent from any rendering engine with examples of how to use it in webgl based engines like threejs. The one big focus is recreating the IK Rig & IK Animations type system that was demoed several years ago from Ubisoft's GDC talk on IK Rigs. With many game engines like Unity and Unreal developing their own IK Rig like systems, this project helps fill the void for web based engines like threejs, babylon, etc. Hopefully with enough help we can create something just as good as the big boys, maybe even better since its free & open source.


Setup

npm install
npm run dev

[ NOTE ] To be able to run the example, you'll need to go into /examples/_res/ and follow the instructions to clone the resource repo. The files are quite large, so they are kept in a seperate repo to keep this project as light weight as possible.

Usage

This example is the basic boiler plate example of how to pull mesh & skeletal data from a GLTF2 file then using a custom THREE.JS Material to render mesh that can be posed or animated.

const gltf = await Gltf2.fetch( '../_res/models/nabba/nabba.gltf' );

//--------------------------------
// Setup Armature
const arm  = new Armature();
for( let j of gltf.getSkin().joints ){
    arm.addBone( j.name, j.parentIndex, j.rotation, j.position, j.scale );
}

//--------------------------------
// Setup Skinning : Matrix
arm.bind( SkinMTX, 0.07 ); 
const mat = SkinMTXMaterial( 'cyan', arm.getSkinOffsets()[0] ); 

//--------------------------------
// Load Mesh
const mesh  = Gltf2Util.loadMesh( gltf, null, mat );
App.add( mesh );

Current Features

  • Armature System built around 'Bones' instead of 'Joints'
  • Several Skinning Plugins
    • Matrices : Typical Skinning, easiest to use in GLSL
    • Dual Quaternions : For better bending of vertices, typically used in places like Disney. It can only handle Rotation & Translation, scaling individual bones don't work.
    • DQ Transform : Experimental & unique to this library. It uses Transform instead of matrices or dualquats to handle the transformation hierarchy & bind pose. When the data is prepared for shader use the worldspace transform gets converted to Dual Quaternions to handle rotation & translation along with an extra buffer that contains scale. The hopes of this hybrid approach is to get the quality of DQ while still being able to scaled individual bones like when using matrix skinning.
  • Bone Springs ( Rotation & Translation )
  • Basic Animator based on Tracks
  • Basic Animation Retargeting for similar skeletal meshes
  • IK Rigs
    • Biped ( Human )
    • Quadruped ( Prototype Phase )
  • IK Animation Retargeting using IK Rigs
  • IK Solvers
    • Aim / SwingTwist
    • SwingTwist - Ends
    • SwingTwist - Chain
    • Limb
    • Arc
    • ArcSin
    • Fabrik
    • Natural CCD
    • Piston
    • Spring
    • Trapezoid
    • ZSolver
    • Catenary / Rope
  • GLTF2 Asset Parsing for cherry picking what you need to load.
  • A few examples using BabylonJS for rendering
  • Several examples using ThreeJS for rendering
    • Some extra fun examples like converting animations to Data Textures
    • Running Full Skinned animation on the GPU with GLSL Example
    • Using Instancing & Data Texture to animate a collection of randomly placed & rotated meshes
  • Ready Player Me Example : Parsing, TPose Generation & Auto IK Rigging
  • Bone Slots : A way to programmically attach assets to bones
  • IK Animation Additives

Future Plans

  • Rewrite IK Rigs
  • Port over starting IK Solvers ( Aim / SwingTwist, Limb, SwingTwistEnds )
  • Rewrite IK Animation Retargeting
  • Port over extra single pass IK Solvers ( Z, Piston, Arc, ArcSin, Trapezoid, Spring )
  • Create an implementation of FABIK
  • Create solver based on Catenary Curve
  • Port over Trianglution Solver ( Alternative to CCD )
  • Port over Natural CCD ( Extended version of CCD )
  • Complete FullBody IK Prototype
  • Revisit FullBody IK, Make it mores stable & user friendly
  • Figure out how to implement VRIK
  • Bone Slots / Attachments
  • Actions or State Machine based Animator
  • Build Examples in other Rendering Engines like BabylonJS
  • Remake Auto Skinning ( Need WebGPU compute shaders for this )
  • Bone Constraints
  • Procedural Animation ProtoTyping
  • Far Future - Create & Share animations with a Web Editor Tool

ossos's People

Contributors

mattrossman avatar sketchpunk avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

ossos's Issues

Why there are "poses" in gltf

"poses" is not a standard field of gltf, so it cannot be read directly in my program. Why not set the data in the TPose directly to the node?

Library build commands don't create `ossos.cjs.js`, must manually rename.

The build commands I use are:

npm run build
npm run build:types

But this creates the following files inside ./dist

image

So I have to either rename the ossos.js manually into ossos.cjs.js or replace every mention of ossos.cjs.js into ossos.js inside package.json. After this step, I can successfully include them in my node project.

Below is the patch. Let me know if I do anything wrong, thank you.
801a3c.patch

[Suggestion] Store resources as submodule

Currently, setting up this project for local testing requires going into examples/_res and following the instructions to clone the resources repository. This can be problematic because:

  • It's an extra step that can be overlooked for contributors
  • The main branch of the repository is not guaranteed to be in sync with commits in this repository

This could be handled more elegantly with the resources repo as a Git submodule of this project.

This would mean that:

  • Both repositories can be cloned in one step with git clone --recurse-submodules
  • The appropriate commit from the resources repository will be versioned with this repository, so examples are guaranteed to work at any point in the project history.

Development of v0.0.4

[ Main Focus ]

  • Animation
    • Blending Animations
    • Learn how to normalize time for smooth blending
    • Figure out how to do phase matching ( https://twitter.com/nan2cc/status/1266792167206338560 )
    • Maybe a framework to handle Blend Trees?
    • Export the results of IK Animation Retargeting. So no need to do realtime ik retargeting all the time.
  • Character controller ( get things to actually move )

[ Secondary ]

  • Bone Springs
    • Not happy with new version, will revisit old prototype version
    • Try to handle Implicit Euler & regular spring force equations, maybe treat them as modes or something.

[ Maybe? ]

  • Make Playcanvas examples

[ Most Likely ]

  • Find & Make usable more models to play with.
  • Try to model my own character

I can't see the examples working from site

Hi:

I tried to see the examples so I clone the repo and follow the installation steps: npm install, npm run dev, I found nothing on localhost:3000, then I tried npm run build:site and npm run preview-site but I found nothing on localhost:5000 then I navigated to http://localhost:5000/examples/threejs/000_gltf2_mesh.html but I got an error on Starter.js at line 51 from console:

Uncaught TypeError: Cannot read properties of undefined (reading 'substring') at Starter.cd2d3b33.js:51

Am I missing some obvious step? How can I see the examples in the browser?

Thanks!

Invalid Retarget transformation on `Armature.offset.set()` for movement scaling & rotation

I'm using the 003_retarget_and_springs.html example with a custom mixamoRig model. The tall/left model is the source, while the short/right model is the target.

If I remove the arm.offset.set() line, this is what I get:

2023-08-19.00-58-46.mp4

Meanwhile, after re-enabling the line:

2023-08-19.00-59-18-00.00.00.000-00.00.08.244.mp4

The target is actually moving downward just a tiny bit. So here you can see that somehow that offset.set() somehow manages to:

  • Rotate the armature properly, but not the movement direction (it goes downward).
  • Scale the armature properly, but not the movement magnitude (it goes much slower).

To emphasize the downward movement direction, here's the clip when I set the offset without the skin.scale (so just rotation & position):

2023-08-19.16-06-39.mp4

Spring bone clamping?

I was wondering if your spring bone implementation should have a maximum stretch clamp from looking at your demo... ;-)

image

Unless of course you left that in just for amusement. This is when the walk animation resets to the original position causing the bust spring bone to snap back due to the super-fast movement back to the origin. (I have had the same problem with hair in other spring bone solutions - moving the character in an animation sequence to a new position causes the hair to think the body moved at 100,000 miles per hour in one tick, which then caused the hair to take a while to settle back down again.)

Using GLTF2 loaded files with <primitive object=...> in ThreeJS Fiber

I am trying to use the current version of the OSSOS library with ThreeJS to take advantage of the retargeting and IK support. I am doing so inside a React app using threeJS Fiber. So I have a <primitive object=...> that I used to use the threejs GLTF loader to load a file and then grab the .scene member to use as the root character.

But the data structure returned by the OSSOS library GLTF2 file loader appears to be a bit different. For example, there is .json instead, and .json.scene exists but has the value of 0, which is not a model.

Can I use OSSOS to get an object compatible with the <primitive object=...> markup used with threejs and fiber?

Note: I am not a deep expert. I was trying to use the threejs retargeting sample code, but it did not work for me. I want to take an animation clip from one model and retarget it to a different bone structure. My fallback is to try and reimplement the whole bone rotation logic you described on the video (which conceptually made sense... but I would rather not have to code it, lol!)

I was trying to apply Mixamo animation clips to a VRM model created by VRoid Studio in case that makes a difference.

Thx for any advice! I could not find much in the way of docs to work it out sorry.

Running examples

I tried to run the examples after cloning and running npm install, but I'm not sure what the process is to set up the dev environment.

I'm getting this error when I try to run build or preview-site

failed to load config from /Users/josh/Downloads/ossos-main/vite.config.js
error when starting preview server:
Error: Build failed with 1 error:
**node_modules/esbuild/lib/main.js:1336:27: ERROR: [plugin: externalize-deps] Missing "./dist/plugin.js" export in "vite-plugin-list-directory-contents" package**
    at failureErrorWithLog (/Users/josh/Downloads/ossos-main/node_modules/esbuild/lib/main.js:1575:15)
    at /Users/josh/Downloads/ossos-main/node_modules/esbuild/lib/main.js:1033:28
    at runOnEndCallbacks (/Users/josh/Downloads/ossos-main/node_modules/esbuild/lib/main.js:1447:61)
    at buildResponseToResult (/Users/josh/Downloads/ossos-main/node_modules/esbuild/lib/main.js:1031:7)
    at /Users/josh/Downloads/ossos-main/node_modules/esbuild/lib/main.js:1143:14
    at responseCallbacks.<computed> (/Users/josh/Downloads/ossos-main/node_modules/esbuild/lib/main.js:680:9)
    at handleIncomingPacket (/Users/josh/Downloads/ossos-main/node_modules/esbuild/lib/main.js:735:9)
    at Socket.readFromStdout (/Users/josh/Downloads/ossos-main/node_modules/esbuild/lib/main.js:656:7)
    at Socket.emit (node:events:513:28)
    at addChunk (node:internal/streams/readable:315:12)

Ultimately, I'm just trying to check out the running examples. Are they hosted somewhere?

Error following setup instructions during `npm run dev`

I'm trying to run this project locally. I followed the setup instructions including cloning the resources repository, but I'm facing an error when running npm run dev:

image

Looking at the docs for vite-plugin-list-directory-contents, it seems the import statement in this project's Vite config is the issue.

The docs say:

import { directoryPlugin } from 'vite-plugin-list-directory-contents';

Compared to the usage here:

import { directoryPlugin } from 'vite-plugin-list-directory-contents/dist/plugin.js';

If I remove /dist/plugin.js from the import path, then I can run the npm run dev command successfully.

I wonder if this behavior is caused by different versions of dependencies getting installed on my end. I notice there is no lockfile being tracked in this project, perhaps that would reduce the chances of contributors facing version mismatches.

demo resourses not work in example htmls

I cloned the res pack and found some of them works well but more of them even not work.

1,works well:
tina / ligerZero / toru / trex / vegeta / xboxavatar_dehvor

2, works,but renders only one part of the gltf.scene
ahsoka_tano / cattus / tyranitar
all these 3 gltf files has moran than one part , the gltf parse function ignored the other parts?

3, not work
1),at_rt /cherubbotpm1 / gine / jinx / ronin /
everything is OK in blender but shows the following error:
Uncaught (in promise) TypeError: Cannot read properties of undefined (reading 'idx')
Uncaught (in promise) SyntaxError: Unexpected token a in JSON at position 6
Uncaught (in promise) TypeError: Cannot read properties of null (reading 'joints')

2), readplayerme
everything is OK in blender but shows the following error:
UtilArm.js:34 Uncaught (in promise) TypeError: Cannot read properties of null (reading 'getSkin')

3),companioncube / sword_war
gltf file has no amature and it shows error:
UtilArm.js:34 Uncaught (in promise) TypeError: Cannot read properties of null (reading 'getSkin')

4), suzanne
gltf file can not importted into blender and it shows the following error:
Uncaught (in promise) SyntaxError: Unexpected token a in JSON at position 6

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.