Comments (5)
Hi @adrida thanks for the interest.
Currently, 4K4D focuses more on the novel view synthesis side and uses a different rendering pipeline than that of the traditional triangle meshes.
Thus for creating 3D assets directly (with such a small view count), you should consider looking into other 3D/4D neural reconstruction methods that focus on surface quality and use human priors (with SDF field or directly optimizes meshes), like AniSDF or Relightable Avatar
from 4k4d.
I see, thanks a lot for your answer and for the references I will check them out. Any idea on projects that would export 3d animated assets of large scenes with multiple persons?
I could have more video angles as input if needed and I don't necessarily need to be able to edit the 3d asset. I am looking to be able just "replay" the scene from different angles and navigate in the 3d space (a bit like static nerf approaches where you can export the 3d reconstruction of an appartement in VR and be in it).
from 4k4d.
Any idea on projects that would export 3d animated assets of large scenes with multiple persons?
Ah, for multi-person reconstruction, I recommend checking out CloseMocap and MultiNB (the first news of EasyMocap).
I could have more video angles as input if needed and I don't necessarily need to be able to edit the 3d asset. I am looking to be able just "replay" the scene from different angles and navigate in the 3d space (a bit like static nerf approaches where you can export the 3d reconstruction of an appartement in VR and be in it).
For replaying the reconstruction, there are generally two approaches:
- Support PCVR in the specific rendering algorithm (like a connector with Unity VR or just natively supports communication with VR devices). An example is the VR-NeRF method you mentioned. We're planning on adding this functionality to EasyVolcap, on which 4K4D is built.
- Export the reconstructed sequence to a renderable format and replay it (like exporting a mesh sequence). This might come at some significant quality loss.
from 4k4d.
I see, thanks a lot for sharing those projects I will take a look.
Any way I could help for the VR feature you are planning on adding to EasyVolcap? Your work is very inspiring, and I see a lot of great potential applications especially in AR/MR.
If we could reconstruct in real time a dynamic scene and render it through a Meta Quest3 or an Apple Vision Pro, it would open-up limitless possibilities.
I am not an expert in 3d reconstruction but from the few papers/surveys I have been reading, I feel like the state of the art is very close to achieving such a thing.
from 4k4d.
Indeed, the field is advancing fast and I also think that future is not far away.
Any way I could help for the VR feature you are planning on adding to EasyVolcap?
I couldn't think of any specific problem to solve as of right now, but we are always looking forward to a PR of any kind. Feel free to contribute!
from 4k4d.
Related Issues (20)
- Are there any plans to release missing files? HOT 2
- CUDA out of memory HOT 2
- About real-time rendering HOT 1
- Is there any plan to add the training code? HOT 2
- Error in opengl HOT 3
- How to convert CamPose.inf to extri.yml? HOT 1
- An error occurred(Segmentation fault) while loading the model HOT 17
- Test view HOT 3
- Question about KPlanes result of DyNeRF dataset? HOT 4
- Questions about the mask loss in the paper HOT 3
- Minimal Dataset extract_images 解帧发绿 HOT 1
- Error during rendering HOT 12
- No mask directory in Mobile-stage dataset HOT 7
- EasyVolcap can't import xxx HOT 2
- Get in trouble with pytorch3d 's installing and tiny-cuda-nn 's compiling. HOT 2
- Can it render poses that have never appeared before? HOT 2
- Licensing HOT 4
- easyvolcap.utils.console_utils -> inner: Runtime exception: 'SuperChargedR4DV is not in the samplers registry' HOT 8
- Bad results than expected. HOT 10
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from 4k4d.