lumalabs / luma-web-examples Goto Github PK
View Code? Open in Web Editor NEWLuma Web Examples, use lumalabs.ai captures directly in your three.js or other WebGL projects!
Home Page: https://lumalabs.ai/luma-web-library
License: MIT License
Luma Web Examples, use lumalabs.ai captures directly in your three.js or other WebGL projects!
Home Page: https://lumalabs.ai/luma-web-library
License: MIT License
the cloud point change to the splats effect is awesome! but i don't konw how to get these code.Anyone can help??
I was expecting to find the implementation in this repo. Thanks! Looking forward to contribute.
I mean, that I can get genie's interface in my application to obtain text to 3d files, stl/obj/glb, etc.
Hi,
I am using the Luma WebGL library in one of my projects. I have noticed that on some older devices the website becomes really slow once the Luma splat has been rendered. I believe it is because of the number of particles that are rendered since when I upload a higher resolution video the lag becomes worse.
Do you have a way of easily reducing the number of particles for example by providing a parameter such as particleDensity
to LumaSplatsThree
? In case there isn't such a functionality, do you plan to add one or is there a way to achieve this in another way?
Does LumaSplatsThree support loading external splat files ?
E.g. I crop unnecessary points in https://playcanvas.com/supersplat/editor and want to import edited splat back to Lumalabs Js library. Is that posible?
I find that in the example of the offical luma website I can remove the camera by clicking somewhere, can I achieve this using the webgl library?
attempting to compile the demo scripts and getting this error ~/luma-web-examples/node_modules/@lumaai/luma-web/misc/inline-worker-plugin.d.ts' not found. doesn't look like the misc
directory is even in the package. wondering if this is something I need to manually create
Hi all,
I just successfully created capture by calling Luma AI API and got it shown in my Google browser through given url, but I do not find any api call can help with downloading this capture to my local machine, is there any api doc can assist in that?
Thanks!
Is there support for user-defined masks within the LumaWebGL renderer? i.e. can we use a given mesh to define a group of points in the splat that are part of the same group & then toggle the visibility of that group in a Three.js scene?
Congrats for the great work!
I did run some captures in WebXR with my Quest3 and I was surprised it runs! It's smooth but get slow when looking at detailed objects in the scene (main subject).
Is there a way to optimize the splat or add some options in the viewer to make the rendering faster for Mixed reality devices?
Thanks!
Hi, maybe noob question is it possible to use luma with webgpu? Trying threejs WebGPURenderer but luma fails to render with
Cannot read properties of undefined (reading 'isWebGL2')
at R.onBeforeRender (@lumaai_luma-web.js?v=e24c7599:917:27)
Branch and code link: https://github.com/xleepy/threejs-app-sandbox/blob/webgpu-api/src/App.tsx#L17
I have a 3D Gaussian Splat output as a .ply file, generated from the original code accompanying the 3DGS publication. Is it possible to load that .ply into the LumaWebGL Renderer?
Default raycaster doesn't detect splat mesh.
This feature would be great for camera navigation tools like click to move. It's implemented in example below, so shouldn't be hard to add to this library.
Example: HOLLYWOOD
useEffect(() => {
Object.values(materials).forEach((material) => {
material.depthTest = false;
material.needsUpdate = true;
})
Object.values(nodes).forEach((node) => {
node.renderOrder = 9000;
})
}, [materials, nodes])
I've tried stuff like this to no avail.
Hi!
When installing npm i @lumaai/luma-web
aside [email protected] I got the following error:
Could not resolve dependency: npm WARN peer three@"^0.157.0" from @lumaai/[email protected]
This is because your package.json (https://www.npmjs.com/package/@lumaai/luma-web?activeTab=code) :
"peerDependencies": {
"@types/three": "^0.157.2",
"three": "^0.157.0"
}
Can you update it to "three": ">=0.157.0"
or "three": "*"
For thoses who cannot wait, bad workaround could be npm i @lumaai/luma-web --force
Thanks!
Hi,
I tried installing luma webGL viewer in React 19 RC.
I got error with npm package:
⨯ node_modules\react-reconciler\cjs\react-reconciler.development.js (498:1) @ $$$reconciler
⨯ TypeError: Cannot read properties of undefined (reading 'ReactCurrentOwner')
var StaticMask = LayoutStatic | PassiveStatic | RefStatic;
497 |
>498 | var ReactCurrentOwner = ReactSharedInternals.ReactCurrentOwner;
^
499 | function getNearestMountedFiber(fiber) { ...
What is the solution here? Seems like it's not fixable in this repo? What should I do to avoid this?
I've been experimenting with integrating splats from the lumalabs app into a threejs experience, and it's amazing! I found a splat editor online and would love to use this to edit my splats. It looks like I need to upload a file, and just provide a link to the capture as in the webgl library. I was wondering if there's a way to get the file from the luma app downloaded so I can edit it before importing it to my threejs app?
Thanks!
Hello.
I am enjoying using Luma AI.
I am posting this issue because I have tried the WebGL library for Luma AI, but I have encountered some problems.
I wrote code based on three.js and the Luma WebGL example, but when I ran the code, nothing was rendered in the browser and I got a lot of errors in the console on the browser.
Below is the full text of the errors.
Uncaught TypeError: Cannot read properties of undefined (reading 'equals')
at setBlending (three.module.js:23355:19)
at Object.setMaterial (three.module.js:23382:6)
at WebGLRenderer.renderBufferDirect (three.module.js:29149:10)
at renderObject (three.module.js:29945:11)
at renderObjects (three.module.js:29914:6)
at renderScene (three.module.js:29777:41)
at WebGLRenderer.render (three.module.js:29593:5)
at animate (main.js:31:12)
I think it might be a Luma AI WebGL issue, since the problem did not occur with the normal three.js example. Any advice would be appreciated.
Here is some information about my environment
code is here
{
"name": "luma-app-vanila",
"private": true,
"version": "0.0.0",
"type": "module",
"scripts": {
"dev": "vite",
"build": "vite build",
"preview": "vite preview"
},
"devDependencies": {
"vite": "^5.0.0"
},
"dependencies": {
"@lumaai/luma-web": "^0.1.14",
"three": "^0.159.0"
}
}
$ bun run node -v
v20.5.1
$ bun --version
1.0.7
(This information is output to the browser console)
LumaSplatsThree git version #8f3367e
This is my unsuccessful attempt:
https://glitch.com/edit/#!/guttural-navy-lentil?path=index.html%3A14%3A9
Is there any basic example I can look at? Thanks
I am getting this error when trying to use Luma splats with React Three Fiber
luma-web.module.js:60 Uncaught TypeError: this.material.updateUniformsAndDefines is not a function
at a.onBeforeRender (luma-web.module.js:60:354411)
at renderObject (three.module.js:29444:1)
at renderObjects (three.module.js:29434:1)
at renderScene (three.module.js:29303:1)
at WebGLRenderer.render (three.module.js:29121:1)
I am following the example you have exactly here
Any idea what might be causing this error?
how would one go about raycasting the splat?
I am trying to get the 3D position of a mouse click inside of the LumaSplats mesh.
This is part of the code that I have written:
function getClicked3DPoint(evt: any) {
// evt.preventDefault();
const canvasPosition = canvasRef.current?.getBoundingClientRect();
console.log('WTF', scene.children)
mousePosition.x = ((evt.clientX - canvasPosition!.left) / canvasPosition!.width) * 2 - 1;
mousePosition.y = -((evt.clientY - canvasPosition!.top) / canvasPosition!.height) * 2 + 1;
rayCaster.setFromCamera(mousePosition, camera); //cameraRef.current!);
let intersects = rayCaster.intersectObjects(scene.children, true);
if (intersects.length > 0)
console.log(intersects[0].point)
return intersects[0].point;
}
<Canvas
ref={canvasRef}
frameloop={frameLoop}
gl={{
antialias: false,
}}
camera={{
position: [0, 0, 0],
}}
>
<group
dispose={null}>
<lumaSplats
onPointerMissed={() => console.log("you clicked, but hit nothing at all")}
onClick={(e) => {
console.log(e.point) // this is the coordinates for the event, eg. mouse click
getClicked3DPoint(e)
}}
onLoad={() => {
setIsLoading(false);
if (camera360Ref.current) {
camera360Ref.current.object.position.x = -5;
camera360Ref.current.object.position.y = 0.4;
camera360Ref.current.object.position.z = 7;
}
}}
loadingAnimationEnabled={true}
semanticsMask={LumaSplatsSemantics.ALL}
// onBeforeRender={}
enableThreeShaderIntegration={false}
particleRevealEnabled={true}
source={src}
position={[0, 0, 0]}
scale={5}
rotation={[0, 0, 0]}
/>
<CustomLine start={[0,0,0]} end={[1,1,0]} />
</group>
</Canvas>
I have created the following luma capture: https://lumalabs.ai/capture/71f64429-3872-417a-a67c-a8b6c6ab16d3
As you can see it is a capture of a normal room. What I want to achieve is to draw lines on the splat. For this, I want to click on the canvas, this will cast a ray to the first intersecting part of the splat. I will take the coordinates of this intersection and then by clicking on another part of the splat I want to draw a line between these two points.
However, most of the times the event on onPointerMissed is being activated and thus no intersecting point is found. Do you have any idea why this is happening and what I can do in order to fix this?
Hello, I am trying to play around with the new luma library. However, when I try to load a model the fps drops a lot and the website pretty much becomes unusable. I have the following code:
import { Object3DNode, extend } from '@react-three/fiber';
import { LumaSplatsThree } from "@lumaai/luma-web";
// Make LumaSplatsThree available to R3F
extend( { LumaSplats: LumaSplatsThree } );
declare module '@react-three/fiber' {
interface ThreeElements {
lumaSplats: Object3DNode<LumaSplatsThree, typeof LumaSplatsThree>
}
}
import { LumaSplatsSemantics } from "@lumaai/luma-web";
import React from 'react';
import './LumaSplatsReact';
export const LumaModel = () => {
return <>
<lumaSplats
semanticsMask={LumaSplatsSemantics.ALL}
// onBeforeRender={}
enableThreeShaderIntegration= {false}
source='https://lumalabs.ai/capture/ca9ea966-ca24-4ec1-ab0f-af665cb546ff'
position={[1, 0, 0]}
scale={2}
rotation={[0, 0, 0]}
/>
</>;
}
import React, { useRef, useState } from 'react';
import {OrbitControls, PerspectiveCamera} from '@react-three/drei';
import { Camera, Scene, WebGLRenderer } from "three";
import { OrbitControls as OrbitControlsStdLib } from 'three-stdlib';
export type DemoProps = {
renderer: WebGLRenderer,
scene: Scene,
camera: Camera,
controls: OrbitControlsStdLib,
}
export const ModelViewer = (props: {
modelReactFn: React.FC<{}> | null,
}) => {
let controlsRef = useRef<OrbitControlsStdLib | null>(null);
return <>
<PerspectiveCamera />
<OrbitControls
ref={controlsRef}
autoRotate={false}
autoRotateSpeed={0.5}
enableDamping={false}
keys={{
LEFT: 'ArrowLeft', //left arrow
UP: 'ArrowUp', // up arrow
RIGHT: 'ArrowRight', // right arrow
BOTTOM: 'ArrowDown' // down arrow
}}
makeDefault
/>
{props.modelReactFn && <props.modelReactFn/>}
</>
}
And I am trying to load the model onto a canvas like this:
<Canvas shadows={false}
camera={{
position: [-6, 7, 7],
}}>
<ModelViewer
key={0}
modelReactFn={LumaModel}
/>
</Canvas>
Can you point me into exactly why my website is slowing so much? Because from your demo I can see that it is indeed possible to load the model and still have a good website performance.
Thanks in advance!
First of all, thanks for this great library and docs! Two quick questions:
I have given it a try and created an online Luma SPLAT Viewer using this library and three.js r159.
It currently only supports URL loading and it does work when using Luma URLs, like this:
https://lumalabs.ai/capture/ca9ea966-ca24-4ec1-ab0f-af665cb546ff
The issue can be seen in the attached picture below when I use a URL like this:
https://raw.githubusercontent.com/GitHubDragonFly/GitHubDragonFly.github.io/main/viewers/examples/legobrick.splat
The library is treating this URL as a "folder", or a location that hosts multiple files, and is trying to get 1 bin and 3 json files out of it.
If I try to use the WebGL Gaussian Splat Viewer then it works, and the following URL can just be entered in a browser:
https://antimatter15.com/splat/?url=https://raw.githubusercontent.com/GitHubDragonFly/GitHubDragonFly.github.io/main/viewers/examples/legobrick.splat
This legobrick.splat
file is just a lego brick made of points and was created from a ply file which was converted to splat by using the above mentioned WebGL Gaussian Splat Viewer.
The library also does not seem to be able to handle local blob URLs either, created with URL.createObjectURL( splat_file )
, which is another issue.
I might be doing something wrong all together but just looking for a second opinion.
Hello,
Thank you for this library, gaussian splatting are progressing fast !
It seems that this repository lacks a proper mention of the licence applied. Could you specify it in the README and add a LICENSE file so that we know if it is opensource or not ?
The licence of the code can be different from the terms and conditions of the Luma service of course.
I have a working project using a Luma splat URL, but I can't make it work with any of the downloaded files (.ply, .luma, .splat)
source can be either of:
- URL to a capture on lumalabs.ai
- path to a luma splats file or folder containing a luma splats artifacts
Am I doing something wrong?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.