Comments (20)
Hey! Unfortunately depth and RGB export functionality is only implemented for Azure Kinects at the moment.
More sensors will be added in the future, including backwards compability for the Kinect v2, but this might take a while.
If you only need depth and rgb images however, it should probably be pretty doable to program a small python program which does this, with the Kinect v2 Python library. https://github.com/Kinect/PyKinect2
from livescan3d.
Thank you very much for sharing. I am very confused about the knowledge of calibration. On LiveScan, I can calibrate two cameras to the same world coordinate system based on the calibration paper, but I don't understand how to calibrate the two cameras using other methods. Because I need to process the images captured by the two cameras first, and finally need to fuse the point clouds captured by the two cameras, I am worried that the ICP fusion effect will be very poor if I don't calibrate them later
from livescan3d.
Hey! I see, yes that makes everything a bit more complicated. The new version of LiveScan3D im currently working (https://github.com/BuildingVolumes/LiveScan3D) on allows for this workflow, but unfortunately only for Azure Kinect devices. I will add support for the Kinect v2 at a later point, but not in the near future, as I don't have the resources to do this at the moment.
I'm also not aware of alternative software that allows for this, but maybe Brekel Pointcloud v2 is able to do this? It has a free trial:
https://brekel.com/pointcloud-v2/
from livescan3d.
Thank you very much for sharing. I'll try it out
from livescan3d.
Wait a moment, are you saying that Azure Kinect can output point clouds and images simultaneously on Livescan? So is the output image a two frame image without fused two cameras.
from livescan3d.
Not simultaneously, you capture either raw images or pointclouds. But I have a tool with which you can convert these raw images into a pointcloud again after the capture. This is no special hardware feature of the Azure Kinect, the Kinect v2 could do the same, but I developed the new version of LiveScan for the Azure Kinect only (so far), as it provides better image quality.
from livescan3d.
Sorry, I may not understand what you mean. Are you saying that the new version of Livescan can allow Azure Kinect to capture images and convert them into point clouds? I understand this part. So, will the new version of Livescan output images or point clouds
from livescan3d.
Sorry for the confusion. When you capture, you only capture the color and depth images. After your capture has finished, you then use another tool to convert these images to pointclouds. So you can have both as an output, but the pointclouds will only be output after the capture and conversion.
from livescan3d.
I understand. Is the output depth image and RGB image similar to the previous version of Livescan point cloud output? After calibration, the left half image and the right half image are combined to form a complete image. Is this what it looks like? Is there any example of an image? I may consider purchasing this type of camera
from livescan3d.
This is an exemplary output of one frame of a capture with five cameras. You get a .jpg with the color picture, and a 16-bit .tiff picture with the depth information for each camera. Additionally metadata about the camera intrinsics, calibration and frametiming are saved to allow for the pointcloud reconstruction later on.
After pointcloud reconstruction, the pointcloud looks like this (this pointcloud is from a different frame of the capture!):
pointcloud.zip
from livescan3d.
This point cloud and the image are not in the same frame, but thank you very much for sharing. I am a second year graduate student, and our research team all use your software. Everyone loves this software very much, and I am also very grateful to you. I have another question, which is how close the Azure Kinect is to the object and how good the collection effect is. Because my project is modeling newborns in an incubator, the point cloud effect of my KinectV2 is very poor due to being too close to the incubator, and it even splits. Due to environmental factors, I can only keep the camera at this height for collection, so I am not sure if Azure Kinect can support static distance shooting
from livescan3d.
Oh wow interesting project! Great to your that this software is being used a lot by your team!
So there seems to be a lot of factors involved here. Generally, the Azure Kinect has a much higher pointcloud quality than the Kinect v2. But I guess you are shooting through the incubator glass? This might introduce refraction effects, which could lower the capture quality. Additionally, the infrared light might be partially blocked by the glass, depending on the material. Maybe you can try shooting with a puppet at first, without any glass, to see how much it affects the capture.
Also the quality of the calibration has a high impact on the capture quality. Unfortunately the calibration quality is not very high at the moment, but we're working on improving this!
from livescan3d.
Yes, the shooting process will be affected by the glass obstruction, I used two KinectV2 can only shoot from the limited space that can be opened on both sides of the incubator, which is very difficult, I considered using two RealSense to record video in the incubator, but I still don't know how to calibrate the two cameras online.
from livescan3d.
Hey! Yeah calibration is unfortunately not super trivial. I don't know how experienced you are with programming in general, but you may need to write your own software for this.
You'll need a marker, take captures of the marker with both cameras and then extract the position and rotation of the camera. It is certainly an advanced task, but possible with libraries like OpenCV.
https://docs.opencv.org/4.x/dc/dbb/tutorial_py_calibration.html
Here is a tutorial which I can recommend!
from livescan3d.
My coding skills are not sufficient to develop calibration software, I am sorry. Recently, I have been trying to switch to Azure Kinect, so that I can ensure the output of images. Besides, it is smaller than Kinect V2. I am considering placing it in an insulated box, but I am not sure if I can successfully calibrate and record point clouds at a relatively close distance. Additionally, when compiling the Livescan-development software, after running it, I only have the client and not the server. I am very frustrated.
from livescan3d.
Please just use the Pre-Release downloads available here :)
https://github.com/BuildingVolumes/LiveScan3D/releases/tag/v1.2.alpha1
If you encounter any issue, it would be great if you could open an issue in that repository, happy to help!
from livescan3d.
I'm sorry, I encountered another difficulty. Instead of using Azure to collect data, my teacher instructed me to use ROS to connect two RealSense cameras. Through Python code, I calculated the distances between corners of a checkerboard pattern, derived the relative transformation matrix from camera 2 to camera 1's coordinate system, and then reconstructed it into a point cloud. However, after visualizing the point cloud, I noticed cracks in overlapping areas and splitting at the top. I specifically need the point cloud of the chest area. I applied ICP and filtering to the point cloud, but the cracks persist. I don't understand the reason; previously, using LiveScan produced separate left and right half-point clouds that aligned perfectly.
from livescan3d.
Hey!
Just as a disclaimer: As this is outside of the scope of Livescan3D support, I won't be able to give you detailled feedback.
This can have multiple reasons, for example imprecise depth data from the cameras, or unprecise calibration. You could try taking multiple samples with the checkerboard and then averaging them.
from livescan3d.
Do you mean to calculate the relative change matrix by averaging multiple sizes of chessboards
from livescan3d.
Hey! No I mean not multiple sizes, but moving the chessboard into different positions and rotations (about 20 different poses), calculating the relative transformation matrix for each pose, and then averaging all matrices into one. When you take only one sample, the calibration can be inaccurate.
from livescan3d.
Related Issues (20)
- LiveScanClient build from source code failed for Azure Kinect. HOT 2
- Azure data corruption, scan merging, color problems, player crashing HOT 4
- Use Hololens 2 sensor data as input HOT 2
- Using AzureKinect, PointCloud is upside down HOT 3
- .bin file in sync while ply files out of sync for two camera recordings HOT 3
- the problem of settings HOT 9
- Place the calibrated cameras in Unity HOT 6
- hi~Thank for share your project , I have a problem is that this project can run on Visual studio 2022 ? HOT 4
- Application's validity HOT 1
- "Kinect.h" no such file or directory
- color mismatch HOT 10
- Problem with Caliberation HOT 51
- Save data in .ply format HOT 6
- How to compile .exe for Azure Kinect? HOT 2
- Problems about showing .ply in LiveScanPlayer.exe HOT 9
- Show nothing when I click 'show live' in the server.exe HOT 5
- Could we save the depth map image at the same time? HOT 4
- Xbox 360 Kinect or Xbox One only? HOT 1
- adding audio HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from livescan3d.