The pixel value of captured depth sensor data (pgm files) about hololensforcv HOT 8 CLOSED

I find that in 'HoloLensForCV\Samples\SensorStreamViewer\FrameRender.cpp' the code takes pixel data in the range of 0 to 4000 as valid and takes each pixel data divided by 1000 as depth. And for short throw depth data, the valid depth range is 0.2 to 1.0. However, note that data in pgm has issue of endian, and you should interchange high 8 bits with low 8 bits to get the right data according to the comment of jiying61306 in issue #19. Maybe this is the result of storing 16 bits data with uint8_t vector in the code of HoloLensForCV\Shared\HoloLensForCV\SensorFrameRecorderSink.cpp.

from hololensforcv.

@cyrineee , as @Huangying-Zhan referred above, the raw pixel value after the endian correction is representing the distance between the depth camera and the object taken in the depth map in millimeters. Since the HoloLens coordinate unit uses the meter as the default (to sure, Unity-based does), we divide the value by 1000 to convert the millimeter to meter.
To convert the pixel in depth map to 3D coordinate, please refer #63. Hope this helped you.

from hololensforcv.

@zwz14 Thank you for the information. I will investigate it based on your comment and post the result. Thanks!

from hololensforcv.

YES!! It worked! The captured sensor data is following Little-Endian. It is required to swap byte order to get correct intensity, e.g. from 0xFF00 to 0x00FF. Now, I got the picture as below. The farther distance got higher intensity. Thank you very much @zwz14 !

from hololensforcv.

In case people still have questions with this issue, please refer to here for an example script that solve the problem.

from hololensforcv.

@Joon-Jung to calculate the distance from object why we divide by 1000 ?
#19 (comment)
is the values in the pgm files refering to depth or distance ?

from hololensforcv.

@Joon-Jung Thaaaaaaanks a lot for your response.
i used the pgm2distance function to calculate the distance from .pgm files in long_throw_depth.tar but i got a distance matrix with the same values for all values of the matrix and for all pgm files ,is there something wrong ? :
def pgm2distance(img, encoded=False):
distance = np.zeros((img.shape))
for y in range(img.shape[0]):
for x in range(img.shape[1]):
val = hex(img[y, x])
val = str(val[2:]).zfill(4)
val = val[-2:] + val[:2]
distance[y, x] = float(int(val, 16))/1000.0
return distance

Another question if it's possible to get the distance of rgb images ?(like we launch the streamerPV on hololens and we receive with sensor_receiver.py the images (i want to calculate the distance of the objects from hololens in this images).

Thanks in advance .

from hololensforcv.

Hi @cyrineee,
I suggest to inspect the .pgm file with image viewer like ImageJ to make sure what you captured and test your code with debugger to inspect the numpy array. I could not see any critical mistake that is making you get a same values from one .pgm file from your code. In my case, I manually swap endian using ImageJ's plugin and run a Python script for making cloud points with short throw.
For using long throw, please have a look on this before you commence.

For your second question, it would be hard to get distance solely based on one RGB image, unless you have something to reference, such as markers (aruco, artoolkit etc). If you want to go with markerless, you might need to (1) align depth camera and RGB camera through coordinate transformation; or (2) use stereo camera method through two VLCs to triangulate the distance like this project's aruco example.

from hololensforcv.