decadenza / directstereorectification Goto Github PK

Hi

Thanks for your great work. And I'm looking forward to your paper.

I have a question about the stereo rectification between pinhole and fisheye.
Is there possible to use your algorithms to realize stereo rectification between pinhole and fisheye?

If possible ,how could i modify your code？

Thanks

1. I have both an X and Y offset between cameras on a fixed rig, where one camera is fronto-planar and the other is at a ~20^ angle relative to the first. Can you confirm this is suitable for DSR?

I see that, unlike OpenCV methods, DSR supports any two arbitrary camera translations, as noted in #5, and it seems camera normals can differ, as noted in #6.

2. Given the camera rotation above, we have less overlap that we'd like between views. This is true for chessboard calibration but also capture, since I've found that rectification towards the image edges using SimpleStereo+DSR has unavoidable disparity, as noted in #2.

Given this tricky rig, it would be helpful to be able to use self-calibration of some kind as an input to DSR. Is there anything recent you can recommend or a lead to follow?

When I run the example.py on my synthetic data using ground-truth extrinsics, I get rectified results that look stretched out horizontally.

On this image you can see

• the DirectStereoRectification results in the first row,
• the cv2.stereoRectify results in the second row,
• and the respective disparity maps in the third row (in this row, both the left and right image correspond to the left image's disparity map).

I'm mostly confused about why the DirectStereoRectification results look horizontally stretched compared to those by stereoRectify, as the latter seem to match the scene geometry more closely. Am I using something wrong here, or is this expected?

Thanks for your intriguing work, continuing in the footsteps of Loop & Zhang.

Could you provide a suitable conversion for calibrated stereo camera data to the left+right extrinsic matrices you require as input to example.py -- for example, using OpenCV's cv2.stereoCalibrate output?

Hi

I have a question about the stereo rectification with different x,y,z.
As i know, opencv.stereorectify only support the stereo rectification which only have x_difference or y_difference.
Can i realize stereo rectification for camera pair with diffrent x, y, z with your code?
For example, the cameras are staggered in a specified position. One camera is high and the other is low.
If not, can you give me some suggestion about code modification?

Thanks

@decadenza Hi, thank you very much for sharing your code! When I want to read your paper, I failed to find it. So can you show me where I can get it?

I'm very intrigued by your paper and this method but when I try to run your code on my own images, I get incorrect results. I'm using synthetic images rendered from Blender. The scene and stereo camera rig looks as follows.

When I run example.py using my own images, intrinsics matrices A1, A2, and extrinsics matrices RT1, RT2, I get this incorrect result:

When I switch the images, such that img1 becomes the image taken by the right camera and img2 becomes the image taken by the left camera (leaving the matrices as before!), I get this result where the images appear to have been "straightened out", yet they still don't line up w.r.t. their y-values.

Moreover, this does not align with your example use, where img1 is the left image and img2 the right image:

I'm rather confused by this, and I'm wondering whether my extrinsics matrices are following a different convention than the one you intend. However, in your paper you state

Call $\mathbf{o}_1 = -\mathbf{R}_1^{-1} \cdot \mathbf{t}_1$ the position of the optical center in the WCS.

You also write

Let $\mathbf{A}_1 \in \mathbb{R}^{3 \times 3}$ be the intrinsic matrix of the left camera, with $\mathbf{R}_1 \in \mathbb{R}^{3 \times 3}$ and $\mathbf{t}_1 \in \mathbb{R}^{3 \times 3}$ its rotation matrix and translation vector, respectively, describing the position of the first Camera Coordinate System (CCS) with respect to World Coordinate System (WCS)

which, to me seems to characterise $\mathbf{t}_1$ as the position of the first camera's optical center, expressed in world coordinates, which seems to be in conflict with "the position of the optical center in the WCS" actually being $\mathbf{o}_1 = -\mathbf{R}_1^{-1} \cdot \mathbf{t}_1.$ In my understanding, $\mathbf{t}_1$ describes the world origin expressed in the coordinate system of the left camera, i.e. camera 1. Am I mistaken there?

In any case, if I do the sanity check to compute

R1 = RT1[:3, :3]
t1 = RT1[:3, 3]

o1 = -R1.T @ t1

I do indeed get that o1 matches the world coordinates of the camera centre (bpy.data.objects.get("cam_L").matrix_world.translation). In case it's relevant, the extrinsics matrices have been extracted via

# Get cameras
cam_L = bpy.data.objects.get("cam_L")
cam_R = bpy.data.objects.get("cam_R")

# Obtain extrinsic matrix
RT1 = np.array(cam_L.matrix_world.inverted())
RT2 = np.array(cam_R.matrix_world.inverted())

What might I be doing wrong here? I'd appreciate any advice or guidance. Thank you! :)

Hi, If I get two images not perfectly rectified, i.e., there are negative disparities, is it possible to rectify there two images with this code?

Hi

Thanks for your kindly help and guidance before.

I have a question about rectification with different fov(different internal parameters).
One is :

[[258.13781734, 0., 954.22342933],
 [0., 258.11148133, 543.9914898],
 [0., 0., 1.]]

the another is ：

[[1.67048218e+03, 0.00000000e+00, 9.49693383e+02],
 [0.00000000e+00, 1.86404224e+03, 5.41460911e+02],
 [0.00000000e+00, 0.00000000e+00, 1.00000000e+00]]

also the have a wide baseline:

R = 
[[ 0.99938327, -0.01944831, -0.02923759], 
 [ 0.00775487,  0.93429903, -0.35640594], 
 [ 0.03424814,  0.3559594,   0.93387364]]

T = [ 0.01480007 -0.58130264 -1.53172435]
when I use DirectStereoRectification:
the perspective distortion is really big: 4077962.957242966

And the rectification failed.

Can you give me some suggestions?

Will this "DirectStereoRectification" algorithms fail with two cameras with completely different internal parameters? Should i scale to the similar focal length? Or is this related to the x_axis baseline length? too small failed?

Thanks