yuliangguo / pytorch_generalized_3d_lane_detection Goto Github PK

你好，我看在训练GeoNet，首先使用了一个固定参数，将2D车道线mask，通过IPM方式转换为俯视图，请问这个固定的参数怎么来的，这个固定参数是否已经包含了相机外参（旋转矩阵和平移矩阵）。这个参数是否只是预先标定的，不太准确的外参。

I was just curious if the curvature that is computed in camera coordinate system is sufficient to use for the steering of the vehicle or is it recommended to compute the curvature in world coordinates as well. If yes, how can XYZ of a particular lane point be used to compute the curvature of the lane ? In traditional IPM based curvature detectection just the X and Y are used keeping Z=0 as it is on the road surface. Since camera to world transformation is rigid can i just rely on the curvature values in camera coordinate system ?

Thanks in advance for any clarification or comments on my understanding. Please do let me know if i am missing something.

Thank you very much for your code, I'm an employee of the national intelligent vehicle innovation center.
In LaneNet3D.py, pred_hcam and pred_pitch are obtained by training, but in GeoNet3D.py I found that they're both directly assigned without training.
I want to know why assign them directly without training? Maybe I misunderstood, Hope to get your reply ~~
Thank you again for your code!

Hi,when i change the calibration of camera,what ever the height or the pitch angel,the model output is totaly error.
Is this algorithm can cover different calibration of camera？
we train this algorithm by usning the real 3d world label with image from different calibration of camera，whether this algorithm can conver different calibration of camera?Or it's cant't to converge directly.

python main_demo_GenLaneNet_ext.py
and don't work

Unexpected key(s) in state_dict: "encoder.0.weight", "encoder.0.bias", "encoder.1.weight", "encoder.1.bias", "encoder.1.running_mean", "encoder.1.running_var", "encoder.1.num_batches_tracked", "encoder.4.weight", "encoder.4.bias", "encoder.5.weight", "encoder.5.bias", "encoder.5.running_mean", "encoder.5.running_var", "encoder.5.num_batches_tracked", "encoder.8.weight", "encoder.8.bias", "encoder.9.weight", "encoder.9.bias", "encoder.9.running_mean", "encoder.9.running_var", "encoder.9.num_batches_tracked", "encoder.12.weight", "encoder.12.bias", "encoder.13.weight", "encoder.13.bias", "encoder.13.running_mean", "encoder.13.running_var", "encoder.13.num_batches_tracked", "lane_out.features.0.weight", "lane_out.features.0.bias", "lane_out.features.1.weight", "lane_out.features.1.bias", "lane_out.features.1.running_mean", "lane_out.features.1.running_var", "lane_out.features.1.num_batches_tracked", "lane_out.features.3.weight", "lane_out.features.3.bias", "lane_out.features.4.weight", "lane_out.features.4.bias", "lane_out.features.4.running_mean", "lane_out.features.4.running_var", "lane_out.features.4.num_batches_tracked", "lane_out.features.6.weight", "lane_out.features.6.bias", "lane_out.features.7.weight", "lane_out.features.7.bias", "lane_out.features.7.running_mean", "lane_out.features.7.running_var", "lane_out.features.7.num_batches_tracked", "lane_out.features.9.weight", "lane_out.features.9.bias", "lane_out.features.10.weight", "lane_out.features.10.bias", "lane_out.features.10.running_mean", "lane_out.features.10.running_var", "lane_out.features.10.num_batches_tracked", "lane_out.features.12.weight", "lane_out.features.12.bias", "lane_out.features.13.weight", "lane_out.features.13.bias", "lane_out.features.13.running_mean", "lane_out.features.13.running_var", "lane_out.features.13.num_batches_tracked", "lane_out.features.15.weight", "lane_out.features.15.bias", "lane_out.features.16.weight", "lane_out.features.16.bias", "lane_out.features.16.running_mean", "lane_out.features.16.running_var", "lane_out.features.16.num_batches_tracked", "lane_out.features.18.weight", "lane_out.features.18.bias", "lane_out.features.19.weight", "lane_out.features.19.bias", "lane_out.features.19.running_mean", "lane_out.features.19.running_var", "lane_out.features.19.num_batches_tracked", "lane_out.dim_rt.0.weight", "lane_out.dim_rt.0.bias", "lane_out.dim_rt.1.weight", "lane_out.dim_rt.1.bias", "lane_out.dim_rt.1.running_mean", "lane_out.dim_rt.1.running_var", "lane_out.dim_rt.1.num_batches_tracked", "lane_out.dim_rt.3.weight", "lane_out.dim_rt.3.bias".

should be
self.anchor_y_steps - self.y_ref

Hello @yuliangguo

Which implementation of 3D-LaneNet did you use to generate the score?

Perhaps you can share it also? as 3D-LaneNet author does not give reference implementation (like you did with Gen-Lanenet). Thanks

ERROR: Could not find a version that satisfies the requirement py3-ortools==5.1.4041 (from versions: none)。
do i need to use ortools.eg:ortools==7.2.6977

Thanks for your amazing work.
I have read Part 3.1 and I just cannot understand the process of estimating z for 3D lane. After the first and second trasformation，I think there should only be red curves on the following image. So where do these blue curves come from? Thank you very much for answering my questions .

RT.
Thank you!

@yuliangguo 感谢你们的杰出工作，期待您的回复！

在Lanet3D_ext.py中，有这样的公式，这公式是怎么来的？

# homograph ground to camera
# H_g2cam = np.array([[1,                             0,               0],
#                     [0, np.cos(np.pi / 2 + cam_pitch), args.cam_height],
#                     [0, np.sin(np.pi / 2 + cam_pitch),               0]])
H_g2cam = np.array([[1,                             0,               0],
                    [0, np.sin(-cam_pitch), args_cam_height],
                    [0, np.cos(-cam_pitch),               0]])

cam的坐标系是在哪定义？为什么是这个样子？

Hello, author:
when I extract model from below file, I got the errors like these:

$ tar -xvf model_best_epoch_29.pth.tar
tar: This does not look like a tar archive
tar: Skipping to next header
tar: Exiting with failure status due to previous errors

How can I solve it? Thank you!

I have two GPU on my desktop, however, even I set os.environ["CUDA_VISIBLE_DEVICES"] = "1", the main_train_GenLaneNet_ext program sill uses the gpu 0. How can I make it use gpu 1?

tools/utils.py文件:

def homograpthy_g2im(cam_pitch, cam_height, K):
    # transform top-view region to original image region
    R_g2c = np.array([[1, 0, 0],
                      [0, np.cos(np.pi / 2 + cam_pitch), -np.sin(np.pi / 2 + cam_pitch)],
                      [0, np.sin(np.pi / 2 + cam_pitch), np.cos(np.pi / 2 + cam_pitch)]])
    H_g2im = np.matmul(K, np.concatenate([R_g2c[:, 0:2], [[0], [cam_height], [0]]], 1))
    return H_g2im

R_g2c是车体坐标系绕x轴旋转90+pitch度的矩阵可以理解, 为什么H_g2im却只取前两列呢?

GeoNet3D_ext.py文件 230行:

        # homograph ground to camera
        # H_g2cam = np.array([[1,                             0,               0],
        #                     [0, np.cos(np.pi / 2 + cam_pitch), args.cam_height],
        #                     [0, np.sin(np.pi / 2 + cam_pitch),               0]])
        H_g2cam = np.array([[1,                  0,               0],
                            [0, np.sin(-cam_pitch), args.cam_height],
                            [0, np.cos(-cam_pitch),               0]])

这里H_g2cam其实是 np.concatenate([R_g2c[:, 0:2], [[0], [cam_height], [0]]], 1)的结果, 但是为什么又与上面的定义方式不同了呢?

Hello, i'm student and study your code.

I use webcam and video file to test the test.py. But i don't know how to use webcam and video file

please to teach how to use webcam and video test.

你好，请问genLaneNet可以使用tusimple训练吗？label里是不是会有缺失的部分

thanks for authors contribution! I have a problem with downloading the data splits and pretrained models, the data is a little big，My internet speed is very slow with google driver. Is anyone has the data in Baidu Cloud. Thanks!

When I run main_demo_GenLanezNet_ext.py, I got the following error:

cuDNN error: CUDNN_STATUS_EXECUTION_FAILED
Traceback (most recent call last):
File "main_demo_GenLaneNet_ext.py", line 147, in
output_geo = output_geo[0].data.cpu().numpy()
NameError: name 'output_geo' is not defined

How can I fix it?

Hey! I have explored your implementation quite nicely. The GeoNet3D_ext model takes as an input the features from the lane segmentation model. Thus those features are then fed to the GeoNet3D_ext model for 3d lane detection. From your implementation, I have realized that you are not predicting camera height and camera pitch instead you are using the gt camera height and gt camera pitch for the same and in your training script you are updating your M_inv using the same gt_camera_height and gt_camera_pitch.

            if not args.fix_cam and not args.pred_cam:
                model2.update_projection(args, gt_hcam, gt_pitch)

I tried to run the train script for GeoNet3D_ext and try to found out that gt_cam_height == pred_cam_height and the same for pitch.

Whereas in the unofficial implementation of 3D lanenet as per the train script, the height and pitch are different from gt's.

**Consider a real-world scenario where I want to use the pre-trained models provided by you for 3D lane Detection: ** in that particular case camera_height and camera_pitch are fixed as not really predicted by the model. Does in that particular case the results for 3d lane detection will differ? Have you tested your approach with any real-world data?

Hi @yuliangguo ,
I think the implementation here https://github.com/yuliangguo/Pytorch_Generalized_3D_Lane_Detection/blob/master/tools/eval_3D_lane.py#L165 is not exactly consistent with the formula in the paper.

Besides, I also want to know why not use num_match_mat or something like match_ratio as the cost_mat.

In Batch Testing of README.md:

Set 'args.dataset_name' to a certain data split to train the model

Is it supposed to be "Set 'args.dataset_name' to a certain data split to test the model"?

你好，路面高度估计是怎么计算的，对应的估计路面高度的loss是什么

I want to get the result segmented to the vehicle in the final result,
so I want to re-learn using erfnet_model_sim3d_7class.tar.

All dataset paths were checked, and the pytorch version and CUDA and cuDNN versions were matched.
But I only got the error below.
Segmentation fault (core dumped)

What is the problem?

If you had the same problem, please share a solution.