bev_lane_det's People
Forkers
rosyapril qianqian121 cuibaby xqpinitial guofenggitlearning virtualxqf ai-jie01 zyg11 happyday-lkj aftermathk chengwei920412 algoai zhousongxin avi9700 levinj waterbearbee liuhang1234 jarygrace jessiepan cv-det sanqingqu shiyong-cui binhelove yutong-gannis yivan-wyygdsg hzm-january vincent-ping seasky100 flyinggh cc8848bev_lane_det's Issues
Confusing about variables in openlane data_loader
Hi,
I'm struggling in understanding the preprocess of openlane dataloader. Would you please give some illustrations of my questions?
- the meaning of cam_w_extrinsics, which seems like the extrinsic params of the camera;
- the meaning of maxtrix_camera2camera_w ;
- difference between cam_extrinsics and cam_extrinsics_persformer;
- why line 176 uses cam_w_extrinsics once and line 178 also uses cam_extrinsics ?
Can you provide information about the training equipment? 4 cards? What model?
I used the source code provided by you to train on 4 cards at 1080, and selected the final round of training results as the validation result. The accuracy was only 56.2%. Therefore, I would like to ask you about the training hardware information you used. Thank you very much for your help!
FPS
Thanks for your great work! I wonder how fast your model detects the lane while inference. Can FPS be greater than 10?
By the way, looking forward to your code.
how to get the vc_intrinsic and vc_extrinsics?
In the papar, it just say 'which are derived from the mean value of the in/extrinsic
parameters of the training dataset'.
I want to use Virtual Camera on other dataset with new in/extrinsic parameters. And I can't find corresponding code in this project. I wonder if you can explain the specific calculation method of the intrinsic parameters and extrinsic parameters of the Virtual Camera.
关于虚拟相机的问题
您好!
我看到您在torch 求虚拟相机参数,是现在图像 大概地面位置取四个点,然后根据z=0,再反投到车体坐标系,再接您求h矩阵的后续
但是再c++示例代码里,却是随机取15个高度为0的店,然后再接您后续h矩阵的后续
请问,这两者有区别有什么特别的含义和思考吗?
Can you provide visual code?
Is your visualization code consistent with that of openlane? Can you provide your result visualization code, thank you very much!
How to choose the best model?
Thanks for your open-source codes. When I train the model, I find that it seems that you don't use tensorboard to record the loss or F1-score. And the total performance of one epoch isn't recorded either. So I wonder how to choose the best model I have got. Should I run the val_openlane.py on each of the models I trained? Or maybe I miss something important.
Thanks for your reply.
A mistake in val_apollo.py
When I try to run the code:"python3 val_apollo.py"
I get a run error:
NameError: name 'Dataset' is not defined
After comparing with val_openlane.py
I think there should be a missing line of code in val_apollo.py which is:
"from torch.utils.data import Dataset,DataLoader"
How can I use this with real time camera video?
How can I use this with real time camera video? It is necessary to change codes in virtual camera?
use tensorrt to inference
how to use tensorrt to do inference
predict lane class
how can I predict lane class of the openlane
Virtual camera problem
This method uses virtual camera, which is a good idea. However, in the data processing stage, only the image is mapped to the virtual camera, and no corresponding transformation is done to the three-dimensional coordinate points.
will you please release the pretrained model?
能否直接通过真实相机与虚拟相机的内外参导出单应矩阵?
您好,
请教一下,看论文包括代码,发现您是通过在真实图像中采样四个点,然后通过一系列转换得到虚拟图像中的四个对应坐标,最终根据四对坐标之间的对应关系求出单应矩阵,我理解这四个点相当于真实图像中所有像素坐标的采样,所以是否有一种方式能够直接根据两个相机的内外参得到对应的单应矩阵那?
感谢
Issue created by accident
Zeroing out the first 2 elements of rotation matrix
In "val_openlane", theres a line as follows:
cam_extrinsics_persformer[0:2, 3] = 0.0
Why is this the case? Isn't this zeroing the x and y elements of the translation component? Wouldn't it need to do cam_extrinsics_persformer[3, 0:3] = 0.0 in order to zero the last row of the matrix?
Thank you
Training with CARLA images
I am trying to train the model with images from CARLA but the extrinsic matrix that I get from CARLA is used to map from camera coordinates to world coordinates, while the "imageview2ego" method uses vehicle oriented coordinates from what I understood. So calculating the homography matrix between the virtual camera and the actual camera is not working well. Do you have any insights as to how to proceed? Thank you.
Will this work be open-sourced in the near future?
Failed to resume training
When I trained the model at the 25th epoch, it halted for mannual stopping, I rewite the code in train_openlane.py
# TODO template config file.
if __name__ == '__main__':
import warnings
warnings.filterwarnings("ignore")
#openlane_cfg_path= os.getcwd()+'/openlane_config.py'
openlane_cfg_path= os.path.abspath(os.path.dirname(__file__)) + '/openlane_config.py'
ckpt_path = '/media/hitbuyi/E8E4A23EE4A20F3E/dataset/openlane1.2/model/BEV_LaneDet/ep025.pth'
#worker_function('./openlane_config.py', gpu_id=[4, 5, 6, 7])
#worker_function(openlane_cfg_path, gpu_id=[0])
worker_function(openlane_cfg_path, gpu_id=[0],checkpoint_path = ckpt_path)
thus
if checkpoint_path:
if getattr(configs, "load_optimizer", True):
resume_training(checkpoint_path, model.module, optimizer, scheduler)
else:
load_checkpoint(checkpoint_path, model.module, None)
was called, but the code failed, it prompted the error in load_model.py
def load_checkpoint(checkpoint, net, optimizer=None, map_loc="cuda"):
sd = torch.load(checkpoint, map_location=map_loc)
net.load_state_dict(sd['models'])
if optimizer and sd['optimizer']:
optimizer.load_state_dict(sd['optimizer'])
return sd
How to solve this issue?
virtual camera and the MLP based STP
hi, thanks for your great work, some questions about the generalization of the virtual camera and the STP,
1.did you test the model trained on openlane with other datasets which the camera extrinsics have large difference with openlane camera ?
2. In principle, is it possible for the MLP based STP to generalize to other scenarios or not? why? @qinjian623
IPM2ego_matrix
Why IPM2ego_matrix is 'M = np.array([[-m_per_pixel, 0, center_x], [0, -m_per_pixel, center_y]])'? Thx
get_seg_offset
cam_w_extrinsics = np.array(gt['extrinsic'])
maxtrix_camera2camera_w = np.array([[0, 0, 1, 0],
[-1, 0, 0, 0],
[0, -1, 0, 0],
[0, 0, 0, 1]], dtype=float)
cam_extrinsics = cam_w_extrinsics @ maxtrix_camera2camera_w #
R_vg = np.array([[0, 1, 0],
[-1, 0, 0],
[0, 0, 1]], dtype=float)
R_gc = np.array([[1, 0, 0],
[0, 0, 1],
[0, -1, 0]], dtype=float)
cam_extrinsics_persformer = copy.deepcopy(cam_w_extrinsics)
cam_extrinsics_persformer[:3, :3] = np.matmul(np.matmul(
np.matmul(np.linalg.inv(R_vg), cam_extrinsics_persformer[:3, :3]),
R_vg), R_gc)
cam_extrinsics_persformer[0:2, 3] = 0.0
matrix_lane2persformer = cam_extrinsics_persformer @ np.linalg.inv(maxtrix_camera2camera_w)
matrix_lane2persformer 这个矩阵代表的是什么
FCTransform 输出的feature大小为啥是(25,5)的
请问 FCTransform 输出的feature大小为啥是(25,5)的,视角转换后 这个 25 和5是怎么设定的?有一定依据吗
license
Could I use your algorithm in an opensource project? And which license should I follow?
是否会开源数据集
can you give me source code
can you give me source code through my email: [email protected]
thank you very much.
When will you release the code?
Cannot reproduce experimental results on Openlane?
Hello all,
I ran the Openlane experiment based on the provided codebase and didn't change any parameters in the config file. I tested multiple models saved at different epochs and only got the best results of 56.3% F-score on Openlane validation set, which is still 2% lower than the number shown in your paper (i.e., 58.4% F-score). Is there anything wrong? Could you please give me some advice?
Evaluation on Apollo
In the original evaluation scheme of the Apollo Synthetic dataset, ground truth lanes are considered in the lateral (x-) range [-30 m, 30 m]. In the paper I only found that the final grid size used for both datasets is [-10 m, 10 m] lateral and [3 m, 103 m] longitudinal. To my understanding there should be no key-points to represent lanes in the range [-30 m, -10 m] and [10 m, 30 m], but still the detection scores are pretty high and far-range errors fairly low.
How is this possible? Was the grid size enlarged for Apollo or was the Apollo evaluation scheme adapted with ground truth lanes being pruned to [-10 m, 10 m] lateral (as it is the case for OpenLane evaluation)?
video demo
Hi, thanks for your great work, did you have any video demos on openlane and ONCE-3DLane?
Looking forward to your reply, thanks! @qinjian623
openlane的f1_score只有55.8?是训练问题吗?
全部是默认设置,epoch=50. val结果如下所示:
{'f1_score': 0.5584564102804471,
'precision': 0.6719318365019449,
'recall': 0.4777714921062119,
'x_error_close': 0.25970785558032083,
'x_error_far': 0.6890227179538931,
'z_error_close': 0.1996908412880719,
'z_error_far': 0.6315434166948418}
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.