tudelft-iv / view-of-delft-dataset Goto Github PK

I downloaded the dataset after receiving your mail. But the data in the radar folder is incomplete. image_2, label_2, pose folders are missing. and many files are of 1 or 2 kb only. please tell me how to proceed.

The 3D figure is not shown.

run command

jupyter nbextension enable --py --sys-prefix k3d

after

jupyter nbextension install --py --sys-prefix k3d

Hi everyone,

First of all, I want to thank you for publishing this wonderful dataset. I am currently working on reproducing the radar results (5 scans) from the paper. When I use the evaluation tool from your repo, I sometimes (not often though) get a ZeroDivisionError:

Traceback (most recent call last):
  File "/home/leon/Projects/OpenPCDet/tools/eval_delft.py", line 21, in <module>
    results = evaluation.evaluate(
  File "/home/leon/Projects/OpenPCDet/vod/evaluation/evaluate.py", line 44, in evaluate
    evaluation_result.update(get_official_eval_result(gt_annotations, dt_annotations, current_class))
  File "/home/leon/Projects/OpenPCDet/vod/evaluation/kitti_official_evaluate.py", line 741, in get_official_eval_result
    mAPbbox, mAPbev, mAP3d, mAPaos, mAPbbox_R40, mAPbev_R40, mAP3d_R40, mAPaos_R40 = do_eval(
  File "/home/leon/Projects/OpenPCDet/vod/evaluation/kitti_official_evaluate.py", line 665, in do_eval
    ret = eval_class(gt_annotations, dt_annotations, current_classes, difficulties, 1,
  File "/home/leon/Projects/OpenPCDet/vod/evaluation/kitti_official_evaluate.py", line 533, in eval_class
    rets = calculate_iou_partly(dt_annotations, gt_annotations, metric, num_parts)
  File "/home/leon/Projects/OpenPCDet/vod/evaluation/kitti_official_evaluate.py", line 432, in calculate_iou_partly
    overlap_part = bev_box_overlap(gt_boxes, dt_boxes).astype(
  File "/home/leon/Projects/OpenPCDet/vod/evaluation/kitti_official_evaluate.py", line 152, in bev_box_overlap
    r_iou = rotate_iou_eval(boxes, q_boxes, criterion)
  File "/home/leon/Projects/OpenPCDet/vod/evaluation/rotate_iou_cpu.py", line 269, in rotate_iou_eval
    iou[n, k] = instance_iou_eval(query_boxes[k], boxes[n], criterion)
ZeroDivisionError: division by zero

I found out that the error occurs in the sort_vertex_in_convex_polygon function in the rotate_iou_cpu.py. Somehow d = math.sqrt(v[0] * v[0] + v[1] * v[1]) becomes zero.
Here is the skript that i wrote using your example:

from vod.evaluation import Evaluation
import os
import wandb
from tqdm import tqdm
#wandb.init("Eval Delft", name="EvalDelft_2 ", reinit=True)
# When the instance is created, the label locations are required.
evaluation = Evaluation(test_annotation_file="/home/leon/Projects/OpenPCDet/data/kitti/training/label_2")

# Using the evaluate method, the model can be evaluated on the detection labels.
EPOCHS = []
default_dir = "/home/leon/Projects/OpenPCDet/output/kitti_models/pointpillardelftradar/default/eval/eval_with_train"
for _, dirs, _ in os.walk(default_dir):
      for dir in dirs:
            if dir.startswith("epoch"):
                  EPOCHS.append(dir)

EPOCHS.sort()
for epoch in tqdm(EPOCHS):
      results = evaluation.evaluate(
          result_path=os.path.join(default_dir, epoch, "val/final_result/data"),
          current_class=[0, 1, 2])

      print("Results: \n"
            f"Entire annotated area: \n"
            f"Car: {results['entire_area']['Car_3d_all']} \n"
            f"Pedestrian: {results['entire_area']['Pedestrian_3d_all']} \n"
            f"Cyclist: {results['entire_area']['Cyclist_3d_all']} \n"
            f"mAP: {(results['entire_area']['Car_3d_all'] + results['entire_area']['Pedestrian_3d_all'] + results['entire_area']['Cyclist_3d_all']) / 3} \n"
            f"Driving corridor area: \n"
            f"Car: {results['roi']['Car_3d_all']} \n"
            f"Pedestrian: {results['roi']['Pedestrian_3d_all']} \n"
            f"Cyclist: {results['roi']['Cyclist_3d_all']} \n"
            f"mAP: {(results['roi']['Car_3d_all'] + results['roi']['Pedestrian_3d_all'] + results['roi']['Cyclist_3d_all']) / 3} \n"
            )

Except for the accidentally swapped input for the Eval function (has been merged into the master in the meantime) I have not changed anything in the code. The training process itself runs smoothly. (I followed your guidelines → not quite reached the benchmark though). Not sure if I'm doing something wrong.... Have any of you ever had such a problem, or any idea what the problem could be ?
Thanks in advance!
Best Regards
Leon

Hi. I had a question about the occlusion setting for evaluation.

The evaluation script in this repo uses [4] for max occlusion, which means it gonna evaluate all objects that have any occlusion.

This setting is different to the original openpcdet setting here and wasn't mentioned in documents and issues.

Could you please confirm this is the correct occlusion setting for evaluation? And pp-radar, pp-lidar results on paper were evaluated with this occlusion setting?

Thank you very much in advance.

As is described in the title, do you have filtered some values from the radar sensor, like removing some noisy data?

when I try to run the file named "1_frame_information.ipynb";when I use the function named ”vis_3d.draw_plot“ in the part named "LiDAR";I find the problem

TypeError Traceback (most recent call last)
~\AppData\Local\Temp/ipykernel_33332/1146023220.py in
6
7
----> 8 vis_3d.draw_plot(lidar_origin_plot=True, lidar_points_plot=True)

G:\view-of-delft-dataset-main\vod\visualization\vis_3d.py in draw_plot(self, radar_origin_plot, lidar_origin_plot, camera_origin_plot, lidar_points_plot, radar_points_plot, radar_velocity_plot, annotations_plot, write_to_html, html_name, grid_visible, auto_frame)
73
74 if lidar_origin_plot:
---> 75 self.plot_lidar_origin()
76
77 if camera_origin_plot:

G:\view-of-delft-dataset-main\vod\visualization\vis_3d.py in plot_lidar_origin(self, label, color, axis_length, label_size)
121 def plot_lidar_origin(self,label = True,color = lidar_plot_color_3d,axis_length = axis_length_3d,label_size = axis_label_size):
122
--> 123 self.plot += k3d_get_axes(self.transform_matrices['lidar'], axis_length)
124 if label:
125 self.plot += k3d.text("lidar",

G:\view-of-delft-dataset-main\vod\visualization\helpers.py in k3d_get_axes(hom_transform_matrix, axis_length)
183 dtype=float)
184
--> 185 origin = [hom_transform_matrix[:3, 3]] * 3
186 axis = hom_transform_matrix.dot(hom_vector.T).T[:, :3]
187

TypeError: 'method' object is not subscriptable

I have trained the radar_5frames_pointpillars.yaml based on the , but the evaluated results are not good. The 'Truncation' in eval.py has also been commented out. Here is my results and config files.

Good morning everyone,

I am trying to run code from vod-tudelft library and it seems that some numpy features you are using are now deprecated.

File "/home/robesafe/.local/lib/python3.10/site-packages/vod/frame/transformations.py", line 379, in project_pcl_to_image
    uvs = project_3d_to_2d(points=radar_points_camera_frame,
  File "/home/robesafe/.local/lib/python3.10/site-packages/vod/frame/transformations.py", line 325, in project_3d_to_2d
    uvs = np.round(uvs).astype(np.int)
  File "/home/robesafe/.local/lib/python3.10/site-packages/numpy/__init__.py", line 305, in __getattr__
    raise AttributeError(__former_attrs__[attr])
AttributeError: module 'numpy' has no attribute 'int'.
`np.int` was a deprecated alias for the builtin `int`. To avoid this error in existing code, use `int` by itself. Doing this will not modify any behavior and is safe. When replacing `np.int`, you may wish to use e.g. `np.int64` or `np.int32` to specify the precision. If you wish to review your current use, check the release note link for additional information.
The aliases was originally deprecated in NumPy 1.20; for more details and guidance see the original release note at:
    https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations. Did you mean: 'inf'?

I say this just to inform you, I could prepare a PR during the next upcoming weeks.

Bests,
Santi

Hi, I met some problems when visualizing the ego-car trajectory w.r.t. the pose infomation.
The transformation between frame #6759 and frame #6760 seems not to match the ego-motion shown in their corresponding image. The computed transform matrix shows that the ego-car has a large yaw angle change between two frames, but on the images, the ego-car is forward.

Could the repo owner explain this or check if the pose information of this frame is converted correctly?

Hello!
I was trying to find the ego vehicle speed for each frame, for eg: from the odometry data! I couldn't fnd anything and I would like to know if I should estimate the vehicle speed from the radar data? Also, I need the exact vehilce velocity in each frame, not the radial one (But I can do the transformation if only radial velocity is available) so that I could comapare it with similar frames on nuscenes! Please let me know how to proceed on this.

Hi,
I'm trying to understand how to calculate azimuth from the annotated data.

(all coordinates in camera coordinates)

print(np.arctan2(label['x'], label['z']) * 180 / np.pi)

but this does not line up with

print(label['alpha'] * 180 / np.pi)

The values I get from the arctan2 angle calculation seem appropriate when drawing them on paper in a coordinate grid. However, in your documentation it clearly states that objects are only annotated within horizontal FoV: ±32°, vertical FoV: ± 22°, making me further question the validity of my azimuth values being > +-32 degrees.

Is my assumption wrong that azimuth=alpha? Or is the arctan2 angle calculation wrong?

Thanks
Eric

I pay high tribute to you and your work, and I wonder what's the pixel size of the camera.

Hi, thanks for your excelent work on the 4D Radar dataset. I have trained my own model based on the training dataset, and evaluated it by val dataset. I would like to ask you how I can evaluate my model in testing set? Thanks.

Dear author, thank you for your meaningful work！
We adapt vod to mmdet3d framework. In addition, eval was modified according to official instructions to evaluate the Entire annotated area. We want to check whether this result is similar to the val set result of 1296 frames in your framework, because we want to see the baseline result of the val set. Thank you and look forward to your reply.

I'm trying to run 4_evaluation.ipynb to evalute my model but it's giving Value error. Please help

ValueError Traceback (most recent call last)
Cell In[4], line 9
6 evaluation = Evaluation(test_annotation_file=os.path.join('E:/vod/view_of_delft_PUBLIC', 'label'))
8 # Using the evaluate method, the model can be evaluated on the detection labels.
----> 9 results = evaluation.evaluate(
10 #result_path=os.path.join('example_set', 'detection'),
11 result_path=os.path.join('E:/vod/view_of_delft_PUBLIC', 'detection'),
12 current_class=[0, 1, 2])
14 print("Results: \n"
15 f"Entire annotated area: \n"
16 f"Car: {results['entire_area']['Car_3d_all']} \n"
(...)
24 f"mAP: {(results['roi']['Car_3d_all'] + results['roi']['Pedestrian_3d_all'] + results['roi']['Cyclist_3d_all']) / 3} \n"
25 )

File e:\github\view-of-delft-dataset\view-of-delft-dataset\vod\evaluation\evaluate.py:44, in Evaluation.evaluate(self, result_path, current_class, score_thresh)
41 gt_annotations = kitti.get_label_annotations(self.test_annotation_file, val_image_ids)
43 evaluation_result = {}
---> 44 evaluation_result.update(get_official_eval_result(gt_annotations, dt_annotations, current_class, custom_method=0))
45 evaluation_result.update(get_official_eval_result(gt_annotations, dt_annotations, current_class, custom_method=3))
47 return evaluation_result

File e:\github\view-of-delft-dataset\view-of-delft-dataset\vod\evaluation\kitti_official_evaluate.py:744, in get_official_eval_result(gt_annotations, dt_annotations, current_classes, pr_detail_dict, custom_method)
741 compute_aos = True
742 break
--> 744 mAPbbox, mAPbev, mAP3d, mAPaos, mAPbbox_R40, mAPbev_R40, mAP3d_R40, mAPaos_R40 = do_eval(
745 gt_annotations, dt_annotations, current_classes, min_overlaps, compute_aos, pr_detail_dict=pr_detail_dict,
746 custom_method=custom_method)
748 ret_dict = {}
749 for j, curcls in enumerate(current_classes):
750 # mAP threshold array: [num_min_overlap, metric, class]
751 # mAP result: [num_class, num_diff, num_min_overlap]

File e:\github\view-of-delft-dataset\view-of-delft-dataset\vod\evaluation\kitti_official_evaluate.py:650, in do_eval(gt_annotations, dt_annotations, current_classes, min_overlaps, compute_aos, pr_detail_dict, custom_method)
647 if custom_method == 3:
648 difficulties = [0]
--> 650 ret = eval_class(gt_annotations, dt_annotations, current_classes, difficulties, 0,
651 min_overlaps, compute_aos, custom_method=custom_method)
653 print("mAP Image BBox finished")
654 mAP_bbox = get_m_ap(ret["precision"])

File e:\github\view-of-delft-dataset\view-of-delft-dataset\vod\evaluation\evaluate.py:44, in Evaluation.evaluate(self, result_path, current_class, score_thresh)
41 gt_annotations = kitti.get_label_annotations(self.test_annotation_file, val_image_ids)
43 evaluation_result = {}
---> 44 evaluation_result.update(get_official_eval_result(gt_annotations, dt_annotations, current_class, custom_method=0))
45 evaluation_result.update(get_official_eval_result(gt_annotations, dt_annotations, current_class, custom_method=3))
47 return evaluation_result

File e:\github\view-of-delft-dataset\view-of-delft-dataset\vod\evaluation\kitti_official_evaluate.py:744, in get_official_eval_result(gt_annotations, dt_annotations, current_classes, pr_detail_dict, custom_method)
741 compute_aos = True
742 break
--> 744 mAPbbox, mAPbev, mAP3d, mAPaos, mAPbbox_R40, mAPbev_R40, mAP3d_R40, mAPaos_R40 = do_eval(
745 gt_annotations, dt_annotations, current_classes, min_overlaps, compute_aos, pr_detail_dict=pr_detail_dict,
746 custom_method=custom_method)
748 ret_dict = {}
749 for j, curcls in enumerate(current_classes):
750 # mAP threshold array: [num_min_overlap, metric, class]
751 # mAP result: [num_class, num_diff, num_min_overlap]

File e:\github\view-of-delft-dataset\view-of-delft-dataset\vod\evaluation\kitti_official_evaluate.py:650, in do_eval(gt_annotations, dt_annotations, current_classes, min_overlaps, compute_aos, pr_detail_dict, custom_method)
647 if custom_method == 3:
648 difficulties = [0]
--> 650 ret = eval_class(gt_annotations, dt_annotations, current_classes, difficulties, 0,
651 min_overlaps, compute_aos, custom_method=custom_method)
653 print("mAP Image BBox finished")
654 mAP_bbox = get_m_ap(ret["precision"])
File e:\github\view-of-delft-dataset\view-of-delft-dataset\vod\evaluation\kitti_official_evaluate.py:536, in eval_class(gt_annotations, dt_annotations, current_classes, difficulties, metric, min_overlaps, compute_aos, num_parts, custom_method)
533 split_parts = get_split_parts(num_examples, num_parts)
535 # iou is calculated in "parts = cluster of frames"
--> 536 rets = calculate_iou_partly(dt_annotations, gt_annotations, metric, num_parts)
537 overlaps, parted_overlaps, total_dt_num, total_gt_num = rets
539 N_SAMPLE_PTS = 41 # what is this number

File e:\github\view-of-delft-dataset\view-of-delft-dataset\vod\evaluation\kitti_official_evaluate.py:406, in calculate_iou_partly(gt_annotations, dt_annotations, metric, num_parts)
402 assert len(gt_annotations) == len(dt_annotations)
403 #here the erorr is occuring Valuerror: need atleast one array to stack
404 #this is commented , please uncomment after it
405 #total_dt_num = np.stack([len(a["name"]) for a in dt_annotations], 0)
--> 406 total_dt_num = np.stack([np.array(len(a["name"])) for a in dt_annotations], 0)
407 total_gt_num = np.stack([len(a["name"]) for a in gt_annotations], 0)
408 num_examples = len(gt_annotations)

File <array_function internals>:180, in stack(*args, **kwargs)

File c:\Users\Realme\AppData\Local\Programs\Python\Python310\lib\site-packages\numpy\core\shape_base.py:422, in stack(arrays, axis, out)
420 arrays = [asanyarray(arr) for arr in arrays]
421 if not arrays:
--> 422 raise ValueError('need at least one array to stack')
424 shapes = {arr.shape for arr in arrays}
425 if len(shapes) != 1:

ValueError: need at least one array to stack

``

Hi Mr. Palffy, I was wondering if you could share with me your opinion on some details regarding the specification of the labeling? When I checked the labels of the dataset, I found that Rider and Cyclist are present at the same time, in addition to the presence of the Bicycle label. This was rather surprising to me. I am looking forward to your response, which will help me a lot. Best wishes! ChenJ.

Hello, I sent a data set download request using the education email. Did you see the email prompt here? I would appreciate it if you could tell me how long it will take to verify my identity;
email：[email protected]

Thank you for your execllent dataset!
I've seen previous issues about low results, and i know you used OpenPCD, I modified the config files of mmdet3d to train pp-lidar according to the previous issues(including point range, voxel sizes and so on). however, I still got very low eval accuracy even ‘0’!!!(i already comented the eval line).
I really don't know how to solve this problem, so I can only ask you for help, thank you again!!!
(this picture is fisrt epoch)

And this is my config:

_base_ = [
    '../_base_/schedules/cyclic_40e.py',
    '../_base_/default_runtime.py'
]

point_cloud_range = [0, -25.6, -3, 51.2, 25.6, 1]
voxel_size = [0.16, 0.16, 5]
max_points_per_voxel = 10

model = dict(
    type='VoxelNet',
    voxel_layer=dict(
        max_num_points=max_points_per_voxel,
        point_cloud_range=point_cloud_range,
        voxel_size=voxel_size,
        max_voxels=(16000, 40000)
    ),
    voxel_encoder=dict(
        type='PillarFeatureNet',
        in_channels=4,
        feat_channels=[64],
        with_distance=False,
        voxel_size=voxel_size,
        point_cloud_range=point_cloud_range),
    middle_encoder=dict(
        type='PointPillarsScatter', in_channels=64, output_shape=[496, 432]),
    backbone=dict(
        type='SECOND',
        in_channels=64,
        layer_nums=[3, 5, 5],
        layer_strides=[2, 2, 2],
        out_channels=[64, 128, 256]),
    neck=dict(
        type='SECONDFPN',
        in_channels=[64, 128, 256],
        upsample_strides=[1, 2, 4],
        out_channels=[128, 128, 128]),
    bbox_head=dict(
        type='Anchor3DHead',
        num_classes=3,
        in_channels=384,
        feat_channels=384,
        use_direction_classifier=True,
        assign_per_class=True,
        anchor_generator=dict(
            type='AlignedAnchor3DRangeGenerator',
            ranges=[
                [0, -25.6, -0.6, 51.2, 25.6, -0.6],
                [0, -25.6, -0.6, 51.2, 25.6, -0.6],
                [0, -25.6, -1.78, 51.2, 25.6, -1.78],
            ],
            # ranges=[
            #     [0, -39.68, -0.6, 69.12, 39.68, -0.6],
            #     [0, -39.68, -0.6, 69.12, 39.68, -0.6],
            #     [0, -39.68, -1.78, 69.12, 39.68, -1.78],
            #     ],
            sizes=[[0.8, 0.6, 1.73], [1.76, 0.6, 1.73], [3.9, 1.6, 1.56]],
            rotations=[0, 1.57],
            reshape_out=False),
        diff_rad_by_sin=True,
        bbox_coder=dict(type='DeltaXYZWLHRBBoxCoder'),
        loss_cls=dict(
            type='FocalLoss',
            use_sigmoid=True,
            gamma=2.0,
            alpha=0.25,
            loss_weight=1.0),
        loss_bbox=dict(type='SmoothL1Loss', beta=1.0 / 9.0, loss_weight=2.0),
        loss_dir=dict(
            type='CrossEntropyLoss', use_sigmoid=False, loss_weight=0.2)),
    # model training and testing settings
    train_cfg=dict(
        assigner=[
            dict(  # for Pedestrian
                type='MaxIoUAssigner',
                iou_calculator=dict(type='BboxOverlapsNearest3D'),
                pos_iou_thr=0.5,
                neg_iou_thr=0.35,
                min_pos_iou=0.35,
                ignore_iof_thr=-1),
            dict(  # for Cyclist
                type='MaxIoUAssigner',
                iou_calculator=dict(type='BboxOverlapsNearest3D'),
                pos_iou_thr=0.5,
                neg_iou_thr=0.35,
                min_pos_iou=0.35,
                ignore_iof_thr=-1),
            dict(  # for Car
                type='MaxIoUAssigner',
                iou_calculator=dict(type='BboxOverlapsNearest3D'),
                pos_iou_thr=0.6,
                neg_iou_thr=0.45,
                min_pos_iou=0.45,
                ignore_iof_thr=-1),
        ],
        allowed_border=0,
        pos_weight=-1,
        debug=False),
    test_cfg=dict(
        use_rotate_nms=True,
        nms_across_levels=False,
        nms_thr=0.01,
        score_thr=0.1,
        min_bbox_size=0,
        nms_pre=100,
        max_num=50))

dataset_type = 'KittiDataset'
data_root = "../../vod_lidar/"
class_names = ['Pedestrian', 'Cyclist', 'Car']
input_modality = dict(use_lidar=True, use_camera=True)
file_client_args = dict(backend='disk')

db_sampler = dict(
    data_root=data_root,
    info_path=data_root + 'vod_dbinfos_train.pkl',
    rate=1.0,
    prepare=dict(
        filter_by_difficulty=[-1],
        filter_by_min_points=dict(Car=5, Pedestrian=5, Cyclist=5)),
    classes=class_names,
    sample_groups=dict(Car=15, Pedestrian=15, Cyclist=15),
    points_loader=dict(
        type='LoadPointsFromFile',
        coord_type='LIDAR',
        load_dim=4,
        use_dim=4,
        file_client_args=file_client_args),
    file_client_args=file_client_args)

train_pipeline = [
    dict(
        type='LoadPointsFromFile',
        coord_type='LIDAR',
        load_dim=4,
        use_dim=4,
        file_client_args=file_client_args),
    dict(
        type='LoadAnnotations3D',
        with_bbox_3d=True,
        with_label_3d=True,
        file_client_args=file_client_args),
    dict(type='ObjectSample', db_sampler=db_sampler, use_ground_plane=False),
    dict(type='RandomFlip3D', flip_ratio_bev_horizontal=0.5),
    dict(
        type='GlobalRotScaleTrans',
        rot_range=[-0.78539816, 0.78539816],
        scale_ratio_range=[0.95, 1.05]),
    dict(type='PointsRangeFilter', point_cloud_range=point_cloud_range),
    dict(type='ObjectRangeFilter', point_cloud_range=point_cloud_range),
    dict(type='PointShuffle'),
    dict(type='DefaultFormatBundle3D', class_names=class_names),
    dict(type='Collect3D', keys=['points', 'gt_bboxes_3d', 'gt_labels_3d'])
]
test_pipeline = [
    dict(
        type='LoadPointsFromFile',
        coord_type='LIDAR',
        load_dim=4,
        use_dim=4,
        file_client_args=file_client_args),
    dict(
        type='MultiScaleFlipAug3D',
        img_scale=(1333, 800),
        pts_scale_ratio=1,
        flip=False,
        transforms=[
            dict(
                type='GlobalRotScaleTrans',
                rot_range=[-0.78539816, 0.78539816],
                scale_ratio_range=[0.95, 1.05]),
            dict(type='RandomFlip3D'),
            dict(type='PointsRangeFilter', point_cloud_range=point_cloud_range),
            dict(
                type='DefaultFormatBundle3D',
                class_names=class_names,
                with_label=False),
            dict(type='Collect3D', keys=['points'])
        ])
]

eval_pipeline = [
    dict(
        type='LoadPointsFromFile',
        coord_type='LIDAR',
        load_dim=4,
        use_dim=4,
        file_client_args=file_client_args),
    dict(
        type='DefaultFormatBundle3D',
        class_names=class_names,
        with_label=False),
    dict(type='Collect3D', keys=['points'])
]

data = dict(
    samples_per_gpu=12,
    workers_per_gpu=4,
    train=dict(
        type='RepeatDataset',
        times=2,
        dataset=dict(
            type=dataset_type,
            data_root=data_root,
            ann_file=data_root + 'vod_infos_train.pkl',
            split='training',
            pts_prefix='velodyne_reduced',
            pipeline=train_pipeline,
            modality=input_modality,
            classes=class_names,
            test_mode=False,
            # we use box_type_3d='LiDAR' in kitti and nuscenes dataset
            # and box_type_3d='Depth' in sunrgbd and scannet dataset.
            box_type_3d='LiDAR',
            file_client_args=file_client_args)),
    val=dict(
        type=dataset_type,
        data_root=data_root,
        ann_file=data_root + 'vod_infos_val.pkl',
        split='training',
        pts_prefix='velodyne_reduced',
        pipeline=test_pipeline,
        modality=input_modality,
        classes=class_names,
        test_mode=True,
        box_type_3d='LiDAR',
        file_client_args=file_client_args),
    test=dict(
        type=dataset_type,
        data_root=data_root,
        ann_file=data_root + 'vod_infos_val.pkl',
        split='training',
        pts_prefix='velodyne_reduced',
        pipeline=test_pipeline,
        modality=input_modality,
        classes=class_names,
        test_mode=True,
        box_type_3d='LiDAR',
        file_client_args=file_client_args)
)

lr = 0.001
optimizer = dict(lr=lr)
optimizer_config = dict(grad_clip=dict(max_norm=35, norm_type=2))
runner = dict(max_epochs=50)
evaluation = dict(interval=1)

hello,I want to know the height of the lidar, camera, and radar above the ground, and the distance between the lidar and radar in the front and back.

Hello, first of all, thank you for your work. I would like to know what is the field of view angle of a radar.

Hi,when I run pp-radar copy from your pp-radar guidelineshttps://github.com/tudelft-iv/view-of-delft-dataset/blob/main/PP-Radar.md.
The output is as follows:

INFO  Car [email protected], 0.70, 0.70:
**bbox AP:22.6309, 28.9965, 23.7317
bev  AP:31.4834, 38.0852, 31.8317
3d   AP:5.7727, 11.2009, 10.5092
aos  AP:21.41, 27.94, 23.06
Car [email protected], 0.70, 0.70:
bbox AP:18.3606, 25.2401, 21.1341
bev  AP:28.1924, 34.1668, 29.1861
3d   AP:4.2473, 8.4565, 7.0809
aos  AP:17.11, 24.20, 20.25
Car [email protected], 0.50, 0.50:
bbox AP:22.6309, 28.9965, 23.7317
bev  AP:42.1046, 48.2684, 40.7786
3d   AP:28.5084, 32.6852, 30.5675
aos  AP:21.41, 27.94, 23.06
Car [email protected], 0.50, 0.50:
bbox AP:18.3606, 25.2401, 21.1341
bev  AP:40.2857, 44.8250, 37.9114
3d   AP:24.4887, 31.3992, 26.7819
aos  AP:17.11, 24.20, 20.25**
Pedestrian [email protected], 0.50, 0.50:
bbox AP:27.3352, 25.2310, 23.8031
bev  AP:21.9817, 19.3415, 18.4007
3d   AP:12.0408, 11.6275, 11.0378
aos  AP:26.68, 24.45, 23.02
Pedestrian [email protected], 0.50, 0.50:
bbox AP:24.2204, 21.3113, 19.0969
bev  AP:17.3610, 15.1017, 13.3590
3d   AP:5.4015, 4.7779, 4.1870
aos  AP:23.44, 20.46, 18.31
Pedestrian [email protected], 0.25, 0.25:
bbox AP:27.3352, 25.2310, 23.8031
bev  AP:43.4798, 38.3300, 36.0452
3d   AP:35.8189, 30.6979, 28.5039
aos  AP:26.68, 24.45, 23.02
Pedestrian [email protected], 0.25, 0.25:
bbox AP:24.2204, 21.3113, 19.0969
bev  AP:41.6760, 36.1811, 32.1889
3d   AP:33.1692, 27.9692, 24.4298
aos  AP:23.44, 20.46, 18.31
Cyclist [email protected], 0.50, 0.50:
bbox AP:54.0451, 49.8941, 43.8147
bev  AP:56.1162, 50.8601, 45.0722
3d   AP:24.1656, 22.1448, 19.4740
aos  AP:53.70, 49.62, 43.58
Cyclist [email protected], 0.50, 0.50:
bbox AP:53.7608, 48.1834, 42.7750
bev  AP:54.6610, 48.8692, 44.0406
3d   AP:22.8138, 20.1624, 17.6555
aos  AP:53.42, 47.87, 42.49
Cyclist [email protected], 0.25, 0.25:
bbox AP:54.0451, 49.8941, 43.8147
bev  AP:76.6063, 70.5171, 63.4638
3d   AP:68.7943, 62.0618, 55.2496
aos  AP:53.70, 49.62, 43.58
Cyclist [email protected], 0.25, 0.25:
bbox AP:53.7608, 48.1834, 42.7750
bev  AP:77.1777, 70.6806, 63.6326
3d   AP:68.7145, 61.9188, 55.3979
aos  AP:53.42, 47.87, 42.49

you can see that，class Car easy perform well than moderate，i do not know how to deal with this problem.
Could you please tell me why it happens... or can you give some guideline

Hi, I was wondering know the camera version details of the camera used for collecting this datasets. And whether the camera used supports auto focus function.

Hi, I have filed multiple requirements for access of the dataset. But I did not receive any reply.

Hi there.

First of all, thank you for releasing this wonderful dataset.

Could you release several trained lidar & radar models in your paper for baseline model evaluation?

Hello Author,

Thanks for the repo!
I am wondering why detections and groundtruth are swapped. like while calling it is called as calculate_iou_partly(dt_annotations, gt_annotations, ..) and in function definiton it is used as calculate_iou_partly(gt_annotations, dt_annotations, ..).
The results are different if we keep same ordering.

Hello Author, could you please explain how v_r_compensated is obtained?
Is there any reference?

Thanks for your outstanding work about the 4D radar datasets which is rarely saw on internet.

I have a question about map frame. So where is the frame reference point located? and what is the rotation relationship between map frame and UTM frame? I have tried to get the relationship manually by matrix manipulation but it seem it varies from time to time.

Hello, Author, thank you for your codes.

After I get the training results with OpenPCdet, how can I test them?

it means :
How should I set "test_annotation_file" and "result_path" in the vod.evaluate?

and the dataset and output are as follow:

When I set them as shown above, the following error will appear.

Evaluating kitti by default
Traceback (most recent call last):
  File "/public/user_data/openpc/OpenPCDet/VoD-main/vod/evaluation/evaluate.py", line 75, in <module>
    main()
  File "/public/user_data/openpc/OpenPCDet/VoD-main/vod/evaluation/evaluate.py", line 57, in main
    current_class=[0, 1, 2])
  File "/public/user_data/openpc/OpenPCDet/VoD-main/vod/evaluation/evaluate.py", line 45, in evaluate
    evaluation_result.update(get_official_eval_result(gt_annotations, dt_annotations, current_class, custom_method=0))
  File "/public/user_data/openpc/OpenPCDet/VoD-main/vod/evaluation/kitti_official_evaluate.py", line 743, in get_official_eval_result
    custom_method=custom_method)
  File "/public/user_data/openpc/OpenPCDet/VoD-main/vod/evaluation/kitti_official_evaluate.py", line 648, in do_eval
    min_overlaps, compute_aos, custom_method=custom_method)
  File "/public/user_data/openpc/OpenPCDet/VoD-main/vod/evaluation/kitti_official_evaluate.py", line 533, in eval_class
    rets = calculate_iou_partly(dt_annotations, gt_annotations, metric, num_parts)
  File "/public/user_data/openpc/OpenPCDet/VoD-main/vod/evaluation/kitti_official_evaluate.py", line 403, in calculate_iou_partly
    total_dt_num = np.stack([len(a["name"]) for a in dt_annotations], 0)
  File "<__array_function__ internals>", line 6, in stack
  File "/home/ubuntu/anaconda3/envs/lib/python3.7/site-packages/numpy/core/shape_base.py", line 422, in stack
    raise ValueError('need at least one array to stack')
ValueError: need at least one array to stack

Hi, thanks for the great dataset and tools. I was wondering what coordinate system is the label/annotations in w.r.t? i.e. what transformations will I have to apply to the radar points such that the annotations and the points are in the same coordinate system? I couldn't seem to find this detail in the jupyter notebooks. Apologies if I missed it.

I have followed steps mentioned in "https://github.com/tudelft-iv/view-of-delft-dataset/blob/main/PP-Radar.md".
However I face below error when running train.py

022-11-28 15:19:03,527 INFO cfg.OPTIMIZATION.LR_CLIP: 1e-07
2022-11-28 15:19:03,527 INFO cfg.OPTIMIZATION.LR_WARMUP: False
2022-11-28 15:19:03,527 INFO cfg.OPTIMIZATION.WARMUP_EPOCH: 1
2022-11-28 15:19:03,527 INFO cfg.OPTIMIZATION.GRAD_NORM_CLIP: 10
2022-11-28 15:19:03,527 INFO cfg.TAG: pointpillar_radar
2022-11-28 15:19:03,527 INFO cfg.EXP_GROUP_PATH: kitti_models
2022-11-28 15:19:03,547 INFO Loading KITTI dataset
2022-11-28 15:19:03,718 INFO Total samples for KITTI dataset: 5139
[320 320 1]
Traceback (most recent call last):
File "train.py", line 223, in
main()
File "train.py", line 125, in main
model = build_network(model_cfg=cfg.MODEL, num_class=len(cfg.CLASS_NAMES), dataset=train_set)
File "../pcdet/models/init.py", line 18, in build_network
model_cfg=model_cfg, num_class=num_class, dataset=dataset
File "../pcdet/models/detectors/init.py", line 37, in build_detector
model_cfg=model_cfg, num_class=num_class, dataset=dataset
File "../pcdet/models/detectors/pointpillar.py", line 7, in init
self.module_list = self.build_networks()
File "../pcdet/models/detectors/detector3d_template.py", line 47, in build_networks
model_info_dict=model_info_dict
File "../pcdet/models/detectors/detector3d_template.py", line 81, in build_vfe
depth_downsample_factor=model_info_dict['depth_downsample_factor']
TypeError: init() got an unexpected keyword argument 'grid_size'

Did anyone faced similar issue and have solution?

Hello, thank you very much for providing the dataset containing 4D Radar. I'm trying to do 3D object tracking based on the dataset. I would like to ask if you can provide a tracking evaluation script. I'm looking forward to your reply!

It seems to provide nothing about pitch and roll annotation, right?
I can only find the annotated yaw in these KITTI-like annoataion files.

I found that some 3D BBoxes at the edges of the images did not match their corresponding objects, as follows:

However, it should do not like this because the images stored in image_2 have been rectified.
.
did I misunderstand these?

As you said, the evaluation results of dataset need to use your function vod.evaluate.

However, the output of this function does not include mAOS.

How should I get this result?

Hello, after using the method similar to PointPillars to convert a point cloud into BEV view, is the vehicle located on the far left with the front facing to the right? The image is a point cloud BEV feature map that I visualized based on the official PP

Good morning everyone.

First of all, thank you for your incredible dataset. Deep Radar Perception can continue its development thanks to dataset like this one.

I was exploring the dataset and training a YOLO v5 model to take advantage of the camera images. I have observed that there is an error that occurs systematically and makes the detector's performance decrease. There are images in which a road user is annotated within the full image (tending to be 100% width and 100% height).

Here you can have an example at 01064.txt.

Car 908 2 -2.6248431176513627 0.0 0.0 1935.0 1215.0 1.9223383609753752 2.0535622747106395 4.999146108042289 2.9836005613724135 1.6872684689437945 2.159001243918928 -1.6804516475522036 1

You can observe there 0.0 0.0 1935.0 1215.0, which is the full size of the image.

Personally, I am working with classes Pedestrian, Cyclist and Car and I have found 183 of these issues in the dataset. If you want more information, I am open to collaborate through this issue.

Best regards.

Hello, thank you for your excellent dataset.
I found in your evaluation, that gt values are added by 0.01. What's the reason for that?

frame 00001 00002 , using API vis_3d.draw_plot(annotations_plot=True),error occurred, but there is nothing wrong with the label.


frame 00356 worked.

Hi!

How do I know the x, y axis values that the ego-car difference between frames has moved?

Hi, I was wondering if it would be possible to publish performance results for pointpillars on the validation set of the data? When and how will we be able to test our own custom models on the test set? Thank you!

Hello, authors.
Could you tell me the difference between radar，radar_3_scans, and radar_5_scans in the dataset?
Best regards.

How to use your assessment method, get mAP and mAOS, here is my data:
#2023-07-05 14:16:18,853 INFO Car [email protected], 0.70, 0.70: bbox AP:25.2931, 35.8688, 30.2304 bev AP:34.3534, 41.2079, 37.0070 3d AP:13.5977, 17.5404, 14.6623 aos AP:22.00, 34.01, 28.79 Car [email protected], 0.70, 0.70: bbox AP:21.6496, 32.8158, 26.9882 bev AP:31.7833, 40.4042, 33.8964 3d AP:7.0489, 12.1813, 9.6848 aos AP:17.99, 30.88, 25.39 Car [email protected], 0.50, 0.50: bbox AP:25.2931, 35.8688, 30.2304 bev AP:53.9248, 58.3119, 50.3598 3d AP:34.7278, 41.2159, 36.9743 aos AP:22.00, 34.01, 28.79 Car [email protected], 0.50, 0.50: bbox AP:21.6496, 32.8158, 26.9882 bev AP:52.7421, 57.3406, 49.3869 3d AP:31.6778, 40.5127, 33.8923 aos AP:17.99, 30.88, 25.39 Pedestrian [email protected], 0.50, 0.50: bbox AP:28.7664, 26.7176, 22.2800 bev AP:26.4072, 21.3220, 20.4711 3d AP:16.2190, 13.5128, 13.1590 aos AP:28.29, 26.09, 21.88 Pedestrian [email protected], 0.50, 0.50: bbox AP:26.2926, 21.6179, 19.5190 bev AP:22.2260, 17.9081, 16.0608 3d AP:9.9817, 8.1055, 6.9494 aos AP:25.61, 20.92, 18.87 Pedestrian [email protected], 0.25, 0.25: bbox AP:28.7664, 26.7176, 22.2800 bev AP:48.2595, 40.4788, 33.9854 3d AP:38.4618, 31.6497, 30.0935 aos AP:28.29, 26.09, 21.88 Pedestrian [email protected], 0.25, 0.25: bbox AP:26.2926, 21.6179, 19.5190 bev AP:46.2196, 37.6173, 33.2550 3d AP:36.3881, 30.1446, 26.2724 aos AP:25.61, 20.92, 18.87 Cyclist [email protected], 0.50, 0.50: bbox AP:60.9057, 58.7827, 52.4480 bev AP:70.8855, 63.8257, 56.6782 3d AP:36.7310, 31.9714, 31.0531 aos AP:60.82, 58.68, 52.36 Cyclist [email protected], 0.50, 0.50: bbox AP:62.0980, 56.1747, 50.1514 bev AP:71.2272, 63.1126, 56.4553 3d AP:33.3966, 29.5327, 26.2988 aos AP:62.00, 56.07, 50.06 Cyclist [email protected], 0.25, 0.25: bbox AP:60.9057, 58.7827, 52.4480 bev AP:84.8932, 76.9164, 68.6398 3d AP:81.7922, 73.7306, 66.0486 aos AP:60.82, 58.68, 52.36 Cyclist [email protected], 0.25, 0.25: bbox AP:62.0980, 56.1747, 50.1514 bev AP:86.0499, 78.9152, 71.5783 3d AP:81.5084, 72.7171, 65.6249 aos AP:62.00, 56.07, 50.06

Hello Author,

I am wondering When will the activity attribute annotations for objects be published.

Thank you so much in advance!

Thank you for your excellent dataset！
I have a question regarding Issue 8, where you explained the poor performance of PP-Lidar. In your explanation, you mentioned that you replaced truncation with other attributes, such as whether the object is moving or not. So, I would like to ask if it is possible to perform motion object detection on this dataset.

Hello, first of all, thank you for your work. I would like to know what is the field of view angle of the camera.

Hi, Thank you for making this great dataset! I have discovered some potential issues during exploration.

First looks like label files for seq 2532-3276 are missing

Some pose files are empty/incomplete
00544.json is incomplete

01312.json is empty

Thank you and hope this will helps. And please let me know if you would like me open as two issues. :D

I try to do the experiments in your paper using OpenPCDet.

For the PP-Lidar experiment, the model was trained with the default config(https://github.com/open-mmlab/OpenPCDet/blob/master/tools/cfgs/kitti_models/pointpillar.yaml), but the following results were obtained.

Compared to Car, Pedestrian and Cyclist have poor results. Especially Cyclist doesn't seem to be learning at all.
What am I missing?

Hi there,

Firstly great thanks for your dataset which has very good quality radar and LiDAR point clouds. However, we found that your data annotation doesn't include object ids, which are mainly for object tracking tasks. KITTI's tracking label format includes two extra parameters at the beginning for sequence number and tracking id, here's an example:

The KITTI tracking annotation format is specified as following:

However, I couldn't find any object id annotations. Will the object id annotation be released later? Many thanks in advance!

Hi everyone,

First of all, thank you for publishing this great dataset. Right now i am working on reproducing the paper results.
I followed the instructions within PP-Radar.md and get nearly the same mAP-scores as reported in the paper.
But the classwise AP-scores seem to differ from the paper-results to some extent.

Here is the comparison of the results for training of PP on single-frame radar data with all 7 features used:

The first line shows the paper results and the second line shows the reproduction results.
Now my question is why the classwise AP-scores differ up to ~10% from the paper results.
Did you use any additional configuration like a weighted loss-function or other things which would explain this difference, or am i missing something here?

Thanks in advance!
Best regards,

Alex