Welcome to The RoboDrive Challenge! 👋

RoboDrive is one of the first competitions that targeted probing the Out-of-Distribution (OoD) robustness of state-of-the-art autonomous driving perception models, centered around two mainstream topics: common corruptions and sensor failures.

There are eighteen real-world corruption types in total, ranging from three perspectives:

• Weather and lighting conditions, such as bright, low-light, foggy, and snowy conditions.
• Movement and acquisition failures, such as potential blurs caused by vehicle motions.
• Data processing issues, such as noises and quantizations happen due to hardware malfunctions.

Additionally, we aim to probe the 3D scene perception robustness under camera and LiDAR sensor failures:

• Loss of certain camera frames during the driving system sensing process.
• Loss of one or more camera views during the driving system sensing process.
• Loss of the roof-top LiDAR view during the driving system sensing process.

Kindly visit our webpage to explore more details and instructions of this challenge. 🚙

This competition is affiliated with the 41st IEEE Conference on Robotics and Automation (ICRA 2024).

ICRA is the IEEE Robotics and Automation Society's flagship conference. ICRA 2024 will be held from May 13th to 17th, 2024, in Yokohama, Japan.

• Useful Info
• Timeline
• Challenge Tracks
• Data Preparation
• Getting Started
• Changelog
• Awards
• License
• Sponsor
• Citation
• Terms & Conditions
• Frequently Asked Questions
• Organizers
• Affiliation
• Acknowledgements

#	Item	Link
🌐	Competition Webpage	https://robodrive-24.github.io
🔧	Competition Toolkit	https://github.com/robodrive-24/toolkit
	Official GitHub Account	https://github.com/robodrive-24
📫	Contact	[email protected]

Note: All timestamps are in the AoE (Anywhere on Earth) format.

• Dec 25 '23 - Team Up; Register for your team by filling in this Google Form.
• Jan 05 '24 - Release of training and evaluation data.
• Jan 15 '24 - Competition servers online @ CodaLab.
• Mar 31 '24 - Phase One deadline.
• Apr 30 '24 - Phase Two deadline.
• May 17 '24 - Award decision announcement @ ICRA 2024.

There are five tracks in this RoboDrive challenge, with emphasis on the following 3D scene perception tasks:

Our evaluation servers were developed based on the CodaLab platform. 🙆‍♂️

CodaLab is an open-source web-based platform that enables researchers, developers, and data scientists to collaborate, with the goal of advancing research fields where machine learning and advanced computation are used.

Kindly refer to DATA_PREPARE.md for the details to prepare the training and evaluation data.

Kindly refer to GET_STARTED.md to learn more usage of this toolkit.

• Jan 22 '24 - The evaluation server of Track 4 is online. [Server4]
• Jan 15 '24 - The evaluation server of Track 5 is online. [Server5]
• Jan 15 '24 - The evaluation server of Track 3 is online. [Server3]
• Jan 15 '24 - The evaluation server of Track 2 is online. [Server2]
• Jan 15 '24 - The evaluation server of Track 1 is online. [Server1]
• Jan 12 '24 - Instructions, baseline models, and results of Track 5 have been released.
• Jan 12 '24 - Training and evaluation data of Track 5 have been released.
• Jan 08 '24 - A list of Frequently Asked Questions (FAQs) has been summarized for better clarity.
• Jan 06 '24 - Instructions, baseline models, and results of Tracks 1 to 3 have been released.
• Jan 05 '24 - Training and evaluation data of Tracks 1 to 4 have been released.
• Dec 25 '23 - Register for your team by filling in this Google Form.
• Dec 01 '23 - Launch of The RoboDrive Challenge at ICRA 2024. More info coming soon!

The top-performing participants of this competition are honored with cash awards and certificates.

Note: The cash awards are donated by our sponsors and are shared among five tracks. We reserve the right to examine the validity of each submission. For more information, kindly refer to the Terms & Conditions section.

Additionally, we provide the following awards for participants that meet certain conditions.

This competition is under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

This competition is supported by HUAWEI Noah's Ark Lab.

The Noah’s Ark Lab is the AI research center for Huawei Technologies, aiming to make significant contributions to both the company and society by innovating in artificial intelligence, data mining, and related fields.

If you find this competition helpful for your research, kindly consider citing our papers:

@article{xie2023robobev,
    title = {RoboBEV: Towards Robust Bird's Eye View Perception under Corruptions},
    author = {Shaoyuan Xie and Lingdong Kong and Wenwei Zhang and Jiawei Ren and Liang Pan and Kai Chen and Ziwei Liu},
    journal = {arXiv preprint arXiv:2304.06719}, 
    year = {2023}
}

@inproceedings{kong2023robodepth,
    title = {RoboDepth: Robust Out-of-Distribution Depth Estimation under Corruptions},
    author = {Lingdong Kong and Shaoyuan Xie and Hanjiang Hu and Lai Xing Ng and Benoit R. Cottereau and Wei Tsang Ooi},
    booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
    year = {2023},
}

@inproceedings{kong2023robo3d,
    title = {Robo3D: Towards Robust and Reliable 3D Perception against Corruptions},
    author = {Lingdong Kong and Youquan Liu and Xin Li and Runnan Chen and Wenwei Zhang and Jiawei Ren and Liang Pan and Kai Chen and Ziwei Liu},
    booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
    pages = {19994-20006},
    year = {2023},
}

@misc{mmdet3d,
    title = {MMDetection3D: OpenMMLab Next-Generation Platform for General 3D Object Detection},
    author = {MMDetection3D Contributors},
    howpublished = {\url{https://github.com/open-mmlab/mmdetection3d}},
    year = {2020}
}

In the meantime, kindly cite the technical reports of the nuScenes dataset and the CodaLab platform:

@inproceedings{caesar2020nuscenes,
    title={nuScenes: A Multimodal Dataset for Autonomous Driving},
    author={Holger Caesar and Varun Bankiti and Alex H. Lang and Sourabh Vora and Venice Erin Liong and Qiang Xu and Anush Krishnan and Yu Pan and Giancarlo Baldan and Oscar Beijbom},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    pages = {11621-11631},
    year={2020}
}

@article{pavao2023codalab,
    title = {CodaLab Competitions: An Open Source Platform to Organize Scientific Challenges},
    author = {Adrien Pavão and Isabelle Guyon and Anne-Catherine Letournel and Dinh-Tuan Tran and Xavier Baró and Hugo Jair Escalante and Sergio Escalera and Tyler Thomas and Zhen Xu},
    journal = {Journal of Machine Learning Research (JMLR)},
    pages = {1-6},
    year = {2023}
}

This competition is made freely available to academic and non-academic entities for non-commercial purposes such as academic research, teaching, scientific publications, or personal experimentation. Permission is granted to use the data given that you agree:

1. That the data in this competition comes “AS IS”, without express or implied warranty. Although every effort has been made to ensure accuracy, we do not accept any responsibility for errors or omissions.
2. That you may not use the data in this competition or any derivative work for commercial purposes such as, for example, licensing or selling the data, or using the data with a purpose of procuring a commercial gain.
3. That you include a reference to RoboDrive (including the benchmark data and the specially generated data for academic challenges) in any work that makes use of the benchmark. For research papers, please cite our preferred publications as listed on our webpage.

To ensure a fair comparison among all participants, we require:

1. All participants must follow the exact same data configuration when training and evaluating their algorithms. Please do not use any public or private datasets other than those specified for model training.
2. The theme of this competition is to probe the out-of-distribution robustness of autonomous driving perception models. Therefore, any use of the corruption and sensor failure types designed in this benchmark is strictly prohibited, including any atomic operation that comprises any one of the mentioned corruptions.
3. To ensure the above two rules are followed, each participant is requested to submit the code with reproducible results before the final result is announced; the code is for examination purposes only and we will manually verify the training and evaluation of each participant's model.

📫 Didn't find a related FAQ to your questions? Let us know ([email protected])!

This competition is developed based on the RoboBEV, RoboDepth, and Robo3D projects.

This competition toolkit is developed based on the MMDetection3D codebase.

MMDetection3D is an open-source toolbox based on PyTorch, towards the next-generation platform for general 3D perception. It is a part of the OpenMMLab project developed by MMLab.

The evaluation sets of this competition are constructed based on the nuScenes dataset from Motional AD LLC.

Part of the content of this toolkit is adopted from The RoboDepth Challenge @ ICRA 2023.