| title | ICCV 2023 SLAM Challenge |
|---|---|
| subtitle | Multi-Robot, Multi-Degraded SLAM Challenge |
| layout | page |
| show_sidebar | false |
| hide_footer | false |
| menubar_toc | true |
| hero_height | is-large |
| hero_image | /img/datasets/dataset_video_short.gif |
td th { text-align: center !important; vertical-align: middle !important; }
.tooltip { position: relative; display: inline-block; }
.tooltip .tooltiptext { opacity: 0%; width: 120px; background-color: black; color: #fff; text-align: center; border-radius: 6px; padding: 5px 0;
position: absolute; z-index: 1; top: 100%; left: 50%; margin-left: -60px; }
.tooltip:hover .tooltiptext { opacity: 40%; } </style>
We provide datasets TartanAir and SubT-MRS, aiming to push the rubustness of SLAM algorithms in challenging environments, as well as advancing sim-to-real transfer. Our datasets contain a set of perceptually degraded environments such as darkness, airborne obscurant conditions such as fog, dust, smoke and lack of prominent perceptual features in self-similar areas. One of our hypotheses is that in such challenging cases, the robot needs to rely on multiple sensors to reliably track and localize itself. We provide a rich set of sensory modalities including RGB images, LiDAR points, IMU measurements, thermal images and so on.
Feel free to check our Paper and cite our work if it is useful ^ ^.
@InProceedings{Zhao_2024_CVPR,
author = {Zhao, Shibo and Gao, Yuanjun and Wu, Tianhao and Singh, Damanpreet and Jiang, Rushan and Sun, Haoxiang and Sarawata, Mansi and Qiu, Yuheng and Whittaker, Warren and Higgins, Ian and Du, Yi and Su, Shaoshu and Xu, Can and Keller, John and Karhade, Jay and Nogueira, Lucas and Saha, Sourojit and Zhang, Ji and Wang, Wenshan and Wang, Chen and Scherer, Sebastian},
title = {SubT-MRS Dataset: Pushing SLAM Towards All-weather Environments},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2024},
pages = {22647-22657}
}
<iframe width="100%" height="400" style="display: block; margin-left: auto; margin-left: auto; width: 250%;" src="https://www.youtube.com/embed/gYeQJt9SNX4" title="Website - Sensor Video" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
Please note challenge deadline: 15th October 2023 11:59 PM EDT
Time remaining:
The SubT-MRS Dataset is an exceptional real-world collection of challenging datasets obtained from Subterranean Environments, encompassing caves, urban areas, and tunnels. Its primary focus lies in testing robust SLAM capabilities and is designed as Multi-Robot Datasets, featuring UGV, UAV, and Spot robots, each demonstrating various motions. The datasets are distinguished as Multi-Spectral, integrating Visual, Lidar, Thermal, and inertial measurements, effectively enabling exploration under demanding conditions such as darkness, smoke, dust, and geometrically degraded environments.Key features of our dataset:
1. Multiple Modalities: Our dataset includes hardware time-synchronized data from 4 RGB cameras, 1 LiDAR, 1 IMU, and 1 thermal camera, providing diverse and precise sensor inputs.
2.Diverse Scenarios: Collected from multiple locations, the dataset exhibits varying environmental setups, encompassing indoors, outdoors, mixed indoor-outdoor, underground, off-road, and buildings, among others.
3.Multi-Degraded: By incorporating multiple sensor modalities and challenging conditions like fog, snow, smoke, and illumination changes, the dataset introduces various levels of sensor degradation.
4. Heterogeneous Kinematic Profiles: The SubT-MRS Dataset uniquely features time-synchronized sensor data from diverse vehicles, including RC cars, legged robots, drones, and handheld devices, each operating within distinct speed ranges.
The TartanAir dataset is collected in photo-realistic simulation environments based on the AirSim project. A special goal of this dataset is to focus on the challenging environments with changing light conditions, adverse weather, and dynamic objects. The four most important features of our dataset are:
1. Large size diverse realistic data: We collect the data in diverse environments with different styles, covering indoor/outdoor, different weather, different seasons, urban/rural.
2. Multimodal ground truth labels: We provide RGB stereo, depth, optical flow, and semantic segmentation images, which facilitates the training and evaluation of various visual SLAM methods.
3. Diversity of motion patterns: Our dataset covers much more diverse motion combinations in 3D space, which is significantly more difficult than existing datasets.
4. Challenging Scenes: We include challenging scenes with difficult lighting conditions, day-night alternating, low illumination, weather effects (rain, snow, wind and fog) and seasonal changes.Please refer to the TartanAir Dataset and the paper for more information.
ROS bag format: Google Baidu
Folder format: Google Baidu
Ground Truth: Google Baidu
| Name | Source | Robot |Sensor | Description | Video | Calibration (Extrinsics) | |---|-----------|---------|-----------|-----------|------------|-----------|-----------|---------------|--------------| | Final_Challenge_UGV1 | SubT-MRS | UGV1 | LiDAR,IMU | Geometry Degraded |link| Google Baidu | | Final_Challenge_UGV2 | SubT-MRS | UGV2 | LiDAR,IMU | Geometry Degraded |link| Google Baidu | | Final_Challenge_UGV3 | SubT-MRS | UGV3 | LiDAR,IMU | Geometry Degraded |link| Google Baidu | | Urban_Challenge_UGV1 | SubT-MRS | UGV1 | LiDAR,IMU | Geometry Degraded |link| Google Baidu | | Urban_Challenge_UGV2 | SubT-MRS | UGV2 | LiDAR,IMU | Geometry Degraded |link| Google Baidu | | Laurel_Cavern | SubT-MRS | Handheld | LiDAR,IMU | Underground Cave |link| Google Baidu | | Lidar_factory | TartanAir | Virtual Sensors | LiDAR,IMU | Snow | | Google Baidu | | Lidar_ocean | TartanAir | Virtual Sensors | LiDAR,IMU | Dynamic Objects | | Google Baidu | | Lidar_sewerage | TartanAir | Virtual Sensors | LiDAR,IMU | Geometry Degraded | | Google
Extra Dataset from Sensor Fusion Challenge
ROS bag format: Google Baidu
Folder format: Google Baidu
Ground Truth: Google Baidu
| Name | Source | Robot |Sensor | Description | Video | Calibration (Extrinsics) | Calibration (Intrinsics) | |---|-----------|---------|-----------|-----------|------------|-----------|-------------|-----------|---------------|--------------| |Multi_Floor|SubT-MRS|SP1|Lidar,RGB,IMU|Multi Floor|link| Google Baidu | Google Baidu | |Long_Corridor|SubT-MRS|RC2|Lidar,RGB,IMU|Multi Floor|link| Google Baidu | Google Baidu | |BlockLiDAR|SubT-MRS|SP1|Lidar,RGB,IMU|Block Lidar|link| Google Baidu | Google Baidu | |BlockVisual|SubT-MRS|SP1|RGB,IMU,Thermal|Block Visual/Thermal|link| Google Baidu | Google Baidu | |SmokeRoom|SubT-MRS|RC7|RGB,Thermal,IMU|Visual Degraded|link| Google Baidu | Google Baidu | |OutdoorNight|SubT-MRS|SP1|RGB,Thermal,IMU|Visual Degraded|link| Google Baidu | Google Baidu | |FlashLight|SubT-MRS|SP1|RGB,Thermal,IMU|Visual Degraded|link| Google Baidu | Google Baidu |
| Name | Source | Robot |Sensor | Rosbag | Calibration (Extrinsics) | Calibration (Intrinsics) | GT Map | GT Traj. | |------|-----------|-----------|-----------|-----------|-------------|-----------|-------------|-----------|---------------|--------------|--------------| |Cave01 |SuperLoc|Handheld|RGB,LiDAR,IMU|link| Google Baidu | Google Baidu | link | link |Cave02 |SuperLoc|Handheld|RGB,LiDAR,IMU|link| Google Baidu | Google Baidu | link | link |Cave03 |SubT-MRS|Handheld|RGB,LiDAR,IMU|link| Google Baidu | Google Baidu | link | link |Cave04 |SuperLoc|Handheld|RGB,LiDAR,IMU|link| Google Baidu | Google Baidu | link | link |Corridor01 |SubT-MRS|RC2 |RGB,LiDAR IMU|link| Google Baidu | Google Baidu |link | link |Corridor02 |SuperLoc|RC1 |RGB,LiDAR IMU|link| Google Baidu | Google Baidu | link | link |Floor01 |SubT-MRS|SP1|RGB,LiDAR,IMU|link| Google Baidu | Google Baidu | link | link |Floor02 (bonus) |SuperLoc|SP1|RGB,LiDAR,IMU|link| Google Baidu | Google Baidu | link | link
We also provide initialization poses for each dataset to support localization modules that allows initial pose configuration. You can find corresponding initial pose config for each dataset here.
Ground truth trajecotry follows TUM format,
timestamp x y z q_x q_y q_z q_wTo convert Velodyne packet message into sensor_msgs/PointCloud2, you can follow this tutorial link.
🚀 The Challenge is Live! 🚀 Join us for three exciting tracks of challenges from the links below.
Visual Inertial Track LiDAR Inertial Track Sensor Fusion Track
🚀 You can participate either Visual-inertial challenge or
LiDAR-inertial challenge Or participate in Sensor Fusion challenge.
Three separate awards will be given for each track. Your SLAM performance in the Sensor Fusion track will not impact the scores in other tracks.
Join us now to become a vital part of cutting-edge advancements in robotics and sensor fusion! 🤖💡 Let your expertise shine in this thrilling competition!
We thank Nvidia for supporting our challenge! The winning team will be awarded with an Orin Developer Kit and Orin Nano Developer Kit!
If you have any question, please do not hesitate to post issues on this github website. We would love to hear from your feedback! Every post will be responded with no spared effort within 36 hours.
Wenshan, Wang et al. "TartanAir: A Dataset to Push the Limits of Visual SLAM" Paper
@misc{wang2020tartanair,
title={TartanAir: A Dataset to Push the Limits of Visual SLAM},
author={Wenshan Wang and Delong Zhu and Xiangwei Wang and Yaoyu Hu and Yuheng Qiu and Chen Wang and Yafei Hu and Ashish Kapoor and Sebastian Scherer},
year={2020},
eprint={2003.14338},
archivePrefix={arXiv},
primaryClass={cs.RO}
}
Kamak Ebadi, et al. "Present and future of slam in extreme underground environments" Paper
@misc{ebadi2022present,
title={Present and Future of SLAM in Extreme Underground Environments},
author={Kamak Ebadi and Lukas Bernreiter and Harel Biggie and Gavin Catt and Yun Chang and Arghya Chatterjee and Christopher E. Denniston and Simon-Pierre Deschênes and Kyle Harlow and Shehryar Khattak and Lucas Nogueira and Matteo Palieri and Pavel Petráček and Matěj Petrlík and Andrzej Reinke and Vít Krátký and Shibo Zhao and Ali-akbar Agha-mohammadi and Kostas Alexis and Christoffer Heckman and Kasra Khosoussi and Navinda Kottege and Benjamin Morrell and Marco Hutter and Fred Pauling and François Pomerleau and Martin Saska and Sebastian Scherer and Roland Siegwart and Jason L. Williams and Luca Carlone},
year={2022},
eprint={2208.01787},
archivePrefix={arXiv},
primaryClass={cs.RO}
}
<script> let calibrations; let datasetsStr, datasets; let datasetTable; let chipsInput; let options = { placeholder: "Filter datasets...", secondaryPlaceholder: "+Tag", autocompleteOptions: { data: { Lidar: null, IMU: null, Thermal: null, Subt: null, UAV: null, UGV: null, }, limit: Infinity, minLength: 0, }, }; document.addEventListener("DOMContentLoaded", function () { let elems = document.querySelectorAll(".chips"); M.Chips.init(elems, options); chipsInput = M.Chips.getInstance(elems[0]); }); datasetTable = document.getElementById("datasetTable"); loadCalibrationCsv(() => { loadDatasetCsv(); }); function openAutoComplete() { setTimeout(function () { if (!chipsInput.autocomplete.isOpen) { console.log("openAutoComplete"); chipsInput.autocomplete.open(); } }, 200); } function onFilterButton() { let startTime = new Date().getTime(); let filter = chipsInput.chipsData.map((chip) => chip.tag.toLowerCase()); filterTable(filter); console.log("Filter time used: " + (new Date().getTime() - startTime) + "ms"); } function loadCalibrationCsv(callback) { makeRequest("/datasets/calibrations.csv", "", (str) => { let rows = $.csv.toArrays(str); calibrations = {}; for (let i = 0; i < rows.length; i++) { calibrations[rows[i][0]] = rows[i][1]; } if (callback) callback(); }); } function loadDatasetCsv(callback) { makeRequest("/datasets/datasets.csv", "", (str) => { datasetsStr = str.split("\n"); for (let i = 0; i < datasetsStr.length; i++) { datasetsStr[i] = datasetsStr[i].toLowerCase(); } let rows = $.csv.toArrays(str); datasets = []; for (let i = 0; i < rows.length; i++) { let dataset = {}; dataset.id = rows[i][0].trim(); dataset.name = rows[i][1].trim(); dataset.location = rows[i][2].trim(); dataset.description = rows[i][3].trim(); dataset.robot = rows[i][4].trim(); dataset.sensors = rows[i][5].trim(); dataset.degraded = rows[i][6].trim(); dataset.trajectoryLength = rows[i][7].trim(); dataset.duration = rows[i][8].trim(); dataset.returnToOrigin = rows[i][9].trim(); dataset.size = rows[i][10].trim(); dataset.image = rows[i][11].trim() !== "" ? rows[i][11].trim() : `/datasets/img/${dataset.id}.png`; dataset.link = rows[i][12].trim(); datasets.push(dataset); } datasetTable.innerHTML = ""; for (let i = 1; i < datasets.length; i++) { let row = datasets[i]; generateRow(datasetTable, i, [ makeDownloadLink(row.name, row.link), row.location, makeRobotIDs(row.robot), row.sensors, row.description, row.degraded, makeLengthDuration(row.trajectoryLength, row.duration), row.returnToOrigin, row.size, makePicture(i), ]); } filterTable(); if (callback) callback(); }); } function filterTable(filters) { if (filters === undefined) filters = []; let count = 0; for (let i = 1; i < datasets.length; i++) { let isIncluded = true; for (let j = 0; j < filters.length; j++) { if (datasetsStr[i].indexOf(filters[j]) < 0) { isIncluded = false; break; } } if (isIncluded) { document.getElementById(`row-${i}`).style.display = ""; count++; } else { document.getElementById(`row-${i}`).style.display = "none"; } } document.getElementById("numResultLabel").innerHTML = `(Showing ${count} datasets)`; } function generateRow(table, rowIdx, dataArr) { let row = document.createElement("tr"); row.id = "row-" + rowIdx; for (let i = 0; i < dataArr.length; i++) { let cell = document.createElement("td"); if (typeof dataArr[i] === "string") { cell.innerHTML = dataArr[i]; } else { cell.appendChild(dataArr[i]); } row.appendChild(cell); } return table.appendChild(row); } function makeLengthDuration(length, duration) { return `${length}
(${duration})`; } function makePicture(idx) { let img = document.createElement("img"); img.id = `picture-${idx}`; img.idx = idx; img.src = datasets[idx].image; img.alt = "Loading..."; img.style.width = "178px"; img.style.height = "100px"; img.style.objectFit = "scale-down"; return img; } function makeRobotIDs(allIds) { let ids = allIds.split(","); let result = ""; for (let i = 0; i < ids.length; i++) { let id = ids[i].trim(); if (id === "") continue; result += makeCalibrationLink(id); if (i < ids.length - 1) result += ", "; } return result; } function makeCalibrationLink(name) { if (calibrations[name] === undefined) alert("Calibration not found: " + name); let link = calibrations[name]; if (link.indexOf("https://") < 0 && link.indexOf("http://") < 0) link = "https://" + link; let result = `${name}`; result += `Download ${name} calibration`; result += ``; return result; } function makeDownloadLink(name, link) { if (link.indexOf("https://") < 0 && link.indexOf("http://") < 0) link = "https://" + link; let result = `${name}`; result += `Download dataset`; result += ``; return result; } function makeDownloadButton(link) { if (link.indexOf("https://") < 0 && link.indexOf("http://") < 0) link = "https://" + link; return `Download`; } function makeRequest(url, data, callback) { let httpRequest = new XMLHttpRequest(); if (!httpRequest) return false; function receivedResponse() { if (httpRequest.readyState === XMLHttpRequest.DONE) { if (httpRequest.status === 200) { console.log("RECEIVED RESPONSE: " + httpRequest.responseText); callback(httpRequest.responseText); } else { console.log("ERROR: Request failed"); alert("Unable to load datasets."); } } } httpRequest.timeout = 0; httpRequest.onreadystatechange = receivedResponse; console.log("SENDING REQUEST: " + url); httpRequest.open("GET", url, true); httpRequest.setRequestHeader("Content-Type", "application/x-www-form-urlencoded"); httpRequest.send(data); } </script> <style> p1 { text-align: center; font-size: 60px; margin-top: 0px; margin-left: 150px; } </style> <script> // Set the date we're counting down to // var countDownDate = new Date("Sept 25, 2023 15:37:25").getTime(); var countDownDate = Date.UTC(2023, 09, 16, 4, 0, 0); // Update the count down every 1 second var x = setInterval(function() { // Get today's date and time var now = new Date().getTime(); // var new=Date.UTC(date.getUTCMonth(), // date.getUTCDate(), // date.getUTCFullYear(), // date.getUTCHours(), // date.getUTCMinutes(), // date.getUTCSeconds()); // let utc= new Date(utcString); // var now = new (Date().getTime()).getTimezoneOffset(); // console.log(dt); // Gives Tue Mar 22 2016 09:30:00 GMT+0530 (IST) // var now=dt.setTime(dt.getTime()+dt.getTimezoneOffset()); // console.log(dt); // Gives Tue Mar 22 2016 04:00:00 GMT+0530 (IST) // var offset = -300; //Timezone offset for EST in minutes. // var estDate = new Date(dt.getTime() + offset*60*1000); // console.log(estDate); //Gives Mon Mar 21 2016 23:00:00 GMT+0530 (IST) // Find the distance between now and the count down date var distance = countDownDate - now; // Time calculations for days, hours, minutes and seconds var days = Math.floor(distance / (1000 * 60 * 60 * 24)); // var days=distance; var hours = Math.floor((distance % (1000 * 60 * 60 * 24)) / (1000 * 60 * 60)); var minutes = Math.floor((distance % (1000 * 60 * 60)) / (1000 * 60)); var seconds = Math.floor((distance % (1000 * 60)) / 1000); // Display the result in the element with id="demo" document.getElementById("demo").innerHTML = days + "d " + hours + "h " + minutes + "m " + seconds + "s "; // If the count down is finished, write some text if (distance < 0) { clearInterval(x); document.getElementById("demo").innerHTML = "EXPIRED"; } }, 1000); </script>
PhD Candidate Carnegie Mellon University |
Master's Student Carnegie Mellon University |
Research Associate Carnegie Mellon University |
Undergradate Student Carnegie Mellon University |
Undergradate Student Carnegie Mellon University |
Senior Field Robotics Specialist Carnegie Mellon University |
Graduate Student Carnegie Mellon University |
Research Associate Carnegie Mellon University |
Research Associate Carnegie Mellon University |
Undergraduate Student Carnegie Mellon University |
Research Associate Carnegie Mellon University |
MSR Student Carnegie Mellon University |
MSME Student Carnegie Mellon University |
Research Associate Carnegie Mellon University |