Result reproduction

[StudentWwg]

Congradulations to your great work that has been accepted by CVPR 2025.
When you train your model or reproduce SparseDrive, have you faced unsteady evaluation results in L2 and collision rate metrices?
I liste the evaluation results that I reproduce SparseDrive with 8 VIDIA GeForce RTX 3090.

+-------------+--------+--------+--------+--------+--------+--------+--------+
| metrics | 0.5s | 1.0s | 1.5s | 2.0s | 2.5s | 3.0s | avg |
+-------------+--------+--------+--------+--------+--------+--------+--------+
| obj_col | 0.742% | 0.713% | 0.684% | 0.659% | 0.649% | 0.638% | 0.670% |
| obj_box_col | 0.020% | 0.020% | 0.026% | 0.073% | 0.148% | 0.309% | 0.134% |
| L2 | 0.2388 | 0.3595 | 0.4970 | 0.6561 | 0.8369 | 1.0406 | 0.6854 |
+-------------+--------+--------+--------+--------+--------+--------+--------+

And I have checked the training losses, everything is normal. Have you faced the same evaluation issue?

[yilutum]

Congradulations to your great work that has been accepted by CVPR 2025. When you train your model or reproduce SparseDrive, have you faced unsteady evaluation results in L2 and collision rate metrices? I liste the evaluation results that I reproduce SparseDrive with 8 VIDIA GeForce RTX 3090.

+-------------+--------+--------+--------+--------+--------+--------+--------+ | metrics | 0.5s | 1.0s | 1.5s | 2.0s | 2.5s | 3.0s | avg | +-------------+--------+--------+--------+--------+--------+--------+--------+ | obj_col | 0.742% | 0.713% | 0.684% | 0.659% | 0.649% | 0.638% | 0.670% | | obj_box_col | 0.020% | 0.020% | 0.026% | 0.073% | 0.148% | 0.309% | 0.134% | | L2 | 0.2388 | 0.3595 | 0.4970 | 0.6561 | 0.8369 | 1.0406 | 0.6854 | +-------------+--------+--------+--------+--------+--------+--------+--------+

And I have checked the training losses, everything is normal. Have you faced the same evaluation issue?

Hi, I’ve run into a similar issue: the training losses look normal, but the evaluation metrics (L2 / collision rates) don’t match the paper’s results. My results are:

+-------------+--------+--------+--------+--------+--------+--------+--------+
| metrics | 0.5s | 1.0s | 1.5s | 2.0s | 2.5s | 3.0s | avg |
+-------------+--------+--------+--------+--------+--------+--------+--------+
| obj_col | 0.742% | 0.713% | 0.684% | 0.659% | 0.649% | 0.638% | 0.670% |
| obj_box_col | 0.234% | 0.215% | 0.163% | 0.181% | 0.250% | 0.358% | 0.251% |
| L2 | 0.2040 | 0.3190 | 0.4552 | 0.6173 | 0.8046 | 1.0162 | 0.6508 |
+-------------+--------+--------+--------+--------+--------+--------+--------+

Have you managed to resolve it? If so, could you share what helped you reach the paper’s results?

[fly-dk]

@yilutum 请问这是你复现BridgeAD的结果吗？
+-------------+--------+--------+--------+--------+--------+--------+--------+
| metrics | 0.5s | 1.0s | 1.5s | 2.0s | 2.5s | 3.0s | avg |
+-------------+--------+--------+--------+--------+--------+--------+--------+
| obj_col | 0.742% | 0.713% | 0.684% | 0.659% | 0.649% | 0.638% | 0.670% |
| obj_box_col | 0.234% | 0.215% | 0.163% | 0.181% | 0.250% | 0.358% | 0.251% |
| L2 | 0.2040 | 0.3190 | 0.4552 | 0.6173 | 0.8046 | 1.0162 | 0.6508 |
+-------------+--------+--------+--------+--------+--------+--------+--------+

[fly-dk]

@Zbozhou 作者你好，我按照同样的设置进行复现，发现检测和跟踪效果相比sparsedrive有一点提升，但是建图效果很差，L2和碰撞率也很差，跟论文结果不一致。请解释一下，或者提供官方权重！
+-------------+--------+--------+--------+--------+--------+--------+--------+
| metrics | 0.5s | 1.0s | 1.5s | 2.0s | 2.5s | 3.0s | avg |
+-------------+--------+--------+--------+--------+--------+--------+--------+
| obj_col | 0.742% | 0.713% | 0.684% | 0.659% | 0.649% | 0.638% | 0.670% |
| obj_box_col | 0.078% | 0.137% | 0.234% | 0.352% | 0.488% | 0.677% | 0.389% |
| L2 | 0.2107 | 0.3384 | 0.4932 | 0.6774 | 0.8898 | 1.1293 | 0.7151 |
+-------------+--------+--------+--------+--------+--------+--------+--------+

mAP: 0.4214
mATE: 0.5655
mASE: 0.2718
mAOE: 0.4897
mAVE: 0.2264
mAAE: 0.1824
NDS: 0.5371

AMOTA: 0.4060
AMOTP: 1.2341
RECALL: 0.5019
MOTAR: 0.6814
MOTA: 0.3720
MOTP: 0.6282
IDS: 474.0

ped_crossing= 0.1125
divider= 0.1403
boundary= 0.2260
mAP_normal= 0.1596

Car / Ped
epa= 0.5063 / 0.4379
ade= 0.6545 / 0.6948
fde= 1.0266 / 1.0024
mr= 0.1382 / 0.1327

obj_box_col: 0.389%
L2: 0.7151

[yilutum]

@yilutum 请问这是你复现BridgeAD的结果吗？ +-------------+--------+--------+--------+--------+--------+--------+--------+ | metrics | 0.5s | 1.0s | 1.5s | 2.0s | 2.5s | 3.0s | avg | +-------------+--------+--------+--------+--------+--------+--------+--------+ | obj_col | 0.742% | 0.713% | 0.684% | 0.659% | 0.649% | 0.638% | 0.670% | | obj_box_col | 0.234% | 0.215% | 0.163% | 0.181% | 0.250% | 0.358% | 0.251% | | L2 | 0.2040 | 0.3190 | 0.4552 | 0.6173 | 0.8046 | 1.0162 | 0.6508 | +-------------+--------+--------+--------+--------+--------+--------+--------+

对，尝试复现了若干次，L2和 col. rate始终无法达到论文效果。