Evaluate on the KITTI Benchmark

[hoangduyloc]

Hi Matssteinweg,
Did you evaluate the result of this code to the official BEV KITTI benchmark?

[matssteinweg]

Hi, no I didn't. The reported evaluation results are computed on 1000 samples taken from the original KITTI training set.

[hoangduyloc]

As my current understanding. In the "load_data.py" file.

• For Train / Val: You will randomly choose 6481 frames of 7481 and then split 90% for training, 10% for validation.
• For Test: You will randomly choose 1000 frames of 7481.
• So I have a question here: Is the Test set may overlap with Train/Val set? Am I right?
  Thank you!

[matssteinweg]

No, I only use two splits, train and test. The official KITTI training set is split into 6481 and 1000 samples. If you look at the "load_data.py" file this should become clear. Also, in "evaluate_model.py" you can see that I only load the test set for evaluation.

[hoangduyloc]

Yes, I see that you load only test set for evaluation. So the train/test set are fixed right? train set starts form [0-6480]; test set starts form [6481-7481]?
Thank you so much.

[matssteinweg]

Yes, the sets are fixed. It is standard practice and used in a number of papers to split the available KITTI training set since there's no labels available for the test set. Unfortunately, I don't remember where I downloaded the exact data split. Sorry for that. I'm sure you can find a suitable split somewhere, though.
Let me know if you have further questions or run into problems with the implementation.

[hoangduyloc]

Thanks for your work and your helps!