Hi,
My name is Henry and I'm a rising senior in high school. This summer I reproduced the segmentation stage of your taxiing localization paper and I ran into something I couldn't figure out on my own and wanted to ask about.
Using best_model_avg.pth from your Hugging Face repo on the shipped validation split, I get centerline 0.7275 and pavement 0.9824 which are both within half a percent of Table 3. My confusion was because hold-short comes out at 0.3505 instead of 0.6645.
I checked whether it was the decision threshold, the split, or how IoU gets averaged, and none of those explain it. The checkpoint is only finding about 40% of the hold-short pixels.
Was Table 3 made with this checkpoint, or a different one? I noticed the training code also saves best_model_worstcase.pth, which isn't on Hugging Face.
Happy to send the full numbers if that's useful. Thanks for releasing the code
and the dataset, I really enjoyed the reading and the opportunity to give it a try.
Best,
Henry
Hi,
My name is Henry and I'm a rising senior in high school. This summer I reproduced the segmentation stage of your taxiing localization paper and I ran into something I couldn't figure out on my own and wanted to ask about.
Using best_model_avg.pth from your Hugging Face repo on the shipped validation split, I get centerline 0.7275 and pavement 0.9824 which are both within half a percent of Table 3. My confusion was because hold-short comes out at 0.3505 instead of 0.6645.
I checked whether it was the decision threshold, the split, or how IoU gets averaged, and none of those explain it. The checkpoint is only finding about 40% of the hold-short pixels.
Was Table 3 made with this checkpoint, or a different one? I noticed the training code also saves best_model_worstcase.pth, which isn't on Hugging Face.
Happy to send the full numbers if that's useful. Thanks for releasing the code
and the dataset, I really enjoyed the reading and the opportunity to give it a try.
Best,
Henry