How to use my own dataset for training?

[xiannsa]

Excuse me, is it true that the method for training my own dataset involves preparing a folder to store images for training, write a new dataloader by imitating wireframe adding it to dataset_util.py, and then following the provided five steps for training?

[rpautrat]

Yes, that is a good summary for re-training on a custom dataset!

[xiannsa]

Excuse me, I imitated Holicity's dataloader, and after the second step of training, no files were generated. So how should I proceed with the third step? I hope you can give some advice.

[abhiagwl4262]

@rpautrat

I converted my dataset which is in lcnn format currently to wireframe format using -
https://github.com/Delay-Xili/F-Clip/blob/master/dataset/wireframe.py &
https://github.com/Delay-Xili/F-Clip/blob/master/dataset/wireframe_line.py

I hope that also works.

[rpautrat]

Excuse me, I imitated Holicity's dataloader, and after the second step of training, no files were generated. So how should I proceed with the third step? I hope you can give some advice.

Did any error happen? Step 2 should be generating the ground truth files. Did you also check that the paths are correct, for example where was the output path that you selected?

[rpautrat]

@rpautrat

I converted my dataset which is in lcnn format currently to wireframe format using - https://github.com/Delay-Xili/F-Clip/blob/master/dataset/wireframe.py & https://github.com/Delay-Xili/F-Clip/blob/master/dataset/wireframe_line.py

I hope that also works.

I am not sure to understand the difference between these two formats, but if you use the one of the original Wireframe dataset, this should be fine.

[xiannsa]

Thank you for your help,I have completed the third step. However, I have encountered a new issue while training in the fourth step: "Input type (torch.cuda.DoubleTensor) and weight type (torch.cuda.FloatTensor) should be the same.

[rpautrat]

This means that there is a mismatch of types: the model weights are torch.float, while some input (probably your input images) are torch.double. You need to convert your images to float with images.float().

[xiannsa]

Hello, thank you for your help earlier. I was able to successfully complete step four, but I noticed that the training performance did not improve significantly as the epoch count increased. During training, I encountered a runtime warning about invalid value encountered in intersection return lib.intersection(a, b, **kwargs). Eventually, when attempting to continue with step five, I received an error stating KeyError: 'ref_junction_map'. Can you please offer some assistance?

[rpautrat]

Hi, did you start training using the pre-trained weights? If so, it would make sense that the training performance will not improve much on your custom dataset.

Regarding the warning, I am not sure, I would need to see the full message, but it is probably not too serious.

For step 5, did you make sure to put the option 'return_type' to 'paired_desc' in the config file of your custom dataset?

[xiannsa]

Thank you for your message. I have already begun the fifth step of my training. Regarding the fourth step, I used my own dataset and did not use a pre-trained model from the first step. I am curious to know if the training results were very poor.

[rpautrat]

These results are actually not very poor, they are similar to what I have. The training metrics are pixel-level precision and recall, so it is very hard to get a good score in these. But the overall quality of your lines should already pretty good now. Did you try visualize them?

[xiannsa]

"I'm on step 5 of my training, and I used the model file from round 11 that was just trained to draw a picture. The result had more lines than when I used the wireframe.tar you provided, but there were also more mistakes. I hope that the models trained in the future will have better results."

[rpautrat]

The matching looks very good at least, but the lines could indeed be improved. Note that you can also tune the hyperparameters to make it better.

One issue here is for example the lack of junctions, which prevent from discovering too many lines. You could try reducing the parameter 'detection_thresh' of the model config file train_full_pipeline.yaml, so that the network can detect more junctions.

[abhiagwl4262]

@rpautrat can you just help me with .mat files for validation data ? what is the scale the lines have been resized to ?

Should I scale my lines to (512, 512) or (500, 500) or something else ?

[rpautrat]

I am sorry, which .mat file? What do you want to validate exactly? If you follow the metrics of the paper, there should be no .mat file needed, the metrics are computed between line detections across two images warped by a homography.