how to get result.json file for evaluation.

[niazahamd89]

Hi hezijian nice work on pose estimation. i have a question let me clear on it please. i work on your model i train it on coco train2017 and val2017 now i want to evaluate it on coco evaluation metric for that purpose i need result.json file, how i can get that json file for evaluation. let me explain that step by step thank you.

[scnuhealthy]

Sorry, I have not finished the evaluation code. You can reorganize the outputs into coco format and use the coco evaluation tool to evaluate. I think the other source code for keypoint detection in github will help you.

[KaiChen1998]

@niazahamd89 @scnuhealthy Same wish!

BTW, is there any python package which can transfer binary instance masks to polygon automatically like the matlab function mask2poly()?

[niazahamd89]

Thank you @scnuhealthy for the response. i am little confuse as you know that on coco evaluation dataset the model generate output in .ckpt format like

model.ckpt-495.data-00000-of-00001
model.ckpt-495.index
model.ckpt-495.meta

do i need to covert this file into coco format for evaluation or is there another way to get that result.json file from evaluation dataset?

let me clear for evaluation on coco demo.ipynb file i need the ground truth file which is provided by coco on there site and other file that i need is the pose detection file in json format. i trained the model now i am confuse what to do to get the second file how i can generate this file.

[scnuhealthy]

To convert binary instance masks to polygon, I think this is a good example: https://github.com/jinfagang/cityscapestococo. CityScape uses binary masks annotation, and COCO uses polygon annotation.

And you need you convert the result based on the output of our model.

[niazahamd89]

Sorry, I have not finished the evaluation code. You can reorganize the outputs into coco format and use the coco evaluation tool to evaluate. I think the other source code for keypoint detection in github will help you.

@scnuhealthy How i can reorganize the outputs into coco format what these outputs means, would you please provide me the link where this issue is solved. i am just stuck here. my main concern is how i can generate these file which is used for evaluation please check in the link

https://github.com/philferriere/cocoapi/tree/master/results

[KaiChen1998]

@niazahamd89 I think you might need to translate your result into COCO format by yourself maybe. Hope the link below will help you:

https://github.com/nightrome/cocostuffapi

[niazahamd89]

@niazahamd89 I think you might need to translate your result into COCO format by yourself maybe. Hope the link below will help you:

https://github.com/nightrome/cocostuffapi

@KaiChen1998 thanks for your response i studied the coco site for the format, i understand that but i don't get the point what the result mean there? they mean the .ckpt file which is generated after model training or plotting pose of one by one picture in evaluation set and store the plotted result?

do you have any idea in the second para here, they are talking about which results?
http://cocodataset.org/#format-results

[scnuhealthy]

.ckpt is just the trained model file, not the output result. I think the predict keypoints and instances masks shown in demo.py is the result you want. The pipeline should be: input one image, and output the detected skeletons and instance masks on the image. Save these results and transfer to COCO format.

[niazahamd89]

.ckpt is just the trained model file, not the output result. I think the predict keypoints and instances masks shown in demo.py is the result you want. The pipeline should be: input one image, and output the detected skeletons and instance masks on the image. Save these results and transfer to COCO format.

Thanks @scnuhealthy yes the demo.py file give the result in image but is it possible to generate this kind of array

{"image_id":136,"category_id":1,"keypoints":[36,181,2,20.25,191,0,35,166,2,20.25,191,0,8,171,2,20.25,191,0,2,246,2,20.25,191,0,20.25,191,0,20.25,191,0,20.25,191,0,20.25,191,0,20.25,191,0,20.25,191,0,20.25,191,0,20.25,191,0,20.25,191,0],"score":0.897}

from image basically this is the coco format for result. and i need the same array to generate for each image. could you please help me to generate these pose plotted result.

[KaiChen1998]

@niazahamd89 Still I think the pycocotool API will definitely help you. You can spend some time looking into its code and findout out how to transfer your result to the COCO format using their API. It should be a very easy job from my perspective and just several lines of code.
Hope that will help you.

[niazahamd89]

Thank you @KaiChen1998 i am trying to generate the pose results but if are concerned to do evaluation after training let me share how you are going to do this.

[scnuhealthy]

@niazahamd89，sorry for I am busy on other experiments. If I finish the evaluation code in the future, I will share on this repository.

[niazahamd89]

Hi @KaiChen1998 i need a little information regarding your repo, when ever i feed the image into the model it gives me the result always in 401x401 dimensions is it possible to get the same output dimensions in which the image was feed? for example i feed the image 640x425 and i get the output with pose in same dimension. i tried it by changing the IMAGE_SHAPE but it gives me error.

[KaiChen1998]

@niazahamd89 Most of the time, we all have different image resolution for different images right? However, the network prefers to take the same resolution all the time. So what you need to do is resize your samples to the same shape, which is [401, 401] in this case, and then resize your inference result back to your original image resolution.
Unfortunately, this part hasn't been done by this repo's owner, so......

[srivatsavagade]

Hi, @scnuhealthy did you finish the evolution part, It would be a great help in my work. Please upload the evaluation.py file
Thanks in advance.