metrics

[wj7486]

How to test the indicators of multiple reference images mentioned in the paper? I would appreciate your help. Thank you very much and I look forward to your reply

[nenhang]

Thank you for your concern.
Our implementation is based on the evaluation framework from XVerseBench (code available here). Below are the specific files implementing each metric:

IDS (Identity Score): idip_face_score.py - Evaluates facial identity consistency against multiple reference faces.
IPS (Image-Patch Similarity): idip_sam-dino_score.py - Measures object alignment using SAM and DINO features.
AES (Aesthetic Score): idip_aes_score.py - Assesses the aesthetic quality of generated images.
ITC (Image-Text Consistency): dpg_score.py - Evaluates semantic alignment between generated images and text prompts.

We hope these will serve you well.

[wj7486]

Thank you for your concern. Our implementation is based on the evaluation framework from XVerseBench (code available here). Below are the specific files implementing each metric:

IDS (Identity Score): idip_face_score.py - Evaluates facial identity consistency against multiple reference faces. IPS (Image-Patch Similarity): idip_sam-dino_score.py - Measures object alignment using SAM and DINO features. AES (Aesthetic Score): idip_aes_score.py - Assesses the aesthetic quality of generated images. ITC (Image-Text Consistency): dpg_score.py - Evaluates semantic alignment between generated images and text prompts.

We hope these will serve you well.

In addition to the metric calculation for LAMICBench++, I would appreciate your guidance on the code related to metric calculation for LayoutSAM-Eval and COCO-MIG. Thank you again for your hard work, and I look forward to your reply.

[nenhang]

Well, the metric of COCO-MIG bench can be found at MIG_Bench, in this script. LayoutSAM-Eval bench's code can be found at CreatiLayout, in this script. We hope these will serve you well.

[wj7486]

Upon careful reading of your paper, I found that the test set for your LAMIC bench++ is not simply a mixture of XVerse and LAMIC, but appears to have created new tuples. Dear author, would it be convenient for you to make LAMIC bench++ publicly available? I look forward to your reply and appreciate your outstanding contribution.

[nenhang]

Upon careful reading of your paper, I found that the test set for your LAMIC bench++ is not simply a mixture of XVerse and LAMIC, but appears to have created new tuples. Dear author, would it be convenient for you to make LAMIC bench++ publicly available? I look forward to your reply and appreciate your outstanding contribution.

Thank you for the careful reading! You're correct — LAMICBench++ is not a simple mixture of XVerse and LAMIC; we constructed new samples with more instances. We have now released the benchmark package (lamicbench_plus_files.tar.gz) along with evaluation scripts in bench/lamicbench_plus/. The package includes benchmark metadata, reference images, and all evaluation code. Please refer to bench/lamicbench_plus/README.md for setup and usage instructions.