Prompting Guideline

[SagiPolaczek]

Hey there!
Nice work and thanks for sharing everything :)

Could you please provide additional information how did you train the LoRA and specifically how to construct the prompt for the model given the grid of 4x4 images? I used a prompt template from your paper (attached below), but the results are not as good as yours.

Thanks in advance!

The prompt I used:

This is a four-panel image grid: [TOP-LEFT]: The original source image. [TOP-RIGHT]: An edited version of the [TOP-LEFT] image, transformed to depict a running motion. [BOTTOM-LEFT]: A second original source image. [BOTTOM-RIGHT]: An edited version of the [BOTTOM-LEFT] image, applying the same running motion transformation as used in [TOP-RIGHT].

[Orannue]

Thank you for your interest in our work!
The training configuration follows the edit_transfer.yml file.
The prompt files we used for training are included in the dataset: https://github.com/CUC-MIPG/Edit-Transfer/blob/main/data/edit_transfer.zip.
All prompts are constructed following the template you provided. The only part we modified in the template for each image is the specific transformation description, such as "running motion."
In our experiments, we observe that the random seeds affect the results. You can try sampling different seeds during inference to observe the variation.
Hope this helps! Feel free to reach out if you have any further questions : )

[panxiying]

Very interesting work. What if I want to use other datasets for fine-tuning? How should I structure the data?

[Orannue]

Very interesting work. What if I want to use other datasets for fine-tuning? How should I structure the data?

Hi @panxiying, thank you for your interest!
To prepare your own dataset, the first step is to collect editing pairs (source, target). For each training sample, you’ll need to create a four-panel image by combining two editing pairs that result in similar changes. You can refer to the format shown here:
https://github.com/CUC-MIPG/Edit-Transfer/blob/main/data/edit_transfer.zip
We recommend including at least two samples per editing type, and covering more than 10 different editing types in total.