Cannot Use Real DRCT GAN Weights for Inference — "Missing keys in state_dict" Error

[DanilaAniva]

Issue:

I'm trying to run inference with the Real DRCT GAN weights, but I'm facing an issue when loading the model with the following command:

bash
python inference.py --input "input_dir" --output "output_dir" --model_path experiments/pretrained_models/Real_DRCT_GAN_SRx4/Real_DRCT_GAN_SRx4_finetuned_from_mse_net_g_latest

However, I receive this error:

RuntimeError: Error(s) in loading state_dict for DRCT:
        Missing key(s) in state_dict: "layers.6.swin1.norm1.weight"

Problem description

It looks like the GAN weights (Real_DRCT_GAN_SRx4_finetuned_from_mse_net_g_latest) are not compatible with the DRCT model architecture used in inference.py. The state_dict in the GAN checkpoint seems to have different keys and/or structure, resulting in missing keys when trying to load them into the DRCT model.

Questions

• What is the correct way to perform inference using the "Real DRCT" GAN weights?
• Should I be using a different architecture or script (other than inference.py) for these specific weights?
• Do you have an example code snippet or pipeline for inference with the Real DRCT GAN models (.pth, not ONNX)?
• I saw ONNX inference script for this model. Where can i download ONNX weights ?

Thank you for your support!

[khanhdc]

Do you have any updates on this? I also encountered the same error when trying to inference my custom-trained model

[limitlesslab]

guys, just download neosr or trainner redux (training tools) and run inference in them using Real DRCT GAN and many other models and architectures, and you can also use Chainner to load and use this model (Real DRCT GAN) for inference on your images AND to convert it to the ONNX format

[somnathsingh-fynd]

If you are trying to load net_g checkpoints "Real_DRCT_GAN_SRx4_mse_net_g_latest" and "Real_DRCT_GAN_SRx4_finetuned_from_mse_net_g_latest" -> In infererence.py change depths= [6, 6, 6, 6, 6, 6] and num_heads= [6, 6, 6, 6, 6, 6]. This should work. These checkpoint are for 6 RDG blocks.