Skip to content

fix(utils/weight_conversion): handle str target_modules/target_parameters in MoE config conversion#3236

Closed
Anai-Guo wants to merge 1 commit into
huggingface:mainfrom
Anai-Guo:fix/moe-config-target-modules-str
Closed

fix(utils/weight_conversion): handle str target_modules/target_parameters in MoE config conversion#3236
Anai-Guo wants to merge 1 commit into
huggingface:mainfrom
Anai-Guo:fix/moe-config-target-modules-str

Conversation

@Anai-Guo

@Anai-Guo Anai-Guo commented May 14, 2026

Copy link
Copy Markdown
Contributor

Summary

Closes #3229.

_convert_peft_config_moe in src/peft/utils/transformers_weight_conversion.py normalized peft_config.target_modules and peft_config.target_parameters via set(value or []) before iterating them against the MoE target-module mapping. When a user passes a single string for these fields (a supported shape elsewhere in PEFT, e.g. target_modules="gate_up_proj"), set("gate_up_proj") silently splits the string into individual characters ({"g","a","t","e","_","u","p","r","o","j"}). The downstream for target in peft_config.target_modules loop then matches none of the expected module names, so config conversion for MoE models silently produced wrong results. Reported in modelscope/ms-swift#9321.

What changed

  • src/peft/utils/transformers_weight_conversion.py (_convert_peft_config_moe): replace
    peft_config.target_parameters = set(peft_config.target_parameters or [])
    peft_config.target_modules = set(peft_config.target_modules or [])
    with a small local _normalize_to_set helper that maps str to a single-element set, None to set(), and otherwise falls back to set(value). Pure normalization change; no other code paths affected.

Verification

  • Behavior preserved for list/tuple/set/None: set([...])/set((...))/set({...}) semantics unchanged; None still becomes set() instead of erroring.
  • New behavior for str: target_modules="gate_up_proj" becomes {"gate_up_proj"} so the per-target loop sees one well-formed module name instead of ten one-character non-matches.
  • No new imports; helper is defined inside the function to keep the public surface unchanged.

AI-assisted, human reviewed.

@BenjaminBossan

Copy link
Copy Markdown
Member

Thanks for the PR but there is already another one (#3232).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] peft 0.19 target_modules (str) use set

2 participants