load_file_from_url() downloads the model from HF_ENDPOINT if it is set

[licyk]

Description

When HF_ENDPOINT is set, load_file_from_url() downloads models from it. Extensions like sd-webui-controlnet that rely on this function will now also use HF_ENDPOINT.

Screenshots/videos:

set HF_ENDPOINT=https://hf-mirror.com

Checklist:

• I have read contributing wiki page
• I have performed a self-review of my own code
• My code follows the style guidelines
• My code passes tests

[a-holm]

Thanks for this contribution! Adding support for HF_ENDPOINT in load_file_from_url is a useful feature, especially for users behind mirrors or in restricted environments.

The core change in modules/util.py correctly implements the intended functionality by replacing the Hugging Face domain based on shared.hf_endpoint. The refinement in modules/shared.py also looks good.

However, the changes included in the diff for modules/dat_model.py, modules/models/sd3/sd3_cond.py, modules/sd_disable_initialization.py, and modules/sd_models.py seem counter-productive to the goal of this PR. They appear to remove dynamic usage of shared.hf_endpoint and replace it with hardcoded https://huggingface.co URLs. This contradicts the intent of allowing an override via HF_ENDPOINT.

Could you please revise the PR to only include the necessary changes in modules/util.py and modules/shared.py? The other modifications seem unrelated or incorrect for this feature.

[licyk]

@a-holm In the four files including modules/dat_model.py and modules/models/sd3/sd3_cond.py, all URLs are downloaded via load_file_from_url(). Therefore, removing the hardcoded shared.hf_endpoint in these files is the correct optimization, it eliminates redundant replacement logic while ensuring that HF_ENDPOINT remains effective after the modification.

[a-holm]

@licyk alright, I just misunderstood the code then.

[Qodo-Free-For-OSS]

Hi, DisableInitialization.transformers_utils_hub_get_file_from_cache only suppresses the known-404 added_tokens.json request when the URL exactly matches the hardcoded huggingface.co form, so it will no longer short-circuit when a non-default HF endpoint is used. This reintroduces the unnecessary 404 request the guard is explicitly trying to prevent.

Severity: action required | Category: reliability

How to fix: Match against configured endpoint

Agent prompt to fix - you can give this to your LLM of choice:

Issue description

modules/sd_disable_initialization.py contains a guard intended to suppress a known-404 request for added_tokens.json. The guard currently compares url to a hardcoded https://huggingface.co/... string, which breaks when the app is configured to use a different HuggingFace base (via HF_ENDPOINT).

Issue Context

The project supports overriding the HuggingFace base URL via shared.hf_endpoint. Any URLs produced/used under that configuration will not match the hardcoded https://huggingface.co/... and the guard won’t trigger.

Fix Focus Areas

• modules/sd_disable_initialization.py[76-86]
• modules/shared.py[94-94]

Implementation notes

Update the guard to also match the configured endpoint (e.g., compare after normalizing the URL’s base, or check for the path suffix /openai/clip-vit-large-patch14/resolve/main/added_tokens.json regardless of domain). Ensure operator precedence is explicit with parentheses.

We noticed a couple of other issues in this PR as well - happy to share if helpful.

---

Found by Qodo code review