Add minimal reference model fetch and metadata caching - #2
Open
TylerZeroMaster wants to merge 10 commits into
Open
Add minimal reference model fetch and metadata caching#2TylerZeroMaster wants to merge 10 commits into
TylerZeroMaster wants to merge 10 commits into
Conversation
Explain new fetch-reference command and how it works
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds a way to download only the minimum required model metadata (safetensors headers, configs, and tokenizers) to get tensor shapes, bypassing the need to download gigabytes of full model weights. It integrates seamlessly with existing conversion scripts.
Why this change?
I used this tool for my first LoRA adapter project because transformers did not want to open my GGUF. I did not want to download full model weights just to get tensor shapes. This adds a way to download the minimum metadata required and save it to the reference model directory.
Changes
fetch-referenceCLI command: Fetches only safetensors header bytes (a few KB) plus config/tokenizer files, caching shapes inungguf_model_meta.json.load_model_metadata()incommon.py: Converters now use cached metadata automatically, with an automatic rescan if local safetensors files are newer.ignore-missing-mtp/ignore-missing-visualflags: Specifically for Qwen3.5: handle GGUFs that omit MTP/vision tensors. The Qwen3.5 verifier remains intact; these flags act as explicit opt-ins for incomplete outputs.The generated cache file also serves as a human-readable snapshot of tensor shapes/dtypes, which I found useful for debugging conversion issues.
Aside
I am still not sure why transformers did not want to open the Qwen3.5 model i downloaded. I suspect the model was converted with the --no-mtp option as it was missing MTP heads. Maybe that, or missing visual keys, is why transformers did not want to open it.