GGUF reader: llama.cpp Mixtral loads as dense Llama with random MLPs; Qwen3 head_dim and Mistral detection not mapped #50197

System Info

transformers 5.20.2, gguf 0.19.0, torch 2.14.1+cpu. Code checked against main on 31 Oct 2026 (integrations/ggml.py, modeling_gguf_pytorch_utils.py).

Problems (GGUF reader config mapping)

1. A llama.cpp Mixtral GGUF silently loads as a dense Llama with randomly initialised MLPs.
llama.cpp writes Mixtral as general.architecture = "llama" with llama.expert_count, llama.expert_used_count and merged blk.N.ffn_{gate,up,down}_exps / ffn_gate_inp tensors. GGUF_CONFIG_MAPPING["llama"] maps no expert keys and llama uses LlamaTensorProcessor, so the file becomes LlamaConfig; the expert and router tensors have no destination and every layer's mlp.{gate,up,down}_proj is newly initialised, with only the usual "newly initialized" warning. On a small llama-with-experts GGUF the result was 0.82 (|a - b| / max(1, max|ref|)) off a reference evaluation of the file's graph; the same tensors loaded as MixtralForCausalLM matched to 1.6e-7.
Expected: route llama + expert_count > 0 to Mixtral (config and stacked expert tensors), or refuse the file instead of initialising the MLPs at random.

2. Qwen3: attention.key_length is not mapped to head_dim.
GGUF_CONFIG_MAPPING["qwen3"] has no attention.key_length, so head_dim takes Qwen3Config's default 128. Official Qwen3 checkpoints happen to use 128; a Qwen3-architecture file with another head size loads without complaint and fails at the first forward (shape '[1, 8, -1, 128]' is invalid).

3. Mistral detection depends on general.name.
if "llama" in architecture and "mistral" in model_name tests membership in the list read_field returns, so it matches only a general.name of exactly "mistral". A converted Mistral (e.g. general.name = "Mistral-7B-v0.1") loads as LlamaForCausalLM without its sliding window; a file named exactly "mistral" then fails in gguf-py 0.19.0 (Unknown gguf model_type: mistral).