Hugging Face Transformers adds native support for running GGUF (llama.cpp) quantized models, starting with Qwen3.5 on Apple Silicon

Hugging Face's transformers library (on its main development branch) now supports loading and running GGUF-format quantised model files, the format used by llama.cpp, directly through its standard from_pretrained/generate APIs. It works by reusing llama.cpp's own ggml kernels via a separate "kernels" package for Metal inference on Apple Silicon Macs. Initial support is limited to the Qwen3.5 architecture, and Hugging Face published benchmarks on an M2 Max showing performance close to llama.cpp's own llama-bench across three checkpoint sizes.

Developers who already use GGUF checkpoints (a large, widely downloaded ecosystem) can now run them through transformers' more feature-rich API instead of being restricted to llama.cpp-based tools like Ollama, LM Studio or Jan. This mainly benefits people building on transformers who want GGUF compatibility without switching toolchains, and it's a technical convergence signal, not a shift in who controls model distribution or compute. Practical impact today is narrow: one architecture (Qwen3.5), one platform (Apple Silicon), and an unreleased main-branch feature rather than a stable release.

The framing implies broad ecosystem support ("millions of downloads") when only a single model architecture on one hardware platform is actually supported at launch. Calling this a replacement for Ollama/LM Studio/Jan is premature since most GGUF users will keep using those tools until (or unless) transformers adds broader architecture and platform coverage. The benchmark comparison is against llama.cpp itself on a single machine (M2 Max), which is a reasonable first proof point but not evidence of general performance parity across hardware or model families.

1. 01Whether transformers extends GGUF support beyond Qwen3.5 to other popular architectures (Llama, Mistral, Gemma) within the next few months
2. 02Whether GGUF support lands on non-Apple platforms (Linux/CUDA, Windows) rather than staying Metal-only
3. 03Whether this ships in a stable transformers release (not just main branch) and whether Ollama/LM Studio users show any measurable shift toward transformers-based tooling as a result