The model you downloaded may be broken, weakened, or steered — and your checks will not catch it

Broken on arrival, with the file intact

In a preprint, researchers audited 305 eligible code-capable GGUF tags across 15 model lines in the Ollama library plus 22 popular Hugging Face community files, and found that seven official tags, 2.3%, solved at most six of 164 HumanEval+ tasks. All of the non-functional tags were low-bit quantizations of models with at most four billion parameters. Rebuilt suspect files matched the original registry files tensor for tensor, ruling out file corruption as the cause: the failure came from uncalibrated quantization recipes, not damaged downloads. m:…7419

The fix the researchers demonstrated is instructive. Adding an importance matrix raised three Qwen2.5-Coder-3B builds from zero to 76 to 113 solved tasks. They released their audit data on the Hugging Face Hub and a screening tool called quantcheck. Their broader finding is institutional: none of the registries they examined documented functional testing of their files. A checksum proves the file you downloaded is the file the registry published. It says nothing about whether the model inside it works. m:…7419

Weakening can strengthen

In a second preprint, researchers report a result that complicates any simple story about quantization degrading models. Their framework, Weakening-Induced On-Policy Self-Distillation, contrasts a reasoning model with its own weaker quantized or pruned version and uses the contrast to generate training signals without privileged information or external teachers. Across tested benchmarks, quantization improved average accuracy by 4.0 percentage points and pruning by 5.5 percentage points over the original models. m:…9551

The individual results are striking: pruning raised Qwen3.5-4B's accuracy from 57.7% to 70.5% on HMMT 2025 and from 39.1% to 56.7% on LiveCodeBench v6, while quantization raised Qwen3.6-35B-A3B from 68.5% to 74.6% on HMMT 2025 and from 61.8% to 69.5% on LiveCodeBench v6. The authors describe the result as state-of-the-art performance without privileged information. Read alongside the broken-artifacts audit, the lesson is that quantization is not a single predictable dial: the same family of techniques can silently destroy a small model, or measurably improve a reasoning model, depending on the recipe. That unpredictability is exactly why a downloaded artifact needs functional testing rather than a version number. m:…9551

Steered, not broken

The third finding is the most consequential for deployment. Arize describes activation steering, in which pinning specific neurons manipulates a model's behavior without altering its weights. The company says steering can turn Qwen3.5-4B hostile by pinning two specific neurons, and that at half strength the steered model retains high capability while bypassing safety refusals, making it difficult to detect. It cites Apple research showing that suppressing a single refusal neuron bypasses safety on about 90% of benchmark prompts, and notes that refusal neurons are present in base models before safety training, suggesting they emerge during pretraining. m:…7056

Arize's argument is that malicious steering can enter through model weights, sidecar files, or inference code, so SBOMs and model signing are insufficient: they verify provenance of components, not what the model actually does. The company notes that Qwen3.5-4B was downloaded more than 7 million times on Hugging Face last month, and that as of October 2026, 8,743 models on Hugging Face have 'abliterated' in their name, a marker of safety removal. Its proposed remedy is continuous behavioral evaluation in production, and it says decision models used as output guardrails blocked all 11 replies where a steered model completed a request it should have refused. Whether guardrails generalize beyond that test is unproven, but the direction is clear: the check that matters is behavioral, not documentary. m:…7056

What to do with a download

The three findings converge on a practical rule. Before a downloaded model touches production, run it: a functional screen like quantcheck catches the broken artifacts, a capability check alone will not catch a half-strength steered model, and a version label will not tell you whether quantization helped or harmed this particular build. The registry, the checksum, and the bill of materials are all necessary and none of them is sufficient. What is sufficient is evidence from running the artifact itself, in the configuration you will deploy, against tasks that include the refusals you need it to keep. m:…7419 m:…7056