/ Benchmark · 10 Nov 2026
Pass rate on the lead tasks: initial 29 Oct run, rule + Clef, and Claude Opus 5.5.
| Model | Initial, 29 Oct | Rule + Clef | Claude Opus 5.5 |
|---|---|---|---|
GLM 5.3 Z.ai | 53% (9 of 17) | 70% (21 of 30) | 93% (28 of 30) |
GLM 5.3 Flash Z.ai | 35% (6 of 17) | 47% (14 of 30) | 93% (28 of 30) |
Qwen 3.6 Alibaba Qwen | 56% (10 of 18) | 53% (16 of 30) | 93% (28 of 30) |
Gemma 4 Google | 88% (14 of 16) | 72% (18 of 25) | 93% (28 of 30) |
DeepSeek DeepSeek | 83% (15 of 18) | 100% (28 of 28, 14 tasks) | 93% (28 of 30) |
Six lead tasks per model; DeepSeek rule + Clef is the 14-task run (28 of 28). Attempts lost to Helmcode errors (HTTP 400/429) are excluded; Opus 5.5 ran on Claude Code, 25 Oct. Small samples.