GLM-5.1: AI Capable of Long-Term Code Optimization Without Plateaus
GLM-5.1 from Z.ai leads on SWE-Bench Pro with 58.4 points, surpassing GPT-5.4 (57.7), Claude Opus 4.6 (57.3), and Gemini 3.1 Pro (54.2). The model is available on Hugging Face under the MIT license. Key breakthrough — resilience to long sessions: it breaks down tasks, experiments, analyzes logs, and adjusts strategies over hundreds of iterations.
Long-Term Optimization Instead of Quick Plateaus
Previous models, including GLM-5, quickly exhaust their tools: they apply standard approaches, reach a local maximum, and stop. GLM-5.1 is designed for multi-hour cycles. Wrapped with feedback, it:
- Breaks the task into subgoals.
- Runs experiments with tools.
- Reads metrics and logs.
- Identifies bottlenecks.
- Rebuilds the strategy.
This allows overcoming plateaus through thousands of tool calls.
Linux Desktop in the Browser: 8 Hours of Autonomous Assembly
Task: implement a web app simulating a Linux desktop — without source code or mockups. Typical models create a skeleton with a taskbar and stubs, then give up.
GLM-5.1 in a review cycle worked for 8 hours:
- Built a file manager with navigation and previews.
- Added a terminal with bash emulation.
- Integrated a text editor with syntax highlighting.
- Implemented a system monitor (CPU, RAM, network).
- Included a calculator and simple games.
All components in a unified UI style. The model itself decided what to refine after each pass.
Record on VectorDBBench: 6x Speedup Over 600+ Iterations
Benchmark: optimizing approximate nearest neighbors (ANN) search on a Rust skeleton. Previous SOTA from Claude Opus 4.6 — 3547 QPS in 50 iterations.
GLM-5.1 in an extended cycle (600+ submissions, 6000+ calls):
- Achieved 21.5k QPS — 6x better.
- Went through 6 structural rebuilds:
- Brute force → IVF index.
- Vector compression to f16.
- Two-stage search: u8 scoring + f16 ranking.
- Hierarchical routing.
Progress graph — a staircase: spot tunings alternate with radical approach changes based on log analysis.
KernelBench Level 3: GPU Optimization
Task: tuning GPU kernels. GLM-5.1 sped it up 3.6x over 1200 iterations. Claude Opus 4.6 leads (4.2×), but GLM shows potential in long sessions.
Availability and Limitations
The model integrates with Claude Code, OpenCode, Cline, OpenClaw. Available to GLM Coding Plan subscribers. Downsides:
- At peak, uses quota 3x faster (until end of April, off-peak — at base rate).
- Struggles with coherence over thousands of calls.
- Self-assessment without metrics is weak.
Z.ai plans optimizations for long-running tasks.
Key Takeaways
- GLM-5.1 beats SOTA on SWE-Bench Pro (58.4%) thanks to long-term thinking.
- On VectorDBBench, achieved 21.5k QPS — 6x the record.
- Assembled a full Linux desktop in the browser in 8 hours without hints.
- Demonstrates staircase progress: 6 strategy rebuilds.
- Needs coherence improvements for extreme sessions.
— Editorial Team
No comments yet.