Articles by tag: swe-bench

GLM-5.1 leads on SWE-Bench Pro: 58.4%

GLM-5.1 from Z.ai outperforms GPT-5.4 and Claude on SWE-Bench Pro. Assembled Linux desktop in 8 hours, x6 on VectorDBBench. Weights on HuggingFace. Test for long-running tasks.

Qwen3.6-Plus: agentic programming and multimodality

Qwen3.6-Plus from Alibaba outperforms competitors in coding on SWE-bench, supports 1M tokens and multimodality. API access, benchmarks, integrations — for developers. Explore details.

Degradation Claude Code: reasoning depth -73%

Analysis of 6852 Claude Code sessions: drop in thinking depth by 73%, cost increase ×122, blind edits 33%. Workaround and metrics for middle/senior dev. Check on your own tasks.

14 LLM Benchmarks: Analysis and Critique

Detailed analysis of SWE-bench, Terminal-Bench, τ² and other tests for LLM. Learn weaknesses and relevance for dev tasks. Choose a model consciously.