Why AI Progress is Only Noticeable in Code and Math: Breaking Down Karpathy's Theses
Andrey Karpathy, co-founder of OpenAI and former AI director at Tesla, explains the gap in assessments of modern models' capabilities. Progress is concentrated in programming, math, and research, where AI achieves peak results. In everyday tasks like searching for information or generating text, improvements are minimal.
Two Groups with Different Experiences
Users fall into two camps: those relying on last year's free ChatGPT, and professionals using advanced tools like Codex or Claude Code.
- The first group judges by outdated models: Advanced Voice Mode gets tripped up in simple scenarios, like choosing between walking or driving to a car wash 50 meters away.
- The second group witnesses 'AI frenzy': models with terminal access can restructure an entire codebase or uncover vulnerabilities in an hour—tasks that take humans days.
This divide leads to misunderstandings: viral fails from old models don't reflect the potential of agentic systems.
Technical Reasons for Code and Math Dominance
Programming and math are perfect for RLHF (Reinforcement Learning from Human Feedback) and RL on verifiable metrics.
- Clear rewards: A unit test either passes or fails—a binary signal for optimization, unlike the subjectivity of text quality.
- Economic focus: B2B tasks drive the bulk of revenue, so AI labs pour resources into code generation and computations.
- Scalability: Agentic systems with tools (terminal, API) tackle complex tasks autonomously.
As a result, top models like Codex can overhaul projects in hours, while voice interfaces struggle with unstructured queries.
Parallel Realities of AI
Karpathy stresses: both views are correct. Voice ChatGPT flounders with memes, but specialized agents rule professional workflows. This isn't a contradiction—it's progress distributed across domains.
Professionals highlight:
- Restructuring legacy code without losing functionality.
- Automated security audits, including zero-day exploits.
- Solving math problems at PhD-level research.
For mid- and senior-level developers, this means integrating AI into CI/CD pipelines and code reviews, where verifiability accelerates iterations.
What Matters
- AI progress is asymmetric: peaks in code and math stem from RL on binary metrics.
- B2B priorities dictate the focus—everyday tasks lag behind.
- Agentic systems with terminal access solve problems orders of magnitude faster than humans.
- Perception gaps arise from differences in the models people use.
- For IT pros, the key is adopting tools like Codex for production workloads.
— Editorial Team
No comments yet.