DeepMind AGI Taxonomy: 10 Cognitive Abilities

DeepMind Redefines AI Measurement: Cognitive Taxonomy Over Benchmarks

DeepMind has introduced not just a new test on the path to AGI—it has proposed a fundamental shift in the paradigm for evaluating artificial intelligence. Instead of a single score or an average result across MMLU, the team offers a ten-dimensional cognitive taxonomy based on empirically validated human cognitive abilities. This is not a tool for marketing claims but a technical protocol for middle and senior developers and researchers, enabling them to diagnose systemic weaknesses in models—even when their overall percentile looks impressive.

Why Current Benchmarks Don’t Work for AGI

Existing benchmark datasets (MMLU, HumanEval, GPQA) suffer from two critical flaws that make them unsuitable for measuring progress toward AGI:

• Contamination of Training Data—models repeatedly encounter answers to these tests even before inference. The result is a high score for memorization rather than understanding. The model solves “what,” but not “why” or “how to adapt to changed conditions.”
• Lack of Separation Between Model and System—when testing ChatGPT or Claude, we evaluate not only the language model but also external components: system prompts, code interpreters, web search, and chain-of-thought reasoning (CoT). Current benchmarks do not distinguish where the model ends and its environment begins.

This leads to false confidence: a system might score 89% on MATH but fail at a basic task like switching strategies in multi-step planning—because executive functions are not tested separately.

Ten Cognitive Abilities: From Perception to Social Cognition

The taxonomy divides general intelligence into ten mutually independent abilities. Eight are fundamental, while two are composite (cannot be reduced to the sum of basic abilities).

Fundamental Abilities:

• Perception—processing sensory signals. For LLMs, this isn’t vision but tokenization as an alternative channel for accessing language. The distinction is crucial: humans see an object, while models receive its description.
• Generation—producing outputs (text, code, actions) and internal reasoning. CoT here isn’t a hack but a manifestation of generative ability, separate from logical deduction.
• Attention—managing focus in dynamic contexts. It’s not the attention mechanism in transformers but a cognitive ability to balance concentration with sensitivity to new stimuli.
• Learning—sustained behavioral change through experience. In-context learning doesn’t count as learning in this sense: knowledge isn’t retained outside the session.
• Memory—four types: declarative (facts), episodic (events), procedural (skills), and prospective (intentions). Forgetting is included as a useful function—modern models can’t selectively delete outdated parameters.
• Reasoning—deduction, induction, abduction, analogy, and mathematical thinking. Automatic pattern matching is excluded: guessing based on formal question cues isn’t reasoning.
• Metacognition—monitoring one’s own confidence, detecting errors, and correcting strategies. The main gap between humans and AI: models don’t know what they don’t know.
• Executive Functions—goal setting, planning, impulse control, cognitive flexibility, and working memory. This is where agents most often break down when switching between subtasks.

Composite Abilities:

• Problem Solving—orchestrating perception, reasoning, planning, and learning. A failure in any basic ability cascades and undermines the solution.
• Social Cognition—modeling others’ beliefs and intentions, cooperation, negotiation, persuasion, and even deception. Deception is included not as an ethical threat but as a diagnostic marker of depth in social understanding.

Measurement Protocol: Three Stages and a Radar Chart

Classification without measurement is academic abstraction. DeepMind proposes a three-stage protocol:

• Isolated Testing of Each Ability—tasks are strictly one-dimensional and do not mix competencies. Task banks are closed and undergo independent auditing. Formats range from multiple-choice to multimodal scenarios with timelines.
• Control Sample of Humans—the same set of tasks, the same instructions, and the same access to tools. No advantages for AI: if the model uses search, humans also get a web interface.
• Percentile Comparison—for each of the ten axes, it calculates what percentage of humans the model outperforms. The result is a radar chart (jagged profile), not a single score.

This allows identifying critical imbalances: for example, a 95th percentile in reasoning and a 20th in metacognition. Such a model is dangerous in high-stakes scenarios—it will consistently err.

What Matters

• The taxonomy doesn’t replace benchmarks—it requires them to be restructured: each test must be clearly tied to a single ability, without “cross-contamination.”
• A jagged profile isn’t a flaw but an objective characteristic. AGI should demonstrate balance across all ten axes, not peak values in individual areas.
• The lack of standard tools for learning, metacognition, attention, executive functions, and social cognition isn’t a technical delay but a methodological gap in the industry.
• Measurement must be relative: not “how many tasks were solved,” but “is the model better than X% of people under identical conditions?”
• Successful implementation of the protocol requires revisiting architectures: for example, long-term memory and forgetting mechanisms should become first-class components, not external plugins.

Five ‘Gaps’ in Modern AI Diagnostics

DeepMind clearly identifies areas where reliable measurement tools are lacking:

• Learning—how do you distinguish acquiring new knowledge from reproducing a learned pattern? You need a transfer test: learning in one domain → applying it in another, without re-fine-tuning.
• Metacognition—there are studies on calibrating confidence, but they’re not standardized or scalable. We need tasks where the model explicitly assesses its probability of being correct before answering.
• Attention—current tests (e.g., “needle in a haystack”) are too simplistic. We need assessments of resilience to dynamic distractions in long dialogues with shifting goals.
• Executive Functions—there are no benchmarks for cognitive flexibility. For example, a system should switch from algorithmic solving to heuristic approaches when constraints change in real time.
• Social Cognition—existing theory-of-mind (ToM) tests don’t differentiate advanced models. We need scenarios involving multi-user negotiations, hidden intentions, and conflicting signals.

DeepMind’s protocol doesn’t make models smarter. It makes them measurable. For developers, this means moving from “we have 87% on MMLU” to “we’re at the 62nd percentile in metacognition—needs refactoring of the self-control module.” This isn’t advertising; it’s a technical specification.

— Editorial Team