LLM Benchmark: GPT-5.4 vs Claude Opus 4.6

# LLM Persuasion Benchmark: GPT-5.4 Leads Over Claude Opus 4.6 in 6300 Dialogues

Models GPT-5.4 (high reasoning) and Claude Opus 4.6 topped the LLM Persuasion Benchmark—a test of the ability to persuade opponents in debates. Researcher Lech Mazur tested 15 LLMs in 6300 multi-round dialogues across 15 contentious topics, from banning cars in city centers to genetic screening of embryos. Each pair of models swapped roles, assessing position shift on a scale from −3 to +3.

Benchmark Mechanics

The benchmark simulates real debates: one model defends a thesis, the other resists. The dialogue lasts 8 turns. The target's position is measured before and after the debate with three hidden questions for accuracy. Each pair runs scenarios both ways—PRO (defense) and CON (refutation). Topics cover ethics, urban planning, and biotechnology.

Total volume: 15 topics × model pairs × 2 directions = 6300 dialogues. This reveals not only eloquence but actual persuasiveness—the ability to change opinions without manipulation.

Persuasiveness Ranking

Leader—GPT-5.4 (high reasoning) with delta 1.71. Followed by Claude Opus 4.6 (1.67), ByteDance Seed2.0 Pro (1.64), Claude Sonnet 4.6 (1.58). Laggards:

• Mistral Large 3: 0.42
• Xiaomi MiMo V2 Pro: 0.52

Top models are more effective in the CON role: tearing down arguments is easier than building them. Average position shift in PRO mode is lower than in CON.

| Model | Persuasiveness (delta) |

|--------|-------------------------|

| GPT-5.4 (high reasoning) | 1.71 |

| Claude Opus 4.6 | 1.67 |

| ByteDance Seed2.0 Pro | 1.64 |

| Claude Sonnet 4.6 | 1.58 |

| Mistral Large 3 | 0.42 |

Resistance to Persuasion

The benchmark also measures susceptibility—how easily a model changes its position. Grok 4.20 Beta is nearly invulnerable (0.015), Claude Opus 4.6 holds at 0.41, Kimi K2.5—0.37. The easiest to shift:

• Xiaomi MiMo V2 Pro (2.0)
• Gemini 3.1 Pro Preview (1.81)
• DeepSeek V3.2 (1.74)

Average susceptibility across the benchmark >1, highlighting the gap between leaders and the rest.

Implications for Developers

Traditional benchmarks (MMLU, HellaSwag) test knowledge but ignore dialogue. LLM Persuasion Benchmark focuses on pragmatics: methodical argumentation, adaptation to counterarguments. A model can generate fluent text but fail to shift positions—or sound monotonous but persuade.

For mid/senior developers, it's a tuning tool:

• Fine-tuning on debates: use logs for RLHF to boost persuasiveness without toxicity.
• Prompt engineering: test chain-of-thought in debates—high reasoning versions lead for a reason.
• Hybrid systems: combine persuasive models (GPT-5.4) with resilient ones (Grok) for chatbots.

Data shows correlation with reasoning quality: top-3 have explicit reasoning modes.

Key Takeaways

• GPT-5.4 leads with 1.71 delta, edging out Claude Opus 4.6 by 0.04—a minimal but statistically significant gap.
• CON arguments outperform PRO: average shift 15–20% higher in refutations.
• Grok 4.20 Beta is the resilience benchmark (0.015), ideal for roles needing position fixation.
• The benchmark exposes weaknesses: eloquence ≠ persuasion, requires multi-round tests.
• 6300 dialogues provide robust statistics, minimizing noise from random responses.

— Editorial Team