STT LLM TTS Benchmarks for real-time translation

30+ Voice AI Benchmarks for Real-Time Translation: Latency, Pricing, Optimal Stack

Developers integrating AI into voice systems know: a 2-second delay kills conversational flow. Tested over 30 STT, LLM, and TTS engines. Result: a stack with ~870ms total latency on Apple M4 outperforms Google Meet in speed and flexibility.

Pipeline chain: STT recognizes speech → LLM translates → TTS synthesizes. Each component is critical for end-to-end latency. Below are benchmarks, protocols, and pitfalls.

STT: Leaders in Streaming and Accuracy

Speech-to-Text kicks off the pipeline. For real-time use, streaming with WER <10% and latency <300ms is essential.

| Provider | Latency | WER | Price/minute | Streaming |

|----------|---------|-----|--------------|-----------|

| Deepgram Nova-3 | <300ms | ~10% | $0.0059 | Yes |

| AssemblyAI Universal-2 | ~300ms | 8.4% | ~$0.006 | Yes |

| ElevenLabs Scribe v2 | 150ms | ~9% | ~$0.01 | Yes |

| Groq Whisper Large v3 | batch | 10.3% | $0.0028 | No |

Deepgram Nova-3 wins on balance: stable streaming, and a $200 signup bonus covers 560 hours of usage. Groq Whisper is unstable (503 errors, 2812ms average).

LLM for Translation: TTFT Matters More Than Tokens/s

Translating short phrases (5–15 words) hinges on Time to First Token. Quality between Llama 3.3 70B and Gemini Flash is close, but speed varies widely.

| Provider | Model | Tokens/s | TTFT |

|----------|-------|----------|------|

| Groq | Llama 3.3 70B | ~750 | ~200ms |

| Cerebras | Llama 8B | 1800 | ~350ms |

| Gemini | 2.5 Flash | 217–245 | 330–450ms |

| Fireworks AI | Llama 3.3 70B | ~800 | ~200ms |

Groq is optimal: 200ms TTFT. Local Llama 3.2 3B (~100 t/s on MLX) lags behind in latency.

TTS: The Bottleneck You Can’t Ignore

TTS breaks the pipeline if TTFB exceeds 500ms. Cloud APIs via WebSocket offer x5 advantage over sync HTTP. Price/hour based on 33,750 characters (two directions, 45 min TTS).

| Provider | Model | TTFB | ELO | Price/1M chars | Price/hour | Russian |

|----------|-------|------|-----|----------------|------------|---------|

| Cartesia | Sonic Turbo | ~40ms | 1054 | $37–47 | $1.26 | Yes |

| Hume | Octave 2 | <200ms | 1562 | $7.60 | $0.26 | Yes |

| ElevenLabs | Flash v2.5 | ~75ms | 1544 | ~$206 | $5.57 | Yes |

| OpenAI | TTS-1 | ~500ms | 1106 | $15 | $0.51 | Yes |

ELO Rating (TTS Arena v2, March 2026):

• Vocu V3.0 (1600)
• Inworld TTS-1.5-Max (1576)
• Hume Octave 2 (1562)
• ElevenLabs Flash v2.5 (1544)

Local models on M4 (MacBook Air 24GB, warm):

| Model | Size | 2–3 words | 10 words | Quality | Russian |

|-------|------|-----------|----------|---------|---------|

| Piper ryan-medium | 63MB | 30–50ms | 137ms | B | Yes |

| Kokoro 82M fp16 | 156MB | 370ms | 730ms | A+ | No |

| ZipVoice 123M | 123MB | ~500ms | 1240ms | B+ | No |

Trend: 0.5–2B parameter models (Chatterbox, Qwen3-TTS) generate phrases in 6–19 seconds without GPU.

Real End-to-End Benchmarks

5 phrases, warm, average TTFB (M4):

| Provider | Model | Protocol | Avg TTFB | Min | Max | Price/1M |

|----------|-------|----------|----------|-----|-----|----------|

| Cartesia | Sonic-2 | WebSocket | 245ms | 208ms | 281ms | $37–47 |

| Kokoro | 82M | local MLX | 313ms | 259ms | 340ms | free |

| ElevenLabs | Flash v2.5 | WebSocket | 395ms | 309ms | 551ms | ~$206 |

| Cartesia | Sonic-2 | sync SDK | 1361ms | 1173ms | 1567ms | $37–47 |

WebSocket vs sync: 245ms vs 1361ms (x5.5 difference).

Key Pitfalls and Discoveries

• Protocol makes the difference: Always use WebSocket for real-time. Sync APIs distort reality.
• Quantization on Apple Silicon slows things down: Kokoro fp16 (373ms) vs INT8 (687ms) — conversion overhead.
• Russian remains a challenge for open-source: Piper is outdated; large models are slow. Cloud = high cost.
• ElevenLabs unit economics: $5.57/hour vs Cartesia $1.26/hour with similar ELO.

Kokoro 82M (fp16, 4 threads): 373ms for 2 words, A+ quality, 28 English voices. No streaming — total time.

Final Stack and Latency

Deepgram Nova-3 (~300ms) → Groq Llama 3.3 70B (~200ms) → StreamChunker (2–3 words, ~100ms) → Kokoro 82M (~370ms). Total: ~870ms to first audio output.

What matters:

• WebSocket is mandatory for TTS under 300ms.
• Groq leads in TTFT for LLMs.
• Kokoro 82M is top open-source for English, but no Russian support.
• ElevenLabs is expensive: calculate $/hour upfront.
• Deepgram offers $200 bonus for 560 hours of STT.

— Editorial Team