VOICE-H: a human evaluation benchmark for text-to-speech, with an interactive leaderboard, head-to-head records, price versus performance, rater demographics, and the preference distribution
VOICE-H: a human benchmark for text-to-speech
Real people judged AI voices against the original human recording, on real conversation, and gave the reason behind every preference.
The VOICE-H leaderboard
Evaluating voice quality is challenging: judgments are inherently subjective, and aggregate preference scores rarely reveal what actually drives them. VOICE-H is grounded in real human speech at both ends, supplying both the prompt and the baseline sample that every model is judged against.
We selected quotes and audio clips from an Askable Labs dataset of over 40,000 real user interviews, spanning topics that naturally elicited emotional expression. Participants compared TTS-generated versions of each quote, rated them for naturalness, accuracy, and emotion/tone, and explained the reasoning behind every rating.
Capturing the qualitative nuances behind preference decisions is what turns benchmark results into actionable insights. VOICE-H goes beyond measuring which voices people prefer, to identify the specific qualities that make speech sound natural, engaging, and human.
The benchmark runs in three languages, and the filter above switches every figure on this page between the pooled result and a single study.
Head to head
Each cell is how often the row's voice was preferred over the column's, across all comparisons. Amber cells win, indigo cells lose.
Price vs performance
Elo against estimated cost per prompt, using recorded list-price assumptions. This chart does not measure latency or billed usage.
What makes a voice feel human
No clear English leader
Eleven v4 has the highest English point estimate, but listener- and prompt-clustered checks leave its leads over Gemini 3.8 Flash, Sonic 3.6 and Gemini 3.1 Flash unresolved.
A clear French lead
Eleven v4 leads the French fit, including a 97.1-point gap over Eleven v3 that remains positive in both clustered checks.
Language changes the result
Qwen is fifth among TTS systems in English, seventh in Spanish and twelfth in French. Its French comments flag voice character and pronunciation.
Timing still matters
Listeners praise well-placed hesitation and criticise unexpected pauses. The research post pairs those observations with the original comparisons.
Specific data opportunities
Contextual emotion, conversational rhythm and language-specific pronunciation are testable development priorities. The benchmark identifies hypotheses, not training-data causes.
Findings above describe the fixed October 2026 release across all languages. Read the evidence and uncertainty checks.
In their own words
Comments from comparisons involving the human recording, including wins, losses and trade-offs. Follows the study filter; non-English comments are translated with the original underneath.
Who judged
Identity-verified listeners. The charts count study sessions; some listeners returned in later rounds.
How strongly raters preferred
Distribution of the five preference labels. Sides were randomised per comparison, and rater-quality filtering removes a small number of votes from each study before these counts.
Methodology
100 English prompts were curated from over 40,000 real Askable user interviews: 10 seconds to three minutes of contiguous, emotional speech, manually reviewed for transcription accuracy. French and Spanish each use 50 prompts and human baseline recordings from separate native datasets in each language, not from the original English interviews. English covers 18 systems including the human recording; French and Spanish each cover 15 across the pooled rounds. Round three adds only pairs involving the new systems: 48 per prompt in English and 36 in French and Spanish. Eleven v2 was not fielded in French or Spanish round three, so its pairs with the three new systems are untested there and marked in the matrix; every other pair has votes. Each session included 12, 15, 16 or 20 randomised comparisons for one prompt, rating each sample for naturalness, accuracy, and emotion/tone, then giving an overall preference on a five-point scale with free-text reasoning. Overall Elo uses ordinal Bradley–Terry on all five preference levels. Dimension Elo uses capped rating-difference weights. Chart intervals use 500 comparison-level bootstrap draws; they do not account for shared listeners or prompts. Additional clustered checks appear in the research post. Side imbalance is uncorrected, and pooling assumes stable comparisons across rounds; round three cannot independently test temporal drift. The All languages view is a single fit over the observed vote mix: 15,217 English, 5,510 French and 5,524 Spanish comparisons. It is English-heavy and some models have English-only coverage. The full methodology, findings, and rater quotes are in the research post.
What's next
VOICE-H is a living benchmark. English, French and Spanish are live, and German is next. Systems join as they ship, and every result lives on one leaderboard rather than a scoreboard per release.
Want your voice model in the next run?
VOICE-H is built on real human conversation and judged by the Askable Labs panel. If you build voice AI, put it in front of real people, and see not just how it ranks but why.