Contents

Open ASR Leaderboard has added private tests for Egyptian- and Nigerian-accented English, featuring a total of 2.89 hours of natural conversation. These provide more relevant evidence when choosing speech recognition models for users whose accents are underrepresented in standard evaluations.

Key Points#

  • Two private datasets from Liva AI add 1.34 hours of Egyptian English and 1.55 hours of Nigerian English.
  • Both contain natural conversational speech, with transcripts preserving punctuation, capitalization, hesitations, and other features.
  • The Private tab now supports comparisons of US and non-US results and displays accent groupings based on Kachru’s “Three Circles” model.
  • These tests broaden evaluation coverage, but teams still need to verify performance with their own users and recording conditions.

What the New Tests Cover#

Announced on November 6, 2026, this update adds two datasets provided by Liva AI.

| Dataset | Accent | Duration | Speaking Style | |---|---|---:|---| | Liva AI Conversational EG | Egyptian English | 1.34 hours | Natural conversation | | Liva AI Conversational NG | Nigerian English | 1.55 hours | Natural conversation |

These tests cover accented English, not Arabic or other languages spoken in Nigeria. This distinction matters when deciding whether the results apply to your product.

The reference transcripts include punctuation, capitalization, hesitations, and other features. Check the transcription and scoring rules before comparing scores with other benchmarks. Differences in transcription conventions and normalization can affect comparisons.

Why Accent-Specific Private Tests Matter#

A high overall speech recognition score does not necessarily mean consistent performance across all accents. For services used by Egyptian or Nigerian English speakers, results for those speaker groups are more useful than an overall score alone.

Private evaluation data also helps reduce opportunities to optimize directly for public benchmark examples. These additions aim to improve both representativeness and reliability by introducing fresh, unused evaluation data and another data provider.

These evaluation categories therefore provide useful evidence for model selection. They do not, however, guarantee production accuracy. Their value lies in revealing performance differences that broad averages can obscure.

How to Read the Breakdowns#

The leaderboard’s Private tab displays two types of accent comparisons. The first compares US and non-US English, with the latter covering the UK, Australia, Canada, India, Nigeria, and Egypt.

The second follows Kachru’s “Three Circles” model.

  • Inner Circle: US, UK, Australia, and Canada.
  • Combined Outer and Expanding Circles: India, Nigeria, and Egypt.

According to the announcement, the Outer and Expanding Circles will be displayed separately as more accents are added. For now, treat the combined category as a broad overview. If your product targets a specific speaker group, check individual accent results whenever possible. Grouped scores can hide differences among the speaker groups they include.

What to Check Before Choosing a Model#

The new datasets total less than three hours. They broaden coverage, but this duration alone cannot establish performance across every speaker group, topic, device, and acoustic environment in both markets.

The supplied announcement does not include model scores or name the best-performing model, and it provides insufficient detail to assess speaker diversity. Claims that a particular system is best for Egyptian or Nigerian English need to be checked against the actual leaderboard results and evaluation documentation.

Use the new results to narrow your shortlist, then verify that the remaining candidates meet your product requirements.

  • Check Egyptian and Nigerian English results separately instead of relying solely on overall rankings.
  • Read the scoring and normalization rules before comparing results across benchmarks.
  • Test speech representative of your intended users, including the recording conditions they will encounter.
  • Compare recognition quality alongside the latency and cost your application can tolerate.

Next Steps#

Open the Private tab on Open ASR Leaderboard, compare your current speech recognition model with one alternative in the Egyptian and Nigerian English evaluation categories, and record each result separately.

Source: https://huggingface.co/blog/open-asr-leaderboard-liva

Contents