Analytical Services

Multilevel Data Analysis

When people listen to someone speak, their ability to understand even simple sentences depends on his or her pronunciation. However, intelligibility, the ability of people to understand a speaker’s sentences, also depends on their hearing and their own dialects. We needed to determine how to measure intelligibility in a way that was fast and reliable: an objective screening test for the intelligibility of non-native English speakers when talking to Americans.

The pronunciation patterns, or phonologies, of speakers depend on the standard phonologies of languages and dialects they know best. That is what distinguishes one accent from another. But what aspects of a phonology matter to being understood? This is a matter of great importance in occupations such as teaching, and of life-and-death importance in others. Which of these aspects can be reliably and automatically measured? A startup company wanted to know.  

For this customer we began with a data set of recordings of words and sentences spoken by native and non-native speakers. Then we measured many American listeners’ ability to understand the same speakers in conversation.

First, we interviewed a domain expert: a phonetician with long experience interviewing accented speakers and assessing their general intelligibility. The we isolated individual features, such as how the “s” in “vision” is pronounced, --features that the phonetician judged to be important, that correlated with being understood, and that we assessed could be feasibly measured by computer. Next, we designed and implemented algorithms to perform the automatic measurements of those features. For every speaker, we scored each word or sentence for its features, combining the scores to produce an overall measure of the speaker’s predicted ability to be understood.  

The original set of listener measurements was rather small. But by combining the scores of different features with fuzzy logic, we were able to control the influence of any one listener, sentence, or feature, to produce a robust estimator from limited data.

The result is an expert system that is competitive with highly trained human judges, but one that can be run quickly and inexpensively from any PC.


As the plot shows, the system's prediction matches the listener evaluations at 92% correlation, over a range of intelligibilty from virtually perfect to nearly incomprehensible.

For more information on this project or any other services available from STAR Analytical Services, please contact us by email at info@STARAnalyticalServices.com or phone 781-861-STAR (7827).