How to Tune Grammars

How to Tune Grammars

There are many places to make effective changes to your speech recognition solution, but generally, we have found grammars to be the easiest and most effective place to start. In this tip, we will look at how to detect errors and modify grammars to optimize performance.

Before we begin, let's define some terms.

General Guidelines

  1. In–Grammar (IG) and Out–of–Grammar (OOG) are labels that look at whether the speech engine matches a path in the grammar with what the caller actually said. If it can, then the spoken words are considered In–Grammar, if not, the spoken words are considered.
  2. Confidence Scores indicate the speech engine's certainty about the answer it returns.
  3. Confirmations are dialog techniques to help the speech application avoid making a mistake, in cases where the results are ambiguous.
  4. Substitutions are a particular kind of error the ASR makes; this occurs when the result from the ASR does not match the words the caller said.

Out–of–Grammar Indicators

There are a number of ways to determine whether or not the error is due to an Out–of–Grammar issue. The easiest, most efficient way to discover this is to use the tools provided by the speech platform or speech engine provider. Pre–configured reports will usually highlight these issues up front. If Out–of–Grammar is a big problem, you will likely receive many customer complaints and low completion and usage rates. Call lots will also have many "No Matches" or empty results.

Finally, when you do obtain results, the confidence score will be significantly lower that the rest of the application. As the call logs are transcribed, look for low accuracies. ASR's will usually get anything that is In–Grammar, so if there is still low accuracy, look for a bunch of Out–of–Grammar speech.

The other good indicator is a high Out–of–Grammar rate. Generally, this is a direct indication that callers are not saying things your grammar understands. There are a few reasons for Out–of–Grammar problems, most of which are easy to resolve. One common reason is that the grammar designer simply forgot to add items, but callers are asking for them. Leaving "next" out of a navigation grammar, or forgetting to add a product name is not uncommon.

Another common error is forgetting commonly used synonyms, for example, "copier", but not "Xerox" or different dialectal versions such as "soda" in the West, and "pop" in the South. Usually you can just add the missing items.

In–Grammar Indicators

Typical In–Grammar issues will often be oriented towards improving the confidence scores of recognition, but you will also confront misrecognitions.

What kinds of issues arise frequently?

Regularly confused phrases are fairly common, and often result because two or more phrases sound quite similar. Another common issue is the result of bad pronunciations in the grammar. Speech engines provide a methodology for arriving at pronunciations for words that aren't in the dictionary, but the automatic pronunciations are not always the best.

So, how can we handle this?

For regularly confused phrases, differentiate them by choosing alternate ways to describe the words. For bad pronunciations, you must add the pronunciations to the dictionary or grammar. There are tools for helping with this task, although they are nearly always ASR–specific.

© 2018 LumenVox, LLC. All rights reserved.