Rss Categories

Out-of-Grammar Tuning

Reference Number: AA-01945 Views: 6462 0 Rating/ Voters

The Out-of-Grammar Tuning Wizard is a way to help identify issues related to users speaking out-of-grammar utterances. It differs from the Out-of-Coverage Tuning Wizard in that it focuses on utterances which contain phrases that might be valid, but are currently not in the grammar (see our Out-of-Coverage Utterances page for more details on the difference between OOG and OOC). You must have transcripts for your data in order for the Tuning Wizard to correctly identify OOG utterances.

By selecting OOG as an issue in the Tuning Wizard, you are able to see the OOG results page:

This page displays a list of all OOG utterances for your selected menu or grammar set. They are sorted by frequency, meaning the most commonly spoken phrases are at the top of the list. You may click the Apply Filter and Exit button to exit the Wizard and apply a filter to the data so that you only have the OOG utterances visible.

Improving OOG Performance

To improve an application that has a large number of OOG responses, there are three main tactics you can pursue:

  1. Reduce the frequency of OOG utterances by improving prompts so that users are more clear on what can be said. High OOG frequency tends to mean that users simply do not understand what they can say, so the prompt should be changed to be explicit. As an example, try replacing open-ended prompts ("What would you like to do?") with directed prompts that include in-grammar phrases ("You can say account balance, transfer funds, or go back.").

  2. Add OOG utterances to your grammars. Using the Grammar Editor, you can modify your grammars to support new phrases and parse those phrases to ensure they work correctly. See our free whitepaper All About Grammars for more tips and best practices on improving grammars.

  3. Improve the way that the application handles out-of-grammar responses. Sometimes you simply cannot reduce the OOG frequency very much, so it becomes important to just make sure that the application is behaving intelligently to OOG phrases. Consider using a confirmation threshold if you are not already, and run the Confidence Tuning Wizard to ensure that your thresholds are tuned to correctly reject OOG answers.

Running Tests

It is important after making changes to grammars for OOG utterances that you use the Tester to run tests on your data. Because adding OOG utterances to a grammar can change the performance for in-grammar utterances, be sure and evaluate the performance of changed grammars across a wide set of data. It is a good idea to run the Confidence Tuning Wizard and examine the confidence histogram after each round of OOG tuning.