Browse
 
Tools
Rss Categories

Out-of-Coverage Items

Reference Number: AA-01951 Views: 1968 0 Rating/ Voters

An important feature introduced in version 12.2 of the Speech Tuner is the ability to flag an utterance as "Out of Coverage" (OOC). An OOC utterance is one which you would never expect a speech application to handle. Examples of OOC utterances might be:

  • Non-speech background noises
  • Cross talk (the ASR user talking to somebody else in the background)
  • Wildly inappropriate phrases or sentences

OOC utterances differ from Out-of-Grammar (OOG) utterances in that OOG utterances are things the application might conceivably handle. For instance, if you had a banking application that asked a user to speak what they wanted to do and a user said "Hear my balance" but the application was expecting "Hear my account balance," it would be reasonable to add the "Hear my balance" phrase into the grammar.

The reason to track these types of utterances separately is because it is very useful to review OOG utterances for potential inclusion into grammars (especially with the OOG Tuning Wizard), but you would never want to modify a grammar to support an OOC phrase. It is useful to track OOC utterances, though, because high numbers of OOC utterances suggest you may have a problem in your application (see the OOC Tuning Wizard article for more information on how to identify and solve such problems).

OOC utterances, like OOG utterances, are not used in the Tester's calculations for word or utterance accuracy. On the Confidence Histogram, they do appear as blue bars (separate from the purple OOG bars) but they are included with the Out of Grammar performance metrics when calculating Correct Reject and False Accept rates on the Histogram.

Marking Utterances as OOC and OOG

To mark an utterance as OOC, simply enter its transcript text as ++OOC++. To help make this faster, you can simply press the Escape key on your keyboard to mark an utterance as OOC from within the Transcriber. You should mark any sort of "junk" as OOC so you can spend very little time transcribing it.

OOG utterances are not marked as such; instead you should simply transcribe what the user said. The Tuner will automatically determine that an utterance is OOG when the transcript text does not produce a valid parse through the grammar. You should transcribe what the user actually said as it may help you determine what phrases to add your grammar.

The ++GARBAGE++ Tag

In versions of the Speech Tuner before 12.2, pressing the Escape key in the Transcriber marked utterances as ++GARBAGE++ which were then counted as OOG. At the time, our recommendation was to just mark "junk" as ++GARBAGE++. The addition of OOC as a separate category changed this, and we now suggest that ++GARBAGE++ no longer be used as the sole transcript text for an utterance (junk utterances should be ++OOC++). You may still use ++GARBAGE++ tags within a longer transcript to denote noises, though generally there are other, more appropriate, noise tags to pick.

If you have an older Tuner database that includes utterances transcribed as ++GARBAGE++ they will show up as OOG. If you would prefer to have them counted as OOC, you can either re-transcribe them as ++OOC++ or you can edit the Tuner database:

  1. Open the .Interactions file in a text editor
  2. Use a string replace or "find and replace" function to replace all instances of ,++GARBAGE++, with ,++OOC++, instead. (Note the commas at the start and end -- this guarantees you will only find cases where the entire transcript is ++GARBAGE++).
  3. Save the Interactions file.

Backwards Compatibility

Users who prefer the older style (only OOG, no OOC) can enable a backwards compatibility mode by doing the following:

  1. Open the Tuner
  2. Go to Edit > Settings.
  3. Click the Advanced tab.
  4. Check the box labeled "Treat OOC as OOG (for backwards compatibility)".
  5. Click OK.