Rss Categories

Performance Graphs

Reference Number: AA-01123 Views: 3445 0 Rating/ Voters

The Tuner displays two graphs based on test results: the Confidence Histogram and the Receiver Operating Characteristics (ROC) graph.

The Confidence Histogram displays a histogram with information about confidence scores. The histogram is a way of looking at confidence scores, with low confidence scores on the left side of the graph and higher scores on the right.

The green, red, and purple vertical bars represent utterances: green bars are correctly recognized utterances, red bars are incorrect, and purple are out of grammar utterances. The height of each bar indicates the frequency of utterances at that confidence level.

The purpose of the confidence histogram is to give you a snapshot of how accurate the recognition is at various confidence levels, so you can set accept, confirm,and reject thresholds in your application. A good speech application will check the confidence score of an utterance and ask the user to confirm the utterance if it the score is too low. If the score is really low, the application will reject the utterance and not waste the user's time asking for a confirmation.

You can gain information about accuracy at a given point by moving the mouse pointer to a spot on the histogram. The upper left portion will display the confidence score at the given location, and the accuracy level of utterances to the right of that location.

With the confidence histogram, you can find the appropriate confirm and reject thresholds for your application. The goal is to minimize the numberof red and purple (incorrect and out of grammar) utterances to the right of the confirm threshold, while minimizing the number of the green utterancesto the left of that threshold.

At the bottom left of the confidence histogram are the average confidence scores for correct and incorrect utterances. Along the very bottom are suggested ranges for rejection, confirmation, and acceptance.

The Receiver Operating Characteristics graph displays theROC information for a test set. This is a fairly complicated graph which is provided for the benefit of those with significant understanding of speech recognition and signal processing.

Describing the exact nature of this graph is out of the scope of this document. However, you may be interested in this Wikipedia article about ROC if you are unfamiliar with the term.