The Tuning Wizard, introduced in version 12.2 of the Speech Tuner, is an interactive tool that can identify and highlight issues with your speech application.
To start the Wizard, simply click the magic wand icon on the toolbar:
You can also click View > Tuning Wizard or press the F8 key to access the Wizard.
Choosing a Task
Once you have started the Wizard, it will ask you which of three tasks you would like to focus on.
These options are:
- I'd like the Tuning Wizard to analyze my data for me and report issues. This is the most common choice, and will let the Wizard help find the issues you want to focus on based on the data you specify.
- Application is not working for certain languages. Used for cases where TTS or ASR works for some languages but not others.
- A certain grammar does not load or work correctly. Troubleshoots a very specific grammar, looking for potential errors.
If you pick the first option, you will ultimately be looking at the following tasks:
The individual knowledge base pages for the tasks describe in some more detail what those tasks and issues consist of; the rest of this page describes the basics of using the Tuning Wizard.
To be successful, speech tuning requires a representative sample of data which is transcribed (for details on this, see our Collecting Data for Speech Tuning whitepaper). For optimal results, you must transcribe this data as well, meaning use the Transcriber tool to type out transcripts that contain exactly what users said.
If you do not have enough transcribed data, you will see this screen:
At this point, you have two options:
- Our recommendation is to transcribe more data as indicated by the Tuning Wizard. This will improve the validity of your tuning results and allow the Wizard to better identify issues. If you are interested in confidence tuning, utterance accuracy, OOG/OOC, or No-Input analysis, having at least some transcripts is required to get meaningful results. You can transcribe data by clicking the Transcribe More Data... button, which will take you in the Transcriber Wizard.
- If you are interested only in issues such as decode failure rates or slow decodes (which do not benefit from transcripts), or you feel you have enough transcribed data and would like to ignore the warning, you can click Next to continue.
If you had enough transcribed data (or you chose to ignore the warning), you will be given the opportunity to tell the Wizard what sorts of issues you are interested in.
You may choose any issue by checking the box next to its name (the Select All Issue Types box lets you check everything, which is the default behavior). The various issue types are explained in more detail by clicking the blue help icon next the count of issues.
Most users will likely want to simply leave all issues selected and click Next to move to the Menu Selection screen. However, advanced users may click Configure to get more options:
Configuring the Issue Analyzer
In the configuration screen, you can adjust the Tuning Wizard's sensitivity to a particular issue. The help icons provide detailed information about how issues are calculated, but in general moving the slider to the left (lower numbers) makes the Tuning Wizard more likely to flag a menu as having issues. So if you are really concerned about No-Input errors, you would move its slider all the way to the left, and if you didn't care at all about decode speed, you would move its slider all the way to the right.
There are a few other options in this configuration screen:
- Only review menus with sufficient transcriptions configures the Tuning Wizard to ignore menus if they do not have the number of transcriptions set by the slider below (the default number is 1000).
- Select analysis depth lets you tell the Tuning Wizard to look at menus or grammar sets. Most users will likely focus on menus, but if your data contains vastly different grammar sets for a single menu, this will let you pick specific grammar sets to look at.
- Reset Defaults resets all sliders to their default values (which most users will find appropriate).
Select a Menu
In this screen, you simply pick the menu you would like the Tuning Wizard to analyze:
You can always choose Global (All menus) to look at all of your data. Click Next once you have picked your menu.
If you select a menu and Apply Filter and Exit, the Tuning Wizard will filter to just the menu you have selected and then exit. This can be used as a shortcut to review the data in a given menu.
Summary of Issues
You will be given a summary screen displaying all the issues which the Tuning Wizard found (based on the issues you flagged for analysis, the sensitivity settings, and the menu/grammar you choose to focus on).
Next to each issue is a number or percentage indicating the frequency of the issue. You can click on an issue you would like to examine and click Next to get to any of the results pages: