LumenVox is excited to announce the release of LumenVox Version 17.0.200. In this release, we have:
- Added support for a new short-utterance transcription (Natural Language) functionality to process audio with a maximum length of approximately 30 seconds.
- Added a new Out of Service configuration option for the ASR (Automated Speech Recognizer) service, allowing system administrators to enter maintenance mode from the Dashboard, which permits currently pending requests to be completed, but any new requests will be rejected (to be potentially handled by other ASR servers in the cluster).
- Added a new feature to the ASR load-balancing mechanism to actively route ASR requests based on the language specified.
Useful for situations where you do not want to be constrained by a specific grammar, or challenged by implementing a more complex and costly Statistical Language Model, the LumenVox Short-Utterance Transcription functionality utilizes a built-in, general Statistical Language Model that has been tuned for everyday use to provide a text representation of supplied audio.
Supporting LumenVox’ commitment to making speech applications more secure and easier to administer, additional enhancements were made to our diagnostic tools and dashboard, including more robust grammar handling within the LumenVox Speech Tuner.
For a comprehensive list of improvements and features released with LumenVox Version 17.0.200, please click here.
I bought a fancy new car about five years ago; all the bells and whistles, every luxury upgrade that money could buy. At 3,000 miles, I got an oil and filter change and put air in the tires. I haven’t taken it to the mechanic since. It’s still running—maybe not as well as it used to—but as long as it gets me from point A to point B, why should I do anything else?
Okay, that’s not a true story. Yet I see this happen all the time with the Cadillac of self-service investments: speech recognition applications. Despite the initial time, effort, and dollars spent, companies often take a set-it-and-forget-it mentality with IVR. Tuning a speech application shouldn’t only happen after the pilot phase; it should occur at regular intervals to ensure the application is running just as smoothly as that vehicle in the driveway.
As we learned in the last Blog post, there are many benefits to tuning, so I won’t reiterate them here. Rather, let’s talk about the process, how it works, and the type of improvements you can expect to see.
The first step is identifying when to initiate a tuning cycle. This can be at pre-defined intervals (annually, for example), following an application enhancement, or even timed in conjunction with an outside event that affects usage and traffic flow to your application. Perhaps you’re an insurance provider, and the government has just mandated coverage and benefit changes. Policyholders may call your application with questions and directives that are new and unexpected. Tuning helps reveal what callers are asking for, and how those needs change over time.
Once you’ve decided to engage in tuning, it’s time to enable utterance capture on your speech recognition server. I usually recommend a minimum two-week period for this, but the interval could be longer or shorter based on call volumes. Before utterance capture is enabled, be sure to play a “your call may be monitored or recorded” message up front to keep the legal department happy, and always double-check that utterance capture is working by listening to a WAV file or two. It’s not unusual for there to be permissions issues writing files to a directory on the server.
While utterance capture is underway, you’ll want to make a list of the dialogs to tune. Prioritize those that get the most use, have high error-out rates, or represent critical “gates” in the call flow. If, for example, failure at a certain prompt prevents a caller from going down an important path (such as making a payment), that “gate” should be tuned for optimal performance. Call event reports will come in handy for identifying any problem areas in the application. In general, I suggest minimizing the number of yes/no or digit dialogs that receive tuning attention because they typically use shared grammars, and any findings at one prompt can be applied to the others.
Armed with utterances, logs, and a tuning plan, it’s now time to load up that data into the LumenVox Speech Tuner. Luckily, the tool provides a very user-friendly interface for transcription and analysis. But once you’re listening to caller recordings, what are you actually looking for? Well, tuning is a fairly subjective process that requires a skilled ear and critical thinking, and it’s sometimes difficult to distinguish trends from outliers. That said, I’m typically on the lookout for: (a) out-of-grammar utterances of a significant sample size, (b) red herring “sound-a-likes” that confuse the speech engine, (c) prompts that mislead the caller into giving unexpected responses, (d) rejection of valid utterances due to confidence scores, and (e) “talk-off” issues where only partial utterances are captured. Addressing such problems may require grammar updates, new phrase recordings, configuration changes, or a combination of all three. A trained voice user interface expert can assist in making and implementing the tuning recommendations based on the data revealed by the Speech Tuner tool.
So, with analysis complete and the application changes in place, what type of measurable improvements can you expect to see? The answer here is always: it varies. You may experience self-service task completion rates that jump from 50% to 75%, but you may not. The fruits of a tuning effort are typically weighed over time and over multiple iterations. False-Accepts (the phrases that the recognizer accepted, but shouldn’t have) and False-Rejects (the phrases that it rejected, but should have accepted) should decrease. Confidence scores for Correct-Accepts should increase. Follow-up tuning cycles will expose these trends, but it’s almost never possible to assign your expectations a hard-and-fast number. Continuing to analyze call event reports will help illuminate where the gains have been made and where to focus your efforts in the next tuning cycle.
Performing these steps in regular intervals will ensure that you’re getting the maximum mileage out of your investment, and protects against a user interface breakdown that could have been avoided with routine maintenance. Not to mention the fact that customer satisfaction will improve when the user experience does!
Please contact your account manager or LVSales@LumenVox.com for more information on the LumenVox Speech Tuner. For more information on the speech tuning process, or to engage Interactive Northwest (INI) in an application tuning cycle, visit the Contact INI page to speak with a qualified speech mechanic. Vroom vroom!
Interactive Northwest, Inc. (INI) develops innovative interactive voice response (IVR), computer telephony integration (CTI), and self-service applications for high-volume contact centers in markets such as government, healthcare, finance, utilities and service industries. A strong commitment to platform expertise, seamless systems integration, and project management excellence uniquely position INI to provide value to its customers. As a long-standing partner in the Avaya DevConnect program and developer of call center speech applications, INI has a deep history in deploying applications on Avaya platforms — making it a reliable partner capable of delivering results that promote the success and profitability of its customers. www.interactivenw.com
LumenVox version 12.2, scheduled for release on Tuesday, Sept. 2, has a large number of exciting new changes. In particular, the Tuner is getting a major series of improvements, and some cool new changes have been added throughout.
From almost top to bottom, we have looked at how we can improve the usefulness of the LumenVox Speech Tuner. One of the first things we realized is that many users have trouble figuring out what they need to tune the most.
Analyzing by Menu
Loading data into the Tuner can be overwhelming, so we added a new concept to the Tuner called a menu. A menu is designed to allow you to filter data so you can tune a specific menu in an IVR or speech application.
The way this works is the Tuner analyzes the grammar files that were in use for each speech interaction. A main menu in a banking application might use the following grammars:
And a “transfer funds” menu might use:
Because the Tuner knows which grammars are active for which speech interactions, it can make logical inferences about which interactions should be grouped together. That grouping is the menu system. A new dropdown allows you to select from the various menus the Tuner recognizes and just pick the one you’d like to focus on.
New to the 12.2 are Tuner Wizards, a series of automated tools that guide you through the process of identifying problems and focusing on the relevant data. You can fire up the new Tuning wizard, pick a menu (or all of the data), and choose from a list of options to focus on. That list includes:
- Confidence Threshold Tuning
- Accuracy Tuning
- Out-of-Grammar Tuning
- Out-of-Coverage Tuning
- No-Input Tuning
- Decode Speed Tuning
- Decode Failure Tuning
The Tuning Wizard will let you know whether your data exhibits any problems related to these issues and then will help you identify which interactions contribute to the particular type of issues you’re facing. It’s a great way to focus your time so that you only pay attention to the items most relevant to you.
Grammar Editor Changes
The Grammar Editor is a long-standing feature in the Tuner, giving developers an easy way to build, edit, and test their grammars. Several new features enhance the capabilities even further:
- Multiple grammar parses. Previously, the Grammar Editor could only parse a sentence against a single grammar at a time. A new option allows developers to parse any combination of loaded grammars, making it easier to test how combinations of grammars will affect grammar coverage.
- Pronunciation Checker. A new module called the Pronunciation Checker shows where pronunciations for grammar items come from: are they in our built-in dictionary? A user-defined lexicon? Or are they being produced by our statistical pronunciation rules? Words which don’t have good pronunciation definitions often lead to errors in recognition, so this is a useful module for troubleshooting performance.
- Random Sentence Generator. This module generates 10 random sentences at a time that are allowed by the grammar. Using it, you can check grammar coverage to make sure that the words and phrases you expect to be in grammar are, while simultaneously ensuring that phrases you don’t expect to be in grammar are not.
We’ve written a lot about speech tuning (for instance: Calculating Speech Recognition Accuracy, Collecting Data for Speech Tuning, Practical Tuning Strategies, and The Importance of Tuning) because it’s such a vital part of building great speech applications.
But as much as we might like tuning because it makes applications better, there’s also a real bottom-line value associated with it. In a new whitepaper on LumenVox.com we describe the ROI of Speech Tuning, showing how to calculate the return on investment (ROI) of speech tuning.
You can go read the full paper to understand how tuning can save you hundreds of thousands of dollars per year, but we’ve also put together this infographic to summarize the process: