What & Why
- Distributed architecture is taking a single computing task and distributing it among
- The primary reason for distributed architecture is redundancy. Let's say you have a
critical system, such as a PBX, which handles your incoming calls. If the power supply
in the PBX machine catches fire, you could lose all of your calls. In a distributed
environment, when one machine fails, the other would pick up the slack.
- Load balancing is another good reason for distributed architecture. This allows you
to split up tasks among multiple machines. If any one machine becomes overburdened with
too much to do the others would help out. The load would be evenly distributed so no
single computer becomes bogged down.
- Distributed architecture is very scalable, which means it's easy to add more machines.
So as the need grows, as you do more business, as traffic grows, you simply plug in more
machines to the cluster of machines and the increased traffic is handled.
The LumenVox Speech Engine works well in distributed environments, because internally we
use a client-server model. There are two parts of the application when the LumenVox Speech
Engine is installed.
- The first part is the speech client, this is the part that talks to your application.
You may have a platform that runs your IVRs, or a PBX, or any other application that
communicates with the Speech Engine using the speech client. This communication could be
by way of MRCP or an API integration. Audio is sent, grammars are sent and the LumenVox
Speech Engine sends back decodes with the actual text of what a user said.
- Internally, the Speech Engine client takes the speech and passes this data to a
speech server. The server application is what actually performs the decode. The client
and the server speak to each other using standard TCP/IP port connections. This is good
because unlike an API integration, which has an integration from your platform to the
LumenVox engine, the API cannot traverse a network. API is used with local machines only.
As opposed to the LumenVox client and server speak to each other using TCP/IP, which
allows communication over a network.
- We can have the speech client and the speech server on the same machine if we choose
to, but we can take the speech engine and put it on a separate machine and have the client
and server still able to talk to one another. The advantage to doing this is that it
allows us to balance the load. Speech recognition can be very resource intensive, it can
require a good deal of memory, and use many CPU's, etc. You may not want this all
happening on your PBX or IVR server, so you could just add the client to the PBX or IVR
server then have the speech server perform decodes on a separate machine which could be
dedicated to speech recognition if you choose.
- For all for this to work, the speech client contains a small routine called the server
monitor. This routine takes a list of speech servers to use and your client application
supplies the monitor with the location of all of the speech servers so that it can
routinely check all of the speech servers. So if you have three speech servers: A, B, and
C, the client is set up on the PBX with 3 separate servers elsewhere. The server monitor
will constantly be communicating with all three servers, A, B, and C. If B suddenly
becomes disabled, the server monitor will realize this and it will no longer send decodes
to server B, it will simply use A and C only. However, if B server comes back online, the
monitor will see this and will begin sending decodes once again. This is good for
redundancy and failover.
- The server monitor also keeps track of how busy the servers are. So if server A
suddenly becomes very busy, the request for decode would be sent to B or C. New decode
request are sent to the server that is least busy. So LumenVox is automatically load
performing a type of load balancing if you have multiple servers set up.
- The final piece that allows LumenVox to work in a distributed environment is our
licensing. Licensing is very convenient for this purpose. LumenVox is not concerned
with how many installations of the engine and the client you have, at least from a
licensing perspective. We don't have per seat or per server types of licenses. The
LumenVox licenses are all per decode; when a speech port is opened to perform a decode,
this is when a license is used. The speech engine and client on as many machines as you
want but it will not take up a license until decodes are done. The client asks for
licenses during a decode, it will look at a license server. Licenses can be installed on
the same machine as the client or on a separate machine. So you could have ten separate
speech clients communicating with twenty separate speech servers all speaking to one
license located on yet another machine.
The following is a graphic representation of the above:
In this graphic one of our customers, Ontelnet has two PBX servers that take incoming
SIP traffic. The servers are set up in a cluster and performing load balancing. Since the
PBX is integrated with LumenVox our client application is on the systems that the PBX's are
running. However, the PBX are very busy and the customer does not want to have to do speech
decodes on the PBX servers because they are handling so many calls. What they have chosen to
do is to move the speech decodes onto different servers. As you can see there are multiple
speech servers set up so now when there is a request for speech recognition that request goes
off to the most appropriate speech server because, as we discussed, the server monitor on
each client has a list of all the speech servers and their status. Also note that we have a
license server, all the clients communicate with the license server, and we also can see that
there is a back up license server that can be used if the primary license server fails for