Telephone self-service is able to provide real value to consumers. It makes access to information available 24/7. It is available without waiting in line. Consumers can purchase things. They can check availability. They can check the status of an order. They can obtain directions. In addition to making customers happy, telephone self-service is also able to save money for the enterprise that is providing it.

Unfortunately, this situation is rarely achieved. Telephone self-service is intensely disliked by many consumers because it is implemented very badly, especially by companies who cut corners just for the sake of extra savings. The caller wants a quick & easy experience and most telephone self-service instead provides an experience that is slow & difficult.

We present here a few of these telephone self-service implementations that were evaluated during the last months. These were selected mostly at random and it’s unlikely that they represent the worst implementations.

Table 1 summarizes the results of this testing. We have adapted the www.get2human.com Voice User Interface (VUI) Standard as the basis for our evaluation. The scoring/rating scheme is simple to understand and scoring of a particular VUI is relatively easy to do. All that is needed is to make a phone call and observe how the system responds. The scheme that we are using has some obvious limitations. The fact that we give each item equal weight is certainly arguable. On the other hand, despite its simplicity, it appears to yield results that seem to correlate with the relative goodness of the self-service system.

For each of the VUIs that we tested, we provide a discussion of it from the perspective of the caller. This is done by Walt Tetschner, who has suffered through many poorly implemented telephone self-service implementations. We also provide a clinical analysis of the system from the viewpoint of a highly-regarded VUI designer (Dr. Walter Rolandi). This identifies what the system is doing incorrectly and offers recommendations regarding what should be done to repair it.

Get2Human Telephone Self-service Rating
Get2Human item NYC Trip Planner Milwaukee Journal Sentinel Common-wealth Choice Dell SSA WSJ NY Times Washington Post Charter Commun-ication Tufts Health
care
American Airlines The Annual Credit Report Service IRS AARP US CIS NSTAR Central Maine Power (CMP)
The caller must always be able to dial 0 or to say "operator" to queue for a human. 0 0 0 0 0 0 0 0 0 0 1 0 0 1 0 1 1
An accurate estimated wait-time, based on call traffic statistics at the time of the call, should always be given when the caller arrives in the queue.  A revised update should be provided periodically during hold time. 0 1 0 0 0 0 0 1 0 0 0 0 0 1 0 NA NA
Callers should never be asked to repeat any information (name, full account number, description of issue, etc.) provided to a human or an automated system during a call. 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1
Callers should always be offered the option to be called back when a human is not available. This option should be offered periodically during hold time. If 24 hour service is not available, the caller should be able to leave a message and be called back promptly when humans are available.. 0 1 0 0 0 0 0 1 1 0 0 0 0 1 0 0 0
Speech applications should provide touch tone (DTMF) fall-back, where appropriate.  0 1 NA 0 0 0 NA 1 NA 1 1 1 NA 1 NA 1 NA
Callers should not be forced to listen to long/verbose prompts.  0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1 0
Callers should be able to interrupt prompts (via dial-through for in DTMF applications and/or via barge-in for speech applications) whenever doing so will enable the user to complete his task more efficiently. 0 0 0 0 0 0 1 1 0 0 1 0 0 1 0 1 1
Do not disconnect for user errors, including when there are no perceived key presses (as the caller might be on a rotary phone); instead queue for a human operator and/or offer the choice for call-back.  0 1 0 1 0 0 0 1 1 0 1 0 0 1 0 1 1
Default language should be based on consumer demographics for each organization.  Primary language should be assumed with the option for the caller to change language.   (i.e. English should generally be assumed for the US, with a specified key for Spanish.) 1 1 0 1 0 1 1 1 0 1 1 1 0 1 0 1 1
All operators/representatives of the organization should be able to communicate clearly with the caller  0 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1
Gethuman Subtotal (Average score) .10 0.60 0.11 0.20 0.10 0.20 0.33 0.70 0.33 .30 0.6 .30 0.11 0.90 0.11 0.78 0.75
Recognize me by caller ID (home or mobile) and then let me enter a four digit PIN to authenticate, so you then have my info on file and don't make me type in long account numbers etc. 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
No "Your call is important to us" greeting.  1 0 1 1 1 0 1 1 1 0 1 1 1 1 1 1 1
No "You can access our website to answer most questions" message. I already know that. 0 1 0 0 0 0 1 1 1 1 0 0 0 1 0 1 0
No "Please listen carefully, since our menu has changed" message.  1 1 1 1 1 1 1 0 1 0 1 1 1 1 0 1 0
Do not play advertisements to me 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
Do not speak to me in legalese or corporate language 0 1 1 1 1 1 1 1 1 1 1 1 0 1 0 1 1
Is it a Quick-&-Easy experience? -3 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1 0
Understand me when I speak a natural number 0 1 NA 0 0 0 NA NA NA NA NA NA 0 NA NA 0 NA
Total Score (Average) 0.00 0.61 0.31 0.33 0.28 0.28 0.50 0.65 0.50 0.35 0.59 0.41 0.24 0.88 0.19 0.76 0.60
 
Get2Human Telephone Self-service Rating (continued)
gethuman item BofA AOL USPS Amtrak CareMark Verizon United Airlines Comcast Eureka FedEx FTC Apple Cingular Verizon Broadband Virgin Mobile
The caller must always be able to dial 0 or to say "operator" to queue for a human. 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0
An accurate estimated wait-time, based on call traffic statistics at the time of the call, should always be given when the caller arrives in the queue.  A revised update should be provided periodically during hold time. 0 0 0 0 0 0 0 0 0 1 0 1 0 0 0
Callers should never be asked to repeat any information (name, full account number, description of issue, etc.) provided to a human or an automated system during a call. 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0
Callers should always be offered the option to be called back when a human is not available. This option should be offered periodically during hold time. If 24 hour service is not available, the caller should be able to leave a message and be called back promptly when humans are available.. 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0
Speech applications should provide touch tone (DTMF) fall-back, where appropriate.  1 1 1 1 1 0 1 1 0 1 0 0 NA 1 1
Callers should not be forced to listen to long/verbose prompts.  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Callers should be able to interrupt prompts (via dial-through for in DTMF applications and/or via barge-in for speech applications) whenever doing so will enable the user to complete his task more efficiently. 0 0 1 0 0 0 0 1 0 0 0 0 0 0 0
Do not disconnect for user errors, including when there are no perceived key presses (as the caller might be on a rotary phone); instead queue for a human operator and/or offer the choice for call-back.  1 0 0 0 1 0 1 1 0 1 0 0 0 1 0
Default language should be based on consumer demographics for each organization.  Primary language should be assumed with the option for the caller to change language.   (i.e. English should generally be assumed for the US, with a specified key for Spanish.) 1 0 1 1 0 0 0 0 1 1 0 1 0 1 0
All operators/representatives of the organization should be able to communicate clearly with the caller  1 0 1 1 1 1 0 1 1 1 1 1 1 0 0
Gethuman Subtotal (Average score) 0.50 0.10 0.40 0.40 0.30 0.10 0.20 0.40 0.20 0.60 0.10 0.30 0.11 0.30 0.10
Recognize me by caller ID (home or mobile) and then let me enter a four digit PIN to authenticate, so you then have my info on file and don't make me type in long account numbers etc. 0 0 0 0 0 0 0 1 0 1 0 0 0 0 NA
No "Your call is important to us" greeting.  1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
No "You can access our website to answer most questions" message. I already know that. 0 0 1 0 1 0 0 0 1 0 0 0 0 0 1
No "Please listen carefully, since our menu has changed" message.  1 1 1 1 0 1 1 1 1 1 0 0 1 1 1
Do not play advertisements to me 0 0 1 1 1 0 1 1 1 1 1 0 0 0 1
Do not speak to me in legalese or corporate language 0 0 1 1 0 1 1 1 1 1 0 1 0 0 0
Is it a Quick-&-Easy experience? 0 0 1 0 0 0 0 1 -7 1 0 0 0 0 0
Understand me when I speak a natural number 0 1 0 0 0 1 1 NA 0 1 1 NA 0 1 0
Total Score (Average) 0.39 0.22 0.56 0.44 0.33 0.28 0.39 0.59 0.00 0.72 0.22 0.29 0.18 0.33 0.29


6/11/09

Walt and
Walt’s VUI View: NYC Transit Trip Planner

The Caller’s View from Walt:


I called the


NYC Transit Trip Planner Voice

to see how well it worked.  I had seen a lot of recent
hype re the new speech-enabled planning that Nuance and
Aspect Software had implemented.


I called 718-330-1234.  After a welcome greeting that
was too long, I encountered a menu that used a different
voice for each menu item.  It had long pauses after each
item.  It only had 5 menu items, but still managed to
use 1 two times.  The response to my entries was
inconsistent.  Sometimes it ignored the actual input (no
overstrike).  Other-times it provided the information
even though it wasn’t requested. (it seemed to insist on
telling the caller about the re-routing of buses on
Broadway, independent of the key that I pressed). 
Getting to the voice service was problematical. 
Pressing 9 to “Continue” was a menu option.   This
seemed to get me to the Trip Planner Voice, but you had
to patiently listened to all of the menu items (over a
minute).    

The
speech system was a highly-directed implementation.  It
explained in much detail how I could specify my
departure and destination points, times, and modes of
tramsportation.  Huge delays between a caller input and
a subsequent response.  I was calling from a quiet
office environment and it had lots of speech recognition
problems.  Classical error recovery torture.  It asked
me when I wanted to leave and I said “
right
away
”.  
It had difficulty recognizing this and forced me to
specify a precise time.   Unnecessary confirmation of my
responses.    It finally read the travel instructions to
me.  It had part of each phrase spoken by the same voice
and used TTS for only the variable portion.  This would
have worked better if they used the TTS voice for the
entire phrase, since the switching of voices is a
distraction.

I
indicated that I wanted to travel from The Empire State
Building in Manhattan to The Yankee Stadium in The
Bronx.  It was 10:30AM when I called.  The system
indicated that the next subway train would not leave
until 11:43AM.   This seemed strange – an hour-&-a-half
wait on a busy weekday morning.  I decided to talk to an
agent.  The automated system did not tell me how long
the wait would be but did state that an agent would be
with me in “just a moment”.  After waiting for 3-4
minutes and being told to go to the MTA website a number
of times, I finally reached an agent.  No functioning
CTI was in place and I had to repeat everything that I
had entered into the IVR.  The agent indicated that the
departure time for the train that I was looking for at
10:30AM, was 10:43AM and that the 11:43AM time that it
gave me was incorrect.  I went to the NYC Transit web
site


www.tripplanner.mta.info
 
 
It
confirmed that the departure time was 10:43AM


This is one of the very worst self-service systems that
I’ve run into.  Virtually everything about the
implementation seems to be wrong.  In addition to it
being an irritating and time-wasting experience, the
information that it provided was wrong.

 

“Dr.” Walt Tetschner examination of NYC Transit Trip
Planner:

Patient:                      
NYC Transit Trip Planner

Task:                                      Get from The Empire State
Building to The Yankee Stadium

Toll Free Number:    1-718-330-1234

 

The Examination Session

Several
calls were made to the IVR on a busy weekday morning.  
Only one call was actually necessary to complete the
task. Call duration was 298 seconds or approximately
four and a half minutes.



Call Details:

The
“system” appears to be two or more systems linked
together.  The call was initially answered by a DTMF
system.  A menu was then presented that used a different
voice for each menu item.  Each menu item was followed
by a long pause (approximately 5-6 seconds), which
seemed to be giving the caller adequate time to
respond.  If the caller did not respond with a key
depression within this pause period, the system appeared
to assume that this item was rejected and proceded to
the next menu item with a different voice. 

Upon
pressing DTMF9 at the last item of the “menu”, the call
is apparently transferred to another system where yet
another voice talent appears.  At this point, the
“system” switches over to speech recognition.  A
significant delay occurs to get switched to the speech
system.

After a
long comment about wheelchair accessibility, it finally
says:

If you need to know how to get from one location to
another, say plan a route.”

 I said: “Plan a route.”

 “All
right. In order to plan a route, I need to know what
type of starting point, your destination and when you
will be traveling.”

Which borough are you starting in?”

I said: “Manhattan.”


 “Sorry
I didn’t get that. Are you starting in Manhattan,
Brooklyn, Queens, the Bronx, Staten Island?”

 I said: “Manhattan.”


 “I need to know which
type of starting point you will be using. So, would you
like to tell me an intersection, landmark, or point of
interest, subway stop, or address?”

I
used the landmark approach going from The Empire State
Building to Yankee Stadium.  This was a tedious
experience with long prompts, long pauses, and frequent
speech recognition errors, where I had to repeat the
input.


After it provided the travel information to me, it
informed me that I could say “goodbye” and hangup to end
the call.

The Diagnosis

1.     
The “menu” that sequentially reads each item and
waits for a response, is a grossly inefficient way to
communicate with a caller. 

2.     
The “system” mixes exclusive interactive
modalities (DTMF only and speech only).

3.     
The database access is providing incorrect
information.

4.     
System uses different voice talents for different
phrases or portions of a phrase.

5.     
Long pauses exist everywhere.

6.     
A long web ad is played.

7.     
Many prompts are unnecessary or far too long.

8.     
The speech system seemed to have difficulties
recognizing simple, easy to recognize utterances.

9.     
The system stupidly apologizes to the user when
it fails to “understand”.

The Prescription

1.     
The existing “menu” structure should be
eliminated in favor of a true menu.  Better yet, a NL
implementation with agent-assisted ASR (AAA) would yield
a superior solution.

2.     
Adopt a single interactive modality universally
supporting both DTMF and speech.

3.     
Fix the database access to assure that the data
that is being delivered is accurate.

4.     
Add CTI and make sure that the agents are using
it.

5.     
Add a queue wait time announcement.

6.     
Implement a consistent overstrike.

7.     
Either shorten or eliminate the web ad.

8.     
Use a single professional voice talent for all
prompts.

9.     
Eliminate or trim any unnecessary prompts and
prompt content.

10. 
Eliminate apologies (users don’t care when
computers are “sorry”).

11. 
Investigate and tune system grammars.

The Prognosis



Good if the prescription above is followed.  The
problems are so severe though, that this requires
virtually a total re-design of the existing system.

 

 

 

Jan
8 2008

Walt and
Walter’s VUI View:


Milwaukee Journal Sentinel

The Caller’s View from Walt:


I called the Milwaukee Journal Sentinel to obtain
information about starting a subscription. I was greeted
by the typical up-front verbage that wasted my time by
forcing me to listen to bunch of unnecessary stuff like
instructing me to use the MJS website.  Finally
gave me a TouchTone menu, but managed to change voices
to cause confusion.  I selected something that was
vaguely close to what I was calling about which put me
into another system that had a different voice and
required speech input.  I responded to the
“Something else” category by saying “new subscription”. 
When requested, I entered my phone number and zip code
and it came back and gave me the rate for a 1-week
subscription.   The rate was $18.42 a week. 
It then started to verify my name and address by
spelling everything out.  The verification process
that they use is painful.  Virtually everything
that you say gets a repeated back to you followed by:
“is that correct?  Please say yes or no?  Half
of the time it does not recognize the yes response
properly and you have to repeat it.  I wanted rates
other than the weekly rate which seemed rather high. 
The automated system didn’t appear to have this
information available so I attempted to get to an agent. 
After hitting 0 a few times, I was connected into a wait
queue that informed me that the weather was bad and
negatively impacting newspaper deliveries and that 19
people were ahead of me in the line.  It offered to
call me back.  I opted for this.  After a few
days, I have still not received a call-back.  I
attempted to use the system again.  I entered all
of my information again and then waited patiently in
line.  When I finally got to the agent, she did not
have any of the information that I had laboriously
entered into the automated system. 

Dr. Walter Rolandi examination of Milwaukee Journal
Sentinel Customer Service:   

Patient:                      
Milwaukee Journal Sentinel

Task:                           Determine the cost of a new
subscription.

Toll Free Number:    1-800-759-6397

 

The Examination Session

Several
calls were made to the IVR over a holiday weekend.  
Only one call was actually necessary to complete the
task, which, if unnecessarily time consuming, was
reasonably easy to complete.  Call duration was 147
seconds or approximately two and a half minutes.

 



Call Details:

The
“system” appears to be two or more systems linked
together.  The call was initially answered by a
DTMF system.  After reading a 30-second series of
introductory messages which included a web ad, the user
is instructed to “press one for our automated phone
system”.  These various messages were apparently
recorded using three different female voices, one
decidedly more professional sounding than the others.

 

Upon
pressing DTMF1, the call is apparently transferred to
another system where yet another voice talent appears. 
At this point, the “system” switches over to speech
recognition.

 

After
an approximately 10-second banner prompt explaining that
inclement weather had caused various delivery problems,
the system says:

 


If that’s not why you’re calling, which of these can I help
you with?


A delivery issue


Vacation hold


Your account or


Something else.

 

The
system then immediately repeated these choices. 
While the “Your account” option seemed like a
possibility, I choose the “Something else” route. 
This quickly led to a four-item submenu, the second
choice of which was “Start a new subscription”.

 

After
saying, “Start a new subscription”, the system prompted
me for my telephone number.   It failed to
recognize the number on the first attempt.  After
the second attempt, the system read my number back
correctly saying:

 


Is that right?


Just say “yes” or “no”.

 

The
system then prompted me for my zip code, again asking me
to confirm what it “heard” with:

 


Is that right?


Just say “yes” or “no”.

 

Oddly,
it failed to recognize me when I answered, “Yes” but got
it on the re-prompt.  About ten seconds of
information about a payment plan was then read out,
finally followed by the subscription cost amount. 
Upon hearing the rate, I hung up.

 

The Diagnosis

1.     
The “system” mixes exclusive interactive
modalities (DTMF only and speech only).

2.     
System uses two, three or four different voice
talents.

3.     
The web ad is an IVR cliché.

4.     
Many prompts are unnecessary or far too long.

5.     
The system stupidly and repeatedly instructs the
user to answer its yes/no questions with, “Just say
‘yes’ or ‘no’”.

6.     
While the production quality of the speech-only
component is very good, the overall production quality
not very professional.

7.     
The speech system seemed to have difficulties
recognizing simple, easy to recognize utterances.

8.     
The system stupidly apologizes to the user when
it fails to “understand”.

 

The Prescription

1.     
Adopt a single interactive modality universally
supporting both DTMF and speech.

2.     
Either shorten or eliminate the wed ad.

3.     
Use a single professional voice talent for all
prompts.

4.     
Eliminate the pointless “Just say ‘yes’ or ‘no’”
prompt.

5.     
Eliminate or trim any unnecessary prompts and
prompt content.

6.     
Eliminate apologies (users don’t care when
computers are “sorry”).

7.     
Investigate and tune system grammars.

 

The Prognosis



Really quite good.

 

 

 

 

 


Dec 8 2008

Walt and
Walter’s VUI View: Commonwealth Choice

The Caller’s View from Walt:


Commonwealth Choice is the government coordinator for
the universal healthcare program that the state of
Massachusetts offers to their residents.  I called
their customer support line (877-623-6765) to find out
what the basic eligibility requirements were. 


Commonwealth Choice starts by playing an ad and then
telling you to go to their web site, describes what it
contains and what the address is.  It then
instructs the caller to press 1 for English.  It
then forces the caller to listen to a long menu of
choices.  If you press 5 at the 1st
menu, you are then requested to indicate the number of
adults and children in your family and then checks that
your income is below the cut-off.  If you qualify,
it then gives you another long menu.  If you press
0, it places you in a wait queue for a human.  The
wait time typically appears to be in excess of 5
minutes.


I noticed that the number of people in your family can’t
be more than 8.  If you enter 9 (or any double
digit number), you are shuffled back to the main menu.  
This looks like a glitch in their logic, since families
with more than 8 members do exist and would appear to be
eligible for State-provided health insurance.


After entering my family information and my income, I
was connected to a human (after a wait of about 5
minutes).  During the wait, the system asked me for
additional information regarding what I was calling
about.  When I finally did get to an agent, she was
polite and seemed knowledgable.  None of the
information that I had provided to the system was passed
to the agent and I had to start all over again. 

 

Dr. Walter Rolandi examination of Commonwealth Choice
Customer Service:          

Patient:                                 
Commonwealth Choice

Task:                                    
Determine eligibility for universal healthcare plan.

Toll Free Number:             
1-877-623-6765

 

The Examination Session


Technically, it was not possible to know whether the
task was actually complete.  After I answered a few
questions, the system transferred the call to a
representative, presumably because my answers implied
that I was not eligible for the plan.  Only one
call was required to navigate to this point and doing so
required 124 seconds to complete.  The entire call
lasted 133 seconds. 

 

Call Details:

The
system is a highly structured DTMF only application.  
The user experience begins with a brief greeting
followed by about 20 seconds of pointless introduction
and a web ad.  Then follows a 14 second Spanish
option.  After selecting “English”, an extremely
wordy 6-item, 46 second Main Menu is heard.  
DTMF choice 5 began with:

 


If you are unsure whether you are eligible….


 

I
entered DTMF 5.   The system then asked for
the number of adults and children living in the
household.  It apparently uses this number to
calculate the household income breaking point for the
number of people one enters.  The system then asked
a yes/no question as to whether the household income
exceeded a particular amount.   After
answering, “yes”, I heard the transfer message.

 

Thus,
the system required over two minutes to obtain three
simple bits of information and transfer the call.

 

The Diagnosis

1.     
System prompts are far too wordy.

2.     
IVR clichés such as a web ad and a Spanish option
are present.

3.     
Many prompts are clearly unnecessary.

4.     
It requires far too much time to perform a
presumably simple sorting process.

5.     
Overall production quality is unprofessional.

6.     
Production quality of the concatenated income
amount is all but embarrassing.  It uses mixed
male-female voices and obviously inappropriate
intonation.

 

The Prescription

1.     
Get rid of the wed ad and Spanish option.

2.     
Chop the Main Menu to three, succinctly stated
items.

3.     
Use professional voice talent.

4.     
Use the same voice talent for all prompts.

5.     
Eliminate or trim down any unnecessary prompts
and prompt content.

 

The Prognosis


Probably all right.

 

 

Nov 6 2008

Walt and
Walter’s VUI View: NSTAR

The Caller’s View from Walt:


I called NSTAR to obtain information on my account
balance.  NSTAR has implemented a speech-enabled
attendant.  It worked pretty well.  It asked
me what I was calling for and it recognized my saying
account information.  It then took my account
number.  This worked fine is I said each digit
individually but did not recognize it when a said it as
a natural number.  When it didn’t recognize the
account number, it defaulted to Touch-Tone right away. 
This is a much better approach than the “did you say ….”
or asking you to say it over again.   I
obtained my account balance in well under a minute. 
I had a question about it and requested to talk to a
human.  I was connected to a human in less than a
minute.  She did not have my account number
available.  NSTAR has implemented a pretty good
self-service capability.  It provides a
quick-&-easy experience to the caller.  They then
spoil it all by not passing the information to the
agent.  Whatever time is saved by having a
quick-&-easy implementation is now a lost by having to
repeat everything to the agent. 


I attempted to talk to a human after normal business
hours.  I was informed that they were closed and
that I should call back when they are open.  No voice
mail or offer to call back the next day.

Dr. Walter Rolandi examination of NSTAR Customer
Service:    

 

Patient:                                 
NSTAR Power Company

Task:                                    
Obtain current account balance
 

Toll Free Number:             
1-800-592-2000

 

The Examination Session

Just
one call was required to complete the task.  The
task was completed in a mere 42 seconds and the entire
call lasted only 83 seconds. 

 



Call Details:

The
system is a speech recognition application with an “open
ended” (“How may I help you?”) Main Menu.  The user
experience begins with the caller hearing:

 


Hello.  Thank you for calling NSTAR. 

For quality purposes, your call may be recorded.


I am an automated assistant.


How may I help you?

 

I
responded, “I need to get my current account balance”.  
After about a three second pause, the system replied:

 


Account balance and recent payments.

 

It was
unclear whether the system response was intended as a
confirmation or a question.   The intonation
of the prompt implied that it was a question but this
was far from obvious.  Confused, I said nothing. 
After another 2.5 second pause, the system continued:

 

Please hold while I look up your account.


OK. 


What account number are you calling about?

 

I then
stated the 11-digit account number I was to use to which
the system responded:

 


OK.


Your account balance is $289.79.

 

At this
point, my task was complete.  The system proceeded
however with an additional 14-plus seconds of unasked
for information about the account.  The additional
account information was followed by a legalese
disclaimer and a series of yes/no questions offering to
provide even more additional account-related information
or services, all of which I declined.  The system
then thanked me for calling, read me a web-ad, said,
“Goodbye” and hung up.

 

The Diagnosis

1.     
This system appears to be very healthy indeed.

2.     
Speech recognition was perfect.

3.     
Task completion was quick and easy.

4.     
Some system feedback was less than clear. (“OK.”)

5.     
The production quality of a number of prompts is
questionable.

6.     
Some prompts sound garbled or electronically
distorted. 

7.     
The functionality of some prompts is questionable
(“I am an automated assistant.”)

8.     
Some prompt content could be trimmed away.

9.     
Small silences could be reduced.

 

The Prescription

1.     
Clean up distorted prompts and rerecord as
necessary.

2.     
Provide more meaningful user feedback.

3.     
Eliminate the few unnecessary prompts and prompt
content.

4.     
Try to eliminate the pauses that follow user
speech.

 

The Prognosis



Excellent.

 





 

Walt and
Walter’s VUI View: Central Maine Power (CMP)

The Caller’s View from Walt:


I called CMP to obtain information on my account. 
They have a structured TouchTone self-service. 
Lots of long-winded instructions.  They tell you
about their web site and to pay attention since their
menu has changed.  I requested my account balance
via the automated service.  It took over 2 minutes
to accomplish this.  This was really tedious to
use.  After getting my account balance, I requested
to be connected to a human.  This was accomplished
quickly (less than a minute).  The agent had my
account information available and was quickly able to
answer the question that I had.  The availability
of CTI compensated for the laborious self-service. 
The only benefit of self-service is to access
information after hours when humans are not available. 
During business hours, agents appear to be readily
available and provide a much more pleasant and quicker
response than the self-service.  No point in using
the self-service during normal business hours. 


I attempted to talk to a human after normal business
hours.  I was informed that they were closed and
that I should call back when they are open.  No
voice mail or offer to call back the next day.

 

CMP

Dr. Walter Rolandi examination of Central Maine Power
Customer Service:   

Patient:                                 
Central Maine Power Company

Task:                                    
Obtain current account balance
 

Toll Free Number:             
1-800-750-4000

 

The Examination Session

One
call was required to complete the task although another
call was made to determine whether the system supported
dial through (it does).  The task required 126
seconds to complete.  The entire call lasted 145
seconds. 

 



Call Details:

The
system is a highly structured DTMF only application.  
The user experience begins with 46 seconds of preemptive
instructions, web ads and additional unrequested
information.   It is too wordy to transcribe. 
Eventually, the system instructs the caller:

 


If you’re calling from a touch-tone phone, please press one.

 

Upon
compliance, the system continues:

 


With this automated phone system, you may use several
self-service options.


For self-service options, press one.

 

Upon
compliance again, the system read out the company’s
variant of the IVR cliché:

 


Please listen carefully before selecting from one of the four
options.

 

The
first option turned out to be “check your account
balance”.  After listening to the entire menu, I
entered DTMF1.   I was then prompted for my
account number which I keyed in.  The system then
laboriously read the account number back, requiring me
to confirm it with (yet another) DTMF1 response. 
The confirmation process began with the pointless
assertion:

 


We have recorded your 13-digit account number as….

 

At this
point, I was finally read the account balance followed
by some additional unrequested account information. 
I was then told that additional account services were
available by pressing (guess what) DTMF1.  My task
complete, I hung up at this point.

 

The Diagnosis


1.   This system is sadly afflicted with many
of the telltale symptoms of poor DTMF IVR design.


2.   It is tedious and laborious to use.


3.   It is full of unnecessary prompts and
prompt content.


4.   It appears at times to be illogically
organized.


5.   It requires what appear to be unnecessary
or pointless steps.


6.   Task completion is painstakingly slow.

7.     
Overall production quality is sub par.

8.     
The confirmation strategy is irrational and
unnecessary.

 

The Prescription

1.     
Get rid of the entire 46-second preemptive
preamble.

2.     
Streamline the organization of the system in
order to make its functionality more immediately obvious
to the caller.

3.     
Eliminate any dialog state that is not absolutely
required.

4.     
Eliminate or trim down any unnecessary prompts
and prompt content.

5.     
Eliminate the confirmation strategy.

6.     
Adopt industry best practices for DTMF design.

 

The Prognosis



Reasonably good.

 

 

 

 

Sep 7
2008

Walt and
Walter’s VUI View: USCIS

The Caller’s View from Walt:


I needed to locate the nearest office of the


US Citizenship & Immigtation Service (USCIS)
.


USCIS is a government agency comprised of 15,000 federal
employees and contractors working in approximately 250
Headquarters and field offices around the world.


 


They are responsible for the administration of
immigration and naturalization adjudication functions
and establishing immigration services policies and
priorities.


 


I called the number that was listed on the

GetHuman web site
(800-375-5283).  After the
press 1 for English option and being told to “please pay
attention since the menu had changed, I ran into the
longest main menu that I’d ever encountered.  The
prompts just went on-&-on.  It used all of the keys
for options and when it was done with all of the
options, went on a long harangue about going to the
Internet to get all of the information on the options
and much more.   It even took the time to
suggest that if you did not have access to the Internet
in your home or business, then you can go to the local
public library to obtain Internet access.  At the
conclusion of the IVR session, I was asked if I wanted
to take a survey.  I took it and it proceeded to
ask about 10 questions on what I thought of the IVR
system.  The US CIS appears to be trying very hard
to provide a good deal of information to their callers
and failing miserably.  They somehow think that
using 10 words when it can be done quite well with a
single word or two, is an effective way of
communicating.  This is one of the most
unnecessarily verbose implementations that I have ever
run into.  It is so tedious that it is virtually
impossoble to use without becoming highly irritated.


 

Dr. Walter Rolandi examination of US CIS Customer
Service:     

 

Patient:                                 
US Citizenship & Immigration
Service

Task:                                    
Obtain location of nearest US
CIS office
 

Toll Free Number:             
1-800-375-5283

 

The Examination Session

Only
one call was necessary to complete the task.  The
task required 213 seconds in total or approximately
three and a half minutes. 

 



Call Details:

The
system is a highly structured DTMF only IVR.  Calls
are answered with a 20 second greeting that includes an
unnecessarily wordy and poorly thought out Spanish
option.   The male voice talent is polished,
professional and of fine production quality. 
Selecting the English language route leads to the
familiar IVR cliché, “Please listen carefully.  The
options on our Main Menu have changed”.   This
is then followed by what appears to be the longest, most
verbose Main Menu I have ever encountered. 

 


Normally, the Call Details section includes
transcription of the patient IVR dialog through the
termination of the Main Menu.  In this special
case, transcription would require more time than I have
to devote to the examination.

 

Suffice
it to say that the Main Menu includes no less than ten
(10) choices, all expressed in pointlessly verbose
language and many of which include preemptive prompt
content.   Additionally, the Main Menu includes a
very lengthy web ad.  The Main Menu ultimately
offers an option to press “*99 to take a 30 second
survey”.

 


Navigating to the end of the system’s Main Menu required
121 seconds (over two minutes).  After this arduous
journey, I entered DTMF4 to find the nearest US CIS.  
This lead to a 27 second submenu wherein the user is
required to enter DTMF1 to obtain the location
information that was previously made available as choice
four on the Main Menu.  Entering DTMF1 lead to a
request for the caller’s zip code.  Upon entering
the zip code, a nine second silence followed by a
readout of the requested information.  The office
location information sounded like a locally recorded
insert.  A female voice talent is used.

 

After
the location information was readout, the “system voice”
returns with some more verbose instructions about how to
repeat the information, return to the Main Menu and hang
up.

 

The Diagnosis

1.     
The system could serve as a textbook example on
what
not to do
when designing an IVR.

2.     
Overall production quality is good.

3.     
Spanish option is too long and poorly planned
out.

4.     
Main Menu is far too long.

5.     
Prompts are far too verbose.

6.     
Preemptive prompt content abounds.

7.     
Unexplained silence follows zip code entry.

8.     
Pace is slow.

9.     
System is unbelievably tedious and time consuming
to use.

 

The Prescription

1.     
The VUI Surgeon’s Scalpel is desperately required

2.     
Extract the Spanish option or rethink its design
and length.

3.     
Cut the number of menu choices.

4.     
Debride all prompts of unnecessary prompt
content.

5.     
Excise all preemptive content.

6.     
Investigate the silence between zip code entry
and response.

 

The Prognosis


Grim without radical surgery.

 




Contact:
Walter Rolandi
, Ph.D.,
The Voice User Interface Company, LLC
+1. 803.252.9995,

wrolandi@wrolandi.com


www.wrolandi.com

 


 

 

 


Aug 7 2008


Walt and Walter’s VUI View: AARP


The


Caller’s View from Walt:


I needed to obtain a replacement AARP membership card.
 I called the main AARP number (888-687-2277). 
After asking me if I was a member (I responded that I
was) it presented a four item menu to me.  One of
the menu items was Membership Card, so I said
“Membership Cards”.  It then asked me for my
10-digit membership number.  I responded by saying
“I don’t know it”.  It then came back and said:
“Did you mean “I don’t know it”. Please say “yes” or
“no””.  I said “yes” and it responded with: “please
hold for the next available representative”.


The inappropriate error recovery after I said “I don’t
know it”, ruined a really good call experience.  No
reason exists to obtain a confirmation that I really
said that I didn’t know my membership number.  Any
response other than me providing the number should have
caused the system to simply go ahead and transfer me to
a CSR.  Jerking the caller around with a “did you
mean?” is just wrong.


When I reached a CSR, she did not know what selection I
had made at the menu and I had to repeat that I needed
to obtain a replacement membership card.

I made
4-5 calls and each time it asked me whether I was a
member or not.  The system does capture the
CallerID but appears to do very little with it. 


 

Dr. Walter Rolandi examination of AARP Customer
Service:       

Patient:                                 
AARP Service IVR

Task:                                    
Find out membership cost

Toll Free
Number:             
888-687-2277

 

The
Examination Session

Two
calls had to be made to the system before concluding
that it apparently does not support the task. 
There is no obvious way to determine membership costs
using the system.  The two calls consumed
approximately 100 seconds. 

 



Call Details:

Calls
are answered with a 2 second audio-icon followed by
cheerful and professional sounding female voice saying: 

 


Welcome to AARP!

This call may be recorded for quality.

After
approximately a 2.5 second pause, the system states:

 


Please say yes or no.


Are you or the person you’re calling for a member of AARP?


 

Not
being a member, I answered, “No” after which the system
told me:

 


Please hold for the next available representative.


 

Not
wishing to bother “the next available representative”, I
hung up.

 

I
immediately called back.  On this call, I answered,
“Yes” to the initial question and heard the following: 

 


Main Menu


Please say if you’re calling for AARP products and services


Insurance


Membership cards


Or to check expiration date

 

I
ventured a guess as to where I might find the cost of
membership and said, “Products and services”. 
After a 3.0 second silence, the “products and services”
menu was read.  There was nothing on the menu that
suggested membership costs although its fourth (and
last) option was to say, “Something else”.  Saying,
“something else” lead to another menu listing several
other services, mostly having to do with travel
planning.

I
returned to the Main Menu and wondered about a bit more
but eventually concluded that membership cost
information was probably not available through the
system.

 

The
Diagnosis

1.     
The system seems to enjoy good health.

2.     
It is well designed and easy to navigate.

3.     
Overall good production quality is good.

4.     
Its menus are short and well land marked.

5.     
Pace is slow but this is probably by design,
given that the system primarily serves the aged.

6.     
Some silences are probably too long

7.     
Access to a human is never far away.

 

The
Prescription

1.     
Investigate the silences between prompts and trim
them to a universal length

2.     
Play audio-icon during (not before) the welcome
prompt

 

The
Prognosis



Excellent.




Contact:
Walter Rolandi
, Ph.D.,
The Voice User Interface Company, LLC
+1. 803.252.9995,

wrolandi@wrolandi.com


www.wrolandi.com

 


 

 



July 9 2008

Walt and
Walter’s VUI View: The Internal Revenue Service (IRS)

The


Caller’s View from Walt:


I needed a Form 8109-C for my quarterly tax deposits. I
called the main IRS number (800-829-1040).  After a
long introduction, I finally got a menu of four choices:
1st
was to obtain information on the rebate.  None of
the other choices mentioned obtaining tax forms, so I
selected an option that offered to provide information
on filing my taxes.  After that, the 1st
menu choice was:
To order tax forms or publications, press 1” 
Thinking that I had found it, I immediately pressed 1.
It responded with: “If you know the the form or
publication that you need, press 1, if you don’t the
form or publication, press 2.”  I responded with a
1.   The system then responded with a long
monologue about the status of the rebate payment. 
To confuse things, this time they referred to it as the
“stimulus payment”.  At this point I bailed out of
the IRS IVR and (after a 10 minute wait) was connected
to a human that took my order for the form that needed.

 

Not
being able to order a form via the IRS IVR is awful. 
It is a waste of my time as well as the time of the
agent.  It appears as though they recently added
the rebate information into the IVR and this somehow
managed to mess up the form request service. 
Beyond this, the IRS self-service is just very poorly
done.


 

Dr. Walter Rolandi examination of Internal Revenue
Service (IRS):        

Patient:                                 
Internal Revenue Service IVR

Task:                                    
Order a 1040 form

Toll Free Number:             
800-829-1040

 

The Examination Session

Three
calls were made to the system before I abandoned the
task.  There is no obvious way to order forms using
the system as I found it on the examination date. 
The three calls consumed approximately six (6) minutes
to complete. 

 

Call Details:

 

A Main
Menu (of sorts) appears after approximately 26 seconds
of the above preliminaries.  It begins:

 


If you are calling for information about the tax rebate,
press 7.


For recorded tax topics, press 1.


For questions about your refund, press 2.


For all other automated services press three.


To hear these options again, press 9.

 


Thinking “all other automated services” was the logical
choice, I entered DTMF 3.  This lead to several
DTMF beeps (apparently a line transfer), five seconds of
silence and an instruction to enter either a social
security or employer number in order “to access your
account information”.  Confused, I did nothing. 
Approximately six seconds later, the system repeated the
request.  Failing to see what my social security
number had to do with ordering a 1040 form, I hung up.

 

I
immediately called back, again experiencing everything
above but paying special attention to the Main Menu,
assuming that I had reacted to it incorrectly on the
initial call.  Again I heard:

 


If you are calling for information about the tax rebate,
press 7.


For recorded tax topics, press 1.


For questions about your refund, press 2.


For all other automated services press three.


To hear these options again, press 9.

 

Still
confused but wanting to be sure, I entered DTMF 9 to
hear the options again.  Astonishingly, the options
were not repeated.  Female #1 simply restated the
greeting, normal business offices and the web ad
followed by Female #2 doing the same but in Spanish. 
The system then promptly hung up.

 

Again, I immediately
called back and navigated through to the Main Menu. 
This time I entered DTMF 1 to see if “recorded tax
topics” could lead to information about ordering forms.  
This lead to another (apparent) transfer, a number of
preemptive prompts and, eventually, an announcement that
“there are seventeen (17) main categories for which
there are topics”.   After listening to the
first seven topics, I hung up. 

Note that it is
possible that the system provides information about form
ordering somewhere between topic eight and topic
seventeen.

 

The Diagnosis

1.     
System is deathly ill.

2.     
It exhibits repeated violations of the de
facto
DTMF standard.

3.     
Overall organization is illogical.

4.     
Jargon is present.

5.     
Spanish option is present.

6.     
Web ads are present.

7.     
Main Menu is illogically laid out.

8.     
One submenu had 17 (!) items.

9.     
DTMF entry is awkwardly (if not stupidly) used of
on 17-item submenu.

10. 
Pointless preemptive prompts are present.

11. 
Pace occasionally seemed slow.

12. 
Overall good production quality is wasted on the
senselessness of the system’s organization and function.

 

The Prescription

1.     
Completely reorganize the entire system.

2.     
Adopt de facto DTMF standard.

3.     
Eliminate jargon.

4.     
Eliminate Spanish option.

5.     
Eliminate web ads.

6.     
Eliminate preemptive prompting.

7.     
Speed up overall interactive pace.

 

The Prognosis



Without the most drastic intervention, very poor indeed.



Contact:
Walter Rolandi
, Ph.D.,
The Voice User Interface Company, LLC
+1. 803.252.9995,

wrolandi@wrolandi.com


www.wrolandi.com

 

 

 

 

 

 


June 9 2008

Walt and
Walter’s VUI View: The

Annual Credit Report Service

 

The


Caller’s View from Walt:


An amendment to the federal Fair Credit Reporting Act
requires each of the nationwide consumer reporting
companies – Equifax, Experian, and TransUnion – to
provide consumers with a free copy of their credit
report, at their request, once every 12 months.
 
I called the Annual Credit Report Service to order a
credit report.  This service is provided by the
three (3) major consumer credit reporting companies. 
I had previously ordered a credit report from this
service, so I was a repeat caller.  You can order
three (one from each of the credit reporting companies)
reports each year, so most calls are repeat calls, since
it makes sense to not obtain the three reports at the
same time.  The service is oblivious to this and
always assumes that you are calling for the first time. 
It forces you to listen to an excessively long and
unnecessary monolog.  Overstrike is ignored. 
The voice relentless continues to babble on describing
the service and what it does.  It describes in
detail all of the ways in which you can order a credit
report such as the Internet and mail.  The system
somehow thinks that the consumer that used the telephone
to request a fee credit report, needs to be told
over-&-over again what the service is, how it works and
the alternate ways that they can request a credit
report. Despite using your phone number to identify who
you are and what your address is, historical information
is not used.  With your phone number, it would be
easy to determine that you are a repeat user and that
the excessively long monolog can be by-passed.  
They do not provide any human backup so the only option
is to just suffer through the horribly implemented
self-service.  The intent of the awful
implementation appears to be to discourage consumers
from requesting a credit report.


 

Dr. Walter Rolandi examination of Annual Credit Report
Service:                                  
Annual Credit Report Service

Task:                                    
Order a report

Toll Free Number:             
877-322-8228

 

The Examination Session

Only
one call was necessary to complete the task.  The
process took just over six (6) minutes however.  It
was at times, laborious, painstaking and annoying.

 

Call Details:


Normally, specifics of the interactive dialog are
included in this section.  However, given the
length of the call, no effort to transcribe its entirety
was made.  Some specific pieces of the dialog are
reviewed.

The
system answers and the caller hears an eight-plus second
initial greeting spoken by a professional sounding
female voice talent.  It then launches into a 24
second monologue describing what the system is and what
it exists to do. 

 


Eventually, the system instructs callers calling from
their home phones to “… press or say, one.  If not,
press or say, two”.  This is the first incidence of
an unnecessarily wordy method for negotiating either-or
logic branches within the application.  
Variants of the template unfortunately are unremittingly
repeated throughout the interaction.

 

Upon
responding, the caller is treated to a 15 second “web
ad” followed by a five second prompt offering to provide
instructions on how one might submit their credit report
request in writing.   At a point soon
thereafter, one finally arrives at the dialog state
where the process of actually making a request begins. 
In my case, navigating to this state had required 99
seconds.

 

After an additional
four minutes and 25 seconds, I successfully completed
the credit report order.  The process included some
notable speech recognition failures.  Among them
were one failure to recognize the utterance, “no” in
response to a “yes/no” question and two failed attempts
to accurately recognize the spelling of my name.  

 

Overall, the
interaction was riddled with lengthy if not actually
unnecessary prompts.   Many were preemptively
instructive and many of the preemptive prompts were
needlessly repeated throughout the session. 

 

Some user inputs
seem to be confirmed unnecessarily.  The system has
a pointlessly verbose and questionably effective error
recovery strategy and, at times, the system prompts
appear to suggest that it has conversational abilities
that it clearly does not possess.

 

The Diagnosis

1.     
This system suffers from neuro-linguistic
infirmities.

2.     
It seems to take every opportunity to say
something that seldom if ever needs to be said.

3.     
It also seems to say things that could be easily
said with just a few words in far too many words.

4.     
There are many preemptive prompts.

5.     
Its time-to-task-completion is painfully slow.

6.     
Overall production quality is good.

7.     
Speech recognition should probably be tuned.

8.     
Some prompts are artificially conversational.

9.     
System response times are occasionally slow.

 

The
system’s health could be dramatically improved with some
effective neuro-linguistic programming
.

 

The Prescription

1.     
Excise all unnecessary prompts and prompt
content.

2.     
Eliminate preemptive prompting.

3.     
Streamline the time-to-task-completion.

4.     
Investigate speech recognition accuracy issues
and tune accordingly.

5.     
Tone down the suggestion of “conversational”
abilities.

6.     
Investigate response time issue.

 

The Prognosis



Very good.



Contact:
Walter Rolandi
, Ph.D.,
The Voice User Interface Company, LLC
+1. 803.252.9995,

wrolandi@wrolandi.com


www.wrolandi.com

 

 

 

 

 


May 9 2008


Walt and Walter’s VUI View: The American Airlines
Customer Support Line


The


Caller’s View from Walt:


I called American Airlines to locate a flight from
Boston to New York.  I requested “Flight and Gate
information” from a four item menu.  Even so, it
came back and confirmed that I had requested Flight and
Gate Information.  I finally nailed down the flight
that I wanted with it repeating everything that I said
and asking for confirmation of each item.  I then
attempted to get to a CSR to book the flight. 
Waited in a queue for about five minutes.  It did
not tell me the wait time or offer to call me back. 
When I finally did get to a CSR, she did not have the
information that I had provided to the flight
information system and I had to provide it to her all
over again.


American Airlines makes using the self-service an
unpleasant experience. It has long and often unnecessary
prompts and parrots virtually everything back to you. 
When you get to the CSR, you need to repeat everything
to the CSR again since nothing is passed from the
self-service to the CSR.


 

Dr. Walter Rolandi examination of American Airlines
customer support line

Patient:                                 
American Airlines

Task:                                    
Find out if service available
from one city to another.

Toll Free Number:             
800-433-7300

 

The Examination Session

The
call was answered by a directed dialog speech
recognition system.  The call was apparently
transferred to another speech recognition system and
eventually back to the original system that appeared to
answer the call upon saying, “Main Menu”.  Only one
call was required to complete the task although it did
entail two “loops” through the system.  

 


Call Details:

The
system answers the call with a pleasant female voice
talent saying:

Thanks for
calling American Airlines.  Para Español, diga,
“Español”.

These
prompts are immediately followed by:


Main Menu.


Please say reservations, flight and gate information,
Advantage services or more options.

Upon my
saying, “Reservations”, the system cheerfully responded:


Ok. Reservations.  If that’s not right, say, “Go back.”

After a
pause of almost two seconds, the system continues:


Now which one would you like? 


Fares, new reservation, existing reservation or flight and
gate information.


After clearing responding, “Flight and gate
information”, the system replied:


I think you want, “Flight and gate information”.  Is
that right?


Answering, “Yes”, the system continued with:

The next time, you can get here more quickly by saying,
“Flight and gate information” at the Main Menu.

This call may be recorded.

Next
came three seconds of silence followed by four seconds
of music-on-hold and an apparent call transfer to
another system:

Welcome to the flight information system for American
Airlines, American Eagle, American Connection and other
American marketed code share flights.

The
intonation and pace of this prompt were unnatural,
suggesting that prompt content had been electronically
edited to include or delete some information.  The
system continues:

To get you up to date on departure and arrival information,
I’m going to ask you a few questions.   First,
what’s the flight number?

A pause
of .82 seconds follows whereupon the system announces:

It’s Ok to say, “I don’t know”.

I said,
“I don’t know”, the system went silent for approximately
two seconds and then carried on:

Ok let’s find out what flight you want.  What’s the
departure city?


“Boston”, said I.

Next!  Tell me the arrival city?


“Chicago”.

Thanks!  Now is
that for arrival or departure info?


“Departure”.

And finally, around what
time does that flight depart?


“Noon”.

Lemme confirm that….

A
laborious restatement of all this information
subsequently follows after which the system located a
relevant flight.   The entire process required
two minutes and twenty-three seconds to complete.

 

The Diagnosis

1.     
This system is reasonably healthy but it is not
without some infirmities.

2.     
It suffers from slight production quality issues.

3.     
Volume, intonation and pace vary significantly
from prompt to prompt.

4.     
At times, pace is so fast that it makes
comprehension difficult. 

5.     
Interaction is artificially conversational.

6.     
At least one pause invites false starts (i.e.,
“First, what’s the flight number?” (.82 second pause) 
“It’s Ok to say, “I don’t know””.

7.     
Persona borders on the annoying.

8.     
Many prompts are unnecessarily long.

9.     
Several prompts contain useless content (i.e.,
“To get you up to date on departure and arrival
information…”)

10. 
There are some preemptive prompts and “hints”
present.

11. 
Prompts contain jargon (i.e., “American marketed
code share flights”)

12. 
Some IVR Clichés are present.

13. 
System response times seem inconsistent.

The
system’s health could be greatly improved with some
minor surgery.

 

The Prescription

1.     
Clean up production quality issues with volume,
intonation and pace.

2.     
Tone down the animation in the persona.

3.     
Re-examine pause and timeout practices.

4.     
Speed up interaction.

5.     
Eliminate IVR Clichés, jargon, preemptive prompts
and “hints”.

6.     
Trim all prompts of unnecessary content.

7.     
Investigate response time issue.

 

The Prognosis


Quite good.



Contact:
Walter Rolandi
, Ph.D.,
The Voice User Interface Company, LLC
+1. 803.252.9995,

wrolandi@wrolandi.com


www.wrolandi.com

 

 

 

 

Apr 12
2008

Walt and
Walter’s VUI View: The Tufts Healthcare Customer Support
Line

The


Caller’s View from Walt:


I needed to obtain a 1099HC from my health insurance
company to prove to the state of Massachusetts that I
had health insurance.  I called the Tufts
Healthcare customer support line.  I was greeted by
a voice that sounded identical to the Amtrak Julie
persona.  So much for unique personas.  I was
calling at 5:30 in the evening.  This turned out to
be outside of normal business hours.  It dumped a
menu at me that gave me the choice of 1) either checking
my bill; 2) obtaining information re my pharmaceutical
coverage; or 3) or filing an appeal of a denial of
services.  After the menu, which was way to
verbose, it then announced that they were closed and
that if I wanted anything else, I should call back when
they were open for business.  Imagine that. Not
even voice mail.


I called back the next day.  Got a 6 item menu for
identifying the type of caller that I was.  This
could have been eliminated by looking at my CallerID. 
Next was a menu that had claimed to have 7 choices and
then gave me 8 choices.  What I was calling about
was not one of the choices, so I chose the
“none-of-the-above”  choice.  When I finally
reached a CSR, none of this information was passed which
meant that I had to provide it all over again.  
By not having CTI implemented, Tufts Healthcare is
wasting my time as well the time of their CSR.  


 

Dr. Walter Rolandi examination of Tufts Healthcare
customer support line

Patient:                                 
Tufts Healthcare

Task:                                    
Obtain name, address and phone number of an eye care
provider

Toll Free Number:             
800-701-9000

The Examination Session

Initial
system branches off to at least one other system.  
First system is a “press or say” directed dialog IVR. 
Second system supports slightly broader speech
recognition but also a directed dialog.  
System had to be called four times in order to discover
how to complete the task.  

Call Details:

The
system answers the call with a female voice talent
saying:


Thank you for calling Tufts Health Plan Medicare
Preferred.  For quality purposes, calls may be
monitored and recorded.  Please listen carefully to
the following options as the menu has changed.

After
this greeting and IVR cliché ordeal is complete, (almost
11 seconds), one hears this monstrous 30-plus second
Main Menu:


           
If you are a provider or are calling on behalf of a
provider, press or say one.

 

If
you are a member of our HMO plan, press or say two.

 

If
you are a member of our PPO plan, press or say three.

 

If
you are a member of our Private Fees for Service Plan,
press or say four.

 

If
you are unsure of your plan type, please look in the
upper right hand corner of your ID card.

 

If
you’re interested in becoming a member of Tufts Health
Plan Medicare Preferred, press or say five.


The system then pauses a mere 1.5 seconds before
announcing:


Please
make your selection now.

I had
been prepared to say, “Two” since I was calling as an
HMO member but the “Please make you selection now.”
prompt confused me and I paused.  After
approximately 3.0 seconds, the system continued:


If
you’re calling from a rotary phone, please stay on the
line. To repeat this menu, say repeat or press star. 
To end this call, simply hang up.

Almost
an entire minute (59 seconds) has now passed since the
system answered the call.   It finally
permitted me to say, “Two”.  After another 2.0
seconds and without any additional feedback, the system
launched into a 63 (!) second submenu which began:



Please listen carefully and select one of our seven
options….


This prompt was also punctuated with the utterly useless
statement:



Please make your selection now.

It was
not immediately clear which menu choice I should
indicate in order to locate an eye care provider.  
One erroneous choice was guessed after which I hung up. 
I then called back two more times, indicating option
“three” because it said something about “Vision Care”
(?).  Both times the system responded that my call
was being transferred to “Eye Med Member Services” (?)
and, assuming that a human would soon answer the call, I
hung up.  On the fourth and last call, I did not
hang up.  After approximately 5 seconds, a line was
heard ringing.  The ringing lasted for over 9
seconds whereupon another system appeared to answer:


Welcome to Eye Med Vision Care (?).

There
are several things that I can help you with. 
Please say one of the following:


Provider locator, plan benefits, claim forms or claim
status.

Upon
saying, “Provider locator”, the system then read an 11
second prompt describing my vision plan and requesting a
zip code where to look for providers.  I spoke the
zip code, “01720” after which the system said,


Hold on while I look that up.

The
system soon came back saying that it hadn’t been able to
locate any provider within 100 miles of the zip code I
that I had provided.  What followed was a series of
nonsensical instructions that required me to return to
the Main Menu in order to say another zip code.  I
eventually muddled through, finally completing the task
in just over 5 minutes and 15 seconds!

The Diagnosis

1.     
This system is deathly ill.

2.     
The system (or systems) supports inconsistent
interactive modalities.

3.     
System attempts to automate far too many tasks.

4.     
Prompts are unbelievably long.

5.     
Preemptive prompts are abundant.

6.     
Menus have too many options and are far too
wordy.

7.     
The system is painstakingly slow to use.

8.     
Feedback is often insufficient.

9.     
Logical organization of the system is
occasionally unapparent.

10. 
Timeout timers are often inappropriate.

11. 
Speech recognition accuracy seems sub par.

12. 
A bevy of IVR Clichés is present.

13. 
System response to input seems slow.

14. 
Production quality is low-to-medium.


Performing this seemingly simple task using this system
(or systems) revealed the severity of its ailments.  
Only the most profound interventions can save this
patient.         

The Prescription

1.     
Bust up the various components into smaller, more
manageable and functionally rational systems.

2.     
Adopt a radically different design philosophy.

3.     
Employ a consistent interactive modality.

4.     
Provide meaningful feedback.

5.     
Speed up interaction.

6.     
Improve ASR accuracy.

7.     
Eliminate IVR Clichés.

8.     
Shorten menus.

9.     
Trim all prompts.

10. 
Improve production quality.

 

The Prognosis



Without fundamental and radical surgery, I am afraid
there is little hope.



Contact:
Walter Rolandi
, Ph.D.,
The Voice User Interface Company, LLC
+1. 803.252.9995,

wrolandi@wrolandi.com


www.wrolandi.com

 

 

 



Mar 7 2008

Walt and
Walter’s VUI View: The Charter Communications Customer
Support Line

 

The


Caller’s View from Walt:


 


I attempted to find out what the cost and availability
of the


Charter Communications

Triple-play package was by calling the telephone
customer support number (866-472-2200). After a brief
greeting, and the press 1 for English prompt, it asks
you to enter your phone number, which I promptly did. 
Charter Comm’s appears to ask you to provide your phone
number for no purpose other than to irritate and waste
the time of the caller.  They do nothing with the
number.  The menu offers me four options. 
They assume that I’m already a subscriber since they are
about billing, scheduling a technician appointment or
obtaining technical assistance.   They had
forced me to provide my phone number from which they
should have easily been able to determine whether or not
I was a subscriber.    I selected the 4th
menu item which was for purchasing services from Charter
Communications.  The options were a new
subscription and a pay-per-view movie.  Again,
offering me the pay-per-view option was inappropriate
for them to do for a non-subscriber.   I was
next connected to a CSR.  I asked him if he had my
phone number or knew what I had requested.  He said
that he did not.   He also indicated that he
handled all types of calls from billing to new services.


 


These people want to be your telephone company. 
Yet, they don’t support rather rudimentary telephony
functions such as CallerID or CTI.   Why they
bother to waste the caller’s time listening to a menu
and responding to it, totally escapes me.  
You are routed to the same CSR no matter what you input,
so the entire IVR process serves no purpose other than
to waste the caller’s time.


 

Dr. Walter Rolandi examination of Charter Communications
customer support line

Patient:                                  
Charter Communications

Task:                                     
Availability and price of broadband service

Toll Free
Number:    866-472-2200

 

The Examination Session

System
is a DTMF-only direct dialog IVR.  It was called
three times.   Apparently it
does not support automated broadband availability and
pricing
information.   Following the “new
service” path eventually leads to a transfer. 



 



Call Details:

The
system answers the call with a female voice talent
saying:

 

Welcome to Charter Communications.

This
Classic IVR Cliché immediately follows the greeting:


 


For quality assurance purposes, your call may be monitored or
recorded.


 

Then
comes a Spanish option:

 


To continue in English, press one.  Para continuar en
Español, oprima el numero dos.


 

Once
one’s language preference has been discerned, the system
then asks the caller to enter his “10-digit home phone
number beginning with area code now”.

After
complying and the passage of approximately three seconds
of silence, the Main Menu emerged.  Just over 22
seconds had elapsed since the call was answered.

 


For billing questions or account balance information, press
one.


If you’re calling in reference to a scheduled appointment,
press two.


If you require technical assistance, press three.


To order new service, to add additional services or to
transfer your service, press four.


To disconnect or remove services, press five.


 

The
Main Menu required over 20 seconds to be read extending
call length to over 40 seconds.  Note that all of
this time was required before being given the
opportunity to indicate why I had called. 

 

Main
Menu choice four seemed the most related to my task. 
When I entered DTMF 4, another 14-second menu followed,
the first choice of which was:

 


To order new Charter Services, press one.


 

I
entered DTMF 1 and after approximately 5 seconds of
silence, two loud DTMF beeps were heard.  Ten
seconds later, an agent named Diane came on the line.

 

 

The Diagnosis

1.     
Spanish option questionably implemented

2.     
Prompts are unnecessarily long.

3.     
Menus are too wordy.

4.     
The Main Menu could be shortened.

5.     
The system does not appear to support the task I
intended to perform.

6.     
IVR Clichés are present.

7.     
System response to input seems slow.

 

Far
from dreadfully ill, the system could be more accurately
described as being, “out of shape”.  It could be
“whipped into shape” very easily however and perhaps be
further improved with little cosmetic surgery.

 

The Prescription

1.     

2..     
Eliminate or redesign Spanish option.

3.     
Shorten Main Menu.

4.     
Trim all prompts.

5.     
Improve production quality (i.e., provide a
transfer message instead of two loud beeps).

6.     
Speed up system response time.

 

The Prognosis


Very promising.

 

 


Contact:
Walter Rolandi
, Ph.D., The Voice User
Interface Company, LLC +1. 803.252.9995,

wrolandi@wrolandi.com


www.wrolandi.com

 

 



Feb 8 2008

Walt
and Walter’s VUI View: The Washington Post Customer
Support Line


The


Caller’s View from Walt:


We called the


Washington Post

this month to see how it differed from The New York
Times and The Wall Street Journal which we tested
previously and found to be wanting. 


I attempted to find out what the cost was to subscribe
to the Washington Post by calling the telephone customer
support number (800-477-4679). After a brief greeting,
it tells you to pay attention since their menu has
changed.  You then encountered a menu where the
first choice was to start a new subscription which I
selected.   The CSR informed me that a
one-year subscription to the Washington Post would cost
me $187.20.  The on-line version would cost an
additional $9.95 per month or $119.40 per year.   


The main menu contained a number of choices.  In
addition to the new subscriber choice, it also had other
selections like


cancel subscription

or report a


damaged/missing paper.  


I tried these out and discovered that they were just
wasting my time since you went to the same CSR pool no
matter what you chose and the CSR did not know which
selection you made so you had to again inform the CSR
what you are calling about.


I decided to call back after business hours. 
Instead of the option to go to the CSR to start a new
subscription, I was instructed as follows:


 
“If
you’re calling for any other reason, please call back
during normal business hours.  Our customer service
representatives are available from 6 AM to 6 PM weekdays 
7 AM to 12PM on Saturdays or holidays and from 7AM to
2PM  Sundays.”


Wow!  Someone wanting to become a subscriber and
The Washington Post doesn’t think that they should
capture their phone number or let them leave a voice
mail message?  The tiniest business typically does
better than this.  They have voice mail at night
and will call you back the next business day.


I also called the Washington Post Advertising Hot Line. 
(800-765-7678). It greeted me with:


The
Washington Post Advertisers hot line is presently
closed.

 
Normal
business hours are 9 AM to 5:30 PM eastern time. 
Please contact us during those hours or if you have
Internet access, please visit our web site at

www.ad
site, that’s a d s I t e dot wash post 
w a s h post dot com. Thank you for calling.
” 


These folks are seeing advertising revenue that is
dropping like a rock.   Yet they don’t even
have a voice mail service where a potential advertiser
can leave a message and can be called back the next day. 
These people deserve to go out of business.  They
are working hard to earn it!

Dr.
Walter Rolandi examination of The Washington Post
customer support line

The Examination Session


Continuing our evaluation of major newspapers, this
month we examined the Washington Post.   For
the sake of consistency, we attempted to perform the
same task that we had tried when calling both the Wall
Street Journal and New York Times: we attempted to
obtain the rate for a one-year subscription to the
newspaper.

 

I
called the system at
800-477-4679.   As was the case with the both
the Wall Street Journal
and New York Times
systems,
the information I sought did not seem to
be available through the IVR.  Only one
call was necessary but several more were made however in
order to clarify a question about the main menu grammar.

Call Details:

The
system answers the call with a female voice talent and
an appropriately succinct greeting:


Welcome to the Washington Post.

The
greeting is immediately followed by the Classic IVR
Cliché:


Please listen carefully as our menus have changed.

The
Main Menu then begins:


To start a subscription, please say, subscription or press
one.


To report a damaged, incomplete or missing paper for today,
Friday, say, complaint or press two.


To temporarily stop your newspaper with a restart date, say
stop or press three.


If you’re calling to cancel your subscription, say, cancel or
press four.


For billing enquiries, say, billing or press five.


To speak with a customer service representative, say, service
or press zero.


Assuming that the “start a subscription” path would lead
to information about subscription rates, I said, “One”.  
Two and one half seconds later, the system responded
with another IVR Cliché:


Your call is being transferred to the next available
representative.

I
immediately hung up.  In order to determine whether
the system actually understood the digit, “one”, I
repeated the experiment several times saying different
numbers all of which appeared to be in the grammar.

 

The Diagnosis

1.     
Someone was clever enough to include digits in
the main menu grammar.

2.     
The main menu could be shortened.

3.     
The system does not appear to support the task I
intended to perform.

4.     
The “say or press” prompting style is ineffective
and unnecessary.

5.     
The “for X, say X” prompting style is
unimaginative and annoying.

6.     
Most prompts are wordy and unnecessarily long.

7.     
IVR Clichés are present.

8.     
Response to speech input seems slow.

 

The
system is not seriously ill but it would greatly benefit
from some cosmetic surgery.

 

The Prescription

1.     
Eliminate the IVR Clichés.

2.     
Eliminate all “or press” DTMF prompt components.

3.     
Reduce main menu choices to three (3).

4.     
Trim all prompts.

5.     
Speed up speech response time.

 

The Prognosis


Quite good.  Better than most.

 


Contact:
Walter Rolandi
, Ph.D., The Voice User
Interface Company, LLC +1. 803.252.9995,

wrolandi@wrolandi.com


www.wrolandi.com

 

 

 

 

 

 



Jan 7 2008

 

Walt and
Walter’s VUI View: The NY Times Customer Support Line

The


Caller’s View from Walt:


 


We called the NY Times this month to see how it differed
from The Wall Street Journal which we tested previously
and found to be wanting. 


 


I attempted to find out what the cost was to subscribe
to the NY Times.   After a brief greeting, you
are confronted with a horrendous menu.  This is a
Touch-Tone system that is pretty poorly done.  Way
to many choices to deal with.   I provided all
sorts of information about what I was calling about. 
I assumed that this information was being used to route
me to the appropriate customer service representative
(CSR).  I quickly found out that this was just
wasting my time since none of the information that I
provided to the IVR was passed to the CSR.  I had
to repeat everything all over again.  No
consistency.  The eight key is used for the repeat
function at the main menu but at other menus it is
identified as an invalid key.   The NY Times
IVR is a pure Touch-Tone service that looks like it was
done a decade ago and has been torturing callers for
that long. 


 


When I followed the instructions to press 1 for a new
subscription, it repeated the instruction all over again
twice.   The 2nd time it admonished
me to have my credit card ready.  I had to again
press 1 for a new subscription.  All of this was a
waste, since I had to repeat everything all over again
to the CSR This service is pretty brain dead.  Hard
to imagine that a major newspaper has their new order
processing operation as messed up as this.  This
appears to be a death wish.


 


The CSR finally told me what the cost to subscribe was. 
They have a relatively complex pricing structure that is
dependent on how you are going to pay.  They also
offer a lower rate for new subscribers. Although you
could argue that this is unfair to existing customers,
it wasn’t as offensive as what the WSJ was doing, which
I thought was simply dishonest.


 


One positive aspect of the NY Times customer service was
that was available 7 days a week and late at night and
early in the morning.  The wait to get to a CSR was
pretty short.  This was in marked contrast to the
service that was provided by The WSJ where it was only
available during working hours and instructed customers
to call back when they were open.


 

Dr. Walter Rolandi examination of the New York Times
customer support line

The Examination Session

 

The
patient today was the New York Times.  Given that
we had previously attempted (and failed) to obtain the
rate for a one-year subscription to the Wall Street
Journal, we thought it only fair to ask the same of the
New York Times’ IVR.


 

I
called the system at
1-800-NYTIMES (1-800-698-4637).  One call should
have been sufficient because, (as was the case with the
Wall Street Journal system),
the information I
sought did not seem to be available through the system. 
Two calls were made however in
order to explore what could only be described as an odd
initial user experience.



 



Call Details:

The
system answers the call with a female voice talent and
an appropriately succinct greeting:

 



Thank you for calling the New York Times.

 

The
system immediately followed the greeting with this:


 


For
verification, your house or building number is: 8 8 8


and
your apartment number is: 4 2 7.


 


If
this is correct, press one.  If not, press two.


 

I
can only imagine that the system was attempting to
identify me (the caller) using ANI.  Since I am not
a current subscriber to the New York Times and the
information the system asked me to confirm had nothing
to do with me, this all seemed very strange and clumsy.  
(Note that on the first call, I entered DTMF1 and, on
the second call, I entered DTMF2.   The
consequences were apparently identical).



A main menu followed:


To
order a new subscription, press one.


To
report a missed paper, a damaged paper or any other
delivery problem, press two.


To
make a payment or obtain billing information on your
account, press three.


If
you are calling to suspend, discontinue or restart your
subscription, press four.


If
you are calling for any other reason, press five.


To
repeat this menu, press eight.


If
you are calling from a rotary phone, please stay on the
line for the next available customer care
representative.


 


Assuming that the “order a new subscription” path would
lead to information about subscription rates, I entered
DTMF1.  The system responded with:

 


To
order a new subscription, press one.



Please have your credit card ready and press one.


For
other customer service assistance, press two.


To
return to the main menu, press nine.


 


Pressing DTMF1 (again) lead to some stereotypical IVR
clichés regarding an imminent transfer to a “customer
care representative”.  I hung up during
music-on-hold.

 

The Diagnosis

1.     
In addition to its physical infirmities, the
system suffers from mental disorders.

2.     
Initial system behavior appears odd.

3.     
Several prompts are clearly irrational.

4.     
There are to many choices on the main menu.

5.     
The system does not appear to support the task I
intended to perform.

6.     
Most prompts are wordy and unnecessarily long.

7.     
Many prompts contain pointless preemptive
content.

8.     
At least one DTMF key use practice violates an
industry practice.

 

The
system does not appear to be very rationally designed. 
With all of its “ors and ands”, the main menu contains a
cognitively overwhelming eleven items.  Upon
electing “to order a new subscription” by pressing
DTMF1, the system immediately requires the user to press
DTMF1 again.   The DTMF9 key is generally
regarded as the
de facto
standard disconnect or “end call” key, not
the “return to main menu” key which is typically “star
star”.
 

 

The Prescription

1.     

Investigate odd initial prompt sequence phenomenon and
adjust accordingly.

2.     
Rethink
all prompts in terms of their rationality.

3.     
Reduce
main menu choices to three (3).

4.     
Trim or
remove any unnecessary prompts.

5.     
Remove
all preemptive prompts or prompt content.

6.     
Support
de facto DTMF standards.

7.     
Speed up
the time to task completion.

 

The Prognosis


Reasonable with thorough adherence to all prescribed
treatments.

 


Contact:
Walter Rolandi
, Ph.D., The Voice User
Interface Company, LLC +1. 803.252.9995,

wrolandi@wrolandi.com


www.wrolandi.com


 

 

 

 

 

Nov
8 2007


Walt and Walter’s VUI View: The Wall Street Journal
(WSJ) Customer Support Line

The


Caller’s View from Walt:


My subscription to the WSJ was expiring and needed to be
renewed.  I received a notification via the USPS
from the WSJ reminding me of this.  The notice
indicated that a renewal for one year would cost $219. 
They indicated how good a deal this was by showing that
it was a $240 saving from the newsstand price.  It
was also identified as “special rate” that would expire
in two weeks.  I also had a subscription to the
on-line version of the WSJ.   Since nothing
about the on-line subscription was mentioned on the
renewal form, I decided to call.  I called at
7:30AM.  Got a long greeting that went on-&-on and
said nothing that a simple greeting that confirmed that
I’d reached the WSJ would have done.  It finally
stopped babbling away and asked me what I wanted. 
It mentioned that it was speech enabled and that I
should say what I wanted.  I said “subscription
renewal”.  It asked me to repeat this a couple of
times.  Finally it said that they could not locate
anyone whose name was “Subscription”.  I then got
instructions re what things I could ask for.  It
listed 18 items that included different magazines,
accounting, investor relations, etc. and finally the
last one was: “Subscriptions and Customer Service”  
Aha!  Now I knew the magic thing to say. 
“Subscription Renewal”  or “Subscription” wasn’t
good enough.  It had to be: “Subscriptions and
Customer Service”.  At that point, I became
enthused that my ordeal was nearly over and I said the
magic phrase.  It recognized what I had said and
spoke it back to.  Next it asked me which
publication I was calling about.  I said that it
was the WSJ and it again parroted this back to me. 
You really feel like telling it to stop repeating
everything that I say back to me because it is becoming
annoying.  Little point in doing this since it told
me which publications that I could request (either The
WSJ or Baron’s) and the system should have no difficulty
in determining which one was said. After all of this
hassle, it then announced that the customer service and
subscriptions department was closed and that I should
call back during normal business hours.  It does
offer an IVR option, which is a Touch-Tone one and the
speech capability is no longer available.  No offer
to call me back.  No voice mail.  No capturing
of my CallerID.   Good grief!  This is
that bastion of business that editorializes about proper
business practices.  A newspaper subscriber that is
calling to renew their subscription and the WSJ tells
them to call back when it is more convenient for them. 
To make it even worse, it wastes a lot of my time before
finally informing me that they were closed. 


 


I called back during normal business hours and went
through the entire process again.  It forced me to
identify that I was calling about the WSJ (not Baron’s)
and that it was it was the print version and not the
on-line version.  The option of both the print and
on-line (which is what I had) didn’t exist, so I
selected the print option.  It again confirmed what
I’d said by parroting it back to me and then transferred
me to a customer service representative (CSR).  The
first thing that the CSR did was ask me just what I was
calling about.  All of the time wasted in telling
the IVR system what I was calling about was for naught. 
Need to start all over again.  After telling the
CSR that I wanted to renew my subscription, I was
informed that I should really go to the WSJ web site
since it would only cost me $79.00.  Wow!  The
newspaper industry is struggling with competition for
advertising revenue from the Internet.  They seem
to be going out of their way to accelerate their demise. 
The WSJ renewal process appears to be designed to screw
it’s historical subscriber base.  Loyal customers
are tricked into paying almost triple what a new
subscriber would pay.    


 

Dr. Walter Rolandi examination of the Wall Street
Journal customer support line

The Examination Session

 

Our
patient this month was the Wall Street Journal.  My
examination task was to call the system to obtain the
rate for a one-year subscription.  The system can
be reached at 1-212-416-2000. 
Only one
call was made because the information I
was asked to obtain did not seem to be available through
this system.  Please note that this may be related
to the fact that the call was made on a weekend.



 



Call Details:

The
system answers the call with a female voice talent who
greets the caller to a 13 second introduction to Dow
Jones’ “speech enabled automated operator”.  After
a 1.5 second pause, the system proceeds:

 


If you know the first and last name of the Dow Jones employee
you wish to speak with say, “Employee directory”.


 


Otherwise, please ask for a Dow Jones department, publication
or service.  Some of your choices are: The Wall
Street Journal; Barrons; the Online Journal; Dow Jones
Newswires; subscriptions and customer service or
advertising.


 


For a complete listing of departments and services say, “What
can I say?”  For immediate live assistance say,
“Operator” or press zero.


 

After
considering the many choices, I said, “Wall Street
Journal”.  The system then replied:


 


The Wall Street Journal!


Which service or department are you looking for?  You
can ask for any one of the same categories that appear
in the printed version of the Wall Street Journal. 
Some of your options are: news department, advertising,
subscriptions and customer service.  For an entire
list of available departments and services say, “What
can I say?”.  For immediate live assistance say,
“Operator” or press zero.


 

The
interaction thus far had required just over 82 seconds. 
When I responded, “Subscriptions”, the system then said
using an apparently different voice talent:

 

Thank you!  Transferring your call to: Wall Street
Journal subscriptions and customer service.

 

After
over 9 seconds of silence, I heard a telephone ring
followed by:

 

Thank you for calling Dow Jones, publisher of the print and
online editions of the Wall Street Journal and Barrons.

Our customer service office is closed at this time.  For
tips on managing your print subscriptions on the web, or
through our automated system, please press one. 
For information on your subscription to W S J dot com,
or Barrons dot com, please press two.

 

The
system paused for approximately 2.0 seconds of silence
and then repeated:

 

Thank you for calling Dow Jones, publisher of the print and
online editions of the Wall Street Journal and Barrons.

Our customer service office is closed at this time.  For
tips on managing your print subscriptions on the web, or
through our automated system, please press one. 
For information on your subscription to W S J dot com,
or Barrons dot com, please press two.

Hearing
nothing about subscription rates, I saw no point in
continuing and I hung up.

 

The Diagnosis

1.     
The system apparently does not support the task I
intended to perform.

2.     
Prompts are wordy and often confusing.

3.     
Prompts and messages seem to be pointlessly
repeated throughout the interaction.

4.     
The System seems to mix DTMF-only modules in with
speech modules.  This may be due to a transfer to
another, independent system however.

5.     
Overall production quality is below average.

6.     
Production quality varies between the two voice
talent prompt sets.

7.     
Almost all system prompts are too long or
unnecessary.

8.     
At least one long silence was observed.

9.     
While purporting to be a speech system, (i.e., a
“speech enabled automated operator”) the system scarcely
permits the user to speak.

10. 
Users must hear numerous and drawn out preemptive
prompts.

 

The Prescription

1.     
Trim or remove any unnecessary prompts and remove
all preemptive prompts.

2.     
Create prompts that more clearly communicate user
choices.

3.     
Eliminate silences.

4.     
Better integrate DTMF and speech modalities.

5.     
Improve production quality and voice prompt
recordings.

6.     
Dramatically speed up the interactive pace.

 

The Prognosis



Average.

 



Contact:
Walter Rolandi
, Ph.D.,
The Voice User Interface Company, LLC
+1. 803.252.9995,

wrolandi@wrolandi.com


www.wrolandi.com

 

 


Sept 10 2007


Walt and Walter’s VUI View: Social
Security Administration


The Caller’s View from Walt:


This system really makes it difficult to
get to an agent.  It tells you to go to the web
site, use the IVR, ignores you, hangs up on you, tells
you to call back when it is less busy.  Much to
verbose and often forces you to listen to long prompts. 
Not at all consistent.   Sometimes T-T
overstrike works and other times it is ignored. Prior to
getting to an agent, I entered my SSN, DOB, Mother’s
Maiden name, and place of birth.  The agent had
none of this information and I had to provide it all
again.  She pretended that it was a security issue,
but finally admitted that she did not have any of the
information that I had entered to the IVR.  The
systems forces you to tell it that English is your
language.  Tells you to go to the web site. 
This is a long and tedious implementation that wastes
the caller’s time.  Not at all quick-&-easy.

Dr. Walter Rolandi examination of the
Social Security Administration customer support line
The Examination Session

The patient this month was a Social
Security Administration IVR.  My examination task
was to call the system to obtain the location of the
nearest Social Security field office.  The system
can be reached at 1-800-772-1213. 
Only one
call was required to accomplish the task although
one more call was made to further explore the “main
menu”.


Call Details:

The
system answers the call using a female voice talent with
a stereotypical greeting, followed by an awkward (almost
2.0 second) silence and some of the “usual IVR cliché
suspects”:


Thank
you for calling Social Security.



(silence)


To
continue in English, press one now or say, English.


 


When I said, “English”, the system
replied:


You’ve
reached our automated services.  If you’d prefer to
speak with an agent, please call this number between
7:00 AM and 7:00 PM, Monday through Friday.


Many services are also available on our
Internet site at double-u double-u double-u dot Social
Security dot G O V.


All this required just over 32 seconds. 
The system then says:


 


Please tell me briefly the reason for
your call.


As I opened my mouth to speak, the system
continued:   


For example, you could say, “I need to
change my address.”


I then said, “I want to find the address
of the closest office”.  After over 3.0 seconds of
silence, the system said:   

 


Sure. 
I can help you find information about the Social
Security field office for your location.


What’s
your five-digit zip code?


 

I replied, “29209” and after another 3.0
seconds of silence, the system continued:

 

 


OK. 
I found it.  You may want to write this down.


And if
you don’t get it all the first time, you can hear it as
many times as you need.


 


For zip
code: 2 9 2 0 9….


 

At this
point, another female voice begins to read 31 seconds of
information about the nearest office.  These
prompts were approximately two times louder than the
prompts played up to this point.  After all of the
information is read out, the less vocal voice talent
returns to say:


 


That’s all of the information I have for
that office.


Note that some offices don’t list their
phone numbers or mailing address or their hours.


Should I repeat that, look up a different
zip code, or, return to the main menu?


I said, “Main menu”.  After four
seconds of silence, the system said:


 


Main
Menu!


If I can
help you with anything else today, just say briefly,
what it is.


If
you’re done, just hang up.


 

Having
completed my task, I hung up.  I called the system
again however, this time saying, “I want to get an
address” at the main menu.  The system launched a
30-plus second information stream about “how to change
your address”.  I could not make it stop or go
back.  Finally, it asked me a yes-no question to
which I replied, “Main menu” which, thankfully, enabled
my escape from the “address change” ordeal.

 

The Diagnosis

1.     


The
system performed the initial task reasonably well but
the second call was an IVR disaster.

2.     


Several
IVR clichés are present.

3.     



Preemptive prompt content is present.

4.     


Strange
and awkward silences follow speech input.

5.     



Production quality is lacking between the two voice
talent prompt sets.

6.     


The
system is only slightly inappropriately anthropomorphic.

7.     


Many
prompts would benefit from trimming.

8.     


Several
prompts are unnecessary and should be eliminated.

9.     



Injudicious use of barge-in prevents user control of the
interaction.

 

The Prescription

1.     



Eliminate IVR clichés.

2.     


Trim or
remove any unnecessary prompts and remove all preemptive
prompts.

3.     


Tune
ASR end pointing to eliminate silences.

4.     



Normalize all voice talent prompts to ensure constant
loudness level.

5.     


Rethink
barge-in policy.

6.     



Incorporate some sort of “cancel”, “stop” or “go back”
function.

7.     


Reduce
time-to-task-completion requirement.

8.     



Consider eliminating the SLM “main menu”.

 

The Prognosis


Reasonably good.

 



Contact:
Walter Rolandi
, Ph.D.,
The Voice User Interface Company, LLC
+1. 803.252.9995,

wrolandi@wrolandi.com


www.wrolandi.com

 


 

 

June 7
2007


Walt &
Walter’s VUI View: on the Dell Customer Support Line
The Caller’s View from Walt

I received a brochure from Dell that was
directed at small and medium businesses.  I called
the number that was shown on the front page of the
brochure (1-877-781-3355).  I encountered a
greeting and menu which attempted to categorize my call
by obtaining additional information from me.  It
also asked me if I was agreeable to participating in a
post-call survey.   After going through all of
this, it then announced to me that they were closed and
that I should call back during “normal” business hours. 
The time was 8:30AM.  It hung up on me without
providing the opportunity for me to leave a message or a
callback phone number.   I followed up with
other calls during “normal” business hours.  At the
Dell web site, they list twelve (12) different numbers
to call.  They categorize them by function (make a
purchase, check order status, etc) or by who you are
(small business, federal government, higher education,
etc.),  This seems nice except that when you call
any of these numbers you find that they seem to be
something that someone dreamt up and forgot to
implement.  CTI is missing.  This means that
all of this categorization and information gathering is
just wasting the caller’s time, since you start all over
again with the agent that you finally reach.


Dr. Walter Rolandi examination of the
Dell customer support line
The Examination Sessions

In today’s examination, my task was to
call a Dell Computer toll free number and get
information about buying a laptop computer.  The
system can be reached at 1-877-781-DELL
(1-877-781-3355).
   While only one call
was necessary to accomplish the task, four additional
calls were made in order to explore other system
features.  The successful “laptop computer
information” call required 138 seconds to get to an
appropriate sales agent.



Call Details:

When the
system answers, the caller immediately hears an
acceptably brief welcome message and an abbreviated
“your call may be recorded” message.  The Main Menu
immediately follows:
 


To dial
an extension, please press one.
To make a purchase, get pricing information or complete
an online order, press two.
For technical support, or for software and technical
questions, press three.
For all other issues related to a purchase already made,
press four.
To repeat a menu at any time, press seven.

All this
requires just over 22 seconds.   When I
Pressed DTMF2, I heard:


Changes
to orders or order status, press one.
Purchases for businesses with fewer than 400 employees,
press two.
Home purchases, three.
For all others, four.

I pressed
DTMF3 and then heard the following 29 second prompt set:


This
voice activated system allows you to speak your response
rather than pressing buttons on the phone.  Feel
free to interrupt me at anytime when you know what to
say.

To get you to the right sales agent please
choose a category that best describes what you’d like to
buy.  You can say the one you want anytime:


Computer
systems
Ink and toner
TV
Electronics and Accessories 
Spare parts
Software or…
None of these.

I
answered, “Computer systems” and after three seconds of
silence I heard:

Are you interested in:


A
desktop, Notebook or an XPS system?

Another
three seconds of silence followed my answer, “Notebook”
and then,


Thanks!
One last thing before I get you to an agent…
Would you like to see if you qualify for financing using
our automated system?

After I
said, “No”, I heard another 29-second message:

Ok.  I’ll connect you to the next
available notebook computer sales agent.  One
moment please.


As part
of Dell’s continuing effort to improve your customer
experience, we would like to invite you to participate
in our random Customer Satisfaction Survey after you
have spoken to one of our representatives.  
Taking the survey will not effect your position in or
call queue.  If you would like to participate in
this survey, please press one now.  If you do not
wish to participate, please press two now.

I spoke
the word, “No” to which the system replied:

Thank you.  We will answer your call
shortly. 

An
appropriate sales associate answered after approximately
11 seconds of beeps, bops, phone rings and music on
hold.


The Diagnosis

1.     

The system mixes DTMF and speech
modalities somewhat unpredictably.

2.     

The system is not inappropriately
anthropomorphic.

3.     

No particular persona excesses
were evident. 

4.     

At least one menu contains too
many options.

5.     

While some prompt trimming is
evident, most prompts are far too long.

6.     

Many prompts are unnecessary and
should be eliminated.

7.     

All of my calls entailed the
“random” invitation to take the Customer Satisfaction
Survey.

8.     

All of my calls entailed an
invitation to apply for financing, even a call in which
I inquired about “spare parts”.

9.     

There is some evidence of
non-standard DTMF usage. (DTMF7 for “repeat”).

10.

Turn-taking pace in speech
modality (3.0 second ASR response) seems slow.


The Prescription


1.
     




Universally support
DTMF, speech of both.


2.
     




Adopt established
industry best practices for the support of DTMF.


3.
     




Reorganize menus
limiting their offerings to three items.


4.
     




Speed up turn taking.


5.
     




Eliminate “financing
invitation” and “random survey invitation”.  Allow
CSR to offer these options to the caller.


6.
     




Further trim all
unnecessary prompt content.


Outpatient surgery would vastly improve
the usability of this system, at least for callers
seeking sales-type information.  Eliminating the
“invitations” alone would probably cut 50 seconds off of
this task.


The Prognosis


Very good indeed.

 



Contact:
Walter Rolandi
, Ph.D.,
The Voice User Interface Company, LLC
+1. 803.252.9995,

wrolandi@wrolandi.com


www.wrolandi.com

 


May 10 2007


Walt & Walter’s VUI View: on the Virgin
Mobile Customer Support Line


The Caller’s View from Walt

This is
the one of the few test/evaluations that I’ve been
involved in where I am not an actual subscriber or
customer.  The Virgin Mobile VUI (1-888-322-1122)
has been touted as an example of a persona
implementation that is targeted at a specific
demographic and quite appropriate for that demographic. 
The original developer (Nuance) has hyped the Virgin
Mobile implementation on a number of occasions.  In
fact, the Virgin Mobile VUI implementation was a
finalist in the the Best Practices competition and
received a “Best Premium Service” award from Nuance  
Since text-messaging is so popular with the user
community that Virgin Mobile targets, I was bit
surprised that Virgin Mobile would even bother to offer
a telephone speech system for their subscribers. 
Why not provide a text-messaging support capability so
that the users can use the communication mechanism that
they prefer?  I asked the Virgin Mobile marketing
people about this and they said that they were working
on this and hoped to have this soon. The VUI is
certainly most irritating and time wasting.  I’m
not part of the Virgin Mobile target demographic, but
neither were the folks that implemented it and crow
about how appropriate it is.  The persona (Simone)
that Virgin Mobile used was that of a young inner city
girl with extremely poor language skills.  My
teenage grandchildren don’t speak like this at all and
I’ve not run into many other teenagers that do.  I
would attribute the use of a voice such as Simones to
nothing more than a disconnect between what Virgin
Mobile marketing views as their target customer and what
they are really like.  I simply can’t imagine that
a reputable enterprise would even consider using Simone
as a spokesperson that would present an appropriate
public image.  Even if you find the voice amusing
at first, I would doubt that it is still as amusing when
you deal with Simone multiple times.  Aside from
the Simone persona, this VUI implementation violated
most of the gethuman standards.  In fact, the only
gethuman standard that it passed was that it did provide
DTMF backup to speech, which is what the vuids folks
were objecting to.  Imagine that – if the Virgin
Mobile implementers followed the guidance of the vuids
persona folks, they would have been able to achieve a
zero gethuman score.   The Virgin Mobile
agents failed the “decent  communication ability”
test, even though it seemed as though they were based in
the U.S.A.   They seemed to have an attitude. 
When they made you repeat information that you had
previously provided to the IVR, no remorse was
expressed.  The attitude seemed to be: “tough
baby!”  This was even more infuriating since the VM
IVR forced you provide information before it would
connect you to the agent. Virgin Mobile should
accelerate the implementation of their text-message
customer support service and then shut Simone down. 
The teenagers have a lot of reasons for using
text-mesaging rather than voice. I would add the
experience that they had with Simone, as another good
reason to prefer text-messaging to speech.  

 


Dr. Walter Rolandi examination of the
Virgin Mobile customer support line


The Examination Sessions


In today’s examination, my task was to
call a Virgin Mobile toll free number and enquire about
opening a new account.  The system can be reached
at 1-888-322-1122.  
The number is advertised on the company’s website on a
“Contact Us” page
(http://www.virginmobileusa.com/about/contact.do).  
One example of the text that describes the number
follows:


 

“If you
have questions or comments about Virgin Mobile, or the
services and products we offer, please contact our
corporate office. Dial * 86 or 1-888-322-1122 from your
Virgin Mobile phone if you need to reach us right away!”

 


Presumably one of the questions a person
would call about is how to initiate a service contract
or get information about a new account.   Be
that as it may, it required two calls to the system to
determine that it does not obviously support such
enquiries.   The system apparently hung up on
me after one timeout and two speech recognition errors
on the first call.  It required 91 seconds.  
On the second call, I was able to transfer to a CSR
after 136 seconds of confused interaction with the IVR. 


 


First Call Details:


When the system answers, the caller hears
approximately 3.5 seconds of silence followed by what
appears to be a truncated musical audicon during which a
man says, “Virgin Mobile at your service”.  Due to
the volume of the accompanying music, I had to listen to
the recording three times to determine what he was
saying. 


 


A Spanish option follows approximately 2
seconds of silence:

For
English, say English.  Or press one.

Para
Español, diga Español.  O marque dos.


I said nothing and after 4.6 seconds of
silence, the system announced:

Hey! 
Wassup?  I’m Simone and this is Virgin Mobile at
your service.

(.6 second
pause)

Whadya
wanna do?  

Say,
Add money or top up or press one.

Get my
account balance or press two.


Activate my phone or press three.

Change
my offer or press four.

Or for
other stuff say, More options or press five.

(.6 second
pause)

You
can tell me the answer just interrupt me as soon as you
know what to say.


Not hearing anything that appeared to be
related to opening an account, I was confused and said
nothing.   After 8 seconds of silence, the
system said:


Sorry. 
What was that?


Say, Add
money or top up or press one.

Get my
account balance or press two.


Activate my phone or press three.

Change
my offer or press four.

Or for
other stuff say, More options or press five.


Still confused, I decided to venture a
guess, hoping that new account information might be
under “More options”.  I said, “Press five”. 
After 2.5 seconds of silence, the system responded,

Sorry,
I’m still not getting that.

If you
wanna add money or top up your account press one.

To
hear your account…


I barged in saying, “Five’.  After
approximately 2 seconds of silence, the system then
said,


Thanks
for calling Virgin Mobile.  Talk to you later.


Not sure what had just happened, I waited
through 8 more seconds of silence at which time a busy
signal became audible.  The system had apparently
hung up on me.


 


Second Call Details:


Again, the system answered the call with
3.5 seconds of silence followed by the truncated musical
audicon.  Wanting to speed things up and quickly
navigate to the “More options” path, I entered DTMF1
(for English) during the audicon.  The system
ignored the input.  As the system then said,


For
English,


I entered DTMF1 again.   The
system responded, suppressing the rest of the Spanish
option prompts.  After approximately 3 seconds of
silence, it then began the main menu,

Hey! 
Wassup?  I’m Simone and this is Virgin Mobile at
your service.

(.6 second
pause)

Whadya
wanna do?  

Say,
Add money or top up or press one.

Get my
account balance or press two.


Activate my phone or press three.

Change
my offer or press four.

Or for
other stuff say, More options or press five.

(.6 second
pause)

You
can tell me the answer just interrupt me as soon as you
know what to say.


Before I could enunciate “More options”,
the system said,


Sorry. 
What was that?


Say, Add
money or top up…


It had apparently responded to background
noise.   I entered DTMF5 to dial-through the
rest of the menu.   After 2 seconds of
silence, the system responded,


Here’s
more.


You can
say, Transfer a phone number or press one.


Buy a
new phone or press two.


Hear
about Virgin XL or press three.


Ask a
question or press four.


Or say,
Start over or press five.


Which
did you want to do?


Still not hearing anything that was
obviously related to opening an account, I responded,
“Start over”.   After 3 seconds of silence,
the system repeated the now familiar and completely
stale menu:

Whadya
wanna do?  

Say,
Add money or top up or press one.

Get my
account balance or press two.


Activate my phone or press three.

Change
my offer or press four…


I barged in saying, “Change my offer”,
thinking that it might lead to new account information. 
After 3 seconds of silence, the system then said,


So,
what’s your Virgin Mobile phone number?


Just
press or say the number starting with the area code or
say, I don’t have one.


Given that my intention was to find out
how to obtain a Virgin Mobile phone number, I said, “I
don’t have one”.   Another 3 seconds of
silence followed after which the system said,


Ok, so I
can get your call to the right live advisor, tell me why
you’re calling today.


Say,
Returns or exchanges or press one.


Activate
your phone or press two.


Problems
making or receiving calls or press three.



Disconnect service or press four.


Or say,
It’s something else or press five.


When I said, “It’s something else”, the
system then said,


Let me
get you to a live advisor, hold on.


They
might ask you to repeat some of the information you gave
me, OK?


Oh and
just so you know, your call may be recorded for quality
assurance.


I then heard two loud beeps followed by 7
seconds of loud music.  A very pleasant CSR
(Christina) soon answered and to whom I explained what I
had been trying to do.  I asked her if new account
information was available through the system she
answered, “yes” indicating that I should have taken the
“Activate your phone” path.   I pointed out
that the choice was far from obvious and she agreed and
apologized.  She went on to explain that in order
to get new account information, one must talk with a
live agent and choosing the “Activate your phone” option
would take the caller to an agent.

 


The Diagnosis

1.     
The system is indeed
gravely ill.

2.     
This appears to be a case
where presumed entertainment value came before simple
and basic functionality in design.

3.     
The system is egregiously
anthropomorphic.

4.     
It has one of the most
inappropriate personas I have ever examined.

5.     
The menu choices are
couched in slang (“add money”, “top up”, “change my
offer”)

6.     
Menu choices are unclear. 
Are “add money” and “top up” the same or different
things?

7.     
Menu choices illogically
include “or press <N>” construction.

8.     
Menu grammars do not permit
users to speak the DTMF numbers.

9.     
Some menus contain too many
options.

10.
Apparently the system
disconnects the call if three errors occur.

11.
Some prompts contain
preemptive instructions that pointlessly tell the user
how to respond.

12.
Essentially all system
prompts are too long, unnecessary or stylistic to no
practical end.

13.
Overall system organization
is questionable.

14.
“Change my offer” path
leads to curious consequences.

15.
The system makes it
difficult to get to a human.

16.
Turn-taking pace is
sluggish.  ASR response seems slow.

17.
There are occasional
unexplained silences.

18.
It has an annoying and
unnecessarily complicated Spanish option.

19.
It contains IVR clichés
(“…your call may be recorded for quality assurance”).

20.
Production quality is at
times questionable.

 


The Prescription

1.     
Eliminate the persona and
all anthropomorphic content.

2.     
Reorganize menus limiting
their offerings to three items.

3.     
Eliminate all slang. 
Describe menu choices in clear, concise and businesslike
terms.

4.     
Eliminate the Spanish
option.  Support a different 800 line for Spanish.

5.     
Adopt established industry
best practices for the support of DTMF.

6.     
Devise a less punitive
strategy for dealing with errors.

7.     
Speed up turn taking.

8.     
Eliminate all preemptive
prompts and all unnecessary prompt content.

9.     
Eliminate trendy audicon
and loud pre-transfer music.

10.
Eliminate or trim the ‘call
recorded for quality assurance’ cliché prompt down to
some minimalistic alternative that serves the same
(loathsome) legal purpose.

11.
Determine cause of and
eliminate unexplained silences.

Only
the most drastic interventions could save this system
but it is not beyond hope.  With major surgery, and
in particular, a lobotomy, the patient might
dramatically improve.

 


The Prognosis


The prognosis would be good if only
Virgin Mobile would realize how badly treatment is
needed.

 

 

 

 

Contact: Walter Rolandi, Ph.D.,
The Voice User Interface Company, LLC +1. 803.252.9995,

wrolandi@wrolandi.com


www.wrolandi.com

 

 

 

 


Apr

9 2007


Walt &
Walter’s VUI View: on the Verizon Broadband Support
Lines


The
Caller’s View from Walt


We had some severe power outages this
month that were caused by a faulty local transformer. 
In replacing the transformer, the power company had
difficulty finding a good ground and managed to fry my
DSL board that is provided by Verizon.  This seemed
like an opportune time to explore converting to the
Verizon Fiber Option broadband offering that Verizon is
promoting.  I called the 800-567-6789 number that
Verizon listed for broadband sales and service
inquiries. What a painfully slow system this is ! 
Huge delays after every response.  Limited
vocabulary but it still gave me the


I’m sorry, I didn’t understand that” a
number of times.
 



Long and useless prompts.  What
value in telling the caller that they are at the Main
menu ?  Had me validate my phone number two times
and when I finally got to the agent, he did not have it
and I had to provide it a third time.   The
system announced that it was going to check if the FiOS
service was available at my address. It finally came
back and announced that it was transferring me to an
agent.  I finally reached an agent that did not
have a clue what I was calling about and started all
over again to check if FiOS was available at my address. 
He finally determined that it was not available but was
not able to tell me when it would be available at my
address.  He announced that it has been available
in my town for over a year, but not at my address.  
It took 163 seconds for the system to figure out what I
was calling about.  A human could have done this in
10 seconds, tops.  Unnecessarily wasting over two
minutes of the caller’s time is terrible.  Worse –
to treat a caller that wants to buy something this way
is really bad business.   I would have called
Comcast to switch to their broadband service, but,
unfortunately, they are just as bad.  I guess that
when you are a monopoly for a hundred years, sales &
marketing skills just are not needed and developing them
is virtually impossible.  After resigning myself to
being stuck with Verizon DSL for the foreseeable future,
I called the technical support line to get assistance
setting up my new modem.  This apparently is
outsourced offshore and I had a difficult time
understanding the technician that had a heavy accent. 
The only positive thing was that he did not say


Great !


after every phrase that he uttered.

 


Dr. Walter Rolandi examination of the
Verizon Broadband customer support line


The Examination Sessions

A Verizon
Online Internet Services IVR was the patient today. 
It can be reached at 1-800-567-6789. 
My task was to call the system and obtain information
about

Verizon FiOS, Verizon’s Fiber Optic broadband service.  
In order to get information about the service, one must
provide a telephone number that is in an area where the
service is offered.  A friend of Walt Tetschner
(and Verizon customer) who lives in Virginia volunteered
his number for the examination.

 

To get
information about Verizon FiOS, one must first locate
the topic in the system’s tree and then be transferred
to a CSR who can help.   Only one call was
required to complete the task.  It took
approximately 163 seconds to get to the point of
transfer.


 

Call Details:

ed some sort of musical stimulus which lasted 0.44
seconds and then about 1.5 more seconds of silence.  
Finally the system says:

 


Starting with the area code please say or
enter your account phone number now or say ‘I don’t know
it’.  If you’re interested in ordering new service,
say, ‘New service’.

 

I
responded, providing our volunteered phone number.  
After 5 seconds of silence, the system said:

 


Lemme
make sure I got that right.



804-NNN-NNNN


Is that
correct?


 

I
answered, “Yes” and after 11.5 seconds of silence, the
system again played the 0.44 musical stimulus and then
announced:

 


You’re at
the Main Menu.  To start over at any time, say Main
Menu.  (ding audicon)


Which are
you calling about?  Tech support, your account or
orders?


 


Confused, I said nothing.  After 5
seconds of silence, a timeout message that elaborated on
the “tech support, your account or orders” options was
read.   It included:

 


…to place
an order or to check the status of an order, say
‘orders’.

 


Appearing to be my best guess, I said,
‘Orders?’  After about 3.5 seconds of silence, the
system said,

 


All
right, orders.


If you’re
calling about an existing order, say, ‘status’. 


Or, to
place a new order, say, ‘new’.


 

Again, I
ventured a guess and replied, ‘New?’  After a full
5 seconds of silence, the system asked,


 


Which
kind of new service are you interested in?


Service
or equipment?


 

Once
again mystified, I guessed, ‘Service?’  After 3
seconds of silence, the system asked,


 


So, which
service would you like to order? 


DSL,
Verizon FiOS or other?


 

Greatly
relieved, I replied, ‘FiOS” and after 3 seconds of
silence, the system explained that it needed to confirm
that the Verizon FiOS service was available in the area
of interest.  It read back the telephone I had
previously provided and asked if it was the number for
which I wanted service.  I answered, ‘yes’. 
After 3 seconds of silence, the system announced that it
was checking to see if service was available “for that
number”.  After 5 seconds of silence, it returned
saying that it had checked the line and that the call
was being transferred to a customer service agent.

 

The
sequence ended with the familiar:

 

By the way, your call may be monitored or
recorded to assure quality customer service.

after
which the call was transferred.

 

The Diagnosis


1.
     




While evincing some of
the industry’s best practices, the system is
nevertheless quite ill.


2.
     




Finding the FiOS
information requires either uncanny luck or near
clairvoyant powers of perception.


3.
     




Almost a third (50 of
163 seconds) of the time needed to stumble toward task
completion was spent listening to ‘dead air’.


4.
     




The menu choices
afforded by the system do not naturally seem to suggest
what follows.


5.
     




The system is somewhat
anthropomorphic.


6.
     




Some prompts contain
preemptive instructions that pointlessly tell the user
how to respond.


7.
     




Some system prompts
are too long or unnecessary.


8.
     




Use of audicons
appeared to be non-systematic.

 


The Prescription


1.
     




Determine the cause of
unexplained silences and reduce or eliminate them.


2.
     




Tone down artificial
attempts to simulate human conversational interaction.


3.
     




Support a different
800 line for Spanish.


4.
     




Infuse menus with more
“clues” as to where they might lead.


5.
     




Trim prompts of
preemptive and unnecessary content.


6.
     




Devise a more coherent
and informative strategy for the use of audicons or
eliminate them.


7.
     




Trim the ‘call
recorded for quality assurance’ cliché prompt down to
some minimalistic alternative that serves the same
(loathsome) legal purpose.

 


Ironically, this system is better
designed than many speech IVRs.  Yet, performing
the ostensibly simple task of obtaining FiOS information
required a series
of lucky guesses.   The pervasive,
unpredictable and unexplained silences make using the
otherwise acceptably designed system an awkward
experience.  If the cause of the silences could be
overcome, menus clarified and prompts trimmed and or
rewritten, things could get a lot better.

 

The Prognosis


Reasonably good. 

 

 

Contact:
Walter Rolandi, Ph.D., The Voice User
Interface Company, LLC +1. 803.252.9995,

wrolandi@wrolandi.com


www.wrolandi.com


 

 


Mar 8 2007


Walt &
Walter’s VUI View: on the Cingular Service Support Lines


The
Caller’s View from Walt



This month we called the Cingular Customer Service
Lines. Since Cingular is one of the largest BeVocal
customers, we thought it was timely to take a peek at
how well they have implemented their customer support
line. I’m not a Cingular customer so I could not access
the Cingular Customer Support service.  I could see
about becoming a Cingular Wireless customer, though. 
On the Cingular web site, I located the following:




Considering Cingular?


Call us for more information about
coverage, offers, cool phones and anything else you need
to know.


1888-333-6651



This looked like the appropriate number
for a potential customer to call, obtain additional
information and become a Cingular Wireless subscriber. 
So I next called this number.

Thank you for calling Cingular Wireless –
now the new AT&T.

For English press 1

(My response: 1)

For assistance with your current account,
press 1

For status on orders placed on
Cingular.com, press 2

If you would like to purchase new service
over the phone or add a line, press 3

(My response: 3)

For assistance with your existing
account, press 1

For assistance upgrading your existing
phone, press 2

To purchase new service, or add a line,
press 3

(My
response: 3)

Thank you for calling
Cingular Wireless
– now The New AT&T
Presently our center is closed. Please try your call
again later. 

These
folks are as incompetent at sales as it gets.  Firstly,
they seem to be having trouble figuring out what their
name should be.  Lets see now: when Cingular
acquired AT&T Wireless a few years ago, they got rid of
the AT&T name and replaced it with Cingular.  Now
AT&T is acquiring Cingular and has decided to go get rid
of the Cingular name and replace it with AT&T.  Or
is it

The New AT&T
These folks should just pick a name that they like, tell
the world what it is, and get on with it.  Stop the
babbling about the name.  No one cares but the marketing
people at Cingular.  I wonder what happened to all of
the AT&T signs that they took down from the stores a
couple of years ago when they changed the name from AT&T
wireless to Cingular Wireless? If you look at the menu
that they dumped on me, rather than responding to the
menu item that I selected, it came back at me with a
menu that repeated most of what was in the first menu. 
It dropped the order status item and referred to the
account as “existing” rather than “current”, but
otherwise was the same as the 1st menu. 
When I again responded with a request to “purchase new
service or add a line”, they announced their name change
again, told me that they were closed and that I should
call back when they were open.  What is wrong with
these people?  I’m a new potential customer and
they don’t  even bother to capture my callerID or
let me leave a voice mail.  Their marketing people
spend millions of $s trying to talk to me and when I
want to talk to them, their call center tells me to go
away.  Wasting my time dealing with all of these
menu choices and then telling me that they were closed
is about as offensive as you can get.

 

The
system that Cingular has to support their existing
subscribers is different than the system for dealing
with new customers.  The voice for the prospective
customer is that of a mature female while the voice for
the system that supports existing customers sounds like
that of a an adolescent female.  One system asks
the 95% of the callers that are English-speaking to
select the preferred language while the other one
requests the Spanish-speaking callers to do the
selection.  The line for potential customers also
accommodates existing subscribers that called this line
in error.  They are rewarded for calling the wrong
number by being offered a menu which directs them to the
proper customer support line.   The downside
is that you are wasting the time of a potential Cingular
customer and working hard to alienate him/her.  If
you are a subscriber that has called this number in
error, when you select the menu choice to inquire about
your existing (or current) account, you are transferred
to the customer support system that my colleague has
analyzed below.  Of course, the caller is subjected
to the Cingular-going-to- AT&T greeting all over again. 
They also decide to play the “this call may be recorded
for quality purposes” announcement which is
inappropriate.  If  the Cingular subscriber
decides to check the status of an order that they had
placed with


www.cingular.com
,
they are routed to a speech-enabled service.  This
service does not announce the name change from Cingular
Wireless to The New AT&T.  No one told them? 
Didn’t attempt to analyze this system in any detail but
noticed that it was not able to deal with a natural
number when I spoke one.  Error response was: “I
heard 01720, is that correct?’ 


 

Dr. Walter Rolandi examination of the
Cingular customer support line

The Examination Sessions

Today we examined a
Cingular Wireless customer support line.  It can be
reached at
800-331-0500. 
 My task was to call the
system and determine how many minutes had been used to
date.   I made several calls to explore the
system although only one call was necessary to complete
my task.


Call Details:

When the DTMF-only
system answers, the caller hears a brief greeting which
includes a marketing plug for “the new AT&T”, followed
by a Spanish option which is available via the “star”
key.  

 

Thank you for calling Cingular Wireless,
now the new AT&T.

Oprima el asterisco para informacion en
Espanol.

 

(2-second pause)

 

Please enter the wireless phone number
you are calling about starting with the area code. 
For information on Cingular products and services or for
help with a recent order, press one for sales.

 

(entered phone number)

 

Please hold while we access your account
information.

 

(4-second pause)

 

Main Menu:

To pay your bill or get other bill
related options, press one.

To check your minutes, press two.

To get help with voicemail, press three.

To report a lost or stolen phone, press
four.

For sales, press five.

To get help with issues not covered by
our automated options and to speak with a customer
service professional, press zero.

To repeat these options, press star.

 

The Main Menu was
read in approximately 22 seconds.  When I then
pressed DTMF 2, the system asked me to enter the billing
zip code for “security”.   What immediately
followed was a barrage of information about various
kinds of minute categories, how much had been used and
how much remained.  It lasted for 66 seconds. 
I completed the entire task in 131 seconds.

The Diagnosis



1.
     





This is a straightforward and reasonably healthy system.



2.
     





It is fairly benign in its
support of DTMF dial through.



3.
     





The system is however afflicted with a few annoying and
superficial illnesses, similar in analogy to “rashes”.



4.
     





It proffers a “sales” opportunity immediately upon
prompting for the customer’s cell phone number.



5.
     





It stupidly responds to user error with a prompt that
laments, “we did not recognize your response”.



6.
     





Many system prompts are too long or unnecessary.



7.
     





System seems to try to compensate for long prompts by
reading them out very quickly.


It is possible to get to a CSR quickly
although Cingular does seem to try to identify the
customer account first.   The system is easy
enough to use and supports fairly rapid task completion.

The Prescription



1.
     





Replace pointless DTMF error response prompts with
something helpful.



2.
     





Support a private line for Spanish speakers.



3.
     



Trim all prompts and eliminate initial
sales pitch.



4.
     



Slow down prompt presentation.


This system could be put into tip-top
shape by simply following the above prescription.  

The Prognosis


Excellent. 


 



Contact:
Walter Rolandi
, Ph.D.,
The Voice User Interface Company, LLC
+1. 803.252.9995,

wrolandi@wrolandi.com


www.wrolandi.com

 

 

 

 



Feb 5 2007


Walt
& Walter’s VUI View: on the Apple Service Support Line


The
Caller’s View from Walt

This
month we called the
Apple
Service
S
upport
line. I was greeted with a 4 item T-T menu that included
a technical support item and a selection that would let
me locate an authorized Apple dealer in my geography. 
I tried the support item which was the first one on the
menu.  Got transferred to another system which was
speech enabled.  When I was asked what Apple
product I was calling about I said “iPod Nano”.  I
was a bit surprised that it responded by saying: “I
think you said iPod Nano.  Is this correct? 
Please say yes or no”.  I thought that I had said
it clearly.  I hung up and tried again.  It
responded in the same fashion.  What is wrong with
these people?  Why are they wasting my time like
this?  They seem to be going through an irritating
and time wasting confirmation process for everything
that is said by the caller.  After dealing with the
awful speech recognition experience, I connected to a
human.  Getting to the human was a difficult
process.  It first refused to recognize my agent
request upfront and dumped me to the speech-enabled
self-service.  I had to go two-out-of-three with it
before it reluctantly transferred me to a human. 
The human did not have any of the information about my
experience with the speech recognition. Next, I thought
that I would try to locate an Apple dealer in my
geography.  After selecting the dealer locator
option, I received the “this call may be recorded
message” which was inappropriate since it was not
transferring me to an agent where the call was likely to
be recorded.  I was then transferred to still
another system (a T-T one where I encountered this menu
that went on-&-on and kept telling me to go to the Apple
web site (and since it must have figured that I was too
stupid to figure it out, that it had to tell me
over-&-over again that it was

www.apple.com

)   It had option after option that it dumped
on me that had nothing to do with finding a dealer in my
geography and for each option I was instructed to go to
the Apple web site.  These folks just don’t relate
to the caller at all.  I’m well aware that Apple
has a web site and don’t really need instructions on how
to find it.  Why they somehow think it is
appropriate and effective to keep on telling me
over-&-over that I should go to their web site escapes
me.  After listening to this long-winded menu, I
selected the one that was for locating a dealer in my
geography.  It then asked me if it was for business
or personal use.  I selected business.  It
then transferred me back to the original system that
announced that they were closed and that I should call
back during regular business hours and hung up on me
without capturing a messaging or even my phone number. 
Wow!  These guys really know how to irritate a
caller.  They tell me that I can find an authorized
dealer in my geography by pressing 2 and then waste my
time with all sorts of options and then finally tell me
to call back when they are open for business. 
After hours, a request to connect to a human is
responded to by telling the caller to call back during
normal business hours (6AM to 6PM PT).  The fact
that the Apple support line does not capture my phone
number is pretty awful.  All of the repetitive
menus are pretty bad the first time that you hear them
but the impact is much worse when you call again and
they dump all of this stuff at you again.  The
Apple support service is made up of a number of
different applications that sound and act different from
each other.  They will repeat information that
another application has already provided and
occasionally even disagree and provide different
information.  They use the # key for Repeat. It
should be pretty apparent why the Apple iPhone is going
to be a huge failure.  Apple knows nothing about
telephony and it shows pretty badly on their telephone
support line.


Dr. Walter Rolandi examination of the
Apple customer support line

The Examination Sessions

An Apple technical
support line was the patient today.  I called it at
800-275-2273.   My
task was to call the system and request technical
support for an iPod Nano.   Only one call was
necessary but I made three or four more just to explore
the options.

Call
Details:

For technical support press one

Status on a previously scheduled repair,
two

 
To extend service or support beyond Apple’s
complementary period or for questions about the
Apple Care Protection Plan, three

To find an authorized Apple reseller or
service provider, four

On one call I
pressed DTMF6 just to see what would happen.  The
system quickly responded, “Your entry was not
understood.”  Pressing DTMF1 transfers the caller
to a speech recognition system that asks the caller to
say the name of the product about which he is calling. 
The system affords the caller some examples:

For example,
you could say, Mac Pro, iPod Nano or Aperture.

 At this juncture,
on three separate occasions, I clearly said, “iPod Nano”
to which the system invariably replied::

 

I think you
said, iPod Nano.  Is that correct?

Upon confirmation,
the system asked one other question and then transferred
my call to a customer support representative.  This
process required approximately 70 seconds.

 

On another call, I
pressed DTMF4 to locate an Apple dealer.  I was
quickly transferred to another DTMF system that
exemplifies many of the most common maladies in IVR. 
The opening line of its 107-second Main Menu was that
old IVR chestnut,

Please
listen carefully to the following menu as the options
have changed.

This was immediately
followed by a web ad.  Then came menu item 1, menu
item 2 which included 4 (four!) more web ads and
instructions to call two 800 numbers for technical
support, one of which was
800-275-2273, (
the number I had called to reach
this system), and then menu item 3, which in turn
contained yet another web ad.

The Diagnosis

1.   Initial
interaction is in DTMF only.  System may route a
call to other systems that support speech or DTMF only.

2.   Opening
Main Menu is succinct and nicely trimmed.

3.   
Saying, “Your entry was not understood” in response to
user errors is pointless and almost punitive.

4.     

Speech recognition component seems
unintelligently designed and poorly tuned.

5.     

Dealer locator subsystem set a new record
for web ad content (six!).

6.     

Dealer locator subsystem prevents
dial-through during opening prompt cliché and initial
web ad, thus forcing users to listen.

7.     

Overall production quality is
questionable, being a hodgepodge of systems and
subsystems.

Other
than appearing unintelligent in its confirmation
prompting, the system process, at least for callers
merely seeking technical support of a product, actually
is quite reasonable.   Some of the other
subsystems that were sampled however are horribly ill. 
In fact, this is the worst case of “web ad-itus” I have
ever encountered.

The Prescription

1.     

Eliminate pointless DTMF error response
prompts.

2.     

Tune speech recognition.

3.     

Rethink the speech confirmation process.

4.     

Eliminate all unnecessary prompt and
prompt content.

5.     

Get rid of all web ads.

6.     

Permit users to dial-through universally.

This
system could be restored to better health relatively
easily with mere cosmetic changes to a number of
prompts.  More invasive interventions may be
necessary to improve the overall production quality of
the system.  To the owners of this system I would
prescribe at least an Apple-call a day.   I am
sure they would perceive the system’s annoyances if they
would only take a critical look.



The Prognosis

Fairly
good. 

 



Contact:
Walter Rolandi
, Ph.D.,
The Voice User Interface Company, LLC
+1. 803.252.9995,

wrolandi@wrolandi.com


www.wrolandi.com

 

 

 

 

 


Jan 6 20077

Walt
& Walter’s VUI View: on the Federal Trade Commission
(FTC) customer support Telephone Self-service Line

The Caller’s View
from Walt

This month we
called the FTC customer support line. 

The FTC is an independent agency that has
the authority to adopt trade regulation rules that
define unfair or deceptive acts in particular
industries. Trade regulation rules have the force of
law. The FTC reports to Congress on its actions.

When I called
877-382-4357, I was greeted by a monologue that just
went on-&-on.  In addition to rattling off menu
choice after menu choice, it managed to violate most of 
the gethuman Core Principles including: wasting my time
telling me about the Spanish option; telling me to go to
their web site and that it was

www.ftc.gov
(surprise);  telling me that their
menu had changed and that I should pay careful
attention.  Thankfully, they did not tell me that
“your call is important to us”.  I guess that they
could not get someone to keep a straight face long
enough to record this one.  The 2nd time that I
made an error, it hung up on me.  No voice mail or
callerID capture. If you want to talk to us, call us
back when we are available. When I called back during
“normal working hours” the next morning. the only
difference that I was able to notice was that when I
requested that I be connected to a human up-front, it
put me into a menu that got me to tell them if this was
my first inquiry and to provide my zip code if it was,
rather than hang up on me.  It still did not
recognize my 1st request for a human and forced me to
request it twice.  It announced a wait time of
“approximately 4 seconds” (which took about 10 seconds
to announce). In the wait announcement, it did manage to
assure me that “your call is important to us’. I was
then connected to a human in about 2 seconds.  His
voice was clearly that of an American male and I made
the assumption that the FTC was not outsourcing it’s
customer support to Mumbai.   He did not have
my zipcode and had me provide it again. I did use the
FTC speech-enabled system to order some of the FTC
telemarketing documents.  It again droned on again
about the FTC web site in gory detail.  When will
this thing ever stop?  Capturing my mailing address
was a painful process.  It also made me provide my
phone number, although it wasn’t apparent why this was
needed.  After I completed this order, I called
again to order some additional FTC literature.  It
did not have my address and made me enter it all over
again.

 


Dr. Walter Rolandi examination of the FTC
customer support line


The Examination Sessions

An
FTC IVR was examined this month.  It can be reached
at
877-382-4357. 
 The system offers seven (7) options on its Main Menu.  
My task was to call the system and order one of the
“more than 400 publications available from the FTC”.  
Only one call was necessary.


Call Details:

The call is
answered with a brief greeting followed by a Spanish
language option.  A 7-item Main Menu (of sorts)
immediately follows.  There is actually a 5-item
Main Menu but it is preceded by two other options one to
“press 6” and another to “press 8).   The menu
items are extraordinarily long.   The
publication ordering option was Main Menu 5, or the
seventh of seven total options offered.  When the
menu completed, I entered DTMF 5. 

           

The
call required approximately 78 seconds to arrive at this
point.  My choice was immediately followed by a
27-plus second prompt encouraging me to order the
publications by using the FTC’s website.  This “web
ad” was apparently intended to discourage ordering
publications using the FTC IVR.   After
reading the lengthy prompt, the system requires the user
to again indicate his desire (by entering DTMF 1) to
order a publication by mail using the system.  When
I did so, the system failed to interpret the DTMF 1
signal and proceeded to repeat the entire 27-plus second
web ad.

When
prompted to do so again, I entered DTMF 1 and held down
the key for approximately 1 second.   I was
immediately transferred to another system, this one
using speech recognition technologies, to complete the
task.  What followed was a 5 minute and 38 second
ordeal of name and address acquisition and confirmation.

 


The Diagnosis

1.     


System
is unbelievably stupid in its organization and
prompting.

2.     


Initial
interaction is in DTMF only.  Subsequent
interaction supports ASR.

3.     


Main
Menu is too long and illogically organized.

4.     


System
is complicated and tries to automate tasks that are
inappropriate for automation.

5.     


System
failed to interpret confirmed DTMF input.

6.     


System
requires far too much time and energy to complete a
simple task.

7.     


The
system’s value proposition is extremely questionable.

8.     


System
terminated the call upon task completion thereby not
permitting me to do anything else without recalling the
system.

9.     



Production quality is below average.

10. 


Most
prompts could be eliminated, the rest trimmed.

Once
again, we have a profoundly unhealthy system.  
More than merely being ill, the system appears to be
mentally ill as well.

 


The Prescription



1.
     

Determine why DTMF input could not be
processed.

2.     


Completely redesign the system
eliminating most of its features.

3.     


Employ well established best practices
for prompting and error handling.

4.     


Either adopt a design philosophy that
systems exist to assist their users or don’t’ bother
with 1-3.

This is
a tragic example of “your tax dollars at work”.  
Consider the absurdity of offering “more than 400
publications available from the FTC”.   The
system is itself symbolic of the government bureaucracy
that created it: most of it is utterly unnecessary and
unwanted.  

 


The Prognosis

Not
good.  Only the most extraordinary interventions
should be contemplated and even radical surgery affords
limited hope. 


 



Contact:
Walter Rolandi
, Ph.D.,
The Voice User Interface Company, LLC
+1. 803.252.9995,

wrolandi@wrolandi.com


www.wrolandi.com

 

 


December 6, 2006

Walt & Walter’s VUI View: on Eureka’s customer support
Telephone Self-service Line

The Caller’s View from Walt

Eureka Company
is also known as Electrolux Home Care Products North
America
. Electrolux is a huge Swedish-based company
that holds a 28% share of the worldwide household
appliances market. A slogan that Electrolux originally
used for marketing their vacuum cleaners in the UK was:
“Nothing sucks like an Electrolux.” We tested the Eureka
telephone support line (800-282-2886). This looked like
it was something out of the past. Even someone like
myself that is afflicted with retrophilia, had
difficulty in finding much good about this
implementation. The conclusion of my colleague (Walter
Rolandi) was that “The Eureka telephone support line
really sucks”. I fully concur with his observation. I
found this to be among the worst telephone self-service
implementations that I’ve ever run into. The challenge
of getting to a human is huge. If you press the right
sequence of keys at precisely the right time, the Eureka
system will begrudgingly attempt to connect you to a
human. Otherwise, it responds to the caller’s “errors”
by abruptly hanging up. Even if you do everything just
the way that they require, you still are not assured of
getting to a human. I received a message: “Due to
circumstances beyond out control, we are temporarily
unable to take your call. Please call back at a later
time.” It then hangs up without offering voice mail or
capturing your callerID. This went on all day. The
Eureka line is both a customer support line and a sales
line. If you indicate that you are interested in buying
a Eureka product, the system tells you to call another
number and hangs up. Imagine that ? a live customer is
on the line that wants to buy something and the Eureka
response is to tell them to hang up and call another
number. Forcing the caller to listen to a long verbose
menu of choices and not permitting overstrike is
attrocious. A very primitive speech-enabled capability
exists. It appears to recognize a caller speaking 8
different digits. Anything else it treats as a caller
error. A Touch-Tone capability exists but you have to
choose one or the other ? either speech or Touch-Tone.
Of the ten gethuman standards, the only one that they
did not appear to violate was the language selection
one. It was so difficult to get to a human that it was
difficult to determine if the agent spoke clearly. I’d
suspect that for the few callers that actually manage to
get to a human are so dazed by the experience that they
don’t even notice if the human speaks clearly or not.
The Eureka telephone self-service looks like something
that was implemented in the early days of telephone
speech (circa 1986). Digits only and no overstrike were
limitations of that era. Basic utility was for persons
that only had a rotary phone. In fact, if you indicated
that you had a Touch-Tone phone, then the system forced
you to use only Touch-Tone. I give them credit for not
telling callers to go to their web site (they problably
did not have a web site when this was implemented). They
also don’t instruct the caller to: “pay attention since
the menu has changed”, and it certainly was not an
overly lively persona. Amazingly, they did have the
nerve to tell the caller that their call was important
to them. When they upgrade, this will give them an
opportunity to add all of the latest
“torture-the-caller” techniques. This is an instance in
which the gethuman evaluation scheme does not do proper
justice to just how badly implemented the Eureka
telephone self-service is. To compensate for this, we
decided to award them an honorary gethuman score of 0.

 


Dr. Walter Rolandi examination of
Eureka’s customer service linee

The Examination Sessions

This month the patient was a Eureka
speech enabled IVR which is available at 1-
800-282-2886. The system offers seven (7) options on its
Main Menu. My task was to call the system and order a
catalog. Two calls were necessary however due to the
system’s modal organization. It is organized in such a
way as to encourage callers having DTMF to use DTMF.
Using speech is an option made available only to callers
who do not indicate they have touchtone phones. Thus,
two calls were needed to traverse both paths of the
tree.

Call One:

The call is answered with an 8.5
second greeting after which the caller is asked if he is
calling from a touchtone phone. If the caller indicates
that he is, he then hears a 7-item Main Menu followed by
an additional (8th) option to repeat the
information:


Thank you for calling the Eureka Company. If
you have a touchtone phone, please press one
now. Otherwise, Please stay on the line.

(User enters DTMF 1)

For information on warranty and where to
obtain service, press one.

To purchase parts or accessories, press two.

To obtain an owner’s manual, or, for
information on how to replace your belt, press
three.

For information on where to buy Eureka
products, to receive product literature, become
a Eureka dealer or for presales information,
press four.

If you are currently a Eureka dealer or
warranty station, press five.

For information on the style of bag and/or
belt your vacuum requires, press six.

For all other information, press seven.

To repeat these options, press nine.





 






The call required approximately 55 seconds to arrive
at this point. Approximately 3 seconds later, before I
had had a chance to respond, the system said,

To avoid being disconnected, please make a selection.


For information on where to buy Eureka
household vacuums, press one now.

For information on where to buy Eureka
built-in vacuums, press two now.

To receive a Eureka product catalog, press
three now.

For questions regarding Sanitaire household
products, Sanitaire commercial products or to
become a dealer, press four now.

For presales information, press five now.

(User enters DTMF 3)


Please enter your five-digit postal zip code now.


(User enters five-digit zip code using DTMF)

At the tone, please say then spell your first
and last names.

(User says name)

Please verify that your name is: (system
plays back recorded name)

If this is correct, press one now. If this is
incorrect, press two now.

(User enters DTMF 1)

At the tone, please say then spell your
street address and then say your city and state.

(User complies)

Please verify that your address is: (system
plays back recorded address)

If this is correct, press one now. If this is
incorrect, press two now.

(User enters DTMF 1)

Please allow seven business days for
delivery. Thank you for calling Eureka.

(System hangs up after 8-second silence)


This entire process required 186 seconds.

 

Call Two:

The second call was basically the
same as the first call except that I intentionally did
not respond when prompted to use DTMF. After about a
6-second silence, the system read the Main Menu above
only slightly modified to prompt user speech.


For information on warranty and where to
obtain service, say one.

To purchase parts or accessories, say two.

To obtain an owner’s manual, or, for
information on how to replace your belt, say
three.

For information on where to buy Eureka
products, to receive product literature, become
a Eureka dealer or for presales information, say
four.

If you are currently a Eureka dealer or
warranty station, say five.

For information on the style of bag and/or
belt your vacuum requires, say six.

For all other information, say seven.

To repeat these options, say nine.


I wanted to see if DTMF was supported
in the speech recognition mode. Upon hearing the entire
alternative Main Menu, I entered DTMF 4 to which the
system immediately responded:


Thank you for calling Eureka.

(System hung up after 8-second silence)




The Diagnosis


  1. System inexplicably
    segregates DTMF and ASR modes.
  2. Main Menu is far too long.
  3. System attempts to
    automate far too many tasks.
  4. Main Menu and sub-menu are
    complicated and confusing.
  5. System requires too much
    time to complete this simple task.
  6. System does not provide
    enough time to respond.
  7. System ominously threatens
    the caller with disconnection if they do not
    promptly respond.
  8. System is horribly
    unforgiving. It hung up on me in speech mode
    presumably for making one error.
  9. System terminated my first
    call upon task completion thereby requiring users to
    call back if they sought to perform more than one
    task.
  10. Production quality is
    questionable.
  11. Almost all prompts could
    be trimmed.

This patient is profoundly ill. Eight
choices on a menu are far too many but if one considers
the “ands and ors” contained on the Main Menu, it
actually contains sixteen (16) choices. This is
cognitively indigestible by most human beings. Overall,
the system tries to do too many things and it is
therefore big and complicated. Its speech interface is
exceptionally primitive, apparently employing only a
10-digit grammar. This suggests antiquated technology
which might perhaps explain the segregation of the DTMF
and speech modes. Whatever the explanation, the system’s
modal organization is senseless.


The Prescription


  1. Completely redesign the
    system eliminating all but the most essential
    features.
  2. Employ well established
    best practices for prompting, timing and error
    handling.
  3. Permit dial through and
    barge-in.
  4. Use professional voice
    talent.

Traditional treatments and cosmetic changes would do
little to improve the health of this system. This is a
case for euthanasia.

The Prognosis

Without a complete reorganization, very grim indeed.




wrolandi@wrolandi.comm



www.wrolandi.com



November 7, 2006

The Caller’s View from Waltt

I called the 800-GO-FEDEX number. 
This implementation was the best that we’ve tested so
far. The gethuman VUI Standard score was 6 out of 10
which positions it 2 points higher than the best that we
had previously seen. Used my CallerID to determine my
location and find the nearest drop-off points. Having
DTMF available but hiding it from the caller seemed like
a strange thing to do. Never was able to figure out how
to get it to repeat. It did not like *. Most of the
calls to this line are from repeat callers. Keeping
track of the caller’s history/preferences would make
sense. The Spanish option and the FedEx web
announcements are irritants when you hear them once.
This gets worse when you hear them every time that you
call. Should not have to provide my account number,
particularly not for an application like requesting a
package pickup. The use of dated TTS technology really
sticks out. The recorded voice is good and then you hear
the address with this antiquated TTS. FedEx makes it
fairly easy to get to an agent most of the time. At a
number of points in the menu, they just block it out,
though. I provided information to the system to
calculate the cost to ship a package. When I went to the
agent, she did not have this information. They do not
provide a wait time estimate or offer a callback option
when you go to the wait queue. From my experience, this
is not necessary since FedEx invariably has adequate
agents and wait queues are non-existent.

 

Dr. Walter Rolandi examination of
FedEx’s customer service line

The Examination Sessions

This month the patient is a FedEx
speech enabled IVR. It can be reached at 1-800-GO-FEDEX
(1-800-463-3339). The system offers seven (7) options on
its Main Menu. My task was to call the system and find a
nearby drop-off location. Only one call was necessary to
perform the assigned task. However, a second call was
made in order to test the system’s error handling
strategy.

Call One:

The call is answered with a brief greeting. Then
comes a Spanish language option followed by
approximately 1.5 seconds of silence.


Welcome to FedEx. Para continuar in Español,
diga Español.

(1.5 seconds silence)

If you know the option you’d like, please say
it now.

(1.5 seconds silence)

Please say: Schedule a pickup


Track a package

Find location

Get rates

Order supplies

Office and print services

Or International 


(1.5 seconds silence)

To hear these options again, say repeat.

To hear more options, say more.


The call required approximately 30 seconds to arrive
at this point.

I responded, “Find location” to which the system
said, “All right. Locations”. The system then explained
that it would use a zip code to find nearby locations
and then prompted me to provide one. I complied, was
thanked by the system, told that it needed some time to
find the appropriate locations and then read a “by the
way” prompt noting that locations can also be found at
FedEx.Com.

The system played an “I’m working” earcon but quickly
returned with three relevant locations. The location
information used high quality (human) audio prompts that
described the class (drop box, official station, etc.)
and time schedules of each pickup location. TTS was used
for the name and address of the locations.

Call Two:

The second call was the same as above except that I
either said nothing or made intentional verbal errors
after the Main Menu to see how the system would respond.

The Diagnosis


  1. Main Menu is too long.
  2. There are some minor
    timing issues.
  3. There are some pointless
    almost annoying “hints” as to what the user should
    say.
  4. Spanish option prolongs
    interaction by about 5.0 seconds.
  5. During Spanish option,
    dial through and barge in are not fully supported.
  6. DTMF is supported but the
    system does not inform the user of the feature. (At
    least it did not do so during the diagnostic calls).
  7. Minor anthropomorphic
    excesses are present.
  8. System contains a web ad.
  9. Some prompts were
    needlessly “conversational”.
  10. Some prompts could be
    trimmed.
  11. TTS leaves a lot to be
    desired.

With the exception of Main Menu’s
length, this is a reasonably healthy system. Its health
however could be significantly improved with some
relatively minor changes. Pace was generally good but
there are issues with timing, mostly after prompting for
user input. The system prompts for a response but then
begins speaking again (with pointless almost annoying
“hints”, no less) before the user has had time to do so.
The Spanish option needs work and the web ad should be
removed. The anthropomorphic and conversational excesses
along with unnecessary prompt content would be easy to
improve. TTS was at times difficult to understand and
made at least one major mistake in reading the name of a
particular location.

The Prescription


  1. Consider eliminating or
    subordinating some Main Menu options.
  2. Permit dial through and
    barge-in during the Spanish option or, preferably,
    totally remove the Spanish option. All callers would
    be better served if Spanish speakers were provided a
    dedicated 800 line.
  3. Tune timers.
  4. Provide some indication
    that DTMF is supported.
  5. Eliminate web ad.
  6. Eliminate existing
    “hints”, anthropomorphic and conversational excesses
    and otherwise trim prompts of all unnecessary
    content.
  7. Consider using a better
    TTS engine.


The Prognosis

Very good to excellent.

Contact: Walter Rolandi, Ph.D., The Voice User
Interface Company, LLC +1. 803.252.9995,




wrolandi@wrolandi.com



www.wrolandi.com

 



October 13, 2006

Walt & Walter’s VUI View: on Comcast’s customer
support Telephone Self-service Line

The Caller’s View from Walt

I called the 800-COMCAST number. 
After asking me for the phone number that had the
service (it did the same thing whether or not I was
calling from the line or not) it then asked me to press
1 for English. It then gave me a menu that asked me if I
was calling about a billing issue, a trouble, a sales
inquiry or a move/change.  This does not make a bit
of sense to me.  The caller that has a
question about his bill is getting the same treatment as
the caller that is interested in switching his telephone
service and broadband to Comcast.  The caller that wants
to purchase something gets another menu that asks him to
further define what product he is interested in
purchasing.  If this isn’t enough aggravation, when
you get to an agent, they don’t have any of this
information and you start all over again. I called the
local Comcast number that was on my bill.  It
seemed virtually identical to the 800-COMCAST line with
the exception that it after the greeting, it started
playing music and told me about the Comcast web site. It
did pull my CallerID and asked me to validate it from
the last four digits. This seemed reasonable until you
realized that it had just wasted my time asking me to
confirm my language preference and this was done after
it told me about the web site in English.

 

Dr. Walter Rolandi examination of
Comcast’s customer service line

The Examination Sessions

The patient this month is that of a
Comcast DTMF IVR. It can be reached at 1-800-COMCAST
(1-800-266-2278). The task was simply to call the system
and navigate to its Main Menu. A valid account number
and zip code were employed to access the system. Two
calls were required. Both were digitally recorded.

 

First Call

On the first call, I provided the
account number when prompted. However the system
responded with, “I did not recognize your selection”. It
then re-prompted me for the information which I again
provided. Apparently accepting the account number, the
system then asked for the zip code where the account is
located. I entered the zip code but was again informed,
“I did not recognize your selection”. Upon re-prompt, I
entered the information again. After about a 3.5 second
pause, I was asked a question apparently intended to
single out prospective sales calls and I was transferred
to a sales representative.

Curious to understand what had
happened, I played the digital recording of the call
back through a DTMF decoder. The recording indicated
that there was a sound quality issue during the first
attempt to enter the account number. This would account
for the initial failure. However the recording showed
that the second entering of the account number and both
zip code entries were correctly decoded and therefore
presumably acceptable to the system. The system however,
appeared to treat the call as an unidentified caller.
The default response to such calls seems to be to treat
them as potential customers and transfer them to a sales
representative. The call required almost two minutes
seconds to get to get to the point of transfer.

Second Call

On the second call, I provided the account number
when prompted which appeared to be immediately accepted
by the system. A transcript follows:


Thank you for calling Comcast.

 Please enter the telephone number including
area code where you have or want service.

(I entered account number)

For English press 1. Para Espanol, oprima el
numero dos.

(I entered DTMF 1)

Thank you. Please hold while we process your
call.

(after 8 seconds of silence)

To help us better serve you please verify
that the last four digits of your account phone
number are 5 5 5 5. If this is correct, press 1.
If this is not correct, press 2.

(I entered DTMF 1)

Please hold while we validate your
information.

(after 4 seconds of silence)

For trouble with your service, press 1

Billing inquiries, press 2

Sales or sales information, press 3

All other questions including moves, changes
to your service or discontinued service, press
4.

 


The call required 72 seconds to arrive at this point.

The Diagnosis

  1. I have no
    explanation for the failed first call. The telephone
    connection could have been so poor as to garble my
    DTMF entries. However the fact that 3 or the 4 DTMF
    entries were clearly decoded from an 8000 Hz, 16-bit
    mono recording of the call suggests otherwise. Thus
    the system may have difficulty decoding DTMF.

  2. The system asks the user
    for his language preference after he enters
    his account number.
  3. The system has what appear
    to be long and awkward silences.
  4. System feedback prompts
    are not particularly informative or helpful.
  5. Several prompts would
    benefit from trimming.

In summary, the first call raises the
specter that this system may have serious problems. On
the other hand, the second call suggests, at least when
one’s account number is recognized, that the system is
not really all that seriously sick.

The Prescription

Cosmetic changes could address the problems in
feedback and unnecessarily verbose prompts.

Reversing the order of the language
preference question would also appear to be fairly
trivial. Ideally, the question would never be asked more
than once: the caller’s language preference should be
retrievable once he has provided his account number.

Eliminating or reducing the long
silences may be more difficult if they are due to
backend issues. Inserting an, “I’m working” earcon would
help alleviate that problem however.

The Prognosis

Quite good, given a little TLC.

Contact: Walter Rolandi, Ph.D., The Voice User
Interface Company, LLC +1. 803.252.9995,



wrolandi@wrolandi.com



www.wrolandi.com



September 13, 2006

Walt &
Walter’s VUI View: on United Airline’s Reservation/Sales
Telephone Self-service Line

The
Caller’s View from Walt

Called the United Airlines
800-864-8331 reservation sales
line. Up-front greeting that goes on and on. Has to tell
me about the wonders of the United web site every time
that I call. Announces that the call is being recorded
as part of the main greeting even though I may never
connect to an agent. Touch-Tone menu up front. Choices
are: 1) flight status 5) Redeem frequent flyer miles; 3)
U.S. travel reservations; and 4) International travel
reservations. The frequent flyer mileage redemption
option is a Touch-Tone one while the others are
speech-enabled ones. The redemption option is the 2nd
item on the menu, yet the instructions are to “press 5”.
Pressing 2 also gets you to the redemption option which
makes this strange. No CTI, so I have to repeat
everything again to the agent and queue wait times are
not provided. The response to requesting an agent is to
connect to self-service speech-enabled self-service
reservation line. If you continue to insist on being
connected to an agent, the self-service system finally
stops arguing with you but not until you tell it if you
are looking to travel within the United States or
international. When I get to an agent, I was connected
to ones in India with thick accents that were difficult
to understand. This is primarily a sales line. If you
have a lost-bag problem, United has a separate line for
this. What possesses these folks to think that it makes
any sense to make it at all difficult for a paying
customer to get to an agent and buy something? United
also lies to the caller. The final phrase of the main
menu is:
If you are calling from a rotary dial
phone, please stay on the line and your call will be
answered by a United Airlines sales representative
.
What really happens is that if you don’t respond at all,
you get connected to the speech-enabled self-service
system which then requires that you to argue with the
self-service before you get connected to a human.

 

Dr. Walter Rolandi examination of
United Airlines customer service line

 

The Examination Sessions

Walt Tetschner suggested that I obtain
the status of a flight using the United Airlines IVR. My
task was simply to call the system, navigate to the
point where I could obtain flight status information,
get the information and hang up.

 

Completing this seemingly simple task
proved to be quite an ordeal entailing three separate
calls and requiring almost 13 minutes.

 

Call Commonalities:

All calls are apparently answered by a
routing application. It first greets the caller,
entreats him to visit the United web site (a.k.a., reads
a “web ad”) and then warns the caller using a
stereotypical statement concerning calls being recorded
“for quality assurance purposes”.

 

The caller then hears the Main Menu:

 

For flight arrival and departure
information please press 1

To redeem miles from your Mileage Plus
Account, please press 5

For travel within the 50 United States,
please press 3

For international travel including Canada
and the Caribbean, please press 4

If you are calling from a rotary dial
phone, please stay on the line and your call will be
answered by a United Airlines sales representative.

 

The system requires 57 seconds to arrive
at this juncture. At times, the voice talent sounds
eerily like Kermit the Frog.

 

First Call

On my first call, I entered
DTMF1 after the initial 57 second opening. I then heard,
“Welcome to United Airlines flight information line”
followed by almost 17 seconds of information about the
system and instructions as to how one should interact
with it. I did not have a particular flight number in
mind so I followed the instructions (saying, “I don’t
know it” when prompted) and then completed the laborious
process of providing and confirming the departure city,
arrival city, time of day, etc. for a flight from
Columbia, SC to Washington, DC. The system collected all
of this information and completely restated it, asking
me to confirm that the information was correct. When I
did so, it returned with departure information on a
Flight 7197.


 

Out of curiosity, I
looked up Flight 7197 on United’s website. To my dismay,
I found that the flight did not fly from
Columbia
to Washington but rather from Columbia to Chicago.


 

Second Call

I decided to give the system another
chance using a known flight number (Flight 7279) which I
obtained from United’s website. I called and again
listened to the 57 second opening. Since I had just
interacted with the system using speech, I inadvertently
said, “One” in response to the Main Menu (above). The
system apparently treated the utterance as a “DTMF
non-response” and after approximately six seconds of
silence, my call was transferred.

 

Note that the Main Menu states:

 

If you are calling from a rotary dial
phone, please stay on the line and your call will be
answered by a United Airlines sales representative.

 

This is not what happened to me however.
My call was transferred to the United Airlines
reservation system. I attempted repeatedly to escape
back to the Flight Status system but to no avail.
Eventually, I hung up.

 

Third Call

I called the system again, entering DTMF1
after the 57 second opening. When prompted for the
flight number, I said, “seventy two seventy nine”. The
system said,

 

I
think you said, “seventy two seventy nine”. Is that
correct? Please say yes or no.”

 

I answered, “yes” and then, when asked,
requested departure information. What then followed was
incomprehensible to me. The system seemed to give
conflicting and thoroughly confusing departure and
arrival information. Upon hanging up, I went back to the
website where I discovered that “Flight 7279” entails to
two separate flights. It appears to be a shuttle that
departs from Washington, arrives in Columbia and then
returns to Washington. I suppose this is why the system
reports confusing arrival and departure times.

 

The Diagnosis

  1. The system is fundamentally flawed:
    it provided information on an incorrect flight
    (7197), it failed to transfer me to a “sales
    representative” and, presumably due to the flight
    naming conventions of United Airlines, it provided
    incomprehensible flight departure and arrival
    information (7279).

  2. The system contains a web-ad.

  3. From the user’s point of view, the
    system mixes modalities, initially requiring DTMF
    while supporting speech and DTMF after the Main
    Menu.

  4. The system seems to talk
    interminably.

  5. The system annoyingly and repeatedly
    tells the user exactly what to say.

  6. There is far too much unnecessary
    prompt content.

  7. The system pointlessly attempts to
    sound conversational.

  8. Many persona excesses and
    anthropomorphisms are evident.

  9. The system ineffectively attempts to
    provide alternative but complicated and
    painstakingly slow methods to find flight status
    information.

  10. The time required to complete a
    simple flight status query task is unnecessarily
    long.

  11. The flow, production quality and
    professionalism of the system’s prompts are at times
    questionable.


 

In summary, this system is gravely if not
fatally ill.

 

 

The Prescription

While some cosmetic changes could
improve, or at least shorten, the user experience, this
system needs major surgery. Again, after the system
confirmed that my query regarded a flight from Columbia
to Washington, it then provided information about a
flight from Columbia to Chicago (see Addenda). This
implies basic logic or data integrity issues. The system
says that calls will be transferred to sales
representatives when they are actually transferred into
another automated system from which there is no apparent
escape. It is difficult to understand why more than one
flight can have the same flight number (see Addenda).
Both of these issues imply faulty business decisions.

 

Cosmetically, the following treatments
could help:

  1. Universally support speech.

  2. Eliminate all unnecessary prompts and
    prompt content. Aggressively trim all prompts.

  3. Eliminate all preemptive prompt
    instructions.

  4. Eliminate persona and anthropomorphic
    excesses.

  5. Simplify and speed up the overall
    process.

  6. Fix production quality problems

 

The Prognosis

Unless the systemic problems can be
surgically excised, the prognosis is far from good.

Addenda


Airline


Flight


Departure


Arrival



 


Frame1


Flight 7197


Columbia, SC (CAE)


Chicago, IL (ORD)


United


Operated by UNITED EXPRESS/MESA AIRLINES


Mon, Sep 11


Mon, Sep 11

 


NOT DEPARTED


Scheduled: 

5:34 PM


Scheduled: 

6:45 PM

 

Details


Estimated:  

5:34 PM


Estimated:

6:45 PM

 

 


Reason: —


Reason: —

 

 


Gate: —


terminal 2 concourse f

 

 

 


Gate:  F1


 

 

 


Baggage claim:  3


Airline


Flight


Departure


Arrival



 


Frame2


Flight 7279


Washington, DC (IAD)


Columbia, SC (CAE)


United


Operated by UNITED EXPRESS/MESA AIRLINES


Fri, Sep 8


Fri, Sep 8


 

 


NOT DEPARTED


Scheduled: 

5:00 PM


Scheduled: 

6:22 PM

 

Details


Estimated:  

5:00 PM


Estimated:

6:22 PM

 

 


Reason: —


Reason: —

 

 


concourse a


Gate:  —

 

 


Gate: A4


Baggage claim:  —

 


Flight 7279


Columbia, SC (CAE)


Washington, DC (IAD)

 


Operated by UNITED EXPRESS/MESA AIRLINES


Fri, Sep 8


Fri, Sep 8

 


NOT DEPARTED


Scheduled: 

7:00 PM


Scheduled: 

8:26 PM

 

Details


Estimated:  

7:00 PM


Estimated:

8:26 PM

 

 


Reason: —


Reason: —

 

 


Gate: —


concourse a

 

 

 


Gate:  A2

 

 

 


Baggage claim:  1


 



August 12, 2006

Walt & Walter’s VUI View: on Verizon’s Customer
Self-service Telephone Self-service Line

The Caller’s View from Walt

I’ve been a Verizon customer for as
long as I can recall. I remember when they were known as
New England Telephone. This was before the breakup of
the original AT&T (1984), and then it became part of
NYNEX which finally merged with Bell Atlantic, acquired
GTE and became Verizon. I had a noise problem on one of
my phone lines so I decided to call Verizon to get it
repaired. To determine the number to call I looked on my
monthly phone bill. Aha! I quickly found that it listed
a separate number for Repair. I called the number and
the first thing that it dumped on me was a menu that
asked me what I was calling about. It gave me an option
via the menu to inquire about most anything (billing,
service changes, product information, etc. in addition
to repair) What is wrong with these people? They
specifically identify a number that you should call for
repair. When you follow their instructions, they turn
around and ask you what you are calling about. This is
one of the most irritating things for a caller. The
caller took the time to figure out what the proper
number to call is. For this, they get rewarded by having
their time wasted having to again explain what they are
calling about. The difference between the Verizon Portal
(which Walter called) and the published Verizon Repair
number is also interesting. The Verizon Repair number
greets you by telling you how to get to Verizon’s web
site. This is so important to them that you can’t
interrupt it and are forced to listen to it. The Verizon
Portal greets you with an advertisement claiming that
they are my broadband company, which, unfortunately, I’m
already aware of. All that is needed and appropriate is
a simple “Verizon”. The Verizon Portal captures your
number via CallerID while the repair line makes you
enter your 10-digit phone number. For the Repair line
menu, if you select an option other than Repair, you are
prompted with: ” Are you calling about 508-551-0413?
This is not even the correct area code for my number. I
called it and found that it was another Verizon Repair
line that functioned differently than the Repair line
that was listed on my bill. No CTI, so you need to
repeat everthing that you entered when you get to a
human. Also without CTI they can’t accurately tell you
how long the wait in the queue is. Just have to listen
to the Verizon ads without knowing how long the wait
will be.

Dr. Walter Rolandi examination of
Verizon Self-service

The Examination Sessions

I called the Verizon IVR a week or so
after Walt Tetschner had used it to report a problem
with his phone service. My task was simply to navigate
to the point where I would report a problem. I called
Verizon toll-free number (800-870-9999) twice and
recorded both interactive sessions. Normally only one
call would have been necessary however something
completely unpredictable occurred on the first call that
obliged be to try again.

First Call

The system answers with a brief greeting
(approximately 4.7 seconds):


Good morning. Thanks for calling Verizon,
your broadband company.


The greeting is followed by a
three-note audio-icon. After about 4.0 seconds of dead
air, the system apparently transferred the call to some
other system. After five rings, a system answered
saying,

You have a choice of regional toll and long distance
providers….

(approximately 26 seconds of some sort of sales
pitch)

Then came another ring after which a
human answered. I explained to the nice lady that I
didn’t know how or why I was transferred to her,
apologized and hung up.

Second Call

Again, the system answered with the same brief
greeting:


Good morning. Thanks for calling Verizon,
your broadband company.


It then read out the number that I
had called from asking if this was the number I had
called about. I answered, “No” and after an awkward
(about 3.0 seconds) pause it said,

Ok. What’s the phone number you are calling about?

I provided a valid Verizon number
that the system then read back to me (after an almost
4.0 second pause) for confirmation with a yes/no
question. I responded, “yes” in less than 1.0 second but
as I was speaking the system played a prompt instructing
me to say “yes”.

The system then played an “I’m working” audio con for
a little over 6.0 seconds after which it announced,

I’ve got your records.

shortly followed by the open-ended prompt,

In just a few words, how can I help you today?

Before I could respond, the system said,


For example, you can say something like I
have a question about a long distance charge on
my bill.


Given my task to find information about repairs, I
finally said,

I’m calling about
repairs.

The system then asked if my trouble was with my,

…DSL service, voicemail, phone line or none of
those.

When I answered, “None of those”, I was transferred
to:

“…the automated system that can help you with
that.”

It is unclear what then happened.
What followed was a series of unnecessary instructions,
another read-back of the number I had called about and
another announcement that my account history had been
found.

Eventually, another open-ended prompt
invited me to briefly describe my problem or ask for a
list of possibilities.

I said, “List” and then hung-up after
“six common types of problems” were read out.

This entire process required 202
seconds. I experienced either two or three confirmations
depending upon one’s interaction of the semantics of the
last telephone number read-back.


The Diagnosis


  1. The first call was
    bizarre.
  2. Regarding the second call,
    there is unnecessary prompt content.
  3. The system ineffectively
    attempts to sound conversational.
  4. Some persona excesses and
    anthropomorphisms are evident.
  5. The system has serious
    timing problems.
  6. The system has serious
    turn taking problems.
  7. The system occasionally
    appears to be unnaturally slow responding
    (presumably a backend issue).
  8. Open-ended prompts provide
    insufficient feedback.
  9. Many prompts contain
    confusing preemptive examples of what the user is
    supposed to say.
  10. The time required to
    complete the task is painful and excessive.
  11. The production quality of
    the system’s prompts is not as good as it might be.
  12. The voice talent on some
    prompts occasionally sounds over-animated while at
    other times sounds almost sedated.
  13. Although it is not clearly
    indicated, DTMF is supported throughout the system
    except at the open-ended prompts.

I may never know what happened on the
first call. Perhaps it was some bizarre fluke or perhaps
the system reacted inappropriately to something I did
not even say. Whether fluke or otherwise, this
experience could signify a substantial flaw in overall
system architecture. Who knows?

Once accessed, the system is not as
bad as it could be (that is, assuming that my transfer
to “…the automated system that can help you with
that.” was logically correct and not actually due to a
false positive response from the recognizer). In any
event, the system has serious usability issues. The
timing problems and preemptive “help” prompts make it
difficult to discern when to speak. The system can also
appear “rude” when it speaks while the user is speaking.
It also has a good bit of unnecessary or pointless
prompt content that only serves to inform the user that
the system is wasting his or her time.

The Prescription

While seriously ill, this system could be put into
excellent health using the following treatments:

  1. Trim all prompts.
  2. Tune timing and
    turn-taking variables.
  3. Eliminate unnatural
    (presumably) backend delays.
  4. Eliminate persona and
    anthropomorphic excesses.
  5. Make the overall process
    faster.
  6. Consider eliminating
    open-ended prompts. Otherwise, provide more
    meaningful feedback.
  7. Fix production quality
    problems

The Prognosis

Pretty good, assuming the turn-taking and timing
problems are not insurmountably linked to back-end or
infrastructure issues.



July 7, 2006

Walt & Walter’s VUI View: on
CareMark’s Customer Self-service Telephone Self-service
Line

The Caller’s View from Walt

I’ve been refilling prescription
drugs via mail order for a number of years. First with
AdvanceRx and now
CareMark
who acquired AdvanceRx. Price is much lower
than buying from the local drugstore. I’ve been calling
them regularly and doing the same thing each time.
Ordering the same prescription drugs, paying for them
with the same credit card and having them sent to the
same address. They have all the ugly stuff that callers
hate. Spanish options to waste their time. No CTI, so
they can’t tell me how long the wait queue is and I have
to repeat everything to the agent. Prescription refill
is a fine application for self-service. The caller knows
precisely what they want to do. They do it over-&-over –
usually from the same phone. Despite this, the CareMark
prescription refill self-service application is a pretty
ugly implementation and probably has a relatively low
usage. Lots of really dopey stuff like telling the
caller that they can say Goodbye. When I say my account
number, I need to say “zero”, since it does not
recognize me saying “oh”.


 

Dr.
Walter Rolandi examination of CareMark Self-service

The
Examination Sessions

I called
the CareMark IVR the day after Walt Tetschner had it to
order some prescriptions. My task was to “check on the
delivery status” of the prescriptions. Accordingly, I
called CareMark toll-free number (888-424-6618)
and recorded my interactive session. Only one call was
needed, given the task at hand.

The
system answers with a brief greeting (approximately 3.3
seconds):

Thank you for calling
CareMark Prescription Service.

A simple
Main Menu then follows:

To refill a prescription,
press one

To check the status of a
mail order prescription, press two

For all other enquiries,
press three

Para escuchar este mensaje
in Espnaol, por favor oprima quattro ahora.

(To hear
this message in Spanish please press four now)

The
system requires only 16 seconds to hear its Main Menu.

Given the
task was to “check on the delivery status” of the
prescriptions, I entered DTMF2. After approximately four
seconds of “dead air”, I heard the sound of a ringing
telephone which was immediately followed by the prompt:

How would you like me to
find your order?

By prescription number

Most recent order

Or by drug name

Otherwise say, repeat,
goodbye, representative or press zero.

I
followed the instructions and completed the task in four
minutes and forty-nine seconds. I experienced two
confirmations and three speech recognition failures, two
of which were at yes/no junctures.

The
Diagnosis

  1. The
    system mixes interactive modalities. Its Main Menu
    system is DTMF only while the “delivery status”
    system is speech only (although it does permit
    pressing zero for a representative).

  2. Most
    prompts are verbose and could be trimmed.

  3. The
    system is at times pointlessly anthropomorphic.

  4. The
    system has severe turn taking and timing problems.

  5. Many
    prompts contain confusing preemptive examples
    presumably intended to help the user know what to
    say.

  6. The
    system supports various methods of tracking
    prescriptions that might be intended to help certain
    classes of users but may in fact only confuse and
    complicate the process for them and others.

  7. The
    “delivery status” task process is unnecessarily
    complicated.

  8. The time
    required to complete the task seems excessive.

  9. The
    Spanish option is probably unnecessary and, in any
    event, ineffectively deployed.

  10. The
    system supports saying “O” for zero in some places
    but not in others.

  11. The
    production quality of the system’s prompts is
    inconsistent: prompt recording quality and volume
    vary in some states.

While
this system does exhibit an appreciation of many VUI
design best practices, it nonetheless suffers from some
severe infirmities. First is that the system takes a
task that should be very simple to perform quickly and
complicates it unnecessarily. Second, mixing modalities
and supporting the use of “O” for zero in some places
but not in others can thoroughly confuse users. Third,
the systems turn taking and timing problems make the
dialog seem needlessly odd. They additionally prompt
user input inappropriately and ineffectively.

The
Prescription

While
seriously afflicted, this system could be put into
excellent health relatively easily. The following
treatments would be required:

  1. Simplify
    the process for obtaining “delivery status”


  2. Eliminate modality switching


  3. Eliminate the need to restrict the use of “O” and
    zero

  4. Fix all
    turn taking and timing problems


  5. Eliminate unnecessary prompt content

  6. Support
    Spanish callers more effectively

  7. Fix
    production quality problems

The
Prognosis

Quite good,
given the will to improve.


 



June 3, 2006

Walt & Walter’s VUI
View: on Amtrak’s Customer Self-service Telephone
Self-service Line

The Caller’s View from
Walt

The
Amtrak customer support number is 800-USA-RAIL. The
first thing that you are greeted with is Julie Amtrak
telling you that about the Amtrak web site and how the
fares may be lower. To torture you more, the caller is
forced to listen to this in its entirety, since
overstrike is enabled. This gets even more irritating
each time that you are forced to listen to it. I have
used the Amtrak web site a number of times and never
found a lower fare. The attempt to create an
anthropomorphic agent is a failure. Did you ever hear a
real person say: “did you mean?” when they did not
understand what was said? The little phrases (hold on,
okay, got it, great) that are supposedly intended to
help establish Julie as a real person, become irritants
since they add nothing and waste the caller’s time. Even
after using the Amtrak self-service a number of times, I
found that it took too long to accomplish a task. The
Amtrak live-agent was invariably able to handle it much
faster. Since the Amtrak hold queue for an agent is
usually non-existent, it makes most sense to simply go
to a live agent up front and not deal with the
self-service. Amtrak needs a Work Force Management (WFM)
system. They are over-staffing their agent positions and
have eliminated the need for a self-service capability.
This situation seems perfectly consistent with the
management of Amtrak. Wasting money on unnecessary
things and not investing in essential thngs such as
adequate train maintenance.

Dr. Walter Rolandi
examination of Amtrak’s Julie

The Examination
Sessions

I called
the Amtrak toll-free number (800-USARAIL) and recorded
three sessions. As with all VUI Reviews, my first
objective was to simply navigate through the system’s
Main Menu. My second call was intended to test the
system using a speakerphone. The objective for the third
call was to comply with Walt Tetschner’s request that I
“check the status of a train”.

First Session:

The system (“Julie”)
answers with a brief (approximately 3.0 second)
greeting:

Hi. I’m Julie, Amtrak’s
automated agent.

The greeting is followed
by an approximately 6.0 second web ad:

I invite you to visit our
website, Amtrak.Com. Lower fares may be available.

A wordy Main Menu then
follows. Simplified, its choices are:

Train Status

Schedules

Reservations

Agent

The system requires only
24 seconds to hear its Main Menu.

Second Session:

Recall
that this call was made to the system on a speakerphone.
I said nothing to the system and made no noise at all
during the call. The system behaved precisely as before
until it was well into the Main Menu whereupon it began
to experience repeated recognition failures. It was
sufficiently intelligent however to suggest that the
errors may be due to a “weak cell phone signal”,
“speakerphone echo or some other background noise”. The
system quickly transferred the call to an agent.

Third Session:

The last
session was essentially equal to the first up to the
point where I was prompted to say, Train Status. I
attempted to get the status on a train leaving from
Columbia, SC to Washington, DC. After a lengthy
interrogative process, the system told me that it had
found two trains. The first was described as a direct
train called the “Silver Star” and the second, also
(oddly) called the “Silver Star”, was said to involve a
connection through Richmond. I was unable to resolve the
ambiguity and obtain the status. The system eventually
transferred me to an agent.

The Diagnosis

  1. The
    system is presumptuously anthropomorphic
  2. The
    system is unnecessarily animated
  3. Most
    prompts contain unnecessary prompt content
  4. The
    Web ad annoyingly wastes of the user’s time
  5. The
    time required to hear the Main Menu could be reduce
    by 40%-50%
  6. The
    “Help” feature is unhelpful and unnecessary

The Prescription

My
failure to complete the task was presumably due to a
functional problem of the system and thus did not figure
largely into my assessment of the VUI. In fact, all
things being considered, this system exhibits a solid
appreciation of VUI design best practices. For example:

  • Its
    Main Menu includes only four choices, one of which
    is to speak with an agent.
  • It
    is easy to reach an agent either verbally or by
    entering DTMF 0 at any point after the web ad.
  • The
    system supports a simple DTMF backup scheme behind
    its menus.
  • Its
    error logic presumably checks for evidence of “weak
    cell phone signal”, “speakerphone echo or some other
    background noise”.
  • A
    Spanish option is offered only after a transfer has
    been initiated.
  • And
    overall production quality is high.

In
summary, the VUI is in pretty good health but it does
however suffer from some behavioral problems and a mild
personality disorder. It could be easily improved by
shortening the greeting, eliminating the web ad,
eliminating the help feature, trimming most of its
prompt and finally, de-animating its anthropomorphic
persona.

The Prognosis

The cure is simple.
Improvement is all but assured.

 



May 3, 2006

Walt & Walter’s VUI View:
On the United States Postal Service (USPS) Customer
Support Line

The Caller’s View from
Walt

The USPS support number is
800-275-8777. My local post office has stopped listing
their local number in the phone book. As a consequence,
I have to call a nationwide 800 number to obtain
information about the local post office. After being
told how to get a Spanish menu and instructed how to get
tax forms from the IRS, a 4 item menu was presented. I
could get a zip code, mailing rates, track & confirm a
delivery, or do something with my delivery services.
Since I wanted the hours of the local post office, and
none of these gave me this, I selected the “More
options” choice. I now found that I could change my
address, schedule a pickup, buy stamps, and finally get
Post Office Information. When prompted, I provided my
zip code and the system spoke back the location of my
post office, the hours and the telephone number. Despite
the shortcomings, the postal service system worked
reasonably well. The Spanish option and the IRS forms
announcement were a irritating but they did it so
quickly that the pain was not that huge. I guess that
when you are a legal monopoly, such as a government post
office, you get pretty efficient at abusing people. The
order of the options did not make a lot of sense. I
can’t imagine many folks calling to: obtain a zip code,
get a mailing rate, or track & confirm a delivery. Yet –
stopping and re-starting delivery is something that is
commonly done. The system does not have CTI implemented,
so that it could not provide a valid queue wait estimate
and I had to provide all of the information all over
again to the human that I finally talked to. When I
tried to get to an agent, the system at first ignored
me. Then it insisted that I provide information that
identified what I was calling about This became even
more irritating when I got to the agent and found that
the information had not been provided to the agent. The
system putting so many options on the menu makes
everything confusing and difficult to use. Attempting to
automate this many functions is a misuse of the
technology. They should get rid of all but the top 2 or
3, in terms of usage.

Dr. Walter Rolandi
examination of USPS VUI

The Examination Session

I called the USPS system at
1-800-275-8777 two times. My first objective was to
simply navigate to the systemҳ Main Menu. The objective
of the second call, as per Walt Tetschnerҳ request, was
to get the location and hours of operation of my local
post office. Although the systemҳ Main Menu is tiered,
getting to and hearing the first or main component of
the Main menu took only 29 seconds.

The system answers with a
brief greeting followed by instructions for Spanish
speakers. The Spanish language option adds 4 seconds to
the call. The system then played a 10 second message
about tax forms and how to contact the IRS. Thus while
it only takes 29 seconds to hear the first tier of the
systemҳ Main Menu, 17 of those seconds are consumed by
the greeting, the Spanish instructions and an IRS
message.

The first tier of the Main
Menu is as follows:


Find a
zip code

Get mailing rates

Track and confirm

Delivery services

Users are also encouraged to
say, “More options” whereupon the following choices are
provided:


Change
of address

Schedule a pickup

Buy stamps

Say Post Office
Information (for post offices hours and locations)

Finally, the system
encourages users to say, “Other information”. When I
obliged, the system asked me if I was calling about
“passport information”. I answered, “no” and was
transferred. The events in this session required 67
seconds to complete.

The second session was
essentially equal to the first up to the point where I
was prompted to say, “Post Office Information”. Saying
“Post Office Information” provided no echoic feedback.
It did however produce two beeps that suggested being
transferred followed by 5 seconds of dead air. The
system then asked me to specify, “My Post Office” or any
other post office. It allowed only 2 seconds to respond
however and interrupted me as I said, “My Post Office”.
This generated a recognition failure, replayed the
prompt and obliged me to repeat my request.

This branch of the system
seemed to employ either mixed female voices or have
quality issues with some of the prompts. The actual
information about “My Post Office” was read out rather
abruptly in a male voice.

The system required 133
seconds to complete the task.

The Diagnosis

This is a relatively healthy
VUI, particularly when one considers just how sick most
others are. It does suffer from one significant problem
(#1 below) but most of its other issues relate to its
production quality.

  1. Like
    many IVR systems, the system offers too many
    choices.
  2. At
    least one turn-taking timing variable needs to be
    adjusted.
  3. Some
    prompt content, (particularly within the initial 17
    seconds) is unnecessary.
  4. The
    system apparently uses an odd and uninformative
    audio-icon for user feedback.

  5. There were pointless silences in several areas of
    the dialog.
  6. The
    system mixes voice talent in an apparently
    meaningless way.

The Prescription

This system is actually in
pretty good shape. It could be even better however if it
were to shed a few of the least requested tasks, trim a
few prompts, tune some timer variables, provide more
meaningful feedback and use voice talents in a more
consistent fashion.

The Prognosis

Improvement is possible and
quite likely.

Contact: Walter Rolandi,
Ph.D., The Voice User Interface Company, LLC +1.
803.252.9995,



wrolandi@wrolandi.com



www.wrolandi.com

 



April 6, 2006

Walt & Walter’s VUI
View: on AOL’s Customer Support Line

The Caller’s View
from Walt

The AOL support number is
800-827-6364. This is also the line for new subscriber
signup, for upgrading from dial-up to broadband and for
service cancellation. Does not answer until the 4th
ring. After the greeting (1st female voice), you get an
emergency announcement (male voice that sounds like the
same voice that BofA is using) that a release of Norton
anti-virus has a bug and that you should call another
number if you are using Norton anti-virus software. I
don’t use Norton anti-virus software so this was just a
waste of my time. Using the same number for functions
that are totally separate is a fundamental mistake. The
first thing that they do is to waste the caller’s time
by getting them to identify which of the three things
they are calling about and then re-routing them. Using
DNIS would be an efficient way of doing this. Aside from
not irritating the caller by wasting their time, it
would cost AOL less since the telephone connect times
are reduced. Don’t use caller ID to obtain my telephone
number which they never use anyway. After you identify
that you are a subscriber by saying “yes” to the
question: “are you a subscriber”, the system then
gathers additional information such as your phone number
and the first three letters of your screen name. Despite
never actually using any of this information for
anything, it wastes your time by laboriously reading it
back to you a letter at a time. Finally, you get to an
eight item menu. Despite having wasted time identifying
that I was a subscriber and providing my phone number
and screen name, the menu includes items for a
non-subscriber to sign up. I found that you can avoid
having to provide the unused telephone number and screen
name by telling the system that you are a
non-subscriber. Actually, for technical support calls,
the fastest way is to just keep hitting the 0 key.
Amazingly, an item that they include on the menu is
service cancellation. After you select technical support
from the menu, you need to identify the sort of computer
that you are using and the broadband connection that you
have. Finally, you have to state what the problem is.
I’ve found that whatever you state as the problem, is
interpreted as an AOL connection problem. Don’t argue
with it &ndash just agree that it’s an AOL connection
problem. Doesn’t matter since none of this information
is used for anything. These folks are just jerking the
caller around. Finally you get put into a queue. AOL
doesn’t have C-TI implemented which means that they
can’t estimate the wait time with any precision so they
just have the guy with the BofA voice announce that it
will be at least so many minutes. You now get 10-20
minutes of music interrupted every few minutes by the
BofA guy informing me of another wonderful feature that
AOL has to offer. Finally, I get to the agent. She does
not have any of the information that I had provided to
the IVR. Unfairly – I dump on her. No wonder that the
agents hate the IVR. Shortly after every contact with
AOL customer service, you receive an e-mail survey from
AOL which claims to attempt to find out how you liked
your customer service experience. This survey is a joke.
The only thing that it asks questions about is the
performance of the agent. AOL pretends to be unaware of
how really awful their telephone self-service really is.
They pretend to care about their subscribers by beating
on the weakest and least able to defend themselves.

 

Dr. Walter Rolandi
examination of AOL VUI

The Examination
Session

I called the AOL system
using account login information provided by Walt
Tetschner and navigated to the system’s Main Menu. This
took 181 seconds. After an initial greeting, the system
plays a 60-plus second prompt about a problem apparently
caused by Norton Utilities. It basically instructs
callers who think they are having this problem to call
another number. Spanish is then offered to the caller
(after over a minute of unsolicited English).

Before arriving at the Main
Menu, the system asks the caller if he is an AOL member,
inviting the caller to say, “yes, no or I don’t know”. I
failed to respond when prompted twice, causing two
no-response or timeout messages to be read. I then
provided a phone number and some alphabetic information
in order to complete the login process. In all, I made
five seemingly appropriate utterances, experienced two
recognition failures and one confirmation.

After listing its six Main
Menu choices, the system prompted that “Help” was
available if further explanation were needed. Saying,
“help”, initiates a 45 second prompt which essentially
restates the system’s Main Menu choices. Note that the
first choice is to “sign up” with AOL.

The session, up to this
point, required over 225 seconds.

The Diagnosis

A substantial number of VUI
infirmities and functional inadequacies were evident:

1. The overall system
organization seemed irrational.

2. The system had difficulty
detecting speech acts.

3. Error timers seem
unnatural.

4. Some menu options
appeared irrational.

5. Many, if not most prompts
were unnecessarily long.

6. The system employs an odd
audio-icon for user feedback. It does so inconsistently
and ineffectively.

7. The system was unbearably
animated.

8. The system is far more
complicated than it need be.

9. The system provides
insufficient movement for the user towards task
completion and therefore leads to user frustration.

10. “Help” was of no help.

11. The system seems
structured in such a way as to painfully discourage
transfers to humans.

12. Even though the system
tells the user that he will soon be transferred, it
repeatedly interrogates the user for additional
information.

The Prescription

This system is a mess. It
may suffer from basic inabilities to detect user speech
acts. It also appears to be stupid: after confirming
that the user is an AOL user, the system offers the user
the option to become an AOL user. This is basically
irrational. Trimming prompts, eliminating
anthropomorphic approximations and other cosmetic
changes could improve the system somewhat but its
organization is fundamentally flawed. Much more
substantial changes would be necessary to overcome its
profound shortcomings.

The Prognosis

Improvement is possible but
it is not likely to result from a casual intervention.
Major surgery is probably required.

Contact:
Walter Rolandi, Ph.D., The Voice User Interface Company,
LLC +1. 803.252.9995,



wrolandi@wrolandi.com



www.wrolandi.com

 

 



March 3, 2006
 

Walt & Walter’s VUI View
(WWVV): on Bank of America’s Consumer Telephone Banking
Telephone Self-service


 

The Caller’s View
from Walt

Back in the July, 2005
issue, we reported on the lousy speech-enabled
implementation that BofA has foisted on me. At that time
the system was relatively new. Over 6 months have gone
by and the excuses that it is having birthing pains and
the problems will be resolved soon, have become a bit
lame. Everytime that I call, I still get provided with
detail instructions on how to disable the speech
service. I called them on February 22nd (two
full days after the holiday) and they were still
including a special message telling me that they were
closed on the holiday and that my account information
would be updated after the holiday. CallerID is not
implemented, so they require me to identify myself by
entering my Social Security Number, which they call my
Access ID. After I’ve followed their directions and
disabled the speech recognition (managed to do this
without crashing the car), I find that it’s really
difficult to enter my 9-digit Social Security Number and
continue driving safely. After I get out of self-service
and go to a human (BofA calls them Specialists), they do
not have any information re the information that I
provided to the self-service. Of course, without C-TI,
they don’t know how many calls are in the queue, so they
can’t inform me how long the wait will be. What is wrong
with these people? They implement speech but not C-TI?
Just ask the callers what they want. Answer is always
the same: They want a quick-&-easy experience! The lack
of any intelligent telephony in this application is
appalling. Using CallerID to recognize who I am would
eliminate the need to provide my Social Security Number
which saves me time and is safer while driving. Having
to repeat information to the agent that I have
previously provided to the IVR is really irritating and
wastes more of my time. The entire BofA telephone
self-service application is slow-&-hard. Long and
unnecessary instructional prompts. I’ve been using this
system since last July. It hasn’t improved since then
which would suggest that it has limited adaptive
capabilities. It would also suggest that BofA has little
real interest in meeting the needs of their callers
(customers). This is even more ugly because the speech
overstrike is disabled and you need to listen to all of
these long prompts. Doesn’t recognize “natural numbers”
being spoken. During the wait queue, BofA has decided to
dump an advertisement on the callers. Captive audience
&ndash no place to go! I don’t at all remember what the
advertisement was all about which suggests how effective
it is. Some things improve with age. This telephone
self-service application from BofA is not one of them.
The BofA phone number for consumers to access their
account information is 800-841-4000. This can also be
accessed by calling 800-900-9000 and selecting option 4.


 

Dr. Walter Rolandi
examination of Bank of America VUI:

The Examination
Session

I called the BoA system
using account login information provided by Walt
Tetschner and navigated to the system’s Main Menu. This
took 99 seconds. I made seven situationally appropriate
utterances, experienced three recognition failures and
one confirmation. The entire session lasted 125 seconds.

The Diagnosis

A number of VUI infirmities
were evident:

  1. The overall pace of
    the system was painfully and artificially slow.

  2. Some intra-prompt
    timers were inappropriate and thus caused turn
    taking errors.

  3. Several menu options
    offered unnecessary, unnecessarily complicated
    or confusing choices.

  4. The system provides
    excessive instructions, presumably intended to
    help the user.

  5. Persona excesses
    were evident.

  6. The system’s speech
    recognition recovery strategy is inefficient.

  7. The system
    ineffectively models the responses it seeks from
    its users.

  8. The system’s speech
    confirmation strategy is questionable and can
    appear silly.

My isolated confirmation
experience tells the “silly” story: After offering the
Main Menu choices, the system eventually suggests:

System: “To
end this call you can say, “goodbye” or just hang up.”



Obligingly, I said, “Goodbye” to which the system
replied:



System: “You said, “goodbye”. Is that correct?”



Here is a prime example of how pretending to be
“natural” or “conversational” can produce silly and
annoying results.

The Prescription

This system could be
considerably improved by adjusting timer values and
better turn taking modeling. Over twenty-five of the 99
seconds required to hear the system’s basic Main Menu
choices consisted of “dead air” between individual
system prompts and between system responses to user
utterances. At least a third of this could be
eliminated. This would move the user through the system
faster and shorten the overall mean call duration.

Several menus could be
significantly improved if simplified and/or shortened.

Almost all prompts are
longer than they need to be. Most system prompts would
benefit from a decisive application of Rolandi’s Razor.
(
http://www.speechtechmag.com/issues/9_4/human_factor/).

All attempts to affect a
human-like personality should be eliminated as well as
most of the system’s expressions of gratitude.

Finally, the system would
greatly benefit from a less conversational and more
informative error recovery strategy.

The Prognosis:
Significant improvement is likely. Walt adds that this
is only so if BofA has the good sense to realize that
providing poor customer service as they are doing is not
good business and invests in the repairs that are
necessary.

Contact: Walter Rolandi,

803-252-9995,



wrolandi@wrolandi.com

 

 

 

 

 

Copyright © 2019 ASRnews