Interface and method of designing an interface6778643Abstract A method of designing an interface system that allows users to map the representation of their task directly to the interface. There are three major phases to the Customer-Centric Approach to Interface Design (C-CAID). End-users' tasks are categorized to determine the frequency of reasons or tasks of why users interact with a particular system. These reasons and their relative frequencies are used to design interface options that emphasize the user's task categories. Finally, the customer-centric interface designs are evaluated and compared with existing system interfaces using usability tests with actual users performing the tasks. The results from usability tests are used to pinpoint the task-option combinations that do not work well and which should be revised. Benefits of this customer-centric design are improved systems performance and increased user satisfaction. Claims What is claimed: Description BACKGROUND OF THE INVENTION
TABLE 1
Number of calls to small business call centers
Call Center Type Call Center Location Number of Calls
AA TX1 561
AR1 222
AA Subtotal = 783
NN MO1 175
TX2 31
NN Subtotal = 206
RR TX3 195
TX4 151
OK1 77
MO2 174
RR Subtotal = 597
CC MO2 495
CC Subtotal = 495
MM OK2 310
MM Subtotal = 310
TOTAL 2391
FIG. 1, step S21, shows that as the service representatives at these centers receive calls, they log the customer's opening statement or reason for calling using an IVR Call Log Sheet that can be found in Table 2. As is readily apparent, utilizing a Call Log Sheet such as the one shown in Table 2, a service representative can easily record the reasons for customers calls for a number of customers and the length of each call.
TABLE 2
Sample Call Log Sheet
Small Business IVR Call Log Sheet Date:
Service Center: .sub.----------
CSR: .sub.--------------
Call Length
Call Reason/Purpose of Customers Call (minutes)
Example: "I'd like to find out the cost of adding two 3.5
more business phone lines."
1
2
3
4
5
6
7
8
9
10
The C-CAID is based upon on the user's representation of a task to be performed while interacting with a particular system interface. FIG. 1, step S2 indicates that user-centric statements should be applied to the design of the interface to assure that items are clearly mapped to the task options that the system can present to the user for selection. User-centric task statements are collected from a sample of users and categorized based on the tasks to be performed by the system. Each task statement is classified or categorized into one of about seventy customer-centric task categories as shown in step S22 of FIG. 1. These customer-centric task categories were logically derived from a general consumer model. A general consumer model is a model based upon the potential tasks that a customer might require of an organization (e.g. telephone company), but does not have to be tailored to the unique organizational structure of a specific telephone company. Subcategories are generated to capture greater details of the lower level general task categories. The categories are then rank-ordered by the number of statements they represent. The IVR Call Log Sheets (Table 2) are electronically recorded and task categories are coded in spreadsheets for analysis. Examples of customer task statements are illustrated in Table 3. They show how many customer statements can be grouped into a higher level category that characterizes the same general information content. It should be noted that the data collection method mentioned above can be performed either manually or automatically. Statistics can be accumulated via a data extraction or data collection algorithm designed to collect data without changing any of the intent, goals or effects of the present invention. This process automation can be applied to any of the steps found in the process flow diagram depicted in FIG. 1.
TABLE 3
Example customer statements to small business call centers.
Call Reason/Purpose of Customers Call Task Category
1 "I'd like to find out the cost of adding two Get information
more business phone lines."
2 "I want to order an ISDN line." Acquire services
3 "I need some help on how to use call Information request
forwarding."
4 "I need to get someone to look at my broken Fix a service
phone." problem
5 "I called to check on my move orders." Relocate service
Before coding the customer task statements into task categories it is important to validate all of the task categories as shown in S23 of FIG. 1. Table 4 lists the different types of task categories than can be used in the design of the small business customer-centric IVR.
TABLE 4
Customer Statement Categorization list
Acquire Service
A0 Unspecified Addition
A1 Request new phone service (Open an Account)
A2 Add Optional Services
A3 Schedule Pending Acquisition
A4 Get Info about a Pending Acquisition
A5 Give Info for a Pending Acquisition
A6 Change a Pending Acquisition
A7 Cancel a Pending Acquisition
A8 Reconnect Service
A9 Acquire Service Temporarily
Discontinue Service
D0 Discontinue Unspecified Service
D1 Disconnect (Close Account)
D2 Discontinue Optional Service
D3 Schedule a Pending Deletion
D4 Get Info about a Pending Deletion
D5 Give Info for a Pending Deletion
D6 Change a Pending Deletion
D7 Cancel a Pending Deletion
D9 Discontinue Service Temporarily
D11 Return a Product
Fix a Service Problem
F0 Fix an unspecified problem
F1 Fix a Service
F2 Fix a Product
F3 Schedule a Pending Repair
F4 Get Info about a Pending Repair
F5 Give Info about a Pending Repair
F6 Change a Pending Repair
F7 Cancel a Pending Repair
F8 Report a Problem
F9 Fix a Problem
Relocate Service
M0 Move unspecified items (I'm moving.)
M1 Request a Move of Service
M3 Schedule a Pending Move
M4 Get Info about a Pending Move
M5 Give Info for the Pending Move
M6 Change a Pending Move
M7 Cancel a Pending Move
M9 Move Service Temporarily
Change Account Information, Services, or Service Features
C0 Change Unspecified
C1 Change Account Data
C2 Change Optional Services
C8 Change a Feature of a Service
C9 Make a Temporary Change
Bill Issues
B0 Unspecified issue with the Bill
B4 Get Information about the Bill
B5 Give Information for the Bill
B6 Dispute the Bill
Pay Arrangement or Report a Payment
P0 Inquire about Payment Options
P1 Set Up Payment Arrangement
P2 Where to Make a Payment
P3 Schedule Payment
P4 Get Info about a Pending Payment
P5 Give Info for a Pending Payment
P6 Change a Pending Payment
P7 Cancel a Pending Payment
Information Requests
I0 Nature of inquiry unspecified
I1 On Services
I2 My Account
I3 Other Service Providers
I11 Other Company Offices
I12 Name/Address/Number
Long Distance/Other Service Provider
L1 Add a Carrier/Provider
L2 Restore a Carrier/Provider
L4 Get Info from the Carrier/Provider
L5 Give Info to the Carrier/Provider
L6 Change Carrier/Provider
L7 Cancel a Carrier/Provider
L8 Billing Issue for a Carrier/Provider
L9 Payment for a Carrier/Provider
L11 PIC Letter
Validation is the process whereby one tests all of the proposed call mapping categories to make sure that they adequately cover all of the reasons that a customer may call the organization for which the IVR is being developed. The interface designer wants to make sure that the IVR can handle any possible reason that a customer would be calling. Different subsidiaries of a company may have different task categories than the ones used for the small business customer-centric IVR. The task categories listed in Table 4 were originally developed for the consumer customer-centric IVR. However, it may be necessary to test the proposed call mapping categories to make sure that they adequately cover all the reasons a customer calls the subsidiary for which the IVR is being developed. This process involves taking a sample of randomized, customer reasons for calling a center and mapping these reasons to the proposed categories. Going through this process allows the designer to amend the task categories to better fit the call center that will interface with the IVR. This approach allows adding or deleting various categories, reorganizing them entirely, or reorganizing any combination thereof. The metrics tabulated in Table 5 through Table 9 illustrate the most frequent reasons that small business customers call the different call centers listed in Table 1.
TABLE 5
Why customers call AA.
Category Description Category %
B4 Get Information about the Bill 33.9%
I2 Get Information on my Account 17.5%
C1 Change Account Data 6.8%
D1 Disconnect (Close Account) 5.4%
B6 Dispute the Bill 5.3%
I1 Get Information on Services 4.7%
D2 Discontinue Optional Service 3.7%
A2 Addition Optional Services 2.6%
P4 Get Info about a Pending Payment 2.6%
L6 Change Carrier/Provider 2.1%
I11 Get Information on other Company 1.8%
Offices
L2 Restore a Carrier/Provider 1.3%
P1 Set Up Payment Arrangement 1.3%
A8 Reconnect Service 1.2%
P5 Give Info for a Pending Payment 1.2%
F1 Fix a Service
P0 Inquire about Payment Options 0.9%
D4 Get Info about a Pending Deletion 0.8%
I0 Nature of inquiry unspecified 0.8%
I12 Get Information on a 0.8%
Name/Address/Number
M1 Request a Move of Service 0.8%
C2 Change Optional Services 0.7%
L7 Cancel a Carrier/Provider 0.7%
A4 Get Info about a Pending Acquisition 0.4%
A1 Request new phone service 0.3%
L5 Give Info to the Carrier/Provider 0.3%
P3 Schedule Payment 0.3%
A5 Give Info for a Pending Acquisition 0.1%
C8 Change a Feature of a Service 0.1%
D9 Discontinue Service Temporarily 0.1%
F2 Fix a Product 0.1%
F8 Report a Problem 0.1%
L4 Get Info from the Carrier/Provider 0.1%
M4 Get Info about a Pending Move 0.1%
P2 Where to Make a Payment 0.1%
Total 100.0%
A0 Unspecified Addition 0.0%
A3 Schedule Pending Acquisition 0.0%
A6 Change a Pending Acquisition 0.0%
A7 Cancel a Pending Acquisition 0.0%
A9 Acquire Service Temporarily 0.0%
B5 Give Information for the Bill 0.0%
C0 Change Unspecified 0.0%
D0 Discontinue Unspecified Service 0.0%
D11 Return a Product 0.0%
D5 Give Info for a Pending Deletion 0.0%
F0 Fix an unspecified problem 0.0%
F3 Schedule a Pending Repair 0.0%
F9 Fix a Problem 0.0%
I3 Get Information on other Service 0.0%
Providers
L1 Add a Carrier/Provider 0.0%
L11 PIC Letter 0.0%
L8 Billing Issue for a Carrier/Provider 0.0%
M5 Give Info for the Pending Move 0.0%
M9 Move Service Temporarily 0.0%
A customer-centric design approach takes the most frequent reasons and uses them as guidance to identify and define the arrangement of menu topics to reflect this frequency of occurrence. Table 5 illustrates the task categories or reasons that customers call center AA. The most frequent category is to "Get Information About the Bill and accounts for 33.9% of the calls. The second most frequent task category occurs in only 17.5% of the calls ("Get Information On My Account"). For the task categories with a frequency above 1.0%, 6 of them are about "Information" and account for 61.7% of the customer calls. This suggests that "Information" should be the prominent topic in a customer-centric IVR menu for small business customer calls to call center AA. Table 6 illustrates the task categories or reasons that customers call center CC. The top category is to "Change Carrier/Provider" and accounts for 29.5% of the customer calls. The second most frequent task category accounts for 20.5% of the customer calls ("Get Information On My Account"). For the task categories with a frequency above 1.0%, 5 are about "Information" and account for 44.5% of the customer calls, and 5 are about "Carriers" and account for 43.2% of the customer calls. This suggests that "Information" and topics relating to "Carriers" should be prominent topics in a customer-centric IVR menu for customer calls to the call center designated as CC. Table 7 illustrates the task categories or reasons that customers call RR. The top category is to "Get Information on Services" and accounts for 20.4% of the customer calls. The second most frequent task category called is add optional services and accounts for 16.9% of the customer calls. For the task categories with a frequency above 1.0%, 5 are about "information" and account for 40.0% of the customer calls; 3 are about "optional services" and account for 24.9% of the customer calls. This suggests that "information" and topics relating to "optional services" should be prominent topics in a customer-centric IVR menu for customer calls to call center RR. Table 8 illustrates the task categories or reasons that customers call MM. The top category is to "Get Information on Services" and accounts for 36.1% of the customer calls. The second most frequent task category is to get information about other company services and accounts for 14.0% of the customer calls. The third most frequent category is to add optional services and accounts for 13.7% of the customer calls. For the task categories greater than 1.0%, 7 are about "information" and account for 67.1% of the customer calls. This suggests that "information" should be the prominent topic in a customer-centric IVR menu for customer calls to center MM. Table 9 illustrates the task categories or reasons that customers call NN. The top category is to "Request New Phone Service" and accounts for 20.1% of the customer calls. The second most frequent task category is to get information about services and accounts for 18.0% of the customer calls. The third most frequent category is to request a move of services and accounts for 10.1% of the customer calls. For these task categories, 7 are about "information" and account for 40.8% of the customer calls; 4 are about "services" and account for 39.7% of the customer calls. This suggests that "information" and topics relating to "services" should be prominent topics in a customer-centric IVR menu for customer calls NN. An important part of the data collection process shown in FIG. 1 step S1 is to collect end-user volume data as shown in step S3 and to analyze the volume data as shown in step S4. Often an IVR designer wants to provide customers with a single access telephone number that can be used for many or all of the centers. Such an arrangement reduces misdirected calls because the end user or customer will not have to guess which of the many company telephone numbers to call. So the question becomes, what if one combines the AA, NN, RR, CC and MM call centers by use of a single customer-access telephone number. In other words, what would the customer-centric call frequency show? To accurately depict a combined and adjusted task frequency as shown in FIG. 1 step S5 across the various call centers it is important to consider the call volume generated to each center. Call volume for each call center in the sample is used to weight the frequency of its different task categories. This is done because a call center that handles billing may get twice as much call volume per year compared with a call center that processes new orders. This volume weighting adjusts task category percentages to account for different call volumes and provides a more accurate analysis of task frequency. The first step in adjusting task frequency for call volume is to consider total call volume for each small business center. First total call volume for each small business call center in a sample is obtained. Based upon data obtained from call volume during a predetermined time period, AA had approximately 610,059 calls, CC had approximately 51,374 calls, RR had approximately 573,871 calls, MM had approximately 40,260 calls, and NN had approximately 308,161 calls. This indicates a 39% weight for AA, a 19% weight for NN, a 36% weight for RR, a 3% weight for CC, and a 3% weight for MM. Table 10 illustrates the relative call volume of the different small business call centers.
TABLE 10
Relative call volume of different small business call centers.
Small Business Calls Received Relative Call
Call Center During Time Period Volume
AA 610,059 39%
NN 308,161 19%
RR 573,871 36%
CC 51,374 3%
MM 40,260 3%
Total 1,583,725 100%
After call volume weights have been computed, task frequency for each call center type is adjusted by multiplying task frequency by the call volume weight. Table 11 illustrates the steps involved in computing adjusted task frequency based upon volume data, as shown in step S5 of FIG. 1.
TABLE 11
Example computation of adjusted task frequency based on
relative call volume.
AA Overall
Call AA NN RR CC MM Adj
AA Vol Adj Adj Adj Adj Adj Task
Task Categories % Wght Freq Freq Freq Freq Freq Freq
Get information .34 .39 13.2% 1.4% 1.0% 0.2% 0.1%
15.9%
about the bill
Get information .05 .39 1.8% 3.4% 7.3% 0.2% 1.1%
13.8%
on services
Get information .18 .39 6.8% 1.2% 3.0% 0.6% 0.2%
11.8%
on my account
The first column shows the task categories to be adjusted by call volume. As an example we will consider the first row task category "Get Information About the Bill". The second column in Table 11 (AA%) represents the percentage of calls made to AA for each task category. In this case, 33.9% of the calls to AA were to "Get Information About the Bill" (see Table 3). The third column in Table 11 (AA weight volume) represents the relative call volume weighting for AA. As shown in Table 10, the relative call weighting for AA was 39%. Column 4 shows the adjusted task frequency which was computed by multiplying the percentage of calls relating to "Get Information About the Bill" by the relative call volume weight. In this example, the adjusted frequency for calls to AA relating to "Get information about the bill" is 13.2% (i.e., 33.9%*39.0%=12.9%). These steps were also performed for the other small business call centers to obtain the adjusted task frequency values in columns 5-8. The adjusted task frequencies for each call center type are then summed to obtain the overall adjusted task category frequency which is represented in the last column (i.e., column 9) of Table 11. To complete the example, the overall adjusted task frequency for "Get Information About the Bill" is approximately 15.9% (i.e., 13.2%+1.4%+1.0%+0.2%+0.1%). The results appear in Table 12 and show the adjusted task category frequencies or reasons customers called all Small Business centers after adjusting or compensating for differences in call volume.
TABLE 12
Why customers call AA, CC, RR, NN and MM.
Category Description Combined %
B4 Get Information about the Bill 15.92%
I1 Get Information on Services 13.85%
I2 Get Information on my Account 11.82%
A2 Add Optional Services 8.98%
C1 Change Account Data 6.25%
A1 Request new phone service 4.67%
D1 Disconnect (Close Account) 4.19%
M1 Request a Move of Service 3.38%
D2 Discontinue Optional Service 3.30%
A4 Get Info about a Pending Acquisition 3.18%
L6 Change Carrier/Provider 2.85%
I11 Get Information on other company 2.54%
Offices
B6 Dispute the Bill 2.53%
C2 Change Optional Services 1.92%
F1 Fix a Service
P4 Get Info about a Pending Payment 1.14%
A8 Reconnect Service 1.14%
A5 Give Info for a Pending Acquisition 0.96%
L4 Get Info from the Carrier/Provider 0.89%
L2 Restore a Carrier/Provider 0.82%
P5 Give Info for a Pending Payment 0.70%
L5 Give Info to the Carrier/Provider 0.70%
D4 Get Info about a Pending Deletion 0.63%
P1 Set Up Payment Arrangement 0.62%
L1 Add a Carrier/Provider 0.52%
C8 Change a Feature of a Service 0.45%
A3 Schedule Pending Acquisition 0.45%
P0 Inquire about Payment Options 0.42%
M4 Get Info about a Pending Move 0.42%
I0 Nature of inquiry unspecified 0.41%
L7 Cancel a Carrier/Provider 0.39%
M5 Give Info for the Pending Move 0.37%
I12 Get Information on a 0.32%
Name/Address/Number
F2 Fix a Product 0.28%
A7 Cancel a Pending Acquisition 0.26%
P2 Where to Make a Payment 0.18%
A9 Acquire Service Temporarily 0.17%
D5 Give Info for a Pending Deletion 0.13%
F8 Report a Problem
D9 Discontinue Service Temporarily 0.11%
P3 Schedule Payment 0.10%
A0 Unspecified Addition 0.06%
A6 Change a Pending Acquisition 0.06%
B5 Give Information for the Bill 0.06%
C0 Change Unspecified 0.06%
D0 Discontinue Unspecified Service 0.06%
F3 Schedule a Pending Repair 0.06%
L11 PIC Letter 0.06%
M9 Service Temporarily Move 0.06%
D11 Return a Product 0.04%
F0 Fix an unspecified problem 0.02%
F9 Fix a Problem 0.01%
I3 Get Information on other Service 0.01%
Providers
L8 Billing Issue for a Carrier/Provider 0.01%
Total 100.0%
As is evident, the top task category is "Get Information About the Bill", and it accounts for 15.92% of all calls. The next most frequent task category is "Get Information on Services", and it accounts for 13.85% of all calls. In fact, the top three reasons why customers call is to: 1) inquire about the monthly bill; 2) inquire about products, services and prices; or 3) inquire about their account. These three information inquiries are responsible for 41.59% of all calls. Further, among the task categories with a frequency greater than 1.0%, 6 are about "information" and account for 48.45% of all calls; and 8 task categories are about "services" (i.e., new, changing, moving or disconnecting service) and account for 35.54%; of the customer calls. This suggests that tasks relating to "information" and "services" should be prominent topics when designing a customer-centric IVR menu for Small Business Call Centers. As seen in FIG. 4, the top ten tasks or reasons customers call small business centers account for 72.2% of the calls in the sample and the top 14 tasks account for 81.6% of the calls. That is, the overwhelming majority of calls to small business centers can be categorized by less than 15 tasks. This means that when designing the IVR menu options S6, one should focus on high frequency tasks because they are the most likely to be of interest to and requested by the small business customers. The most frequent tasks are located early in the IVR menu selections to quickly address common calls; however, less frequent categories may be grouped with more frequent categories for logical reasons. For example, in the customer-centric design approach to the small business IVR, we combine the frequent task categories into menu options that are ordered by the percentage of calls for which they account (see Table 12). As noted above, there are six categories involving "Get Information." Three of them (rank orders 1, 2, and 3) concern either the customer's account, the bill or the products and services. These three account for 41.59% of the calls. Therefore, these tasks are used to word the Top-Level Menu "Get information about your account or our services". All categories relating to the "Get Information" option appear in an associated Second-Level Menu as shown in Table 13. According to the data collected organized and analyzed as discussed above and as shown in the steps S1-S5 of FIG. 1 menu options of the interface are defined in step S6. A sample of such an IVR are shown in Table 13.
TABLE 13
Example Top-Level, Second-Level, and Third-Level IVR Menu Options.
Top-Level Menu
Choices Second-Level Menu Choices Third-Level Menu Choices
To get information For information about an For bill items other than
about your account or item on your bill, press 1 DSL or ISDN, press 1.
our services and prices, For DSL bill items, press
2.
press 1. For ISDN bill items, press 3.
For DSL or ISDN bill items,
press 4.
For all other billing
questions, press 5.
For information about our If you have an existing
services, products, and account, press 1.
prices, press 2. If you are a new account,
press 2.
For the information we have about your account, press
3.
For information about a For repair, press 1.
repair or installation order, For installation, press
2.
press 4.
For information about an order for additional lines or
services, press 5.
For information about high- For DSL, press 1.
speed data services, press 6. For ISDN, press 2.
For all other questions about
high-speed data services,
press 3.
Within any business, there is often an organizational need to route similar requests to different agents for task handling and execution. For example, staffing levels or specialization of training at different call centers may dictate distinguishing among similar generic tasks on some basis that may be transparent, irrelevant or unknown to the customer. In Small Business, for example, billing questions involving high-speed data products are handled separately by agents who are specialized by technology and according to other areas of expertise. Thus, the Second-Level option "Information About an Item On Your Bill" leads to a Third-Level Menu that differentiates ISDN and DSL and "other." It should be noted that not all IVR menu categories are included in all IVR designs. For example, task categories such as "Get Information About a Pending Acquisition" and "Get Information About Other Company Offices" were not included as options in the current small business IVR although these items account for 5.2% of the call volume. These items were both included in the customer-centric IVR design. As FIG. 1 step S7 shows, after the initial user-interface is designed, a prediction model based on expected user task volume and task frequency is used to provide an early indication of how the interface will perform. A predictive comparison of the newly designed system to the existing system is performed to estimate performance gains (see steps S8, S9 and S10 in FIG. 1). Prototypes of the system are then tested on a sample of users performing real-world scenarios (i.e., tasks) extracted from the initially collected customer-centric data. In an effort to estimate the routing performance of the current IVR design as compared to the customer-centric design, a designer needs to directly compare predicted routing performance of both IVR designs. Estimates of correct call routing were predicted using 2 independent judges (i.e., testers). Each judge reviewed 24 task categories that each contained three real-world scenarios for a total of 72 scenarios. The scenarios were extracted from the initial 2391 customer-task statements that were collected from the customer call centers. The judges read each scenario and attempted to resolve it (i.e., correctly route it) using both the existing and newly designed customer-centric small business IVR. For example: Task Category=Get information about the bill: Scenario #1=I have a question about the charges on my bill. Scenario #2=I need to know to who some of these numbers on my bill belong. Scenario #3=I need to know the balance of my account. Scores were converted to a "routing percentage score" by multiplying percent correct routing by the task frequency adjusted for call volume across call centers. For example, if the two judges examining scenarios correctly routed 5 of 6 scenarios within a task category, then the "routing percentage score" equaled 83.3%. Further, if the task category frequency adjusted for call volume across calling centers was 15.92% then the "Predicted Routing Percentage Score" would be 13.26% (i.e., 83.3%.times.15.92%=13.26%). The "Predicted Routing Percentage Scores" are multiplied and summed across the 24 task categories arrive at the overall predicted routing percentage for each IVR. These scores are used to estimate performance differences between the current small business IVR and the redesigned customer-centric small business IVR. Table 14 describes the menu topics based on call frequency adjusted for call volume.
TABLE 14
Small business customer-centric menu topics based on frequency.
Specific
General Specific Task Routing Predicted
Topics Topics Code Call Freq % Score Score
Get Get B4 15.92% 100% 15.92%
Information information
about the bill
Get I1 13.85% 100% 13.85%
information
on services
Get I2 11.82% 100% 11.82%
information
on my
account
Get A4 3.18% 100% 3.18%
information
about a
pending
acquisition
Get I11 2.54% 33.3% 0.84%
information
about other
company
offices
Get P4 1.14% 100% 1.14%
information
about a
pending
payment
Get L4 0.89% 33.3% 0.28%
information
about a
carrier
Subtotal = 47.03%
Change your Add optional A2 8.98% 83.3% 7.48%
account or services
change your Change C1 6.25% 33.3% 2.08%
services account
information
Delete D2 3.30% 100% 3.30%
optional
services
Request a M1 3.38% 100% 3.38%
move of
service
Change a L6 2.85% 100% 2.85%
carrier
Change C2 1.92% 100% 1.92%
optional
services
Restore a L2 0.83% 83.3% 0.68%
carrier
Get M5 0.37% 33.3% 0.12%
information
about a
pending
move
Cancel or L7 0.39% 100% 0.39%
delete a
carrier
Add a carrier L1 0.52% 100% 0.52%
Subtotal = 22.72%
Open an Discontinue D1 4.19% 100% 4.19%
account or service
disconnect Open a new A1 4.67% 100% 4.67%
services account
Subtotal = 8.86%
Repair Fix a service F1 1.36% 100% 1.36%
Fix a product F2 0.28% 100% 0.28%
Subtotal = 1.64%
Discuss your Dispute the B6 2.53% 100% 2.53%
bill or bill
payments Make P1 0.62% 100% 0.62%
payment
arrangement
Reconnect A8 1.14% 66.6% 0.76%
service
Subtotal = 3.91%
Grand 84.16%
Total =
The "general" and "specific" columns represent the most frequent reasons why customers called small business centers. The "task code" column shows the identifier used when the calls were classified in designing the IVR. The "call frequency" column shows the percentage of calls that were for a particular task; this is the percentage of callers who called the center to accomplish this particular task. The "routing percent score" shows the percentage of tasks correctly routed by the two judges. The "predicted score" shows the theoretical prediction of routing performance based on the adjusted call frequency and the routing percentage score. The results of the analysis provide support for the theoretical prediction that there will be an improvement in correct call routing performance by redesigning an existing small business IVR using a customer-centric approach. Specifically, the results of the analysis predict that the customer-centric design shown in Table 14 will correctly route approximately 84% of customers to the appropriate service center (see Grand Total of 84.16%). For comparison purposes, the predicted routing model was applied to the existing small business IVR. The analysis is shown in Table 15 and suggests that approximately 73% of the customers will be routed to the appropriate service center (see Grand Total of 73.00%) with the existing design. Therefore, a customer-centric approach has the potential to produce a significant increase (i.e., 84.16%-73.00%=11.16%) in the number of callers who are correctly routed. This comparison illustrates a primary recommended use of this approach--to compare related IVR designs as to their relative customer-centric operational performance.
TABLE 15
Existing Small Business IVR: Business-centric prompts,
with frequency data.
General Specific Specific Routing Predicted
Topics Topics Task Code Call Freq % Score Score
Orders and Add or make A2 8.98% 100% 8.98%
product changes to C2 1.92% 100% 1.92%
information service A8 1.14% 50% 0.57%
M5 0.37% 66.6% 0.25%
Establish A1 4.67% 100% 4.67%
new service M1 3.38% 100% 3.38%
or move
existing
service
Telephone I1 3.38% 100% 13.85%
systems or
equipment
Subtotal = 3.62%
Billing Change long C1 6.25% 16.7% 1.04%
Inquiries or distance L6 2.85% 33.3% 0.95%
change long carrier L2 0.82% 100% 0.82%
distance L1 0.52% 100% 0.52%
carrier L7 0.39% 100% 0.39%
Payment P4 1.14% 100% 1.14%
arrangements P1 0.62% 100% 0.62%
Completely B4 15.92% 100% 15.92%
disconnect I2 11.82% 100% 11.82%
service or all D2 3.30% 33.3% 1.10%
other billing B6 2.53% 100% 2.53%
questions L4 0.89% 100% 0.89%
Subtotal = 37.74%
Repair Fix a product F1 1.36% 100% 1.36%
or service F2 0.28 100% 0.25%
Subtotal = 1.64%
Grand 73.00%
Total =
The C-CAID approach to interface design has many tangible and useful results including increased user satisfaction and optimized system performance. One of the biggest problems associated with menu option systems is the occurrence of misdirected calls. These are incoming calls that are "misdirected" or incorrectly routed to the wrong servicing organization. Misdirected calls are a significant problem in both business and consumer markets. An example of the impact that these misdirected calls may have on a business is discussed below. It is estimated that approximately 31% of the total calls to a company's small business call centers (i.e. AA, CC, RR, MM and NN) are misdirects. In 1998, there were approximately 6.2 million calls to those Small Business call centers, resulting in approximately 1.9 million misdirected calls. A recent study indicates that each misdirected call costs approximately $91. Thus, the annual cost of misdirected calls is approximately $1.75 million (1.9 million misdirected calls times $0.91 each). The projected cost savings achieved by implementing the customer-centric small business IVR design is estimated at $400,000 dollars based on the 25% predicted reduction in misdirects for the customer-centric small business IVR. After design of the C-CAID based interface system, the system was tested. Participants in the test were instructed that they were to make two phone calls. One call was to the existing small business IVR and the second call was to the customer-centric IVR. Participants were presented with real-world tasks that were generated using example scenarios obtained from gathered data. Example tasks might require participants to obtain information about a bill or to order an optional phone service. Participants were told to select category options from either the current or redesigned IVR to resolve the particular scenario with which they were presented. After completing both phone calls, participants were administered a short questionnaire to assess their impression of each system. An example questionnaire can be found in FIG. 5. Internal administrations of this survey found that customers greatly preferred an IVR designed using the C-CAID methodology as opposed to ones that did not. Two objective measures are used to evaluate the performance of the customer-centric IVR. These measures are cumulative response time (CRT) and a routing accuracy score and are described in detail below. This evaluation step is shown in FIG. 1 step S11. Two objective measures are collected during usability testing. Cumulative response time (CRT) and task accuracy are used to evaluate performance of the systems. CRT is a measure of the time per task that end-users spend interacting with each system. Task accuracy is used to measure how effectively the goals of the end-user were mapped to each system via the user interface. These two metrics (CRT and task accuracy) are then graphed into a CRT/accuracy matrix (S11 of FIG. 1) that depicts end-user performance for these systems in a convenient and easy to understand manner. This CRT/accuracy matrix is a very powerful tool for the designer to evaluate the strengths and weaknesses of a specific interface design. The designer can observe and evaluate those specific tasks that the end-users are performing poorly while using the interface. From this evaluation, one can redesign the interface design in order to improve performance. If a particular task does not effectively map to the associated interface options then one can expect that the resulting accuracy of the navigation will be poor or higher CRTs will result. The CRT/accuracy matrix is instrumental in identifying the deficiencies of an existing interface system and then serves as a guide and feedback monitor to indicate how introduced changes positively or negatively impact the system as a result of the redesign activity (S12 of FIG. 1). The goal of the retesting efforts is to determine whether performance on the redesigned interface options improved and if the redesign negatively impacted unchanged aspects of the interface design. To calculate the cumulative response time, the time in seconds that subject spends interacting with the IVR is measured. Participants (i.e., subjects) are not credited or penalized for listening to the entire IVR menu before making a selection. For example, if the main menu of the IVR is 30 seconds in length, and the participant listens to the whole menu and then makes a selection, they receive a CRT score of 0. If the participant only listens to part of a menu, hears their choice, and barges in, choosing an option before the whole menu plays, then they would receive a negative CRT score. For instance, if they choose option 3, fifteen seconds into a 6 option, 30 second menu, they would receive a CRT score of -15. Conversely, if they were to repeat the menu after hearing it once, and then chose option three on the second playing of that menu, they would receive a CRT score of +15 for that menu. Referring to the examples illustrated in FIG. 3, the first subject took two seconds after the end of each announcement to make the selection, resulting in a score of 4. The second subject repeated the announcement and made both selections after two seconds, resulting in a score of 12. The third subject made the selection before completion of each announcement and thus received a score of -4. Once the subject reaches their final destination in the IVR (regardless of accuracy), the CRT scores for each menu level are summed, and that becomes the total CRT score for that subject on that particular task. This metric accurately measures the total time a user is interfacing with the system, regardless of menu length. Using this method, it was found that longer IVR menus don't necessarily mean that the participant spends more time in the system. In fact, the customer-centric consumer IVR has menus that measured approximately 90 seconds in length while the current consumer version's menus were about 30 seconds in length. However, CRT scores indicated that although the customer-centric design is almost 3 times longer than the current consumer IVR, the averaged CRT scores for both the customer-centric and current IVR's were not significantly different. The likely explanation for this could be that the subjects got better descriptions, with more detail, in the customer-centric design. Therefore, they did not need to repeat the menus as much. Route scoring provider information as to whether or not the user is routed to the correct call center. A score of -1, 0, or 1 could be awarded for each task. A score of -1 indicates that a subject actively chose an IVR path that did not route them to the correct destination for the task that they were trying to complete. A score of 0 indicates that the subject got to a point in the IVR and made no selection either by hanging up or automatically timing out. Finally, a score of 1 indicates that the subject successfully navigated the IVR for a given task and was routed to the correct destination. Both CRT and routing accuracy (i.e., route scoring) are used to create a CRT/Routing matrix that is used to estimate performance on specific IVR menu options. CRT is plotted on the Y axis and route scoring on the X axis. An example CRT/Routing matrix is depicted in FIG. 2. From this matrix one can determine which tasks are handled well by the IVR, and which tasks are not. Data points located in the upper left quadrant of the matrix are optimal performers because there was both high routing accuracy and a short CRT. In this case, the short CRT time shows that participants are confident that their selected task option matches their goal or reason for calling. Therefore, participants are "barging in" to the menuing system, spending less time with the IVR, and their call is correctly routed. In contrast, data points that fall in the lower right quadrant represent tasks that did not perform well because there was both poor routing accuracy and a long CRT. In this case, participants had to repeat the menus and still did not choose the correct menu options in the IVR. As shown in FIG. 1, step S12, the CRT/routing matrix provides a powerful tool when it comes to evaluating the strengths and weaknesses of the IVR design. For example, a designer can look at the specific tasks that participants perform poorly, and redesign or reword aspects of these IVR menu choices in order to improve performance. That is, if a particular task does not effectively map to the associated menu option then one can expect poor routing accuracy or high CRT's. The CRT/routing matrix can be used to identify deficiencies and tweak the IVR design for optimal performance. Pending completion of the redesigned IVR, the modified design is retested using a similar user sample as in the initial testing sample as shown in step S13 of FIG. 1. The purpose of the retest is to evaluate any changes made to the IVR based on results of the CRT/routing matrix. The goals of retesting efforts are to determine whether performance of the redesigned menu items improves and to determine whether changes made had negative impact on unchanged menus in the IVR. Evaluations of IVR performance using the CRT/routing matrix and retesting are iterative processes and should continue until the design maximizes performance or improvement goals are met. After the initial user-interface is designed, a prediction model based on expected user task volume and task frequency is used to provide an early indication of the how the interface will perform. This establishes a baseline interface that provides an indication of how the new interface will perform. The newly designed interface is then compared to the previous one and performance gain estimates can be determined with respect to the previous interface. The resulting system prototypes are then tested on a sample of users. The users perform real-world scenarios extracted from the initially collected customer-centric data to test the system. As also shown in FIG. 1, upon completion of the redesign test (S13) an updated interface can be designed using the same principles by looping back to step S6 or to another appropriate point. Thus, the interface can be periodically improved. While the invention has been described with reference to a preferred embodiment, it is understood that the words which have been used herein are words of description and illustration, rather than words of limitations. Changes may be made, without departing from the scope and spirit of the invention in its aspects. Although the invention has been described to a particular method, the invention is not intended to be limited to the particulars disclosed herein; rather, the invention extends to all functionally equivalent structures, methods and uses. For example, the interface can be an interface between a user and a computer application or between a user and an e-mail system. Also, the interface can also be between two communicating machines or devices. The communications layers can be optimized so that the machines are set up optimally to communicate more statistically relevant tasks than those communications that may not occur as frequently. In addition, the updating of the design interface may be done at various time increments. Additionally, the information gathered is not limited to customer service requests for a telephonic system. The information can be the data gathered from an online e-commerce system that looks at those items or preferences that occur or are accessed more frequently by the end user. Accordingly, the invention is now defined with respect to the attached claims.
|
Same subclass Same class Consider this |
||||||||||