Subclass 80	Subclass 1	Subclass 2
AUTOMATED ELECTRICAL FINANCIAL OR BUSINESS PRACTICE OR MANAGEMENT ARRANGEMENT

Employee analysis based on results of an education business simulation

6944596

Abstract

Analyzing employees based on results of an education business simulation. A network is utilized to present a training simulation to a user for achieving a training goal. Information is integrated into the training simulated utilizing the network that helps motivate achievement of the goal by the user. Progress of the user towards achieving the goal is measured utilizing the network. For further motivating achievement of the goal, feedback is also provided to the user utilizing the network. There are three levels of feedback, where each is progressively more detailed and is selected based on the number of mistakes made by the user, the user's progress towards the goal is subsequently reported to the employer of the user utilizing the network.

Claims

1. A method for analyzing performance in a network based training simulation, comprising the steps of:

(a) presenting across a network a goal-based training simulation to a user for achieving a goal, wherein the goal is improving a user's decision-making process for his job;

(b) integrating information into the training simulated utilizing the network that motivates achievement of the goal by the user;

(c) measuring progress of the user towards achieving the goal utilizing the network;

(d) providing feedback to the user utilizing the network that further motivates achievement of the goal, wherein the feedback is selected from at least three levels of feedback such that each of the three levels of feed back is progressively more detailed and the level of feedback selected is proportionate to a number of mistakes made by the user;

(e) reporting the measured progress towards achievement of the goal utilizing the network;

(f) comparing the measured progress of the user to a measurement of a subsequent other user's progress; and

(g) gauging effectiveness of the training simulation in teaching achievement of the goal to the user based on the measured progress and the feedback.

2. A method as recited in claim 1, wherein the goal is selected from at least one of: service excellence, billing excellence, sales and marketing excellence, deregulation transition excellence, commercial/industrial excellence, credit and collections excellence, and back office excellence.

3. A method as recited in claim 1, wherein the goal includes at least one business objective and at least one learning objective.

4. A method as recited in claim 1, further comprising the step of presenting information indicative of the goal utilizing the network prior to the presenting of the training simulation.

5. A method as recited in claim 1, further comprising the step of: storing the measurement of the user's progress in a database utilizing the network.

6. A method as recited in claim 1, wherein the network is a wide area network capable of communicating using TCP/IP and IPX protocol.

7. A method as recited in claim 1, wherein the training simulation is displayed in a browser.

8. A computer readable medium containing a computer program which, when loaded on a computer and executed, causes the computer to perform the functions of:

utilizing a network to present a goal-based training simulation to a user for achieving a goal, wherein the goal is improving a user's decision-making process and work related skills;

integrating information into the training simulated utilizing the network that motivates achievement of the goal by the user;

measuring progress of the user towards achieving the goal utilizing the network;

providing feedback to the user utilizing the network that further motivates achievement of the goal; and

reporting the user's progress towards the goal utilizing the network.

9. A computer readable medium as recited in claim 8, wherein the goal is selected from at least one of: service excellence, billing excellence, sales and marketing excellence, deregulation transition excellence, commercial/industrial excellence, credit and collections excellence, and back office excellence.

10. A computer readable medium as recited in claim 8, wherein the goal includes at least one business objective and at least one learning objective.

11. A computer readable medium as recited in claim 8, further comprising the function of presenting information indicative of the goal utilizing the network prior to the presenting of the training simulation.

12. A computer readable medium as recited in claim 8, further comprising the function of storing the measurement of the user's progress in a database utilizing the network, and the function of comparing the measurement of the user's progress to a measurement of the progress of subsequent other user.

13. A computer readable medium as recited in claim 8, wherein the network is a wide area network capable of communicating using TCP/IP and IPX protocol.

14. A computer readable medium as recited in claim 8, wherein the training simulation is displayed in a browser.

15. A system for analyzing performance in a network based training simulation, comprising;

(a) logic that presents across a network goal-based training simulation to a user for achieving a goal, wherein the goal is improving a user's decision-making process and work related skills;

(b) logic that integrates information into the training simulated utilizing the network that motivates achievement of the goal by the user;

(c) logic that measures progress of the user towards achieving the goal utilizing the network;

(d) logic that provides feedback to the user utilizing the network that further motivates achievement of the goal, wherein the feedback is selected from at least three levels of feedback such that each of the three levels of feed back is progressively more detailed and the level of feedback selected is proportionate to a number of mistakes made by the user;

(e) logic that reports the measured progress towards achievement of the goal utilizing the network;

(f) logic that compares the measured progress of the user to a measurement of a subsequent other user's progress; and

(g) logic that gauges effectiveness of the training simulation in teaching achievement of the goal to the user based on the measured progress and the feedback.

16. A system as recited in claim 15, wherein the goal is selected from at least one of: service excellence, billing excellence, sales and marketing excellence, deregulation transition excellence, commercial/industrial excellence, credit and collections excellence, and back office excellence.

17. A system as recited in claim 15, wherein the goal includes at least one business objective and at least one learning objective.

18. A system as recited in claim 15, further comprising logic that presents information indicative of the goal utilizing the network prior to the presenting of the training simulation.

19. A system as recited in claim 15, further comprising logic that stores the measurement of the user's progress in a database utilizing the network.

20. A system as recited in claim 15, wherein the network is a wide area network capable of communicating using TCP/IP and IPX protocol.

21. A method utilizing a network for analyzing employee performance in a network goal-based training simulation for improving employee decision-making skills, comprising the steps of:

(a) providing a goal-based training simulation accessible by utilizing a network;

(b) presenting a training simulation to one or more than one employee for achievement of a goal;

(c) motivating achievement of the goal by the one or more than one employee by utilizing the network to integrate information into the training simulation;

(d) measuring progress of the one or more than one employee towards achievement of the goal in the training simulation;

(e) providing feedback to the one or more than one employee for further motivating achievement of the goal and improving employee decision-making skills of one or more than one employee utilizing the network; and

(f) reporting progress of the one or more than one employee towards achievement of goal.

22. The method of claim 21, wherein after the step of presenting further comprising the step of selecting a goal for achievement.

23. The method of claim 21, wherein the step of measuring further comprising calculating a measurement indicative of the progress of the employee towards achievement of the goal.

24. The method of claim 23, further comprising reporting the calculated measurement of the progress of the employee towards achievement of the goal.

Description

FIELD OF THE INVENTION

The present invention relates to training programs and more particularly to employee analysis based on results of an education business simulation.

BACKGROUND

The International Data Corporation projects that the U.S. market for Web-based training will exceed a total of $6 billion by 2002. Using this same data the Web-based training market will grow at a rate higher than 20% per year. During a similar timeframe, Training Magazine predicts that total U.S. training market will grow to approximately $72 billion by 2002. Web-based training currently represents a very small percent of overall training budgets in the U.S. (note: these are U.S. only statistics) and will continue growing at a healthy rate going forward.

When building a knowledge based system or expert system, at least two disciplines are necessary to properly construct the rules that drive the knowledge base, the discipline of the knowledge engineer and the knowledge of the expert. The domain expert has knowledge of the domain or field of use of the expert system. For example, the domain expert of an expert for instructing students in an automotive manufacturing facility might be a process control engineer while the domain expert for a medical instruction system might be a doctor or a nurse. The knowledge engineer is a person that understands the expert system and utilizes the expert's knowledge to create an application for the system. In many instances, the knowledge engineer and domain expert are separate people who have to collaborate to construct the expert system.

Typically, this collaboration takes the form of the knowledge engineer asking questions of the domain expert and incorporating the answers to these questions into the design of the system. This approach is labor intensive, slow and error prone. The coordination of the two separate disciplines may lead to problems. Although the knowledge engineer can transcribe input from the expert utilizing videotape, audio tape, text and other sources, efforts from people of both disciplines have to be expended. Further, if the knowledge engineer does not ask the right questions or asks the questions in an incorrect way, the information utilized to design the knowledge base could be incorrect. Feedback to the knowledge engineer from the expert system is often not available in prior art system until the construction is completed. With conventional system, there is a time consuming feedback loop that ties together various processes from knowledge acquisition to validation.

Educational systems utilizing an expert system component often suffer from a lack of motivational aspects that result in a user becoming bored or ceasing to complete a training program. Current training programs utilize static, hard-coded feedback with some linear video and graphics used to add visual appeal and illustrate concepts. These systems typically support one "correct" answer and navigation through the system is only supported through a single defined path which results in a two-dimensional generic interaction, with no business model support and a single feedback to the learner of correct or incorrect based on the selected response. Current tutorial systems do not architect real business simulations into the rules to provide a creative learning environment to a user.

SUMMARY OF THE INVENTION

Analyzing employees based on results of an education business simulation. A network is utilized to present a training simulation to a user such as an employee for achieving a training goal. Information is integrated into the training simulated utilizing the network that helps motivate achievement of the goal by the user. For example, integrated information may include information for helping the user develop skills for more efficiently achieving the goal. Progress of the user towards achieving the goal is measured utilizing the network. This progress measurement may include, for example, tracking skill areas that the user has difficulty mastering. For further motivating achievement of the goal, feedback is also provided to the user utilizing the network. There are three levels of feedback where each is progressively more detailed and is selected based on the number of mistakes made by the user. The user's progress towards the goal is subsequently reported to the employer of the user utilizing the network.

In an aspect of the present invention, the goal may be one of: (1) service excellence for teaching the user skills in handling a high percentage of customer calls correctly; (2) billing excellence for teaching the user skills for consistently making decision and taking actives that lead to correct customer bills; (3) sales and marketing excellence for teaching the user skills in increasing sales of products and services, (4) deregulation transition excellence for teaching the user skills for correctly responding in competitive scenarios, (5) commercial/industrial excellence for teaching the user skills for increasing revenue, customer satisfaction, and retention, (6) credit and collections excellence for teaching the user skills that help increase revenue and decrease bad debts, and (7) back office excellence for teaching the user skills that increase throughput of back office work items. In another aspect of the present invention, the goal may include at least one business objective and at least one learning objective.

In an embodiment of the present invention, information indicative of the goal may be presented utilizing the network prior to the presenting of the training simulation. In another embodiment of the present invention, the measurement of the user's progress may be stored in a database utilizing the network sot that the measurement of the user's progress may be compared to a measurement of the progress of subsequent other users.

In an additional aspect of the present invention, the network may be a wide area network capable of communicating using TCP/IP and IPX protocol. In yet a further aspect, the training simulation may be displayed in a browser such as an Internet browser like Microsoft's MS Internet Explorer and Netscape's Navigator.

DESCRIPTION OF THE DRAWINGS

The foregoing aspects and embodiments are better understood from the following detailed description of the invention with reference to the drawings, in which:

FIG. 1 is a schematic diagram of a representative hardware environment in accordance with an embodiment of the present invention;

FIG. 2 illustrates a model for the various aspects of training in accordance with an embodiment of the present invention;

FIG. 3 illustrates a model for focusing the core principles on various types of employees in accordance with an embodiment of the present invention;

FIG. 4 is a flowchart for a process for a network-based training simulation payment scheme in accordance with an embodiment of the present invention;

FIG. 5 is a flowchart for a process for analyzing performance in a network based training simulation in accordance with an embodiment of the present invention;

FIG. 6 is a flowchart for a process for tying training to business effectiveness in a network based simulation environment in accordance with an embodiment of the present invention;

FIG. 7 is a flowchart for a process for capturing employee capability data in a network based training simulation in accordance with an embodiment of the present invention;

FIG. 8A is a flowchart for a process for performing reporting and data analysis in a network based educational business simulation in accordance with an embodiment of the present invention;

FIG. 8B is a flowchart for a process for developing module content in accordance with an embodiment of the present invention;

FIG. 9A is a block diagram of a system architecture in accordance with an embodiment of the present invention;

FIG. 9B depicts the timeline and relative resource requirements for each phase of development for a typical application development in accordance with an embodiment of the present invention;

FIG. 9C depicts the potential savings in both functional and technical tasks in accordance with an embodiment of the present invention;

FIG. 9D illustrates commonalties in accordance with an embodiment of the present invention;

FIG. 9E illustrates a development architecture approach in accordance with an embodiment of the present invention;

FIG. 9F illustrates a small segment of a domain model for claims handlers in the auto insurance industry in accordance with an embodiment of the present invention;

FIG. 9G illustrates an instantiated domain model in accordance with an embodiment of the present invention;

FIG. 9H illustrates an insurance underwriting profile in accordance with an embodiment of the present invention;

FIG. 10 illustrates a transformation component in accordance with an embodiment of the present invention;

FIG. 11 illustrates the use of a toolbar to navigate and access application level features in accordance with an embodiment of the present invention;

FIG. 12 is a GBS display in accordance with an embodiment of the present invention;

FIG. 13 is a feedback display in accordance with an embodiment of the present invention;

FIG. 14 illustrates a display in which a student has made some mistakes in accordance with an embodiment of the present invention;

FIG. 15 illustrates a journal entry simulation in accordance with an embodiment of the present invention;

FIG. 16 illustrates a simulated Bell Phone Bill journal entry in accordance with an embodiment of the present invention;

FIG. 17 illustrates a feedback display in accordance with an embodiment of the present invention;

FIGS. 18 and 19 illustrate a feedback display in accordance with an embodiment of the present invention;

FIG. 20 illustrates a feedback display in accordance with an embodiment of the present invention;

FIG. 21 illustrates a simulation display in accordance with an embodiment of the present invention;

FIG. 22 illustrates the steps of the first scenario in accordance with an embodiment of the present invention;

FIGS. 23 and 24 illustrate the steps associated with a build scenario in accordance with an embodiment of the present invention;

FIG. 25 illustrates how the tool suite supports student administration in accordance with an embodiment of the present invention;

FIG. 26 illustrates a suite to support a student interaction in accordance with an embodiment of the present invention;

FIG. 27 illustrates the remediation process in accordance with an embodiment of the present invention;

FIG. 28 illustrates a display of journalization transactions in accordance with an embodiment of the present invention;

FIG. 29 illustrates the objects for the journalization task in accordance with an embodiment of the present invention;

FIG. 30 illustrates the mapping of a source item to a target item in accordance with an embodiment of the present invention;

FIG. 31 illustrates target group bundles in accordance with an embodiment of the present invention;

FIG. 32 illustrates a TargetGroup Hierarchy in accordance with an embodiment of the present invention;

FIG. 33 illustrates a small section the amount of feedback in accordance with an embodiment of the present invention;

FIG. 34 illustrates an analysis of rules in accordance with an embodiment of the present invention;

FIG. 35 illustrates a feedback selection in accordance with an embodiment of the present invention;

FIG. 36A illustrates a portion of a flowchart of the feedback logic in accordance with an embodiment of the present invention;

FIG. 36B illustrates another portion of a flowchart of the feedback logic in accordance with an embodiment of the present invention;

FIG. 36C illustrates an additional portion of a flowchart of the feedback logic in accordance with an embodiment of the present invention;

FIG. 36D illustrates a further portion of a flowchart of the feedback logic in accordance with an embodiment of the present invention;

FIG. 37 illustrates an example of separating out some mistakes for the interface to catch and others for the ICAT to catch has positive and negative impacts in accordance with an embodiment of the present invention;

FIG. 38 is a block diagram of the hierarchical relationship of a transaction in accordance with an embodiment of the present invention;

FIG. 39 is a block diagram illustrating the feedback hierarchy in accordance with an embodiment of the present invention;

FIG. 40 is a block diagram illustrating how the simulation engine is architected into an embodiment of the present invention of the invention;

FIG. 41 is a block diagram setting forth the architecture of a simulation model in accordance with an embodiment of the present invention;

FIG. 42 illustrates the arithmetic steps in accordance with an embodiment of the present invention;

FIG. 43 illustrates a drag & drop input operation in accordance with an embodiment of the present invention;

FIG. 44 illustrates list object processing in accordance with an embodiment of the present invention;

FIG. 45 illustrates the steps for configuring a simulation in accordance with an embodiment of the present invention;

FIG. 46 illustrates a distinct output in accordance with an embodiment of the present invention;

FIG. 47 is a block diagram presenting the detailed architecture of a system dynamics model in accordance with an embodiment of the present invention;

FIG. 48 is graphical representation of the object model which is utilized to instantiate the system dynamic engine in accordance with an embodiment of the present invention;

FIG. 49 is a PInput Cell for a simulation model in accordance with an embodiment of the present invention;

FIG. 50 is a PInput backup cell in a simulation model in accordance with an embodiment of the present invention;

FIG. 51 is a display illustrating a POutput cell in accordance with an embodiment of the present invention. The steps required to configure the POutput are presented below;

FIG. 52 is an overview diagram of the logic utilized for initial configuration in accordance with an embodiment of the present invention;

FIG. 53 is a display of the source item and target configuration in accordance with an embodiment of the present invention;

FIG. 54 is a display of video information in accordance with an embodiment of the present invention;

FIG. 55 illustrates a display depicting configured rules in accordance with an embodiment of the present invention;

FIG. 56 illustrates feedback for configured rules in accordance with an embodiment of the present invention;

FIG. 57 illustrates a display with follow-up configuration questions in accordance with an embodiment of the present invention;

FIG. 58 illustrates configuration of aggregate rules in accordance with an embodiment of the present invention;

FIG. 59 illustrates a set of coach items in accordance with an embodiment of the present invention;

FIG. 60 is an ICA Meeting Configuration tool display in accordance with an embodiment of the present invention;

FIG. 61 illustrates an ICA utility in accordance with an embodiment of the present invention;

FIG. 62 illustrates a configuration utility display in accordance with an embodiment of the present invention;

FIG. 63 illustrates an object editor toolbar in accordance with an embodiment of the present invention;

FIG. 64 illustrates the seven areas that can be configured for a simulation in accordance with an embodiment of the present invention;

FIG. 65 illustrates a display that defines inputs in accordance with an embodiment of the present invention;

FIG. 66 illustrates a list editor in accordance with an embodiment of the present invention;

FIG. 67A illustrates a define student display in accordance with an embodiment of the present invention;

FIG. 67B illustrates a ControlSourceItem display in accordance with an embodiment of the present invention;

FIG. 68 illustrates a simulation workbench in accordance with an embodiment of the present invention;

FIG. 69 illustrates an object viewer in accordance with an embodiment of the present invention. As shown in

FIG. 70 illustrates an Object Viewer Configuration in an Utilities menu in accordance with an embodiment of the present invention;

FIG. 71 illustrates a log viewer in accordance with an embodiment of the present invention;

FIG. 72 illustrates a Doc Maker display in accordance with an embodiment of the present invention;

FIG. 73 illustrates a Feedback display in accordance with an embodiment of the present invention;

FIG. 74 is an object editor display that illustrates the use of references in accordance with an embodiment of the present invention; and

FIG. 75 presents the detailed design of smart spreadsheets in accordance with an embodiment of the present invention.

DETAILED DESCRIPTION

FIG. 1 is a schematic diagram of one possible hardware implementation by which the present invention may be carried out. As shown, the present invention may be practiced in the context of a personal computer such as an IBM compatible personal computer, APPLE MACINTOSH computer or UNIX based workstation.

A representative hardware environment is depicted in FIG. 1, which illustrates a typical hardware configuration of a workstation in accordance with one embodiment having a central processing unit 110, such as a microprocessor, and a number of other units interconnected via a system bus 112. The workstation shown in FIG. 1 includes a Random Access Memory (RAM) 114, Read Only Memory (ROM) 116, an I/O adapter 118 for connecting peripheral devices such as disk storage units 120 to the bus 112, a user interface adapter 122 for connecting a keyboard 124, a mouse 126, a speaker 128, a microphone 132, and/or other user interface devices such as a touch screen (not shown) to the bus 112, communication adapter 134 for connecting the workstation to a communication network (e.g., a data processing network) and a display adapter 136 for connecting the bus 112 to a display device 138.

The workstation typically has resident thereon an operating system such as the MICROSOFT WINDOWS NT or WINDOWS/95 Operating System (OS), the IBM OS/2 operating system, the MOS, or UNIX operating system. Those skilled in the art will appreciate that the present invention may also be implemented on other platforms and operating systems.

An embodiment of the present invention of the present invention is written using JAVA, C, and the C++ language and utilizes object oriented programming methodology. Object oriented programming (OOP) has become increasingly used to develop complex applications. As OOP moves toward the mainstream of software design and development, various software solutions require adaptation to make use of the benefits of OOP.

OOP is a process of developing computer software using objects, including the steps of analyzing the problem, designing the system, and constructing the program. An object is a software package that contains both data and a collection of related structures and procedures. Since it contains both data and a collection of structures and procedures, it can be visualized as a self-sufficient component that does not require other additional structures, procedures or data to perform its specific task. OOP, therefore, views a computer program as a collection of largely autonomous components, called objects, each of which is responsible for a specific task. This concept of packaging data, structures, and procedures together in one component or module is called encapsulation.

In general, OOP components are reusable software modules which present an interface that conforms to an object model and which are accessed at run-time through a component integration architecture. A component integration architecture is a set of architecture mechanisms which allow software modules in different process spaces to utilize each others capabilities or functions. This is generally done by assuming a common component object model on which to build the architecture. It is worthwhile to differentiate between an object and a class of objects at this point. An object is a single instance of the class of objects, which is often just called a class. A class of objects can be viewed as a blueprint, from which many objects can be formed.

OOP allows the programmer to create an object that is a part of another object. For example, the object representing a piston engine is said to have a composition-relationship with the object representing a piston. In reality, a piston engine comprises a piston, valves and many other components; the fact that a piston is an element of a piston engine can be logically and semantically represented in OOP by two objects.

OOP also allows creation of an object that "depends from" another object. If there are two objects, one representing a piston engine and the other representing a piston engine wherein the piston is made of ceramic, then the relationship between the two objects is not that of composition. A ceramic piston engine does not make up a piston engine. Rather it is merely one kind of piston engine that has one more limitation than the piston engine; its piston is made of ceramic. In this case, the object representing the ceramic piston engine is called a derived object, and it inherits all of the aspects of the object representing the piston engine and adds further limitation or detail to it. The object representing the ceramic piston engine "depends from" the object representing the piston engine. The relationship between these objects is called inheritance.

When the object or class representing the ceramic piston engine inherits all of the aspects of the objects representing the piston engine, it inherits the thermal characteristics of a standard piston defined in the piston engine class. However, the ceramic piston engine object overrides these ceramic specific thermal characteristics, which are typically different from those associated with a metal piston. It skips over the original and uses new functions related to ceramic pistons. Different kinds of piston engines have different characteristics, but may have the same underlying functions associated with it (e.g., how many pistons in the engine, ignition sequences, lubrication, etc.). To access each of these functions in any piston engine object, a programmer would call the same functions with the same names, but each type of piston engine may have different/overriding implementations of functions behind the same name. This ability to hide different implementations of a function behind the same name is called polymorphism and it greatly simplifies communication among objects.

With the concepts of composition-relationship, encapsulation, inheritance and polymorphism, an object can represent just about anything in the real world. In fact, our logical perception of the reality is the only limit on determining the kinds of things that can become objects in object-oriented software. Some typical categories are as follows:

Objects can represent physical objects, such as automobiles in a traffic-flow simulation, electrical components in a circuit-design program, countries in an economics model, or aircraft in an air-traffic-control system.
Objects can represent elements of the computer-user environment such as windows, menus or graphics objects.
An object can represent an inventory, such as a personnel file or a table of the latitudes and longitudes of cities.
An object can represent user-defined data types such as time, angles, and complex numbers, or points on the plane.

With this enormous capability of an object to represent just about any logically separable matters, OOP allows the software developer to design and implement a computer program that is a model of some aspects of reality, whether that reality is a physical entity, a process, a system, or a composition of matter. Since the object can represent anything, the software developer can create an object which can be used as a component in a larger software project in the future.

If 90% of a new OOP software program consists of proven, existing components made from preexisting reusable objects, then only the remaining 10% of the new software project has to be written and tested from scratch. Since 90% already came from an inventory of extensively tested reusable objects, the potential domain from which an error could originate is 10% of the program. As a result, OOP enables software developers to build objects out of other, previously built objects.

This process closely resembles complex machinery being built out of assemblies and sub-assemblies. OOP technology, therefore, makes software engineering more like hardware engineering in that software is built from existing components, which are available to the developer as objects. All this adds up to an improved quality of the software as well as an increased speed of its development.

Programming languages are beginning to fully support the OOP principles, such as encapsulation, inheritance, polymorphism, and composition-relationship. With the advent of the C++ language, many commercial software developers have embraced OOP. C++ is an OOP language that offers a fast, machine-executable code. Furthermore, C++ is suitable for both commercial-application and systems-programming projects. For now, C++ appears to be the most popular choice among many OOP programmers, but there is a host of other OOP languages, such as Smalltalk, Common Lisp Object System (CLOS), and Eiffel. Additionally, OOP capabilities are being added to more traditional popular computer programming languages such as Pascal.

The benefits of object classes can be summarized, as follows:

Objects and their corresponding classes break down complex programming problems into many smaller, simpler problems.
Encapsulation enforces data abstraction through the organization of data into small, independent objects that can communicate with each other. Encapsulation protects the data in an object from accidental damage, but allows other objects to interact with that data by calling the object's member functions and structures.
Subclassing and inheritance make it possible to extend and modify objects through deriving new kinds of objects from the standard classes available in the system. Thus, new capabilities are created without having to start from scratch.
Polymorphism and multiple inheritance make it possible for different programmers to mix and match characteristics of many different classes and create specialized objects that can still work with related objects in predictable ways.
Class hierarchies and containment hierarchies provide a flexible mechanism for modeling real-world objects and the relationships among them.
Libraries of reusable classes are useful in many situations, but-they also have some limitations. For example:
Complexity. In a complex system, the class hierarchies for related classes can become extremely confusing, with many dozens or even hundreds of classes.
Flow of control. A program written with the aid of class libraries is still responsible for the flow of control (i.e., it must control the interactions among all the objects created from a particular library). The programmer has to decide which functions to call at what times for which kinds of objects.
Duplication of effort. Although class libraries allow programmers to use and reuse many small pieces of code, each programmer puts those pieces together in a different way. Two different programmers can use the same set of class libraries to write two programs that do exactly the same thing but whose internal structure (i.e., design) may be quite different, depending on hundreds of small decisions each programmer makes along the way. Inevitably, similar pieces of code end up doing similar things in slightly different ways and do not work as well together as they should.

Class libraries are very flexible. As programs grow more complex, more programmers are forced to reinvent basic solutions to basic problems over and over again. A relatively new extension of the class library concept is to have a framework of class libraries. This framework is more complex and consists of significant collections of collaborating classes that capture both the small scale patterns and major mechanisms that implement the common requirements and design in a specific application domain. They were first developed to free application programmers from the chores involved in displaying menus, windows, dialog boxes, and other standard user interface elements for personal computers.

Frameworks also represent a change in the way programmers think about the interaction between the code they write and code written by others. In the early days of procedural programming, the programmer called libraries provided by the operating system to perform certain tasks, but basically the program executed down the page from start to finish, and the programmer was solely responsible for the flow of control. This was appropriate for printing out paychecks, calculating a mathematical table, or solving other problems with a program that executed in just one way.

The development of graphical user interfaces began to turn this procedural programming arrangement inside out. These interfaces allow the user, rather than program logic, to drive the program and decide when certain actions should be performed. Today, most personal computer software accomplishes this by means of an event loop which monitors the mouse, keyboard, and other sources of external events and calls the appropriate parts of the programmer's code according to actions that the user performs. The programmer no longer determines the order in which events occur. Instead, a program is divided into separate pieces that are called at unpredictable times and in an unpredictable order. By relinquishing control in this way to users, the developer creates a program that is much easier to use. Nevertheless, individual pieces of the program written by the developer still call libraries provided by the operating system to accomplish certain tasks, and the programmer must still determine the flow of control within each piece after it's called by the event loop. Application code still "sits on top of" the system.

Even event loop programs require programmers to write a lot of code that should not need to be written separately for every application. The concept of an application framework carries the event loop concept further. Instead of dealing with all the nuts and bolts of constructing basic menus, windows, and dialog boxes and then making these things all work together, programmers using application frameworks start with working application code and basic user interface elements in place. Subsequently, they build from there by replacing some of the generic capabilities of the framework with the specific capabilities of the intended application.

Application frameworks reduce the total amount of code that a programmer has to write from scratch. However, because the framework is really a generic application that displays windows, supports copy and paste, and so on, the programmer can also relinquish control to a greater degree than event loop programs permit. The framework code takes care of almost all event handling and flow of control, and the programmer's code is called only when the framework needs it (e.g., to create or manipulate a proprietary data structure).

A programmer writing a framework program not only relinquishes control to the user (as is also true for event loop programs), but also relinquishes the detailed flow of control within the program to the framework. This approach allows the creation of more complex systems that work together in interesting ways, as opposed to isolated programs, having custom code, being created over and over again for similar problems.

Thus, as is explained above, a framework basically is a collection of cooperating classes that make up a reusable design solution for a given problem domain. It typically includes objects that provide default behavior (e.g., for menus and windows), and programmers use it by inheriting some of that default behavior and overriding other behavior so that the framework calls application code at the appropriate times.

There are three main differences between frameworks and class libraries:

Behavior versus protocol. Class libraries are essentially collections of behaviors that one can call when one wants those individual behaviors in a program. A framework, on the other hand, provides not only behavior but also the protocol or set of rules that govern the ways in which behaviors can be combined, including rules for what a programmer is supposed to provide versus what the framework provides.
Call versus override. With a class library, the code the programmer instantiates objects and calls their member functions. It's possible to instantiate and call objects in the same way with a framework (i.e., to treat the framework as a class library), but to take full advantage of a framework's reusable design, a programmer typically writes code that overrides and is called by the framework. The framework manages the flow of control among its objects. Writing a program involves dividing responsibilities among the various pieces of software that are called by the framework rather than specifying how the different pieces should work together.
Implementation versus design. With class libraries, programmers reuse only implementations, whereas with frameworks, they reuse design. A framework embodies the way a family of related programs or pieces of software work. It represents a generic design solution that can be adapted to a variety of specific problems in a given domain. For example, a single framework can embody the way a user interface works, even though two different user interfaces created with the same framework might solve quite different interface problems.

Thus, through the development of frameworks for solutions to various problems and programming tasks, significant reductions in the design and development effort for software can be achieved. An embodiment of the present invention of the invention utilizes HyperText Markup Language (HTML) to implement documents on the Internet together with a general-purpose secure communication protocol for a transport medium between the client and the Newco. HTTP or other protocols could be readily substituted for HTML without undue experimentation. Information on these products is available in T. Berners-Lee, D. Connoly, "RFC 1866: Hypertext Markup Language—2.0" (November 1995); and R. Fielding, H, Frystyk, T. Berners-Lee, J. Gettys and J. C. Mogul, "Hypertext Transfer Protocol—HTTP/1.1: HTTP Working Group Internet Draft" (May 2, 1996). HTML is a simple data format used to create hypertext documents that are portable from one platform to another. HTML documents are SGML documents with generic semantics that are appropriate for representing information from a wide range of domains. HTML has been in use by the World-Wide Web global information initiative since 1990. HTML is an application of ISO Standard 8879; 1986 Information Processing Text and Office Systems; Standard Generalized Markup Language (SGML).

To date, Web development tools have been limited in their ability to create dynamic Web applications which span from client to server and interoperate with existing computing resources. Until recently, HTML has been the dominant technology used in development of Web-based solutions. However, HTML has proven to be inadequate in the following areas:

Poor performance;
Restricted user interface capabilities;
Can only produce static Web pages;
Lack of interoperability with existing applications and data; and
Inability to scale.

Sun Microsystem's Java language solves many of the client-side problems by:

Improving performance on the client side;
Enabling the creation of dynamic, real-time Web applications; and
Providing the ability to create a wide variety of user interface components.

With Java, developers can create robust User Interface (UI) components. Custom "widgets" (e.g., real-time stock tickers, animated icons, etc.) can be created, and client-side performance is improved. Unlike HTML, Java supports the notion of client-side validation, offloading appropriate processing onto the client for improved performance. Dynamic, real-time Web pages can be created. Using the above-mentioned custom UI components, dynamic Web pages can also be created.

Sun's Java language has emerged as an industry-recognized language for "programming the Internet." Sun defines Java as: "a simple, object-oriented, distributed, interpreted, robust, secure, architecture-neutral, portable, high-performance, multithreaded, dynamic, buzzword-compliant, general-purpose programming language. Java supports programming for the Internet in the form of platform-independent Java applets." Java applets are small, specialized applications that comply with Sun's Java Application Programming Interface (API) allowing developers to add "interactive content" to Web documents (e.g., simple animations, page adornments, basic games, etc.). Applets execute within a Java-compatible browser (e.g., Netscape Navigator) by copying code from the server to client. From a language standpoint, Java's core feature set is based on C++. Sun's Java literature states that Java is basically, "C++ with extensions from Objective C for more dynamic method resolution."

Another technology that provides similar function to JAVA is provided by Microsoft and ActiveX Technologies, to give developers and Web designers wherewithal to build dynamic content for the Internet and personal computers. ActiveX includes tools for developing animation, 3-D virtual reality, video and other multimedia content. The tools use Internet standards, work on multiple platforms, and are being supported by over 100 companies. The group's building blocks are called ActiveX Controls, small, fast components that enable developers to embed parts of software in hypertext markup language (HTML) pages. ActiveX Controls work with a variety of programming languages including Microsoft Visual C++, Borland Delphi, Microsoft Visual Basic programming system and, in the future, Microsoft's development tool for Java, code named "Jakarta." ActiveX Technologies also includes ActiveX Server Framework, allowing developers to create server applications. One of ordinary skill in the art readily recognizes that ActiveX could be substituted for JAVA without undue experimentation to practice the invention.

Improving Employee Performance

Over the next decade, human performance will be the most important differentiator in a future world where access to technology, customers, and markets no longer provide significant entry barriers. As such, companies must link improvements in human performance to overall business performance, effectively "creating new business value through people." Improved human performance will result in better business performance.

A solution for improving human performance is training. For a training program to be an effective performance-based program, a number of elements are recommended:

Clear and focused curriculum.
Effective training tools and resources.
Training environment structured to mirror the job.
Consistent coaching and timely feedback.
Skills and behaviors learned in a business simulated environment. The present invention enables human performance to be delivered anytime and anywhere. FIG. 2 illustrates a model for the various aspects of training including: (1) a simulation 200 of the job and development of employee's ability to perform their job (technical, business, and customer skills; (2) a training environment 202 to deliver training anytime and anywhere; (3) coaching tips 204 and learning resources 206 to enhance performance development; and (4) A curriculum 208 of content addressing high volume, recurrent development needs.

There are many potential benefits from business offerings utilizing this model for training. A potential benefit includes making employees more successful at their jobs—business simulations maximize retention & reinforcement of desired employee behaviors. A second potential benefit includes addressing high business impact skill areas—regulated & retail energy suppliers—areas of significant cost or customer service impact. A third potential benefit includes tailoring to each employer's specific business needs and objectives—cost effective delivery through common business process & technology platform. A fourth potential benefit includes delivering training to the point of need—Internet-based—anytime, anywhere.

A fifth potential benefit includes developing content rapidly—leverage platform/training assets while facilitating client involvement/approval. A sixth potential benefit includes improving management insight—summary & detail reporting of utilization, certification, scoring, and feedback. A seventh potential benefit includes minimizing fixed training investment—pay-per-use model—costs are tied to training benefits (assessment). An eighth potential benefit includes making training more successful—give training another toolset to address training requirements proactively.

Business simulations may serve as the basis for delivering such performance and provide a learning environment that enables participants to acquire technical and social skills better (i.e., with higher retention) and faster (i.e., less time to competence) by:roviding a goal, simulating the workplace (with a focus on application of knowledge and skills), providing a risk-free environment, allowing the learner to make mistakes, allowing the learner to set own pace, providing support/resources/expertise as needed, and allowing performance to confirm employee's competence.

In an illustrative embodiment of the present invention a customer service curriculum may be implemented to provide excellent customer service, decrease costs, and increase revenue through employee skills development. FIG. 3 illustrates a model for focusing the core principles 300 on various types of employees including new hire employees 302, current employees 304, specific targeted employees 306. One module that may be included in the customer service curriculum is a service excellence module for teaching employees to handle 99% of all call types correctly the first time, within AHT goals, while referring zero calls. In this module, the following sub-modules maybe included: (1) Resolve Billing Inquiries, (2) Effective High Bill Resolution, (3) Efficient Call Handling, (4) Proactive Customer Problem Solving, (5) Issue Change Meter Orders, (6) Issue New Service Requests, (7) The Benefits of EFT, (8) Maintain Account Information, (9) Issue Outages and Emergencies, (10) Issue Service Orders, and (11) Issue Connects/Disconnects. Another module that may be included in the customer service curriculum is a billing excellence module for teaching employees to consistently make decisions and take actions that lead to correct customer bills. In this module, the following sub-modules may be included: (1) Resolve Billing Inquiries, (2) Adjustments: When are they appropriate, (3) Effective High Bill Resolution, (4) Initial Billing Issue Analysis, (5) Budget Billing: When, How, and Why?, (6) Issue Change Meter Orders, (7) Summary Billing as a Service, (8) Resolve Non-service Billing Issues, (9) Resolve Miscellaneous Order Issues, (10) The Benefits of EFT, and (11) Obligations: When are they appropriate.

A third module that may be included in the customer service curriculum is a sales and marketing excellence module for teaching employees skills for increasing sales of products and services. In this module, the following sub-modules may be included: (1) Effective Marketing and CRM, (2) Effective Cross-selling, (3) Know Your Customer, (4) Value-added Products and Services, (5) Billing-related Products and Services, (6) Efficiency Products and Services, and (7) Non-regulated Products and Services.

An additional module that may be included in the customer service curriculum is a deregulation transition excellence module for teaching employees skills for responding to competitive scenarios correctly 99% or the time. In this module, the following sub-modules may be included: (1) What is Deregulation, (2) Understanding the Energy Merchant's Role, (3) Understanding the Energy Merchant's Code of Conduct, (4) Understanding the ISO/PX, (5) Effective Marketing and CRM, (6) Effective Cross-selling, (7) Understanding the Components of an Unbundled Bill, (8) Understanding Different Billing Models, (9) Electronic Communications: Understanding EDI, (10) Understanding Enrollment and Registration, (11) Understanding Exception Processing, and (12) Understanding Order Tracking.

A further module that may be included in the customer service curriculum is a commercial/industrial excellence module for teaching employees how to increase revenue, customer satisfaction and retention. In this module, the following sub-modules may be included: (1) Effective Marketing and CRM, and (2) Effective Rate Selection and Calculation.

A credit and collections excellence module may also be included in the customer service curriculum for teaching employees skills for helping to increase revenue and to decrease bad debt. In this module, the following sub-modules may be included: (1) Understanding the Collections Process, (2) Extensions, Arrangements, and Agreements, and (3) Understanding Deposits, Returned Checks, and Bad Debt.

Yet another module that may be included in the customer service curriculum is a back office excellence module for teaching employees skills for increasing throughput of back office work items. In this module, the following sub-modules may be included: (1) Effective Service Order Dispatching and Completion, (2) Understanding Meter Reading and Reroutes, and (3) Resolve Miscellaneous Order Issues, (4) Efficient Work Item Management and Completion.

Sub-modules are designed to translate business objectives into learning objectives in order to achieve business benefits. As an illustrative example, for a high bill sub-module (in the billing excellence module) the business objective may be to improve the ability to effectively & consistently address high bill complaints. From this business objective, the derived learning objectives may include teaching employees skills for initial customer interaction responses, analyzing root cause of complaints, proper responses to complaints, and correct customer interaction styles. These learning objectives will help achieve business benefits such as increasing customer satisfaction through consistent customer interactions, reinforcing behaviors (selling, collections, etc) to meet business objectives, and reducing lost time/effort resulting from resolution errors. As another illustrative example, in a no bill sub-module (again in the billing excellence module) the business objective may be to improve the identification and quick resolution of no billed accounts by employees. In such a sub-module, the learning objectives may include correctly identifying no bill accounts, quickly analyzing cause of no bill, identifying correct simple no bill fixes, and correctly referring difficult no bills. The potential business benefits from such a sub-module may include the increased ability of more employees to handle no bill accounts, the reduced total number of no bill accounts through increase in resolution, and the more efficiently handling of difficult no bill accounts.

In closer detail, the present invention provides for an Internet-based service that delivers high-impact training at the point of need on an anytime, anywhere basis. The present invention is based on the proven teaching concept that people "learn best by doing". The present invention uses interactive multimedia technology to simulate real-world situations and to guide and advise employees as they respond to those situations. The present invention encourages learning by letting employees understand and meet performance goals, and giving them access to a wealth of information from video clips of expert advice to relevant background data to targeted learning resources. The simulation techniques used by the present invention allow employees to work at their own pace, and to learn and make mistakes in a realistic but risk-free environment.

In an embodiment of the present invention, the present invention provides employee development on a pay-per-employee basis. Once an employee is enrolled in a training module, he or she can use it over and over again to refresh or enhance their skills without additional cost. As employees complete the present invention scenarios, their progress is measured and captured. The data captured is reported to managers allowing them to gauge the effectiveness of training, to see which individuals are mastering skills, and to identify skill areas that are particularly difficult for employees in general.

The present invention development architecture allows for rapid customization of pre-defined business attributes to deliver learning modules. For example, when an employee accesses training in the present invention, he or she will receive training that teaches their company's desired behaviors using their company's specific business scenarios as exercises. This creates a learning context where employees can safely learn to apply new skills and reinforce behaviors in the manner of their company's specific culture and objectives. This also allows for company-specific tailored content to be developed at a price that is competitive to generic content.

Functions and services provided include:

Service Category	Main Elements

Host and deliver	Sales and Market	Manage:	Reporting
content for client	Planning Support	Client Usage	Per Client/
skill development	Content	New Sales	Curriculum
needs	Development	Tech Envmts	Profitability
	Customer Service	Customer	Cost Management
	Technical Support	Relation-
	Business Case	ships
	Analysis	Usage
	Risk Management	Billing

Development and	Learning Development
Maintenance of	Subject Matter Expertise
Content	Development Management
	Configuration Management
	Development Billing
	Content Quality Management
Other Learning	Collaborative Learning (not currently available)
Services	Bulletin Board Capabilities (not currently available)
	Additional Learning Integration (not currently
	available)
	Application HR Integration (not currently available)
	Learning network consulting services (not currently
	available)
	Pricing of Additional Features (not currently available)

The present invention provides for full end-to-end integrated skill development service offering by: (1) providing content customized to meet client business objectives at the point of need, anytime, (2) providing larger content offerings and benefits larger than any one client requires, (3) hosting and support services provided, (4) providing content creation/maintenance services, (5) tailored employee assessment and feedback within context of each module, (6) ease of maintenance over time and priced accordingly, (7) reporting capabilities of per employee skill achievement, participation, and training management, (8) integration of clients existing learning resources into critical learning points within teaching content, plus (9) creation of additional learning resources to fill identified learning gaps.

Reduce Training Delivery and Development Costs

Improved contribution to bottom line by accelerating the time it takes learners to reach high levels of performance. Simulation teaching approach reduce productive time away from the job by as much as 80% over traditional learning approaches. Faster acceptance of company values and culture by the viewing of war stories and messages told by company experts and highly respected executives.

FIG. 4 is a flowchart for a process 400 for a network-based training simulation payment scheme in accordance with an embodiment of the present invention. A user such as an employee is permitted to initiate a training simulation session utilizing a network in operation 402. The network is then utilized in operation 404 to present the training simulation session to the user. In operation 406, a session charge is assessed each time the training simulation session is initiated. The session charge may be a fixed per use fee that is charged each time a user initiates a session. An invoice is then generated that includes the assessed session charge in operation 408.

In an embodiment of the present invention, the invoice may be transmitted utilizing the network to a customer such as the user's employer. In another embodiment of the present invention, a user identifier may be assigned to the user upon initiation of the training session. This user identifier is associated with the associated session charge and stored in a database. In one aspect, the user identifier may include information identifying an employer of the user. This way each session initiated by employees of a particular customer (i.e., an employer) can be tracked.

In a further embodiment of the present invention, the generation of the invoice may be accomplished by retrieving the session charges associated with a common customer from the database so that the retrieved session charges may be added together to determine a total charge to the customer. The invoice may then be created which includes the total charge to the customer. Optionally, the created invoice may include an itemized list of the retrieved session charges.

In an aspect of the present invention, the network may be a wide area network capable of communicating using TCP/IP and IPX protocol. In another aspect of the present invention, the presenting of the training simulation session utilizing the network may further include presenting a simulated environment to a user for achieving a goal utilizing the network and then utilizing the network to integrate information into the simulation environment that motivates achievement of the goal by the user. Progress of the user towards achieving the goal may then be measured utilizing the network with feedback being provided that further motivates achievement of the goal to the user utilizing the network.

Improve Employee Proficiency

Increase employee proficiency in a given business process by 15-25% over other training methods. Content developed focuses on skill application rather than content or delivery focus. Employees taught "skills" are not usually greater than 50% successful at retaining these skills, let alone being able to apply them when the time arises.

FIG. 5 is a flowchart for a process 500 for analyzing performance in a network based training simulation in accordance with an embodiment of the present invention. In operation 502, a network is utilized to present a training simulation to a user such as an employee for achieving a training goal. Information is integrated into the training simulated utilizing the network that helps motivate achievement of the goal by the user in operation 504. For example, integrated information may include information for helping the user develop skills for more efficiently achieving the goal. In operation 506, progress of the user towards achieving the goal is measured utilizing the network. This progress measurement may include, for example, tracking skill areas that the user has difficulty mastering. For further motivating achievement of the goal, feedback is also provided to the user utilizing the network in operation 508. The user's progress towards the goal is subsequently reported to the employer of the user utilizing the network in operation 510.

In an aspect of the present invention, the goal may be one of: (1) service excellence for teaching the user skills in handling a high percentage of customer calls correctly; (2) billing excellence for teaching the user skills for consistently making decision and taking actives that lead to correct customer bills; (3) sales and marketing excellence for teaching the user skills in increasing sales of products and services, (4) deregulation transition excellence for teaching the user skills for correctly responding in competitive scenarios, (5) commercial/industrial excellence for teaching the user skills for increasing revenue, customer satisfaction, and retention, (6) credit and collections excellence for teaching the user skills that help increase revenue and decrease bad debts, and (7) back office excellence for teaching the user skills that increase throughput of back office work items. In another aspect of the present invention, the goal may include at least one business objective and at least one learning objective.

In an embodiment of the present invention, information indicative of the goal may be presented utilizing the network prior to the presenting of the training simulation. In another embodiment of the present invention, the measurement of the user's progress may be stored in a database utilizing the network sot that the measurement of the user's progress may be compared to a measurement of the progress of subsequent other users.

In an additional aspect of the present invention, the network may be a wide area network capable of communicating using TCP/IP and IPX protocol. In yet a further aspect, the training simulation may be displayed in a browser such as an Internet browser like Microsoft's MS Internet Explorer and Netscape's Navigator.

Tie Training to Business Effectiveness

Deliver business results and measure gaps with desired impacts to update training content/delivery. Content developed with alignment to business objectives targeted. Content learning objectives identified by review of decision points with business process that impact business performance. All content within an individual module developed around defined learning objective.

FIG. 6 is a flowchart for a process 600 for tying training to business effectiveness in a network based simulation environment in accordance with an embodiment of the present invention. In operation 602, a business objective for a business process is first selected. An example of a business objective is the objective to improve the ability to effectively and consistently address high bill complaints. In operation 604, the business process is reviewed to identify at least one decision point in the business process at which a decision is required for achieving the business objective. In other words, achieving/attaining the business objective requires the making of a decision at one or more decision points. A training simulation is then presented for achieving the business objective (i.e., a goal) utilizing a network in operation 606. Information is incorporated into the training simulation utilizing the network that motivates the making of a correct decision each decision point for achieving the business objective in operation 608. For example, integrated information may include strategies for helping the user develop decision making skills to help the user make the correct decision at the decision point in order to achieve the business objective. In operation 610, the network is also utilized to monitor progress towards achieving the business objective and to provide feedback that further motivates achievement of the business objective.

In an embodiment of the present invention, information may be presented indicative of the correct decision utilizing the network prior to the presenting of the training simulation. In another embodiment of the present invention, a session charge may be assessed each time the simulated environment is presented.

In an aspect of the present invention, the business objective is selected from at least one of: service excellence, billing excellence, sales and marketing excellence, deregulation transition excellence, commercial/industrial excellence, credit and collections excellence, and back office excellence. In another aspect of the present invention, the network is a wide area network capable of communicating using TCP/IP and IPX protocol.

Improve Integration with Training Administration

Create cost savings around administration of training delivery and completion. Provide the capability needed to manage and administer training through providing adequate to feed performance management and training administration systems. Evolve as needed to create automated feeds to both systems within component based architecture.

Improve Employee Capability Data Capture

Increase quality of data capture of actual displayed behaviors around a desired employee skill. Design of data tracking and feedback captures employee behaviors and rationales over variety of business scenarios. Provides capture of data that proves value of training based on employee application of knowledge.

FIG. 7 is a flowchart for a process 700 for capturing employee capability data in a network based training simulation in accordance with an embodiment of the present invention. A training simulation is presented to a plurality of employees utilizing a network in operation 702. The performances of the employees in the training simulation are analyzed to generate employee performance measurements for the employees utilizing the network in operation 704. The employee performance measurements (i.e., performance data) are stored in a database in operation 706. Performance trends are then determined utilizing the stored employee performance measurements in operation 708.

In an embodiment of the present invention, the employee performance measurements may be utilized to provide behavioral correlation to employee reviews. In another embodiment of the present invention, the employee performance may be related to business performance metrics. Progress towards the goal may also be monitored utilizing the network so that feedback can be provided to further motivate accomplishment of the goal utilizing the network. As an option, the analysis of the performances of the employees in the training simulation to generate employee performance measurements may also include the calculating of a quantitative degree of correctness in achieving the goal.

In a further aspect of the present invention, presenting the training simulation may further require that information indicative of a goal be presented utilizing the network and then integrating information that motivates accomplishment of the goal. In an aspect of the present invention, the network is a wide area network capable of communicating using TCP/IP and IPX protocol. In yet another embodiment of the present invention, the determined performance trends may be transmitted to an employer of the employees utilizing the network.

Performance Reporting and Data Analysis

Captures data that could feed a performance library for determining performance trends and analysis of skill issues within clients' business model. Performance data captured in order to provide behavioral correlation to employee reviews. Where applicable, The present invention will attempt to show employee performance data relevance to business performance metrics.

FIG. 8A is a flowchart for a process 800 for performing reporting and data analysis in a network based educational business simulation in accordance with an embodiment of the present invention. First, in operation 802, a business objective is selected and then in operation 804 a training simulation is presented to an employee relating to the business objective utilizing a network. In operation 806, information is integrated into the training simulation that motivates accomplishment of the business objective by the employee utilizing the network. The employee's progress towards accomplishment of the business objective is tracked in operation 808 utilizing the network and feedback is provided in operation 810 utilizing the network that further motivates achievement of the business objective. The tracked progress towards the business objective is then stored in a database in operation 812.

In an aspect of the present invention, the business objective may be selected from: service excellence, billing excellence, sales and marketing excellence, deregulation transition excellence, commercial/industrial excellence, credit and collections excellence, and/or back office excellence. In another aspect of the present invention, the tracking of the employee's progress may include tracking the employee's behavior and rationales during the training stimulation. In a further aspect of the present invention, the network may be a wide area network capable of communicating using TCP/IP and IPX protocol.

In an embodiment of the present invention, a report may be generated based on the tracked progress towards the business object. In another embodiment of the present invention, the effectiveness of the training simulation in teaching accomplishment of the business objective to the employee may be gauged based on the tracked progress and feedback. In yet a further embodiment of the present invention, difficulties encountered by the employee in accomplishing the business object may also be identified based on the tracked progress and feedback.

Developing Module Content

FIG. 8B is a flowchart for a process 850 for developing module content in accordance with an embodiment of the present invention. In general, module specifications are first created in operation 852. Next, learning program detailed design is performed and a learning prototype is created in operations 854 and 856. From this basis, a learning product is created in operation 858 which is then tested in 860. Subsequently, continuous improvements to the learning product may be made in the future in operation 862.

In general, the creation of module specifications in operation 852 includes the creating of a module specification package. The performance of learning program detailed design in operation 854 may include the operations of defining detailed learning objectives for the learning program, designing learning activities for the learning program, designing reference items for the learning program, preparing a learning test plan for the learning program, and finally prepare a learning design report for the learning program. The creation of a learning products prototype in operation 856 may include creating a prototype and then conducting a review of the prototype.

Generally, the creating of the learning product in operation 858 may include the developing of learning content, media, reference items, evaluation materials, and a maintenance and support plan. Further, the creating of the learning product may also include the reviewing and assembly of the product. The testing of the learning product in operation 860 may include validating a learning test plan, preparing and executing the learning test plan and then performing any necessary fixed on the learning product. Continuous improvement of the learning product in operation 862 may be conducted during and after the execution of each module of the learning product.

Create Module Specifications 852

The module specifications may be created both at a user's site and offsite. The module specifications may be a "hand-off" of information to a functional team offsite.

Inputs

Analysis of a business problem
Interviews with SMEs
Interviews with user executives
Observations of users
Points of Contact
Discussions with SMEs
Interaction between development team at user's site and development team offsite
SME Review
Final Sign Off (Module Specification Package)
Required Tasks
Determine Business Issue
Determine the business issue that will be addressed in the module
Determine Impact on Organization
Determine all impacts that the business issue will have on the organization (financial impacts, impacts on the various departments within the organization, etc.)
Create Business Process Flow
Determine and document the flow of the business process that will be addressed in the module.
Determine Target Audience
Determine Module Development Schedule
Determine high level budget (in days) for the module (not costs). Include the estimated length (in hours) for the module so that developers will have an idea of the work effort involved.
Determine Learning Goal and User Performance Expectations
Compile Available Information Resources
Determine Motivational Factors
Determine SMEs and Reviewers
Determine the SMEs who will be available for consultation during the entire development of the module. For each SME, include area of expertise, phone number, email address, and preferred method of contact. Also, include the reviewer(s) for the module, areas of module that will be reviewed by this person(s), and dates of the reviews.
Team Lead Review
SME Review
Deliverables
Module Specification Package:
Business Issue to be Addressed
Impact on Organization
Business Process Flow
Target Audience
Module Development Schedule (days)
Learning Goal and User Performance Expectations
Available Information Resources
Motivational Model (ARCS)
List of SMEs and Reviewers
Signoff
Best Practices
Some of the content developers offsite should be part of the team at the user's site.
The team creating deliverables offsite should be in close contact with the team at the user's site.
When soliciting input from the user, ensure that input is being received from the right group. Ensure that it is a representative sample of the target audience. It is important to access all skill levels including beginners to experts.
Learning Program Detailed Design Performance 854
Define Detailed Learning Objectives

The Module Specifications Package will outline much of the information required to complete a module. From this information, detailed learning objectives should be established. These objectives will serve as the building blocks for the specific activities within the module, and will include terminal, enabling, and performance objectives. The learning objectives will then guide the design and development of the activities, all the way down to the exercise-specific level. All content developed will directly support one or more of these learning objectives.

Inputs

Module Specifications Package
Points of Contact
Discussions with SMEs
Interaction between development team at user's site and development team offsite
Required Tasks
Create List of Objectives
- Determine which objectives must be achieved by which target audience
- Identify which objectives address similar issues and group them accordingly
- Prioritize the Learning Objectives based on impact to the company
Group Objectives into Activities
Conduct Team Lead Review
Deliverables
Prioritized List of Objectives
List of Activities (Grouped Objectives)
Design Learning Activities

Once the objectives have been determined, ranked, and grouped, these groups will lead to the creation of activities. Designing learning activities consists of adding more details to the format, structure, and function of the activities and providing information on how the learning objectives will be best addressed in the exercises. The information added in the design phase consists of the format for the exercise, the types of quizzes to be used, how many practice vs. assessed exercises to provide, etc. Outlining the specific characteristics of example accounts to be used within activities must also take place at this time. The design of learning activities will provide a higher level overview of the activity without the specific content details that will be added during development.

Inputs

List of Tools for Content Design and Development
Business Writing Standards Document
Electronic Folder of All Documentation and Templates
Design and Development Procedures Document
List of Activities (Grouped Objectives)
Points of Contact
Discussion with tech team
Discussion with SMEs
Required Tasks
Design Exercises
Define Account Criteria to support Exercises
Determine the Functional Requirements for the Module
Develop Certification Approach
Conduct Team Lead Review
Deliverables
Activity Detailed Designs
Account Criteria
Functional Requirements for Module
Certification Approach
Best Practices
Discuss and review all activities with each content developer to ensure that overlap is minimized and that the correct amount of emphasis is placed on each learning objective
Adhere to standards for the module to make the development process easier and more accurate
Clear any new ideas or modifications with Team Leads and tech team to ensure viability
Include several appropriate accounts on the Account Lists so that alternatives exist if there are retrieval or screen capturing problems
Ensure that developers receive and incorporate SME and Team Lead feedback to minimize changes during development
Design Multi-Media Content

Once the design of the activities and exercises has proceeded to a certain point, and more specifics for each are outlined, the content developer will have a better idea of the specific subject matter and objectives that are contained in each activity. Multimedia peripherals should now be identified as being appropriate to the activity and in support of the learning objectives. Omega Productions in Atlanta, Ga. has been used successfully to create these products. Content developers offsite generate a list of "talking points" for videos that would be appropriate for inclusion in the module. Audio clips may also be included in exercises as appropriate.

Inputs

Design and Development Procedures Document
Activity Detailed Designs
Screen Scrapes
Required Tasks
Develop the Interviewee Talking Points
- Identify crucial areas in detailed design that directly relate back to the learning objectives
- Create list of Talking Points for each of these areas and include general information on the types of stories desired from SMEs for inclusion in videos
- Generate video clip ideas and determine which clips will be placed in each different activity of the module
List User Interviewees
Create Call Scripts
- Develop Audio Scripts for each scenario to be read by professional actors
Conduct Team Lead Review
Deliverables
Interviewee Talking Points
Interviewee List
User Call Scripts
Best Practices
SME and Team Lead review should be solicited to ensure scripts are realistic and appropriate
Video talking points should be general enough to allow SMEs to improvise and customize their stories, but specific enough to ensure appropriate stories are received
Expert lists should be generated keeping in mind that the expert must not only be knowledgeable, but also eloquent enough to successfully relate stories in well-planned manner
SME and Team Lead review should be solicited to ensure that scenarios are accurate and realistic.
In contrast to the videos, the scripts for the audio scenarios should be very specific and written verbatim to ensure the proper information is introduced exactly as intended by the developer.
Understand what types of clips would be most effective in the module prior to filming, as opposed to viewing scripts and then placing them in the module where appropriate.
Design Reference Items

After learning activities are designed and subject matter is finalized, the peripheral reference items may be designed. These consist of the Reference section and the What's Next sections. The content developer will take the information from the available list of resources (in the Module Specifications Package) and create the Job Aids and What's Next requirements. Additionally, content specific links within feedback to exercises should be identified for inclusion in the Reference section.

Inputs

Activity Detailed Designs
Prioritized List of Objectives
Available information Resources from Module Specification Package
Points of Contact
Interaction between development team at user's site and development team offsite
Required Tasks
Determine what topics should be explained through Job Aids
Identify the need for Job Aid links in the Activities.
Create list of What's Next items for each screen
Conduct Team Lead Review
Deliverables
Job Aid Needs=Includes a description of reference materials and job aids. Also includes the list and alphabetical layout of all reference items.

What's Next? Needs=An excel spreadsheet which includes the section page, page key, question and answer for all questions in the "What's Next" section. This spreadsheet will also be used in development as the actual content for the answers is developed.

Best Practices

Create What's Next? Links assuming that users have no familiarity with Internet applications.
Incorporate user's current job aids.
Prepare Learning Test Plan

The "Learning Test" will be conducted at the user's site. This test will occur after the module has been completed and tested thoroughly offsite, but before the module is released for production at the user's site. When preparing the approach to the learning test, developers will determine how to test users before full-scaled production to ensure that the module is effective and provides a consistent learning experience. The Learning Test Approach will be owned by the offsite Team with interaction from the user's site.

Inputs

Prioritized Objectives
Points of Contact
Interaction between team at user's site and team offsite
Required Tasks
Create the Learning Test Approach
Team Lead Review
Deliverables
Learning Test Approach=Document which includes a high level of the testing approach. The document contains the roles in the testing, allotted time for testing, purpose of the testing, suggested evaluation techniques, a test approach matrix, defined risks, and data to be collected.
Prepare Learning Design Report

The Learning Design Report consists of the Content Design Deliverable Packet that will be submitted to reviewers before final sign-off. Then the completed and reviewed packet will be used as the input for the development phase of the module.

Inputs

Learning Hierarchy/Strategy
Certification Approach
Functional Requirements for Module
Activity Detailed Designs
Interviewee List
Interviewee Talking Points
User Call Scripts
Job Aid Needs
What's Next? Needs
Points of Contact
Required Tasks
Combine design deliverables into one comprehensive deliverable packet
Deliverables
Design Deliverable Packet
- For the Developers
Learning Hierarchy/Strategy
Certification Approach
Functional Requirements for Module
Activity Detailed Designs
Interviewee List
Interviewee Talking Points
User Call Scripts
Job Aid Needs
What's Next? Needs
- For User/Reviewer Signoff
Certification Approach
Activity Detailed Designs
Job Aid Needs
Conduct Team Lead Review
Signoff
Best Practices
Signoff from the user/reviewers will include acceptance of the Content Design Deliverable Packet and the Lo-Fi Prototype (see below)
Prototype Learning Product 856
Create Prototype

Once the Detail Design is created, a Lo-Fi Prototype is produced using KX Front Page. The prototype displays the general design and layout of the Module down to the Activity and Exercise level. The exact content of the exercises is not displayed, but the general idea of each exercise is laid out. In developing the Lo-Fi Prototype, Content Developers create a storyboard depicting all of the visuals (background, window shots, icons, etc) that make up the module. This prototype is then used to display the Module to the SMEs.

Inputs

Functional Requirement for Module
Activity Detail Designs
Screen Scrapes
Interviewee Talking Points
Interviewee List
User Call Scripts
Job Aid Needs
What's Next Needs
Conversations with Tech Team Lead
Points of Contact
Interaction between development team at user's site and development team offsite
Required Tasks
Develop Lo-Fi Prototype of Module
- Construct a paper model of the Module.
- Depict every screen within the activities and exercises and also include each of the icons in the Navigation Bar.
Team Lead Review
Deliverables
Lo-Fi Prototype of Module
Conduct Prototype Review

In the prototype review, SME's, Team Leads and/or developers will step through the Lo-Fi Prototype of the Module. The SMEs and Team Leads will critique the prototype for usability, aesthetics and validity. In addition, all design documents will be compiled in the Design Deliverable Packet.

Inputs

Lo-Fi Prototype of Module
Points of Contact
Interaction between development team at user's site and development team offsite
Required Tasks
Test Lo-Fi Prototype with SME's and Reviewers
Assemble Design Deliverable Packet
SME Review
Team Lead Review
Deliverables
Test Lo-Fi Prototype w/SMEs and Reviewers
Finalized Design Deliverable Packet
Signoff
Best Practices
Walk though the Prototype a few times and polish the presentation before presenting the final product to the SMEs.
When testing the Prototype, three or more developers (including the Team Lead) are necessary. One person should facilitate and talk to the user as they go though the module. A second person should act as the 'computer' and coordinate the changing of the paper model. All additional persons should record the SMEs comments on note cards.
Sort the SME's comments into groups by topic. As a Content Team, discuss each topic and modify the Prototype where necessary.
Signoff from the user/reviewers will include acceptance of the Content Design Deliverable Packet and the Lo-Fi Prototype
Create Learning Product 858
Develop Learning Content

Once the design and prototype have been finalized, the Content for the Module must be developed. This includes the titles, instructions, scenarios, audio, questions, answers and feedback for the exercises and the introduction and Wrap-Ups for the activities and the Module. In addition, the What's Next, Reference Job Aids and Videos for the entire Module will be developed. All Content should be loaded into the database via the Content Maintenance Tool.

Inputs

List of Tools for Content Design and Development
Business Writing Standards Document
Electronic folder of all documentation and templates
Design and Development Procedures Document
List of Objectives (Terminal, Performance, Enabling)
Learning Hierarchy/Strategy
Certification Approach
Functional Requirements for Module
Activity Detailed Design
Account Criteria
Points of Contact
Discussions with SMEs
Required Tasks
Develop the Activities
Develop the Module Introduction Page
Develop the Module Wrap-Up and Certification pages
Continue to log development questions in the SME Question Log
Team Lead Review
Deliverables
Activities Complete
Module Introduction Page Complete
Module Summary Page Complete
SME Question Log
Data Loaded into the Database
Best Practices
Continue to involve Team Leads and SMEs in informal feedback/review sessions to ensure developed content is correct.
Do not use detailed information on a customer (address, phone number, etc) in the exercise question when Screen Scrapes of the accounts will be included in the exercise. This way, if the Screen Scrapes need to be changed, or even a different account must be substituted, there will be fewer necessary updates.
Review standards and adhere to them to ensure future updates are minimized.
Develop Media

Video and Audio clips are used to enhance the Module. The video clips are developed for the Module as a whole and then divided out and incorporated into the individual activities. The Talent for the Video clips are actual user employees, adding credibility to the information taught. The Audio clips are developed for a specific activity that is scenario based. The Audio Talent includes professional actors thus the Audio clips are not necessarily user specific.

Inputs

Interviewee Talking Points
Interviewee List
User Call Scripts
Points of Contact
Interaction between development team at user's site and development team offsite
Required Tasks
Conduct Video Shoot
- Plan Video topics and determine at least three potential interview questions per topic
- Coordinate Video Shoot with filming company and user
Edit Videos and Select Video Clip Short List
View Video Clip Short List and Select Final List
Complete Video
- Map Videos to individual activities
- Enter Videos into the Content Manager
Audio Clip Final List
- Develop Audio scenarios and scripts, and outline Talent characteristics.
Complete Audio Clips
- Select Final Audio Clip
Team Lead Review
Deliverables
Video Shoot
Video Clip Short List
Video Clip Final List
Video Clips Complete
Audio Clip Final List
Audio Clips Complete
Data Loaded into Database
Best Practices
It is best to create more than one Audio script for each scenario. Each should have a different tone or underling emotion so that after the recording, the clip with the highest impact can be chosen for each scenario.
Develop Reference Items

The Reference section contains the Reference Icon, the What's Next Icon and the Window Tips for the Screen Scrapes. The Reference Icon contains a list of Job Aids that can be accessed through the Reference Icon or though links in the activities. What's Next offers answers to different 'commonly asked questions' for every page of the Module. The Window Tips appear at the top of each CSS Screen Scrape and further explain the CSS window displayed.

Inputs

Job Aid needs
What's Next needs
CSS Help
User Intranet/Internet site
Account Criteria
Points of Contact
Team Lead Review
Required Tasks
Create Additional Job Aids.
Develop the What's Next? Section
Compile Account Lists
Capture Screen Scrapes
Develop Window Tips
Team Lead Review
Deliverables
Job Aids Complete
What's Next? Complete
Account Lists
Screen Scrapes
Window Tips Complete
Data loaded into Database
Best Practices
Do not forget to include What's Next for the Module Menu page, Activity Menu page and the Module Wrap-Up.
Develop Evaluation Materials

The goal for the user in taking the Training is twofold: first, to gain knowledge and second to demonstrate efficient performance at a level desired by the user executives. The process that ensures effective performance is the Certification Plan/Skills Plan. This plan assigns a point value to each question in the exercise within the activity and a weight for each activity in the module. The user must answer a predetermined percentage of the questions correctly throughout the module in order to become certified in the module topic.

Inputs

Certification Approach/Skills Approach
Points of Contact
Team Lead
Required Tasks
Develop certification plan/skills plan
Team Lead Review
Deliverables
Certification Plan/Skills Plan
Best Practices
Weigh each activity by impact to the business process
Weigh activities with fewer questions slightly less than activities with many questions so no one activity has a large impact on the user becoming certified. This way, users are not penalized for lack of partial credit for some questions.
Develop Maintenance and Support Plan

A Maintenance and Support Plan exist for all Training through the Service Desk, but additional support needs to be planned for the individual Module in three areas. All maintenance/support at the Module level must be tracked. All items in a Module specific to one user must be tracked and modified when the Module is customized for a new user. All CSS windows captured in a Module must be tracked and updated in case these windows are later modified within the CSS application itself.

Inputs

Service Desk Documentation
Activities, Module Introduction and Module Summary Complete
Points of Contact
Interaction between development team at user's site and development team offsite
Discussions with SMEs
Required Tasks
Create Module Specific Maintenance Plan
Create Customization Approach Document
Track all items in the Module that are user specific and include them in this portion. Use the Customization Approach Document to determine necessary changes when the Module is reconfigured for a new user.
Compile a list of CSS Screen Scrapes used in the Module
- This list will be supplied to the SoCo Fix-it Manager. As the CSS Windows change, the SoCo Fix-it Manager will notify the Team about necessary updates.
Team Lead Review
Deliverables
Module Specific Maintenance Plan
Customization Approach Document
List of CSS Windows Captured in Module
Review Products and Assemble

The final step in the Create Learning Products phase is the assembly and test of the Module. Once all of the data has been entered though the Content Maintenance Tool into the database tables, the Module is ready for testing. First the individual components are tested and then assembled together. The assembled pieces are tested for continuity. The final Module is application tested in the production environment.

Inputs

Activities, Module Introduction and Module Summary Complete
Video Clips Complete
Audio Clips Complete
Job Aids Complete
What's Next Complete
Window Tips Complete
Points of Contact
Team Lead
Required Tasks
Confirm all content is correctly loaded into the Database.
Conduct User Component Test
Conduct Application Acceptance Test
Team Lead Review
Signoff
Deliverables
Application complete
User Component Test Complete
Application Acceptance Test Complete
Learning Product Testing 860
Validate Learning Test Plan

The "Learning Test" will be conducted at the user's site. This test will occur after the module has been completed and tested thoroughly offsite, but before the module is released for production at the user's site. A pre-selected group of user users and SMEs will take the module and validate its effectiveness and accuracy prior to the official release of the module. The team at the user's site will be responsible for creating the Learning Test Plan. This deliverable will document the detailed plans for executing the Learning Test.

Inputs

Learning Test Approach (Task 6263.5/Prepare Learning Test Plan)
Points of Contact
Interaction between development team at user's site and development team offsite
Required Tasks
Create Learning Test Plan
The Learning Test Plan is based on the Learning Test Approach and documents how the Learning Test will be executed at the user's site.
Determine the participant profile, sample size, learning test methods, and purpose the Learning Test.
Include procedures for logging issues/fixes/enhancements that arise from the Learning Test and determine how to categorize and send these items to the development team offsite.
Deliverables
Learning Test Plan
Prepare Learning Test Plan

This task centers on implementing the recommendations/procedures in the Learning Test Plan. Several communications, scheduling, and set up tasks must be completed prior to the execution of the Learning Test. This task will be owned by the development team at the user's site.

Inputs

Learning Test Plan
Points of Contact
Communications with user Training Department
Communications with Learning Test Participants
Communications with user Technical Operations Department
Communicate with the development team offsite
Required Tasks
Implement the Learning Test Plan
Communicate with the user Training Department so that they are aware of the Learning Test and its purpose
Determine schedules for participants in the Learning Test
Communicate with the participant group so that they understand their roles and time commitments
Communicate with the user Technical Operations Department to ensure that the equipment necessary to run the test is ready and available
Ensure that the development team offsite understands the fix-it procedures for the issues arising from the Learning Test
Deliverables
Complete Set-up of Learning Test Environment
Execute Learning Test

The development team at the user's site will execute the Learning Test. All results of the Learning Test should be documented, prioritized, and categorized and then sent to the development team offsite.

Inputs

Completed Set-up of Learning Test Environment
Points of Contact
Learning Test participant group
Interaction between development team at user's site and development team offsite
Required Tasks
Execute Learning Test
Document Results of Learning Test
Send results (divided into "fixes" and "enhancements" with priorities) to the development team offsite
Deliverables
Learning Test Fixes/Enhancements Log
Best Practices
Determine the priorities of the results of the Learning Test. Identify which items need to be fixes immediately ("fixes") and which items are "nice-to-haves" ("enhancements").
Send feedback to the development team offsite regarding what worked well for the users and what did not.
Perform Learning Product Fixes

A team offsite will review and implement the fixes that result from the Learning Test and plan the post-production enhancement schedule. Fixes will be "fixed" immediately; enhancement will be reviewed and scheduled for future implementation. The Post-Production Enhancement Schedule will be sent to the development team at the user's site for reference (in case any communication needs to occur with the user).

Inputs

Completed execution of the Learning Test
Learning Test Fixes/Enhancements Log
Business Writing Standards Document
Points of Contact
Interaction between development team at user's site and development team offsite
Possible communication between the development team at user's site and the Learning Test participant group
Required Tasks
Review Learning Test Fixes/Enhancements Log
The functional team lead offsite will review this deliverable to determine which fixes will be approved and which will be rejected.
The functional team lead will assign the approved fixes to functional/technical developers
Perform Learning Test Fixes
Create Post-Production Enhancement Schedule
Deliverables
Learning Test Fixes Complete
Post-Production Enhancement Schedule
Best Practices
Ensure that all content fixes are consistent with development standards.
Continuous Improvement 862

Before the start of a new module, the team responsible for the new module will also become the owners of the process documentation for content development. The content development process for the current module will be reviewed and revised as necessary. The streamlined processes will be documented and implemented by the team responsible for the new module.

Inputs

Current Process Flow document
Current Checklist document
Points of Contact
Interaction between the team for the current module and the CI representative from the team for the new module
Interaction with the SW Delivery Excellence team
Interaction with Process Excellence experts
Required Tasks
Review Process Update Suggestions
- Approve/reject the on-going list of suggestions logged by the current development team
Update the Process Flow Diagram
Update the Checklist document
Deliverables
Revised Process Flow Diagram
Revised Checklist document
Best Practices
The members of the current module development team should be documenting potential improvements to the process on an on-going basis.
The members of the CI meeting should be: the entire development team for the current module, the team lead for the current module, the team lead for the user's site team, the team lead for the next module, and one representative from the development team for the new module.
Later, when the process becomes more streamlined and less revisions will be made the members of the CI meeting will become reduced and the CI meetings will become discontinued altogether.

The following portion is included to aid in the design and development of the individual deliverables. Each deliverable is labeled by its task number in the checklist.

SME Question Log

Purpose:

The SME Question Log is a living spreadsheet that tracks all of the issues encountered by the Content Development Team. All questions/issues should be entered into the SME Log so that SMEs may review them either formally in interviews or informally, and log responses. If a question is answered during an interview that was not originally in the SME log, enter both the question and response after the interview. If maintained and used correctly, this log should be one of the most valuable resources used during the module design and development process.

Procedures:

When logging an issue, be sure to fill out the following:

Date Logged—current date
Owner—name of person logging issue
Proposed Type of Respondent(s)—records which type of SME (offsite, User Site or Team Lead) would best be able to address the question
Question—explain in detail the issue/question encountered. Try to write the questions in the same format you will present to the user, using correct terminology and detailed information. This way, the questions can be directly lifted from the log and submitted to the SMEs prior to an interview.
Type—The Type column is a numeric based key unique to each Module. It can be used to sort all of the Q & A by topic. The key will appear as a footer on each page of the spreadsheet. For example, the key for Module 3 reads:
1=Job Aid/Reference
2=Date Wanted
3=Obligations
4=Research/Resources, etc.

When logging a response, be sure to fill out the following:

Date Answered—date SME response is given
Answered by (Operating Company)—SME name followed by their company in parenthesis. Number each SME so that when multiple SMEs are consulted per issue logged, their responses can be clearly identified.
Response—explain in detail the SME's response. Be sure to number the response with the corresponding SME number for that issue.
Job Aid Needs
Purpose
Job Aids are available for the users in the Reference section of the Module. These Job Aids are tools for the user that provides information in an easy to use format. The Job Aids are reference items that the user can access at any point throughout the Module in order to better understand a topic or concept. These job aids may also be in the form of a tool that guides the user to do a task more efficiently or more effectively.
Research
Work with content developers for each Activity to understand the content and types of information needed in the Job Aids/Reference section.
Access CSS Help either through CSS or Map to it by:
1. Open Windows Explorer
2. Tools:/Map Network Drive:
3. Any Drive
4. Path: Scroll down to AS240210 (double click)
5. Scroll down to T3 (double click): This adds the drive with path to Explorer
6. Select the Drive with the new path
7. Select "HLPs" Folder
8. Select Systcomm application
Review CSS reference and on-line help system. The majority of job-aids and reference material will most likely come from this source.
CSS Help has search capabilities that allows you to manipulate your search.
Use of the Internet is also a possibility if the necessary materials are not found in the CSS. Previous Modules have used other Utility Co. web pages for content specific information. So keep in mind as a possibility.
After finding relevant information, you can choose to copy that information or use the information given to develop a job-aid to put into the Module.
All job-aids and reference material is contained in the Reference Section of the Module
Be sure to discuss all information chosen for each Activity with the respective content developers.
For the design phase the deliverable for Job Aids/Reference Needs should contain a list of topics and subtopics. The subtopic is the Job Aid name that you have pulled from research and will be expanded into a Job Aid during development. See Job Aid/Reference Template for the deliverable.
What's Next Needs
Purpose
The What's Next? feature answers commonly asked questions that the user may have related to using and/or navigating through the module.
The What's Next? Feature changes with each screen. Content was developed for each level: Module Selection Page, Activity Selection Page, Exercise Selection Page, Exercise Page, Activity Wrap Up Page and Module Wrap Up Page.
Researching
Each What's Next Screen has hard coded text at the top that says, "What do I do next . . . " The text under this always starts with specific information for the screen the user is on starting with " . . . I just" For example: . . . I just entered the Assessed Exercise Screen. These sentences are links that when clicked expand to give information for what the user should do from that point. After the " . . . I just" information there is more general information that is Module or page specific. The more general information does not have to start with . . . I just.
Each Screen should have a list of general " . . . I just" sentences telling the user what to do at each point they could be at in that particular screen. For Example: . . . I just entered the Activity Screen. . . . I just pushed the Assessment Exercise button. . . . I just finished reading the feedback
The next set of sentences is more general information that may be throughout the Module or specific to the level they are on: Activity Selection, Exercise Selection, Assessed Exercises, etc. These can be viewed and copied if needed from Modules 1 and 2. Examples of general sentences for entire Module: What if I need to stop the training before I am finished? Examples of general sentences specific for a level: What is Assessment? What if I do not successfully demonstrate my skills in assessment?
After deciding all of the What's Next general and specific information for each page in the Module it is important that you review the Design Document for each Activity. Each content developer will fill in a What's Next section for any additional information they feel should be incorporated for that page. These points should also be added to the What's Next section for all necessary pages.
Refer to a What's Next Template to prepare the What's Next deliverable. This template is an Excel spreadsheet that has the sections of the module listed on the left side, areas for page keys, super keys, the question asked, and the answer. There are also standards and tips located on the bottom of the template for your reference.
Test Approaches
Assembly Test Approach

This test is designed to ensure that all components within the application are able to communicate with each other effectively. The Assembly Test was performed immediately after the Component Test. The development team members on both the functional and technical sides conducted this test on the pages they respectively developed. This was decided since the developer of the page is most familiar with the components and links they developed, and could therefore easily navigate through the application. Additionally, their familiarity with the individual pages allowed them to quickly identify whether or not anything was missing or malfunctioning more easily than someone who was unfamiliar with the page.

Component Test Approach

The component test was designed to ensure that all the components of each individual page of the application function correctly and as expected. This test was performed prior to all other tests so that a majority of the system bugs could be identified and fixed prior to the later sequences of testing. The development team members (on the functional and technical sides) were responsible for conducting this test since they were most familiar with the individual components they developed and could most easily navigate through the untested application efficiently.

Scripting

To test the Module as a 'day in the life', testers needed to work through the application as if they were users. In order to thoroughly and completely work through the application, testers used scripts. These scripts allowed the testers to complete the training module while simulating all the actions and procedures that the end-users may eventually execute, hopefully encountering any potential problems before the program actually went into production.

Product Test Approach

The Product test was designed to ensure that all pages and components of the application worked correctly and as expected. This test was performed in conjunction with all other tests (functional, performance, assembly, etc.). The component test ensures that each of the individual parts should already be tested and functioning correctly as stand-alone pieces of the whole. The product test then grouped all of these individual components together and allowed The development team to test the application as a whole before rollout to the end users.

Process for Fixing an Error (Functional & Technical)

The testing process uncovered a variety of errors within the execution of the application. Each of these errors had to be corrected before rollout to the end user. There were distinct processes and procedures in place to address issues as they came up, funnel them to the appropriate "fixer", and retest the subject matter again once the error had been fixed. SIRs were filled out to log the error, and were then handed off to the point person to log the issue and assign it to be fixed. Once fixed, the subject was retested to identify that the error no longer existed.

Design and Build—From a Functional Perspective

Training and Development Checklist

The purpose of this portion is to supply a list of items that the client Subject Matter Experts (SMEs) should use/check off when reviewing the content of a given Module. By creating this portion, we are reducing the chance that any usability/functionality issues are not tracked during the SME review process as well as educating the SMEs about what they should be looking for when reviewing the Modules.

Training Detail Design and Build Approach

The purpose of this portion is to outline the methodology that was followed for the detailed design and development of training by the Project. It is intended to provide the team with a common understanding of their roles and responsibilities, as well as information on the key detailed design and development tasks, deliverables and deadlines. This portion should be read in conjunction with the other portions of the documentation, in order to gain a full understanding of the entire process from design to implementation.

Training and Development Standards

This portion is to assist content developers by outlining the standard format and guidelines when designing and developing content for component based training modules. Templates have been developed to provide an illustration of the standards and should be utilized in the design and development phases. This portion should be considered a "living document" and as such will be updated when appropriate improvements are identified.

This portion includes the following information:

Getting Started

Terminology Standards

Formatting Standards

Html Tags

Naming conventions

Activity Design and Development Templates

Page Instructions

Writing Feedback for multiple choice quizzes (for correct and choice and incorrect answer choices)
Developing Video

The purpose of this portion is to outline the approach to developing the video content of the Modules.

This portion describes in detail the steps taken to prepare, film, edit, select and place all of the video clips used for Module 2.

Developing Audio-Clips

The purpose of this portion is to outline the approach to developing the audio content of the Modules.

This portion describes in detail the steps taken to prepare, script, record, select and place all of the video clips used for Module 2.

Certification Plan and Approach

CSS Windows

Capturing CSS windows allows builders to use real life examples in training modules. In this reference guide it is assumed the audience has had experience with CSS/Customer 1 applications. The purpose of the portion is to outline the steps needed to capture and manipulate images to a usable format.

Tool Bar Development

The purpose of this portion is to identify each icon in the tool bar and document the necessary research needed for each during the creation of a new module.

What's Next

The What's Next? feature answers some commonly asked questions that the user may have related to using and/or navigating through the module.

Notebook

The Notebook feature allows the user to either directly type in notes or cut and paste notes into the workspace provided. The user can save their notes as they continuously enter the notebook. At the end of the module, the user can and print out the their notes.

Video

The Video feature contains all of the videos marked as links throughout the module. The Video feature presents all of the videos that appear throughout the module and groups them by topic.

Reference

Reference materials may be gathered for each training module. This section details how to research existing company job aids and reference materials located on CSS (Customer Service System) and develop new reference materials. This portion also details how to enter reference materials into the database.

Save and Exit

The Save and Exit icon allows the user to save his/her work and exit the module. The next time the user enters the module, he/she may resume where they left off.

Map

The Map is a navigational tool used to access any portion of a module that they user has already entered.

Training Modules

Following is a brief description of the modules, and a summary of the activities that took place during design and development of Module 2.


Module Name	Training Method	Duration

Change Meter Orders	Internet-based,	2 hours
	Self-paced
High Bill Inquiries	Internet-based,	2 hours
	Self-paced

Formulation of Learning Objectives

Each module was further broken down into distinct Activities, each of which addressed several distinct learning objectives. SME's at were consulted to provide information on the issues that the training should address. Since Module 2 was designed to replace existing Instructor-Led-Training, the concepts covered in that training were discussed. The Team and the SME's also decided which additional value-added subjects should be covered in Module 2 to increase the effectiveness of the training, and differentiate the module from its predecessor. The result of these discussions was a list of Learning Objectives, or general areas of subject matter, that guided further design and development of the module details.

Design of the Module

Following is a description of the design of Module 2. This will examine the design of the Activities and Exercises, which taken together compose the entirety of the complete module.

Design of Activities

The list of Learning Objectives was then examined to determine which objectives could be grouped together