System and methods for improved program testing5475843Abstract The present invention includes a Computer-based Training system (CBT) having one or more Application Translation Units (ATUs), a Message Engine, and a Script Engine. For one or more target applications of interest, an ATU is provided for processing events specific to that application, thereby trapping events and translating them into abstract messages or "meta-messages" for conveying information about a particular event to the system. A computer-aided software testing embodiment of the present invention is also described. The system provides prefabricated building blocks for constructing a high-level model of an application's User Interface (UI). This high-level model serves as a middle ground between test scripts and the application being tested. The knowledge of how a given UI element is controlled or how it can be observed is retained in the model rather than in a test script. Consequently, the test script consists of easy-to-maintain, high-level testing commands only. Claims What is claimed is: Description COPYRIGHT NOTICE
______________________________________
while (GetMessage (&msg, NULL, 0, 0))
{
TranslateMessage (&msg) ;
DispatchMessage (&msg) ;
}
return msg.wParam ;
______________________________________
where a message (&msg) is retrieved by a call to GetMessage (step 263); if needed, the retrieved message may be translated by a call to TranslateMessage() and then dispatched by a call to DispatchMessage (step 269). This "while" loop continues until the GetMessage function returns a value of zero--indicating that the loop has read a WM.sub.-- QUIT message from the queue, telling the application to end (yes at step 265). In addition to the messages generated by Windows, applications can create their own messages and place them in the application queues of other applications. The SendMessage and PostMessage functions let applications pass messages to their windows or to the windows of other applications. The PostMessage function directs Windows to post the message by placing it in the application queue. Control returns immediately to the calling application, and any action to be carried out as a result of the message does not occur until the message is read from the queue. The SendMessage function, on the other hand, directs Windows to send a message directly to the given Windows function, thereby bypassing the application queue. Windows does not return control to the calling application until the Windows function that receives the message has processed it. Such an unqueued message, however, is generally a message that affects the window only. The general mechanism for retrieving and dispatching messages in an event-based system, such as Microsoft Windows, is known in the art; see, e.g., Petzold, C., Programming Windows, Second Edition, Microsoft Press, 1990. Additional information can be found in Microsoft's Window Software Development Kit, including: 1) Guide to Programming, 2) Reference, Vols. 1 and 2, and 3) Tools, all available from Microsoft Corp. of Redmond, Wash. The disclosures of each of the foregoing are hereby incorporated by reference. First embodiment: Computer-based Training System The following description of the CBT system of the present invention will focus on the presently preferred embodiment which includes components implemented in an event-driven architecture with the C++ programming language. In particular, an object-oriented model is adopted whereby new objects may be created from existing objects (classes). The general features of C++, including data encapsulation, inheritance, and polymorphism, as well as C++ techniques for implementing class hierarchies and class methods are known; see e.g., Ellis, M. and Stroustrup, B., The Annotated C++ Reference Manual, Addison-Wesley, 1990. Additional information about object-oriented programming and C++ in particular can be found in Borland.RTM. C++3.1: 1) User's Guide, 2) Programmer's Guide, and 3) Library Reference, all available from Borland International of Scotts Valley, Calif. The disclosures of each of the foregoing are hereby incorporated by reference. A. Overview Referring now to FIG. 3, a Computer-based Training system 300 of the present invention includes one or more Application Translation Units (ATUs) 340, a Message Engine 350, and a Script Engine 330. Also shown, the system 300 includes a CBT window object 370 operably coupled to the Script Engine; CBT window object 370, in turn, may be linked to one or more custom Dynamic Link Libraries (DLLs) 380. Driven by a set of instructions or script 320, the system 300 monitors various events of the Windows system and desired target applications. Messages from these events, including system messages 310 and target application messages, are trapped by the ATUs 340 and reported to the Message Engine 350 as CBT messages. The Message Engine, in turn, dispatches the messages according to handlers specified by the Script Engine 330, operating under the control of script 320. Messages of interest are processed as desired; others are simply allowed to pass through. The construction and operation of these components will now be described in further detail. 1. Application Translation Units and their Target Applications In normal operation of the system 100, a user is using one or more Windows application programs, for example, programs 145 of FIG. 1B, which can be a spreadsheet, wordprocessor, database, or the like. For each application where CBT support is desired, an application-specific ATU 340 is provided for processing events specific to that application (now shown as target application 360 in FIG. 3). Thus, each ATU 340 is a module, preferably implemented as a Dynamic Link Library (DLL), for processing messages for a specific target application. In addition to the application-specific DLLs, ATUs 340 include a Windows ATU. In contrast to the other ATUs which hook into specific target applications, the Windows ATU processes all Windows events, including system messages 310. In this manner, general system events, that is, ones not specific to any particular application, may be processed by the CBT system 300 as well. Whether application specific or not, each ATU functions to trap events and convert them into "CBT messages"--a lingua franca or common language for all events, whether Windows or application-specific, occurring within the system. More particularly, a CBT message is a meta-message, that is, a high-level message describing one or more events which have occurred. For instance, instead of monitoring numerous, low-level system messages, such as WM.sub.-- RBUTTONDOWN, WM.sub.-- LBUTTONDOWN, WM.sub.-- RBUTTONUP, WM.sub.-- LBUTTONUP, and the like, the user/script writer need only monitor a single message CBT.sub.-- MOUSECLICK; the task of determining what the Windows system or application is doing is left to the ATUs. By abstracting low-level system messages into high-level (and more meaningful) CBT messages, the system of the present invention greatly simplifies script design and writing. A CBT message, which is stored internally as an integer, represents a particular type of event. Information or data particular to each event, such as active window, cursor location, and the like, on the other hand, are packaged as a discreet data object (EventInfo object) fully describing the event. EventInfo objects, along with CBT messages, are dispatched from the ATU to the Message Engine 350 for further processing. 2. Message Engine and Scripts The Message Engine 350 determines which of the many CBT messages it receives is of particular interest to the CBT system, as it operates under the direction of a script 320. At startup, each target application of interest to the script 320 is registered with the Message Engine. The script 320, which itself includes a collection of objects, may be compiled, pre-compiled, pseudocompiled, or the like, as desired for a particular platform. In a preferred embodiment, script 320 is pre-tokenized, whereby source listings and comments (optionally) are stored in a compact form which may be reconstructed into the original source. Thus, script 320 need only be a set of instructions; no particular format is required by the present invention. For each application registered, the Message Engine maintains one or more "handlers" or modules for processing events specific to that application (as represented by the CBT messages). Thus, the Message Engine manages a list of target handlers for determining which CBT messages are of interest to the script 320, that is, which messages map into the list. Messages which are of interest to the system, i.e., those specified by the script 320, are trapped and forwarded to a designated handler. Those messages which are not of interest are allowed to "fall through" (i.e., be passed on to) the individual target applications 360. In essence, the Message Engine filters the events for a particular application so that only those of interest, that is, those having a handler defined for the event, are processed. B. System operation 1. Method: CBT session Referring now to FIGS. 4A-B, the general method of operation for the system 300 is illustrated by a flowchart 400. While the general methodology remains the same from one CBT session to another, the reader should understand that the specific steps of any one CBT session are dictated by instructions and script objects defined in the script 320. For instance, the script 320 states which specific target applications will be registered with the Message Engine and which events of those applications will be dispatched. Thus, the following description illustrates the basic framework in which the CBT system operates. Under the direction of the script 320, at step 410 the CBT system registers a target application of interest by creating a CBTApplication object. Serving as the main entry point for the CBT system, this object initializes and executes the script tutorial. Moreover, the object initiates sessions with the Message Engine and Script Engine and acts as a centralized dispatch point for all external functions and object method calls within each CBT lesson. From here individual CBT lessons are loaded and performed. When a lesson script is loaded the CBTApplication object creates a CBTLesson object which is responsible for managing that particular lesson. The CBTLesson object reads the lesson script and builds a deck of CBTScene objects, with each corresponding to a scene described in the lesson script. Alternatively, each scene may be constructed on-the-fly, particularly in high-performance environments. The CBTLesson object resembles a state machine; it maintains the active scene (active state) and sends appropriate script scenes (CBTScene objects) to the Script Engine 330 for presentation. Each object is directly accessible to the script writer through script variables; for example, the CBTApplication object is accessed through a theCBTApp global variable, the CBTLesson object through a theCBTLesson global variable. To complete the initialization of step 410, the target application is registered with the Message Engine 350. In particular, hooks are installed by a corresponding ATU 340 so that events within the target application of interest are trapped. As set forth below, these events will, in turn, be reported to the Message Engine 350 as CBT messages. At step 420, script-specified links are established to individual resources of interest. Within the target application itself, various resources (e.g., buttons, menus, and the like) may be of interest to the script writer. For example, if one were interested in a particular button of the target application, such as a "help" button, that button may be registered with the Message Engine 350. Events associated with that button (e.g., "mouse enter" the button) are then trapped for processing by an ATU. The links are specified within the CBT script 320 as follows. In a preferred embodiment, a link is established to an individual resource or control by indicating a selected one of its Windows class name, Window title, or Resource ID, all of which are readily accessible Windows data structures. Commonly, a link will be established by using the Resource ID. Particular links may be created or removed during a session, as desired. At step 430, the system traps various events which are relevant to the established links. This operation, shown in further detail in FIG. 4B, occurs as follows. Every time that a message arrives at the message queue of a target application, a message filter hook function (WH.sub.-- MsgFilter) 431 is called. First, the hook function determines if the message represents a task which has been "hooked," that is a specific link for this event of the target application has been created. If not, the event is passed down the hook chain for processing by one of the other hooks (i.e., hooks 433, 435, 437). The Windows hook 437, for example, traps all window messages (WM.sub.--). In this fashion, which hook function is invoked depends on what type of message comes into the target application. By way of illustration, hooks may be installed as follows:
__________________________________________________________________________
BOOL installHooks(CbtEntry *pEntry)
if( ! pEntry )
return FALSE;
// Note that the fourth parameter below may receive a task handle of
// a target application, whereupon the hook is installed in that
// application. When receiving a NULL parameter, the hook installs
// to applications.
// CBT hook -- allow CBT system to stop events from progressing
hCBTHook =
SetWindowsHookEx(WH.sub.-- CBT, (HOOKPROC)cbtFilter, hInstance, NULL);
// Msg filter hook -- dialog boxes and menus
hMsgHook =
SetWindowsHookEx(WH.sub.-- MSGFILTER, (HOOKPROC)msgFilter,
hInstance, NULL);
// Get msg hook
hGetMsgHook =
SetWindowsHookEx(WH.sub.-- GETMESSAGE, (HOOKPROC)getmsgFilter,
hInstance, NULL);
// Windows hook
hCallWndProcHook =
SetWindowsHookEx(WH.sub.-- CALLWNDPROC, (HOOKPROC)callWndProcFilter,
hInstance,NULL);
return( hCBTHook && hMsgHook &&
hCallWndProcHook && hGetMsgHook);
}
__________________________________________________________________________
As shown, a callback function is installed for each hook; each callback function, in turn, serves as an entry point into an ATU. Additional information on the installation and use of hooks in Microsoft Windows can be found in the Microsoft Windows Software Development Kit referenced above. In addition to installing hooks to trap system messages, one or more application-specific hooks (callback functions) 439 are installed as well. For instance, a target application may be a spreadsheet which includes its own internal set of messages, for example, SS.sub.-- CELLSELECT, SS.sub.-- CELLEDIT, SS.sub.-- BLOCKDEFINE, and the like. To monitor these messages, an ATU may register an application-specific callback function with the target application, in effect dynamically binding the ATU to its target application. At runtime, the application invokes the callback function for dispatching internal or application-specific messages. Thus, the CBT system of the present invention is not limited to Windows events and their messages; instead, the CBT system may receive and translate any messages of interest, whether system-wide or strictly internal. At step 440, events which are trapped by the Application Translation Units 340 are "dispatched" to the Message Engine 350 as CBT message/event information objects. As shown in particular detail in FIG. 4B, the dispatch module of each ATU includes a function corresponding to each Windows event. Thus, for the WM.sub.-- COMMAND, WM.sub.-- MENUSELECT, WM.sub.-- BUTTONDOWN, and WM.sub.-- SETCURSOR messages, the following translate functions may be defined: int doWM.sub.-- Command (CbtEntry *pEntry, MSG *msg); int doWM.sub.-- MenuSelect(CbtEntry ,pEntry, MSG *msg); int doWM.sub.-- ButtonDown(CbtEntry *pEntry, MSG *msg); int doWM.sub.-- SetCursor (CbtEntry *pEntry, MSG *msg); each designed for processing its particular event. The operation of an ATU dispatcher will be demonstrated by the processing of Windows messages for determining if the cursor has traversed a window boundary (i.e., entered a new window); this example illustrates how a multitude of Windows WM.sub.-- SETCURSOR messages can be converted into MOUSELEAVE and MOUSEENTER meta-messages. The dispatching of other events as CBT messages is set forth hereinbelow as Appendix A. As shown in FIG. 5A, a doWM.sub.-- SetCursor dispatch method 500 is invoked whenever a SetCursor message is trapped by the ATU (i.e., captured by a hook function before the event has been received by the application). Here, the script writer is not interested in the screen cursor simply entering an already active window; thus, the method simply "allows" the Windows message to be passed to the target application at step 501 and returns. Specifically, since the screen cursor is entering a window which is already active, no particular CBT-generated message or other intervention is desired by the script writer at this point; hence, the WM.sub.-- SETCURSOR message is allowed to pass through. Continuing the example, the script writer may specify that the event of a cursor leaving an active window is of interest and should be trapped. Since the cursor is not simply re-entering the active window (no at step 501), the window which the cursor is leaving should be notified of the event. The CBT system notifies the Message Engine of this action as follows. First, at step 502, the information pertaining to the window where the event occurred is encapsulated into a C++ object (which is derived from an EventInfo class hierarchy, described in further detail hereinbelow). At step 503, the information object and a "MouseLeave" message are dispatched to the previous (departing from) window, with the message being denoted as a "NOTIFY" message. In a preferred embodiment, two different classes of messages are provided: CBT.sub.-- NOTIFY and CBT.sub.-- CONFIRM. Those messages which are purely informational, such as mouse movements, are CBT.sub.-- NOTIFY messages. Those which can be stopped before they reach the target application, on the other hand, are called CBT.sub.-- CONFIRM messages. Each is registered with Windows as an application-specific Windows event. Using two methods defined within EventInfo objects, the script 320 can allow or prevent a CBT.sub.-- CONFIRM type message from reaching the target application. Specifically, a stopMessage method is invoked which determines (based on script instructions) whether to allow the message to pass through to the target application. After step 503, the method proceeds to alert the Message Engine that the cursor is entering a new window. In a manner similar to sending the "MouseLeave" message, the method first builds an EventInfo object at step 504. Next, a "MouseEnter" message of type CBT.sub.-- NOTIFY is dispatched to the application, along with the information for the event (EventInfo object), at step 505. At step 506, an active window flag is set; this is the flag that is read in step 501 to determine if the mouse cursor is entering the active window. Finally, the method concludes by passing the message on to the application (i.e., "allow" message) at step 507. At the conclusion of the method, memory for the EventInfo objects may be recovered (e.g., using manual or automatic garbage collection techniques). For purposes of illustration, one may implement such a method in the C++ programming language as follows:
__________________________________________________________________________
int doWM.sub.-- SetCursor(CbtEntry *pEntry, MSG *msg)
EventInfo *pObject = (EventInfo *)NULL; // init EventInfo
if( pEntry->hActiveWnd == (HWND)(msg->wParam) ) // steps 510/507
return MSG.sub.-- ALLOW;
pObject = new WindowInfo( pEntry->hActiveWnd ); // step 502
DispatchToCbt(pEntry, CBT.sub.-- NOTIFY,
TM.sub.-- MOUSELEAVE, (LPARAM)pObject); // step 503
pObject = new WindowInfo( (HWND)(msg->wParam) ); // step 504
DispatchToCbt(pEntry, CBT.sub.-- NOTIFY,
TM.sub.-- MOUSEENTER, (LPARAM)pObject); // step 505
pEntry->hActiveWnd = (HWND)(msg->wParam); // step 506
return MSG.sub.-- ALLOW; // step 507
// garbage collection performed by the system
}
__________________________________________________________________________
Here, pEntry is a pointer to a record, CbtEntry, which is updated. The record includes handles to the hooked task (application that has been hooked) and the currently active window:
______________________________________
typedef struct .sub.-- CbtEntry {
HTASK hHookedTask;
HWND hActiveWnd;
} CbtEntry; // pEntry points to this
______________________________________
As shown, a meta-message may maintain its own data structures for tracking events at a higher level (e.g., the status of the active window). The DispatchToCbt function, on the other hand, is conveniently viewed as two halves of a process. Specifically, the Message Engine registers a callback function with the ATU. The operation proceeds as follows. On the ATU side, the ATU passes to the DispatchCBTMessage method a task handle for identifying a particular application; since the system processes multiple applications, this mechanism serves to distinguish between different applications (and their instances):
__________________________________________________________________________
void DispatchToCbt(HTASK hHookedTask, UINT Msg, WPARAM wParam, LPARAM
lparam)
if( pCbtDispatchFunc )
(pCbtDispatchFunc)(hHookedTask, Msg, wParam, lparam);
}
__________________________________________________________________________
In this manner, an ATU filters or groups events by target application and communicates its events as meta-messages to other CBT components. At the completion of step 440 of FIG. 4A, the ATU has dispatched the CBT message, its type, and the EventInfo object to the Message Engine 350, thereby fully communicating an event which it has trapped. Not all events are of interest, however. Thus, the events should be filtered so that only those of interest are acted upon. At step 450, the Message Engine performs this task by comparing the CBT message against known event handlers. In other words, the engine attempts to dispatch CBT messages of interest--ones having a handler define for the event. Thus on the Message Engine side, the Message Engine determines which entry in its table corresponds to that task:
__________________________________________________________________________
void CALLBACK MessageEngine::DispatchCBTMessage(HTASK hTarget, UINT
cbtmsg, WPARAM wParam, LPARAM lParam)
CbtSession *pSession = GetSessionFromTask(hTarget);
if( pSession )
SendMessage(pSession->hMsgPane, CbtMessages[cbtmsg],
wParam, lparam);
// Notify application handler
// where pSession is the current application session
// (determined from hTarget);
// CbtMessages[cbtmsg] is the table lookup for the
// CBT message ("confirm" or "notify";
// wParam is the CBT message type (TM.sub.-- msg); and
// lParam is a pointer to the EventInfo object.
}
__________________________________________________________________________
With particular reference to FIG. 4B, this process is illustrated. Message Engine filters CBT messages through a plurality of message handlers, including, for example, a TargetApplication handler 451, a TargetWindow handler 453, a Custom handler 455, and a Default handler 457; other exemplary handlers are set forth hereinbelow as Appendix B. CBT messages of interest will be passed to a particular handler. As shown in Appendix B, each message belongs to a particular Message Handler Class and is either informational (CBT.sub.-- NOTIFY) or preventable (CBT.sub.-- CONFIRM). A "mouseEnter" message, for example, belongs to a TargetWindow Handler Class; the message is therefore processed by the TargetWindow handler 453. An application specific event, such as an SS.sub.-- EDITCELL message from a spreadsheet target application, on the other hand, would typically be passed to the TargetApplication handler 451. Finally, messages without a handler, that is, those which the script writer has no particular interest, may be passed to a default handler (e.g., for ignoring, discarding, or otherwise invoking a desired default action); thus, the script writer need only enumerate handler methods for messages of interest. If matched with a handler, the message is then dispatched. Specifically, the handler extracts properties for the message and the accompanying EventInfo object. For a message of TargetWindow handler class, for instance, available object properties include: 1) Name: Title string of the control; 2) Class: Windows class of the control; 3) ID: Resource ID of the control; 4) Style: Style flags of the control; 5) Enable: Whether the control is enabled; 6) Position: Coordinates of the control; and 7) EventInfo: Current EventInfo object, if any. Additional exemplary properties which are available for various messages handler classes are set forth hereinbelow as Appendix C. As an alternative to defining several event handlers, a more preferred embodiment provides only two basic event handlers: an Application Link handler and a Window Link handler. Each is an object having various handler methods for appropriately responding to each message passed. Based on the lookup performed by the Message Engine (i.e., NOTIFY or CONFIRM), an Application Link handler may, for example, effect the dispatching as follows:
__________________________________________________________________________
RESULT CALLBACK ApplicationLink::wndProc(HWND hwnd, UINT Message,
WPARAM wParam, LPARAM lparam)
// First, get handler (from link registered)
ApplicationLink *pWindow = (ApplicationLink *)GetWindowLong(hWnd, 0);
if( pWindow )
{
if( Message == CBT.sub.-- notify ) // Msg is a NOTIFY msg
{
EventInfo *pInfo = (EventInfo *)lParam;
// Recall that Event info includes one or more of a win class,
// name, and resource ID. If exact information is not provided
// (e.g., just "OK" button), do "fuzzy" match (i.e., match as
// much as possible:
pWindow->doDispatchNotify(wParam, pInfo);
pInfo->Finished( );
return TRUE,
}
else if ( Message == CBT.sub.-- confirm // Msg is a CONFIRM msg
{
EventInfo *pInfo = (EventInfo *)lParam; // Event info
pWindow->doDispatchConfirm(wParam, pInfo);
pInfo->Finished( );
return TRUE,
}
}
// on return, call WinProc instantiated w/ applic. link
return DefWindowProc(hWnd, Message, wParam, lParam);
}
__________________________________________________________________________
Here, the doDispatch- methods communicate directly with the Script Engine. In turn, the Script Engine responds according to script objects defined within the active script. By invoking the stopMessage method for specifying whether an event is allowed, for example, events may be stopped from reaching a target application; in most instances, however, the script writer will simply specify the default--that the event should simply pass on through to the target application. The script writer may provide methods for handling the various events of interest, or he or she may rely on default methods which are defined by the CBT system. In operation, a CBT message is passed to the objects. Each object, in turn, is responsible (through its method definitions) for knowing which messages are of interest, and how each one of interest should be processed. In a target application, for example, if the script writer hooks a window link up to a list box of the application, he or she should provide methods for handling the event (as communicated by CBT messages) of that list box. Referring now to FIG. 5B, the overall method of dispatching messages is summarized. In a doDISPATCH method 520 of the present invention, a CBT message arrives at the Message Engine and is processed as follows. First, in step 521, an attempt is made to match the message to an application link handler. If the attempt is unsuccessful (no at step 522), then the message is simply allowed to "fall through" (i.e., left unprocessed, or processed by a default handler). Otherwise (yes at step 522), at step 523 the Message Engine forwards the CBT message (with EventInfo object) to the identified Application Link handler. At step 524, the Application Link handler examines the EventInfo object and attempts to match it with a registered window link. If this step is unsuccessful (no at step 525), then a Default handler will be assigned for processing the event at step 526. At step 527, the message is forwarded to the Window Link handler. The Window Link handler, in turn, dispatches the message to the Script Engine at step 528. At this point, the Script Engine identifies the event by mapping the message into its set of known reserved words. At step 529, the Script Engine processes the message according to the instructions of the script (i.e., effects the action desired by the script writer, as indicated by the use of the matching reserve word). Upon completion of this step, the method has successfully dispatched the meta-message, with appropriate actions being effected in response to that message. 2. Building CBT Lessons As a tutorial is designed, the CBT script writer creates a "storyboard" showing the visual appearance as well as the flow of the tutorial. The storyboard becomes the basis for the CBT lesson script. CBT scripts are written in a simple language which contains both message handling and object-oriented features. Each lesson script is organized as a collection of scenes, with each scene describing the actions that take place at a particular point in the lesson. For example, a scene might instruct the CBT system to display a window containing some text when a particular menu item is chosen in the target application. As the lesson script proceeds, new scenes can be performed. This process continues until the user chooses to exit the CBT or the lesson is finished. To control the target application, the CBT system intercepts all Windows events for the application and translates them into CBT messages. These messages will trigger any corresponding message handlers which are defined within a scene. When a message handler is triggered, its script is executed. Within each scene, message handlers are defined for each UI control in the application which is of interest. For example, to respond to a button click within the script the following handler is defined:
______________________________________
script for Scene1
TargetButton theButton(120)
on theButton.buttonClick
theCBTLesson.perform("Scene2")
end
end
______________________________________
This hypothetical scene creates a TargetButton object which is linked to the UI control in the target application; the UI control Resource ID is 120. Next, a Message Handler is defined for the buttonClick message from the TargetButton object. When this message is received, the Script Engine performs a new scene, Scene2. Thus, the statement: theCBTLesson.perform("Scene2") calls the perform method of the global object "theCBTLesson" (the CBT Lesson object). In addition to controlling user actions, the CBT lesson also drives the target application autonomously by sending appropriate messages. Alternatively a sequence of events can be recorded (e.g., using a tool similar to MS-Windows Recorder) for later replay. The script may also query the target application for its current properties. If a window link is established to a button, for instance, the script may query the button for its properties, such as its size, its title, and the like. One should note that the ability to query for properties operates independently of the processing of events and their messages. As another example, a target application could be asked to enumerate all the buttons of a dialog box. The script may, in turn, act on the queried information, including modifying selected resources. In this fashion, the resources of a target application may be dynamically varied, thereby providing the target application with an alternative user interface--one having UI components which may be altered on the fly. Appended herewith as Appendix D is a source listing illustrating an exemplary script syntax and methodology for operating the CBT system of the present invention. Additional reference materials illustrating a preferred script syntax are appended herewith as Appendix F. 3. Multiple-application Lessons As shown by FIG. 3, the system of the present invention is operative with one or more applications 360. More particularly, according to the present invention, a single script 320 may be employed to not only control multiple applications concurrently, but also control interaction between multiple applications. A script may be provided for tutoring the user in the operation of cutting and pasting between applications, for instance, cutting text from a word processor and pasting it into a database application. Thus, the CBT system 130 is not application bound; instead, it is a complete subsystem--one which may control multiple applications, including interactions between applications and/or the operating system, even launching additional applications as needed. 4. Event Information (EventInfo) Objects An EventInfo object, which stores information about a particular event, is instantiated from an EventInfo class 550. FIG. 5C illustrates the EventInfo inheritance tree and lists the properties for each class. As shown, the EventInfo class hierarchy 550 includes nine derived EventInfo classes which contain the state information about the various standard CBT messages. At the root of the hierarchy is the EventInfo base class 551. In a preferred embodiment, this class may be constructed with the following C++ class definition:
______________________________________
class SHARED.sub.-- CLASS EventInfo : public CbtObject,
public pEventInfo {
public:
ATOMTABLES(Keyword, 7)
protected:
HWND hwndTarget;
BOOL bAllowMsg;
BOOL bIsProcessing;
public:
EventInfo(HWND hwnd);
virtual .about.EventInfo( );
virtual
int Supports(hProtocol &Hdl) const;
inline HWND WindowHandle( ) const;
virtual
const char *
WindowName( ) const = 0;
virtual
const char *
WindowClass( ) const = 0;
virtual
int WindowId( ) const = 0;
virtual
LONG WindowStyle( ) const = 0;
virtual
BOOL AllowMessage(BOOL bFlag, BOOL
bState);
virtual
BOOL ProcessingMsg( ) const;
virtual
void Finished( );
inline void * operator new(unsigned size);
inline void operator delete(void *p);
ATOMMETHODS(Keyword)
CLASSMETHODS(EventInfo, "EVENTINFO")
};
______________________________________
As shown, the EventInfo class 551 provides access to the Windows name and its class, its resource ID, its Windows style, and whether the message is allowed (according to the script 320). Derived from EventInfo class 551 is WindowInfo class 552, a pure virtual base class for other EventInfo classes. The subclass affords the same four pieces of information which were provided by the base class 551. In addition, for a given window handle, the object will extract a window name, class, ID, and style. The class may be constructed with the following C++ class definition:
______________________________________
class SHARED.sub.-- CLASS WindowInfo : public EventInfo {
protected:
int iWindowId;
LONG lWindowStyle;
char *
strWindowName;
char *
strWindowClass;
public:
Windowinfo(HWND hWnd);
virtual .about.WindowInfo( );
virtual
int Supports(hProtocol &Hdl) const;
virtual
const char * WindowName( ) const;
virtual
const char * WindowClass( ) const;
virtual
int WindowId( ) const;
virtual
LONG WindowStyle( ) const;
CLASSMETHODS(WindowInfo, "WINDOWINFO")
};
______________________________________
In addition to the windowing information, other events are also of interest, particularly mouse and keyboard events. These other events are accessible through subclasses of WindowInfo class 552. Specifically, the WindowInfo class spawns five subclasses: WinHelpInfo class 561, WinPositionInfo class 563, WinShowInfo class 565, WinSelectInfo class 567, and KeyboardInfo class 569. As shown, objects may be instantiated from these subclasses for accessing help text, window position, menu information, keyboard events, and the like. WinHelpInfo, for instance, contains the text which was sent by the WinHelp engine to the CBT. This text can be a sequence of script statements which are executed or simply a string of text. WinPosition provides the coordinates of the window. WinShowInfo contains the SWP.sub.-- flags corresponding to the Windows ShowWindow() function. WinSelectInfo contains the name of the selected menu or control window item. KeyboardInfo contains the key that was pressed as well as any flags indicating if the <ALT>, <SHIFT>, or <CTRL> keys were also pressed. Two classes, WinPositionInfo class 563 and WinSelectInfo class 567, spawn additional subclasses. As shown, MouseInfo class 571, which is derived from WinPositionInfo class 563, provides direct access to mouse events; it contains the mouse button that was clicked and whether it was a single or double click as well as the position of the mouse. WinSelectInfo class 567, on the other hand, spawns MenuInfo class 573 and ListInfo class 575. The former provides access to menu IDs and menu flags, the flags indicating whether the menu item is grayed, highlighted, or the like; the latter provides the index of the selected item, which is important if the list does not contain text. Second embodiment: Computer-aided Software Testing System A. Introduction An alternative embodiment of the present invention, one adapted for automated testing of GUI applications, will now be presented. Before discussing the construction and operation of the computer-aided software testing embodiment in detail, it is helpful at the outset to briefly review parameters for quantifying the effectiveness of software testing. By automating the tasks of "controlling" and "observing" the target software, the burden of testing for the QA engineer is substantially reduced. In other words, the more successfully the QA engineer can control and observe the software program, the more effective the test becomes to automate. This can be restated as a formula: Test Automation Effectiveness=%Controllability X %Observability where Controllability is a rough measure of how effectively the program can be automatically controlled into a given state, and Observability is the measure of how effectively a bug can be automatically observed when the program being tested is in a faulty state. Test Automation Effectiveness is the product of these two factors for the program under examination. Consider, for example, the task of testing a software compiler having both command-line and GUI components (such as found in commercially-available compilers from Borland and Microsoft). Conceptually, testing the GUI portion of the compiler should be easier than testing the command-line portion, owing to the latter's complex internal operations. In practice, however, it is far easier to test the command-line portion. The QA engineer need only develop test programs or data files that are run through the compiler using a simple batch file. Here, it is easy to achieve total control for the non-GUI portion. Observability is also high because return values or variables denoting the state of the compiler can be tested and written to a file using simple program statements that print the results. The task of automating the control of a GUI program, in contrast, is particularly difficult. Specifically, the task of maintaining pre-recorded input (i.e., keyboard and mouse actions) and test scripts is not only very time consuming but also prone to error. If the degree of controllability and observability of the automated tests can be increased, the task of testing GUI software may be made easier. According to the present invention, the QA engineer constructs a high-level model of an application's UI using prefabricated building blocks. This high-level model serves as a middle ground between test scripts and the application being tested. The knowledge of how a given UI element is controlled or how it can be observed is retained in the model rather than in a test script. Consequently, the test script consists of easy-to-maintain, high-level testing commands only. For instance, an exemplary script for opening a particular file (e.g., "testl.txt") may be defined as follows:
__________________________________________________________________________
TheApp.FileMenu.Open.Select( )
// Select File open menu
// ActiveDialog now represents the File Open dialog
ActiveDialog.Filename.Set("test1.txt")
// Type in the file name
ActiveDialog.OK.Click( ) // Click on the OK button
__________________________________________________________________________
One need only include high-level statements defining the operation, such as shown above. B. System overview Referring to FIG. 6, a software testing automation system 600 of the present invention includes Generic Element Models (GEMs) library 610, Application-specific Testing Models (ATMs) 615, Resource Database 620, one or more Model Generators 625, Test Runtime Library 630 (for storing Test Objects or Test Functions, typically as one or more Dynamic Link Libraries (DLLs)), Script (interpreter) Engine 635, Message Engine with one or more Application Translation Units (ATUs) 640, and an in-memory Testing Model 655 (of the application under test), and one or more test scripts 660. Communication directly with the application under test 665 is effected through Windows API 670 and/or Test Port 650. Using Windows API 670, various Windows messages may be sent or posted to the application for querying the application or effecting particular operations, as is known in the art. For instance, one may determine at runtime whether a checkbox screen object is checked (i.e., toggled to its "Checked" stated) by invoking the standard Windows API call: IsDlgButtonChecked (hwndDlg, idButton) where, hwndDlg is the handle of dialog box and idButton is the identifier of the button/checkbox. Documentation for the Windows API is provided in the Microsoft Windows Software Development Kit (SDK), available directly from Microsoft (and several other vendors). In addition to communication with the application under test 665 through the Windows API, application-specific messages may be registered with the Windows OS, as is known in the art. Test port 650 represents the use of such application-specific messages for communicating with the application under test 665. In general operation, the system employs the Model Generator 625 to decompose the application under test 665 to generate the Application-specific Testing Models (ATMs) 615. Each ATM is a high-level model for a specific component of the application being tested, such as a File Open dialog. ATMs describe the actual component which they represent in terms of Generic Element Models (stored in GEMs Library 610), which will now be introduced. A GEM encapsulates the behavior of irreducible user interface elements such as push buttons, checkboxes, listboxes, and the like. GEMs are instantiated when an ATM corresponding to an active state of the application under test is activated by the Model Manager 645. GEMs load their expected results from the Resource Database 620 and are capable of binding themselves dynamically to the actual object on the screen which they represent. Each GEM therefore can perform a "self test" on itself by simply comparing its expected result (as stored in the Resource Database 620) to its actual behavior (as displayed on the screen at runtime). In this manner, an entire application can perform a self test by simply asking all its components to test themselves in turn. Driven by a test script 660, the Testing Model 655 of the application under test 665 employs both Windows APIs 670 and/or the Test Port 650 to control the application being tested into various states; in each state, the results generated may be observed for error. The Model Manager 645 monitors the state of the application under test using the Message Engine and Application Translation Units (ATUs) 640 (components which are the same as those described above for the first embodiment). ATUs translate low-level messages into high-level messages, dispatching those events that the Model Manager 645 has registered an interest in. Driven by the changes in the state of the application under test 665 and/or events occurring in the system, the Model Manager 645 instructs the Script Engine 635 to load and execute the appropriate ATM 615 which corresponds to the active state of the application under test. The test script 660 employs the Test Library 630 and the Testing Model of the Application 655 to carry out the test execution task. The construction and operation of the system components will now be described in detail. C. Application-specific Testing Models Application-specific Testing Models (ATMs) provide high-level representation for specific components of the application under test. An ATM for a File Open dialog, for instance, describes the actual dialog in terms of the fundamental, pre-constructed building blocks which it comprises. In this manner, an entire application can be described using a library of ATMs. In a preferred embodiment, at least five categories of ATMs are automatically provided by the system: Menus, ToolBars, Dialogs, Client Area Windows, and a Status Line. Each of these categories corresponds to a base class pre-supplied with the system. Each base class encapsulates functionality associated with its category. A menu base class, for instance, includes data members for describing menus and methods operative on those data members. The base classes simplify the Model Generator's task of creating ATMs. Moreover, the base classes allow the QA engineer or software developer to construct custom ATMs by simply deriving specific instances of the pre-supplied base classes. Consider, for instance, an application having a File Open dialog. An ATM representing such a dialog may be created by deriving a FileOpenDlg class from the pre-existing base class for dialogs. The pre-supplied base class for dialogs ("BaseDig" class) already encapsulates the behavior and properties of a generic dialog box. Thus, the task of modeling a File Open dialog is greatly simplified. In a like manner, other aspects of the application may be modeled using the base classes which are provided for the other categories. For instance, the application's menu and client area may be modeled using pre-supplied BaseMenu and BaseClientArea classes, respectively. Before describing ATMs in further detail, it is helpful to review runtime construction of screen objects, such as dialog boxes in Microsoft Windows. Consider a typical dialog, such as confirmation message box dialog 700 shown in FIG. 7A. The dialog 700 comprises a screen window 705 which includes a plurality of elements, including caption bar 710, static text 715, and buttons 720, 725, 730. Each of these "children" may, in turn, include still further elements. Button 720, for instance, includes a caption text or label: "Yes". FIG. 7B shows the resource or programming statements attendant to the creation of each element of dialog 700 (e.g., in Microsoft Windows). The first line of resource file 750, for instance, defines a DIALOG screen element, named DIALOG.sub.-- 1 (identifier), and includes screen coordinates (111, 98, 151, 59) specifying a starting location and size (ending location). The second line specifies various window attributes or "window styles" that the dialog 700 is to assume. Dialog 700, for instance, is to be a "popup" window (Microsoft window style=WS.sub.-- POPUP) with a caption bar (Microsoft window style=WS.sub.-- CAPTION) and a system menu (Microsoft window style=WS.sub.-- SYSMENU). A caption statement (CAPTION "Confirmation") specifies that the text "Confirmation" is to be displayed in the caption bar 710. Also shown in the resource script are the children screen elements of the dialog 700. Specifically, the static text 715 is defined by the LTEXT statement, while buttons 720, 725, 730 are defined by PUSHBUTTON statements. The definitional statement for each of these includes information similar to that of the parent dialog window--namely, a caption or label, an identifier (resource identifier), a starting location and size, and window styles. Button 710, for instance, is defined by a resource statement having a caption of "&Yes" (& tells Windows to underline the immediately following character), an identifier of 101, screen coordinates of 24, 35, 24, 14, and window style of WS.sub.-- CHILD (child window), WS.sub.-- VISIBLE (visible), and WS.sub.-- TABSTOP (tab stop for setting input focus to this element with tab key). The foregoing relationship between resource (.RC) script files and screen objects is well known in the field of Windows programming. For those readers unfamiliar with these concepts (i.e., non-Windows programmers), it is strongly suggested that the following references be consulted: Petzold, C., Programming Windows (particularly, Section III: Using Resources); Borland C++: Resource Workshop; and Microsoft Windows Software Development Kit/Microsoft Visual C++. FIG. 7C illustrates that resource statements in a resource file are at runtime actually translated into Windows API calls. In particular, a given resource is created at runtime by calling Windows CreateWindow function, with the aforementioned attributes passed to Windows as parameters. CreateWindow is a standard Windows API call (and is therefore fully documented in the Petzold and Microsoft materials, as well as numerous other references). The "Yes" button 720, for instance, is created by the following CreateWindow call:
______________________________________
CreateWindow (
button, // window class name
"&Yes", // window caption
WS.sub.-- CHILD
// window style
WS.sub.-- VISIBLE
WS.sub.-- TABSTOP,
247cxchar,4, // initial x position
351cyChar*8, // initial y position
241cxChar*4, // initial x size
141cyChar*8, // initial y size
hDlg, // parent window handle
NULL, // window menu handle
hInstance, // Program instance
handle
NULL // creation parameters
);
______________________________________
At runtime, therefore, the Message Engine may trap each CreateWindow API call for determining each screen object which is about to be created. Using known techniques, the Model Generator 625 may decompose the resources of an application to automatically construct a model specific for the application--the Application-specific Testing Model or ATM. Known techniques include reading Windows resource information. Since resources may change dynamically at runtime (e.g., menu item becoming "grayed out"), it is more preferable to dynamically poll resources at runtime. Thus in a preferred embodiment, the resource information is learned dynamically--t the application's runtime--using Windows API (calls) 670, or Test Port 650 (in the instance of a proprietary control object). Exemplary calls are provided below (with reference to the Notepad example of FIGS. 8 and 9). Once the application's resources are learned, the Model Generator proceeds to construct each ATM by declaring the individual resources in a class definition derived from BaseDlg--the generic dialog class which encapsulates all the properties and functionality found in an empty dialog box. In an exemplary embodiment, BaseDlg may be constructed from the following class definition:
__________________________________________________________________________
// BaseDlg class definition
class BaseDlg( )
//Begin Constructor
Moniker = 0 //Live link to the actual dialog on the screen
Id = 0 //Unique id used as index to fetch expected
//values for this dialog from the resource dbase
Caption = 0 //Dialog's expected caption
ParentName = 0 //Parent Window's Caption or Name
Style = 0 //This dialog's style
Dimension = 0 //Coordinates and size
Children = 0 //Reference to the components of this dialog
NumOfKids = 0 //Number of components in this dialog
ParentPtr = 0 //Reference to application which owns this dialog
//End constructor
function Init( Cl, Cap, App, ResId ) // Class, Caption, Application
// reference and Unique Id
//Establish a live link to the actual dialog on the screen
//by instantiating a WindowProxy object called "Moniker"
WindowProxy Moniker("", Cl, Cap )
//Set ParentPtr and Id
ParentPtr = App
Id = ResId
//Fetch expected values from the resource database
//using the "Id" key
Defaults.LoadData( Id, Caption, ParentName, Style, Dimension
//Bind each component to the corresponding element on the screen
i = 0
while ( i < NumOfKids )
Child = Children [ i ]
Child.Bind( ) //See BaseGem's Binds member function
i + 1
end
end
function DetachChildren( )
// Detach from the actual dialog on the screen
i = 0
while ( i < NumOfKids )
Child = Children[ i ]
Child.Detach( )
i = i + 1
end
Monker = 0
end
function SelfTest( )
//Bist or Built-in-self-test controls verification and logs
//messages to the appropriate file in a specified format
Bist.Start( "Dialog selftest", Id )
Bist.Verify( "Caption", Caption, Moniker.Caption )
Bist.Verify( "Parent", ParentName, ParentPtr.Moniker.Caption )
Bist.Verify( "Size", Dimension, Moniker.GetSize( ) )
Bist.Verify( "Style", Style, Moniker.GetStyle( ) )
Bist.Verify( "Children",NumOfKids, Moniker.GetNumOfKids( ) )
//Instruct each component of the dialog to test itself
i = 0
while ( i < NumOfKids )
Child = Children[ i ]
Child.SelfTest( ) //Each Child has a SelfTest( )
//member function
i = i + 1
end
Bist.EndSection( )
end
// Capture attributes of this dialog and its children
function Capture( )
// . . . place holder for custom code; insert as desired
end
// Move window X pixels horizontally and Y pixels vertically
function Move( X, Y )
// . . . place holder for custom code; insert as desired
end
// Size window X pixels horizontally and Y pixels vertically by
// stretching the bottom-right corner.
function Size( X, Y )
// . . . place holder for custom code; insert as desired
end
end
__________________________________________________________________________
As shown, BaseDlg includes a Moniker data member. Moniker is a live link to the actual (corresponding) object on the screen. The link is created by calling WindowProxy, a C++ class which encapsulates Windows information for the object. In an exemplary embodiment, WindowProxy includes the following C++ class constructor:
__________________________________________________________________________
WindowProxy::WindowProxy(const char *strTask,
const char *strClass,
const char *strTitle)
strWindowName = (char *)new char[MAX.sub.-- WINDOW.sub.-- NAME + 1];
strWindowClass = (char *)new char[MAX.sub.-- WINDOW.sub.-- NAME + 1];
strTaskName = (char *)new char[MAX.sub.-- WINDOW.sub.-- NAME + 1];
*strWindowClass = *strWindowName = *strTaskName = ' 0';
if( strTask )
STRCPY(strTaskName, strTask);
hTargetTask = getTaskHandle(strTask);
hWndTarget = FindWindow(strClass, strTitle);
BindToWindow(hWndTarget);
}
__________________________________________________________________________
The constuctor includes a call to a BindToWindow method, which may be constructed as follows:
__________________________________________________________________________
BOOL WindowProxy::BindToWindow(HWND hWnd)
BOOL bStatus;
if( hWnd && iWindow(hWnd) )
{
GetWindowText( hWnd, strWindowName, MAX.sub.-- WINDOW.sub.-- NAME );
GetClassName ( hWnd, strWindowClass, MAX.sub.-- WINDOW.sub.-- NAME );
iWindowId = GetWindowWord(hWnd, GWW.sub.-- ID);
lWindowStyle = GetWindowLong(hWnd, GWL.sub.-- STYLE);
hWndTarget = hWnd;
bStatus = TRUE;
}
else
{
*strWindowName = *strWindowClass = ' 0';
iWindowId = 0;
lWindowStyle = OL;
hWndTarget = (HWND)NULL;
bStatus = FALSE;
}
hCurrItem = (HWND)NULL;
return bStatus;
}
__________________________________________________________________________
As shown, WindowProxy's constructor encapsulates Windows handle, task handle, caption, and the like for the object. In the event that the object is not a Windows control (e.g., it is a spreadsheet or other implementation-specific object), the information may be nevertheless encapsulated (as shown by the above "else" clause of BindToWindow). Other exemplary method prototypes of WindowProxy are set forth hereinbelow in Appendix E. Regardless of how it is implemented, the Moniker, once created, is an encapsulation of the actual object on the screen. The properties of the Moniker fully describe the corresponding object on the screen, thus giving a live link to the actual screen object. The Resource Database, on the other hand, stores expected data for the object. The two may be compared against each other (e.g., during SelfTest) for detecting runtime errors, as well as modeling problems. In addition to the Moniker, the BaseDlg class includes two other data members: Children--a reference to the components of the dialog, and NumOfKids--number of components (kids) in the dialog. These are employed in the SelfTest method of BaseDlg.
______________________________________
function SelfTest( )
i = 0 // init to first kid
while ( i < NumOfKids )
Child = Children[ i ] // test this kid
Child.SelfTest( )
i = i + 1 // increment to next kid
end
end
______________________________________
As shown, this allows a dialog object to test itself by asking its children to, in turn, test themselves (by calling their self test methods--Child. SelfTest()). Moreover, SelfTest may include various levels of testing--each level denoting more extensive testing. For instance, maximum (all levels) testing may be performed by:
______________________________________
function SelfTest( MaxLevels )
i = 0
iLevel = 0
while ( iLevel < MaxLevels ) // step through all levels
while ( i < NumOfKids )
Child = Children[ i ]
Child.SelfTest( iLevel )
i = i + 1
end
iLevel = iLevel + 1 // increment to next level
end
______________________________________
In contrast, more expedient (but less thorough), high-level testing may be performed by:
______________________________________
function SelfTest( iLevel )
i = 0
while ( i < NumOfKids )
Child = Children[ i ]
Child.SelfTest( iLevel ) // just test single level
i = i + 1
end
end
______________________________________
During creation of the dialog at runtime, the Message Engine traps the CreateWindow message for the dialog of interest (which the Model Manager has registered an interest in). In this manner, the Model Manager 645 receives notification from the Message Engine 640 that the dialog is about to be created. Upon receipt of the notification, the Model Manager locates an existing model of the dialog. If a model is found, the system is instructed to load the dialog and execute it, whereupon the Init() method of the BaseDlg class is invoked. Init initializes the Moniker, thereby creating the live link to the actual dialog. Also during Init (as shown above), the children are asked to bind themselves to their corresponding screen objects. If one of the children is a checkbox, for instance, it becomes bound to the actual checkbox on the screen. The opposite of Init is Detach. Upon the Model Manager's receipt of a message that the dialog is about to be removed from the screen (upon occurrence of a DestroyWindow message for the dialog), the Detach method is invoked for freeing memory allocated for the BaseDlg instance which had been created. The Moniker for the dialog is also freed (set to 0). This process will be illustrated for the confirmation message box dialog 700. For the dialog 700, a model may be constructed from the following class definition, derived from BaseDlg:
______________________________________
class ConfirmationDlg of BaseDlg
Button Oui ( Self, 101 )
// line 2
Button No ( Self, 102 )
Button Cancel ( Self, 103 )
Static TheText ( Self, -1 )
// line 5
NumOfKids = 4
// Component array
Components = [ NumOfKids ]
Components[ 0 ] = Oui
Components[ 1 ] = No
Components[ 2 ] = Cancel
components[ 3 ] = Text
Children = Components
end
______________________________________
The first line declares a class, ConfirmationDlg, which is derived from BaseDlg, the existing class. As shown above, each ATM includes a component array containing all its elements. The member functions of the BaseDlg class manipulate these components once they have been constructed. ConfirmationDlg inherits the data members and member functions of BaseDlg and instantiates the list of elements for FIG. 7A, as shown in lines 2 to 5 above. Oui, No, and Cancel represent objects within the confirmation dialog 700 which are instances of a Generic Element Model (GEM) of type Button. In an exemplary embodiment, each GEM (described in detail below) takes two parameters to construct itself: a reference to the parent dialog (Self) and a unique identifier (either its pre-existing resource id, or one derived by the system). In the present example, the Yes button 710 on the message box 700 has been represented by an identifier called Oui in order to emphasize that there are no assumptions about textual information made in the ATM. The object Oui loads its expected label text (the text string "Yes") from the Resource Database during initialization time. Consequently, the ATM above can represent different language versions of the Confirmation dialog without any modifications. The language difference is encapsulated in the Resource Database (described below) which is generated automatically. As shown above, ConfirmationDlg itself is a class definition. To use ConfirmationDlg at runtime, an instance is created. In an exemplary operation of the preferred embodiment, an instance (variable), ActiveDig, of the ConfirmationDlg class is created at runtime. More particularly, the instance is created when the Model Manager 645 detects that the actual Confirmation dialog has been activated, as shown by the following test script instructions:
______________________________________
ConfimationDlg ActiveDlg( )
// Instantiate ActiveDlg
ActiveDlg.Init( ClassName, Caption,
Application, ResourceId )
// Initialize
______________________________________
When instantiated, ActiveDig (or more specifically, its class constructor) loads the expected attributes for its components from the Resource Database 620. ActiveDig also loads its own attributes, such as its expected caption and location/size, from the Resource Database as well. In this manner, each component of ActiveDig (as well as ActiveDlg itself) binds itself at initialization to the actual screen object which it represents. D. Resource Database The Resource Database 620 stores expected or baseline information about the objects referred to in ATMs. Recall that the ATMs consist of instances of Generic Element Models. The Resource Database is where expected attributes of these GEM instances, such as label text, size, color, and the like, are stored. Thus, the database serves as an inventory of all resource information for an application under test. By storing object attributes in a database, the need for "hard coding" textual or positional information within ATMs or within test scripts is eliminated. This is particularly advantageous because any textual or positional assumptions about a UI element often change, especially when the application is to be localized (i.e., translated to different languages) for several international versions. In this manner, the need for altering the ATMs or the test scripts every time the text label for a UI element changes is eliminated. In a preferred embodiment, the Resource Database 620 itself is implemented using object database methodology. For instance, an exemplary resource database may be constructed using a single table defined as follows:
______________________________________
Field Name
Type Description
______________________________________
UniqueId String Key field which uniquely identifies an
object
Label String The text associated with an object
Parent String The object's parent name
PreferredName
String Variable name used by the Model
Generator to instantiate this object as
DefaultState
Integer Default state of an object (checked,
grayed, and the like)
Dimension String Expected location of this object relative
to its parent origin (x1,y1,x2,y2) format
Attributes
Binary Misc. attributes to be tested
Picture Binary Screen shot of the object preferably in a
device independent format
______________________________________
The table is keyed by (indexed on) the UniqueId and Label fields. Some fields are multi-purpose, depending on what object is stored in the record. For a menu item record, for instance, the Dimension field stores the order of the item within its popup menu. Further description of the fields is provided hereinbelow, during presentation of the Notepad example. E. GEM Library 1. Overview A Generic Element Model or GEM is perhaps best described by way of example. Consider an application program such as a Desktop Publisher (DTP) application. The application may be decomposed into its major UI components: Menus, Dialogs, ToolBar, Client Windows (which include scroll bars and system menus), and a Status Line. Each component can be broken down further. Menus, for instance, can be further reduced to four subtypes: top level pop-up menus, system menus, menu items, and side cars ("fly out" menus). The entire menu tree of any application can be described as instances of these four types of menus. Each one of these four types of menus can then be modeled by four corresponding types of Generic Element Models. Using these four GEMs, ATMs can be built which collectively represent the entire menu tree of the application. Thus given a small number of GEMs, one can build ATMs which represent the user interface of very complex applications. As described above, a GEM encapsulates the behavior of irreducible user interface elements such as push buttons, checkboxes, listboxes, menu items, and the like. When a GEM is instantiated, it takes two parameters: a reference to its parent and a resource id which uniquely identifies this GEM among its siblings. During construction time, the GEM loads its expected results from the Resource Database using, in an exemplary embodiment, a key consisting of its parent's unique id concatenated with its own id. The GEM binds itself to the actual UI element on the screen which it represents, when requested to do so by its parent. At this point, the GEM can be instructed to run a self test ("SelfTest" method) on itself by simply comparing its expected attributes (loaded from the resource database) against its actual attributes (retrieved from the actual element on the screen which the GEM represents). 2. Detailed construction In a preferred embodiment, all GEMs are derived from a base class, BaseGEM, which encapsulates the basic functionality offered by a empty or "faceless" UI element, such as processing a single or double mouse clicks, or getting input focus. BaseGEM may be constructed from the following class definition:
______________________________________
//class definition
class BaseGem( Pops, ResId )
//Begin constructor
ParentPtr = Pops //Reference to parent
ParentWHndl = 0 //Parent Window Handle
Id = ResId //Unique id for this GEM among its siblings
Moniker = 0 //Live link to the actual UI element on screen
ObjectType = 0 //Type of this GEM
//These are expected values fetched from the resource data
base
Label = 0
ParentName = 0
DefState = 0
Location = 0
ShortCut = 0
//End constructor
function Bind( )
//Load expected values from the resource database
Defaults.LoadData( Pops.Id + "," + Id, Label, ParentName,
DefState, Location )
Shortcut = GetNmemonic( Label )
//Bind to the actual UI element on screen
Moniker = ParentPtr.Moniker.FindItem( Id )
ParentWHndl = ParentPtr.Moniker.WinHandle
end
function Detach( )
Moniker = 0
end
function Select( )
if (Shortcut = = 0)
return
end
KeyIn( ParentWHndl, "{ALT}" + ShortCut )
//KeyIn( ) resides in Test RTL 630
end
function Click( )
MouseClick( Moniker.WinHandle, LeftButton )
//MouseClick( ) resides in Test RTL 630
end
function DblClick( )
MouseDblClick( Moniker.WinHandle, LeftButton )
//MouseDblClick( ) resides in Test RTL 630
end
function RightClick( )
MouseClick( Moniker.WinHandle, RightButton )
//MouseClick( ) resides in Test RTL 630
end
function RightDblClick( )
MouseDblClick( Moniker.WinHandle, RightButton )
//MouseDblClick( ) resides in Test RTL 630
end
KeyInto( Chars )
KeyIn( Moniker.WinHandle, Chars )
end
function IsA( )
return ObjectType
end
function Capture( )
//Virtual function
//Override w/ capture routine specific for object
end
function HasFocus( )
DlgHndl = FindTheWindow( ParentPtr.Caption )
return Moniker.Focus
end
function IsEnabled( )
return IsWindowEnabled( Moniker.WinHandle )
end
function GEMTest( )
Bist.Start ( XsA( ), Id )
Bist.Verify( "Parent Name", ParentName,
ParentPtr.Moniker.Caption )
Bist.Verify( "Location", Location, Moniker.GotLocation( ) )
end
end //End class definition of BaseGem
______________________________________
As shown above, BaseGem includes data members of a parent pointer, a parent window handle, a unique identifier, and a "Moniker". The parent pointer or ParentPtr is passed in (to the constructor) at initialization; for a dialog, the dialog passes in "self" (i.e., passes in a pointer to itself). The parent window handle (ParentWHndl) is set to the window handle of the parent pointer's moniker during operation of the bind function. The unique identifier or Id is a way to identify a particular control among its siblings. This may simply be the resource identifier (provided for Windows), or it may be an identifier assigned by the system. Again, this is passed in by the dialog at creation. The Moniker is a handle or live link to the actual UI element on screen. The object type or ObjectType indicates a windows type, such as checkbox, pushbutton, and the like. The class also includes data members for storing expected values for the object (fetched from the Resource Database): Label, ParentName, DefState, Location, and ShortCut. The values for these are fetched from the Resource Database upon instantiation by calling the Defaults.LoadData method; ShortCut is determined from analyzing the Label (e.g., menu item label of &File indicates a ShortCut keystroke of Alt-F). During self test, the actual values may be compared to these expected values. Also shown, BaseGem includes class methods or functions. The Bind() function binds the object to its actual screen object. In particular, the method employs a Windows Id to return a unique handle or "Moniker" for the screen object. A corresponding Detach() function is provided for detaching from the screen object; the Moniker is reset (to zero). Other class methods are provided for simulating user events. For instance, the Click(), DblClick(), RightClick(), and RightDblClick() methods perform their respective mouse clicks, by sending the corresponding Windows event message to the target object. Select(), on the other hand, keys in the shortcut (e.g., keyboard accelerator) for the object. Other types of GEMs may be derived from BaseGEM. A CheckBox class, for example, may be derived. CheckBox inherits all the capabilities of BaseGEM and adds any behavior unique to it. It may be constructed as follows:
______________________________________
class CheckBox( Pops, ResId ) of BaseGEM( Pops, ResId )
ObjectType = "CheckBox"
// Get the state of the CheckBox
function GetState( )
if ( IsDlgButtonChecked( ParentWHndl, Id ) == 1 )
// Note IsDlgButtonChecked( ) is std Win API (call) 670
return "1"
else
return "0"
end
end
function SelfTest( )
GEMTest( )
Bist.Verify( "Label", Label, Moniker.Caption )
PrevState = GetState( )
Click( )
Bist.NonEqual( "Changing State", PrevState, GetState( ) )
Click( )
Bist.Done( )
end
function Capture( )
dbDictionary Attributes( )
// dbDictionary is test object provided by Test RTL
Attributes.AddEntry( "Label", Moniker.Caption )
Attributes.AddEntry( "Location",
Moniker.Left + "," + Moniker.Top )
Attributes.AddEntry( "State", GetState( ) )
return Attributes
end
end
______________________________________
As shown, CheckBox is a derived class and includes state behavior specific for its type, namely "Checked" (state "1") or "Unchecked" (state "0") for a CheckBox. Since the CheckBox knows about its own behavior and attributes, it includes methods for handling these specifics. For instance, CheckBox includes a GetState() method which returns whether the checkbox instance is "checked" or "unchecked." CheckBox adds its own SelfTest() function, which runs the GemTest() function (standard self test) as well as running additional tests specific for a checkbox (i.e., clicking itself and comparing its resulting state with its previous state). Capture() function for capturing the attributes of itself. Specifically for a checkbox, this entails capturing information about its Label (i.e., text displayed with the checkbox) and its state (whether it is "checked" or "unchecked"). These may be stored as entries in a database. 3. GEM Functionality GEMs will now be summarized in terms of their specific functionality. In a preferred embodiment, GEMs include the following fundamental functionality: Self load, Self test, Binding to screen objects, Attribute capture, Next state, User interaction simulation, and Resource tracking. Each will be described in turn. (a) Self load GEMs contain no information about their characteristics (e.g., text labels) in their declaration or definition. Instead, a GEM loads its characteristics from the Resource Database during construction. Each GEM (which is a derived instance of a BaseGem class, presented above) is constructed with a unique resource identifier (ResId). Thus when a GEM is instantiated, the object containing the GEM passes a unique key to the constructor of the GEM. The constructor uses this key to load in the GEM's expected properties from the Resource Database: Defaults.LoadData (Pops.Id+", "+Id, Label, ParentName, DefState, Location) Each GEM is constructed only on an as-needed basis. (b) Self test All GEMs have a SelfTest member function. SelfTest enables the GEM to test itself by comparing its characteristics to the actual element on the screen that it represents. The basic test includes checking the position, label, parent name, and default state of the element. A CheckBox class, for instance, may include the following SelfTest class method:
______________________________________
function SelfTest( )
GEMTest( )
Bist.Verify( "Label", Label, Moniker.Caption )
PrevState = GetState( )
Click( )
Bist.MonEqual( "Changing State", PrevState, GetState( ) )
Click( )
Bist.Done( )
end
______________________________________
As shown, the Checkbox. SelfTest method begins by calling GEMTest(), which verifies the position and parent for the object. SelfTest then verifies the text label for the checkbox screen object. Finally, the SelfTest method proceeds to test the checkbox with mouse click events. Here, the current state is saved (to PrevState). Then, a click event is executed (by calling the inherited Click() method for the object). At this point, the SelfTest compares the state of the object with the previously-saved state. As a checkbox functions as a toggle, its now-current state (i.e., after receiving the click event) should differ from the previously-saved state. This is confirmed by: Bist.NonEqual ("Changing State", PreyStats, GetState() ) If the states are instead equal (i.e., the checkbox on the screen did not toggle), the test will fail. Upon completion of the test, the checkbox object is restored to its previous state. In this fashion, all checkboxes of a program may test themselves to assure they are not defective. In addition to testing display attributes and receipt of input events, additional tests can be added to GEMs at any time in the QA cycle. When a test is added to the self test section, all instances of that GEM will automatically include the new test as well. (c) Binding to screen objects GEMs can attach themselves to the actual object on the screen that they represent. This ability to bind to screen elements enables any given visual element on the screen to be examined at runtime, greatly increasing the level of observability offered by the test automation system. (d) Attribute capture A GEM can construct a container object which captures the attributes of the actual screen element the GEM represents. This container object is an instance of the standard C++ dictionary class with an extra feature of persistence. The attributes of the screen element can be stored in a database (e.g., in the Resource Database) for later reference and comparison. (e) Next state GEMs may contain knowledge of what the next state of the program will be after the control the GEM represents is selected. For example, when a menu item is selected, a dialog box might appear. In this case, the menu item and the dialog box are related. As another example, clicking an OK button should cause the button to tell its parent (i.e., the dialog box containing the button) to destroy itself. Next state relationships among GEMs are established using links within ATMs. Maintaining links among related GEMs in practice usually proves problematic and inefficient however. Therefore the burden of "knowing" the next state can be disabled within GEMs and transferred to a higher level which in this system is the Model Manager. (f) User interaction simulation Each GEM can simulate any possible operation that a user would perform an any given UI element, such as a mouse click, to getting focus, pressing a key, and the like. As shown above (in the BaseGEM class definition), for instance, a test script may simulate a click event at an object by invoking that object's Click() method. (g) Resource tracking GEMs can monitor resource usage. If there is a sudden drop or leakage in overall system resources, for example, GEMs may log warning messages. F. Model Manager The Model Manager 645 functions to monitor the active state of the application under test and load the appropriate model of the application under test at any given state. The Model Manager provides a key benefit. Instead of keeping the entire model of an application under test in memory, only a model corresponding to the active or live portion of the application under test need be loaded and executed. This approach yields improved performance and decreased intrusion over the application being tested. The approach also frees the script developer from knowing the instance name of every single component within the application. Consider, for example, the commands for bringing up a file open dialog and clicking the OK button: FileMenu.Open.Select () FileOpenDialog.OK.Click () In the above code, the script writer must know that after executing the first line, the resulting File Open dialog which was activated is referred to as FileOpenDialog. Keeping track of the names of every component within a large application is tedious and error-prone. Moreover, the instance FileOpenDialog must be instantiated even though the script may never use it. In contrast, the following code performs the same as the one above, yet incurs much lower memory overhead and is easier for the script developer to maintain: FileMenu.Open.Select () ActiveDlg.OK.Click () As shown, once the first line is executed, a global variable (ActiveDig) is automatically constructed by the Model Manager 645 by loading and executing the ATM which models this dialog. The Model Manager can also instruct the Model Generator to create a model on-the-fly if one does not already exist. G. Model Generator The Model Generator 625 generates ATMs 615 and the Resource Database 620. The Generator may employ resource information embedded in a program, live data on the screen, or a combination of both. In the event that all the necessary information is not available in a program resource, a control script may be constructed to take the application under test into necessary states so that the model generator may generate the appropriate ATM and Resource Database entries. The Model Generator 625 also provides information about what changes were made between the current build and the previous run. In addition, the Model Generator shields old scripts from breaking (i.e., incurring errors) by using a field in the database called preferred name. Application Modeling: Windows Notepad application The method of the present invention for modeling a application will now be illustrated using a well-known application--Microsoft Windows Notepad. In this manner, the reader may focus on the teachings of the present invention without distraction from a complex application. The creation of an Application Test Model (ATM) for the Windows Notepad application is illustrated in FIGS. 8A-D. In FIG. 8A, the interface for the Notepad application is shown. The interface includes a main or application window 800 having a caption bar 801, a menu bar 820, and a client area 840. Menu bar 820, in turn, invokes a plurality of submenus, such as submenu 830. A. Menu As shown in FIG. 8B, the menu bar 820 and its submenus may be decomposed into a resource file 870. Menu resource file 870 includes "popup" sections defining the submenus which are attached to the menu bar 820. Submenu 830, for instance, is defined by popup section 875. Thus as shown, the menu bar 820 and its submenus may be decomposed for tracking its individual elements. The decomposition into Application-specific Test Model (ATM) may be accomplished either by reading resource data from the EXE file (static method), or by executing the EXE file and inquiring about menu information (from the running application during runtime). A Model Generator implemented using the latter method provides an added advantage since it can extract dynamic menu attributes (grayed, checked, and the like) from the application. These attributes are generally not available in the resource file. The method for generating menu ATMs using the dynamic approach is illustrated in FIG. 8C. At step 881, the application under test is launched. Next, at step 882, the application's window handle is retrieved. From the window handle the menu handle can be determined; this is indicated by step 883. For each top level menu, the following steps are performed at step 884. At step 884A, an unique id is constructed for this top level menu. At step 884B, the resource database is searched for the record under this id. At step 884C, the top level menu string from the menu handle is retrieved. A preferred name (if any) is also retrieved at step 884D; otherwise, a name based on the menu string is synthesized (e.g., "FileMenuATM" for menu item of "File"). Using the preferred (or synthesized) name, a class for this popup menu is declared at step 884E: class FileMenuATM(Parent, Id) of PopUpMenu(Parent, Id) When the class is instantiated, a reference to parent is passed as the Parent parameter, and this menu's unique id is passed as the Id parameter. At step 884F, the information for this top level menu in the Resource Database is updated, and at step 884G any discrepancies (changes between last run and this run) are logged in the log file. For each submenu, the following steps are performed at step 885. At step 885A, the <menu string> and <menu id> of the menuitem are retrieved. A unique id for this menuitem is determined at step 885B, based on its parent's id and this menu item's id. At step 885C, the system looks up the record corresponding to this unique id in the Resource Database. At step 885D, a preferred name (if any) is retrieved from the PreferredName field (or is synthesized based on the menu string). This is used, in turn, at step 885E to instantiate a variable of type MenuItem as a data member of this ATM class: MenuItem New(Self, 9) The first parameter is a reference to this instance of the ATM class; the second parameter is the <menu id> of this menu item. At step 885F, the record in the Resource Database is updated An exemplary record for the File, New menu item may be constructed as follows: Unique Id: Notepad,#1,9 Label: &New Parent: &File PreferredName: New State: integer denoting the state which is enabled, not grayed, not checked Dimension: 1 (denotes the position of this menuitem within the popup menu). For menu items, the Dimension field of the Resource Database is used to describe the position of the menu item (Attributes and Picture fields are not used for the Notepad application menus.) Finally at step 885G, any differences in the log file are recorded. The menu class, FileMenuATM 915, is constructed for the menu bar 820 of the Notepad application as shown in FIG. 8D. FileMenuATM is a derived class of PopupMenu--a class which encapsulates the behavior of a popup; its relation to resource information 910 is shown. In a similar manner, a menu class for the Edit menu, EditMenuATM, is constructed from the resource information of the Edit menu of the Notepad application, as shown in FIG. BE. B. Dialog When the user invokes a File, Open command (e g., by selecting "Open" from the submenu 830), the Notepad application 800 displays the Open dialog 900, shown in FIG. 9A. The dialog 900 includes a plurality of screen objects. As shown in further detail in FIG. 9B, the dialog 900 comprises a window 911 with caption bar and includes the following children: static text fields 912, 915, 917, 922, 923, edit text field 913, list boxes 914, 919, combo box 916, 918, and buttons 920, 921. Such a dialog is defined by the following resource information:
__________________________________________________________________________
1536 DIALOG LOADONCALL MOVEABLE DISCARDABLE 36, 24, 264, 134
STYLE DS.sub.-- MODALFRAME WS.sub.-- POPUP WS.sub.-- CAPTION
WS.sub.-- SYSMENU
CAPTION "Open"
FONT 8, "Helv"
LTEXT "File &Name:", 1090, 6, 6, 76, 9
EDITTEXT 1152, 6, 16, 90, 12, ES.sub.-- AUTOHSCROLL ES.sub.-- OEMCONVERT
WS.sub.-- BORDER WS.sub.-- TABSTOP
LISTBOX 1120, 6, 12, 90, 68, LBS.sub.-- STANDARD NOT LBS.sub.-- NOTIFY
1
LBS.sub.-- OWNERDRAWFIXED LBS.sub.-- HASSTRINGS LBS.sub.-- DISABLEROSC
ROLL
NOT WS.sub.-- BORDER WS.sub.-- TABSTOP
LTEXT "&Directories:", -1, 110, 6, 92, 9
LTEXT 10, 1088, 110, 18, 92, 9, SS.sub.-- NOPREFIX WS.sub.-- GROUP
LISTBOX 1121, 110, 32, 92, 68, LBS.sub.-- STANDARD NOT LBS.sub.--
NOTIFY
LBS.sub.-- OWNERDRAWFIXED LBS.sub.-- HASSTRINGS LBS.sub.-- DISABLECOS
CROLL
NOT WS.sub.-- BORDER WS.sub.-- TABSTOP
LTEXT "List Files of &Type:", 1089, 6, 104, 90, 9
COMBOBOX 1136, 6, 114, 90, 36, CBS.sub.-- DROPDOWNLIST CBS.sub.--
AUTOHSCROLL
WS.sub.-- BORDER WS.sub.-- VSCROLL WS.sub.-- TABSTOP
LTEXT "Dri&ves:", 1091, 110, 104, 92, 9
COMBOBOX 1137, 110, 114, 92, 68, CBS.sub.-- DROPDOWNLIST
CBS.sub.-- OWNERDRAWFIXED CBS.sub.-- AUTOHSCROLL CBS.sub.-- SORT
CBS.sub.-- HASSTRINGS WS.sub.-- BORDER WS.sub.-- VSCROLL WS.sub.--
TABSTOP
DEFPUSHBUTTON "OK", 1, 208, 6, 50, 14, BS.sub.-- DEFPUSHBUTTON
WS.sub.-- GROUP WS.sub.-- TABSTOP
PUSHBUTTON "Cancel", 2, 208, 24, 50, 14, WS.sub.-- GROUP WS.sub.--
TABSTOP
PUSHBUTTON "&Help", 1038, 208, 46, 50, 14, WS.sub.-- GROUP WS.sub.--
TABSTOP
CHECKBOX "&Read Only", 1040, 208, 68, 50 , 12, BS.sub.-- AUTOCHECKBOX
WS.sub.-- GROUP WS.sub.-- TABSTOP
}
__________________________________________________________________________
Application-specific Testing Model (ATM) generation for the dialogs of the Notepad application can be accomplished using the dynamic method as well. In order to bring up each dialog of the Notepad application, a small control script is constructed which "walks" the menu tree, brings up each dialog, and invokes the dialog model generator to create ATMs for each dialog. Referring now to FIGS. 9C-D, Model Generation for dialogs, using the dynamic method 950, is illustrated. In step 951, the following files are opened: the Resource Database for updating, a Script file for writing, and a MapFile for reading. (MapFile provides a mapping from the type of UI elements on the screen to the corresponding GEM type). In step 952, the dialog to be modeled is brought up (on the screen) and a live link is established with the dialog, using either a window handle or the Test Port. At step 953, the live link is used to determine actual caption, class, and unique id for this dialog. If, at step 954, the unique id (UniqueId) is known, the method proceeds to step 957. Otherwise, the method searches the Resource Database, based on caption and/or class of this dialog, in an attempt to locate the record corresponding to this dialog. If one is found (step 956), the method proceeds to step 957. If not found, however, a unique id is assigned to this dialog at step 958, with a new record being created for this dialog in the Resource Database (indexed by the unique id). At step 957, the values in the record (corresponding to this dialog in the Resource Database) are compared to the actual dialog attributes (as displayed on screen). Any differences are logged; the record in the database is updated. At step 959, a class definition statement for this dialog is generated in the script file. Next, the children controls are processed. At step 961, the (next) child control within this dialog is located. If child is NULL (no more children) at step 962, the method proceeds to step 967; otherwise, the method goes to step 963. At step 963, a live link is established with the child control (window). Based on the info available from the link to this child window (e.g., type, id, and label), the following values are assigned at step 964: A) <GEM Type> gets its value from the MapFile based on the type of this child window. If no mapping available <GEM Type> set to BaseControl. B) <Id> is set to the unique id, usually resource id, for this control. C) <PreferredName> is set to the value under field PreferredName of the record in resource database corresponding to this control. If none found, synthesize a name based on the label or type of this control. At step | ||||||