Server system for coordinating utilization of an integrated test environment for component testing6574578Abstract Method and system aspects for utilizing a server to coordinate component testing in an integrated test environment network are described. In a method aspect, the method includes accepting a logon attempt by a client system and requesting a desired partition for the client system from a partition server. The method further includes reporting results from the client system on a database server, supplying report data to a Web server, and logging the client system off the server. Claims What is claimed is: Description TECHNICAL FIELD
private String
EOL = System.getProperty("line.separator"),
ENCODING = System.getProperty("file.encoding");
//***************************************
// PROTECTED ATTRIBUTE INITIALIZATION
//***************************************
//
if( testObject.lineSeparator.equals(EOL))
outcome = TestCase.PASS;
else
outcome = TestCase.FAIL;
execute(new TestCase( component, certUnit, "init-lineSeparator",
outcome, testObject.lineSeparator == null ? "null":
testObject.lineSeparator) );
if(testObject.fileInputStream == null)
outcome = TestCase.PASS;
else
outcome = TestCase.FAIL;
execute(new TestCase( component, certUnit, "init-fileInputStream",
outcome, testObject.fileInputStream == null ? "null":
testObject.fileInputStream.toString() ));
//****************
// PUBLIC METHOD
//****************
// Returned default encoding should match default JAVA encoding
java.lang.String r2;
r2 = testObject.getEncoding();
if( r2.equals(ENCODING))
outcome = TestCase.PASS;
else
outcome = TestCase.FAIL;
execute( new TestCase( component, certUnit, "getEncoding#1", outcome,
r2 == null ? "null": r2.toString() ) );
//
// To "8859_1" (ISO Latin-1) without failing
//
java.lang.String p3 = "8859_1";
testObject.setEncoding( p3 );
outcome = TestCase.PASS;
execute( new TestCase( component, certUnit, "setEncoding#1",
outcome) );
//
// Verify that getEncoding() returns "8859_1" ISO Latin-1 after set
//
r2 = testObject.getEncoding();
if( r2.equals( p3 ))
outcome = TestCase.PASS;
else
outcome = TestCase.FAIL;
execute( new TestCase( component, certUnit, "getEncoding#2", outcome,
r2 = null ? "null" : r2.toString() ));
The testcases above are examples of background tests. Interactive tests take a prompt string in place of the last two arguments passed to the execute method. This prompt is then presented by the testing framework, at runtime, and the true/false response is taken as the outcome of the test. For example: chartObject.setChartType(Chart.PIECHART); execute(new TestCase(component, certUnit, "Chart#test1", "Do you see an a PIE CHART now?")); A checklist is a special case of an interactive test suite. The checklist items presented by the framework are not based on introspection of class files, but rather, use entries placed into the integrated test environment database 214 to determine the content of the checklist. The checklist items are presented to the test engineer at test execution time, just like the previous interactive test example, and the test engineer chooses between responses like: PASS, FAIL, BYPASS, KNOWN PROBLEM, as each checklist item is executed manually. Checklists are intended to support manual testing efforts, like downloading a product off the Web, for example: 1. Navigate to website 2. Go to download page 3. Download software component to client machine 4. Extract the contents of the downloaded file 5. Install the component software from the extracted files 6. Review license and copyright information The ITE test software preferably runs in its own partition on the ITE client 206. The C:.backslash.partition of the ITE client 206 is treated as transient data that gets overlaid with images from the partition server 204, as described previously. The test suites are run locally on the ITE client machines 206, and the outcomes of the test cases are stored locally in memory on the ITE client 206, so that latency in transacting with the ITE server 202 does not impact the test results. Further, the present invention utilizes a central repository of production test suites in the ITE server 202. The test suites are loaded from the repository, at run time, to perform regression testing (i.e., repeating an established suite of tests to make sure there are no unintended changes in the behavior of the software induced by the addition of new features or fixes). When test suite code is being developed or revised, it is copied onto an ITE client machine 206 and is then modified and executed using the client version, not the server version of the code. In other words, the repository of test suites is managed like a code repository with the only difference being that the runtime hook determines if the test suite should be fetched locally or from the remote repository. A number of commercially available tools such as IBM's CMVC and Rational Corporation's ClearCase are suitable for use to manage the test suite repository. In this way, a test engineer always knows where to go to get the production version of a test suite, so that any team member can perform regression testing without having to worry about which machine holds the current version of the test. Further, repository software usually implements the idea of checking components in and out, since it is easily determined if a test suite is being modified and who is making modifications. In accordance with a further embodiment of the present invention, an automatic test case code generator is utilized to reduce the amount of manual coding required to develop each individual test case within a given test suite. When testing components, e.g., JavaBeans, each executable class within the component must be tested. About 1000-1200 lines of code are typically needed to do low level API (Application Programnming Interface) testing of a class file. In order to reduce the tedious and time-consuming process of hand-coding tests for each method within a component, the present invention utilizes a test case code generator software program within the ITE clients 206, as described with reference to FIG. 5, to generate a basic core API test skeleton program that systematically exposes all protected methods, constructors and attributes, and makes calls to all methods contained in a subject class. The utilization includes reading a class file for testing (step 400) and testing by the test case code generator (step 402). In a preferred embodiment, the test case code generator generates the skeleton test cases by way of the java.introspection facility from Java's java.lang.reflect package to test the API calls within a class and provides an exhaustive core API unit test case of a given class. The introspection facility of JAVA is provided at run-time to query a class and identify what methods are in the class. The introspection facility also determines what arguments a method takes, the data type of each argument, what data types are returned and which exceptions are thrown, if any. The test case code generator portion of the ITE client 206 suitably performs automatic test case generation based on return type when possible, as well as trapping exceptions. For example, when a Boolean setter method is detected, the test case code generator creates code to automatically check that the default Boolean value is correct, changes it to the opposite Boolean value, insures that the change occurs properly, and returns the value to the default value. Thus, a skeleton test suite program is generated (step 404) that can then be individualized for specific test cases of the component and integrates into the testing execution framework to automatically execute each test case and post test results to the common repository. The skeleton program, as its name implies, provides a basic test structure and improves the testing environment by reducing the hand-coding of individual test cases for a method to a minimal number of lines, which saves time and improves consistency, while ensuring the integrity of the lowest level API calls of a method. In accordance with the present invention, the testing skeleton supports definition and execution of three major types of tests: background automated tests, interactive automated tests, and automatic checklist processing to support manual testing activities, as previously described in the code example presented above in conjunction with FIG. 3. Thus, background tests include those where the expected outcome of the test is known, so that a simple comparison between the expected outcome and the actual outcome can be performed in the background to test for proper functioning. Interactive tests refer to those tests where feedback is required, e.g., `Do you see a bar graph now?`, with the identification of proper functioning determined by the interactive feedback provided. Checklists provide a straightforward mechanism for ensuring that quality checks are performed in a complete and consistent manner. FIG. 6 illustrates a diagram of the class relationship of the test execution framework that run on the ITE client 206 in accordance with the present invention. The framework on the ITE client 206 instantiates the software objects being tested, drives the execution of test suites, captures responses for interactive test cases, catches and deals with exceptions and generates the test results transaction, which it then passes to the ITE application server 202 to be posted to the database server 214. The test engineer developing a test suite gets all of these services "for free" by extending the Driver Panel and Test Suite classes. The abstract class ADriverPanel 600 implements methods that facilitate and control the execution of test suites on the ITE client 206. AdriverPanel 600 constructs a TestFrame 601 frame window that contains a TestingPanel object 602 which, in turn, contains any graphical elements of the component software being tested. AdriverPanel 600 also creates a TestLog 603 object to optionally record test results to a local text file on the ITE client 206. ADriverPanel is further responsible for posting the test results to the database server 214 via the postResult( ) method. AtestSuite 604 is executed by a "panel class" that extends ADriverPanel 600 which calls the addSuite( ) method to add an instance of each TestSuite that is to be run within the scope of the driver and then calls the engageDriver( ) method to actually execute the TestCases contained within the TestSuite. A sample of the preferred code for the text file reader example component follows:
// FileTextReaderPnl.java generated by the SkelGen utility on Wed Dec 23
13:38:24 PST 1998
// (.COPYRGT.) Copyright 1998, IBM Corporation. Author Tim Logan.
//
// FileTextReaderPnl.java
package Tests.FileTextReader;
import j ava.awt.*;
import java.awt.event.*;
import java.util.*;
import com.sun.java.swing.*;
import util.Streams.*;
import QE.Client.Console;
import QE.TestHarness.*;
import Tests.FileTextReader.FileTextReaderCore;
/**
* This class drives the execution of the FileTextReaderCore test suite
*/
//***********************************************************
public class FileTextReaderPnl extends ADriverPanel
//***********************************************************
{
//
--------------------------------------------------------------------------
--------
public FileTextReaderPnl
( Console console, String driverTitle )
//
--------------------------------------------------------------------------
--------
{
super( console, driverTitle);
};
//
--------------------------------------------------------------------------
--------
public void runTestSuites()
//
--------------------------------------------------------------------------
--------
{
//
// Instantiate the TestSuite thread objects that will be run serially.
// The first one (the core API TestSuite) is generated for you.
//
addSuite( new FileTextReaderCore( this ) );
//
// Your custom TestSuites . . .
//
engageDriver();
};
};
A test suite extends the abstract class ATestSuite 604. TestSuites are
owned by AdriverPanel 600 and contain TestCases 605 in a parent
relationship, as shown in the diagram of FIG. 6.
//***********************************************************
public class FileTextReaderCore
extends ATestSuite
//***********************************************************
As shown by the diagram of FIG. 6, each AtestSuite 604 object runs on it's own thread by inheriting from java.lang.Thread. The testing logic is contained in an implementation of the run( ) method which executes calls to the parent A TestSuite.execute( ) method, passing new instances of TestCase 605 objects as follows:
//
--------------------------------------------------------------------------
-------------
public void run()
//
--------------------------------------------------------------------------
-------------
{
int outcome = TestCase.PASS;
//
//*************
// background test
//*************
//
if( *WHATEVER YOUR TEST CONDITION IS* == true)
outcome = TestCase.PASS;
else
outcome = TestCase.FAIL;
execute( new TestCase( "componentName", certificationUnitName",
"testName", outcome, resultObject == null? "null":
resultObject.toString() ) );
//
//*************
// interactive test
//*************
execute( new TestCase( "componentName", certificationUnitName",
"testName", "Interactive Prompt String Goes Here") );
//
// when all TestCases have been executed, call . . .
//
complete();
};
The VerifyDialog 606 object that is created whenever an interactive prompt string is passed in a TestCase constructor blocks the TestSuite thread using the A TestSuite$Semaphore 607 object while waiting for the user to respond to the interactive prompt. Referring again to FIG. 3, once the tests are completed in the ITE client 206, the test suite test results are posted from the ITE client 206 to the ITE server 202 (step 312), and the ITE server 202 posts the test results as a single transaction to the DB server 214 (step 314). The ITE client 206 also makes a report request to the Web server 216 (step 316). The Web server 216 then requests the report data from the ITE server 202 (step 318), which the ITE server 202 sends in response to the Web server 216 (step 320). The Web server 216 provides the report Web pages generated to the ITE client 206 (step 322). The client can then continue testing with another test suite if desired, with testing via the ITE environment completed for the client 206 when the client logs off of the ITE server 202 (step 324). The above-described interactions with the DB server 214 occur through the ITE server 202. Data is exchanged between the ITE server 202 and the DB server 214 as needed. The interactions described between the ITE client 206 and the ITE server 202 are transaction-based, as are the exchanges between the Web server 216 and the ITE server 202, as is well understood by those skilled in the art. Through the present invention, an integrated test environment successfully performs component testing with consistency and efficiency. In addition, the integrated test environment may be extended to provide utility classes to instrument code, perform lint-like code clean-up, enforce coding standards, support conditional compilation, do method signature deprecation analysis between releases, and automatically generate documentation, for example. Although the present invention has been described in accordance with the embodiments shown, one of ordinary skill in the art will readily recognize that there could be variations to the embodiments and those variations would be within the spirit and scope of the above-described system and method. For example, to include remote client machines, some adjustments to the network may be needed, such as the inclusion of firewall machines, as is well understood by those skilled in the art. Further, although the aspects of the test case code generator are described as a portion of the ITE client application, these aspects may also be achieved for use as a stand-alone application, if desired. Accordingly, many modifications may be made by one of ordinary skill in the art without departing from the spirit and scope of the appended claims.
|
Same subclass Same class Consider this |
||||||||||