Analysis of network performance6990616Abstract A method for testing of a communication network, using a plurality of traffic agents coupled to communicate via the network. The method includes transmitting a sequence of data packets via the network from a first one of the traffic agents to a second one of the traffic agents and recording arrival characteristics of the packets in the sequence, responsive to receiving the packets at the second traffic agent. The arrival characteristics of different packets in the sequence are compared so as to determine a measure of variability in transmission of the packets via the network. Claims What is claimed is: Description FIELD OF THE INVENTION
Preferably, assessing the relative irregularity of packet loss includes detecting bursts of lost packets. There is also provided, in accordance with a preferred embodiment of the present invention, a method for testing of a communication network, using a plurality of traffic agents coupled to communicate via the network, the method including:
Preferably, comparing the order of arrival includes finding a measure of discrepancy between the order of arrival and the order in which the packets were transmitted. There is additionally provided, in accordance with a preferred embodiment of the present invention, a method for testing of a communication network, using a plurality of traffic agents coupled to communicate via the network, the method including:
There is further provided, in accordance with a preferred embodiment of the present invention, a method for testing of a communication network, using a plurality of traffic agents coupled to communicate via the network, the method including:
There is moreover provided, in accordance with a preferred embodiment of the present invention, a method for testing of a communication network, using a plurality of traffic agents coupled to communicate via the network, the method including:
There is furthermore provided, in accordance with a preferred embodiment of the present invention, a method for testing of a computer application accessed via a communication network, using a plurality of traffic agents coupled to communicate via the network, the method including:
Preferably, running the instance of the application on the first computer includes running an application server, and exchanging the application data packets includes transmitting application client messages from the second computer to the first computer. Alternatively or additionally, running the instance of the application includes running a distributed computing application on the first computer, and exchanging the application data packets includes running another instance of the application on the second computer. Further alternatively or additionally, comparing the exchange characteristics includes comparing a delay in the exchange of application data between the first and second computers relative to the exchange of test data. There is also provided, in accordance with a preferred embodiment of the present invention, a method for determining a round-trip transmission delay in a communication network, including:
Preferably, transmitting the second data packet includes transmitting the second packet without waiting to receive the first data packet at the second endpoint. Additionally or alternatively, comparing the first and second transmit times and the first and second receive times includes using transmit and receive times recorded in accordance with different clocks maintained at the first and second endpoints. Most preferably, comparing the first and second transmit times and the first and second receive times includes canceling out a relative offset between the different clocks, substantially without an a priori knowledge of the offset. There is additionally provided, in accordance with a preferred embodiment of the present invention, a method for testing of a communication network, using a plurality of traffic agents coupled to communicate via the network and having respective agent clocks that are generally independent of one another, the method including:
Preferably, determining the start time includes choosing a time to start the test that is delayed relative to expected times of sending the start messages. Further preferably, synchronizing the initiation of the test includes starting the test at each of the traffic agents at the start time, as indicated by the respective agent clock, corrected responsive to the time of sending contained in the respective start message. There is further provided, in accordance with a preferred embodiment of the present invention, apparatus for testing of a communication network, including:
Preferably, the apparatus includes a testing center, coupled to the network, which is adapted to receive the indication of the packets that were lost in transmission and to assess the relative irregularity in the occurrence of packet loss. There is moreover provided, in accordance with a preferred embodiment of the present invention, apparatus for testing of a communication network, including:
There is furthermore provided, in accordance with a preferred embodiment of the present invention, apparatus for testing of a communication network, including:
There is also provided, in accordance with a preferred embodiment of the present invention, apparatus for testing of a computer application accessed via a communication network, including:
Preferably, the instance of the application running on the first computer includes an application server, and wherein the second computer acts as a client of the application. There is additionally provided, in accordance with a preferred embodiment of the present invention, apparatus for determining a round-trip transmission delay in a communication network, including:
There is further provided, in accordance with a preferred embodiment of the present invention, apparatus for testing of a communication network, including:
There is moreover provided, in accordance with a preferred embodiment of the present invention, apparatus for testing of a communication network, including:
There is furthermore provided, in accordance with a preferred embodiment of the present invention, a computer software product for testing of a communication network, the product including a computer-readable medium containing program instructions, which instructions, when executed by first and second computers coupled to communicate via the network, cause the first computer to transmit a sequence of data packets via the network to the second computer, and further cause the second computer to receive the data packets and to record arrival characteristics of the received packets, wherein the arrival characteristics of different packets in the sequence are compared so as to determine a measure of variability in transmission of the packets via the network, wherein the measure of variability is a measure relating to a quality selected from a group of qualities consisting of changes in packet transmission delay, loss of transmitted packets and an order in which transmitted packets are received. In a preferred embodiment, the program instructions, when executed by a third computer, cause the third computer to act as a testing center, which receives the arrival characteristics from the first and second computers and determines the measure of variability. There is furthermore provided, in accordance with a preferred embodiment of the present invention, a computer software product for testing of a computer application accessed via a communication network, the product including a computer-readable medium containing program instructions, which instructions, when executed by first and second computers coupled to communicate via the network, wherein the first computer is also running an instance of the application, cause the first and second computers to exchange test data packets via the network, generally independent of the application, so as to determine test packet exchange characteristics, and to exchange application data packets via the network between the second computer and the instance of the application running on the first computer, so as to determine application packet exchange characteristics, whereby the exchange characteristics of the application and test packets are compared. There is additionally provided, in accordance with a preferred embodiment of the present invention, a computer software product for determining a round-trip transmission delay in a communication network, the product including a computer-readable medium containing program instructions, which instructions, when executed by first and second computers coupled to communicate via the network, cause the first computer to transmit a first data packet through the network to the second computer at a first transmit time, and cause the second computer to receive the first data packet at a first receive time and to transmit a second data packet through the network to the first computer at a second transmit time, substantially independent of the first transmit and receive times, which second data packet is received by the first computer at a second receive time, wherein the first and second transmit times and the first and second receive times are compared so as to determine the round-trip transmission delay. There is also provided, in accordance with a preferred embodiment of the present invention, a computer software product for testing of a communication network, the product including a computer-readable medium containing program instructions, which instructions, when executed by a plurality of computers coupled to communicate via the network and having respective local clocks that are generally independent of one another, cause a first one of the computers to determine a start time at which a test of the network is to begin and to send respective start messages to the other computers, each start message containing the start time and a time of sending the start message determined with reference to the local clock maintained by the first computer, and which instructions further cause the other computers to synchronize initiation of the test, responsive to the respective start messages. The present invention will be more fully understood from the following detailed description of the preferred embodiments thereof, taken together with the drawings in which: BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a block diagram that schematically illustrates a distributed network testing system, in accordance with a preferred embodiment of the present invention; FIG. 2 is a flow chart that schematically illustrates a method for network testing, in accordance with a preferred embodiment of the present invention; FIG. 3 is a flow chart that schematically illustrates another method for network testing, in accordance with a preferred embodiment of the present invention; FIG. 4 is a block diagram that schematically illustrates message flow among elements of the system of FIG. 1, in accordance with a preferred embodiment of the present invention; FIG. 5 is a block diagram showing elements of a distributed network testing system, which schematically illustrates still another method for network testing, in accordance with a preferred embodiment of the present invention; FIG. 6 is a flow chart that schematically illustrates yet another method for network testing, in accordance with a preferred embodiment of the present invention; and FIGS. 7A and 7B are block diagrams that schematically illustrates message flow among elements of the system of FIG. 1, in accordance with preferred embodiments of the present invention. DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS FIG. 1 is a block diagram that schematically illustrates a distributed testing system 20, used to perform diagnostic testing on a network 22, in accordance with a preferred embodiment of the present invention. Network 22 may comprise substantially any network known in the art that is capable of transmitting data packets, such as a local- or wide-area network (LAN or WAN), a public switched telephone network (PSTN), the Internet or an intranet, an asynchronous transfer mode (ATM) network, an optical or wireless network, or some combination of these different types. The network comprises switching hardware, represented schematically in FIG. 1 as switches 32. Testing system 20 comprises a plurality of end-point traffic agents 26, 28, 30, which are coupled to ports of network 22. Preferably, the traffic agents serve as both traffic generators, transmitting packets through the network, and as traffic analyzers, receiving packets and assembling information regarding the received packets, as described in detail hereinbelow. Traffic agents 26, 28, 30 are typically implemented as software processes running on host computers connected to the network. Alternatively, when network 22 is a very fast network, such as an ATM or a Gigabit Ethernet network, some or all of the hosts may comprise add-on hardware devices to accommodate the needs of the traffic agents. Further alternatively or additionally, the traffic agents may be implemented as stand-alone devices, independent of host computers. Preferably, testing system 20 also comprises one or more network management agents 34, which are associated with one or more of switches 32. These and other aspects of the testing system are described in U.S. patent application Ser. No. 09/164,488, now U.S. Pat. No. 6,269,330, which is assigned to the assignee of the present patent application, and whose disclosure is incorporated herein by reference. System 20 further includes a testing center 24, which is preferably implemented as a software process executed at a network management host. The host of testing center 24 preferably comprises a graphical workstation or a personal computer. The software for the testing center, as well as software for carrying out the functions of the traffic agents, is typically conveyed to the respective computers via network 22. Alternatively, the software may be supplied on tangible media, such as CD-ROM, for installation on the respective computers. Preferably, testing center 24 communicates through network 22 with traffic agents 26, 28, 30, as well as with network management agents 34. Alternatively or additionally, different communication apparatus, independent of network 22, such as modem dialup lines or Internet connections, are used to communicate with some or all of the traffic agents. Testing center 24 preferably conducts tests by transmitting one or more commands to at least some of the traffic agents and network management agents, and subsequently receiving reports from the agents. Preferably, testing center 24 processes the reports, evaluates network 22 based on the reports and displays test results to a network operator. The tests may be initiated by the operator, or they may be carried out automatically by the test center on a predetermined schedule or when a fault condition is suspected. When the tests are carried out automatically, and a fault condition is detected, the testing center preferably generates an alarm, so as to notify the operator that a fault has occurred. FIG. 2 is a flow chart that schematically illustrates a method for testing network 22 using system 20, in accordance with a preferred embodiment of the present invention. This method tests loss of packets in transmission through the network, and particularly the regularity or "burstiness" of such loss. Testing center 24 commands one of the traffic agents, such as agent 26, to transmit a sequence of a predetermined number of packets, say N packets, to another traffic agent, such as agent 28. Multiple transmitting and/or receiving traffic agents may likewise be employed in this method and in other test methods described hereinbelow. Each packet carries an index identifying its position in the sequence, and the packets are preferably transmitted at roughly uniform intervals. Thus, when agent 28 receives the packets, its is able to determine, based on the sequence indices, which of the packets have not arrived. Agent 28 thus records the times of arrival of the packets and, particularly, the numbers of any lost packets, typically including packets corrupted in transit. Once all of the N packets have been sent, agent 28 tabulates the results of the test, including the total number of lost packets and their indices, and conveys this information to testing center 24 for processing. The testing center computes an overall loss ratio, p, equal to the number of lost packets divided by the total number of packets sent, N. If the average interval between transmission of successive packets in the sequence is T, then the average time between successive lost packets will be T/p. The testing center checks a histogram of the difference in the indices of the lost packets, indicating the actual time between packets that were lost, against this average. If the histogram is substantially peaked at short intervals, the packet loss is considered to be "bursty." (In other words, most of the lost packets are in closely-spaced groups, separated by times much longer than T/p between successive groups.) In normal operation of a large network, particularly a datagram network, such as an Internet Protocol (IP) network, such bursty loss is to be expected from time to time as a result of network congestion. On the other hand, if the histogram is generally grouped around the average time T/p, then the packet loss is considered to be roughly regular, or "non-bursty." In this case, the packet loss is probably not due to congestion, but rather to random noise that may be the result of a hardware fault, such as a loose connection or bad interface. Thus, at the conclusion of the test, testing center 24 generates a report indicating the number of lost packets and the relative "burstiness," or variability, of the loss. Based on the histogram analysis described above, and/or on other statistical analyses, as are known in the art, the testing center indicates the probably cause of the packet loss. When the loss of packets exceeds some threshold (which may also depend on the relative burstiness of the loss), the testing center preferably generates an alarm to notify the operator of a possible fault situation. FIG. 3 is a flow chart that schematically illustrates another method for testing network 22, in accordance with a preferred embodiment of the present invention. In this method, as in the method of FIG. 2, agent 26 is commanded to send an ordered sequence of N packets, each packet containing a running index I. As agent 28 receives the packets, it assigns each packet its own index J, corresponding to the order in which the packets were received. Preferably, agent 28 then computes an order score M given by the total deviation between the sending index I and the receiving index J over all of the received packets, for example: ##EQU1## This formula may be modified to take other factors into account, such as lost packets. Other, more complex measures of the variability in packet arrival order can similarly be derived. At the conclusion of the test, the testing center analyzes and reports to the network operator on the deviation of the packets from ordered arrival. If all of the packets arrived in order, M will have the value zero. If the order of the arriving packets was severely disturbed, M will be large. It may then be concluded that there is a fault or disruption in the network, such as a routing problem or rapidly-occurring changes in network link quality. Preferably, when M (or some other, comparable variability measure) exceeds a given threshold, the testing center generates an alarm for the attention of the network operator. FIG. 4 is a block diagram that schematically illustrates message flow among elements of system 20 associated with yet another method for testing network 22, in accordance with a preferred embodiment of the present invention. In this method, a round-trip packet transmission time is estimated accurately between two network endpoints, such as agents 26 and 28, using uncorrelated one-way transmissions between the agents. To understand how this estimate is made, let t be a clock time maintained by the testing center, while t′ is the clock value maintained by agent 26 and t" is the clock value maintained by agent 28. Typically, the clocks are not precisely synchronized, so that t′=t+M1, and t"=t+M2. Here the values of the clock offsets M1 and M2 are unknown, but they can be assumed to be very nearly constant over short periods of time. To begin the measurement, testing center 24 sends a command to agent 26 to transmit a packet to agent 28 at an assigned time, ta. Agent 26 receives the command, and transmits the package at time ta′, i.e., at the time assigned by the testing center, but measured using the local clock of the agent. Agent 28 receives the packet, and records its arrival at a time tb". Meanwhile, testing center 24 sends a command to agent 28 to transmit a packet to agent 26 at a time tc, which is preferably close to the expected value of tb (although at the time that it sends the command to agent 28, the testing center has no knowledge of the actual value of tb). This packet is transmitted at a time tc" and is recorded as having reached agent 26 at a time td′. Agents 26 and 28 then preferably compute the values of (ta′-td′) and of (tb"-tc"), respectively, and report these values to the testing center. Alternatively, the raw values of ta′, td′, tb" and tc" are reported. Typically, this process is repeated a number of times. Testing center uses the values of (ta′-td′) and (tb"-tc") as follows to calculate the estimate of the round-trip transmission time between agents 26 and 28: ##EQU2## In other words, the relative clock offsets cancel out, and the testing center is left with an accurate measurement, in its own time frame, of the packet transit times from agent 26 to agent 28 and from agent 28 to agent 26. A similar approach can be used to monitor variability in one-way transmission delays through network 22. In this case, successive values of ta′ and tb" are measured with respect to transmission of packets from agent 26 to agent 28. Although the exact transmission time (ta-tb) is not measured, due to the different clock offsets of the two agents, testing center 24 can measure changes in the relative value of (ta′-tb") in order to monitor trends toward greater or lesser transmission delays. FIG. 5 is a block diagram that schematically illustrates elements of system 20, useful in understanding yet another method for testing of network 22, in accordance with a preferred embodiment of the present invention. This method is used to diagnose faults and other service complaints associated with an application that runs on a server and is accessed by clients via the network. A similar method can be used for diagnosing problems encountered in distributed computing applications. For the purposes of this method, traffic agents 62 and 72 run respectively on computer hosts 60 and 70, which communicate via network 22. An application server 64 is installed on host 60. Agent 72, under the command of testing center 24, exchanges test traffic with agent 62, independently of the application. At the same time, agent 72 is configured to emulate a client 74 of application server 64, so as to exchange application traffic with the application server. (Alternatively, the agent may work in cooperation with an actual client.) The two types of traffic that are exchanged are evaluated to assess their relative performance, typically in terms of qualities such as packet throughput, delay, loss, jitter and other measures described herein or otherwise known in the art. If there are significant performance variations between the two types of traffic, it is a likely indication that difficulties being encountered by users of the application are attributable to application or server problems. On the other hand, to the extent that similar results are obtained for both types of traffic, the application difficulties are probably attributable to network problems. In this case, the test can be repeated using another traffic agent, preferably one that is relatively close to host 60, in order to determine whether the situation improves when the communication path through the network is shortened or otherwise altered. Another option is for testing center 24 first to test communications between agent 72 and application server 64. While doing so, the testing center receives and analyzes the traffic generated by the application, in order to derive information such as the length of the data received by agent 72, or even the full contents of communication transactions. Then, during a second stage, the testing center generates parameters to be used in a test conducted between agents 62 and 72, so that the test traffic exchanged in the second stage will be as similar as possible to the actual application traffic in the first stage. FIG. 6 is a flow chart that schematically illustrates a method for synchronizing the initialization of testing performed by multiple traffic agents, in accordance with a preferred embodiment of the present invention. This synchronization comes to compensate, at least in part, for the relative offsets of the clocks maintained by different traffic agents, as mentioned hereinabove with reference to the embodiment of FIG. 4. Furthermore, when a large number of traffic agents (and possibly network management agents 34, as well) participate in a given test scenario and are required to begin testing simultaneously, the present method generally provides more accurate synchronization than methods known in the art, which rely on sending a "start" command to all of the test participants at the same time. Testing center 24 determines a time, TST, at which a given diagnostic test is to begin. Preferably, TST is set sufficiently far in advance so that the testing center is assured of having time to communicate with all of the test participants so as to complete the following synchronization procedure before time TST arrives. TST is determined in accordance with a local clock, TCL, maintained by the testing center. The agents participating in the test, however, maintain their own clocks, which typically differ from TCL. Therefore, to compensate for this difference, the test center sends an initialization message to each of the participating agents that includes both the value of TST and a value of time TCL sampled at the time that the respective initialization message is sent. Preferably, sampling of TCL is delayed after preparation of the initialization message so as that the sampling takes place as close as possible to the actual transmission of the message. When a given agent receives its respective initialization message, it compares the sampled value TCL to its own local clock TAL. The agent then sets its own schedule to begin the test at a time given by TAL+TST-TCL. In this manner, the offsets between the different clocks in system 20 are zeroed out, and all of the agents begin the test at roughly the same time. Although there may still be a residual discrepancy in the start times of the agents, due to the different transit times of the respective initialization messages, this discrepancy is still substantially smaller than the discrepancy that occurs when the testing center must simultaneously send a "start" message to each participant in a long sequence. FIGS. 7A and 7B are block diagrams that schematically illustrate other testing configurations that can be implemented in system 20, in accordance with a preferred embodiments of the present invention. Although some preferred embodiments are described hereinabove with reference to pairs of participating traffic agents, other, more complex test configurations can also be used in these tests. Thus, for example, in FIG. 7A, testing center 24 instructs traffic agent 26 to send a packet to traffic agent 28, which responds by sending another packet on to traffic agent 30. In FIG. 7B, both of agents 26 and 30 exchange packets with agent 28, preferably substantially simultaneously. Measures of packet arrival variability and transmission delay, such as those described hereinabove, may similarly be made in these more complex configurations. The network management agents may also be involved in such measurements. It will be appreciated that the preferred embodiments described above are cited by way of example, and that the present invention is not limited to what has been particularly shown and described hereinabove. Rather, the scope of the present invention includes both combinations and subcombinations of the various features described hereinabove, as well as variations and modifications thereof which would occur to persons skilled in the art upon reading the foregoing description and which are not disclosed in the prior art.
|
Same subclass Same class Consider this |
||||||||||