Selection of addressed processor in a multi-processor network

Abstract

A multiple processor network is described whereby a "Sender" processor can address a "Receiver" processor within a system of processors and select the first processor which is found to be in an idle condition, and whereby a Sender can address processors of a specially indicated type. A Global Memory Module (GMM) and a system hierarchy of processors is described which provides access to a plurality of addressable memory storage units. A multiple number of processors or computer systems are connected to one or more Global Memory Modules whereby memory resources may be shared by multiple processor systems and where control and communications are provided between the processors through the Global Memory Modules. The Global Memory Module may be organized into a hierarchy of Global Memory Module systems whereby processors attached to "lower level" GMM systems may access memory in "higher level" GMM systems. Means are provided whereby a processor in one GMM system may send commands and messages to a processor in another GMM system. Means are provided by which one processor can address another specific processor in the system network or whereby one processor can address an "available" processor in a system designated under a system name, and the network will choose the processor which is "idle" or, if there is no idle processor available, will then choose a processor which is "not engaged", that is to say, a processor which when it finishes its currently scheduled activities, will then be available for processing of a received command and message.

Claims

What is claimed is:

1. A multiprocessor system network wherein a plurality of processors are formed into system-groups wherein each system-group includes a plurality of processors connected to a common memory-module-system-controller, and a plurality of said memory-module-system-controllers are interconnected to provide for intercommunication between system-groups, and wherein said memory-module-system-controllers are connected to establish a hierarchy of levels in which higher level system-groups of processors exert a master control status over lower level groups of processors which function as slaves, the network comprising:

(a) a plurality of processors, wherein individual groups of said processors are connected to a common memory-module-system-controller to form a system-group of processors;

(b) a plurality of memory-module-system-controllers, each of said controllers including:

(b1) a plurality of memory storage units for storing data in addressable locations;

(b2) a memory control unit for reading or writing data having addresses contained within said memory storage units and including a repeater port for transmitting higher level addresses to higher level memory-module-system-controllers;

(b3) a plurality of input ports, each input port connected to a local processor to form said system-group of processors;

(b4) a system control unit for inter-system communication and sequentially granting sending status to processors connected to said input ports;

(c) connection means between said plurality of memory-module-system-controllers for interchange of data and control signals, said connection means including:

(c1) a system communication-control bus;

(c2) a memory repeater bus;

(d) means to provide a system name to a system-group of processors, wherein said system name is indicative of the hierarchical level of each system-group in the network;

(e) means, in said system control unit, for a sender processor to communicate with a plurality of processors by addressing them via a system name;

(f) means, in said system control unit, to select, from those processors having the addressed system name, the first processor which is idle, said first idle processor being selected to receive a message from said sender processor.

2. The network of claim 1 which includes:

(e) means for detection by a Sending Processor that a receiving processor has been halted;

(f) means for signaling, to the Sending Processor, the identity of said halted processor.

3. The network of claim 2 including:

(g) means, in a halted processor, to cause the halting of every other processor encompassed by the same system name.

4. The system network of claim 3 including:

(h) means for reconfiguration of the hierarchy of processors in the network by the re-naming of the processors.

5. The network of claim 1 wherein said means for providing name signals includes:

mask means for identifying each individual processor connected within the same system.

6. The network of claim 1 which includes:

means for selecting, among those processors carrying the addressed system name, a processor that is "non-engaged" should there not be available a processor that is idle, said condition of "non-engaged" being recognizable by said system control unit receiving a signal indicating that the message buffer of that processor is empty of data;

means, in each of said input ports, for receiving and temporarily storing a message from processors other than the processor fixedly connected to that input port, said means being designated as a message buffer.

7. In a multi-processor network in whcih processor-systems are connected sequentially via a system communication-control bus and a memory-repeater bus to establish a hierarchical level of systems, and wherein each processor-system consists of a plurality of processors, connected, via separate input ports, to a common system-control-memory-control module designated as a global memory module controller, the combination comprising:

(a) a plurality of global memory module controllers, each of said controllers including:

(a1) a plurality of memory storage units for storing data in addressable locations, said addressable locations having address number values which increase relative to the location of said memory storage units in the hierarchy level;

(a2) a memory control unit for reading or writing data located at addresses contained within the memory storage units situated at the local hierarchical level, and including:

a repeater port for transmitting higher level addresses to higher level global memory module controllers;

(a3) a system control unit for inter-system control and communication and for sequentially enabling processors, connected to the input ports, to become sending processors;

(a4) a plurality of input ports, each input port providing connection to a local processor to form a system-group of processors having a hierarchical level related to the position of their global memory module controller in the network, and wherein each of said input ports includes a processor port adapter which includes:

(a4a) a response buffer for receiving and storing commands and data from a locally connected processor;

(a4b) a message buffer for receiving message data information from other processors in the network and for transmitting said information to said locally connected processor;

(b) sequential connection means between said plurality of global memory modules for interchange of data and control signals, said connection means between any two adjacent global memory modules in the hierarchy including:

(b1) a system communication-control bus;

(b2) a memory repeater bus;

(c) means to provide a system name to a system-group of processors wherein said system name is indicative of the hierarchical level of those processors in that system-group in the network;

(d) means, using said system name, to address a plurality of processors as potential receivers in said network by transmitting the system name to the message buffers of the processors being addressed;

(e) means in said system control unit to select a receiving processor to be interrupted, said means including:

means to select a first processor in a system of commonly named processors which is found to be idle;

means to select the first "not-engaged" processor should none of the processors in the named addressed system be found to be idle.

8. The network of claim 7 wherein each processor includes:

means to establish its own processor name in the network;

means to establish a processor name for another processor in the network;

means to concurrently address a plurality of processors in a system of processors;

means to address a subset of processors in a system of processors where said subset of processors have specific capabilities as indicated by their name.

9. The network of claim 7 wherein each Sender Processor includes:

means to address a plurality of processors in a system whereby all of such addressed processors will be interrupted if they are not presently "engaged";

means to place, in the Sending Processor's response buffer, signal indications to indicate that processors actually received the sender's command, and to indicate processors which were engaged and did not receive the Sender's command.

10. The network of claim 7 wherein each processor includes:

means whereby a Sending Processor may address a plurality of receiving processors and each of said receiving processors will be interrupted regardless of whether or not they are engaged.

11. The network of claim 7 wherein each of said processors in the network includes:

means for a Sending Processor to address a plurality of processors and to cause these processors all to be halted;

means for said Sending Processor to address a plurality of processors and to cause each of the addressed receiving processors to be cleared.

12. The network of claim 11 wherein said processor port adapter includes:

(a4) an access control register which is settable to determine which areas of said memory storage units may be written-into or read-from by the local processor connected to said processor port adapter, said access control register including a Write-access register section and a Read-access register section.

13. The network of claim 12 in which each receiving processor which has been cleared will also clear all the digit positions of its Write-access control register and its Read-access control register.

14. The network of claim 7 in which each of said processors in the network includes:

means for a Sending Processor to address a plurality of receiving processors and to cause each of said receiving processors to load information or programs from disk or tape into a memory resource in the network;

means for a Sending Processor to address a plurality of receiving processors and to cause said receiving processors to start operating.

15. The network of claim 7 wherein each of said processors includes:

means for a Sending Processor to address a plurality of receiving processors in the network;

means to select one of said plurality of receiving processors on a locational left-to-right sequential priority basis.

16. The network of claim 15 wherein each receiving processor includes:

means, in its processor port adapter, to return a status code signal, to the Sender Processor, indicating the conditional state of the receiving processor;

and wherein the receiving processor will not be interrupted during execution of the addressing cycle;

said addressing of the receiving processors being accomplished by means of a processor name signal initiated by said sending processor.

17. The network of claim 12 wherein said network includes:

means for a Sending Processor to access a Write-access control and Read-access control register number "i", where "i" represents a numbered memory storage unit location;

means in said system control unit for computing the level at which the memory module "i" resides in the hierarchy;

means for the Sender Processor to address a plurality of receiving processors;

means in said system control unit to verify that the memory storage unit "i" is commonly accessible to both the Sender Processor and the receiver processor.

18. The network of claim 17 which includes:

means to modify the receiver's Read-access control register and Write-access control register so that access to the memory storage unit module "i" is given to the selected receiving processor.

19. The network of claim 17 which includes:

means to re-set the bits in the Sender's Write-access control register and Read-access control register, thus denying further access to the memory module "i" to the Sending Processor.

20. The network of claim 17 wherein each of said processors in said network includes:

means wherein the Sender Processor's system control unit does not reset the Sender's Write-access and Read-access control registers, thus to permit the sharing of the memory storage unit module "i" by both the Sending Processor and the receiving processor.

21. The network of claim 17 wherein each of said processors in said network includes:

means for a Sending Processor to cause its system control unit to reset the Sending Processor's Write-access and Read-access control registers position "i" to deny further memory access to memory storage unit "i" by the Sending Processor.

22. The network of claim 21 wherein each of said processors in said network includes:

means for a Sender Processor to address a plurality of receiving processors by a generalized system name;

means to address a plurality of processors in the network to find whether each of the addressed processors has a memory storage unit module number "i" set in its Write-access control register;

means for each such receiving processor that fulfills said "set" condition to send its processor name back to the Sending Processor;

means to address a plurality of receiving processors in the network by a generalized system name, and to select those receiving processors which have memory storage unit module number "i" set in their Read-access control registers;

means for each of such receiving processors which meet this set "i" condition to send its processor name back to the Sending Processor.

23. The system of claim 22 wherein said system control unit includes:

a single bit error register for registering the occurrence of an error condition, and

wherein each of said plurality of processors in said network includes:

means to address a receiving processor whereby the contents of that receiving processor's single bit error register is communicated back to the response buffer of the Sending processor, thus to signal the occurrence or non-occurrence of an error condition.

24. The network of claim 12 wherein said system control unit includes:

a First Word Address Register which is set to provide the number "i" of the lowest numbered memory storage area associated with that global memory module controller, and said network includes:

means for a Sending Processor to address a receiving processor in said network by identification number;

means in a selected receiving processor to return the digital contents of its First Word Address Register (FWAR) back to the response buffer of the Sending Processor;

means for accessing the receiver processor's Write-access control register and Read-access control register;

means, in said receiver processor, for generating a response word and for placing, in said response word, the contents of the "i"th position of the receiver's Write-access control register and the "i"th position contents of the receiver's Read-access control register.

25. The network of claim 7 wherein each of said processors in said network includes:

means to identify itself with a system name having first and second qualifiers;

means for a Sender Processor to address all other processors with the same system name and the same values of the first and second qualifiers;

means to send a manual Halt command to all such addressed receiver processors which will all then be halted by said manual Halt command.

26. The network of claim 25 wherein each of said processor port adapters includes:

a dependent status register for indicating, when set, that said processor is a slave to another master.

27. The network of claim 26 which includes:

means, in each of said system control units, for communicating to lower level processor subsystems, a manual Halt command sent by a higher level processor system;

and whereby each processor, in said lower level subsystem, will also be halted if its dependent status register is set.

28. The network of claim 27 which includes:

means for said Sender Processor to address and transmit a "manual clear" command to the processors with the same system name and to their subordinate lower level subsystem processors to cause all of said processors to clear.

29. A multiprocessor system network wherein a plurality of processors are formed into system-groups and each system-group includes a plurality of processors connected to a common memory-module-system-controller, and a plurality of said memory-module-system-controllers are interconnected to provide for intercommunication between system-groups, said memory-module-system-controllers being connected to establish a hierarchy of levels in which higher level system-groups of processors exert a master control status over lower level groups of processors which function as slaves, the network comprising:

(a) a plurality of processors, each group of processors, which connect to the same memory-module-system-controller, forming a system-group having a position or "level" in a network of system-groups;

(b) a plurality of memory-module-system-controllers, each of said controllers including:

(b1) a plurality of memory storage units for storing data in addressable locations, said addressable locations having address values unique to that memory-module-system-controller;

(b2) a memory control unit for reading or writing data having addresses contained within said memory storage units and including:

(b2-1) a repeater port for transmitting higher level address requests to higher level memory-module-system-controllers;

(b3) a plurality of input ports, each input port connected to a local processor, each input port including:

(b3-1) a Response Buffer for receiving data in response to commands and/or queries put out by that processor;

(b4) a system control unit for inter-system communication and for sequentially granting sending status to processors connected to said input ports, said system control unit including:

(b4-1) means to establish a processor-identification number for each processor connected to an input port of a memory-module-system-controller;

(c) connection means between said plurality of memory-module-system-controllers for interchange of data and control signals, said connection means including:

(c1) a system communication-control bus;

(c2) a memory repeater bus.

30. The network of claim 29 wherein said means to establish a processor identification number includes:

register means, designated as a processor identification register, for storing the identification number of a processor, said identification number being formed by concatenating the numbers of the input ports traversed by the communication path from the memory-module-system-controller having the highest memory address to the particular processor having the number designation.

31. The network of claim 30 wherein said system control unit includes:

means for a sender processor to select a receiver processor by means of said processor identification number;

means for a sender processor to communicate with its system control unit to inquire as to its processor number identification;

means, within said system control unit, to provide a sender processor with information as to the value of its processor identification number.

32. The network of claim 30 wherein said system control unit includes:

means, for a selected receiving processor, through its own system control unit, to provide a response signal as to the value of its processor identification number, said response signal being placed in the Response Buffer of the processor port of the sending processor, thus establishing, to the sending processor, the locational position level in the physical hierarchical network that the receiving processor occupies.

Description

CROSS REFERENCE TO RELATED APPLICATIONS

The system described in this application is related to other applications on the same or similar subject matter as follows:

An application entitled "Module for Coupling Computer-Processors", filed Dec. 21, 1978, Ser. No. 971,677, inventors Clifford Bellamy and John Besemer.

An application entitled "Multi-Processor Communication Network", filed Dec. 21, 1978, Ser. No. 971,890, inventors Clifford Bellamy and John Besemer.

An application entitled "Hierarchical Multi-Processor Network for Memory Sharing", filed Dec. 22, 1978, Ser. No. 972,431, inventors John Besemer and Clifford Bellamy.

BACKGROUND OF THE INVENTION

This invention relates to data processing systems and methods for linking or coupling data processing systems with system resources such as memory and other processor units or processing systems.

With the development of computer systems, processing units and system resources such as memory, input/output units, data communications systems, etc., there has been more desire to form integrated computing system networks whereby a multiplicity of computers may be connected to one another, and even wherein different types of computer systems may be coupled to one another for mutual intercommunication and distribution of processing tasks. One of the major obstacles impeding the design of a system network architecture has been the lack of a comprehensive means for interconnecting the various processing units of the system.

In a multiprocessor network with a multitude of resources, there is always the problem of how to allocate the use of the resources and how to allocate and control which processor systems may communicate with which other processor systems and what usage they may make of these other systems.

In the embodiment of the present invention, an architecture and methodology is disclosed for the coupling of a multiple number of processing systems whereby any one system may be permitted to access and control a selected one, or one of a group of processors in another system, in a controlled fashion whereby certain memory resources may be shared and certain memory resources may be restricted; or where certain memory resources are permitted to have only a limited use to a given processor. Thus, the herein described system network provides for executive control of requesting processors and slave processors in an orderly scheduled basis plus the allocation of resources and the general network control which will permit an efficient use of all the available resources.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a drawing of a multiprocessor system whereby individual processors are coupled through global memory modules.

FIG. 2A shows a multiprocessor network which is coupled by global memory modules showing the input ports of a global memory module and other major elements of the global memory module.

FIG. 2B is a schematic drawing of a global memory module showing how each port of the global memory module provides a processor port adapter and a requestor port adapter which intercooperate respectively with the global system control adapter and memory ports which connect to memory storage units.

FIG. 3 is a schematic drawing of a typical input port of a global memory module, showing in more detail the requestor port adapter and the processor port adapter.

FIG. 4 is a schematic drawing of a section of the global memory module known as the global system control adapter and showing the major elements thereof.

FIG. 5 is an overall schematic drawing of the global memory module cabinet, showing the major sections thereof.

FIG. 6 is a schematic drawing showing a hierarchy of global memory modules whereby global memory modules are connected in a hierarchy and how a processor or a global memory module (GMM) can be identified by an identification number, PID.

FIG. 7 is a schematic drawing showing global memory modules connected in a hierarchy, and wherein each GMM has a first word address register (FWAR) which identifies the memory modules (memory storage units) connected to it.

FIG. 8A is a schematic drawing of a hierarchical network whereby systems of processors are interconnected to form a hierarchy of systems indicated as a top system, mid system and bottom system.

FIG. 8B shows a hierarchical system of global memory modules and processors together with the symbolic name and processor number which identifies each unit.

FIG. 8C shows three cases of logical memory structures using a global memory module together with illustrations of the access control registers (ACR). FIG. 8C is on two sheets, 8C-1 and 8C-2.

FIG. 8D shows the global memory module configuration of FIG. 8B together with the logical structure and the settings of the access control register. FIG. 8D is on two sheets, 8D-1 and 8D-2.

FIG. 9 is a drawing showing the fields used to identify a particular processor and which includes a mask of 12 positions to permit identification of any one of 12 processors in a given system-group of processors connected to a global memory module.

FIGS. 10A through 10I show the global command words used to operate the various capabilities of the network system.

FIG 11A is a schematic drawing showing how the GSC (global system control) bus provides communication from a Sender-processor to a series of possible Receiver-processors in a network.

FIG. 11B illustrates the circuit whereby the ports of a global memory module alternately receive a global system control bit which gives the port having the control bit the temporary control of the global system control bus.

FIG. 12 is a circuit diagram showing the generation of the Sender code signal SNDR.sub.p which makes a particular port p assume the condition of being a "Sender" to other processors in the network.

FIG. 13 is a diagram showing how each of the input ports of a global memory module provides comparators for signal comparison and also how the global system adapter (GSC) provides comparator circuitry. FIG. 13 is on two sheets, 13-1 and 13-2.

FIG. 14 shows the organization of a PROM for the comparator A IN and comparator B IN of the global system control of FIG. 13.

FIG. 15 is a schematic diagram of a global memory module showing more detail of the global system control (GSC) and the processor port adapter.

FIG. 16A is a circuit drawing of the inhibit change circuit which provides an output to the mode control of the state register flip-flop.

FIG. 16B illustrates the timing arrangements for this circuit.

FIG. 17 is a schematic drawing of communications between two different processors attached to two different global memory modules and their use of the Response Buffer and the Message Buffer.

FIG. 18 is a circuit diagram showing the micro-code control circuits of the global system control (GSC) of the global memory module (GMM). FIG. 18 is on three sheets, 18-1, 18-2 and 18-3.

FIG. 19 is a circuit diagram of the priority resolution circuitry used in the global memory module. FIG. 19 is on two sheets, 19-1, 19-2.

FIG. 20 is a timing diagram showing the timing relationships between the global memory module (GMM) and a requesting processor.

FIG. 21 is a circuit drawing of the provisions made for the left-right priority selection as between global memory module cabinets.

FIG. 22 is a timing diagram illustrating the signals for the resolution of priority for three selected situations.

FIG. 23 is a circuit diagram illustrating the data paths between a particular processor, the Message Buffer, the Response Buffer and the repeator port output cable. FIG. 23 is on four sheets which follow the pattern of: 23-1 . . . 23-2 and 23-3 . . . 23-4.

FIG. 24 illustrates the circuit by which the select flip-flop in a processor port is activated.

FIG. 25 is a circuit diagram showing the elements used in the comparison of Sender and Receiver processors in determining which level memory storage units may be shared by the processors and also whereby the processor identification PID of a receiver processor can be compared against the sender processor's PID; and whereby the processor name and mask of a Receiver-processor can be compared against the name and mask field sent by the Sending processor.

SUMMARY OF THE INVENTION

A system architecture is provided whereby a "global memory module" placed in a hierarchy is used in coupling different sets of computer processors to each other. A global memory module is provided which can be organized into multiple heirarchical units wherein each of the global memory modules can couple at least four processors or computer systems. The global memory module provides memory storage which can be shared by any of the processors attached to it, or shared by processors attached to other global memory modules within the hierarchical memory system. Thus, the global memory module provides a way of coupling computers into a multi-processor system.

The global memory module includes four input requestor ports, each of which can support a connection to a processor-computer system. A repeater port of the global memory module provides for connection to a "higher level" global memory module to permit access to additional memory storage when the address of requested data is beyond the scope of the first accessed global memory module. Further, the global memory module includes a global memory control which provides access to at least four memory storage units which are part of the global memory module, or alternatively to a repeater port which provides access to the memory of higher level global memory modules. The requestor ports are part of a larger control unit called the global system control (GSC) which provides intercommunication and control between a given level global memory module and the subsequent levels of global memory modules.

The hardware structure is utilized with the development of a master control program operating system (MCP) which is applicable to the various types of computer involved, such as for example the Burroughs B 6000 series of computers. The operating system provides for two types of processor couplings, which are: (a) tight coupling of processors and (b) a loose coupling of processors.

In "tight coupling" of processors there may be two or more B 6000 processors which are run under the control of a single MCP as a "tightly coupled" system. On the other hand, two or more processors running under the control of "separate" operating systems may communicate with each other and access common memory as "loosely coupled" systems. A loose coupling may exist between two processors, a processor and a tightly coupled system, or two tightly coupled systems.

The master control program (MCP) is primarily related to the utilization of the hierarchical memory structure which is provided by the global memory modules and the techniques provided for "sharing" resources. All the programs written for the Burroughs B 6000 systems will run without modification under the master control program (MCP).

From a multi-processor system point of view, the B 6000 series of computers may form a network system which provides an advanced architecture. However, the advanced architecture of a computer such as the Burroughs B 6800 can be organized into an improved system by adopting the global memory method of coupling systems. The background and operation of the Burroughs B 6800 computer is described in a publication of the Burroughs Corporation of Detroit, Michigan 48232 as covered by manual:

500 1290, copyright 1977

By coupling systems, the following improvements are made possible:

1. The efficiency of a tightly coupled multi-processor system can be improved by reducing the "contention" between processors.

2. The process predictability can be improved by isolating sets of processes in memory storage which is local to a processor.

3. Complex multi-processor systems can be constructed by coupling together architecturally simple, single processor systems.

4. System reconfiguration capabilities, including the detection and the identification of failing units, are improved.

5. By using one memory subsystem local to a processor, and using another memory subsystem when memory is shared between processors, the local memory can be made simpler, faster and less complex. The cost and performance of the single processor system is not influenced significantly by allowing for the possibility of its use with accommodating additional processors.

DESCRIPTION--GENERAL

The Global Memory Module (GMM) has the purpose of arranging for a multitude of different processors to be coupled together to form a multiprocessor installation that can be dynamically (via software) configured to suit a variety of conditions. By providing communication paths between the various processors and by providing memory modules (Memory Storage Units, MSU) that are common to two or more processors, the GMM facilitates the use of software control to couple processors together to form systems and to couple, between these systems, to form a processor-system hierarchy tailored to the particular requirements of the specific task at hand. The processor-system hierarchy provides a "processor name" to each processor which "processor name" is not directly related to the physical configuration of the installation except as arranged by the software.

The Global Memory Module (GMM) is divided into four functional areas as follows:

(a) Global System Functions

(b) Memory Control Functions

(c) GMM Maintenance Functions

(d) Power Supplies and Clock

Global System Functions

In order to perform global system functions, it is necessary that the (i) processors, (ii) the GMM cabinets, (iii) and the memory storage modules, in the network be "identified" as to their respective physical locations. Further, the memory storage unit modules of the global memory, that can be shared, must be identified and also the processors must be identified by a logical name structure.

Processor Identification

As will be seen in FIG. 6, each Global Memory Module has four input requestor ports designated, in this figure, as 1, 2, 4, and 8. If there are higher level GMM's in the hierarchy, then a repeater port (R) is provided for the particular Global Memory Module which is lower in the hierarchy than the highest level Global Memory Module.

The physical identification (PID-processor identification) which is placed on any given processor in the system is determined by the relationship of the input requestor ports through which the processor has a path to the highest level GMM (level 0).

In the preferred embodiment, generally only four levels of GMM's are used in the network. Thus, as will be seen in FIG. 6, the processor identification, PID, is a four-digit number with the "leftmost" digit representing the connected requestor port of the level 0 GMM. For example, in FIG. 6, the lowest level processor, P.sub.D, is designated with a PID of 8148. This means that the processor P.sub.D has a connected path to the highest level GMM at port 8 of the highest level GMM (level 0 GMM); then the connected path moves through port 1 of the level 1 GMM; then through port 4 of the level 2 GMM and then through port 8 of the level 3 GMM. Thus, tracing the connected path from the highest level GMM (level 0 GMM) to the lowest level GMM (level 3 GMM); it is found that the processor P.sub.D connects through ports 8148 and thus this number is given to the processor as the processor identification number, PID.

Within each Global Memory Module there is located (FIG. 4, FIG. 6) a three-digit processor identification register (PIDR) in which the PID digits (representing the path from this GMM's Repeater Port to the highest level [level 0] GMM) are set by manual switches at the time of installation. If there are three higher level Global Memory Modules (GMM's), the first "three" PID digits are set manually. However, if there are only two higher level GMM's, then only the first two digits are set manually. If there is only one higher level GMM, then only the first PID digit of that particular GMM is manually set.

When a Global Memory Module is the highest level Global Memory Module in the system (level 0), then no PID digits are set manually in that particular GMM's processor identification register, PIDR.

The port number of a processor which is connected to a GMM is "concatenated" (by the PID circuitry) with the manually set digits of the PIDR, processor identification register. Thus, the PID circuitry places the processor's port number (i.e., 1, 2, 4 or 8) into the leftmost "zero" position of the PIDR of the GMM, that is, the digit position that is equal to 0.

The Global Memory Module 10 of FIG. 2A is made of two main functional units: (a) the global memory control (GMC) 20 and (b) the global system control (GSC) 30.

The GMC 20 is a time-multiplexed memory control with four requestor ports R.sub.A, R.sub.B, R.sub.C, R.sub.D and repeater port 25.sub.3. Thus, if a requestor (as processor P.sub.A) provides a memory address greater than the highest address in the memory storage unit module (MSU 10.sub.l, 10.sub.m, 10.sub.n, 10.sub.p) connected that particular GMC, then the request is passed through the repeater port 25.sub.3 to the requestor port R.sub.A of another Global Memory Module GMM.sub.2, if one is connected thereto.

The Burroughs B 6800 and B 6900 processors have local memory controls which operate on a similar basis and which will pass memory access requests out through a repeater port when the address required is greater than the local maximum address. This mechanism allows a number of processors or subsystems to be interconnected by a tree-like memory structure as can be seen in FIGS. 1 and 2A. Thus, while a particular Burroughs B 6800 processor such as P.sub.D may be connected to the Global Memory Module GMM.sub.3 of FIGS. 1 and 2A, the 512 K words of the local memory M.sub.D will be enhanced--since the processor P.sub.D can access the memory storage unit modules 10.sub.l -10.sub.p of GMM.sub.3 and in addition can use the repeater port 25.sub.3 to access further memory storage unit modules in the higher level Global Memory Modules, such as GMM.sub.2, GMM.sub.1, etc. Thus, the memory accessibility of any given processor in the hierarchical system is considerably enhanced because of the communication paths providing access to other memory modules.

Each processor which is connected to a Global Memory Module will have associated with it a Read Access Control Register (RACR) and also a Write Access Control Register (WACR), seen in FIG. 3. These registers, RACR and WACR, reside in the processor port adapters (PT.sub.1, PT.sub.2, PT.sub.4, PT.sub.8 of FIGS. 1, 2A) of each port which is associated with a processor connected to a given Global Memory Module.

The Read Access and the Write Access Control Registers are used to define the memory space which a processor will have access to. Thus, the RACR and WACR of the processor port adapter PT will regulate (for the processor connected to it) the type of access that processor will have to any given memory area.

Thus, not only can a single GMM be used to couple, for example, four B 6800 processors but it can also regulate what areas of memory will be "accessible" and what "type" of access (Read or Write) will be permissible. A system with both Read and Write access to an area of memory can "grant" another system "full access" or it can grant "read only" access or it can retain for itself exclusive access to an area of memory.

The Global System Control GSC 30 of FIG. 2A senses the "status" of each processor to determine whether it is running, not running, or super halted. If a processor is super halted or not running, the GSC 30 will automatically transmit a message to another processor in the same system to enable the master control program (MCP) to take recovery action by sending that processor an alarm interrupt or by initiating a halt-load sequence.

When a processor in a tightly coupled system is "halted", the GSC 30 senses this and automatically halts the other processors in the system. The same condition applies when a system is "cleared".

As seen in FIG. 2A, each Global Memory Module has a series of repeater ports 25.sub.3, 25.sub.2, 25.sub.1, 25.sub.0, and control busses B.sub.10, which are used to build a "hierarchy" of global memory modules connecting systems of processors.

Thus, for example, Burroughs B 6800 processors, as shown in FIG. 2A, can be operated with:

A. one master control program (MCP) as a "tightly" coupled system.

B. as separate independent systems making use of global memory as an extension of their local memory.

C. as "loosely coupled systems", each with its own master control program (MCP) that communicates through the common memory of a GMM and the global memory bus C.sub.10.

As seen in FIG. 2A, the GSC bus B.sub.10 is connected from one level of the Global Memory Module hierarchy, such as GMM.sub.3, to the global system control GSC of higher level Global Memory Modules in the hierarchy, such as GMM.sub.2, GMM.sub.1, GMM.sub.0. Each Global Memory Module, such as GMM.sub.3, will have a global system control adapter, GSC 30, having a multi-cabinet adapter as 26.sub.3. The GSC bus B.sub.10 is used for transmitting interprocessor interrupts and transmitting system control information.

Thus, the concept of using Global Memory Modules to couple processors in a network of systems which form a hierarchy of levels provides a number of improvements and advantages for multiprocessor systems. The Global Memory Module concept provides facilities for:

(1) a plurality of processors capable of accessing a common memory within a particular GMM or higher level GMM.

(2) communication between processors by passing messages between processors together with an associated interrupt signal.

(3) putting constraints on any given processor so that it may only access specifically designated areas of a given Global Memory.

(4) the addressing of processors by means of a "name" so that a particular individual processor may be selected; or addressing a processor or processors by name in terms of a "set" from which one or more processors may be selected.

(5) addressing a set of processors in a given system and then selecting, among these processors, the "idle" processor member among the set of processors, thus bypassing the selection of those processors which may be "engaged" or those which, while "idle", are still carrying information which will cause them to become engaged.

(6) the confining of communication paths between processors according to the hierarchy defined by their names, that is, names assigned to the processors. Thus, the names given to the processors in a system hierarchy will determine which processors may communicate with each other and which processors may not communicate with each other.

(7) the starting and the stopping of one processor by another processor within the system hierarchy or the network hierarchy.

(8) programmatic control, or the way in which a network installation is configured through the "setting of names", and through the use of memory access control registers in the Global Memory Control (GMC) 20 of the Global Memory Module (GMM) 10.

(9) the automatic detection of any processor which is in an "abnormal state" together with the transmission of a message concerning this condition to other processors which may try to communicate with the "abnormal state" processor.

There are a number of ways in which processors in an installation may be related. Referring to FIG. 6, for example, one processor PID 8840 may be the slave of a processor PID 8820, where processor 8820 is a B 6800 computer and the processor PID 8840 is a data communications processor connected to a Global Memory Module. Or there could be another relationship where the processors 8820 and 8840 are identical and equal processors which can operate "independently" one of the other and wherein one cannot start or stop the other.

Thus, any given processor may be considered to be operating in a different environment according to its relationship to other processors and to the system in which it resides.

In order to better describe the conditions under which a processor operates, the following definitions will be useful for later discussions of the Global Memory Module network hierarchy:

System:

A set of processors that are functionally equal from the point of view of providing some sort of service to a requestor. This set of processors will have the same name and is referred to as a "system". In the preferred embodiment, the Global Memory Module provides for a maximum of 12 processors in a "system".

Equal Systems:

Two systems that "cooperate" but have no direct control of each other.

Installation:

A computer installation would be said to consist of a "set" of coupled systems, that is, a set of systems that can communicate with each other.

Master System:

A system that has the ability to exercise some control over another system is considered to be a "master". The one system may be the master of several systems while at the same time being slave to another system.

Slave System:

A system that can be controlled by another is considered to be a slave system. It can be a slave to one system, while on the other hand being a master to other systems.

As an example of the above definitions regarding "coupled systems", the following examples may be used to more clearly illustrate possible configurations:

I. Referring to FIG. 1, if there are four "equal" Burroughs B 6800 processors (P.sub.A -P.sub.D) connected by a Global Memory Module 10, and which operate under the control of one operating Master Control Program, then there is formed what may be called a "tightly coupled" system. An external processor to this system, such as Q.sub.D ' (having local memory D.sub.D '), which is addressing a message to this system does not mind which one of the four processors, P.sub.A, P.sub.B, P.sub.C, P.sub.D, is interrupted to provide the service requested; nor is the external processor Q.sub.D ' concerned about the number of processors in the system.

II. Referring to FIG. 2A, a Burroughs B 6800 "system Q" consisting of two processors is coupled to another system R over which it has control. Both systems Q and R themselves are "tightly coupled" systems, but the coupling between system Q and system R is "loose". System Q might be managing a large data base and system R might be carrying out all data communication functions for the installation. In this illustration the system Q is the "master" system, while the system R is the "slave" system.

III. Referring again to FIG. 2A, a situation could exist where the Q system was the master, the R system was both slave/master, and S system was a slave. For example, if the processor S.sub.D was a data communications processor connected to a Global Memory, GMM.sub.o, and processor S.sub.D was working as a slave to system R which is the master of system S; and at the same time system S is a slave to system Q. The processors in system S, while being members of one system, could execute their own copies of the same code and each be capable of establishing an outward connection or receiving an incoming call over a switched telephone network. System R then addresses system S as a system to have one of its processors dial out, then addresses that processor specifically to transmit blocks of data down the line.

Referring to FIG. 1, the Global Memory Module 10 provides at least four memory interface hubs, L, M, N, P, which connect to global memory storage units (MSU) 10.sub.l, 10.sub.m, 10.sub.n, 10.sub.p. These physical memory units may be referred to as memory storage modules. Thus, a preferred embodiment of a Global Memory Module in the system will have the capability, through its memory hubs, of connecting to four memory storage modules. These memory modules may be of 384K bytes or even 768K bytes. Each memory module (memory storage unit) is given an identification by means of a two-digit module number which is manually set by the field engineer at the time of installation. This two-digit module number is set in accordance with the following rules:

(a) Each module number associated with a given GMM must be unique.

(b) All module numbers associated with a given GMM must be higher in value than any module number associated with GMM's that have a path to this GMM's requestor ports.

(c) Duplicate module numbers may exist within the system but only so long as rule (b) above is not violated.

Within each Global Memory Module there is provided a global system control GSC adapter 30 which has a first word address register, FWAR, as seen in FIGS. 4 and 7. The FWAR consists of four two-digit fields, which provide a total of eight bits. The FWAR is used to indicate the lowest numbered memory module connected to the particular GMM, and also the lowest module number of the memory storage module (MSU) connected to each higher level GMM that is accessed by this GMM's repeater port.

In the FWAR of GMM.sub.3 of FIG. 7, the "leftmost" set of digits indicates the lowest module number connected to the highest level GMM. The "rightmost" set of digits indicates the lowest module number connected to the "level 3" GMM. If there are less than three levels of GMM's in this system, the memory storage unit module number connected to the lowest existing level GMM is repeated in the remaining positions as in GMM.sub.1a and GMM.sub.1b. Thus, the FWAR in each Global Memory Module identifies the memory module hierarchy available to its input requestor ports.

The FWAR is manually set at the time of installation. Neither the memory module numbers nor the FWAR register are changed unless the system is physically changed. FIG. 7 shows an example of how memory modules are identified in the system network. It should be noted that the memory module numbers in any given FWAR are in descending order from left to right, that is, in regard to the hierarchy of GMM's from the highest level 0 down to the lowest level 3. This is in conformity with rule (b) for setting the two digit memory storage module numbers.

In FIG. 2A, the Global Memory Module, GMM.sub.3, contains a Global Memory Control (GMC) 20 in order to service requests to memory, and also contains a Global System Control (GSC) 30 in order to service interprocessor control and communications, in addition to enabling the dynamic configuration of the computer systems involved and enabling use of the common memory facilities.

In the preferred embodiment, the Global Memory Module (GMM) 10 of FIG. 2A provides interface input requestor ports for four processors and memory hubs L, M, N, P for four sets of homogenous memory storage modules. It also provides a repeater port 25.sub.3 for memory bus C.sub.10 interface to a higher level Global Memory Module (GMM.sub.2) and a GSC control port 26.sub.3 to the Global System Control bus B.sub.10 for interprocessor communications.

The Global Memory Module 10 has two operating modes, M1 and M2. These are distinguished by the type of request being presented to the Global Memory Module.

Mode 1: If the request is for a memory access, the Global Memory Control 20 will perform the following steps:

(a) recognize a memory request;

(b) resolve priority in the event of simultaneous requests;

(c) dynamically connect a selected requestor to the repeater port 25.sub.3 if the requested memory address is not contained in this level of Global Memory;

(d) dynamically connect a selected requestor to the requested memory storage unit, FIG. 2A, and initiate the memory cycle if the address being selected is contained within this level of the Global Memory.

(e) parity check the requestor's address and also the address sent to memory;

(f) check write-data for errors and set the appropriate flags;

(g) check read-data for errors and set the appropriate flags;

(h) generate control signals and error signals to the requestor.

Mode 2: If the request is for a Scan Operation, the Global Memory Control 20 will perform the following steps:

(a) it will recognize a Scan request and verify that the unit type being addressed is Global Memory (Address Bits [19:4]=1011). Note: The statement [19:4] signifies that reference is made to the 4 bits 19, 18, 17, 16 and that these 4 bits start at 19;

(b) generate the control and error signals to the requestor.

The Global System Control 30 will:

(c) regenerate the error correction code and check the write-data presented during Scan-Out;

(d) correct single errors on write-data and set the error flags;

(e) perform the operation which is specified in the Scan Word;

(f) generate the error correction code on the message to the receiving unit and send (to the receiving processor) an external or alarm interrupt, if appropriate;

(g) generate the error correction code on the response from the receiving Global System Control, and store the code and the response in the Response Buffer, FIGS. 3, 16;

(h) return information from the Response Buffer on a Scan-In request if the Address Bits [15:1]=0;

(i) return information from the Message Buffer, FIGS. 3, 16, on a Scan-In request if the Address Bit [15:1]=1.

Global System Control Bit:

In FIGS. 1 and 2A there may be seen the global system control bus B.sub.10. This bus connects each one of the global memory modules to each one of the other global memory modules through the individual global system control adapter 26 in each Global Memory Module. The global system control bit is a digital signal which is passed between each of the global memory module cabinets and between each of the input ports within each global memory module cabinet. The global system control bit is used to determine control of the global system bus, since only one processor port, at any given time, can have control of the GSC bus B.sub.10.

The GMM cabinet identification number is seen in FIG. 5 as placed in the cabinet number register 28.sub.cr, which is part of the multi-cabinet adapter 26 which connects to the global system control bus B.sub.10.

The global memory module cabinet identification number (cabinet number) is used only to determine the order in which each of the GMM cabinets will receive the global system control bit. This bit is passed between each of the cabinets and then between the ports within each cabinet, in order to determine control of the global system bus. The GMM cabinet number consists of four binary weighted switches, which are set at the time of installation. The maximum number of GMM cabinets in an installation will generally be limited to 16 cabinets.

Processor Name Structure:

In order to establish the desired relationships among processors and global memories, and to be able to reconfigure the system by software, a processor name structure is established to define processors in this system but which bears no fixed relationship to the physical structure of the system. The processor name structure is an artificial structure which can be determined by the software system.

As can be seen in FIG. 3, the requestor port has a section designated PT for "processor port adapter", in which there resides a register, PNR, which is the "processor name register" and in which information can residue as to the "name" of a processor being used in the system network.

As seen in FIGS. 9, 8A, 8B, each processor is given a "name" according to the format: NQ.sub.1 Q.sub.2 Mask. These symbols constitute fields of data in which the entire processor name consists of 24 bits and in which the field N is the "system name", the field Q.sub.1 is the first qualifier of this system name, the field Q.sub.2 is the second qualifier of the system name, and the field MSK (mask) identifies one of the 12 possible processors within the system. In the preferred embodiment of the network, the design arranges for the capability of 12 possible processors in each system.

The first Q.sub.1 and the second Q.sub.2 qualifiers of the system name identify the "position" of a system in a three-level system hierarchy, FIG. 8A, which are interpreted as follows:

(a) if Q.sub.1 and Q.sub.2 are equal to 0: the system is at the first level (Top) of the hierarchy.

(b) if Q.sub.1 is not equal to 0 and Q.sub.2 is equal to 0: the system is at the second level (MID) of the hierarchy.

(c) if Q.sub.1 is not equal to 0 and Q.sub.2 is not equal to 0: the system is at the third level (BOT) of the hierarchy.

Thus, a system of processors can, as designated by the "processor name", by definition reside at the highest level (first level-top) of the hierarchy, or residue at the second level, or at the third level of the hierarchy; and the very name given to the processor thus indicates in what "system" the processor resides, what its position is in the system hierarchy, and the individual identification number of which one (of up to a possible 12 processors) may be designated.

FIG. 8A shows a schematic representation of how the processor name of FIG. 9 is used to define the "position" of a particular processor in a system, and also the "position" of the system in the system hierarchy. For example, a group of processors in the top level hierarchy are seen to have their Q.sub.1 and Q.sub.2 qualifiers all equal to 0, and each system will have a separate N (name symbol) identifier to distinguish that particular system from the other systems which are "equal" to it. In addition, each of the separate systems can identify up to 12 separate processors attached to that system by means of the 12 mask bits from 0-11. It will be noted that in the top hierarchy, there are three system names designated as 100, A00, and F00.

Likewise, in the middle hierarchy there are three system names designated A10, AB0 and AF0.

The bottom hierarchy of systems in FIG. 8 are seen to be designated as AB1, AB2, ABE, ABF.

As an example of use and understanding of the processor name, the processor named AB2-2 (processor having mask bit 2 in the system AB2) is the number 3 processor within system AB2. The system AB2 is "equal" to any other system which has ABx as its system name. Little x is any designation from 1 to F (hexadecimal notation 1-9, A-F).

Additionally, it should be noted that system AB2 is a slave to system AB0. System AB0 is "equal" to any other system which as Ax0 as its system name. Further, the system AB0 is "slave" to system A00.

The system A00 is "equal" to any other system that has x00 as its system name. The systems that have slave systems are defined as "master" systems. A system can be a "master" system, an "equal" system, and a "slave" system all at the same time. The system AB0 of FIG. 8A is just such a system.

Global Commands:

A "sending" processor will scan out a command word in order to execute certain operations upon the system. Such Global command words are used to:

(i) configure the system hierarchy;

(ii) to establish the memory module access restrictions;

(iii) to exercise control of one processor by another; and

(iv) to interrogate the global status of one processor by another processor.

A global command word can address the "receiving" processor either by means of (a) its processor identification PID or by (b) the processor name PNR, depending on the type of command word used. A command word consists of 60 bits and is scanned out of the "sending" processor to the Global Memory Module where it is placed in the Response Buffer of the processor port adapter (P.sub.T) associated with that particular processor. FIGS. 10A through 10I show the format of the command words scanned out from the sending processor. The bits of each Global Command Word are numbered 0 through 59. Bit 51 is always the parity bit and bits 50, 49, 48 are always the tag bits. Bits 47-40 represent the OP code.

The global system control function operates on the concept that, at any given time, only one processor (one processor port) can be a "sender" and that all other processors in the system are potential "receivers".

The Addressing of Processors and Systems in the Network:

The above mentioned examples of "coupled" systems illustrate the ways needed for systems and processors to address each other. A particular problem that arises within the system is that while the processors in a system may be functionally identical, they still may have different capabilities. This occurs because:

(a) the processors may not be all able to address the same set of peripheral devices, that is, only some processors are connected to the magnetic tape subsystem, for example;

(b) the processors may not have access to the same amount of local memory; and

(c) the ways in which the processors are connected to the Global Memory Modules may be different.

Unfortunately, there is no way in which processors within a system can be allocated a "single" hierarchical name or address which will cope with and handle these different requirements and which will enable a processor to refer by name to:

(i) both the subset processors that are connected to a particular GMM and;

(ii) the subset of processors connected to a particular peripheral exchange.

The solution which is adopted thus uses:

(i) one method of addressing for processors "within" systems; and

(ii) another method for addressing of processors "between" systems.

In the preferred embodiment of the invention, "systems" are allocated names which reflect the hierarchy to which they belong. For example, in FIG. 8A, the system ABE is the slave of system AB0 which in turn is the slave of the system A00.

A processor "within" a system is identified by a single bit in a processor mask, MSK, FIG. 9, which is then "concatenated" to the system "name". Thus, one processor in a system, say system AB0, can then address a subset of processors by specifying AB0 (MSK) where the bits of MSK, corresponding to each member of the subset, are set to 1; the address, for example, AB0 (11 . . . 11) will address "all" the processors in system AB0.

Again referring to FIG. 8A, there is illustrated a large installation together with the "naming" of the different systems and processors. Systems 100, A00 and F00 are "equal" systems and have slave systems A10, AB0, AF0. The system AB0 is then seen to have "slave" systems, such as AB1, AB2, ABE, and ABF.

Communication is permitted within systems, that is, between "equal" processors in a system and also along the lines which indicate the processor relationships of FIG. 8A. For example, the processors of system A00 can communicate only with 100 and F00 and can also communicate with its slave system AB0 which is seen as being "one level" down in the hierarchy. The processors of system AB0 can communicate with its master system A00 "one level" up and AB0 can also communicate with its slaves AB1, AB2, ABE, and ABF which are "one level" down.

It should be noted that a system such as ABE cannot directly communicate with the system A00 or 100 or F00, either for passing messages between programs or for transmitting control commands.

Within the given system, such as for example A00 or ABE, one processor can address another processor by specifying the appropriate bit in the name field; for example in the system F00, the processor number 1 (F00 [00-01]) sends a message to processor 2 by specifying address (F00[00-010]).

Now, a processor in the system A00, for example, can send a message to the system AB0 by specifying that "any processor may respond" by setting the processor bit mask to all "1's", for example, AB0 [11--11]. The A00 system may also select a specific processor in the system in the AB0 by setting, for example, only one bit in the processor bit mask, as AB0[00-01].

Global Memory Module Numbers and Processor Numbers:

The global memory module system network provides for a processor to be addressed by its "name" which is set by the program when the system is initialized. Additionally, processors can be addressed by "number". Thus, the identification of a processor and the means by which it can be addressed may be done in two ways, namely:

(i) the "name" of the processor;

(ii) the "number" of the processor.

Thus, as seen in FIG. 8A, the processor can be identified by the "name" which is a combination of the system name (N, Q.sub.1, Q.sub.2) plus the bit indication of a MASK which is shown in FIG. 9.

On the other hand, the processor "number" is fixed for each particular installation and is formed by concatenating the numbers of the GMM ports traversed by the path from the GMM of the highest memory address to the particular processor it is desired to get the number of.

Referring to FIG. 8B there is shown a hierarchy of global memory modules from the highest level (level 0) through level 1 and level 2. The processors in the network of FIG. 8B are given short system names, such as A001, A002 and B001, B002, etc. The chart at the bottom of FIG. 8B shows the relationship between the "short" system name for a processor, its full name in terms of the bit mask involved, and the "number" of the processor which would be organized according to the numbers of the GMM ports traversed by the path from the highest level GMM to the processor in question.

The ports of each GMM are numbered 1, 2, 4, 8. The "highest" port in the network memory structure is the most significant digit of the processor number.

FIG. 8B thus provides an example of the processor numbers and also the GMM numbers which are formed on this basis. For example, the "most global" GMM is numbered 0000. The first one of the level 1 GMM's in the hierarchy is numbered 1000 since that GMM connects to port 1 of the highest level GMM. Likewise, the level 1 GMM 8000 is numbered as such because it connects to port 8 of the highest level global memory module GMM 0000. At the level 2 of the hierarchy it will be seen that the leftmost GMM connects upward to port 1 of the level 1 GMM 1000 and thence to the level 0 GMM 0000 at port number 1. Hence, the leftmost GMM at level 2 is numbered 1100.

Likewise, it will be seen in FIG. 8B that the rightmost GMM 1800 of level 2 connects upward in the hierarchy to port 8 at level 1 and thence to port 1 at level 0; hence, the number 1800 is used as the number for the rightmost GMM.

Thus, FIG. 8B shows three columns entitled "name", "full name", and "number" to show the relationship between the "name" and the "number" of a processor. For example, the B 6800 processor having the short name B004 is connected to the second port of GMM 1800 of level 2. From the chart it will be seen that its full name is B00 (0-1000) and that its processor number is 1820 since the connection path down from the topmost GMM of the hierarchy is through ports 1, 8, 2 and since there are no further connections, the final digit becomes a zero to form 1820 as the number of the processor B004.

In FIG. 8B there is also illustrated a mix of different types of processors in which those marked P may be typically Burroughs B 6800 processors and those blocks marked D may be other types of processors, such as a data communications processor.

Since "systems" were defined as being a set of processors that are functionally equal from the point of view of providing some sort of service to a requestor, it will be seen that the group of processors of FIG. 8B, for example, such as A001 and A002, attached to the level 0 GMM form a "system" which is given the name A00.

Likewise, at level 2 we find the processors B001, B002 are attached to GMM 1100 are equal to processors BOO3 and BOO4 (attached to GMM 1800) to form a group or system of processors of relatively equal service and thus these four processors have been given the system name B00 with the last-numbered suffix to differentiate the different processors within the system B00.

Referring to FIG. 8B to the central schematic sketch therein, it is seen that the primary system A00 is "master" over system AB0 which then is "master" over system ABC and ABD.

Likewise, BOO is another system which is equal to system A00 and system BOO is "master" over system BCO.

Classes of Processor to Processor Communication:

The communications from one processor to another processor in the GMM system network use the global system control GSC 30 and are divided into the following four types of classes:

Class 1: Within (intra):

A "within" system communication command permits a processor to address processors in the same (equal) system.

Class 2: Upward:

An upward communication command permits a processor in a particular system to address processors in that system's master system (one level up).

Class 3: Downward:

A downward communication command permits a processor in a particular system to address processors in "slave" systems (one level down).

Class 4: Across:

An "across" systems communication command permits a processor in an "equal system" to address processors in other "equal systems", that is, a processor in system A00 (FIG. 8B) can send a message to system B00, and the GSC 30 (global system control) will select a processor based on the mask bit of the mask shown in FIG. 9.

Slave systems may also address "across" to other slaves of the same master; for example, a processor in system ABC (FIG. 8B) may address a processor in system ABD.

The "class" of communication that a processor can initiate depends on its position in the hierarchy of the system of which it is a member, and the type of communication that is specified. This is illustrated by the following Table I which shows the communication permitted by the systems AOO, ABO and ABC for the common HEYU command that transmits a message, and also the control command such as HALT, in reference to FIG. 8B.

                  TABLE I
    ______________________________________
                    Sender
    Command     Variant   A00      AB0     ABC
    ______________________________________
    HEYU        Within    A        A       A
                Up        D        A       A
                Down      A        A       D
                Across    A        A       A
    HALT        Within    A        A       A
                Up        D        D       D
                Down      A        A       D
                Across    D        D       D
    ______________________________________
     A = Allowed
     D = Denied

    ______________________________________
                Read Access Clocks
                              Cycle Clocks
    Function      Min        Max      Min   Max
    ______________________________________
    Read          5          7        7     12
    Clear/Write   --         --       --    10
    Read Modify Write
                  5          7        8     12
    ______________________________________

                  TABLE III
    ______________________________________
    SCAN-OUT WORD FORMAT
    ______________________________________
    Bits       59:8      Error Correction Code
               51:1      Word Parity Bit
               50:3      Tag Bits (000)
               47:6      Operation Code
               41:2      Variant Bits
               39:24     Information
               15:16     Receiver Address
    ______________________________________

                  TABLE IV
    ______________________________________
    RESPONSE WORD: TWO FORMATS
    Format 1: No transmission parity problem (2:1 = 0)
    ______________________________________
    Bits 59:8          Error Correction Code
         51:1          Word Parity Bit
         50:3          Tag Bits (000)
         47:6          Operation Code of the SCAN-OUT
         41:2          Variant Bits
         39:24         Information from Register Read
         15:1   On     Interrupt Pending
         14:1   On     WACR Set by Test and Set
         13:1   On     Time Out Waiting to Receive
         12:1   On     Receiver could not see Module 1