Concurrency-control method and apparatus in a database management system utilizing key-valued locking

Abstract

The concurrency-control mechanisms in a database-management system achieves high concurrency by using a lock-mode set larger than that conventionally employed for multi-granularity locking. In a system of key-valued locking in which locks on key-value ranges are acquired separately from the locks on the key values with which they are associated, the IX lock mode conventionally acquired on a range by update, insert, and delete operations is replaced with three separate lock modes respectively associated with those operations and invoked by them for range locking. In key-valued-locking systems in which ranges are locked commonly with the key-values associated with them, the mode set is further expanded so that each mode represents a different combination of range and key-value locks.

Claims

We claim:

1. For managing a database of accessible records that contain resources of first and second types, each of a plurality of the resources of the second type being uniquely associated with a respective resource of the first type, and for providing accesses to the resources in response to implementing instructions of user-requested transactions, a method comprising the steps of:

A) providing a lock manager characterized by a predetermined compatibility matrix that indicates, for a plurality of composite lock modes, which pairs of the composite lock modes are compatible with each other, the lock manager maintaining a lock table that identifies locked database resources and the composite lock modes in which they are locked, receiving lock requests that designate resources and composite lock modes in which those resources are to be locked, and responding to the lock requests by generating compatibility indications that indicate whether the lock modes designated by the lock requests are compatible in accordance with the compatibility matrix with the lock modes of locks acquired by other transactions on the database resources that the lock requests designate; and

B) executing a plurality of transaction routines for performing respective transactions, each transaction routine comprising instructions that implement database-access operations, associated with respective database accesses, that apply lock requests to the lock manager and perform the associated database accesses only if the compatibility indications generated by the lock manager in response to the lock requests are positive, the resource identifiers in the lock requests being so mapped from the resources of the first and second types that each resource of the second type is mapped to a resource identifier the same as that to which the resource of the first type associated therewith is mapped, the database-access operations implemented by the instructions of the plurality of transaction routines including a set of operations that includes a plurality of operations of which each requires locks both on at least one resource of the first type and on the resource of the second type with which it is associated, the lock requests applied to the lock manager by the instructions that implement such a transaction's operations requesting, in the absence of an already existing lock acquired by the given transaction, a composite lock whose mode is associated with constituent first-resource-type and second-resource-type lock modes whose compatibility sets within a set of constituent lock modes would be sufficient to maintain serializability if the constituent first-resource-type and second-resource-type lock modes individually locked the resources of the first and second resource types, respectively, on which that operation requires locks, a given composite lock mode being compatible with another composite lock mode if and only if the constituent first-resource-type and second-resource-type lock modes associated with the given composite lock mode are respectively compatible with the first-resource-type and second-resource-type lock modes associated with that other composite lock mode.

2. A method as defined in claim 1 wherein the resources of the first type are existing key values contained in respective accessible records and belonging to an ordered sequence of possible key values and the resources of the second type are key-value ranges, each of a plurality of which extends from the existing key value associated therewith to an existing key value in front of the associated key value.

3. In a method for managing a database of accessible records that contain respective existing key values belonging to an ordered sequence of possible key values divided into key-value ranges, each of a plurality of which extends from an existing key value uniquely associated therewith to an existing key value in front of the associated key value, and for providing accesses to the records in response to implementing instructions of user-requested transactions, which method comprises the steps of:

A) providing a lock manager characterized by a compatibility matrix that indicates, for a plurality of lock modes, which pairs of the lock modes are compatible with each other, the lock manager maintaining a lock table that identifies locked database resources and the modes in which they are locked, receiving lock requests that designate resource and lock modes in which those resources are to be locked, and responding to the lock requests by generating compatibility indications that indicate whether the lock modes designated by the lock requests are compatible, in accordance with the compatibility matrix, with the lock modes of locks acquired by other transactions on the database resources that the lock requests designate; and

B) executing a plurality of transaction routines for performing respective transactions, each transaction routine comprising instructions that implement database-access operations, associated with respective database accesses, that apply lock requests to the lock manager and perform the associated database accesses only if the compatibility indication generated by the lock manager in response to the lock requests is positive, the improvement wherein the compatibility matrix includes at least eight distinct lock modes.

4. For managing a database of accessible records that contain respective existing key values belonging to an ordered sequence of possible key values divided into key-value ranges, each of a plurality of which extends from an existing key value uniquely associated therewith to an existing key value in front of the associated key value, and for providing accesses to the records in response to implementing instructions of user-requested transactions, a method comprising the steps of:

A) providing a lock manager characterized by a predetermined compatibility matrix that indicates, for a plurality of lock modes, which pairs of the lock modes are compatible with each other, the compatibility matrix including delete-next range, insert-next-range, scan, and updated modes and indicating that the delete-next-range mode is incompatible with the insert-next range mode and the scan mode but compatible with the update mode which the compatibility matrix indicates to be incompatible with the scan mode, the lock manager maintaining a lock table that identifies locked database resources and the modes in which they are locked, receiving lock requests that designate resources and lock modes in which those resources are to be locked, and responding to the lock requests by generating compatibility indications that indicate whether the lock modes designated by the lock requests are compatible in accordance with the compatibility matrix with the lock modes of locks acquired by other transactions on the database resources that the lock requests designate; and

B) executing a plurality of transaction routines for performing respective transactions, each transaction routine comprising instructions that implement database-access operations, associated with respective database accesses, that apply lock requests to the lock manager and perform the associated database accesses only if the compatibility indications generated by the lock manager in response to the lock requests are positive, the resource-identifiers in the lock requests being so mapped from the key values and ranges that the resource identifier to which each range is mapped is the same as that to which the key value associated therewith is mapped, the database-access operations implemented by the instructions of the plurality of transaction routines including at least:

i) a scan operation that scans through key-value ranges, lock requests applied to the lock manager by the instructions that implement a given transaction's scan operations requesting, in the absence of an already existing lock acquired by the given transaction, locks of the scan mode and including resource identifiers that represent the key-value ranges through which the scan operation scans;

ii) a delete operation that deletes a record, one said lock request applied to the lock manager by the instructions that implement the delete operation in a given transaction requesting, in the absence of an already existing lock acquired by the given transaction, a lock of the delete-next-range mode and including a resource identifier that represents the key-value range immediately behind the range with which the key value of the record to be deleted is associated;

iii) an insert operation that inserts a record, one said lock request applied to the lock manager by the instructions that implement the insert operation in a given transaction requesting, in the absence of an already-existing lock acquired by the given transaction, a lock of the insert-next-range mode and including a resource identifier that, before insertion of the record to be inserted, represents the key-value range that encompasses the key value thereof; and

iv) an update operation that updates a record, one said lock request applied to the lock manager by the instructions that implement the update operation in a given transaction requesting, in the absence of an already-existing lock acquired by the given transaction, a lock of the update mode and including a resource identifier that represents the key-value range into which the key value of the record to be updated falls,

whereby one transaction's scan of, or insertion of a record into, a key range from which a record has been deleted by another uncommitted transaction can be prevented without preventing the one transaction from updating the record whose key value is associated with that key range.

5. A method as defined in claim 4 wherein:

A) the compatibility matrix includes a singleton-read mode and an update-scan mode;

B) the database-access operations implemented by the instructions of the plurality of transaction routines additionally include:

i) a singleton-read operation that reads a single record, one said lock request applied to the lock manager by the instructions that implement the singleton-read operation in a given transaction requesting, in the absence of an already-existing lock acquired by the given transaction, a lock of the singleton-read mode and including a resource identifier that represents the key value of the record to be read; and

ii) an update-scan operation that scans through key-value ranges and updates records having key values in those ranges, lock requests applied to the lock manager by the instructions that implement the update-scan operations requesting, in the absence of an already existing lock acquired by the same transaction, locks of the update-scan mode and including resource identifiers that represent the key-value ranges through which the update-scan operation scans;

C) the compatibility matrix includes at least five distinct lock modes IU, IIn, ID, S and SIX, whose compatibility combinations are as follows:

    ______________________________________
    IU            IIn    ID         S    SIX
    ______________________________________
    IU     X          X      X
    IIn    X          X
    ID     X
    S                                 X
    SIX
    ______________________________________

    ______________________________________
    MODES   IS-S   IU-X    IIn- IIn-X  ID-  S-   ID-X
    ______________________________________
    IS-S    X              X           X    X
    IU-X                   X           X
    IIn-    X      X       X    X
    IIn-X                  X
    ID-     X      X
    S-      X                               X
    ID-X
    ______________________________________

    ______________________________________
    IU            IIn    ID         S    SIX
    ______________________________________
    IU     X          X      X
    IIn    X          X
    ID     X
    S                                 X
    SIX
    ______________________________________

    ______________________________________
    MODES   IS-S   IU-X    IIn- IIn-X  ID-  S-   ID-X
    ______________________________________
    IS-S    X              X           X    X
    IU-X                   X           X
    IIn-    X      X       X    X
    IIn-X                  X
    ID-     X      X
    S-      X                               X
    ID-X
    ______________________________________

    ______________________________________
    IU            IIn    ID         S    SIX
    ______________________________________
    IU     X          X      X
    IIn    X          X
    ID     X
    S                                 X
    SIX
    ______________________________________

    ______________________________________
    MODES   IS-S   IU-X    IIn- IIn-X  ID-  S-   ID-X
    ______________________________________
    IS-S    X              X           X    X
    IU-X                   X           X
    IIn-    X      X       X    X
    IIn-X                  X
    ID-     X      X
    S-      X                               X
    ID-X
    ______________________________________

Description

BACKGROUND OF THE INVENTION

The present invention is directed to resource-management systems and in particular to the locking strategies that they employ.

A resource-management system is typically implemented in a computer, including its various types of storage apparatus, programmed with appropriate software. A common type of resource-managment system, namely, a database-management system ("DBMS"), can be thought of as operating in two phases, the first of which may be called an design phase and the second of which may be called an execution phase.

FIG. 1 represents the design phase. In this phase, the DBMS 10 provides a database designer with ways of defining the "structure" of the data, i.e., the manner in which their storage is organized, and of defining transactions in which the data thus stored will be accessed by ultimate users.

The first function, performed in what we can call a "metadata manager" 12, typically responds to structure-defining (data-definition) commands entered by the designer in a high-level database language such as SQL. A relational DBMS, for instance, may accept a data-definition instruction of the following form:

CREATE TABLE INCOME

NAME CHAR (20),

INC DECIMAL (10,2)

SSN CHAR (9);

Such a statement may establish that the database will include a relation, or table, called "income," in which each tuple, or record, includes three attributes, or fields, including a "NAME" field in the form of twenty characters, an "INC" (income) field of ten decimal digits with a decimal point two places from the right, and an "SSN" (social security number) field in the form of nine characters.

The database designer may use similar statements to define other tables that the database will include. He may further define one or more indexes, whose maintenance he knows will be valuable in the use of the database. For instance, he may employ a command such as:

CREATE INDEX ON INCOME (INC);

to create an index of the records ordered by the values of the respective records' income fields.

The result of such commands is to cause entries reflecting them in a database catalog 14 that the DBMS creates and maintains for use in its access operations. Note that none of the foregoing activities provides actual database contents; attributes have been named, but no values of those attributes have necessarily been entered.

We will assume for present purposes that it is the end users who supply the database contents by various manipulating transactions. However, it is the database designer who defines the types of transaction routines that the end users invoke for this purpose. (Actually, a separate application programmer may perform this task. From here on, we will refer collectively to the various personnel, other than the end users, who operate on the database as the "definer.") That is, the definer generates transaction routines, which the DBMS's run-time system will perform in response to requests from end users. To produce these routines, the definer employs a group of DBMS modules to which we will refer as a "query compiler" 16.

The definer may write the transaction definition in a general-purpose language such as PL/1, but he will typically embed in the PL/1 program instructions written in the high-level database language, such as SQL, to which the DBMS responds. This can be done in PL/1, for instance, by preceding, say, SQL commands with "EXEC SQL," which is a signal that the overall PL/1 source code will have to be submitted to a precompiler before PL/1 compilation so as to strip out the SQL statements and replace them with appropriate subroutine calls. The stripped-out SQL statements would be presented to the query compiler, which would compile them into instructions that call upon the services of various operative modules within the DBMS's run-time supervisor.

A transaction-defining statement in the high-level database language may be in the form:

START TRANSACTION

SELECT INC

FROM INCOME

WHERE SSN="123456789"

COMMIT WORK;

When the routine specified by such a statement runs, it searches the database for a record in which the SSN (Social Security Number) field is 123 45 6789 and fetches the value in the income field of that record.

To compile this command, the query compiler 16 consults the database catalog, which contains the definer's definitions of the data organization, such as that "INCOME" is a relation and INC and SSN are two of its attributes. It also determines the best manner in which to search for the indicated information, and it employs the index information in order to do this. If, for instance, the definer has required that an index ordered by social-security number be provided, then the query compiler produces a transaction routine that accesses the record by way of that index. In such a situation, the SSN attribute is the "key" by which the system finds the desired record (or records).

It should be emphasized at this point that, although we depict the query compiler as operating in the design phase rather than in the execution phase, many of what will be described below as features of the query compiler are only implicit in most DBMS implementations until the execution phase is in progress. In particular, the typical query compiler places in the transaction routine 17 calls to operation subroutines chosen from a subroutine set resident only during the execution phase and commonly called by most transaction routines. Since the contents of these subroutines are all part of the translation from transaction definition to implementation instructions, we consider them features of the query compiler, and the remaining discussion will not distinguish between the functions that the query compiler performs directly during the design phase and those that its design-phase actions perform only indirectly, by calling existing subroutines resident during the execution phase.

Among the capabilities of many DBMSs is that of maintaining what is known as "serializability." In writing transaction definitions, the database definer is defining what will happen when an end user submits a request to the system. One resulting transaction routine, for instance, might be invoked by a user at an automatic teller machine to transfer money from one account to another. Another might be invoked by a bank executive from a management-information-system terminal to request the total of all account balances for a given branch of the bank. In each case, the end user submits his transaction request to a run-time supervisor 18 (FIG. 2), which calls the specified transaction routine. The transaction routine obtains records from a storage medium 19, such as a magnetic disk or cached copies of its contents, through the operation of a buffer manager 20. The actual central-processor time involved in each of these transactions may be very small, but the time required between the beginning and end of the transaction may be considerable, since time is usually required to await data from storage facilities, inputs from the human user, etc. In the operation of most large databases, it is therefore important that the central processor be freed to perform operations of other transactions between the individual operations of any single transaction. But this interleaving can cause problems if steps are not taken to prevent them.

For example, the ATM user's transfer of money from one account to another may actually be implemented in a number of operations, which may be interspersed with inputs from the user. The transfer involves the separate steps of removing money from one account and adding it to another. Similarly, the bank executive's transaction of obtaining account totals actually includes a number of separate operations, each of which involves reading an account balance and adding it to a running total. If these operations are interleaved, the account-totaling operation may copy the balance from the ATM user's first account before the transfer and from the second account after the transfer, and this would indicate to the executive that the bank total is greater than it actually is.

This would not result, of course, if the two transactions occurred serially. One of the functions of most large DBMSs is therefore to perform transactions in such a way that concurrently performed sets of them are serializable, i.e., that their results are the same as that of a series of non-concurrent transactions, without requiring actual serial transaction ordering. To this end, the query compiler usually causes the transaction routine to include certain operations that invoke the services of a DBMS module known as a "lock manager" 21, which is active in the execution phase depicted in FIG. 2 (and actually in the design phase, too, for purposes not relevant here) and maintains a lock table 22, whose contents indicate which "resources" are currently involved in transactions in such a manner that certain operations on them by other transactions must be postponed until the previous transactions have been completed. That is, if a transaction performed by such a routine includes an access to a certain resource, it will also request that the lock manager post a lock in the lock table identifying the designated resource as one to which access is restricted. (We use the more-general term resources instead of records for reasons that will become apparent. Until those reasons do become apparent, however, little harm results from reading "records" for "resources.")

When an end user invokes a transaction routine in the execution phase, that routine will request that the lock manager post a lock on the required resource. The lock request includes a "resource ID," which typically is, say, a hashed or otherwise encoded version of the key value, and an indication of the requested lock's "mode." (As will be explained below, locks can differ in their degrees of restrictiveness.) In response, the lock manager will consult the lock table to determine whether any lock has been posted for that resource ID. If so, it will consult its "compatibility matrix" to determine whether the requested lock is compatible with the already-posted lock, and the transaction that has requested the lock will be at least temporarily prevented from proceeding if the lock modes are not compatible. Otherwise, the lock manager will produce a positive compatibility indication and (except in certain "instant lock" cases that will be described below) enter a lock in the lock table, thereby restricting other transactions' access to the locked resource.

Of particular interest in the present context is what is known as "range locking." The serializability of some types of transactions is not assured by simply locking the several records to which they require access. An example of such a transaction is one that includes a scan operation, which accesses all records in which a certain attribute's values are within a specified range.

For instance, one transaction may be to find the total income within an income range. A concurrent transaction may be to insert records for a group of new taxpayers who first have reportable income within a given period. It is desirable that the computed total reflect either all or none of the new taxpayers, but individual record locking does not insure this result. If the insertion transaction inserts one record in a part of this range that the totaling transaction has already searched and another in a part that it has not, then neither transaction will encounter the other's locks, and the total-computing transaction's result will be erroneous. This results from the "phantom" records that were inserted into a range after that range's original occupants were locked.

To remedy this problem, database management systems must lock entities other than just records. For example, a DBMS may lock a whole file when a scan operation is performed in it. But such an expedient can produce a considerable reduction in the system's achievable concurrency, i.e., in the degree to which operations of separate transactions can be interleaved. Alternatively, therefore, DBMSs sometimes employ range locking, which treats not only individual key values but also key-value ranges as resources to be listed in the lock table. Therefore, when a transaction routine includes an operation directed to all records within a certain target key range, it causes the lock manager to post locks directed not only to the individual key values to which the transaction will obtain access but also to a set of lockable ranges that covers the target range. And, if another operation is directed to a specific key value, that transaction not only requests a lock on that key value but also has the lock manager check for locks on any lockable range into which that key value falls. This latter function is performed by additionally identifying any such range resource and requesting a lock on it.

The routine for inserting records of new taxpayers, for instance, would identify all ranges into which the records that it intends to insert fall, and it would check the lock manager to determine whether there are locks on these ranges. Since the total-computing transaction will have acquired locks on ranges into which the new taxpayer records would otherwise have been inserted, the record-insertion transaction is postponed until the total-computation transaction has been completed, and serializability is thereby maintained.

Note that the use of ranges as lockable resources does not change the lock manager. A conventional lock manager is quite simple. It does not have to "know" what kind of resource a resource ID represents. It needs to "know" only the compatibility matrix and the resource IDs against which it has already posted locks.

We digress at this point to note that locking a key value or range thereof is not the same as locking the record or records that the key value or range designates. The record identified by the name-attribute value John Doe may be the same as that identified by the social-security-number-attribute value 123 45 6789. But the key-value locking performed by an operation that uses John Doe as its key value and derives the resource ID from that value when it acquires a lock on it does not by itself restrict access to that record by an operation that requests a lock on the social-security number. Typically, of course, the operation will also perform further locking, such as record-ID locking, which will cause access to that record from other paths to be restricted. Indeed, one of the keys may in essence be the record ID. But we are not concerned with that here. Here we are concerned only with locking on the basis of a given key or range thereof, so we will refer to key-value or range locking, which may or may not be accompanied by or equivalent to record locking.

Although it is better than locking the whole file, locking ranges of key values can itself reduce concurrency significantly. To lessen this concurrency reduction, DBMSs employ more than one mode of lock, each lock mode differing from the others in restrictiveness. This restrictiveness variation is useful even if the DBMS does not use range locking. For example, a lock acquired by a transaction as a result of an operation that only reads records does not need to prevent other transactions from reading those same records, but a lock resulting from a write operation does. In recognition of this fact, a simple, two-mode locking system may employ lock-table entries that include an indication not only of whether a lock is in place but also of whether the lock is a share-mode ("S") lock, requested for read operations, or an exclusive-mode ("X") lock, requested by record-modifying operations. A transaction requesting an S lock will be deterred only by X locks on the target resource, not by S locks, while transactions that request X locks will be deterred by locks of both types.

Database systems that employ range locking often obtain additional concurrency by means of "multi-granularity locking," or MGL, which conventionally employs five lock modes, as FIGS. 3, 4, and 5 illustrate. FIG. 3 is a lock-mode table, which indicates the types of locks acquired by transactions in accordance with one application of the MGL scheme for the different types of database-access operations of which they may be comprised. In addition to the simple "covering" lock modes S and X, there are three "intention" lock modes, designated IS, IX, and SIX, whose purpose is to indicate, with respect to a key range, that a further, covering lock will be acquired on a key value that falls within that range. (Actually, the SIX mode, as will be seen below, is both a covering lock and an intention lock.)

FIG. 4 is an exemplary lock table that will be employed to explain intention locking. In the example, we will assume that one of the attributes in a relation defined for a given database is labeled "INC," for "income." That is, if the relation is thought of as a table, each row of the II table represents a taxpayer, and one of the columns represents those taxpayers' incomes. Let us further assume that the database designer has required (not necessarily in principle but nearly unavoidably in practice) that an index be maintained of incomes and that he has further specified that the DBMS is to treat certain predetermined ranges of incomes as lockable resources. The reason for this may be that a purpose of the database is to support studies of characteristics of various predetermined income ranges.

Now, suppose that a user calls for performance of a transaction routine for totalling all incomes within the $30,000-$70,000 income range and that the database designer has specified two predetermined ranges of $30,000 to $50,000 and $50,000 to $70,000. The DBMS's run-time supervisor responds to this request by running the requested transaction routine and assigning this instance of that routine's operation the transaction name T.sub.1. Since transaction T.sub.1 will need to read all records within the two ranges but not to update any of them, it requests S locks on the two ranges, as the first row of the FIG. 4 table indicates. The S lock is a covering lock, since it implicitly locks each key value in the range. But the transaction does not explicitly place a separate lock on each key value--i.e., each income level--that it finds, since the query compiler will have so designed any transaction routine that similarly searches for records by income as to cause the lock manager to check the income range into which the indicated income falls.

The "read scan" entry in the FIG. 3 mode table reflects this lock-mode selection. It shows that, upon a scan-type read operation, i.e., one which requests all records within a given range, a lock of the S type is acquired on the range or ranges involved but not on the individual key values.

As FIG. 5 indicates, lock mode S is compatible with only two lock modes, IS and S. The latter compatibility is exemplified by a concurrent transaction ("T.sub.2 "). Transaction T.sub.2 requires access to the S-locked ranges, but only to read them, so it requests S-mode locks of those ranges. The lock manager, which implements the compatibility table of FIG. 5, informs T.sub.2 that the requested lock mode is compatible with the existing (S-mode) locks on those ranges in its lock table. It therefore posts the lock, as the FIG. 4 lock table indicates in the T.sub.2 row.

To observe compatibility of the S mode with the IS mode, let us consider a transaction ("T.sub.3 ") whose purpose is to fetch, say, the identity of the taxpayer whose income is $39,547. This involves a "singleton read" operation, i.e., one that targets an individual key value, not a range, although the key value may fall within a range. As FIG. 3 indicates, the query compiler requires such an operation's transaction to request not only an S lock on the actual key value to which this operation is directed but also an IS lock on any range in which that key values falls. Accordingly, as FIG. 1 shows, T.sub.3 acquires an S lock on income value $39,547 and an IS lock on income range $30,000 to $50,000. The lock manager grants this lock, even though locks have already been acquired on that income range, because the lock manager's compatibility matrix (FIG. 5) indicates compatibility between the IS and S locks.

The IS lock is not a covering lock: the operation that caused it is not directed to the range against which the IS lock is posted. Instead, it is an intention lock: it indicates the intention to acquire a lock on a different resource, namely, a key value, that the range includes (or, for some resources, otherwise overlaps). Its purpose is to prevent another transaction from acquiring a covering lock on the range that would be inconsistent with the first transaction's access to the included (or overlapped) resource.

The difference between the IS and S locks becomes apparent when one considers a new transaction T.sub.4, whose purpose is to correct the income value for the taxpayer whose income is $41,290. This is a "singleton update," and FIG. 3 indicates that the involved range and key value must be locked in modes IX and X, respectively. FIG. 4 shows that no previous transaction has obtained a lock on the key value of $41,290. But that value falls within the $30,000 to $50,000 range, on which previous transactions have already acquired locks, and the mode, S, of two of those locks is incompatible with the IX-mode lock that transaction T.sub.4 requests on that range. Transaction T.sub.4 therefore cannot proceed until the S-lock-holding transactions T.sub.1 and T.sub.2 have ended, either by aborting or by running to completion, i.e., in database parlance, by "committing."

Now, let us suppose that transactions T.sub.1 and T.sub.2 commit. A committing transaction notifies the lock manager, which responds by removing the transaction's entries. FIG. 4 uses checkmarks to represent such removals for transactions T.sub.1 and T.sub.2. The only lock remaining on the $30,000 to $50,000 range after transactions T.sub.1 and T.sub.2 commit is therefore the IS-mode lock acquired by transaction T.sub.3. Remember that transaction T.sub.3 requested only an IS-mode lock on the range, since it did not need to read all of the range but only needed to indicate that it would be reading something within the range. Thus, as FIG. 3 indicates, the IS mode is compatible with the IX mode requested by transaction T.sub.4 on the $30,000 to $50,000 range. After transactions T.sub.1 and T.sub.2 have committed, therefore, transaction T.sub.4 can acquire the locks that it requires before it will proceed.

Note that transaction T.sub.4 acquires a different, X-mode lock on the particular key value, $41,290, to which its singleton update operation is directed. This mode is incompatible with the S-mode lock acquired by T.sub.3, but there is no interference, because the two transactions' S- and X-mode locks are directed to different resources. If T.sub.4 's update operation were instead directed to the income value $39,547, on which transaction T.sub.3 has acquired a lock, the requested locks would not have been granted to T.sub.4 ; if this were not so, T.sub.4 would be allowed, contrary to the serializability requirement, to modify a record that transaction T.sub.3 is reading.

The remaining lock mode in FIG. 5, SIX, has a compatibility combination that is the intersection of those of the S and IX modes: it is compatible with IS, with which S and IX are compatible, but it is not compatible with IX or S, since IX is not compatible with the latter and S is not compatible with the former. This is the mode requested by an "update scan" operation, which scans all records within a target range and updates any that meet a predetermined criterion.

For instance, if the records have as an additional attribute the name of the taxpayer's state, a transaction may be to correct a reporting anomaly by adding a fixed amount to the income field of each Kentucky taxpayer whose income falls within an income range of $35,000 through $60,000. Although the particular requested range is not among the resources that the database has been arranged to lock, the required serializability can be insured by locking the two illustrated ranges, which together include the requested range. This operation is in essence both a read scan and a series of singleton updates. The query compiler therefore causes such transaction routines to request that the ranges involved be locked in the SIX mode, which both prevents all transactions prevented by the S-mode lock and prevents all transactions prevented by the IX-mode lock. As FIG. 3 shows, such transactions additionally request X-mode locks on the individual key values that are in fact updated.

To complete our discussion of this particular type of range locking, we comment on two further details. The first is that the IS-mode range lock in the first row in FIG. 3 is only implicit in some implementations: in those implementations, the transaction does not actually request this lock and the lock manager does not post it. As reference to the compatibility matrix of FIG. 5 reveals, the only locking mode with which mode IS is incompatible is the X mode, which, as FIG. 3 shows, is ordinarily requested only for individual key values, not for ranges. Therefore, so long as the query compiler does not produce transaction routines that request X locks on ranges, a singleton read transaction does not need to acquire a range lock. But some MGL systems may save lock overhead by, say, locking the range in the X mode for insert and delete operations and thereby avoiding the need to acquire a key-value lock. In such arrangements, the IS lock mode would have to be explicit.

Incidentally, if the use of the X mode is to be restricted, as it is in FIG. 3, to key values rather than ranges, then it can be replaced with the SIX mode, which is incompatible with the only other mode used for key values, namely, the S mode.

The second detail is that, although FIG. 3 primarily describes the action of the query compiler and the routines that it generates, while FIG. 5 characterizes the lock manager, certain aspects of FIG. 3 can instead be implemented by the lock manager. Specifically, the first range-lock entries in FIG. 3 for singleton updates, inserts, and deletes are all IX. This is the lock mode that a transaction requests if that same transaction has not previously obtained an S-mode or SIX-mode lock on the same resource. If it has obtained such a lock, on the other hand, it acquires an SIX-mode lock instead of an IX-mode lock. It can be the transaction routine's responsibility to make this substitution, but one usually would incorporate this function in the lock manager instead.

It can be appreciated that range locking of this type eliminates the phantom problem yet affords considerable concurrency. As the last example indicates, however, the use of such definer-specified ranges is somewhat inflexible. Specifically, the target income range mentioned above for the Kentucky anomaly did not fit the predetermined ranges very closely, and locking enough predetermined ranges to cover the target range resulted in less concurrency than would have been possible if there had been a closer fit. Additionally, the approach to MGL just described requires considerable locking overhead, since it requires that two resources, namely, a key range and an individual key value, be separately locked for most operations.

A system that deals with these problems is the ARIES/KVL system described in Mohan, "ARIES/KVL: A Key-Value Locking Method for Concurrency Control of Multiaction Transactions Operating on B-TREE Indexes," Proc. Very Large Databases Conference, Brisbane, Australia (August 1990). This system dynamically re-defines key-value ranges in accordance with the current population of key values. Specifically, the system maintains a key-value-ordered index, and the possible lockable ranges are the ranges between each pair of successive key values that currently exist in the index. That is, if the existing key values are k.sub.1, k.sub.2, . . . , k.sub.i, . . . such that k.sub.i <k.sub.i+1, then the ranges are the disjoint semi-open intervals (k.sub.i, k.sub.i+1 ], and each such range is identified by the upper bounding key value. Equivalently, of course, the range could be identified instead by the lower attribute value, in which case the range would be open at the top rather than at the bottom. Since locking granularity is finer if the ranges are disjoint, it is preferable for one end of the interval to be open. A lock-request resource identifier in this system identifies both a key value and the ranges that the key value represents. This enables a transaction to lock with a single lock request both a range and the key value that represents it.

FIG. 6 lists the lock modes required in the ARIES/KVL system for the various operations that a transaction might include. Unlike FIG. 3, FIG. 6 includes only one column for the first four operations listed, because an ARIES/KVL operation does not separately lock key values and key ranges. The locks listed in FIG. 6 are roughly the more restrictive of the range and key locks required by the system of FIG. 3 for the same operations. FIG. 6 does include a second column for insert and delete operations, but this does not indicate that key values and ranges are locked separately for these operations. Instead, it represents a separate lock on what will be described below as the "next" key value/range. This is necessitated by the fact that inserting and deleting records in a KVL system divides or extends existing key ranges.

Suppose that a record having a key value of k.sub.i is to be deleted. Clearly, one transaction's deletion of a record that has been written, read, or inserted by a second uncommitted transaction potentially compromises the serializability of those transactions. Therefore, an operation that deletes k.sub.i should not be permitted if another transaction has any lock on k.sub.i, regardless of the mode of that lock. A deleting transaction accordingly tests for any such lock by requesting an X-mode lock on k.sub.i. FIG. 6 includes the "instant" notation to indicate that the transaction need only request a compatibility indication and not actually acquire the lock, for a reason that will presently become apparent.

As FIG. 6 shows, the deleting transaction requests a lock not only on the targeted key value/range but also on the "next" key value/range, i.e., on key value/range k.sub.i+1, which includes the range previously represented by the deleted key value k.sub.i. Since that range has now been modified, no access to it should be permitted. The lock acquired on range k.sub.i+1 is therefore an X-mode lock.

For an insertion, on the other hand, the requested locks are not so restrictive. Let us suppose, for instance, that an inserting transaction is to insert a new key value k.sub.i ' between k.sub.i and k.sub.i+1. This will reduce the width of the range previously represented by key value k.sub.i+1, so range k.sub.i+1 must be checked for locks. For this purpose, ARIES/KVL uses an instant lock in the IX mode if the inserting transaction has not previously locked that key/range. Clearly, if that range had been, say, scanned by another uncommitted transaction, as indicated by an S, X, or SIX lock, that range should not be modified by inserting a new record into it. Testing by means of an IX-mode lock prevents this. However, there is no reason why the k.sub.i ' record cannot be inserted by one transaction just because another uncommitted transaction has previously inserted the k.sub.i+1 record, as indicated by a previously existing IX-mode lock. Since the requested IX-mode lock is compatible with an IX-mode lock, such an insert in front of another insert can occur. (In front of and behind are defined by the direction in which key-valued ranges extend: a range extends in front of the key value that represents it.) The inserting transaction requests only an instant lock on k.sub.i+1 because there is no reason why one transaction's insertion of k.sub.i ' should prevent another transaction's access to k.sub.i+1.

It is now apparent why the current-key-value/range lock required by a deleting transaction can be instant: subsequent insertion of a record having the deleted key value is prevented by the delete operation's next-key-value/range lock, which is incompatible with the next-key-value/range lock requested by the insert transaction.

Remember that the lock manager tests for compatibility only with existing locks of other transactions; if an inserting transaction has itself previously acquired an S, SIX, or X lock on the next key value k.sub.i+1, the lock manager does not produce an incompatibility indication even though the IX lock requested for that key is incompatible with those modes. Yet, because an insertion divides the range previously identified by k.sub.i+1, on which the requesting transaction has acquired such locks, provision must be made to insure that the entire previous range remains locked at least as restrictively after the insertion as it was before. Instead of IX, therefore, the lock on inserted key value k.sub.i is in mode X if the locking transaction has previously acquired an S-, SIX-, or X-mode lock on the next key value k.sub.i+1.

As was stated above, the ARIES/KVL system differs from the previously described MGL system. Instead of employing a fixed set of lockable ranges that is independent of the current population of existing key values, it dynamically so redefines the lockable ranges as key values are added and removed so that the currently existing key values bound the currently lockable ranges. This contributes to concurrency, because it tends to result in ranges whose granularity more nearly equals the required granularity.

SUMMARY OF THE INVENTION

We have recognized, however, that the ARIES/KVL system additionally employs a feature that tends to reduce concurrency. Specifically, the resource identifiers that it employs in its lock requests do not distinguish between key values and the ranges that they represent, so a whole range can sometimes be locked when it is only necessary to lock a single key value. In so doing, ARIES/KVL system incurs a concurrency penalty. But the range-definition approach used in ARIES/KVL, which we will call "key-value locking" ("KVL"), can be implemented in a system that, like the other MGL approach described above, locks key values and ranges separately. Such a system would yield greater concurrency.

We have further recognized that concurrency can be increased even over that obtainable in such a system, and we have achieved this additional concurrency by using a different set of lock modes, a set in which the number of modes is great enough that all the concurrency achievable in a KVL system by the conventional MGL mode set is maintained, while the modes of the locks on ranges required by update and scan operations, and on the "next" ranges by insert and delete operations, all differ from one another in transactions that have obtained no previous locks on those resources. With such a set of lock modes, the mode of the lock obtained on the next range by a delete operation can be incompatible with that of an insert operation's next-range lock and still be compatible with that of the range lock acquired by an update operation, with which that of the lock acquired by a scan operation must be incompatible.

This yields concurrency that the conventional MGL mode set does not: it permits updating of a record whose key represents a range that has been widened by an uncommitted transaction's delete operation. We have found that this result can be achieved by replacing the conventional IX mode with three separate modes, which we call IU, IIn, and ID, and using them in place of the IX mode on update, insert, and delete operations, respectively.

Moreover, by adding still further lock modes, this concurrency can be obtained even in systems in which lock requests employ identical resource identifiers for key values and the ranges they represent. By adding enough lock modes to achieve certain further compatibility combinations, which will be set out in more detail below, one can enable such systems to perform certain operations that the ARIES/KVL system prohibits. Specifically, embodiments of the present invention can delete or insert records whose key values immediately precede those of records read or updated by uncommitted transactions, and they can read or update records whose key values represent ranges divided or extended by operations of uncommitted transactions. We have been able to achieve these combinations by using a total of eight lock modes.

BRIEF DESCRIPTION OF THE DRAWINGS

These and further features of the invention are described below by reference to the accompanying drawings, in which:

FIG. 1 is a block diagram that depicts parts of a DBMS that a definer uses during database design;

FIG. 2 is a block diagram of parts of a DBMS employed to respond to requests from an ultimate user;

FIG. 3 is a lock-mode table that characterizes a type of prior-art DBMS query compiler;

FIG. 4 depicts an exemplary lock table maintained by a lock manager;

FIG. 5 is a compatibility table that characterizes certain prior-art lock managers;

FIG. 6 is a lock-mode table that characterizes the query compiler of another prior-art DBMS;

FIG. 7 is a compatibility table that characterizes a lock manager employed in one embodiment of the present invention;

FIG. 8 is a lock-mode table that characterizes a query compiler employed in the same embodiment;

FIG. 9 is a compatibility table that characterizes the lock manager of another embodiment of the present invention;

FIG. 10 is a lock-mode table that characterizes the query compiler of the same embodiment; and

FIG. 11 is a table of the concurrencies that the teachings of the present invention enable a DBMS to achieve; and

FIG. 12 is a flow diagram illustrating the method of the present invention.

DETAILED DESCRIPTION OF ILLUSTRATIVE EMBODIMENTS

The compatibility matrix of FIG. 7 and lock-mode table of FIG. 8 characterize the lock manager and query compiler of one embodiment of the present invention. Before we proceed with a description of this new DBMS, however, we should make more explicit the aspect of the locking process to which the present invention is directed. The following discussion therefore explains that the present invention is directed to the range and key locking that occur in a single path employed to access the database.

A database-management system provides access to a database by requesting the file manager in its host computer system to add, retrieve, or replace records whose storage on computer disks or other storage media the file manager (indirectly) controls. The DBMS specifies these records by respective unique record IDs. An operation in a transaction routine, on the other hand, typically specifies desired records by means of a key. In the case of a primary key, the key is an attribute (or group of attributes) that is essentially the same as the record ID in the sense that records whose key values are adjacent are clustered in close proximity. But a key may instead be an attribute, or combination of them, whose corresponding records are physically scattered.

If the system is to avoid a linear search when a key other than the record ID is used, the database designer will have to have directed that an index on the key be maintained, i.e., that the system maintain a file in which each entry has key field and a pointer field and in which the entries are ordered by the value in the key field and hence clustered. For instance, the definer may require that the system keep a taxpayer-name index, in which the key field in each record contains a name and the pointer field in that record typically contains a pointer to the record in the database proper that contains that name. In principle, however, the pointer can be the value of a key in a further index, which will also have to be consulted to find the target record.

An operation thus can in principle proceed through a number of keys, including the record identifier, and it could-acquire another lock for each new key. Moreover, most databases provide more than one such path; i.e., they allow different sequences of one or more keys to be used for searching. The teachings that follow can be applied to any leg of any path.

It should also be understood that, although the description of the invention will refer only to the key values (and ranges thereof) as the locked resource, the invention is applicable to any lockable resource of which a key value or key range is a component. For example, one can conceive of a DBMS in which the resource identifier used in a lock request can differ in accordance with the source operation's target attribute: for an operation that is to access target attribute A in the record identified by key value k, the system may use a resource identifier different from the one used for a request from an operation that is to access a different target attribute B in the record identified by the same key value. In such a case, the resource identifier used in the lock request would be determined by the combination of key value and attribute, but range locking would still be possible. Therefore, although the following description will not separately consider systems that distinguish locked resource not only by key value but also by target attribute, the teachings of the present invention are applicable to such systems, too.

It should be further understood that the term key has a broader meaning in the context of the present invention than it does in some other contexts. Clearly, an attribute or attribute combination has usefulness as a key if its values can be ordered and thus used for non-linear searching, and its use in the literature has obviously resulted from keys used in indexes for this purpose. But the term key has sometimes been limited to an attribute or combination thereof whose values uniquely identify records in a relation. Use-of the present invention is not restricted to unique keys, so we do not so limit that term here. However, statements herein concerning maximum possible concurrency are based on the assumption that the keys are unique; for non-unique keys, it may be possible to modify the illustrated embodiments so as to achieve even greater concurrency.

Moreover, a key as used in the present context need not be limited to those derived from attributes or used in an index. Not all scans employ indexes, because not all attributes are indexed, and not all predicates controlling a scan can make effective use of an index. For these scans, one needs to search through an entire file. These scans may need to guard against phantoms, just as index scans do. For this purpose, the key from which a lock request's resource identifier is derived could be the record ID, and a range lock on that ID would be interpreted as locking the thus-identified record and the entries physically disposed between it and its immediate physical predecessor. Such a key would not necessarily be related to any attribute--the file may be a heap file, for instance--but it could clearly be used for range locking.

Finally, it should be recognized that, although the present invention is directed to key-valued locking, it is not restricted to systems in which ranges are defined a they are in the ARIES/KVL system. As was stated above, key-valued locking employs ranges that are bounded by successive existing key values. But it is possible for a KVL system to use additional values, too, to define key-value ranges. To avoid ranges that bridge index pages, for instance, a KVL system may additionally use index-page boundaries to define ranges, as is disclosed in our commonly assigned U.S. patent application entitled Key-Range Locking with Index Trees and filed on even date herewith, which we hereby incorporate by reference.

Furthermore, ranges in a KVL system may in fact overlap even though they are defined by successive key values. The same Lomet et al. application discloses a method of using key-valued locking with "bounded disorder" indexes, in which the index trees' leaf nodes consist of pluralities