Transaction clash management in a disconnectable computer and network5878434Abstract A method and apparatus are disclosed for detecting and handling clashes that may occur when transactions performed on disconnected replicas of a database are merged after the computers carrying the replicas are reconnected. A variety of clashes are addressed, including those which arise from inconsistent add, remove, move, and modify operations. Transient clashes that would resolve themselves without significant intervention are distinguished from persistent clashes. Log patching, regression, key modification, duplication, and a variety of other tools for handling the clashes are described. Claims What is claimed and desired to be secured by patent is: Description FIELD OF THE INVENTION
__________________________________________________________________________
CLASS ha.sub.-- server
SUPERCLASS
ndr.sub.-- dodb.sub.-- object.sub.-- header;
PARENT ndr.sub.-- dodb.sub.-- database;
PROPERTY NDR.sub.-- OS.sub.-- CLASS.sub.-- FLAG.sub.-- FULLY.sub.--
REPLICATED;
ATTRIBUTE
{
ha.sub.-- server.sub.-- name
server.sub.-- name
PROPERTY NDR.sub.-- OS.sub.-- ATTR.sub.-- FLAG.sub.-- SIBLING.sub.--
KEY;
}
}
CONSTANT
HA.sub.-- VOLUME.sub.-- NAME.sub.-- MAX=32;
DATATYPE
ha.sub.-- volume.sub.-- name
STRING HA.sub.-- VOLUME.sub.-- NAME.sub.-- MAX;
DATATYPE
ha.sub.-- volume.sub.-- id
BYTE;
A volume has a name, which must be unique within the
server and can be used as the root component of a path name:
CLASS ha.sub.-- volume
{
SUPERCLASS
ndr.sub.-- dodb.sub.-- object.sub.-- header;
PARENT ha.sub.-- server;
PROPERTY NDR.sub.-- OS.sub.-- CLASS.sub.-- FLAG.sub.-- NAMESPACE.sub.--
ROOT;
ATTRIBUTE
{
ha.sub.-- volume.sub.-- name
volume.sub.-- name
PROPERTY NDR.sub.-- OS.sub.-- ATTR.sub.-- FLAG.sub.-- SIBLING.sub.-
- KEY .linevert split.
NDR.sub.-- OS.sub.-- ATTR.sub.-- FLAG.sub.-- IS.sub.--
DOS.sub.-- FILENAME;
ha.sub.-- volume.sub.-- id
volume.sub.-- id;
}
}
__________________________________________________________________________
In order to allocate unique volume identifiers this object holds the next free volume ID. Initially this is set to 1, so that the SYS volume can be given ID 0 when it is added to the database, in case any applications make assumptions about SYS:
______________________________________
CLASS ha.sub.-- next.sub.-- volume
SUPERCLASS ndr.sub.-- dodb.sub.-- object.sub.-- header;
PARENT ha.sub.-- server;
PROPERTY NDR.sub.-- OS.sub.-- CLASS.sub.--
FLAG.sub.-- UNREPLICATED;
ATTRIBUTE
{
ndr.sub.-- dodb.sub.--
dummy.sub.-- key
dummy.sub.-- key
PROPERTY NDR.sub.-- OS.sub.-- ATTR.sub.-- FLAG.sub.--
SIBLING.sub.-- KEY
COMPARISON ndr.sub.-- dodb.sub.-- dummy.sub.-- key.sub.-- compare
VALIDATION ndr.sub.-- dodb.sub.-- dummy.sub.-- key.sub.-- validate;
ha.sub.-- volume.sub.-- id
next.sub.-- free.sub.-- volume.sub.-- id;
}
}
______________________________________
A file or directory name can be 12 (2-byte) characters long:
______________________________________
CONSTANT HA.sub.-- FILENAME.sub.-- MAX=24;
DATATYPE ha.sub.-- filename
STRING HA.sub.-- FILENAME.sub.-- MAX;
______________________________________
The ha.sub.-- file.sub.-- or.sub.-- dir.sub.-- id is a compound unique key embracing the file or directory ID that is allocated by the server, as well as the server-generated volume number. The latter is passed as a byte from class 87 NetWare Core Protocols from which it is read directly into vol (declared as a byte below). Elsewhere in the code the type ndr.sub.-- host.sub.-- volume.sub.-- id (a UINT16) is used for the same value.
______________________________________
DATATYPE ha.sub.-- file.sub.-- or.sub.-- dir.sub.-- id
ULONG file.sub.-- or.sub.-- dir;
ha.sub.-- volume.sub.-- id
vol;
}
______________________________________
Files and directories have many shared attributes, the most important being the file name. This must be unique for any parent directory.
__________________________________________________________________________
CLASS
ha.sub.-- file.sub.-- or.sub.-- dir
PARENT ha.sub.-- directory;
SUPERCLASS
ndr.sub.-- dodb.sub.-- object.sub.-- header;
ATTRIBUTE
{
ha.sub.-- filename filename
PROPERTY NDR.sub.-- OS.sub.-- ATTR.sub.-- FLAG.sub.-- SIBLING.su
b.-- KEY .linevert split.
NDR.sub.-- OS.sub.-- ATTR.sub.-- FLAG.sub.-- IS.sub.--
DOS.sub.-- FILENAME;
ha.sub.-- file.sub.-- or.sub.-- dir.sub.-- id
id
PROPERTY NDR.sub.-- OS.sub.-- ATTR.sub.-- FLAG.sub.-- GLOBAL.sub
.-- KEY .linevert split.
NDR.sub.-- OS.sub.-- ATTR.sub.-- FLAG.sub.-- UNREPLICAT
ED
GROUP file.sub.-- or.sub.-- dir.sub.-- id.sub.-- group;
ULONG attributes;
SHORT creation.sub.-- date;
SHORT creation.sub.-- time;
ndr.sub.-- dodb.sub.-- auth.sub.-- id
creation.sub.-- id;
SHORT access.sub.-- date;
SHORT archive.sub.-- date;
SHORT archive.sub.-- time;
ndr.sub.-- dodb.sub.-- auth.sub.-- id
archive.sub.-- id;
}
}
__________________________________________________________________________
A file has some additional attributes not present in a directory, and may contain a contents fork which can be accessed via a file distributor 90 (FIG. 3):
__________________________________________________________________________
CLASS ha.sub.-- file
SUPERCLASS
ha.sub.-- file.sub.-- or.sub.-- dir;
PROPERTY NDR.sub.-- OS.sub.-- CLASS.sub.-- FLAG.sub.-- DEFINE.sub.--
REPLICAS .linevert split.
NDR.sub.-- OS.sub.-- CLASS.sub.-- FLAG.sub.-- HAS.sub.--
PARTIALLY.sub.-- REPLICATED.sub.-- FILE .linevert split.
NDR.sub.-- OS.sub.-- CLASS.sub.-- FLAG.sub.-- HAS.sub.--
FILE.sub.-- PATH.sub.-- NAME .linevert split.
NRD.sub.-- OS.sub.-- CLASS.sub.-- FLAG.sub.-- PARENT.sub.--
HAS.sub.-- RSC;
ATTRIBUTE
{
BYTE execute.sub.-- type;
SHORT update.sub.-- date
property NDR.sub.-- OS.sub.-- ATTR.sub.-- FLAG.sub.-- UNREPLICATED;
N
SHORT update.sub.-- time
property NDR.sub.-- OS.sub.-- ATTR.sub.-- FLAG.sub.-- UNREPLICATED;
O
ndr.sub.-- dodb.sub.-- auth.sub.-- id
update.sub.-- id
property NDR.sub.-- OS.sub.-- ATTR.sub.-- FLAG.sub.-- UNREPLICATED;
N
ULONG length
property NDR.sub.-- OS.sub.-- ATTR.sub.-- FLAG.sub.-- UNREPLICATED;
O
}
}
__________________________________________________________________________
A directory does not possess a contents fork for file distributor 90 access. The access rights mask is inherited and should be managed by like access control lists ("ACLs"):
__________________________________________________________________________
CLASS ha.sub.-- directory
SUPERCLASS
ha.sub.-- file.sub.-- or.sub.-- dir;
PROPERTY NDR.sub.-- OS.sub.-- CLASS.sub.-- FLAG.sub.-- DEFINE.sub.--
REPLICAS .linevert split.
NDR.sub.-- OS.sub.-- CLASS.sub.-- FLAG.sub.-- HAS.sub.--
FILE.sub.-- PATH.sub.-- NAME .linevert split.
NDR.sub.-- OS.sub.-- CLASS.sub.-- FLAG.sub.-- HAS.sub.-- RSC;
//replication support count
ATTRIBUTE
{
BYTE access.sub.-- rights.sub.-- mask;
SHORT update.sub.-- date;
SHORT update.sub.-- time;
ndr.sub.-- dodb.sub.-- auth.sub.-- id
update.sub.-- id;
SHORT rsc
PROPERTY NDR.sub.-- OS.sub.-- ATTR.sub.-- FLAG.sub.-- IS.sub.--
RSC .linevert split.
NDR.sub.-- OS.sub.-- ATTR.sub.-- FLAG.sub.-- REPLICATED;
}
}
__________________________________________________________________________
The root directory must appear at the top of the hierarchy below the volume. Its name is not used; the volume name is used instead. This is the top of the replication hierarchy and therefore is the top level RSC in this hierarchy:
______________________________________
CLASS ha.sub.-- root.sub.-- directory
SUPERCLASS ha.sub.-- directory;
PARENT ha.sub.-- volume;
PROPERTY NDR.sub.-- OS.sub.-- CLASS.sub.-- FLAG.sub.-- DEFINE.sub.--
REPLICAS .linevert split.
NDR.sub.-- OS.sub.-- CLASS.sub.-- FLAG.sub.-- HAS.sub.--
RSC;
}
______________________________________
In one embodiment, schemas such as the schema 84 are defined in a source code format and then compiled to generate C language header files and tables. The named source file is read as a stream of lexical tokens and parsed using a recursive descent parser for a simple LL(1) syntax. Parsing an INCLUDE statement causes the included file to be read at that point. Once a full parse tree has been built (using binary nodes), the tree is walked to check for naming completeness. The tree is next walked in three passes to generate C header (.H) files for each included schema file. The header generation passes also compute information (sizes, offsets, and so forth) about the schema which is stored in Id nodes in the tree. Finally, the complete tree is walked in multiple passes to generate the schema table C source file, which is then ready for compiling and linking into an agent's executable program. Each disconnectable computer 40 also includes a replica manager 46 which initiates and tracks location-specific updates as necessary in response to database manager 42 requests. The replica manager is discussed in detail in connection with later Figures. A file system interface 48 on each computer 40 mediates between the replica manager 46 and a storage device and controller 54. Suitable file system interfaces 48 include well-known interfaces 48 such as the File Allocation Table ("FAT") interfaces of various versions of the MS-DOS.RTM. operating system (MS-DOS is a registered trademark of Microsoft Corporation), the XENIX.RTM. file system (registered trademark of Microsoft Corporation), the various NOVELL file systems (trademark of Novell, Inc.), the various UNIX file systems (trademark of Santa Cruz Operations), the PCIX file system, the High Performance File System ("HPFS") used by the OS/2 operating system (OS/2 is a mark of International Business Machines Corporation), and other conventional file systems. Suitable storage devices and respective controllers 54 include devices and controllers for the media disclosed above in connection with the storage medium 34 (FIG. 1) and other conventional devices and controllers, including non-volatile storage devices. It is understood, however, that the database replicas 56 stored on these media are not necessarily conventional even though the associated devices and controllers 54 may themselves be known in the art. Each computer 40 also has a network link manager 50 that is capable of establishing a network connection 52 with another disconnectable computer 40. Suitable network link managers 50 include those capable of providing remote procedure calls or an equivalent communications and control capability. One embodiment utilizes "DataTalk" remote procedure call software with extended NetWare Core Protocol calls and provides functionality according to the following interface:
______________________________________
rpc.sub.-- init( )
Initialize RPC subsystem
rpc.sub.-- shutdown( )
Shutdown RPC subsystem
rpc.sub.-- execute( )
Execute request at single
location
rpc.sub.-- ping( )
Ping a location (testing)
rpc.sub.-- claim.sub.-- .sub.-- next.sub.-- execute( )
Wait until the next rpc.sub.-- execute( )
is guaranteed to be used by this
thread
rpc.sub.-- free.sub.-- next.sub.-- execute( )
Allow others to use rpc.sub.-- execute(
______________________________________
)
Those of skill in the art will appreciate that other remote procedure call mechanisms may also be employed according to the present invention. Suitable network connections 52 may be established using packet-based, serial, internet-compatible, local area, metropolitan area, wide area, and wireless network transmission systems and methods. FIGS. 2 and 3 illustrate one embodiment of the replica manager 46 of the present invention. A replica distributor 70 insulates the database manager 42 from the complexities caused by having database entries stored in replicas 56 on multiple computers 40 while still allowing the database manager 42 to efficiently access and manipulate individual database objects, variables, and/or records. A replica processor 72 maintains information about the location and status of each replica 56 and ensures that the replicas 56 tend to converge. A consistency distributor 74 and a consistency processor 76 cooperate to maintain convergent and transactional consistency of the database replicas 56. The major processes used include an update process which determines how transaction updates are applied, an asynchronous synchronization process that asynchronously synchronizes other locations in a location set, a synchronous synchronization process that synchronously forces two locations into sync with each other, an optional concurrency process that controls distributed locking, and a merge process that adds new locations to a location set. In one embodiment, processes for synchronization and merging are implemented using background software processes with threads or similar means. The concurrency process may be replaced by a combination of retries and clash handling to reduce implementation cost and complexity. Each location is identified by a unique location identifier. A "location sync group" is the group of all locations that a specific location synchronizes with. The location sync group for a database replica 56 on a client 20 is the client and the server 16 or other computer 28 that holds a master replica 56; the computer 28 holding the master replica 56 is the "storage location" of the target database. The location sync group for the computer 28 that holds the master replica 56 is all computers 28 connectable to the network that hold a replica 56. A "location set" is a set of presently connected locations in a location sync group. Locations in an "active location set" have substantially converged, while those in a "merge location set" are currently being merged into the active location set. Objects are read at a "reference location" and updated at an "update location," both of which are local when possible for performance reasons. To support concurrency control, objects require a "lock location" where they are locked for read or update; the local location is the same for all processes in a given location set. According to one update process of the present invention, the updates for a single transaction are all executed at one update location. Each group of updates associated with a single transaction have a processor transaction identifier ("PTID") containing the location identifier of the update location and a transaction sequence number. The transaction sequence number is preferably monotonically consecutively increasing for all completed transactions at a given location, even across computer 28 restarts, so that other locations receiving updates can detect missed updates. The PTID is included in update details written to an update log by an object processor 86. An update log (sometimes called an "update stream") is a chronological record of operations on the database replica 56. Although it may be prudent to keep a copy of an update log on a non-volatile storage device, this is not required. The operations will vary according to the nature of the database, but typical operations include adding objects, removing objects, modifying the values associated with an object attribute, modifying the attributes associated with an object, and moving objects relative to one another. The PTID is also included as an attribute of each target database object to reflect the latest modification of the object. In one embodiment, the PTID is also used to create a unique (within the target database) unique object identifier ("UOID") when a target database object is first created. A target database object may contain attributes that can be independently updated. For instance, one user may set an archive attribute on a file while a second user independently renames the file. In such situations, an object schema 84 may define attribute groups. A separate PTID is maintained in the object for each attribute group, thereby allowing independent updates affecting different attribute groups of an object to be automatically merged without the updates being treated as a clash. The consistency distributor 74 gathers all of the updates for a single transaction and sends them, at close transaction time, to the update location for the transaction. The consistency processor 76 on the update location writes the updates to a transaction logger 88. In one embodiment, the transaction logger 88 buffers the updates in memory (e.g. RAM). If the update location is not local then the updates are committed to the transaction log and the PTID for the transaction is returned, so that the same updates can be buffered locally; this allows all updates to be applied in order locally. In this manner the transaction updates are applied to the update location. An objective of one asynchronous synchronization process of the present invention is to keep the rest of the locations in the location set in sync without unacceptable impact on foreground software process performance. This is achieved by minimizing network transfers. A process of the consistency processor 76 (such as a background software process) either periodically or on demand requests the transaction logger 88 to force write all pending transactions to the log and (eventually) to the target database. The consistency processor 76 also causes the batch of updates executed at an update location to be transmitted to all other locations in the current location set as a "SyncUpdate" request. These updates are force written to the log before they are transmitted to other locations, thereby avoiding use of the same transaction sequence number for different transactions in the event of a crash. The SyncUpdate requests are received by other locations in the same location set and applied to their in-memory transaction logs by their respective consistency processors 76. Each consistency processor 76 only applies SyncUpdate transactions which have sequence numbers that correspond to the next sequence number for the specified location. The consistency processor 76 can determine if it has missed updates or received them out of order by examining the PTID. If updates are missed, the PTID of the last transaction properly received is sent to the consistency distributor 74 that sent out the updates, which then arranges to send the missing updates to whichever consistency processors 76 need them. Acknowledged requests using threads or a similar mechanism can be used in place of unacknowledged requests sent by non-central locations. Non-central locations (those not holding a master replica 56) only need to synchronize with one location and thus only require a small number of threads. To promote scalability, however, central locations preferably use unacknowledged broadcasts to efficiently transmit their SyncUpdate requests. The asynchronous synchronization process causes SyncUpdate requests to be batched to minimize network transfers. However, the cost paid is timeliness. Accordingly, a synchronous synchronization process according to the present invention may be utilized to selectively speed up synchronization. The synchronous synchronization process provides a SyncUptoPTID request and response mechanism. In one embodiment, the SyncUptoPTID mechanism utilizes a SyncState structure which is maintained as part of a location state structure or location list that is managed by a location state processor 80 in the memory of each computer 28. The SyncState structure for a given location contains a location identifier and corresponding transaction sequence number for the most recent successful transaction applied from that location. The SyncState structure is initialized from the update log at startup time and updated in memory as new transactions are applied. A SyncUptoPTID request asks a destination to bring itself up to date with a source location according to a PTID. The destination sends a copy of the SyncState structure for the source location to that source location. The source location then sends SyncUpdate requests to the destination location, as previously described, up to an including the request with the PTID that was specified in the SyncUptoPTID request. In a preferred embodiment, the central server is a NetWare server and the SyncUptoPTID requirements are approximately 100 bytes per location, so scalability is not a significant problem for most systems. A merge process according to the present invention includes merging location sets when disconnected disconnectable computers are first connected or reconnected. For instance, merging location sets normally occurs when a computer new to the network starts up and merges into an existing location set. Merging can also happen when two sets of computers become connected, such as when a router starts. Merging occurs when two replicas 56 are resynchronized after the computers 28 on which the replicas 56 reside are reconnected following a period of disconnection. Either or both of the computers 28 may have been shut down during the disconnection. A set of updates are "merged atomically" if they are merged transactionally on an all-or-nothing basis. A distributed database is "centrally synchronized" if one computer 28, sometimes denoted the "central server," carries a "master replica" with which all merges are performed. Portions of the master replica or portions of another replica 56 may be "shadowed" during a merge. A shadow replica, sometimes called a "shadow database", is a temporary copy of at least a portion of the database. The shadow database is used as a workspace until it can be determined whether changes made in the workspace are consistent and thus can all be made in the shadowed replica, or are inconsistent and so must all be discarded. The shadow database uses an "orthogonal name space." That is, names used in the shadow database follow a naming convention which guarantees that they will never be confused with names in the shadowed database. A "state-based" approach to merging compares the final state of two replicas 56 and modifies one or both replicas 56 to make corresponding values equal. A "log-based" or "transaction-based" approach to merging incrementally applies successive updates made on a first computer 28 to the replica 56 stored on a second computer 28, and then repeats the process with the first computer's replica 56 and the second computer's update log. A hybrid approach uses state comparison to generate an update stream that is then applied incrementally. The present invention preferably utilizes transaction-based merging rather than state-based or hybrid merging. As an illustration, consider the process of merging a single new location A with a location set containing locations B and C. In one embodiment, the following performance goals are satisfied: (a) Use of locations B and C is not substantially interrupted by synchronization of the out-of-date location A with B and C; and (b) Users connected to location A (possibly including multiple users if location B is a gateway) are able to see the contents of the other locations in the set within a reasonable period of time. Merging typically occurs in three phases. During a "merging out" phase location A sends newer updates to location B. For instance, if A's location list contains PTID 50:14 (location identifier:transaction sequence number) and B's location list contains PTID 50:10, then the newer updates sent would correspond to PTID values 50:11 through 50:14. During a "merging in" phase new updates in the merge location B are merged into A's location. For instance, suppose A's location list contains PTIDs 100:12 and 150:13 and B's location list contains PTIDs 100:18 and 150:13. Then the new updates would correspond to PTID values 100:13 through 100:18. If updates are in progress when merging is attempted, the initial attempt to merge will not fully succeed, and additional iterations of the merging in and merging out steps are performed. In one embodiment, merging does not include file contents synchronization. Instead file contents are merged later, either by a background process or on demand triggered by file access. This reduces the time required for merging and promotes satisfaction of the two performance goals identified above. In embodiments tailored to "slow" links, merging is preferably on-going to take advantage of whatever bandwidth is available without substantially degrading the perceived performance of other processes running on the disconnectable computers. In embodiments employing an update log, the log is preferably compressed prior to merging. Compression reduces the number of operations stored in the log. Compression may involve removing updates from the log, altering the parameters associated with an operation in a given update, and/or changing the order in which updates are stored in the log. In one embodiment, all Object Database calls come through the consistency distributor 74, which manages distributed transaction processing and maintains consistency between locations. Almost all calls from a location distributor 78 are made via the consistency distributor 74 because the consistency distributor 74 supports a consistent view of the locations and the database replicas 56 on them. The consistency distributor 74 and an object distributor 82 support multiple concurrent transactions. This is needed internally to allow background threads to be concurrently executing synchronization updates. It could also be used to support multiple concurrent gateway users. In an alternative embodiment, multiple concurrent transactions on the same session is supported through the consistency distributor 74. In one embodiment, the consistency distributor 74 and the consistency processor 76 are implemented in the C programming language as a set of files which provide the functionality described here. Files CD.H and CD.C implement part of the consistency distributor 74. A separate module having files CD.sub.-- BG.H and CD.sub.-- BG.C is responsible for background processes associated with merging and synchronization. A module having files CDI.H and CDI.C contains functions used by both the CD and CD.sub.-- BG modules. These modules provide functionality according to the following interface:
______________________________________
cd.sub.-- init Init CD
cd.sub.-- shutdown
Shutdown CD
cd.sub.-- create.sub.-- replica
Create a replica of a specified
database
cd.sub.-- remove.sub.-- replica
Remove a replica of a specified
database
cd.sub.-- load.sub.-- db
Load an existing database
cd.sub.-- unload.sub.-- db
Unload an existing database
cd.sub.-- merge.sub.-- start
Start merge of active and merge
location sets
cd.sub.-- merge.sub.-- stop
Stop merge
cd.sub.-- start.sub.-- txn
Start a CD transaction
cd.sub.-- set.sub.-- txn.sub.-- ref.sub.-- loc
Set reference/update lid
(location identifier) for txn
(transaction)
cd.sub.-- get.sub.-- txn.sub.-- desc
Get a txn descriptor given a txn
id
cd.sub.-- abort.sub.-- txn
Abort a CD transaction
cd.sub.-- end.sub.-- txn
End a CD transaction
cd.sub.-- commit Commit all previously closed txns
to disk
cd.sub.-- execute.sub.-- txn
Execute locks and updates for a
txn
cd.sub.-- read Do read or lookup request
cd.sub.-- readn Do readn
cd.sub.-- lookup.sub.-- by.sub.-- uoid
Do lookup using UOID
cd.sub.-- add.sub.-- lock
Add an object or agent lock
cd.sub.-- remove.sub.-- lock
Remove an object or agent lock
cd.sub.-- modify.sub.-- attribute
Modify a single attribute in a
previously read object
cd.sub.-- init.sub.-- new.sub.-- doid
Setup all fields in a new doid
cd.sub.-- add Add a new object
cd.sub.-- remove Remove an object
cd.sub.-- move Move an object
cd.sub.-- set.sub.-- marker
Add marker point to txn
cd.sub.-- revert.sub.-- to.sub.-- marker
Revert txn state to last marker
cd.sub.-- get.sub.-- effective.sub.-- access.sub.-- right
Get the effective access rights
for the current session and
object
cd.sub.-- convert.sub.-- uoid2doid
Convert UOID to DOID
cd.sub.-- sync.sub.-- object
Get the server to send a newly
replicated object
cd.sub.-- bg.sub.-- init
Initialize CD background
processes
cd.sub.-- bg.sub.-- merge
Execute a background merge
cd.sub.-- bg.sub.-- sync.sub.-- remote.sub.-- upto.sub.-- ptid
Bring remote location up to date
with local PTID
cdi.sub.-- init
cdi.sub.-- shutdown
cdi.sub.-- execute.sub.-- ack.sub.-- sys
Execute acknowledged request
using system session
cdi.sub.-- execute.sub.-- ack
Execute acknowledged request
cdi.sub.-- apply.sub.-- locks
Apply locks for txn
cdi.sub.-- abort.sub.-- prc.sub.-- txn
Remove all locks already set for
a txn
//Forced update location (used to change update location
when executing clash handler functions)
cdi.sub.-- register.sub.-- forced.sub.-- update.sub.-- location
Register location to be used as
update location for thread
cdi.sub.-- unregister.sub.-- forced.sub.-- update.sub.-- location
Unregister location to be used as
update location for thread
cdi.sub.-- get.sub.-- forced.sub.-- update.sub.-- location
Get forced update location for
thread
cdi.sub.-- sync.sub.-- upto.sub.-- ptid
Bring location up to date with
PTID
cdi.sub.-- sync.sub.-- upto.sub.-- now
Bring location up to date with
latest PTID
cdi.sub.-- sync.sub.-- loc.sub.-- list
Make my location list consistent
with destination location list
and return info on mismatch of
PTIDs
cdi.sub.-- read.sub.-- loc.sub.-- list
Read location list
cdi.sub.-- sync.sub.-- upto.sub.-- dtid
Bring location up to date with
DTID
______________________________________
Since updates are cached during a transaction, special handling of reads performed when updates are cached is required. In one embodiment, the caller of cd.sub.-- read() or cd.sub.-- readn() sees the results of all updates previously executed in the transaction. In an alternative embodiment, for cd.sub.-- read() reads will see all previously added objects and will see the modified attributes of objects, but will not see the effects of moves or removes. Thus if an object is removed during a transaction the read will behave as if it has not been removed. The same is true for moved objects. Modifications to keys will have no effect on reads using the keys. The cd.sub.-- readn() function behaves as if none of the updates in the current transaction have been applied. In one embodiment, the consistency processor 76, which processes all distributed object database requests, includes background processes that manage object database updates on local locations and synchronization of locations. Within this embodiment, a CP module contains a dispatcher for all requests which call functions that have a prefix of "cpXX.sub.-- "; a CPR module processes read requests; a CPU module processes update and lock requests; a CPSM module processes synchronization and merging requests; a CP.sub.-- BG module controls background processing which includes scheduling multiple background threads, controlling the state of all local locations and synchronization of local locations with local and remote locations; and a CPUI module provides functions that are shared by the CP.sub.-- BG and CPx modules. These modules provide functionality according to the following interface:
______________________________________
cp.sub.-- init Includes performing mounting of
local locations and recovery of
TL (transaction logger 88) and OP
(object processor 86)
cp.sub.-- shutdown
Shutdown CP
cp.sub.-- process
Process a consistency request
cp.sub.-- clear.sub.-- stats
Reset CP statistics
cp.sub.-- dump.sub.-- stats
Dump CP statistics to the log
cpr.sub.-- process.sub.-- read
Process OP read or lookup request
cpr.sub.-- process.sub.-- readn
Process readn request
cpu.sub.-- register.sub.-- dtid
Register use of a DTID at a
reference location
cpu.sub.-- execute.sub.-- txn
Execute single txn at reference
location
cpu.sub.-- commit
Commit all txns for session
cpu.sub.-- add.sub.-- locks
Add list of locks
cpu.sub.-- remove.sub.-- locks
Remove list of locks
cpu.sub.-- abort.sub.-- prc.sub.-- txn
Remove object locks for specified
transaction
cpsm.sub.-- sync.sub.-- upto.sub.-- ptid
Bring remote locations up to date
as far as given PTID
cpsm.sub.-- get.sub.-- latest.sub.-- ptid
Obtain the latest PTID
cpsm.sub.-- get.sub.-- sync.sub.-- object
Remote machine wants to sync a
newly replicated object
cpsm.sub.-- sync.sub.-- object
Add a newly replicated object to
the local database
cpsm.sub.-- get.sub.-- sync.sub.-- update
Get a local sync.sub.-- update
cpsm.sub.-- sync.sub.-- update
Apply multiple update txns to
location
cpsm.sub.-- read.sub.-- loc.sub.-- list
Read list of locations and states
cpsm.sub.-- sync.sub.-- loc.sub.-- list
Sync location list
cpsm.sub.-- merge.sub.-- loc.sub.-- list
Attempt to merge my location list
with other location list
cpsm.sub.-- sync.sub.-- finished
Remote machine is notifying us
that a sync.sub.-- upto.sub.-- ptid has
completed
cpsm.sub.-- request.sub.-- merge
Request a merge of this location
with the central server
cpui.sub.-- init Initialize internal structures
cpui.sub.-- shutdown
Shutdown CPUI subsystem
cpui.sub.-- execute.sub.-- txn
Execute update txn at a local
location
cpui.sub.-- apply.sub.-- update.sub.-- list.sub.-- to.sub.-- db
Apply an update list to an OP
database
cpui.sub.-- commit
Commit all txns at location
cpui.sub.-- flush
Flush all txns to object database
at location
cpui.sub.-- replay.sub.-- logged.sub.-- transactions
Replay transactions from the log
that have not been committed to
OP
cp.sub.-- bg.sub.-- init
Initialize CP.sub.-- BG subsystem
cp.sub.-- bg.sub.-- shutdown
Shutdown CP.sub.-- BG subsystem
cp.sub.-- bg.sub.-- handle.sub.-- distributed.sub.-- request
Handle a request that requires
remote communication
cp.sub.-- bg.sub.-- notify.sub.-- close.sub.-- txn
Notify CP.sub.-- BG of a closed
transaction
cp.sub.-- bg.sub.-- notify.sub.-- commit
Notify CP.sub.-- BG that all txns are
committed at a location
cp.sub.-- bg.sub.-- attempt.sub.-- send.sub.-- flush
Attempt to send out and flush
txns
cp.sub.-- bg.sub.-- notify.sub.-- load
Notify CP.sub.-- BG of a newly loaded DB
cp.sub.-- bg.sub.-- notify.sub.-- unload
Notify CP.sub.-- BG of a newly unloaded
DB
cp.sub.-- bg.sub.-- flush.sub.-- upto.sub.-- ptid
Force all transactions upto the
specified ptid to the migrated
state
______________________________________
The location distributor 78 in each replica manager 46 and the location state processor 80 are used to determine the storage locations of database entries. In one embodiment, the location state processor 80 uses a cache of the current state of locations and maintains state information on the merging process. The location state processor 80 is responsible for processing remote requests which pertain to the location list. All locations that are up at any time within a sync group are in either the ACTIVE or MERGE location sets. The ACTIVE location set contains all locations that are in sync with the local location up to certain sync watermarks. The MERGE location set contains all nodes that are not in sync with the local location, either through not having updates the active set does have, or through having updates the active set does not have. Locations in the MERGE set enter the ACTIVE set through the two-way merging process described above, under control of the consistency distributor 74 and the consistency processor 76. Once in the ACTIVE set, a location should never leave it until the location goes down. Each location continuously sends out its local updates to other members of its active location set as part of the merging process. The PTID in a location's log that was last sent out in this manner is called the location's "low watermark" PTID. For a location to enter the active set it must have all PTIDS in its local log up to the low watermark PTID; only the merging process used to move a location from the MERGE to the ACTIVE location set is capable of propagating early transactions. Each location also maintains a "high watermark" PTID which is the last transaction (in local log order) that has been committed, and is thus a candidate for sending out in a background sync update. The replica managers 46 track the last transaction sequence number made by every location up to the low watermark PTID in order to know whether a location is up to date with another location's low watermark. The log ordering may be different in different locations, up to an interleave. One embodiment of the location state processor 80 provides functionality according to the following interface:
______________________________________
ls.sub.-- init Initialize LS
ls.sub.-- shutdown
Shutdown LS
ls.sub.-- close.sub.-- db
Clear out all entries for a
database
ls.sub.-- allocate.sub.-- new.sub.-- lid
Allocate a new location
identifier for use by a new
replica
ls.sub.-- add Add a new location
ls.sub.-- remove Remove a location
ls.sub.-- modify.sub.-- local.sub.-- tid
Modify a location entry's local
transaction identifier (sequence
number)
ls.sub.-- modify.sub.-- state
Modify a location entry's state
ls.sub.-- get.sub.-- loc.sub.-- list
Get list of locations
ls.sub.-- get.sub.-- loc.sub.-- sync.sub.-- list
Get list of locations for syncing
ls.sub.-- get.sub.-- next.sub.-- loc
Get next location
ls.sub.-- get.sub.-- first.sub.-- in.sub.-- loc.sub.-- list
Get first location in list that
is in current location set
ls.sub.-- get.sub.-- loc.sub.-- entry
Get location entry given lid
(location identifier)
ls.sub.-- get.sub.-- first.sub.-- ref.sub.-- loc
Get nearest reference location in
provided list
ls.sub.-- get.sub.-- first.sub.-- ref.sub.-- loc.sub.-- in.sub.--
Get first reference location in
provided list
ls.sub.-- get.sub.-- lock.sub.-- loc
Get lock location for location
set
ls.sub.-- higher.sub.-- priority
Determine which location has
highest priority
ls.sub.-- complete.sub.-- merge
Complete the merge process
ls.sub.-- set.sub.-- sync.sub.-- watermarks
Set the high and low watermark
PTIDS used in syncing and merging
______________________________________
The object distributor 82 manages ACLs and otherwise manages access to objects in the database. In one embodiment, the object distributor 82 provides functionality according to this interface:
______________________________________
typedef void* ndr.sub.-- od.sub.-- db.sub.-- handle;
//open database handle
//lint -strong(AJX, ndr.sub.-- od.sub.-- txn.sub.-- id)
//object distributor transaction instance identifier
typedef void* ndr.sub.-- od.sub.-- txn.sub.-- id;
#define NDR.sub.-- OD.sub.-- INVALID.sub.-- TXN.sub.-- ID
(ndr.sub.-- od.sub.-- txn.sub.-- id)0
typedef struct //Txn info returned by NdrOdGetTxnInfo
ndr.sub.-- od.sub.-- db.sub.-- handle
db; /* database */
ndr.sub.-- dodb.sub.-- session.sub.-- type
session; /* session */
} ndr.sub.-- od.sub.-- txn.sub.-- info;
//Start a new clash txn for this session
ndr.sub.-- ret EXPORT
NdrOdStartClashTxn (
ndr.sub.-- od.sub.-- db.sub.-- handle
db.sub.-- handle,
/* --> Handle to the open DB */
ndr.sub.-- dodb.sub.-- session.sub.-- type
session, /* --> session */
ndr.sub.-- od.sub.-- txn.sub.-- id
*txn.sub.-- id);
/* <-- txn.sub.-- id */
//Find out what databases are available
ndr.sub.-- ret EXPORT
NdrOdEnumerateDBs (
ndr.sub.-- od.sub.-- enum.sub.-- flags
flags,
/* -->Determines which databases are included in search */
ndr.sub.-- os.sub.-- db.sub.-- name
search.sub.-- name,
/* --> The database name (may be wild)
*/
ndr.sub.-- os.sub.-- db.sub.-- type.sub.-- name
search.sub.-- type,
/* --> The database type (may be wild)
*/
ndr.sub.-- dodb.sub.-- database id.sub.-- type
search.sub.-- id,
/* --> The database id (may be wild) */
ndr.sub.-- os.sub.-- db.sub.-- name
name,
/* <-- The database name */
ndr.sub.-- os.sub.-- db.sub.-- type.sub.-- name
type,
/* <-- The database type */
ndr.sub.-- dodb.sub.-- database.sub.-- id.sub.-- type
*id,
/* <-- The database id */
UINT16 *index);
/* <--> Set to 0 to start else use
previous returned value */
//Start a new txn for this session
ndr.sub.-- ret EXPORT
NdrOdStartTxn (
ndr.sub.-- od.sub.-- db.sub.-- handle
db.sub.-- handle,
/* --> Handle to the open DB */
ndr.sub.-- dodb.sub.-- session.sub.-- type
session,
/* --> session */
ndr.sub.-- od.sub.-- txn.sub.-- id
*txn.sub.-- id);
/* <-- txn id */
______________________________________
The interface includes NdrOdCloseTxn(), which closes updates for the current transaction and causes all updates since the last NdrOdStartTxn() call to be applied. Either all updates will be applied, or none will be applied. NdrOdCloseTxn() does not commit the updates, that is, they are not written to disk. NdrOdCommit() is used to commit closed updates to disk. However, after calling NdrOdCloseTxn(), no further updates may be applied in the transaction. This function is also where all the locking and updates previously cached actually get done. Consequently, most locking and/or consistency errors are reported here (after synchronization) so that the transaction can be retried:
______________________________________
ndr.sub.-- ret EXPORT
NdrOdCloseTxn(ndr.sub.-- od.sub.-- txn.sub.-- id
txn.sub.-- id);
/* --> txn.sub.-- id */
______________________________________
The NdrOdEndTxn() function ends the current transaction and executes an implicit NdrOdCloseTxn(). No error is returned if no transaction is currently open:
__________________________________________________________________________
ndr.sub.-- ret EXPORT
NdrOdEndTxn(ndr.sub.-- od.sub.-- txn.sub.-- id
txn.sub.-- id);
/* --> txn id */
The NdrOdCommit function commits all previously closed
transactions for the session to disk:
ndr.sub.-- ret EXPORT
NdrOdCo.linevert split.it(
ndr.sub.-- od.sub.-- db.sub.-- handle
db, /* --> DB to commit */
ndr.sub.-- dodb.sub.-- session.sub.-- type
session);
/* --> session */
The interface also includes the following functions:
//Abort current txn
ndr.sub.-- ret EXPORT
Ndr.sub.-- OdAbortTxn(ndr.sub.-- od.sub.-- txn.sub.-- id
txn.sub.-- id);
/* --> txn.sub.-- id */
//Get info on current txn
ndr.sub.-- ret EXPORT
NdrOdGetTxnInfo(
ndr.sub.-- od.sub.-- txn.sub.-- id
txn.sub.-- id,
/* --> txn.sub.-- id */
ndr.sub.-- od.sub.-- txn.sub.-- info*
txn.sub.-- info);
/* <-- txn.sub.-- info */
//Lookup an object using parent Distributed Object Identifier
//(DOID; encodes location info to assist in sending distributor
//requests to the right machine; includes UOID) & sibling key
or
//using global key; the key value MUST be a contiguous
structure.
ndr.sub.-- ret EXPORT
NdrOdLookupByKey(
ndr.sub.-- od.sub.-- txn.sub.-- id
txn.sub.-- id,
/* --> txn.sub.-- id */
ndr.sub.-- dodb.sub.-- access.sub.-- rights.sub.-- type
rights.sub.-- needed.sub.-- on.sub.-- parent,
/* --> rights needed on parent */
ndr.sub.-- os.sub.-- class
class.sub.-- id,
/* --> Class id. of superclass to match */
/* Acts as filter when key contains wildcard. */
ndr.sub.-- dodb.sub.-- doid.sub.-- class*
parent.sub.-- doid,
/* --> Parent DOID */
ndr.sub.-- os.sub.-- attribute
key.sub.-- id,
/* --> Type of unique key */
UINT16 key.sub.-- length,
/* --> Length, in bytes, of the key value */
VOID* key, /* --> Key value */
ndr.sub.-- dodb.sub.-- doid.sub.-- class* doid);
/* <-- Pointer to returned DOID of object */
//Lookup an object using DOID
//This checks the existence of the object and updates its DOID
ndr.sub.-- ret EXPORT
NdrOdLookup(
ndr.sub.-- od.sub.-- txn.sub.-- id
txn.sub.-- id,
/* --> txn.sub.-- id */
ndr.sub.-- dodb.sub.-- access.sub.-- rights.sub.-- type
rights.sub.-- needed.sub.-- on.sub.-- parent,
/* --> rights needed on parent */
ndr.sub.-- dodb.sub.-- doid.sub.-- class*
doid, /* --> DOID */
ndr.sub.-- dodb.sub.-- doid.sub.-- class*
new.sub.-- doid);
/* <-- Updated DOID of object */
//Lookup an object's parent using DOID.
ndr.sub.-- ret EXPORT
NdrOdLookupParent(
ndr.sub.-- od.sub.-- txn.sub.-- id
txn.sub.-- id,
/* --> txn.sub.-- id */
ndr.sub.-- dodb.sub.-- access.sub.-- rights.sub.-- type
rights.sub.-- needed.sub.-- on.sub.-- parent,
/* --> rights needed on parent */
ndr.sub.-- dodb.sub.-- doid.sub.-- class*
doid, /* --> DOID */
ndr.sub.-- dodb.sub.-- doid.sub.-- class*
parent.sub.-- doid);
/* <-- Parent DOID of object */
//Read an object using parent DOID and sibling key or using
//global key. It's always OK to read an object with an out of
//date parent.sub.-- doid as the parent's lid is not used to get the
//reference location. The key value MUST be a contiguous
//structure.
ndr.sub.-- ret EXPORT
Ndr.sub.-- OdReadByKey(
ndr.sub.-- od.sub.-- txn.sub.-- id
txn.sub.-- id,
/* --> txn.sub.-- id */
ndr.sub.-- dodb.sub.-- access.sub.-- rights.sub.-- type
rights.sub.-- needed.sub.-- on.sub.-- parent,
/* --> rights needed on parent */
ndr.sub.-- os.sub.-- class
class.sub.-- id,
/* --> Class id. of superclass to match */
/* and superclass structure to be returned */
ndr.sub.-- dodb.sub.-- doid.sub.-- class*
parent.sub.-- doid,
/* --> Parent DOID */
ndr.sub.-- os.sub.-- attribute
key.sub.-- id,
/* --> Type of unique key */
UINT16 key.sub.-- length,
/* --> Length, in bytes, of the key value */
VOID* key, /* --> Key value */
UINT16 max.sub.-- length,
/* --> Max length of data read */
UINT16* length,
/* <-- Final length of data read */
ndr.sub.-- os.sub.-- object*
object);
/* --> Pointer to object buffer */
//Read an object using DOID
ndr.sub.-- ret EXPORT
Ndr.sub.-- OdRead(
ndr.sub.-- od.sub.-- txn.sub.-- id
txn.sub.-- id,
/* --> txn.sub.-- id */
ndr.sub.-- dodb.sub.-- access.sub.-- rights.sub.-- type
rights.sub.-- needed.sub.-- on.sub.-- parent,
/* --> rights needed on parent */
ndr.sub.-- os.sub.-- class
class.sub.-- id,
/* --> Class id. of superclass to match */
/* and superclass structure to be returned */
ndr.sub.-- dodb.sub.-- doid.sub.-- class*
doid, /* --> DOID */
UINT16 max.sub.-- length,
/* --> Max length of data read */
UINT16* length,
/* <-- Final length of data read */
ndr.sub.-- os.sub.-- object*
object);
/* --> Pointer to object buffer */
__________________________________________________________________________
An NdrodReadn() function which reads multiple objects using parent DOID and wildcards behaves as if none of the updates in the transaction have been applied. Interpretation of wildcard values in the key is done by registered keying functions. NdrOdReadn() reads either up to max.sub.-- objects, or up to the maximum number of objects that will fit in the max.sub.-- length object buffer:
__________________________________________________________________________
ndr.sub.-- ret EXPORT
Ndr.sub.-- OdReadn(
ndr.sub.-- od.sub.-- txn.sub.-- id
txn.sub.-- id,
/* --> txn.sub.-- id */
ndr.sub.-- dodb.sub.-- access.sub.-- rights.sub.-- type
rights.sub.-- needed.sub.-- on.sub.-- parent,
/* --> rights needed on parent */
ndr.sub.-- os.sub.-- class
class.sub.-- id,
/* --> Class id. of superclass to match
and superclass structure to be returned */
ndr.sub.-- os.sub.-- class
read.sub.-- as.sub.-- class,
/* --> Class id. target objects are to be read as */
ndr.sub.-- dodb.sub.-- doid.sub.-- class*
parent.sub.-- doid
/* --> Parent Doid */
ndr.sub.-- os attribute
key.sub.-- id,
/* --> Type of unique key */
UINT16 key.sub.-- length,
/* --> Length, in bytes, of the key value */
VOID* key,
/* --> Key value to match, can contain wildcard.
NULL implies match all objects under parent containing
the key id */
UINT16 max.sub.-- length,
/* --> Max length of data read */
UINT16* length,
/* <-- Final length of data read */
ndr.sub.-- dodb.sub.-- object.sub.-- list*
object.sub.-- list,
/* --> Pointer to object buffer */
UINT16 max.sub.-- objects,
/* --> Max number of objects read. Use OD.sub.-- MAX.sub.-- OBJECTS to
read max that will fit in buffer */
ndr.sub.-- dodb.sub.-- context.sub.-- type*
context);
/* < > --> set to DODB.sub.-- CONTEXT.sub.-- START to start a new read,
or a previously returned context to continue a previous
read. <-- set to DODB.sub.-- CONTEXT.sub.-- END if all objects read,
or a value that can be used to continue reading at the
next object */
#define NDR.sub.-- OD.sub.-- MAX.sub.-- OBJECTS
0xFFFF
__________________________________________________________________________
The NdrOdLock() function explicitly adds an exclusive or shared lock to an object using the object's DOID. The lock call is called implicitly for all updates, but should be called explicitly if read locks are required. The lock is only taken when the transaction is initially executed. It is not executed when the update is merged. The lock is applied at the end of a transaction. If it fails the transaction is aborted and should be re-tried by the caller. One embodiment does not utilize locks to control concurrency but instead relies on retries and clash handling:
__________________________________________________________________________
ndr.sub.-- ret EXPORT
NdrOdLock(
ndr.sub.-- od.sub.-- txn.sub.-- id
txn.sub.-- id,
/* --> txn.sub.-- id */
ndr.sub.-- dodb.sub.-- class*
doid, /* --> Objects's DOID */
BOOLEAN is.sub.-- exclusive);
/* --> TRUE => take exclusive lock */
The interface also includes:
//Add agent defined lock to object
ndr.sub.-- ret EXPORT
NdrOdAddAgentLock(
ndr.sub.-- od.sub.-- txn.sub.-- id
txn.sub.-- id,
/* --> txn.sub.-- id */
ndr.sub.-- dodb.sub.-- doid.sub.-- class*
doid, /* --> Objects's DOID */
ndr.sub.-- dodb.sub.-- lock.sub.-- type
lock.sub.-- type,
/* --> Type of lock */
ndr.sub.-- dodb.sub.-- lock.sub.-- flags.sub.-- type
lock.sub.-- flags,
/* --> Flags that allow multiple locks to be taken
in single call. Each bit corresponds to a separate
lock, e.g. used for read/write flags on file open */
ndr.sub.-- dodb.sub.-- lock.sub.-- deny.sub.-- flags.sub.-- type
deny.sub.-- flags);
/* --> Bits set that correspond to lock.sub.-- flags bits
causes the corresponding lock to be denied */
//Remove agent defined lock
ndr.sub.-- ret EXPORT
NdrOdRemoveAgentLock(
ndr.sub.-- od.sub.-- txn.sub.-- id
txn.sub.-- id,
/* --> txn.sub.-- id */
ndr.sub.-- dodb.sub.-- doid.sub.-- class*
doid, /* --> Objects's DOID */
ndr.sub.-- dodb.sub.-- lock.sub.-- type
lock.sub.-- type);
/* --> Type of lock */
__________________________________________________________________________
The following four calls are used to append various types of updates onto an open transaction. Any of them may return NDR.sub.-- OK indicating success, NDR.sub.-- CD.sub.-- EXCEEDED.sub.-- TXN.sub.-- LIMITS indicating that transaction limits have been exceeded, or some other error indicator. In the case of exceeded transaction limits the transaction state will not have been changed and the failed call will have had no effect. The caller is expected to commit or abort the transaction as appropriate. In all other error cases the transaction is automatically aborted before returning the error to the caller:
__________________________________________________________________________
//Modify a single attribute in a previously read object
//The object distributor caches the modifications and only
//applies them at close txn time
ndr.sub.-- ret EXPORT
Ndr.sub.-- OdModifyAttribute(
ndr.sub.-- od.sub.-- txn.sub.-- id
txn.sub.-- id,
/* --> txn.sub.-- id */
ndr.sub.-- dodb.sub.-- access.sub.-- rights.sub.-- type
rights.sub.-- needed.sub.-- on.sub.-- parent,
/* --> rights needed on parent */
ndr.sub.-- dodb.sub.-- doid.sub.-- class*
doid,
/* --> DOID of previous read version of object.
Used to verify object has not been modified by another
user since previously read */
ndr.sub.-- os.sub.-- attribute
attribute.sub.-- id,
/* --> Identifies attribute to be modified */
VOID* value); /* --> New attribute value */
//Add a new object
//The DOID attribute does not need to be filled in by the
caller.
//The DOID will be set up before writing the object to the
//database.
ndr.sub.-- ret EXPORT
NdrOdAdd(
ndr.sub.-- od.sub.-- txn.sub.-- id
txn.sub.-- id,
/* --> txn.sub.-- id */
ndr.sub.-- dodb.sub.-- access.sub.-- rights.sub.-- type
rights.sub.-- needed.sub.-- on.sub.-- parent,
/* --> rights needed on parent */
ndr.sub.-- dodb.sub.-- doid.sub.-- class*
parent.sub.-- doid,
/* --> Parent DOID */
ndr.sub.-- os.sub.-- class
class.sub.-- id,
/* --> Class id of object */
ndr.sub.-- os.sub.-- object*
object);
/* --> Pointer to agent object */
//Remove an object using DOID
ndr.sub.-- ret EXPORT
NdrOdRemove(
ndr.sub.-- od.sub.-- txn.sub.-- id
txn.sub.-- id,
/* --> txn.sub.-- id */
ndr.sub.-- dodb.sub.-- access.sub.-- rights.sub.-- type
rights.sub.-- needed.sub.-- on.sub.-- parent,
/* --> rights needed on parent */
ndr.sub.-- dodb.sub.-- doid.sub.-- class*
doid); /* --> DOID */
//Move an object using DOID
ndr.sub.-- ret EXPORT
NdrOdMove(
ndr.sub.-- od.sub.-- txn.sub.-- id
txn.sub.-- id,
/* --> txn.sub.-- id */
ndr.sub.-- dodb.sub.-- access.sub.-- rights.sub.-- type
rights.sub.-- needed.sub.-- on.sub.-- parent,
/* --> rights needed on parent */
ndr.sub.-- dodb.sub.-- doid.sub.-- class*
doid, /* --> DOID */
ndr.sub.-- dodb.sub.-- doid.sub.-- class*
target.sub.-- parent.sub.-- doid);
/* --> Target parent DOID */
//Set a marker in an open transaction. The state of the
//transaction at the time the marker is set can be reverted
//to at any time before the transaction is closed by
//calling NdrOdRevertToMarker( ).
//Only the last marker in a transaction is significant.
//This call may return NDR.sub.-- CD.sub.-- EXCEEDED.sub.-- TXN.sub.--
LIMITS which
//should be treated as for the update appending calls above
ndr.sub.-- ret EXPORT
NdrOdSetMarker(ndr.sub.-- od.sub.-- txn.sub.-- id
txn.sub.-- id);
/* --> txn.sub.-- id */
//Revert a txn's state to the last previously marked state
ndr.sub.-- ret EXPORT
NdrOdRevertToMarker (ndr.sub.-- od.sub.-- txn.sub.-- id
txn.sub.-- id);
/* --> txn.sub.-- id */
//Add a <user-id, rights-mask> pair to an object's
//access rights, overwriting any previous rights-mask for
//that user
ndr.sub.-- ret EXPORT
NdrOdAddAccessRight(
ndr.sub.-- od.sub.-- txn.sub.-- id
txn.sub.-- id,
/* --> txn.sub.-- id */
ndr.sub.-- dodb.sub.-- dboid.sub.-- class*
doid, /* --> Object DOID */
ndr.sub.-- dodb.sub.-- auth.sub.-- id.sub.-- type
user,
/* --> User to whom rights are to be granted */
ndr.sub.-- dodb.sub.-- access.sub.-- rights.sub.-- type
rights);
/* --> Rights to be granted to that user */
//Remove any <user-id, rights-mask> pair from an object's
//access rights for a given user-id
ndr.sub.-- ret EXPORT
NdrOdRemoveAccessRight(
ndr.sub.-- od.sub.-- txn.sub.-- id
txn.sub.-- id,
/* --> txn.sub.-- id */
ndr.sub.-- dodb.sub.-- doid.sub.-- class*
doid, /* --> Object DOID */
ndr.sub.-- dodb.sub.-- auth.sub.-- id.sub.-- type
user);
/* --> User whose rights are to be revoked */
//Get the array of all <user-id, rights-mask> pairs for an
object
ndr.sub.-- ret EXPORT
NdrOdGetAccessRights(
ndr.sub.-- od.sub.-- txn.sub.-- id
txn.sub.-- id,
/* --> txn.sub.-- id */
ndr.sub.-- dodb.sub.-- dboid.sub.-- class*
doid, /* --> Object DOID */
UINT16* acl.sub.-- count,
/* <-- Number of ACL entries for that object */
ndr.sub.-- dodb.sub.-- acl.sub.-- element.sub.-- type*
acl);
/* <-- Rights information for that object */
//Get the effective access rights for the current session
//for an object
ndr.sub.-- ret EXPORT
Ndr.sub.-- OdGetEffectiveAccessRight(
ndr.sub.-- od.sub.-- txn.sub.-- id
txn.sub.-- id,
/* --> txn.sub.-- id */
ndr.sub.-- dodb.sub.-- doid.sub.-- class*
doid, /* --> Object DOID */
ndr.sub.-- dodb.sub.-- access.sub.-- rights.sub.-- type*
rights);
/* <-- Effective rights for the current session */
//Convert UOID to DOID
ndr.sub.-- ret EXPORT
NdrOdConvertUoid2Doid(
ndr.sub.-- os.sub.-- class
class.sub.-- id,
/* --> Class id. of object */
ndr.sub.-- dodb.sub.-- uoid type*
uoid, /* --> UOID */
ndr.sub.-- dodb.sub.-- doid.sub.-- class*
doid); /* <-- Updated DOID */
//Convert UOID to DOID
ndr.sub.-- ret EXPORT
NdrOdConvertUoid2LocalDoid(
ndr.sub.-- os.sub.-- class
class.sub.-- id,
/* --> Class id. of object */
ndr.sub.-- dodb.sub.-- lid.sub.-- type
location,
/* --> Location on which object exists */
ndr.sub.-- dodb.sub.-- uoid.sub.-- type*
uoid, /* --> UOID */
ndr.sub.-- dodb.sub.-- doid.sub.-- class*
doid); /* <-- Updated DOID */
__________________________________________________________________________
The object processor 86 provides a local hierarchical object-oriented database for objects whose syntax is defined in the object schema 84. In one embodiment, the object processor 86 is built as a layered structure providing functionality according to an interface in the structure which is described below. The embodiment also includes a module for object attribute semantics processing, a set of global secondary indexes, a hierarchy manager, a B-tree manager, a record manager, and a page manager. Suitable modules and managers are readily obtained or constructed by those familiar with database internals. A brief description of the various components follows. The page manager provides functionality according to a logical file interface of free-form fixed length pages addressed by logical page number. Rollback and commit at this level provide anti-crash recovery. The record manager provides for the packing of variable length keyed records into fixed length pages. The B-tree manager uses the facilities of the record and page managers to provide general B-trees supporting variable length records and variable length keys. The hierarchy manager imposes a hierarchical structure on records by use of structured B-tree keys and a global UOID.fwdarw.full name index. The secondary index manager provides generalized global indexing capabilities to records. The attribute manager interprets the schema 84 in order to raise the interface of the object processor 86 from a record-level to an object-level interface. The interface module of the object processor 86 uses lower level interfaces to provide functionality according to the following interface:
______________________________________
op.sub.-- init Initializes object processor
op.sub.-- shutdown
Shuts down object processor
op.sub.-- add.sub.-- database
Creates a new volume
op.sub.-- mount.sub.-- database
Mounts a specified volume for use
op.sub.-- dismount.sub.-- database
Dismounts the specified volume
op.sub.-- remove.sub.-- database
Removes a specified volume
(permanently)
op.sub.-- read Read an object by UOID
op.sub.-- readn Read one or more objects with
wildcards
op.sub.-- execute.sub.-- update.sub.-- list
Apply one or more updates
op.sub.-- commit Commit updates to a specified
volume
op.sub.-- rollback
Rollback to the last committed
state
op.sub.-- free.sub.-- inversion.sub.-- list
Free up an inversion list
returned from update execution
op.sub.-- clear.sub.-- stats
Clear object processor statistics
op.sub.-- dump.sub.-- stats
Dump statistics to the log
______________________________________
Due to higher level requirements of trigger functions in a set of trigger function registrations 94, in one embodiment it is necessary to have the old values of modified attributes available on a selective basis. This is done by means of a `preservation list` produced by op.sub.-- execute.sub.-- updates(). The preservation list contains an update list specifying old attribute values for all executed updates that require it (as determined by a callback function), together with pointers to the original causative updates. These updates may not actually be present in the input update list, as in the case of an object removal that generates removes for any descendant objects it may have. Preservation lists reside in object processor 86 memory and must thus be freed up by the caller as soon as they are no longer needed. The transaction logger 88 provides a generic transaction log subsystem. The logs maintained by the logger 88 provide keyed access to transaction updates keyed according to location identifier and processor transaction identifier (PTID). In one embodiment, a non-write-through cache is used to batch uncommitted transaction updates. The transaction logger 88 is used by the consistency processor 76 to support fast recovery after a crash. Recovery causes the target database to be updated with any transactions that were committed to the log by the logger 88 but were not written to the target database. The log file header contains a "shutdown OK" flag which is used on startup to determine if recovery is required for the location. The transaction logger 88 is also used by the consistency processor 76 to support fast synchronization. The update log created by the logger 88 is used to replay the updates from one location to a second location using minimal disk and network 10 transfers. The file distributor 90 distributes file contents to appropriate locations in the network 10. A file processor 92 supports each file distributor 90 by carrying out requested read, write, lock, or other operations locally. The file distributor 90 hides from agents the complexities caused by the distributed nature of files. To the extent possible, the interface portion of the file distributor 90 resembles file system interfaces that are familiar in the art. An open file is denoted by a numeric fork.sub.-- id and functions are provided to read, write, open, and otherwise manipulate and manage files and their contents. However, a class in the schema 84 can be given a REPLICATED.sub.-- FILE property. Whenever an object of such a class is created in the database, a distributed file is created by the file distributor 90 and file processor 92 to hold the file contents associated with that object. For instance, the Hierarchy Agent might create such an object to denote a leaf node in the directory hierarchy. In short, in one embodiment the file distributor 90 neither has nor needs an explicit externally called mechanism for creating files. Moreover, the distributed file is deleted from storage when the corresponding object is deleted from the database. The locations at which the file is stored are precisely those at which the object exists. When a file with more than one replica 56 is modified and closed, the file distributors 90 and file processors 92 at the various locations holding the replicas 56 ensure that all replicas 56 of the file receive the new contents. It is not necessary for the agent to expressly manage any aspect of file content distribution. A distributed file is identified by the UOID of the corresponding object; no built-in hierarchical naming scheme is used. A transaction identifier is also required when opening a file, to identify the session for which the file is to be opened. In one embodiment, the file distributor 90 and file processor 92 provide functionality according to the following interface:
______________________________________
//An ndr.sub.-- fd.sub.-- fork.sub.-- id is the Id by which an FD open
fork is known
typedef SINT16 ndr.sub.-- fd.sub.-- fork.sub.-- id;
#define
NDR.sub.-- FD.sub.-- NOT.sub.-- A.sub.-- FORK.sub.-- ID (-1)
//An ndr.sub.-- fd.sub.-- open.sub.-- mode is a bit-mask which specifies
whether a
//fork is open for reading and/or writing
typedef UINT16 ndr.sub.-- fd.sub.-- open.sub.-- mode;
#define
NDR.sub.-- FD.sub.-- OPEN.sub.-- READ.sub.-- MODE
0x0001
#define
NDR.sub.-- FD.sub.-- OPEN.sub.-- WRITE.sub.-- MODE
0x0002
#define
NDR.sub.-- FD.sub.-- OPEN.sub.-- EXCL.sub.-- MODE
0x0004
#define
NDR.sub.-- FD.sub.-- OPEN.sub.-- EXTERNAL.sub.-- MODES
0x0007
//The remaining open modes are private to the replica managers
#define
NDR.sub.-- FD.sub.-- OPEN.sub.-- SYNC.sub.-- MODE
0x0008
#define
NDR.sub.-- FD.sub.-- OPEN.sub.-- CLOSE.sub.-- ON.sub.-- EOF.sub.--
MODE 0x0010
#define
NDR.sub.-- FD.sub.-- OPEN.sub.-- READ.sub.-- NOW
0x0020
______________________________________
In one alternative embodiment, opening a file with an NdrFdOpenFile() function returns pointers to two functions together with a separate fork.sub.-- id for use with these two functions only. These pointers are of the type ndr.sub.-- fd.sub.-- io.sub.-- function, and may be used as alternatives to NdrFdReadFile() and NdrFdWriteFile() when accessing that open file only. The functions should be at least as efficient as NdrFdReadFile() and NdrFdWriteFile() and will be significantly faster when the file access is to a local location. Their use does require that the caller maintain a mapping from the open fork id onto these function pointers. For this reason, NdrFdReadFile() and NdrFdWriteFile() should always be available for all open files in this alternative embodiment:
______________________________________
typedef ndr.sub.-- ret EXPORT (*ndr.sub.-- fd.sub.-- io.sub.-- function)(
ndr.sub.-- fd.sub.-- fork.sub.-- id
fork.sub.-- id,
/* --> Id of open fork
UINT32 offset,
/* --> Offset at which to start reading */
UINT16* length,
/* <--> desired length on entry, actual length on
exit. These will only differ if an error
is encountered (such as end of file) */
UINT8* data,
/* <--> Data read or written */
ndr.sub.-- od.sub.-- txn.sub.-- id
txn.sub.-- id);
/* --> txn.sub.-- id
______________________________________
A "clash" occurs during synchronization when two desired changes to the database are inconsistent. Clashes arise from "independent" updates, namely, updates performed on separate replicas 56 while the computers holding the replicas 56 were disconnected. Thus, clashes always take place between a pair of "clashing updates" which together define a "clash condition." A "repairing update" is an update that removes a clash condition caused by a clashing update. A "transient clash" is a clash that is not present in the final states of the two replicas 56 being merged. Transient clashes only arise when log-based or hybrid merging is used. For instance, suppose two users each create a file of a given name at two locations 36, 38 while those locations are disconnected. The user at the first location 36 then deletes (or renames or moves) the file in question before reconnection such that it no longer clashes with anything on the second location 38. On merging the replicas 56 of the two locations 36, 38, the original add update for the file from the first location 36 will clash with the replica 56 of the second location 38, yet the final result of applying the update stream from the first location 36 to the replica 56 on the second location 38 is a state that is compatible with that replica 56. By contrast, "persistent clashes" create inconsistencies that are present in the final states of two replicas 56. A clash whose type is unknown is a "potential clash." A "file contents clash" occurs when a file's contents have been independently modified on two computers 28, or when a file has been removed from one replica 56 and the file's contents have been independently modified on another replica 56. An "incompatible manipulation clash" occurs when an object's attributes have been independently modified, when an object has been removed in one replica 56 and the object's attributes have been modified in another replica 56, when an object has been removed in one replica 56 and moved in the hierarchy in another replica 56, when a parent object such as a file directory has been removed in one replica 56 and has been given a child object in another replica 56, or when an object has been independently moved in different ways. Thus, although clashes are discussed here in connection with files and the file distributor 90, clashes are not limited to updates involving files. A "unique key clash" occurs when two different objects are given the same key and both objects reside in a portion of the database in which that key should be unique. In a database representing a file system hierarchy, for instance, operations that add, move, or modify files or directories may create a file or directory in one replica 56 that clashes on reconnection with a different but identically-named file or directory in another replica 56. A "permission clash" occurs when a change in file access or modification permissions that is made to a central server replica 56 would prohibit an independent update made to a mobile or client computer replica 56 from being applied to the server replica 56. A permission clash is an example of an "external clash," namely, a clash detected by reference to a structure external to the database. Permission clashes and other external clashes may be detected by trigger functions. A "grouped attribute" is a database object attribute that is associated with other database object attributes such that changing the value of any attribute in a group creates a clash with the other attributes in the group. For instance, filename and rename-inhibit attributes are preferably grouped together, while filename and file-access-date attributes are preferably not grouped together. Without attribute grouping, a change to any attribute of an object is assumed to clash with a change to any other attribute of the object or another change to the same attribute. "Eliminating a clash" means identifying the basis for the clash and eliminating it. "Recovering from a clash" means identifying the basis for the clash and either eliminating that basis or presenting alternative resolutions of the clash to a user to choose from. "Regressing an update" means undoing the update on at least one replica 56. Creating a "recovery item" means moving an object into a recovery storage, creating a duplicate object in recovery storage, or adding a new object to recovery storage. In the case of duplicating an object, uses of the object's key may be remapped so that subsequent updates are performed on the recovery item instead of the original object. Suitable recovery storage means include directory hierarchies in a file system and container hierarchies in a directory services database. If the database represents a file system hierarchy, recovery items may be gathered in a "single directory hierarchy" or "recovery directory" that contains a directory at the root of the volume, recovered items, and copies of any directories necessary to connect the recovered items properly with the root. A clash handler function of one of the types below can be registered with the file distributor 90 for a database type to be called whenever the file distributor 90 detects a clash caused by disconnected modification or removal of a file's contents. The parameters are those of a regular clash handler plus the object DOID with NDR.sub.-- OS.sub.-- CLASS.sub.-- FLAG.sub.-- HAS.sub.-- PARTIALLY.sub.-- REPLICATED.sub.-- FILE property (the file object defined by the object schema 84) and possibly a duplicated object return:
______________________________________
//Call back to a husk in respect of clashes detected at the
//database level
typedef ndr.sub.-- ret EXPORT (*ndr.sub.-- fd.sub.-- object.sub.--
clash.sub.-- fn) (
ndr.sub.-- od.sub.-- db.sub.-- handle
db, /* --> Database */
ndr.sub.-- dodb.sub.-- session.sub.-- type
session,
/* --> session to use in od.sub.-- start.sub.-- txn */
ndr.sub.-- od.sub.-- clash.sub.-- info*
info,
/* --> Information on clash */
ndr.sub.-- dodb.sub.-- doid.sub.-- class*
old.sub.-- doid,
/* --> DOID of file with clashing contents */
ndr.sub.-- dodb.sub.-- doid.sub.-- class*
new.sub.-- doid);
/* <-- Doid of duplicated file */
//Call back to the husk in respect of clashes detected at the
//filesystem level
//(via pre trigger functions)
typedef ndr.sub.-- ret EXPORT (*ndr.sub.-- fd.sub.-- filesys.sub.--
clash.sub.-- fn) (
ndr.sub.-- od.sub.-- db.sub.-- handle
db, /* --> Database */
ndr.sub.-- dodb.sub.-- session.sub.-- type
session,
/* --> session to use in od.sub.-- start.sub.-- txn */
ndr.sub.-- od.sub.-- clash.sub.-- info*
info,
/* --> Information on clash */
ndr.sub.-- dodb.sub.-- doid.sub.-- class*
doid);
/* --> DOID of file with clashing contents */
______________________________________
A parameter block such as the following is passed to clash handling functions to provide them with information about the clash:
______________________________________
typedef struct
ndr.sub.-- dodb.sub.-- ptid.sub.-- type*
ptid;
/* --> PTID of clashing txn */
ndr.sub.-- od.sub.-- clash.sub.-- type
clash.sub.-- type;
/* --> Clash type */
ndr.sub.-- os.sub.-- class
class.sub.-- id;
/* --> Class id of object causing the clash */
ndr.sub.-- os.sub.-- attribute
attr.sub.-- id;
/* --> Attr id of object causing the clash */
ndr.sub.-- dodb.sub.-- update.sub.-- list*
update.sub.-- list;
/* --> Update list of transaction */
ndr.sub.-- dodb.sub.-- update*
update;
/* --> Update causing clash (always a pointer
into `update.sub.-- list` */
BOOLEAN is.sub.-- higher.sub.-- priority;
/* --> Relative priority of location
to which update is being applied.
TRUE=> Applying to location with higher
priority (e.g. to location set with
central location) */
void* agent.sub.-- merge.sub.-- info;
/* --> Value which is reserved for (arbitrary)
use by agent clash handlers. It is
guaranteed to be set to NULL on the
first clash of a merge, and preserved
for all subsequent clashes within that
merge */
} ndr.sub.-- od.sub.-- clash info;
______________________________________
A close handler function of type ndr.sub.-- fd.sub.-- close.sub.-- fn can be registered with the file distributor 90 for a database type to be called whenever the file distributor 90 closes a modified local copy of the file contents, passing the new length and modification date/time and user identifier:
______________________________________
typedef ndr.sub.-- ret EXPORT (*ndr.sub.-- fd.sub.-- close.sub.-- fn) (
ndr.sub.-- od.sub.-- db.sub.-- handle
db, /* --> Database */
ndr.sub.-- do.sub.-- db.sub.-- session.sub.-- type
session,
/* --> session to use in od.sub.-- start.sub.-- txn */
ndr.sub.-- os.sub.-- class
class.sub.-- id,
/* --> Class ID of file */
ndr.sub.-- dodb.sub.-- uoid.sub.-- type*
uoid, /* --> UOID */
UINT32 length,
/* --> length of closed file */
UINT16 time,
/* --> modification time */
UINT16 date,
/* --> modification date */
UINT32 updator);
/* --> modification user */
______________________________________
A creation handler function of type ndr.sub.-- fd.sub.-- creation.sub.-- fn can be registered with the file distributor 90 for a database type to be called whenever the file distributor 90 creates a local copy of the file contents. This allows the replica manager 46 on a central server computer 28 to update the master copy of the file to reflect the attributes of the file created while disconnected:
______________________________________
typedef ndr.sub.-- ret EXPORT (*ndr.sub.-- fd.sub.-- creation.sub.--
fn)(
ndr.sub.-- od.sub.-- txn.sub.-- id
txn.sub.-- id,
/* --> txn.sub.-- id */
ndr.sub.-- os.sub.-- class
class.sub.-- id,
/* --> Class ID of file */
ndr.sub.-- dodb.sub.-- uoid.sub.-- type*
uoid); /* --> UOID of file */
______________________________________
The file distributor 90 embodiment also provides the following:
__________________________________________________________________________
//Return aggregated information about all volumes
ndr.sub.-- ret EXPORT
NdrFdVolumeInfo(
ndr.sub.-- od.sub.-- txn.sub.-- id
txn.sub.-- id,
/* --> txn.sub.-- id
UINT32 cluster size,
/* <-- Number of bytes per cluster */
UINT16* total.sub.-- clusters,
/* <-- Total number of clusters */
UINT16* free.sub.-- clusters);
/* <-- Number of free clusters */
//Add a file
ndr.sub.-- ret EXPORT
NdrFdAddFile(
ndr.sub.-- od.sub.-- txn.sub.-- id
txn.sub.-- id,
/* --> txn.sub.-- id */
ndr.sub.-- dodb.sub.-- doid.sub.-- class*
doid,
/* --> Uoid of file created */
UINT32 length);
/* --> Length of existing file (0 when new) */
//Remove a file
ndr.sub.-- ret EXPORT
NdrFdRemoveFile(
ndr.sub.-- od.sub.-- txn.sub.-- id
txn.sub.-- id,
/* --> txn.sub.-- id */
ndr.sub.-- dodb.sub.-- uoid type*
uoid);
/* --> Uoid of file removed */
//Open a file for reading or writing by a task
ndr.sub.-- ret EXPORT
NdrFdOpenFile(
ndr.sub.-- od.sub.-- txn.sub.-- id
txn.sub.-- id,
/* --> txn.sub.-- id */
ndr.sub.-- os.sub.-- class
class.sub.-- id,
/* --> Class ID of file to open */
ndr.sub.-- dodb.sub.-- uoid.sub.-- type
uoid,
/* --> Uoid of file to open */
ndr.sub.-- fd.sub.-- open.sub.-- mode
open.sub.-- mode,
/* --> Open for read and/or write? */
ndr.sub.-- fd.sub.-- fork.sub.-- id*
fork.sub.-- id,
/* <-- FD Fork Id of open file */
BOOLEAN is.sub.-- create,
/* --> TRUE if open as part of create
ndr.sub.-- fd.sub.-- io.sub.-- function*
read.sub.-- function,
/* <-- Function to be used for READ operations */
ndr.sub.-- fd.sub.-- io.sub.-- function*
write.sub.-- function,
/* <-- Function to be used for WRITE operations */
ndr.sub.-- fd.sub.-- fork.sub.-- id*
io.sub.-- fork.sub.-- id,
/* <-- FD Fork Id used with above two functions (only) */
UINT16* num.sub.-- forks.sub.-- remaining);
/* <-- Number of forks remaining to be opened
on same machine */
//Read from a file
ndr.sub.-- ret EXPORT
NdrFdReadFile(
ndr.sub.-- od.sub.-- txn.sub.-- id
txn.sub.-- id,
/* --> txn.sub.-- id */
ndr.sub.-- fd.sub.-- fork.sub.-- id
fork.sub.-- id,
/* --> Id of open fork */
UINT32 offset,
/* --> Offset at which to start reading */
UINT16 req.sub.-- length,
/* --> Number of bytes requested to read */
UINT8* data, /* <-- Data read */
UINT16* act.sub.-- length);
/* <-- Actual number of bytes read */
//Write to a file
ndr.sub.-- ret EXPORT
NdrFdWriteFile(
ndr.sub.-- od.sub.-- txn.sub.-- id
txn.sub.-- id,
/* --> txn.sub.-- id */
ndr.sub.-- fd.sub.-- fork.sub.-- id
fork.sub.-- id,
/* --> Id of open fork */
UINT32 offset,
/* --> Offset at which to start writing */
UINT16 req.sub.-- length,
/* --> Number of bytes requested to write */
UINT8* data); /* --> Data to be written */
//Get the current length of an open file
ndr.sub.-- ret EXPORT
NdrFdGetOpenFileLength(
ndr.sub.-- od.sub.-- txn.sub.-- id
txn.sub.-- id,
/* --> txn.sub.-- id */
ndr.sub.-- fd.sub.-- fork.sub.-- id
fork.sub.-- id,
/* --> Id of open fork */
UINT32* length);
/* <-- Length of that open file */
//Lock or Unlock a range of bytes in an open file
ndr.sub.-- ret EXPORT
NdrFdClearPhysicalRecord(or NdrFdLockPhysicalRecord(
ndr.sub.-- od.sub.-- txn.sub.-- id
txn.sub.-- id,
/* --> txn.sub.-- id */
ndr.sub.-- fd.sub.-- fork.sub.-- id
fork.sub.-- id,
/* --> Id of open fork */
UINT32 offset, /* --> Offset for lock */
UINT32 req.sub.-- length);
/* --> Number of bytes requested to lock */
//Ensure a file's contents are on disk
ndr.sub.-- ret EXPORT
NdrFdCommitFile(
ndr.sub.-- od.sub.-- txn.sub.-- id
txn.sub.-- id,
/* --> txn.sub.-- id */
ndr.sub.-- fd.sub.-- fork.sub.-- id
fork.sub.-- id);
/* --> Id of open fork
//Close a file, having completed reading and writing
ndr.sub.-- ret EXPORT
NdrFdCloseFile(
ndr.sub.-- od.sub.-- txn.sub.-- id
txn.sub.-- id,
/* --> txn.sub.-- id */
ndr.sub.-- fd.sub.-- fork.sub.-- id
fork.sub.-- id);
/* --> Id of open fork */
//Given a UOID to a file or directory return its name
//in the specified namespace, along with its parent's UOID
ndr.sub.-- ret EXPORT
NdrFdGetFilename(
ndr.sub.-- od.sub.-- db.sub.-- handle
db,
/* --> handle to current database */
ndr.sub.-- dodb.sub.-- uoid.sub.-- type*
file.sub.-- or.sub.-- dir.sub.-- id,
/* --> Uoid of object whose name is wanted */
ndr.sub.-- os.sub.-- attr.sub.-- property
namespace,
/* --> Namespace (e.g. DOS) of name wanted */
void* name.sub.-- buffer,
/* <-- Buffer to receive name */
UINT16* name.sub.-- size,
/* --> Size of provided buffer */
ndr.sub.-- dodb.sub.-- uoid.sub.-- type*
parent.sub.-- dir.sub.-- id);
/* <-- Parent UOID of object (NULL at root) */
//Callback functions to be used with
//NdrFdRegisterChangedIdCallback
typedef ndr.sub.-- ret EXPORT
(*NdrFdChangedIdCallback)(
ndr.sub.-- od.sub.-- db.sub.-- handle
db, /* --> Database Id */
ndr.sub.-- os.sub.-- class
class.sub.-- id,
/* --> Class ID of file or dir */
ndr.sub.-- dodb.sub.-- uoid.sub.-- type*
uoid, /* --> Uoid of file or dir
UINT32 new.sub.-- id);
/* --> New Id allocated by underlying file system */
__________________________________________________________________________
A NdrFdRegisterChangedIdCallback() function provides registration of a callback function to be called when a change to a file or directory's unique identifier is made. On a NetWare 4.x server this normally happens only when the file or directory is created by an internal file distributor 90 trigger function. However the identifier will be needed by agents for tasks such as directory enumeration. Because trigger functions cannot directly modify replicated objects, a record of the identifier change is queued within the file distributor 90 and the callback is made asynchronously:
__________________________________________________________________________
ndr.sub.-- ret EXPORT
NdrFdRegisterChangedIdCallback(
ndr.sub.-- os.sub.-- db.sub.-- type.sub.-- handle
db.sub.-- type,
/* --> Database type */
NdrFdChangedIdCallback
fn); /* --> Callback function */
__________________________________________________________________________
The interface also provides the following:
__________________________________________________________________________
//Register clash handlers for contents clashes for files held
in
//a database of the given type.
ndr.sub.-- ret EXPORT
NdrFdRegisterClashHandlers(
ndr.sub.-- os.sub.-- db.sub.-- type.sub.-- handle
db.sub.-- type,
// --> Database type
ndr.sub.-- os.sub.-- class
class.sub.-- id,
// --> Class ID of contents `container` eg file
ndr.sub.-- fd.sub.-- object.sub.-- clash.sub.-- fn
object.sub.-- clash.sub.-- fn,
// --> Clash handler for dealing with conflicts
// --> between objects (e.g. contents modification
// and removal)
ndr.sub.-- fd.sub.-- filesys.sub.-- clash.sub.-- fn
filesys.sub.-- clash.sub.-- fn,
// --> Clash handler for conflicts that arise
// through some characteristic of the file
// system (e.g. access rights on delete)
ndr.sub.-- fd.sub.-- filesys.sub.-- clash.sub.-- fn
filesys.sub.-- clash.sub.-- fn1);
//Register a trigger-like routine to be called when a local
//replica of a file is modified. The routine takes the length
//and modification date/time of the local replica of the file.
ndr.sub.-- ret EXPORT
NdrFdRegisterCloseHandler(
ndr.sub.-- os.sub.-- db.sub.-- type.sub.-- handle
db.sub.-- type,
// --> Database type
ndr.sub.-- os.sub.-- class
class.sub.-- id,
/* --> Class ID of file */
ndr.sub.-- fd.sub.-- close.sub.-- fn
close.sub.-- fn);
/* --> Clash handler to call */
//Register a trigger-like routine to be called when a local
//replica of a file is has been created. This allows the
//replica manager on a central server to update the
//server's master copy of the file to reflect the attributes
//of the file created during the disconnection.
ndr.sub.-- ret EXPORT
NdrFdRegisterCreationHandler(
ndr.sub.-- os.sub.-- db.sub.-- type.sub.-- handle
db.sub.-- type,
/* --> Database type */
ndr.sub.-- os.sub.-- class
class.sub.-- id,
/* --> Class ID of file */
ndr.sub.-- fd.sub.-- creation.sub.-- fn
creation.sub.-- fn);
/* --> Creation handler to call */
//De-register a clash | ||||||