System for and method of providing delta-versioning of the contents of PCTE file objects5623661Abstract A method and system for providing delta-versioning of data stored in an object-based data repository. In the preferred embodiment, delta-versioning is provided for "file" objects in a PCTE implementation in a manner which is relatively versatile for developers of PCTE compliant tools and programs and which is largely transparent to the users of those tools and programs. Further, redundant restoration of delta-versioned data is minimized and automatic removal restored data which is no longer needed is provided. Claims The embodiments of the invention in which an exclusive property or privilege is claimed are defined as follows: Description BACKGROUND OF THE INVENTION
______________________________________
int IBM.sub.-- pcte.sub.-- contents.sub.-- create.sub.-- delta (
Pcte.sub.-- object.sub.-- reference
file.sub.-- object,
Pcte.sub.-- boolean delta.sub.-- successor,
Pcte.sub.-- string delta.sub.-- program
);
______________________________________
In this interface, file.sub.-- object is an object identifier to indicates the object to be delta-versioned, delta.sub.-- successor is a boolean flag to indicate whether the successor object(s) to the specified object should also be delta-versioned (described below) and delta.sub.-- program is a string which is the name of a program that has been written to comply with the program specification in Appendix 1 and is to be used to perform the actual delta-versioning work. In the presently preferred embodiment of the present invention, the "file" object includes two additional, private, attributes. Specifically, there is provided: a "delta.sub.-- prgm" attribute which indicates which delta-versioning program is used to delta or to recreate the contents, if this attribute does not exist, the "file" object has not been delta-versioned; and a "delta.sub.-- version" attribute which indicates which particular delta within the delta control file represents the contents of the "file" object. Further, in the presently preferred embodiment if the contents of the "file" are delta-versioned, the "contents" attribute includes the path and filename of the delta control file for the contents. In the present invention, the "file" object's private attributes are modified such that, when a program attempts to read or write the contents of a "file" object, the object first checks for the existence of the "delta.sub.-- prgm" attribute to determine whether delta-versioning has been designated for the object. For a read operation, if the "delta.sub.-- prgm" attribute does not exist, the contents of the "file" object are accessed in the conventional fashion, i.e.--the file which stores the contents is read from the host file system 40. If the "delta.sub.-- prgm" attribute exists, the delta-versioning program indicated by the "delta.sub.-- prgm" attribute is called (transparently to the user) to restore the version of the contents required by the user from the delta control file. Specifically, the PCTE system employs the delta-versioning program indicated by the "delta.sub.-- prgm" attribute and passes the delta-versioning program appropriate parameters (as described in Appendix 1) including parameters to indicate that a restore operation is to be performed, the version which must be restored (as indicated by the "delta.sub.-- version" attribute), the path and filename of the appropriate delta control file (from the "contents" attribute) and a filename for the restored file created by the delta-versioning program. To create a first (original) version of a "file" object, the user creates the "file" object and its contents in the normal manner. If the contents of the "file" object are to be delta-versioned, a call is made to the delta-versioning API to create a delta control file. Specifically, the API is called with suitable parameters including: the "file" object of interest; the delta-versioning program to be employed. The API sets the "file" object's "delta.sub.-- prgm" attribute to the delta-versioning program to be used and invokes the delta-versioning program to create the delta control file for the "file" object's contents. The "contents" attribute in the "file" object is updated to include the path and filename of the delta control file which has been created and the "file" object's "delta.sub.-- version" attribute is set to a value representing the version identifier which is returned by the delta-versioning program. Finally, the original, non-delta versioned, contents file on host file system 40 is deleted. FIG. 4 shows a "file" object 46 which is similar to object 10 in FIG. 2 but whose contents have been delta-versioned. As shown in the Figure, a delta control file 48 has been created for the contents of object 46 on file system 40. The "contents" attribute has been updated to include the name and path to delta control file 48, the "delta.sub.-- prgm" attribute has been set to indicate that the delta was created by the SCCS delta versioning program and the "delta.sub.-- version" attribute has been set to the version identifier returned by the delta-versioning program, in this example indicating that the version is .DELTA.0, i.e.--the original version. FIG. 5 shows the result of a new version of "file" object 46 of FIG. 4 being created with the "version.sub.-- revise" API. The contents of the resulting "file" object 50 has also been delta-versioned by a call to the delta-versioning API with parameters including object 50 and the same delta-versioning program employed for delta-versioning the contents of object 46. The "contents" attribute of object 50 includes the path and filename of delta control file 48 which has been updated to include a delta to reconstruct the contents of object 50. Object 50's "delta.sub.-- prgm" attribute is set to SCCS and its "delta.sub.-- version" attribute is set to .DELTA.1, as returned by the delta-versioning program. The "successor" link of object 46, "0.successor", is set to indicate object 50 and the "predecessor" link of object 50, "0.predecessor", is set to indicate object 46 in the conventional manner. FIG. 6 shows the results of creating multiple versions of a "file" object which have been delta-versioned. As mentioned above, the present invention allows a user to delta-version a "file" object and its predecessors and successors. Accordingly, the present invention must be able, given a "file" object as a parameter to the delta-versioning API, to identify each predecessor and successor object of the given object. This is slightly complicated by the fact that PCTE allows an object to have multiple predecessor and/or successor objects, resulting in version graphs such as that shown in FIG. 7. In the present invention, given an object as a parameter to the delta-versioning API, its version graph is determined during the execution of the API by first traversing the predecessor links of the object and, in turn, the predecessor links of its predecessors. If an object has multiple predecessors, the traversal fans out to encompass each branch of those predecessor links. The traversal of the objects in a branch ceases when the traversal reaches the earliest object of the branch or when the traversal reaches an object which has been previously identified during the traversal. In some circumstances, such as wherein a traversal starts from object V6 of FIG. 7, one or more objects such as object V3 will not be located by the predecessor traversal. Once the traversal of the predecessor links has been completed, the successor links are traversed from the given object until the leaf objects of each branch are reached. At this point, and "missed" predecessor objects are identified by again traversing the predecessor links, as described above, starting from each identified leaf object. For the specific example of FIG. 7, object V3 would be identified by traversing the predecessor link from object V8. Once a complete version graph has been obtained for the given object, the delta-versioning API invokes the delta-versioning program, which was passed to this API as a parameter, to "implement" the delta-versioning. The delta-versioning program is invoked to create a delta control file for the original object and to create and add deltas for each subsequent version object. The above-mentioned attributes of the delta-versioned objects are set appropriately and, as last step which occurs only after a successful creation of a delta control file and/or delta (described further below), the contents files are removed from the host file system. The present invention also allows delta-versioning of only part of a version graph by selecting an object for delta-versioning and not specifying that its successor objects should also be delta-versioned. In this case, a value of false is passed to the boolean parameter of the delta-versioning API indicating that only the predecessors of the given object are to be delta-versioned, thus excluding its successors. FIG. 8 shows an example wherein "file" objects V4 and its predecessor objects V3 through V1 have been delta-versioned without delta-versioning its successor objects V5 through V7. If a user subsequently specifies that object V7 is to be delta-versioned, the present invention first cheeks to see if object V7 has a predecessor object. If it does not, unlike the example shown in FIG. 8, V7 is the original object and the delta-versioning program will be invoked to create a delta control file for the contents as previously described above. If object V7 does have a predecessor object, i.e. object V7 is a subsequent version of another "file" object as is the case in FIG. 8, the present invention will determine the version graph for V7, as described above. In the trivial case, none of the objects in the version graph have been delta-versioned and a delta control file is created for the original object in the version graph and deltas are added for each subsequent object, in turn, as described above. In the non-trivial case, such as that shown in FIG. 8 wherein some of the preceding objects have been delta-ed and some have not, the present invention identifies the earliest preceding object in the version graph which has been delta-versioned and delta-versions each succeeding object, in turn, by adding an appropriate delta to the delta control file indicated by the "contents" attribute of that earliest preceding delta-versioned object. In the case, such as object of object V7 in FIG. 7, that an object has two or more predecessor objects, the delta is created between the object and the earlier of its predecessors. The attributes of each of the newly delta-versioned objects are appropriately updated and, upon completion of a successful delta-versioning, the contents files on the host file system are removed. If desired, the user can specify that the present invention also cheeks to see if object V7 has any successor objects which should be delta-versioned. If delta-versioning is specified for successor objects, the present invention will check to see if object V7 has successor objects. If it does, they are delta-versioned, in turn, to complete the delta-versioning of the entire version graph. An additional feature of the present invention is that of shared storage for restored contents files and automatic clearing of file system storage space when the restored contents are no longer needed. When a read operation (invoked by a "contents.sub.-- open" or similar API) is performed on a "file" object whose contents have been delta-versioned, a check is performed to see if any other user already has restored the desired contents. Specifically, a check is performed to see if the restored contents file is already present on host file system 40. If so, the user is provided with a file handle to that restored contents file. If no other user is accessing the desired contents (i.e.--the restored contents do not already exist on the host file system), the contents are restored by the delta-versioning program (as described above) to a file on the host file system and the user is supplied with the file handle of the restored file while the delta control file and associated delta remain unchanged. When the contents are no longer needed by a user, they are de-accessed (by a "contents.sub.-- close" or similar API). When there is no longer any user accessing the restored contents file, the file is closed in the conventional manner and the file handle freed. In this manner, redundant storage is reduced and storage space which is no longer needed is freed. By definition in PCTE, any API must be atomic, meaning that an API call must either completely succeed or have no effect if it fails. To comply with this requirement, the delta-versioning program provides two calls which allow the delta-versioning API to perform the necessary error correction steps if an invocation of the API fails. Specifically, the delta-versioning program includes a call which deletes a delta control file, thus removing all delta versions, and a call which removes deltas from a delta control file when the deltas are subsequent to a specified version. These two calls, in addition to the create, update and restore calls, allow for the required atomic implementation of the delta-versioning API. Presently, many delta-versioning programs do not directly support the delta-versioning program specification of the present invention. Accordingly, the program specification has been designed to work with existing delta-versioning programs which have been encapsulated to correspond with the specification. Specifically, in the presently preferred embodiment the SCCS program has been encapsulated by a wrapper' implemented as a UNIX.RTM. KORN Shell script, as shown in Appendix 2. Of course, any suitable wrapper or encapsulation method may be employed to allow other delta-versioning programs to be employed, as would occur to those of skill in the art. As mentioned above, the present invention is not limited to any particular delta-versioning system or program. It is contemplated that under some circumstances the implementer of a PCTE tool or program may wish to provide different delta-versioning programs for different types of contents. For example, SCCS may be provided for text contents while another delta-versioning program is provided for binary contents. In such a case, the tool implementer may first check the file contents to find out what program (SCCS for example) is suitable. When a suitable delta-versioning program has been determined, the API is called to perform the delta-versioning with the identified program. The present invention provides for the delta-versioning of data associated with objects, such as the PCTE "file" object, in a manner which is transparent to the user and yet flexible for the implementer of the tool or program. The present invention also provides for the automatic freeing of storage space occupied by restored versions of contents files which are no longer required.
APPENDIX 1
__________________________________________________________________________
A delta.sub.-- command string comprises the name of the particular
delta-versioning
program selected followed by one of:
COMMAND
IDENTIFIER
PARAMETER(S)
DESCRIPTION
__________________________________________________________________________
-c initial.sub.-- version.sub.-- file
this call creates a delta control file
from the original version of the file
"initial.sub.-- version.sub.-- file". The call
returns
the path and filename of the delta
control file created and the version
identifier produced by the delta
program.
-d delta.sub.-- control.sub.-- file,
this call creates a delta between the
version.sub.-- file
file "version.sub.-- file" and its previous
version in the delta control file
"delta.sub.-- control.sub.-- file". The call returns
the path and filename of the delta
control file and the version identifier
produced by the delta program.
-g delta.sub.-- control.sub.-- file,
this call restores a version with the
restored.sub.-- file.sub.-- version,
version identifier "version.sub.-- identifier"
version.sub.-- identifier
from the delta control file
"delta.sub.-- control.sub.-- file" and gives it the
filename "restored.sub.-- version.sub.-- file".
-x delta.sub.-- control.sub.-- file
this call erases the delta control file
"delta.sub.-- control.sub.-- file".
-z delta.sub.-- control.sub.-- file,
this call erases subsequent delta
version.sub.-- identifier
versions of a version identifier
"version.sub.-- identifier" from the delta
control file "delta.sub.-- control.sub.-- file".
-f version.sub.-- file
this call tests the contents of the input
file "version.sub.-- file" to determine if the
contents are valid for the delta.sub.--
versioning mechanism of the delta
program. The call returns a 0 if the
contents are valid, a 1 if they are not
valid.
__________________________________________________________________________
Each of the program calls also returns an exit code which indicates whether the call has succeeded or failed.
APPENDIX 2
__________________________________________________________________________
#!/bin/ksh
for x
do
case $x in
-c)
ROOT.sub.-- FILE.sub.-- NAME = $2
SUFFIX = echo $0 .vertline. awk -F/ `{print $NF}`
LAST.sub.-- FNAME.sub.-- COMP = echo $ROOT.sub.-- FILE.sub.-- NAME
.vertline. awk -F/ `{print $NF}`
S.sub.-- FILE.sub.-- NAME = echo $ROOT.sub.-- FILE.sub.-- NAME
.vertline. .backslash.
sed `s/`$LAST.sub.-- FNAME.sub.-- COMP`/s.`$LAST.sub.-- FNAME.sub.--
COMP`.`$SUFFIX`/`
VERSION = 1.1
if [! -f "$S.sub.-- FILE.sub.-- NAME" ]
then
$0 -d $S.sub.-- FILE.sub.-- NAME $ROOT.sub.-- FILE.sub.-- NAME
rc = $?
exit $rc
fi
admin -i$ROOT.sub.-- FILE.sub.-- NAME -r$VERSION $S.sub.-- FILE.sub.--
NAME >>/dev/null 2>&1
rc = $?
if [ rc -ne 0 ]
then
exit $rc
fi
echo $S.sub.-- FILE.sub.-- NAME
echo $VERSION
exit 0
;;
-g)
S.sub.-- FILE.sub.-- NAME =$2
TARGET.sub.-- FILE.sub.-- NAME = $3
VERSION = $4
LAST.sub.-- T.sub.-- FNAME.sub.-- COMP = echo $TARGET.sub.-- FILE.sub.--
NAME .vertline. awk -F/ `{print $NF}`
LAST.sub.-- S.sub.-- FNAME.sub.-- COMP = echo $S.sub.-- FILE.sub.-- NAME
.vertline. awk -F/ `{print $NF} `
TARGET.sub.-- DIR = echo$TARGET.sub.-- FILE.sub.-- NAME.vertline.sed's/`$
LAST.sub.-- T.sub.-- FNAME.sub.-- COMP`/./`
DELTA.sub.-- FILE.sub.-- NAME = echo $LAST.sub.-- S.sub.-- FNAME.sub.--
COMP .vertline. sed's/s.//`
CURRENT.sub.-- DIR = pwd
cd $TARGET.sub.-- DIR
EXIST.sub.-- FLAG = 1
while [ EXIST.sub.-- FLAG -ne 0 ]
do
if [ -f $DELTA.sub.-- FILE.sub.-- NAME
then
sleep 1
else
EXIST.sub.-- FLAG = 0
get -s -r$VERSION $S.sub.-- FILE.sub.-- NAME >>/dev/null 2>&1
rc =$?
if [ rc -ne 0 ]
then
cd $CURRENT.sub.-- DIR
exit $rc
fi
mv -f $DELTA.sub.-- FILE.sub.-- NAME $LAST.sub.-- T.sub.-- FNAME.sub.--
COMP
cd $CURRENT.sub.-- DIR
fi
done
exit 0
;;
-d)
S.sub.-- FILE.sub.-- NAME = $2
TARGET.sub.-- FILE.sub.-- NAME = $3
LAST.sub.-- T.sub.-- FNAME.sub.-- COMP = echo $TARGET.sub.-- FILE.sub.--
NAME .vertline. awk -f/ `{print $NF}`
LAST.sub.-- S.sub.-- FNAME.sub.-- COMP = echo $S.sub.-- FILE.sub.-- NAME
.vertline. awk -F/ `{print $NF}`
TARGET.sub.-- DIR = echo $TARGET.sub.-- FILE.sub.-- NAME .vertline. sed
's/`$LAST.sub.-- T.sub.-- FNAME.sub.-- COMP`/./`
DELTA.sub.-- FILE.sub.-- NAME = echo $LAST.sub.-- S.sub.-- FNAME.sub.--
COMP .vertline. sed 's/s.//`
P.sub.-- FILE.sub.-- NAME = echo $S.sub.-- FILE.sub.-- NAME .vertline.\
sed 's/`$LAST.sub.-- S.sub.-- FNAME.sub.-- COMP`/p.`$DELTA.sub.--
FILE.sub.-- NAME`/`
CURRENT.sub.-- DIR = pwd
cd $TARGET.sub.-- DIR
get -s -e $S.sub.-- FILE.sub.-- NAME >>/dev/null 2>&1
rc = $?
if [ rc -ne 0 ]
then
cd $CURRENT.sub.-- DIR
exit $rc
fi
rm -f $DELTA.sub.-- FILE.sub.-- NAME
cp -f $LAST.sub.-- T.sub.-- FNAME.sub.-- COMP $DELTA.sub.-- FILE.sub.--
NAME
VERSION = cat $P.sub.-- FILE.sub.-- NAME .vertline. awk `{print $2}`
delta -s -y $S.sub.-- FILE.sub.-- NAME >>/dev/null 2>&1
rc = $?
cd $CURRENT.sub.-- DIR
if [ rc -ne 0 ]
then
exit $rc
fi
echo $S.sub.-- FILE.sub.-- NAME
echo $VERSION
exit 0
;;
-x)
S.sub.-- FILE.sub.-- NAME = $2
rm -f $S.sub.-- FILE.sub.-- NAME
rc = $?
fi
exit 0
;;
-z)
S.sub.-- FILE.sub.-- NAME = $2
VERSION.sub.-- TO.sub.-- KEEP = $3
prs -r$VERSION.sub.-- TO.sub.-- KEEP = -1 -d":I:" $S.sub.-- FILE.sub.--
NAME .vertline..backslash.
while read NEW.sub.-- VERSION
do
if [ "$NEW.sub.-- VERSION" != "$VERSION.sub.-- TO.sub.-- KEEP" ]
then
rmdel -r$NEW.sub.-- VERSION $S.sub.-- FILE.sub.-- NAME >>/dev/null 2>&1
rc = $?
if [ rc -ne 0 ]
then
exit $rc
fi
fi
done
exit 0
;;
-f)
FILE.sub.-- NAME = $2
FILE.sub.-- INFO = file $FILE.sub.-- NAME
TEST.sub.-- TYPE = echo $FILE.sub.-- INFO .vertline. awk `/text/`
TEST.sub.-- EMPTY = echo $FILE.sub.-- INFO .vertline. awk `/empty/`
TEST.sub.-- ILLEGAL = echo $FILE.sub.-- INFO .vertline. awk `/illegal/`
TEST.sub.-- DATA = echo $FILE.sub.-- INFO .vertline. awk `/data/`
if [ ! -z "$TEST.sub.-- ILLEGAL" ] .vertline..vertline. [ ! -z
"$TEST.sub.-- DATA" ]
then
exit 1
fi
if [ -z "$TEST.sub.-- TYPE" ] && [ -z "$TEST.sub.-- EMPTY" ]
then
exit 1
fi
exit 0
;;
*)
exit 1
;;
esac
done
__________________________________________________________________________
|
Same subclass Same class Consider this |
||||||||||