Federated searching of heterogeneous datastores using a federated query object6233586Abstract A computer method and system capable of searching multiple heterogeneous datastores with heterogeneous data types by employing an object oriented data model to define a federated query object, a federated collection object and a federated datastore object. The federated query object translates a generic query into the appropriate queries for each datastore, the federated datastore object acts as a virtual datastore for multiple heterogeneous datastores with the ability to map concepts between datastores, and the federated collection object represents results from a federated query in a hierarchy that maintains sub-grouping information from each datastore to allow accessing of results by datastore or as a single collection of results. The federated objects thus provide user applications with enhanced accessibility of different hierarchies of information, as well as more robust search capabilities. Claims We claim: Description TABLE OF CONTENTS
Class Name Description
Collection 1 represents a collection of objects
Iterator 36 to iterate over collection members. It has subclasses
to match each collection type.
SequentialCollection a collection which supports sorting and provides
2 sequential access to its member, forward and
backward.
QueryableCollection a sequential collection to store the result or scope
5 of a query, which, in turn, can be queried further
Results 6 a sequential collection to store the result or scope
of a query
FederatedCollection a nested collection that contains the result of a
8 federated query
QueryEvaluator 11 to evaluate queries
QueryManager 10 to create and evaluate queries
Datastore 9 represents a data storage facility. In general, it
supports query creation and processing, connection,
transaction, CRUD operations (add, retrieve, update,
and delete), etc.
Query Classes FIG. 2 shows the class hierarchy for query classes. Query classes have query processing capabilities, such as preparing, executing, and monitoring the status of query executions. There is a query class for each type of query. The query classes work with a query manager 10 or datastore class 9 in processing a query. The results are returned in the form of a Collection 1 of objects, e.g. Results 6 (FIG. 1). For federated queries 19, the result is returned in a federated collection 8, that is a nested collection of results of each query's components. A user may then create a federated iterator 38 (FIG. 8) to iterate over the results across collection boundaries. Alternatively, the user may chose to create a sequential iterator 36 to access member collections and process them separately.
Class Name Description
QueryBase 12 the base class for query objects
Query 13 the class for a query object which is associated with a
specific datastore.
ParametricQuery represents a parametric query which can be prepared,
14 executed, and queried on the status of query processing
TextQuery 15 similar to parametric query objects, except for text
ImageQuery 16 similar to parametric query objects, except for images
SQLQuery 17 similar to parametric query objects, except for SQL
queries
CombinedQuery represents combined queries; a combination of
18 parametric, text, and image queries, which can be
executed together
FederatedQuery represents an aggregation of heterogeneous queries,
19 e.g. DL Parametric query, OnDemand, and other
queries to heterogeneous datastores
Data-Object classes FIG. 3 shows the class hierarchy for Data Object classes. The objects stored in and manipulated by the datastore 9 and query operations belong to data object classes. These objects are returned as the result of a query, or created and used in CRUD (add, retrieve, update, delete) operations.
Class Name Description
DataObjectBase the abstract base class for all data objects known by
20 datastores. It has a protocol attribute, that indicates
to the datastore which interface can be used to operate
on this object.
XDOBase 21 the base class to represent user-defined-types (UDT) or
large objects.
XDO 22 represents complex UDTs or large objects (LOB). This
object can exist stand-alone or as a part of a DDO 28.
Therefore, it has a persistent object id and CRUD
operations capabilities.
Blob 23 a base class for BLOBs as a placeholder to share all
generic operations pertaining to BLOBs.
Clob 26 a base class for CLOBs (Character Large Objects) as a
placeholder to share all generic operations pertaining
to CLOBs.
DBClob 27 a base class for DBCLOBs (database character large
object) as a placeholder to share all generic operations
pertaining to DBCLOBs.
BlobDL 25 represents a BLOB specific to DL
ClobDL represents a CLOB specific to DL
DBClobDL represents a DBCLOB specific to DL
PersistentObject represents a specific object whose code is statically
30 generated and compiled. This type of object will not
be covered in this document.
DDO 28 (Dynamic Data Object) represent generic data objects
which are constructed dynamically at runtime. This
object fits well with query and browsing activities
in Portal where objects are only known and generated
at runtime. It supports the CRUD operations (add,
retrieve, update, and delete), therefore, with the
help of its associated datastore DDO 28 can put itself
in and out of the datastore.
DDOBase 29 the base class for DDO 28, without the CRUD
methods
4 Class Dynamics This section provides an overview on how the above classes interact with one another. To get a better understanding of the scenario presented, refer to the particular class descriptions in the next sections and Section 12 , "Sample Programs". As shown in FIG. 4, when a user wants to submit a query, he/she can start by creating a specific datastore object 9 to give him/her access to the query processing functions provided by that datastore 9. Such a datastore 9 could be DatastoreDL, for example. The next step would be to call the "evaluate" method on the datastore 9 and supply a query string and other parameters, or a query object 13. The result of the "evaluate" method 39 is a queryable collection object 5, which also includes the "evaluate" method 39, so that it can evaluate the next query, and so on. In the preferred embodiment, the subsequent query evaluated by a queryable collection must match or be consistent with the member of the queryable collection. Otherwise, an exception is thrown. Usually the result of a query evaluation is a subset of the present collection, i.e. it could be the proper subset or even an empty collection. In addition to the query evaluation capability, a queryable collection object can be used as an input scope for the execute method in query objects. It is recommended that users create a queryable collection object 5 using the "evaluate" method 39, either in the datastore 9 or in the source queryable collection 5. However, a user can create a queryable collection 5 and fill it up with elements he/she created. Since a queryable collection may need help from a datastore object 9 in its query processing tasks, the user may need to associate it with the corresponding datastore object 9. This is done by passing a datastore object 9 as an input to the constructor. As shown in FIG. 5, another way to process a query is by creating a query object specific to the type of query language. Query objects 13 are created using the createQuery() method 40 in the datastore 9. This method of creation is preferable to ensure that the created query object 14-19 will have all the necessary information and can always get help from the datastore 9 in processing the query. A query object 14-19 can prepare and execute the query. As mentioned before, the execute method 41 may take a queryable collection 5 as an input parameter to limit the scope of the query 14-19. As shown in FIG. 6, the result of a query is a collection 1 of objects containing Dynamic Data Objects (DDOs) 28 inside. A DDO 28 has attributes, with type, value, and properties. The value of an attribute can be a reference to another DDO 28 or XDO 22, or a collection of DDOs 28 or XDO 22s, thus it may form a tree structure. This structure is quite general and flexible enough to represent folder and document data models in DL, as well as an object-oriented view of relational databases. As shown in FIG. 6, the user creates a datastore 9 (DL or DB2) then executes a proper query. The result is a collection of DDOs 28, which can be iterated over using an iterator 36 (FIG. 8). Representing DL Folders and Documents Depending on the query, each element in the resulting collection 1 can be a combination of documents, documents with parts, folders, or folders containing other folders and documents. All of these results are represented as DDOs 28. Parts 4 within a document becomes a data-itern in the DDO 28 with the name "Parts", and its value is a pointer to a collection of XDOs 22 representing DL parts. A folder DDO 28 has attributes and properties of the corresponding DL folder. It has a data-item with the name "Folder" which is a pointer to a collection of DDOs 28 representing the content of the folder. Each DDO 28 in the folder, in turn, can be another folder or simply a document. DDOs associated with DL datastores have several standard fields of information, described as follows:
Name Description
ItemType a property of the whole DDO 28 (not to a particular data-item)
taking the value of DOCUMENT or FOLDER
ItemId ItemId is not represented directly as a data_item (DDO 28
attribute) but kept in Pid as a data member Id. The user can
access it via method ddo.getPid( ).getId( ). This scheme
avoids duplication and is consistent with the underlying
representation inside DL datastore.
Rank an integer number representing the computed ranking of this
result. Rank is stored as a DDO 28 data-item (attribute) with
the reserved name "Rank". This DDO 28 data-item has a
transient property (not persistent), since it does not actually
exist as a DL item attribute. Ranking is only appropriate for
query results coming from a non-parametric query, such as
text, image query, or combined query.
5 Object Collection classes 5.1 Collection Collection 1 defines a base public interface for the collection of objects. In the preferred embodiment, collection 1 can not evaluate a query. A collection 1 may have a name (the default name is an empty string), but it could be set to anything, for example the name of the DL index class or table where the members belong. It functions as described in IBM manual, Digital Library Application Programming Guide and Reference for the Internet Connection, Doc. No. SC26-8868-00. A sample definition of the collection class is defined below.
Collection.java
package COM.ibm.mm.sdk.server;
public interface Collection
{
public int cardinality( ) ;
public Object retrieveElementAt(Iterator where) ;
public Iterator createIterator( ) ;
public void addElement(Object element) ;
public void addAllElements(Collection elements) ;
public void insertElementAt(Object element, Iterator where) ;
public void replaceElementAt(Object element, Iterator where) ;
public void removeElementAt(Iterator where) ;
public void removeAllElements( ) ;
public String getName( ) ;
public void setName(String collName) ;
};
5.2 SequentialCollection SequentialCollection 2 is a subclass of collection 1, which supports sorting and sequential accessing, in forward and backward directions. It functions as the SimpleCollection described in IBM manual, Digital Library Application Programming Guide and Reference for the Internet Connection, Doc. No. SC26-8868-00. A sample sequential collection is defined below.
SequentialCollection.java
package COM.ibm.mm.sdk.server;
public class SequentialCollection extends Collection
{
public SequentialCollection( ) ;
public int cardinality( ) ;
public Iterator createIterator( ) ;
public Object retrieveElementAt(Iterator where) ;
public void addElement(Object element) ;
public void addAllElements(Collection elements) ;
public void insertElementAt(Object element, Iterator where) ;
public void replaceElementAt(Object element, Iterator where) ;
public void removeElementAt(Iterator where) ;
public void removeAllElements( ) ;
public int sort(Object SortFunction, int SortOrder) ;
};
5.3 Folder Folder 3 is a subclass of sequential collection 2. Its purpose is to hold document DDOs 28 and folder DDOs 28 as members of a DL folder. The DDO 28 representing a DL folder has an attribute with a reserved name Folder as a reference to a Folder 3 collection. The public interface of Folder 3 is the same as of sequential collection 2, but internally, it keeps track of member addition and deletion to be reflected when the object is saved to the DL datastore 9. In addition to the methods inherited from the sequential collection class 2, Folder 3 has two additional methods: public void addMember(DDO folder, DDO member) throws Exception; Adds the member item into the folder and reflects the results immediately in the datastore. At the end of the operation the member item will be in the Folder 3 collection in memory as well as in the folder in the datastore. public void removeMember(DDO folder, DDO member) throws Exception; Removes the member item from the folder and reflects the results immediately in the datastore. At the end of the operation the member item will not be in the Folder 3 collection in memory and also removed from the folder in the datastore. 5.4 Parts Parts 4 is a subclass of sequential collection 2. Its purpose is to hold Part XDO 22 members of a DL document. The DDO 28 representing a DL document has an attribute with a reserved name Parts as a reference to a Parts 4 collection. The public interface of Parts 4 is the same as of sequential collection 2, but internally it keeps track of member addition and deletion to be reflected when the object is saved to the DL datastore. In addition to the methods inherited from the sequential collection class 2, Parts 4 has two additional methods: public void addMember(DDO item, XDO member) throws Exception; Adds the member part into the parts collection and reflects the results immediately in the datastore. At the end of the operation, the member item will be in the part collection in memory as well as inside the item in the datastore 9. This operation has the same effect as the sequence of member. add () and parts. addElement (member) where member is the part xdo and parts is the collection of parts in this item. public void removeMember(DDO item, XDO member) throws Exception; Removes the member part from the parts collection and reflects the results immediately in the datastore 9. At the end of the operation the member part will not be in the Folder 3 collection in memory and also deleted from the item in the datastore 9. This operation has the same effect as the sequence of parts.removeElement(member) and member.del() where member is the part xdo and parts is the collection of parts in this item. 5.5 QueryEvaluator A query evaluator 11 is an abstract class which has the capability of evaluating queries over an implicit or explicit collection of objects 1. Its subclass includes QueryableCollection 5, which maintains an explicit collection, and Datastore 9, which has an implicit collection (i.e. the database). Since Java does not allow multiple inheritance, QueryEvaluator 11 is defined as an interface so that QueryableCollection 5 can inherit from both SequentialCollection 2 and this interface. A sample QueryEvaluator class is defined below.
QueryEvaluator.java
package COM.ibm.mm.sdk.server;
public interface QueryEvaluator
{
public Object evaluate(String query,
short ql_type,
NVPair params[ ]) ;
};
5.6 QueryableCollection A queryable collection 5 is a subclass of both query evaluator 11 and sequential collection 2 classes, therefore it can evaluate queries on its members. It can be used to store the result or scope of a query, which, in turn, can be queried further. For example, the result of the Datastore. evaluate() method is a QueryableCollection object 5 which also implements the evaluate() method 39, so it can evaluate a query using its current collection of members as the scope. Queryable collection 5 functions as described in IBM manual, Digital Library Application Programming Guide and Reference for the Internet Connection, Doc. No. SC26-8868-00. A sample QueryableCollection is defined below.
QueryableCollection.java
package COM.ibm.mm.sdk.server;
public abstract class QueryableCollection extends SequentialCollection
implements QueryEvaluator
{
public QueryableCollection( ) ;
public QueryableCollection(Datastore ds) ;
public QueryableCollection(QueryableCollection fromCollect) ;
public abstract int cardinality( ) ;
public abstract Iterator createIterator( ) ;
public abstract Object retrieveElementAt(Iterator where) ;
public abstract void addElement(Object element) ;
public abstract void addAllElements(QueryableCollection elements) ;
public abstract void insertElementAt(Object element,
Iterator where) ;
public abstract void replaceElementAt(Object element,
Iterator where) ;
public abstract void removeElementAt(Iterator where) ;
public abstract void removeAllElements( ) ;
public abstract Object evaluate(String query, short ql_type,
NVPair params [ ]) ;
};
5.7 FederatedCollection The structure and behavior of a preferred embodiment of a FederatedCollection 8 is illustrated in FIG. 8. In the preferred embodiment, a FederatedCollection 8 allows an application program to process data objects resulting from a query as a group or collection and at the same time preserves the sub-grouping relationships that exist between them. It provides the user with a mechanism: to aggregate several collections of data objects while preserving each individual collection's information to treat these collections as one whole unit of collection, ignoring collection boundaries A federated collection 8 is a collection containing Results objects 6; it is created to hold the results of a PederatedQuery 19, which may come from several heterogeneous datastores 9. Each Results 6 contains the results of a subquery of the FederatedQuery 19 submitted to a specific native datastore 9 associated with the federated datastore 37. As shown in FIG. 8, the biggest oval represents the FederatedCollection 8 containing several smaller ovals which are collection objects 1. A FederatedCollection 8 can also contain individual Results objects 6. The FederatedIterator 38 traverses through the whole contents of FederatedCollection 8, across collection 1 boundaries, each time returning a DDO 28 (a non-collection object). The first Iterator 36 is a regular iterator for FederatedCollection 8, thus each time returning a collection object 1. The second Iterator 42 is an iterator for a collection object 1, therefore it returns DDOs 28 as members of the collection 1. The method SetToFirstCollection() in the FederatedIterator 38 will set the iterator position to the first non-collection element of the FederatedCollection 8; in this case, it would be the first element of the first collection object 1. At this point, if the method setToNextCollection() is invoked, it will set the iterator position to the first non-collection element of the second collection 1. The method setToLastCollection() in the FederatedIterator 38 will set the iterator position to the last non-collection element of the FederatedCollection 8; in this case, it would be the last element of the last collection object 1. As the opposite of setToNextcollection(), at this point, if the method setToPreviousCollection() is invoked, it will set the iterator position to the last non-collection element of the previous collection 1. Thus to iterate over a FederatedCollection 8, a user/application has a number of tools or approaches. A user can create and use Iterator 36 or a SequentialIterator (not shown) to pull each collection object 1. Then he/she could create another Iterator 36 on this collection 1 to iterate over it and to process the result according to its datastore origin. Alternatively, the user can create a federated iterator, FederatedIterator 38, and use it to iterate over all element members across collection boundaries, regardless of which datastore 9 the result came from. A FederatedCollection public class interface is set forth below. A federated collection object 8 can contain other nested collections up to any arbitrary depth. In the preferred embodiment shown below, a federated collection object is not queryable, however, this capability could be added if desired by changing "SequentialCollection" in the second line to "QueryableCollection." See also 5.11 . "FederatedIterator", 5.9 "Iterator", 5.10, "SequentialIterator".
FederatedCollection.java
package COM.ibm.mm.sdk.server;
public class FederatedCollection extends SequentialCollection
{
public FederatedCollection( ) ;
public int cardinality( ) ;
public int memberCardinality( ) ;
public Iterator createIterator( ) ;
public Iterator createMemberIterator( ) ;
public Object retrieveElementAt(Iterator where) ;
public void addElement(Object element) ;
public void addAllElements(QueryableCollection elements) ;
public void insertElementAt(Object element, Iterator where) ;
public void replaceElementAt(Object element, Iterator where) ;
public void removeElementAt(Iterator where) ;
public void removeAllElements( ) ;
};
5.7.1 Methods public int cardinality(); /*Returns the number of all individual (leaf) elements (non-collection) across the collection boundaries. This is the total of all elements in each of the sub-collections. */ public int memberCardinality(); /*Returns the number of elements in the collection. The element could be a collection object, i.e. a Results object 6, not necessarily a leaf element (non-collection). */ public Iterator createIterator(); /*Creates a new FederateIterator for this federated collection. This iterator can be used to iterate over the federated collection, across collection boundaries, returning a (non-collection) member one at a time. */ public Iterator createMemberIterator(); /*Creates a new iterator for this federated collection. The new iterator would be supporting SequentialIterator interface. That is, the method next of this iterator would return a collection, such as Results object 6. A user can then create an iterator on the Results 6 to iterate over its members. */ 5.8 Results Results 6 is a subclass of QueryableCollection 5, therefore it supports sorting and bi-directional iterators, and is queryable. Element members of Results 6 are objects, instances of DDO 28, which represent hits from a query. The iterator created by this class is SequentialIterator or BidirectionalIterator. Besides being queryable, Results 6 functions as described in IBM manual, Digital Library Application Programming Guide and Reference for he Internet Connection, Doc. No. SC26-8868-00. Below is an example of a Results 6 definition.
Results.java
package COM.ibm.mm.sdk.server;
public class Results extends QueryableCollection
{
public Results(Datastore ds) ;
public Results(Results fromResults) ;
public int cardinality( ) ;
public Iterator createIterator( ) ;
public Object retrieveElementAt(Iterator where) ;
public void addElement(Object element) ;
public void addAllElements(QueryableCollection elements) ;
public void insertElementAt(Object element, Iterator where) ;
public void replaceElementAt(Object element, Iterator where) ;
public void removeElementAt(Iterator where) ;
public void removeAllElements( ) ;
public Object evaluate(String query, short ql_type,
NVPair params[ ]) ;
};
5.9 Iterator Iterator is a base abstract class of iterators. Iterators, such as those shown in FIG. 8 (36, 42) are used to iterate over collections of members. The Iterator class has subclasses to match each collection type. It functions as described in IBM manual, Digital Library Application Programming Guide and Reference for the Internet Connection, Doc. No. SC26-8868-00. Below is an example of an iterator definition.
Iterator.java
package COM.ibm.mm.sdk.server;
public interface Iterator
{
public Object next () throws Exception;
public void reset ();
public boolean more ();
};
5.10 SequentialIterator A sequential iterator is bidirectional, it can go forward and backward over members of SequentialCollection 2. It functions as does the SimpleIterator described in IBM manual, Digital Library Application Programming Guide and Reference for the Internet Connection, Doc. No. SC26-8868-00. Below is an example of a Sequential Iterator definition.
SequentialIterator.java
package COM.ibm.mm.sdk.server;
public interface SequentialIterator extends Iterator
{
public Object previous () throws Exception;
public Object at () throws Exception;
public boolean setToFirst ();
public boolean setToLast ();
public boolean setToNext ();
public boolean setToPrevious ();
};
5.11 FederatedIterator A federated iterator 38 is used to iterate over the collective members of FederatedCollection 8 across collection boundaries. The next() method will return DDOs 28 until all collections 1 are iterated over. This iterator is created by invoking the method createIterator() in the FederatedCollection object 8. See also 5.9. "Iterator" and 5.10, "SequentialIterator".
FederatedIterator.java
package COM.ibm.mm.sdk.server;
public interface FederatedIterator extends SequentialIterator
{
// from Iterator
public Object next () throws Exception;
public void reset ();
public boolean more ();
// from SequentialIterator
public Object previous () throws Exception;
public Object at () throws Exception;
public boolean setToFirst ();
public boolean setToLast ();
public boolean setToNext ();
public boolean setToPrevious ();
// new methods
public boolean setToFirstCollection ();
public boolean setToLastCollection ();
public boolean setToNextCollection ();
public boolean setToPreviousCollection ();
};
In the preferred embodiment, the federated iterator methods always return or point to an element which is not a collection. If the element is a collection, it goes inside the collection to find the proper element of this collection, and so forth until it finds an element which is not a collection. 5.11.1 Methods public Object next() throws Exception; /* Returns the current element in the collection 1 and advances the iterator to the next position. In case the element is a collection 1, it goes inside that collection 1 and retrieves the first element recursively, until it finds an element which is not a collection 1. When the current collection 1 is exhausted, this iterator will find the next collection 1 and extract the first element (non-collection) from it.*/ public void reset(); /*Resets the iterator to the beginning of FederatedCollection 8.*/ public boolean more(); /*Returns true if there are more elements in the FederatedCollection 8. In this case, an element implicitly means a collection member which is not a collection 1 (a leaf).*/ public Object previous() throws Exception; /*Returns the current element in the collection 1 and moves the iterator backward one position. In case the element is a collection 1, it goes inside that collection 1 and retrieves the last element, until it finds an element which is not a collection 1. When the current collection 1 is exhausted, this iterator will find the previous collection 1 and extract the last element (non-collection) from it. */ public Object at() throws Exception; /*Returns the current element in the collection 1 without moving the iterator position. The returned element is not a collection 1.*/ public boolean setToFirst(); /*Set the iterator to the first element in this federated collection 8. The first element is not a collection 1. Returns true if the operation is successful, otherwise it returns false.*/ public boolean setToLast(); /*Set the iterator to the last element in this federated collection 8. The last element is not a collection 1. Returns true if the operation is successful, otherwise it returns false.*/ public boolean setToNext(); /*Set the iterator to the next element in this federated collection 8. The next element is not a collection 1. Returns true if the operation is successful, otherwise it returns false.*/ public boolean setToPrevious(); /*Set the iterator to the previous element in this federated collection 8. The previous element is not a collection 1. Returns true if the operation is successful, otherwise it returns false.*/ public boolean setToFirstCollection(); /*Set the iterator to the first element in the first collection in this federated collection 8. The first element is not a collection 1. Returns true if the operation is successful, otherwise it returns false.*/ public boolean setToLastCollection(); /*Set the iterator to the last element in the last collection 1 in this federated collection 8. The last element is not a collection 1. Returns true if the operation is successful, otherwise it returns false.*/ public boolean setToNextCollection();/*Set the iterator to the first element in the next collection 1 in this federated collection 8. The first element is not a collection 1. Returns true if the operation is successful, otherwise it returns false.*/ public boolean setToPreviousCollection(); /*Set the iterator to the last element in the previous collection 1 in this federated collection 8. The first element is not a collection 1. Returns true if the operation is successful, otherwise it returns false.*/ 6 Object Query classes 6.1 QueryManager A query manager 10 is a more powerful subclass of query evaluator 11, it can evaluate queries, as well as create query objects 13. It assists query objects 13 in query processing and in monitoring query execution. It functions as described in IBM manual, Digital Library Application Programming Guide and Reference for the Internet Connection, Doc. No. SC26-8868-00. A more specific version of query manager 10 is Datastore 9, which represents a data storage facility and provides the functionalities of the underlying datastore (i.e. a relational data base, Digital Library, etc). An example of a QueryManager interface is set forth below.
QueryManager.java
package COM.ibm.mm.sdk.server;
public interface QueryManager extends QueryEvaluator
{
public abstract Query createQuery(String query,
short ql_type,
NVPair params []);
};
6.2 QueryBase Querybase 12 is an interface for a query object 13 which can be associated with zero or more datastores 9. With the help of its datastores, query objects 13 perform query processing tasks, such as preparing and executing a query, monitoring the status of query execution, and storing the results. The result of a query is usually a Results object 6. It functions as described in IBM manual, Digital Library Application Programming Guide and Reference for the Internet Connection, Doc. No. SC26-8868-00. A sample QueryBase interface is defined below.
QueryBase.java
package COM.ibm.mm.sdk.server;
public interface QueryBase
{
public void prepare (NVPair params []) throws Exception;
public void execute (NVPair params []) throws Exception;
public int status ();
public Object result ();
};
6.3 Query Query 13 is an interface for a query object associated with one specific datastore. Objects which implement this interface are created by datastore classes 9. The result of a query is usually a Results object 6. Examples of concrete implementations of Query 13 interfaces are ParametricQuery 14, TextQuery 15, ImageQuery 16, etc, which are created by their associated datastore. An example of a Query 13 interface definition is set forth below.
Query.java
package COM.ibm.mm.sdk.server;
public interface Query extends QueryBase
{
public short qlType ();
public String queryString ();
public Datastore datastore ();
};
6.4 ParametricQuery ParametricQuery 14 is a concrete implementation of Query 13, created by a datastore object 9, to represent and execute a query involving parametric data. Parametric queries refer to a class of queries requiring an exact match of the query predicate with the values stored in the datastore 9. They function as described in IBM manual, Digital Library Application Programming Guide and Reference for the Internet Connection, Doc. No. SC26-8868-00. An example of a ParametricQuery class definition is set forth below.
ParametricQuery.java
package COM.ibm.mm.sdk.server;
public class ParametricQuery implements Query
{
public ParametricQuery(Datastore creator, String queryString);
public ParametricQuery(ParametricQuery fromQuery);
public void prepare(NVPair params []);
public void execute(NVPair params []);
public int status ();
public Object result ();
public short qlType ();
public String queryString ();
public Datastore datastore ();
};
6.5 TextQuery TextQuery 15 is a concrete implementation of Query 13, created by a datastore object 9, to represent and execute a query involving textual data. Text queries refer to a class of queries on the content of text fields for an approximate match with the given text search expression, i.e. the existence (or non-existence) of certain phrases or word-stems. The text query object 15 performs the necessary data format conversion internally (e.g. datastream conversion for Text Miner), so the methods relevant to these conversions do not need to be exposed to public. With that exception, a text query object functions as described in IBM manual, Digital Library Application Programming Guide and Reference for the Internet Connection, Doc. No. SC26-8868-00.
TextQuery.java
package COM.ibm.mm.sdk.server;
public class TextQuery implements Query
{
public TextQuery(Datastore creator, String queryString);
public TextQuery(TextQuery fromQuery);
public void prepare(NVPair params []);
public void execute(NVPair params []);
public int status ();
public Object result ();
public short qlType ();
public String queryString ();
public Datastore datastore ();
};
6.6 ImageQuery ImageQuery 16 is a concrete implementation of Query 13, created by a datastore object 9, to represent and execute a query involving image data Image queries refer to a class of queries on the visual content of images for an approximate match with the given image search expression, i.e. the existence (or non-existence) of certain color percentages, layout, and textures. An example of an ImageQuery 16 class definition is set forth below.
ImageQuery.java
package COM.ibm.mm.sdk.server;
public class ImageQuery implements Query
{
public ImageQuery(Datastore creator, String queryString);
public ImageQuery(ImageQuery fromQuery);
public void prepare(NVPair params []);
public void execute(NVPair params []);
public int status ();
public Object result ();
public short qlType ();
public String queryString ();
public Datastore datastore ();
};
6.7 CombinedQuery CombinedQuery 18 is a concrete implementation of QueryBase 12, to represent and execute a combination of parametric, text, and image queries. Combined queries provide multi-search capability on the specified datastore by coordinating the execution of their subqueries. They function as described in IBM manual, Digital Library Application Programming Guide and Reference for the Internet Connection, Doc. No. SC26-8868-00. An example of a CombinedQuery 18 class definition is set forth below.
Combined Query.java
package COM.ibm.mm.sdk.server;
public class CombinedQuery implements QueryBase
{
public CombinedQuery( ) ;
public CombinedQuery(CombinedQuery fromQuery) ;
public void prepare(NVPair params[ ]) throws Exception;
public void execute(NVPair params[ ]) throws Exception;
public int status( ) ;
public Object result( ) ;
};
6.8 OnDemandQuery OnDemandQuery (not shown) is a concrete implementation of Query 13, created by a datastore object 9, to represent and execute a query involving parametric data in OnDemand datastores. See also parametric query above. An example of an OnDemandQuery class is set forth below.
OnDemandQuery.java
package COM.ibm.mm.sdk.server;
public class OnDemandQuery implements Query
{
public OnDemandQuery(Datastore creator, String queryString) ;
public OnDemandQuery(onDemandQuery fromQuery) ;
public void prepare(NVPair params[ ]) ;
public void execute(NVPair params[ ]) ;
public int status( ) ;
public Object result( ) ;
public short qlType( ) ;
public String queryString( ) ;
public Datastore datastore( ) ;
};
6.9 FederatedQuery A FederatedQuery 19 is a concrete implementation of Query 13 which can be associated with more than one physical datastore. Its purpose is to represent and execute queries across heterogeneous datastores. This query can be a combination of a DL parametric query, OnDemand query, and other query types involving supported datastores. The FederatedQuery object 19 delegates the query processing task to each of the native datastores or to its associated federated datastore 37. The latter will translate the query into several native queries, one for each of the native datastores, submit them for processing and collect the result back. See 9.6.3 , "Federated query processing" for further details. The result is a FederatedCollection object 8, a collection of collection objects 1, which in turn contains data objects in a consistent and uniform structure as represented by dynamic data objects, DDOs 28. The federated collection 8 gives the user a choice to process the results regardless of the datastore origin or to process them according to each native datastore where the results came from. See 5.7 , "FederatedCollection". See also 5.11 , "FederatedIterator", 5.9, "Iterator", 5.10 , "SequentialIterator", 9.6 , "DatastoreFederated", and 9.6.2 , "Federated query string". An example of a FederatedQuery 19 class definition is set forth below.
FederatedQuery.java
package COM.ibm.mm.skd.server;
public class FederatedQuery implements Query
{
public FederatedQuery(Datastore creator, String queryString) ;
public FederatedQuery(FederatedQuery fromQuery) ;
public void prepare(NVPair params[ ]) ;
public void execute(NVPair params([ ]) ;
public int status( ) ;
public Object result( ) ;
public short qlType( ) ;
public String queryString( ) ;
public Datastore datastore( ) ;
};
7 Data Object classes 7.1 DataObjectBase DataObjectBase 20 is an abstract base class for data objects, data value objects (UDTs), and extender data objects defined in CDO. These are the objects known by datastores 9.
DataObjectBase.java
package COM.ibm.mm.sdk.server;
public abstract class DataObjectBase
{
public abstract short protocol( ) ;
public abstract String getObjectType( ) ;
};
DataObjectBase 20 has a protocol method, which indicates what protocol to use to store and retrieve the persistent data to/from a datastore 9. Objects with the same protocol are handled in a similar manner by the datastore 9. The preferred embodiment supports the DDO, Data value, XDO, PO, and PDDO protocols. Potentially, the data access class library can be extended to support additional subclasses and their associated protocols, for example, stream data objects and protocols. 7.1.1 Methods public abstract shortprotocol(); /*Gets the protocol type associated with this data object.*/ public abstract String getObjectType(); /*Gets the type of this object.*/ 7.2 DataObject DataObject 31 is an abstract class, and a subclass of DataObjectBase 20. DataObject 31 is used to represent objects which can have persistent data. DataObject 31 must be subclassed in order to implement a real object. A subclass of DataObject 31 is DDO 28--dynamic data object.
DataObject.java
package COM.ibm.mm.sdk.server;
public abstract class DataObject extends DataObjectBase
{
public DataObject( ) ;
public Pid getPid( ) ;
public void setPid(Pid aPid) ;
};
DataObject 31 has a Pid--persistent object identifier. Pid identifies the location of the persistent data of this data object in the datastore. If the Pid is not set, it will be created automatically after the first data manipulation operation. DataObject 31 inherits protocol from its parent DataObjectBase 20, and supports the DDO, PO, and PDDO protocols. Methods public Pid getPid(); Gets the Pid of this DataObject. public void setPid(Pid apid); Sets the Pid of this DataObject. 7.3 DDO Base--Dynamic Data Object Base DDO 28 provides a representation and a protocol to define and access object's data, independent of object's type. The DDO 28 protocol is implemented as a set of generic methods to define, add, and access each data items of an object. This protocol allows a client to create an object dynamically and get its persistent data in and out of the datastore, independent of datastore's type. The implementation may utilize schema mapping information, registered in the datastore class. The schema maps each individual persistent data items to its underlying representation in the datastore. A DDO 28 has a set of attributes, each having a type, value, and properties associated with it. The DDO 28 itself may have properties which belong to the whole DDO 28 instead of to a particular attribute. DDO 28 is a subclass of DataObject 31.
DDOBase.java
package COM.ibm.mm.sdk.server;
public class DDOBase extends DataObject
{
public DDOBase( );
public DDOBase(short initialSize);
public DDOBase(DDOBase ddo);
public DDOBase(String objectType, short initialSize);
public DDOBase(Pid pid, short initialSize);
public short protocol( );
public String getObjectType( );
public void setObjectType(String toObjectType);
public boolean updatable( );
public short addData( ) throws Exception;
public short addData(String dataName) throws Exception;
public short addDataProperty(short data_id,
String property_name) throws Exception;
public short addDataProperty(short data_id,
String property_name, Object property_value)
throws Exception;
public short dataCount( );
public short dataPropertyCount(short data_id) throws Exception;
public void setDataProperty(short data_id, short property_id,
Object property_value) throws Exception;
public Object getDataProperty(short data_id,
short property_id) throws Exception;
public void setDataPropertyName(short data_id,
short property_id, String property_name)
throws Exception;
public String getDataPropertyName(short data_id,
short property_id) throws Exception;
public void setData(short data_id, Object data_value)
throws Exception;
public Object getData(short data_id) throws Exception;
public void setDataName(short data_id,
String data_name) throws Exception;
public String getDataName(short data_id) throws Exception;
public short dataId(String data_name) throws Exception;
public short dataPropertyId(short data_id,
String property_name) throws Exception;
public void setNull(short data_id) throws Exception;
public boolean is Null(short data_id) throws Exception;
public boolean isDataSet(short data_id) throws Exception;
public boolean isDataPropertySet(short data_id,
short property_id) throws Exception;
public Object getDataByName(String data_name)
throws Exception;
public Object getDataPropertyByName(String data_name,
String property_name) throws Exception;
public Object getDataPropertyByName(short data_id,
String property_name) throws Exception;
//properties of this DDO
public short addProperty( ) throws Exception;
public short addProperty(String property_name)
throws Exception;
public short addProperty(String property_name,
Object property_value) throws Exception;
public short propertyCount( ) throws Exception;
public void setProperty(short property_id,
Object property_value) throws Exception;
public Object getProperty(short property_id) throws Exception;
public void setPropertyName(short property_id,
String property_name) throws Exception;
public String getPropertyName(short property_id)
throws Exception;
public short propertyId(String property_name)
throws Exception;
public Object getPropertyByName(String property_name)
throws Exception;
public boolean isPropertySet(short property_id) throws Exception;
};
Methods DDOBase--Constructors public DDOBase(short initialSize); public DDOBase(DDOBase ddo); public DDOBase(String objectType, short initialSize); public DDOBase(Pidpid, short initialSize); DDOBase(DDODef ddoDej); /*A DDOBase 29 object can be created by calling the constructor with our without supplying any parameter, or by copying information from another DDOBase 29.*/ public short addData() throws Exception; public short addData(String data_name) throws Exception; /*Adds a new data-item to a DDO 28 and returns the new data-item number (data-id).*/ public short addDataProperty(short data_id) throws Exception; public short addDataProperty(short data_id, Stringproperty_name) throws Exception; public short addDataProperty(short data_id, String property_name, Object property_value) throws Exception; /*Add a new property to a given data-item and returns the new property number.*/ public short dataCount();/*Gets the number of data-items in this DDO 28*/ public short dataPropertyCount (short data_id) throws Exception; /*Gets the number of properties associated with this data-item.*/ public void setDataProperty (short data_id, property_id, propertyValue) throws Exception; /*Sets the value of a given property in a data-item.*/ public Object getDataProperty (short data_id, property_id) throws Exception; I*Gets the value of a given property in a data-item.*/ public void setDataPropertyName (short data_id,shortproperty_id, String property_name) throws Exception; /*Sets the name of a given property-id in a data-item.*/ public StringgetDataPropertyName(short data_id, short property_id) throws Exception; /*Gets the name of a given property-id in a data-item.*/ public void setData (short data_id, Object dataValue) throws Exception; /*Sets the value of a given data-item.*/ public Object getData (short data_id) throws Exception; /*Gets the value of a given data-item.*/ public void setDataName(short data_id, data_name) throws Exception; /*Sets the name of a given data-item.*/ public String getDataName (short data_id) throws Exception; /*Gets the name of a given data-item.*/ public short dataId (String data_name) throws Exception; /*Gets the data-id of a given data-item name*/ public short dataPropertyId (short data_id, Stringproperty_name) throws Exception; /*Gets the property-id of a given property-name in a data-item.*/ public void setNull(short data_id) throws Exception; /*Sets the value of a data-item to null. The data-item must be nullable and the DDO 28 must be updatable.*/ public boolean isNull(short data_id) throws Exception; /*Returns true if this data-item value is null.*/ public boolean isDataSet (short data_id) throws Exception; /*Returns true if this data-item value is set already.*/ public boolean isDataPropertySet (short data_id, short propertyid) throws Exception; /*Returns true if the given property of a data-item is set.*/ public Object getDataByName (String data_name) throws Exception; /*Gets the value of a data-item, given its name.*/ public Object getDataPropertyByName (String data_name, property_name) throws Exception; Object getDataPropertyByName (short data_id, property_name) throws Exception; /*Gets the value of the given property-name of a data-item.*/ public String getObjectType(); /*Gets the object type of this DDO 28.*/ public void setObjectType(String aObjectType); /*Sets the object type of this DDO 28 and synchronize it with the object type of its Pid.*/ public short addProperty() throws Exception; public short addProperty(String property_name) throws Exception; public short addProperty(Stringproperty_name, Object property_value) throws Exception; /*Add a new property to this DDO 28 and returns the new property number.*/ public short propertyCount() throws Exception; /*Gets the number of properties associated with this DDO 28.*/ public void setProperty(short property_id, Object property_value) throws Exception; /*Sets the value of a given property id in this DDO 28.*/ public Object getProperty(shortproperty_id) throws Exception; /*Gets the value of a given property id in this DDO 28.*/ public void setPropertyName(shortproperty_id, String property_name) throws Exception; /*Sets the name of a given property id in this DDO 28.*/ public String getPropertyName(short property_id) throws Exception; /*Gets the name of a given property id in this DDO 28.*/ public short propertyId(String property_name) throws Exception; /*Gets the property-id of a given property-name in this DDO 28.*/ public Object getPropertyByName(String property_name) throws Exception; /*Gets the value of the given property-name of this DDO 28.*/ public boolean isPropertySet(short property_id) throws Exception; /*Returns true if the given property id of this DDO 28 is set.*/ 7.4 DDO--Dynamic Data Object A DDO 28 is a special class of DDOBase 29 that keeps the information about its associated datastore 9 and has CRUD methods (add, retrieve, update, delete). So, with the help of the datastore 9, it can send itself in and out of datastore 9. A datastore query returns a collection of DDOs 28. A sample DDO definition is set forth below.
DDO.java
package COM.ibm.mm.sdk.server;
public class DDO extends DDOBase
{
public DDO(short initialSize) ;
public DDO(DDO ddo) ;
public DDO(String objectType, short initialSize) ;
public DDO(Datastore ds, String objectType, short initialSize) ;
public DDO(Pid pid, short initialSize) ;
public DDO(Datastore ds, Pid pid, short initialSize) ;
public void setDatastore(Datastore ds) ;
public Datastore getDatastore( ) ;
public short protocol( ) ;
// CRUD methods
public void add( ) ;
public void retrieve( ) ;
public void update( ) ;
public void del( ) ;
};
7.4.3 Methods DDO--Constructors public DDO(short initialSize); public DDO(DDO ddo); public DDO(String objectType, short initialSize); public DDO(Datastore ds, String objectType, short initialSize); public DDO(Pidpid, short initialSize); public DDO(Datastore ds, Pidpid, short initialsize); /*DDO 28 can be created by calling the constructor with or without supplying any parameter, or by copying information from another DDO 28.*/ public void setDatastore(Datastore ds);/*Sets this datastore as the one associated with this DDO 28, i.e. the datastore to keep the persistent copy of this DDO 28.*/ public Datastore getDatastore(); /*Gets the associated datastore for this DDO 28.*/ public short protocol(); /*Returns the protocol supported by this object. In this case, it is PDDO, which essentially means that this object knows about the datastore it is associated with and, with the help of the datastore, this object can store itself in and out of the datastore.*/ CRUD methods public void add(); public void retrieve(); public void update(); public void del(); /*Insert, retrieve, update, or delete this object from persistent store. In the preferred embodiment, the add, del, and update operations only affect the object in the persistent store; they do not change the copy in main memory.*/ 7.5 Pid A persistent data identifier (Pid) provides a unique identifier to locate the persistent data of data objects in a set of known datastores. A DataObject 31 needs to have a Pid in order to store its data persistently.
Pid.java
package COM.ibm.mm.sdk.common;
public class Pid
{
public Pid( ) ;
public Pid(String sourcePidString) throws Exception;
public Pid(Pid pid) ;
public String getDatastoreType( ) ;
public void setDatastoreType(String sourceDatastoreType) ;
public String getDatastoreName( ) ;
public void setDatastoreName (String sourceDatastoreName) ;
public String getId( ) ;
public void setId(String sourceId) ;
public String pidString( ) ;
public String getObjectType( ) ;
public void setObjectType(String anObjectType) ;
public boolean isSet( ) ;
};
7.5.2 Methods Pid--Constructors public Pid(); public Pid(String sourcePidString) throws Exception; public Pid(Pidpid); sourcePidString is a String obtained as a return value from calling the pidString() method. public String getDatastoreType(); /*Gets datastore type.*/ public void setDatastoreType (String aDatastoreType); /*Sets the datastore type to the given string value.*/ public String getDatastoreName();/*Gets datastore name.*/ public void setDatastoreName (String aDatastoreName); /*Sets the datastore name to the given string value.*/ public String getId();/*Returns a string consisting of a datastore specific id of this DataObject 31 persistent data. This id contains information to locate the persistent data in the given datastore.*/ public void setId (String sourceId); /*Sets the datastore specific id of the DataObject 31 persistent data to the given string value.*/ public String pidString();/*Returns a string representation of this Pid. This string can be used as an input parameter to re-construct the Pid using its constructor. The string format may vary between datastore types.*/ public String getObjectType();/*Gets the type of the object owning this Pid.*/ public void setObjectType(String anObjectType); /*Sets the type of the object owning this Pid.*/ public boolean isSet();/*Returns true if all components of this Pid are set to their intended values.*/ 8 XDO Classes FIG. 3 shows the Data Object Class hierarchy, which includes the Extended Data Object (XDO 22) classes. The base XDO class is XDOBase 21, a subclass of DataObjectBase 20. The purpose of an XDO class is to represent a user defined data type, i.e. a data value(s) with all the methods relevant to it. A subclass of XDOBase 21 is XDO 22, which provides a public interface to support persistency and may exist as a stand-alone object (without the associated DDO 28). Other subclasses of XDOBase 21 may be defined by users to represent a specific user defined type, such as USDollar or shoe-size. However, only subclasses of XDO 22 have persistency. A more detailed description of XDO 22 classes and their properties is given in Java Grand Portal Design Notes, Lee et al., Dec. 9, 1997, and Doug Hembry. Extended Data Objects (XDO) Class Library High Level Specification and Design, (Aug 7, 1996); IBM's U.S. application Ser. No., 08/852,051, filed May 6, 1997, entitled OBJECT AGGREGATION REPRESENTATION OF RELATIONAL DATABASE ROWS HAVING NONTRADITIONAL DATATYPES, by Daniel T. CHANG, Douglas M. HEMBRY, Basuki N. SOETARMAN, and Robert N. SUMMERS; IBM's U.S. application Ser. No., 08/852,062, filed May 6, 1997, entitled CLASS HIERARCHY FOR OBJECT AGGREGATION REPRESENTATION OF RELATIONAL DATABASE ROWS WITH CELLS HAVING NONTRADITIONAL DATATYPES, by Daniel T. CHANG, Douglas M. HEMBRY, Basuki N. SOETARMAN, and Robert N. SUMMERS; IBM's U.S. application Ser. No., 08/852,055, filed May 6, 1997, entitled FLEXIBLE OBJECT REPRESENTATION OF RELATIONAL DATABASE CELLS HAVING NONTRADITIONAL DATATYPES, by Daniel T. CHANG, Douglas M. HEMBRY, Basuki N. SOETARMAN, and Robert N. SUMMERS; and IBM's U.S. application Ser. No. 08/852,052, filed May 6, 1997, entitled OBJECT REPRESENTATION OF RELATIONAL DATABASE CELLS HAVING NONTRADITIONAL LARGE OBJECT DATATYPES, by Douglas M. HEMBRY, which are herein incorporated by reference. A sample program to illustrate the use of stand-alone XDO 22 is given in Section 12.4, "Example of Add, Retrieve, Update, Delete and Open from XDO object". 8.1 XDOBase XDOBase is an abstract class, subclass of DataObjectBase, to represent a data value, ranging from a simple UDT (User Defined Type) to a complex LOB (Large Object). XDOBase provides a set of methods specifically used to communicate with Datastore objects. A user should subclass from XDOBase and provide methods that correspond to their specific UDT or LOB behaviors. 8.2 XDO The XDO 22 is an abstract class that can represent a complex UDT (User Defined Type) or LOB (Large Object). The user should subclass from XDO 22 and provide methods relevant to the UDT or LOB being represented. This user defined XDO 22 subclass can either work with an associated DDO 28 or as a stand alone-object. XDO 22 extends the public interface of XDOBase 21 mainly by defining independent datastore access CRUD functions (add, retrieve, update, del). These functions enable an application to store and retrieve the object's data to and from a datastore without the existence of an associated DDO 28 class object (stand-alone XDO 22). A stand-alone XDO 22 must have its Pid set in order to be able to locate the position in the datastore where it belongs. If the XDO 22 is used in conjunction with the DDO 28 the pid is set automatically. XDO 22 inherits protocol from its parent DataObjectBase 20. 8.3 BLob Blob 23 is an abstract class that declares a common public interface for basic binary large object (BLOB) data types in and other datastores. The concrete classes derived from Blob 23 share this common interface, allowing polymorphic processing of collections of blob objects originating from heterogeneous datastores. Blob 23 is a subclass of XDO 22 and supports the XDO 22 protocol. 8.4 BlobDL The BlobDL 25 class is one of the concrete subclass of Blob 23. BlobDL 25 defines the public interface for BLOB parts in DL. It inherits the public interface of Blob 23 (and therefore from XDO 22 and XDOBase 21) and implements any abstract functions present in the interfaces of those base classes. 8.5 CLob Clob 26 is an abstract class that declares a common public interface for basic character large object (CLOB) data types in Digital Library datastores. The concrete classes derived from Clob 26 share this common interface, allowing polymorphic processing of collections of character clob objects originating from data stored in heterogeneous datastores. In the preferred embodiment, the interface for CLob is the same as that defined for Blob 23. It is expected that over time, the interfaces would diverge, with more binary oriented function being added to the Blob 23 interface, and function suitable for character data being added to Clob 26. 8.6 ClobDL The ClobDL class is one of the concrete subclass of Clob 26. ClobDL defines a public interface for CLOB parts in DL. It inherits the public interface of Clob 26 (and therefore from XDO 22 and XDOBase 21) and implements any pure virtual functions present in the interfaces of those base classes. 8.7 PidXDO Persistent data identifier for XDO 22 provides a unique identifier to locate the persistent data of an XDO 22 in a known datastore. An XDO 22 needs to have an PidXDO in order to store its data persistently. PidXDO is a subclass of Pid. 8.8 PidXDODL PidXDODL is a specific PidXDO for XDOs stored in Digital Library. This class has information on partid where the data comes from. 8.9 Annotation Annotation represents the digital library annotation information structure(ANNOTATIONSTRUCT). The data structure provides information about an annotation affiliated with an object. 9 Data Access classes 9.1 Datastore A Datastore 9 object represents and manages a connection to a datastore, transactions, and the execution of datastore commands. Datastore 9 is a special version of query manager class 10. It supports the evaluate method, so it may be considered as a subclass of query manager 10. Datastore 9 should be subclassed to provide a specific implementation of the target datastore. For example: DatastoreDL DatastoreTS DatastoreQBIC DatastoreOD etc.
Datastore.java
package COM.ibm.mm.sdk.server;
public interface Datastore extends QueryManager
{
public void connect (String datastore_name,
String user_name,
String authentication,
String connect_string) throws Exception;
public void disconnect ( ) throws Exception;
public Object getOption (int option) throws Exception;
public void setOption (int option, Object value)
throws Exception;
public Object evaluate(String command, short
commandLangType, NVPair params[ ]) ;
public Object evaluate(Query query) ;
public Object evaluate(CQExpr queryExpr) ;
public ResultSetCursor execute (String command,
short commandLangType,
NVPair params[ ])
throws Exception;
public ResultSetCursor execute (Query query)
throws Exception;
public ResultSetCursor execute (CQExpr queryExpr)
throws Exception;
public Query createQuery (String command,
short commandLangType,
NVPair params [ ])
throws Exception;
public Query createQuery (CQExpr queryExpr) throws Exception;
public void addObject( DataObject ddo ) throws Exception;
public void deleteObject( DataObject ddo ) throws Exception;
public void retrieveObject( DataObject ddo ) throws Exception;
public void updateObject( DataObject ddo ) throws Exception;
public Object listServers( ) throws Exception;
public Object listEntity( ) throws Exception;
public Object listEntityAttributes (String entityName)
throws UsageError, DatastoreAccessError;
public void commit ( ) throws Exception;
public void rollback ( ) throws Exception;
public boolean isConnected ( ) ;
public String datastoreName( ) ;
public String datastoreType( ) ;
public Handle connection( ) ;
public String userName( ) ;
};
9.1.2 Methods public void connect (String datastore_name, String userName, authentication, connect_string) throws Exception; /*Connects to the datastore. Authentication is the password, and datastore_name is the name of the database or library server.*/ public void disconnect () throws Exception; /*Disconnects from the datastore.*/ public void setOption (int option, Object value) throws Exception; /*Sets a datastore option.*/ The valid options and values are as follows:
Option Value Description
OPT_ACCESS_MODE (for DatastoreDL
and TS)
READONLY Indicates that this
connection supports
read-only access
to the datastore
READWRITE Indicates that this
connection supports
read and write
access to the
datastore
OPT_TYPE_CONVERSION (for DatastoreDL
and TS)
TRUE Indicates that base
types from a data-
store are to be
returned in a
retrieved DDO.
FALSE Indicates that base
types from a data-
store are converted
to strings in a
retrieved DDO.
OPT_DL_WAKEUPSRV TRUE Notify the search
program that there is
indexing work to be
done for each XDO.
FALSE Notify the search
program that there is
indexing work to be
done.
OPT_FORCE_UPDATE TRUE If XDO has not
changed an update
will be performed.
FALSE If XDO has not
changed an update
will not be
performed.
OPT_DL_ACCESS Indicates the type
of access to DL
requested
SS_CONFIG Indicates that the
current access is an
administrative
session
SS_NORMAL Indicates that the
current access is a
non-configuration
session and pro-
hibits the same
userid from logging
in to the datastore
multiple times
OPT_TS_CCSID (TS only) Indicates
the type of coded
character set
identifier. See IBM
manual, Digital
Library Application
Programming
Reference Volume
2: Text Search
Server, Doc. No.
SC26-8653-00, for
the details
OPT_TS_LANG (TS only) Indicates
the type of language
identifier. See IBM
manual, Digital
Library Application
Programming
Reference Volume
2: Text Search
Server, Doc. No.
SC26-8653-00, for
the details
OPT_TRANSACTION_STATE (for DatastoreDL)
ACTIVE Indicates that a user
is in a transaction.
INACTIVE Indicates that a user
is not in a trans-
action.
public Object getOption (int option) throws Exception; /*Gets the datastore option. The valid options and values for getting options are the same as for setting options.*/ public Object evaluate(String command, short commandLangType, params[]); public Object evaluate(Query query); public Object evaluate(CQExpr queryExpr); /*Evaluates a query and returns the result as a Object containing QueryableCollection 5. The second form takes a query object. The third form takes a compound query expression as an alternate form of a query string.*/ public ResultSetCursor execute(String command, short commandLangType, params[]) throws Exception; public ResultSetCursor execute (Query query) throws Exception; public ResultSetCursor execute(CQExpr queryExpr) throws Exception; /*Executes a command to the datastore and returns a result set cursor. The second form takes a query object, and the third form takes a compound query expression.*/ public Query createQuery (String command, commandLangType, params[]); public Query createQuery (CQExpr queryExpr); /*Creates a query object using the given parameters. The second form takes a compound query expression as an alternate form of a query string.*/ public void addObject (DataObject do) throws Exception; /*Adds the data object to the datastore.*/ public void deleteObject (DataObject do) throws Exception; /*Deletes the data object from the datastore.*/ public void retrieveObject (Data Object do) throws Exception; /*Retrieve the data object from the datastore.*/ public void updateObject (DataObject do) throws Exception; /*Updates the datastore with this data object.*/ public Object listServers() throws Exception; /*Returns a list of valid servers to connect to. The return value is a sequential collection containing objects describing the server. The object will be specific for each datastore.*/ public Object listEntity() throws Exception; /*Returns a list of entity names accessible in the current server. An entity corresponds to an index-class in DL or a table in a relational database system. A user establishes a connection to the server before calling this method. The return value is a sequential collection containing the entity information that is specific for each datastore.*/ public Object listEntityAttributes(String entityName) throws UsageError, DatastoreAccessError; /*Returns a list of attribute information for a given entity. Please refer to the information under the specific datastore implementation.*/ public void commit () throws Exception; /*Commits the current transaction.*/ public void rollback () throws Exception; /*Rolls back the current transaction.*/ public boolean isConnected(); /*Returns true if it is connected to datastore.*/ public String datastoreName(); /*Gets datastore name.*/ public String datastoreType(); /*Gets datastore type.*/ public Handle connections; /*Returns the connection handle of this datastore.*/ public String userName(); /*Returns the user name of this datastore.*/ 9.2 DatastoreDL This class is a specific version of Datastore 9 to implement a Digital Library (DL) datastore. It provides Documents, Parts 4 and Folders 31 storage and retrieval mechanisms, as well as query, search, and other document processing features supported by DL. The execute and evaluate methods of DatastoreDL take parametric query strings expressed in the parametric query language type. The syntax of this query string is described below. The ParametricQuery object 14 accepts queries in this syntax; in fact the ParametricQuery 14 object delegates the low level query processing tasks to the DatastoreDL. By itself, DL only supports parametric queries. Integration with other search engines, such as SearchManager/TextMiner, or QBIC, gives DL the capability to index texts and images, and collectively process combined queries. Combined query processing is done by the CombinedQuery 18 class with the help of the associated datastore classes (DatastoreDL, TS, QBIC, etc). In addition to the functions described in Datastore 9, DatastoreDL provides some other functions for accessing catalogs.
DatastoreDL.java
package COM.ibm.mm.sdk.server;
public class DatastoreDL implements Datastore
{
. . .
. . .
public DatastoreDL ( )
throws Exception;
public void connect ( String datastore_name,
String user_name,
String authentication,
String connect_string) throws Exception;
public void startTransaction ( ) throws Exception;
public void wakeUpService(String serviceName) throws Exception;
public Handle transactionConnection( ) throws Exception;
public Object listServers( ) throws Exception;
public Object listEntity( ) throws UsageError;
public Object listEntityAttributes (String entityName)
throws UsageError, DatastoreAccessError;
public void checkout(DataObject dob)
throws UsageError, DatastoreAccessError;
public void checkIn(DataObject dob)
throws UsageError, DatastoreAccessError;
};
9.2.1 Methods public void connect (String datastore_name, userName, authentication, connect_string) throws Exception; /*Connects to the datastore. Authentication is the password and datastore_name is the name of the library server. The connect string is optional. It is used to change the password. To change the password of a userid, the user must set the DatastoreDL option OPT_DL_ACCESS to SS_CONFIG and pass in a connect string with the new password.*/ Below is sample of the connect string a user can supply.
Connect String
NPWD=NEWPASSWORD
public void startTransaction() throws Exception; /*Start a transaction. The user can user commit or rollback methods to end the transaction.*/ public Handle transactionConnection() throws Exception; /*Returns the transaction connection handle of this datastore.*/ public void wakeUpService(String serviceName) throws Exception; /*Notify the search program that there is indexing work to be done.*/ public Object listServers() throws Exception; /*Returns a list of valid servers to connect to. The return value is a sequential collection containing ServerInfoDL objects describing the server.*/ public class ServerInfoDL { public String serverName(); public String serverType(); }; /*The user needs to cast the returned object to SequentialCollection 2 and iterate over it to get ServerInfoDL objects.*/ public Object listEntity() throws UsageError; /*Returns a list of index class names in the current DL server. The user needs to establish a connection to the server before calling this method. The return value is a sequential collection containing string objects, which are index class names. The user needs to cast the returned object to SequentialCollection 2 and iterate over it to get index class names.*/ public Object listEntityAttributes(String entityName) UsageError, DatastoreAccessError; /*Returns a list of attribute information for a given index class name. The input entityName is the name of the desired index class. The user needs to establish a connection to the server before calling this method. The return value is a sequential collection containing AttributeDef objects, each of which describes attribute information. The user needs to cast the returned object to SequentialCollection 2 and iterate over it to get AttributeDef.*/ public void checkOut(DataObject dob) throws UsageError, DatastoreAccessError; /*Checks out a document or folder item from the datastore. The user will have an exclusive update privilege of the item until he/she checks the item back in. However, other users can still read the item.*/ public void checkIn(DataObject dob) UsageError, DatastoreAccessError; /*Checks in a document or folder item previously checked out from the datastore. The user releases his exclusive update privilege of the item.*/ 9.2.2 DL parametric query string The syntax of DL parametric query string is as follows:
DL Parametric Query Syntax
SEARCH=([INDEX_CLASS=index_class_name,]
[MAX_RESULTS=maximum_results,]
[COND=(expression)]
[; . . . ]
) ;
[OPTION=([CONTENT=yes_no,]
[TYPE_QUERY=type_of_query,]
[TYPE FILTER=doc_and_or_folder]
)]
In the above syntax, words in uppercase are keywords. Lowercase words are parameters supplied by users, and are described below: index_class name--the name of the index class to be searched. If it is not specified, all available index classes will be searched. maximum_results--the maximum number of results to be returned. If MAX_RESULTS is 0 or not specified, all items that match the search criteria will be returned. expression--is a conditional expression composed of attribute_name and attribute_value pairs joined with relational operators, and two subsequent pairs are connected with logical operators. Each pair is enclosed in parenthesis. If a condition is not specified, an index class must be specified and it will return all contents in the index class up to the maximum result. (Similar to SELECT * in relational databases). Logical operators include NOT or , AND or &, and OR or .vertline.. Relational operators include EQ or.dbd., LEQ or <=, GEQ or >=, LT or <, GT or >, NEQ or <>, IN, NOTIN, LIKE, NOTLIKE, BETWEEN, and NOTBETWEEN. These last two operators take a pair of attribute values as a range. An example of a conditional expression is: (DLSEARCH_Date >= "1990") AND (DLSEARCH_Date <="1993") AND (DLSEARCH_Volume BETWEEN 1 3) For information about supported operators and examples of valid expressions refer to the LIBSEARCHCRITERIASTRUCT section of the IBM Manual, Digital Library Application Programming Reference Volume I, Doc. No. SC26-8652-00. The sequence of INDEX_CLASS, MAX_RESULTS, and COND is terminated by a semicolon and can be repeated for other index classes. yes_no--YES or NO. If the content is YES, the resulting documents and folders will have their contents retrieved. If the value is NO, only the document or folder ids are set. The default is YES if not specified. For example, content value YES would cause the resulting document or folder DDOs to have their Pid, object type, properties and all attributes set. If the content is NO, the attributes of the resulting DDOs are not set. Regardless of the content option, the Parts 4 attribute of a document is always set to a collection of parts with NO content. Similarly, the Folder 3 attribute is also set to a collection of DDOs with NO content. The part or DDO 28 content can be retrieved explicitly when needed.
type_of_query - the valid values are:
STATIC process the request using static SQL query. If no
static query is available the function uses a
dynamic query.
DYNAMIC process request using a dynamic SQL query.
BUILDONLY generates a static SQL query based on the input
search expressions and saves it for future
searches.
doc_and_or_folder takes the value of FOLDERDOC, to search for
folders and documents, DOC to search for
documents only, and FOLDER to search for
folders only.
9.3 DatastoreTS This class is a specific version of Datastore 9 to implement Text Search (TS) datastore or Text Miner. Essentially, TS provides text indexing and search mechanisms. It does not really store documents or folders. TS indexes text parts of documents and process search requests using this index. The results of a text query submitted to TS are DL item identifiers, which are keys to retrieve the actual documents from DL datastore. The execute() and evaluate() methods of DatastoreTS take text query strings expressed in text query language type. The syntax of this query string is described below. TextQuery 15 object accepts queries in this syntax, in fact TextQuery 15 object delegates the low level query processing tasks to DatastoreTS.
DatastoreTS.java
package COM.ibm.mm.sdk.server;
public class DatastoreTS implements Datastore
{
. . .
. . .
public DatastoreTS ( ) throws Exception;
public void connect ( String datastore_name,
String user_name,
String authentication,
String connect_string) throws Exception;
public void connect ( String server_name,
String port,
char communication_type) throws Exception;
public Object listServers( ) throws Exception;
public Object listEntity( ) throws UsageError;
};
9.3.1 Methods public void connect (String datastore_name, String userName, String authentication, String connect_string) throws Exception; Connects to the datastore. Authentication is the password. The username and authentication are for a digital library server. Datastore 9 name is the name of the search service. The connect string is optional. It is used to provide the communication type and port number as well as a list of library server, userid, authentication groupings. Below is a sample syntax of a connect string a user can supply.
Connect String
[COMMTYPE={T .vertline. P};PORT=portnumber;
LIBACCESS=(libraryserver,userid,auth; . . . )]
connect string /*addition connect parameters*/ COMMTYPE /*communication type. This can be set to T (TCPIP) or P (PIPES)*/ PORT /*port number. This parameter must be include if the COMMTYPE is specified.*/ LIBACCESS /*Library access group. If this parameter is passed the user should not specify the userName and authentication parameters in the connect method. Each library access group is related to a digital library server. If one library access group is specified the parenthesis are not needed. The user can specify one or more library access groups. Each library access group consist of the library server name (i.e. LIBSRVR2), userid and authentication of a digital library server which is where the text parts are stored.*/ There are different ways to use the connect method. Below is a list of the different ways to connect to Text Search. connect with datastore name (search service) connect with datastore_name (text search server) and specify connect_string with COMMTYPE and PORT. connect with datastore_name (search service), and specify the connect_string with LIBACCESS. connect with datastore_name (text search server) and specify connect_string with COMMTYPE, PORT and LIBACCESS. connect with datastore_name (search service) and userName, authentication. public void connect (String server_name, String port, char communication_type) throws Exception; /*Connects to the datastore. The server_name is the text search server. The user needs to specify the communication type _CTYP_TCPIP--TCPIP or CTYP_PIPES--PIPES) and port number.*/ public Object listServers() throws Exception; /*Returns a list of valid servers to connect to. The return value is a sequential collection containing ServerInfoTS objects describing the server.*/ public class ServerInfoTS { public String serverName(); public char serverLocation(); }; /*The user needs to cast the returned object to SequentialCollection 2 and iterate over it to get ServerInfoTS objects.*/ serverLocation will return either (_SRV_LOCAL (local server) or SRV_REMOTE (remote server)). public Object listEntity() throws UsageError; Returns a list of search index names in the current TS server. The user needs to establish a connection to the server before calling this method. public class IndexTS { public String indexName(); public String libraryId(); }; /*The indexName is the search index name and the libraryId is the library identifier of the search index. The return value is a sequential collection containing IndexTS objects. The user needs to cast the returned object to SequentialCollection 2 and iterate over it to get IndexTS objects.*/ 9.3.2 TS text query string The syntax of text query string is as follows:
TS Text Query Syntax
SEARCH=(COND=(text_search_expression)
) ;
[OPTION=([SEARCH_INDEX={search_index_name .vertline. (index_list) };]
[MAX_RESULTS=maximum_results;]
[TIME_LIMIT=time limit;]
[THES_NAME=thesaurus_index_name;]
[THES_DEPTH=depth_for_query_expansion;]
)]
In the above syntax, words in uppercase are keywords. Lowercase words are parameters supplied by users, and are described below: text_search_expression /*an expression composed of a sequence of text_search_criteria connected by logical operators.*/ [unary_operator] text_search_criteria [[binary_operator text_search_criteria] . . . ] /*Binary operators are AND or &, OR or .vertline., NAND and NOR. NOT is the only unary operator.*/ text_search_criteria is: {search_argument $DOC$ '{'search_argument '}'.vertline. $PARA$ '{'search_argument '}'.vertline. $SENT$ '{'search_argument '}' } /*The dollar sign delimits the keyword/option.*/
$DOC$ reserved word indicating that the search proximity expression
in search argument has a scope of the whole document
$PARA$ indicating that the search proximity expression in search
argument has a scope of a paragraph
$SENT$ indicating that the search proximity expression in search
argument has a scope of a sentence
search_argument is: {[$search_option$] {word/phrase}.vertline. [$search_option$] '{'free_text'}' }. . . /*The dollar sign delimits search_option. Options inside a pair of dollar sign are separated by comma, and can have the following values:*/
SYN search includes synonyms of the current term
L search terms are reduced to its their form
(PREC_LEMMA_FORMS).
V less precise matches are also considered as valid
(PREC_VARY_PHRASE).
P words in the current term must exactly match the
words in the document text (PREC_PRECISE)
E the term precision should be as exact as possible
for the current index (PREC_EXACT)
MC=symbol symbol to indicate a sequence of optional char-
acters or for a single optional word, i.e. wild
card character, usually an asterisk (*).
SC=symbol symbol to indicate a single required character,
usually a question mark (?). word is a word in the
specified search language, phrase is quoted
words, and free_text is words inside a pair of
braces {}.
search_index_name the name of one search index to be searched
index_list the list of search index names to be searched,
separated by commas
maximum_results the desired maximum number of results to be
returned
time_limit specifies the maximum processing time of the text
search server for a Boolean query or the Boolean
part of a hybrid query.
An example of a text search expression to search for documents contains the word "internet" and "DB2" in the same paragraph, and a free text "internet commerce is booming", and a word started with "Net", is as follows: $PARA$ {internet DB2} AND {internet commerce is booming} AND $MC=*$ Net* For more information about this terminology please refer to IBM Manual, Application Programming Reference Volume 2, Text Search Server, Doc. No. SC26-8653-00. 9.4 DatastoreQBIC This class is a specific version of Datastore 9 to implement the Query By Image Content (QBIC) datastore. QBIC provides image indexing and search mechanisms for Digital Library data. However, QBIC itself does not store any image data. QBIC indexes image parts in documents and process search requests using this index. The results of an image query submitted to QBIC are DL item identifiers, which are keys to retrieve the actual documents and their parts from a DL datastore. Since DatastoreFederated 37 implements the Datastore 9 interface, the user can use this same interface for accessing other datastores, such as DatastoreDL, DatastoreOD, etc. From this interface level, DatastoreFederated 37 is consistent with other datastores. The execute() and evaluate | ||||||