Semantic and cognition based image retrieval5930783Abstract A computer implemented method for searching and retrieving images contained within a database of images in which both semantic and cognitive methodologies are utilized. The method accepts a semantic and cognitive description of an image to be searched from a user, and successively refines the search utilizing semantic and cognitive methodologies and then ranking the results for presentation to the user. Claims What is claimed is: Description FIELD OF THE INVENTION
TABLE 1
______________________________________
Terminology Base
______________________________________
TV is-a Appliance
Computer is-a Appliance
Radio is-a Appliance
Man is-a Person
Woman is a Person
Airplane is-Transportation
Car is Transportation
Bird is an Animal
Dog is-an Animal
Cat is-an Animal
Tree is-a Plant
Flower is-a Plant
______________________________________
The terminology manager 216 maintains a terminology database 226 that is used for query relaxation and term heterogeneity resolution. An example of a terminology base is shown in Table 1. A helpful way to view the terminology base is as a forest that contains many trees in which each tree represents related concepts structured hierarchically. For instance, the terminology in Table 1 can be visualized as shown in FIG. 7. When presented with a term, the terminology manager 216 can return multiple matching alternatives with a corresponding ranking. The rankings are then used during the query processing to generate the final ranking of the results. The semantics-based query processor 218 provides answers to queries concerning the semantics information stored in images. The image semantics required for query processing is pre-extracted from the images using COIR 212 and the image semantics editor 214. The image semantics data 224 is stored in an image semantics database 224 resulting in a quick response time. An example of semantic query that can be answered by the semantics-based query processor 218 is: "Which objects are to the right of a tree in image
TABLE 2
______________________________________
Information maintained in SEMCOG and Its Usage
Information
Maintained By
Used By Purposes
______________________________________
Terminology
Terminology
Facilitator Query relaxation
database Manager
Image semantics
Image Semantics-based
Semantics-based
database semantics query query
editor processor processing
Image component
COIR COIR Image component
catalog identification
Image metadata
COIR COIR Cognition-based
store matching
Image store
COIR COIR and Image storage
facilitator
______________________________________
As can be readily appreciated by those skilled in the art, SEMCOG maintains a large body of information. The information maintained in SEMCOG and its usage is summarized in Table 2. Note that our inventive method and apparatus is designed as an open architecture that can be applied to other media, such as audio and video. The information maintained in SEMCOG can be categorized into two types as follows Media independent information, such as the terminology database 226 and the image semantics database 224; and Media dependent information, such as the image component catalog 222, the image metadata store 232, and the image store 242. Advantageously, our design permits seamless application to other media by replacing the image component catalog 222, the image metadata store 232, and the image store 242 with components that are more suitable to other media. With reference now to FIG. 6, there it shows the data structure of image semantics stored in SEMCOG. Here we layout the image semantics as tables. For ease of illustration, we assume that an image database ImageDB contains six images (I001, I002, I003, I004, I005 and I006), which are shown further in FIG. 8. A table of the objects that are contained within each image is listed in Table 3. Note that in operation, these objects are extracted from the images by the COIR system. As preferably implemented, the coordinates are pointers pointing to components in images. The properties of the objects maintained are described in Table 4. As can be seen, each object has a corresponding image, a set of coordinates describing its location within that image, and a set of semantics. In operation, the semantics are returned by the COIR system along with corresponding similarity ranking range between 0.0 and 1.0. For example and with reference to Table 4, object O201 is similar to a computer with ranking of 0.98 and it is similar to a television with ranking of 0.85. Finally, SEMCOG also maintains the spatial relationships between the objects in the database. Table 5 shows how these relationships are represented in SEMCOG. SEMCOG maintains this information about the images so that queries can be expeditiously processed off line as when performing cognition-based image matching.
TABLE 3
______________________________________
Objects contained within the images
Images Adjectives Objects
______________________________________
I001 O101 O102 O103
I002 O201 O202 O203
I003 Elegant O301 O302 O303
I004 O401 O402 O403 O404
I005 O501 O502
I006 O601 O602
______________________________________
We now present an overview CSQL (Cognition and Semantics-based Query Language), that is used as the query language in SEMCOG. In so doing, we introduce the terminologies and syntax in CSQL and illustrate its capabilities and expressive power. As can be appreciated, an image or an object can be interpreted either semantically or visually. A semantic interpretation of an image is the "real world meaning" of the image. A visual interpretation of an image, on the other hand, is how this image looks. Ideally the semantic interpretation and visual interpretation of an image should be consistent with one another. For example, the semantic interpretation (meaning) of an image of a man is "man". However, the visual interpretation of this image can be "human" if we are unable to determine whether the image is that of a man or a woman. Another example can be an image of a doll. In this example, the image's semantic interpretation is "doll" but its visual interpretation can be "woman" or "human" since it may not be possible to determine whether or not the object is animate. In SEMCOG, there are three types of entities. They are 1) semantic entities, 2) image entities; and 3) dual entities. These entities form the basis of all user queries and their definitions follow. Semantic entities are those entities having only semantic identities. More specifically, semantic descriptions of objects, or terminologies, are treated by SEMCOG as semantic entities. As such, a user may use such semantic descriptions to retrieve an image. Some examples of semantic entities are "man", "red car", and "plane". An image entity is an entity having only image identities. Besides the semantic descriptions, users may also use example images and/or sketches to describe what they are searching for. Such images and sketches constitute image entities. Referring to FIG. 9, image entities may be atomic, i.e. may contain a single object, or may be compound, i.e. may contain multiple objects. Dual entities are polymorphic entities which may behave as semantic entities or image entities as needed. A dual entity has both semantic identity and image identity, As such, when individual objects within a component image entity are extracted, these extracted objects are stored as dual entities, because such objects have both semantic identities and image identities as shown in FIG. 9. The image identity and semantic identity may not be consistent due to limitations of current image processing capabilities as previously described. Importantly, dual entities can never be inputs or outputs of the queries, instead they only exist during the query processing. The semantic identities are semantically described whereas the image identities are kept as pointers to the corresponding portions of the images. Note that the image identity of a dual object can not be compound. Importantly, SEMCOG allows users to present queries for semantic entities and image entities. For example, 1. "Retrieve the semantic means of the objects present in this image", and 2. "Retrieve all objects present in this image" are both valid queries. Note that, in example query 1 above, we are asking for the semantic identity of a dual entity and in the second query, we are asking for the image identity of the same dual entity. The components contained in images are dual entities, hence they must be converted into semantic entities or image entities to be returned as outputs. Advantageously, users of our inventive method and system may use the entities introduced above to describe different types of selection criteria. Query criteria based on the entities and operators involved may be categorized as follows: Semantics-based selection criteria: With semantics based selection, users specify semantic expressions to retrieve images. Since semantic terminologies are not standardized, in many cases, users may not be able to provide precise semantics of the entities. Hence, in SEMCOG, users are allowed to describe different levels of strictness for semantic relationships between entities using the following binary predicates: 1. --is: The is predicate returns true if and only if both of the arguments have identical semantics (man vs. man, car vs. car). 2. --is.sub.-- a: The is.sub.-- a predicate returns true if and only if the second argument is a generalization of the first one (car vs. transportation, man vs. human). 3. --s.sub.-- like: The s.sub.-- like predicate returns true if and only if both arguments are semantically similar (man vs. woman, car vs. truck). Note that arguments of the above predicates can be semantic, image, or dual entities. In all these cases, the comparison of the entities is made on their semantic identities because the type of comparison is defined by the operators. If the argument is a dual entity, its semantic identity is used. If the argument is an image entity, then its semantics must be extracted before the comparison. Consequently, a more expensive comparison results. If the image entity is compound (i.e. the entity has more than one component), then such a comparison may not be meaningful since the components may have different semantics. Cognition-based selection criteria: Cognition based selection may be used by users to retrieve images. In this regard, SEMCOG allows users to describe different cognition-based selection criteria between entities using the following binary predicates: 1. --i.sub.-- like: The two arguments to the i.sub.-- like predicate are either image entities or dual entities. In this case, the system compares the two entities based on their image identities. Since the semantics are not involved in the process, the input images can be compound as well. 2. --contains: The first argument of the predicate must be an image entity. The binary contains predicate returns true if the second argument, which is a dual entity, is contained in the first argument, which is an image entity. This predicate is mainly used to extract the components of a compound image. The reason why the first argument must be a compound image entity is because the granularity of the first argument must be in a higher level than the second argument, which is atomic. Scene-based selection criteria: As we discussed earlier, CSQL allows users to describe the spatial relationships between the entities that constitute an image. 3. --to.sub.-- the.sub.-- right.sub.-- of . . . in: This predicate takes two entities (semantic or dual) and an image entity, and it checks whether the first of the entities is to the right of the second entity and within the image entity. If the user provides a dual entity as one of the first two predicates, the system uses its semantic identity unless the dual entity is extracted from the given image entity. In that case, on the other hand, the system uses its image identity (and its coordinates) to check for the truth of the predicate. 4. --to.sub.-- the.sub.-- left.sub.-- of . . . in, above.sub.-- of . . . in. Please note that these spatial relationships are those that can be automatically identified by COIR 212. Other spatial relationships, such as in.sub.-- front.sub.-- of and behind, are more difficult tasks for any image processing technique to perform fully automatically. These predicates are similar to the to.sub.-- the.sub.-- right.sub.-- of . . . in predicate, and their predicate names are self explanatory. We will add an attached predicate, to find the adjacent objects within an image to SEMCOG after we add the corresponding functionality to COIR. Besides the above selection criteria and predicates, CSQL also allows users to elaborate on their query specifications using adjectives. There are two types of adjectives defined in CSQL: color adjectives and image adjectives. Color adjectives apply to semantic entities as well as their associated atomic image entities. An atomic image entity car can be further described as red car or white. On the other hand, image adjectives apply to the whole image entities including compound and atomic image entities. An example of image adjectives is (elegant imageID). The elegant adjective has been implemented in COIR 212. An image is considered elegant based on the color combination of its regions. Dual entities can have both types of the adjectives; the color adjectives apply to their semantic identities and the image adjectives apply to their image identities. Please note that in the work, we only consider those adjectives that can be automatically identified by COIR 212, such as colors. Other adjectives such as large and small are more difficult to identify since these adjectives are relative, not absolute. Additionally, CSQL allows nested queries through the use of two filtering predicates: best.sub.-- of.sub.-- the and one.sub.-- of.sub.-- the. The best.sub.-- of.sub.-- the predicate filters out the best matching answer of a query. Note that the query must have only one free variable. The best.sub.-- of.sub.-- the can be used for reducing the number of queries generated by query relaxation. On the other hand, the one.sub.-- of.sub.-- the predicate returns all the answers to the input query. Again, the input query must have only one free variable. In summary, SEMCOG allows its users to retrieve images based on the following criteria: Image similarity. In many image database applications, users are interested in similar-based retrievals where user provides a base image and the image database returns similar one(s). In a preferred embodiment, a user may either pick the base images from one or more image pools or they may draw sketches to describe what they expect as the result. This is a cognition-based selection criteria. Containment: In some other applications, a user may desire to select those images that contain particular entities. An entity of interest may be an atomic image entity, a sketch of an object, or it may be a semantic description of an object. In such queries, it is usually insufficient to compare the entity with the images. Instead, the system must evaluate the semantics of the user inputs, and it must retrieve images that satisfy the corresponding semantic description. An example of this type of queries is (the underlined words correspond to the terms or images provided by the user): "select image P where (P contains X) and (X is man) and Xi.sub.-- like <<a sketch>>)." Spatial relationship: Spatial relationship may be used by a user to describe the spatial relationships between the entities that should be available in the retrieval images. Importantly, the user may be interested in not only the objects which are available within an image, but also in the spatial relationships of these objects and their properties. For example, a real estate broker may ask a query to retrieve all images of candidate real estate in which there is a tree to the right of the house. In the most basic approach, the real estate broker may formulate the above query by drawing a sketch of a house and a tree that is to the right of the house. SEMCOG then would use the image similarity criterion to find all images that looks like the one he drew. However, with this approach, the success of the search would highly depend on the artistic capabilities of the real estate broker. Note that this is an expensive operation since it is a pure cognition-based approach. In contrast, using CSQL the above query can be stated as follows: "select image P where (X is house) and (Y s.sub.13 like tree) and (Y to.sub.-- the.sub.-- right.sub.-- of X)". A user can use a combination of all three types of selection criteria discussed above with more descriptive adjectives to formulate a more complicated query such as: "Select all images where there is a man standing to the left of a red car, where the car looks like <<a sketch>>." In the above query, the user not only verbally describes the objects and their relationships, but also provides a sketch, i.e. more detailed description of what he expects to see. Such a query can be formulated in CSQL as: "select image P where (P contains X) and (P contains Y) and (X is to.sub.-- the.sub.-- right.sub.-- of Y in P) and (X is man) and (Y is red car) and (Y i.sub.-- like pictureID)." where pictureID is a pointer to a sketch provided by the user. With this background in place, we may now describe the query language used in SEMCOG in greater detail. We have introduced the entities and predicates used in SEMCOG and CSQL. Further examples showing the use of this query language and the screen dumps of the user interface based on this query language are shown in FIGS. 10 and 13. SEMCOG provides an "SQL-like" query language (CSQL) which lets a user to query for the images in a database or for the objects within those images. It allows three types of queries: semantics-based, cognition-based, and relationship-based. SEMCOG also allows users to create complex queries which may involve two or more of the above mentioned query types. In addition to these, users can create nested queries. Table 6 provides an overview of the predicates that SEMCOG uses for handling semantics- and cognition-based image retrieval. In the columns 1.sup.st argument and 2.sup.nd argument, we use s, d, and i to indicate the types of entities are allowed. The s, d, and i stand for semantic entity, dual entity, and image entity respectively. In the following section we describe the mechanics of the query processing and we show how these predicates are used and processed.
TABLE 6
______________________________________
Predicates and their usage
1.sup.st 2.sup.nd 3.sup.rd
Predicate
argument argument argument
Discussion
______________________________________
is s/d/i s/d/i semantics are
compared
is.sub.-- a
s/d/i s/d/i semantics are
compared
s.sub.-- like
s/d/i s/d/i semantics are
compared
i.sub.-- like
d/i d/i images are compared
contains
i d semantics of dual is
searched within image
to.sub.-- the.sub.--
s/d s/d i positions of entities are
right.sub.-- of compared: if argument
. . . in is a semantic entity or
a dual entity outside of
image i semantics of
the argument is used,
otherwise image of the
dual is used.
to.sub.-- the.sub.-- left
s/d s/d i as above
.sub.-- of . . . in
above.sub.-- of
s/d s/d i as above
. . . in
below.sub.-- of
s/d s/d i as above
. . . in
best.sub.-- of.sub.-- the
query nested
one.sub.-- of.sub.-- the queries are processed:
these are not regular
predicates. The
arguments of these
quasi-predicates are
queries themselves.
______________________________________
We now use examples to show the query processing in SEMCOG. The databases used for the examples are shown in FIG. 8 and tables 3, 4, and 5 presented previously. EXAMPLE 1 Retrieve all images in which there is a TV. In CSQL syntax, this query can be posed as follows: select image X where X contains TV The processing of this query involves the following steps: Step 1: Facilitator 210 asks the semantic query processor 218 to find all images that contain a TV. Step 2: The semantics-based query processor 218 checks the semantic information database 224 to find all the images that contain a TV by checking with Table 4. The result of this step, Ids of the matching images, are returned to the facilitator 210 with the corresponding similarity ranking values as follows:
______________________________________
Sim- Sim-
Ob- Im- Seman- ilar-
Seman- ilar-
ject age tics ity tics ity . . .
Coord Adj.
______________________________________
O101 I001 Compu- 0.93 TV 0.65 . . .
ter
O201 I002 Compu- .98 TV 0.85
ter
O301 I003 TV 0.85
______________________________________
Step 3: The facilitator 210 returns the final results to the user. The final results are images with associated degrees of confidence as shown in FIGS. 14(a)-14(e). For FIG. 14(a) (image I003), the ranking is 0.85. For FIG. 14(b) (image I002), the degree of confidence is 0.85. For FIG. 14(c) (image I001), the degree of confidence is 0.65. Step 4: If the user wants to see more results, then the facilitator 210 asks the semantics-based query processor 218 to relax the term TV. Step 5: The semantics-based query processor 218 relaxes the comparison with TV and returns a new set of matching answers. EXAMPLE 2 Retrieve all images in which there is a man and a transportation and this person is to the right of the transportation. This query can be posed as follows: select image X where X contains Y and X contains Z and Y is a man and Z is.sub.-- a transportation and Y to.sub.-- the.sub.-- right.sub.-- of Z in X There are multiple ways to process this query. One such possible execution plans is as follows: The facilitator 210 Step 1: creates one image entity for X (ix), two dual entities for Y and Z (dy and dz respectively), and links these dual entities to the image entity ix. Step 2: assigns man to the semantic component of dy. Step 3: consults the semantics manager 216 for a list of transportations, and assigns the results to the semantic component of dz.
______________________________________
transportation
Airplane
Car
______________________________________
Step 4: passes ix, dy and dz to the semantics-based query processor 218 for the list of triples that satisfies the to.sub.-- the.sub.-- left.sub.-- of . . . in criterion. The query processor 218 returns answers as long as both the spatial criterion and the links created in Step 1 are simultaneously satisfied.
______________________________________
Sim- Sim-
Ob- Im- Seman- ilar-
Seman- ilar-
ject age tics ity tics ity . . .
Coord.
Adj.
______________________________________
O103 I001 Man 0.75 Woman 0.69 . . .
O202 I002 Man 0.88 Statue 0.45 . . .
O203 I002 Man 0.76 Statue 0.55 . . .
O403 I004 Man 1.00 . . .
O602 I006 Man 0.79 Chim- 0.62 . . .
panzee
O401 I004 Air- 0.60 Bird 0.30 . . .
plane
______________________________________
Here, O403 and O401 satisfy the link created in Step 1, i.e. both of them the same image, 1004. They also satisfy the spatial constraint:
______________________________________
Image Orientation Objects
______________________________________
I004 horizontal O401 O402 O403 O404
______________________________________
Hence, as a result of this step, ix gets bound to I004 with confidence value 0.6. Step 5: returns the Ids of the images returned by the semantics-based query processor 218. Since the previous step returned only I004, the result of the query is as shown in FIG. 15. In FIG. 15 (image I004), the ranking is 0.6. If, on the other hand, the above query was specified as select image X where X contains Y and X contain Z and Y is human and Z is.sub.-- a.sub.-- transportation and Y to.sub.-- the.sub.-- right.sub.-- of Z in X (by replacing man with human), then the query processing would be as follows: Step 1: creates one image entity for X (ix), two dual entities for Y and Z (dy and dz respectively), and it links these dual entities to the image entity ix. Step 2: assigns human to the semantic component of dy. Step 3: consults the semantics manager 216 for a list of transportations, and assigns the results to the semantic component of dz.
______________________________________
Transportation
Airplane
Car
______________________________________
Step 4: Facilitator 210 passes ix, dy, and dz to the semantics-based query processor 218 for a list of triples that satisfies the to.sub.-- the.sub.-- left.sub.-- of . . . in criterion. The query processor returns answers as long as both the spatial criterion and the links created in Step 1 are simultaneously satisfied.
______________________________________
Sim- Sim-
Ob- Im- Seman- ilar-
Seman- ilar-
ject age tics ity tics ity . . .
Coord.
Adj.
______________________________________
O303 I003 Human 1.00 . . .
O401 I004 Air- 0.60 Bird 0.30 . . .
plane
______________________________________
Now, since O303 and O401 are in different images, the link constraint imposed by step 1 is not satisfied. Hence, the result of this step is an empty set. Step 5: Since there are no matching answers, facilitator 210 replaces the statement "Y is human" with "Y is.sub.-- a human". Note that at this point there are many ways to relax the given query. In this example, we picked one of the possible relaxations. Step 6: Facilitator 210 consults the semantics manager 216 for a list of humans, and assigns the results to the semantic component of dy.
______________________________________
Human
Man
Woman
______________________________________
Step 7: Next, facilitator 210 passes in, dy and dz to the semantics-based query processor 218 for a list of triples that satisfies the to.sub.-- the.sub.-- left.sub.-- of . . . in criterion. The query processor 218 returns answers as long as both the spatial criterion and the links created in Step 1 are simultaneously satisfied.
______________________________________
Sim- Sim-
Ob- Im- Seman- ilar-
Seman-
ilar-
ject age tics ity tics ity . . .
Coord.
Adj.
______________________________________
O401 I004 Air- 0.60 Bird 0.30 . . .
plane
O103 I001 Man 0.75 Woman 0.69 . . .
O202 I002 Man 0.88 Statue
0.45 . . .
O203 I002 Man 0.76 Statue
0.55 . . .
O303 I003 Human 1.00 . . .
O403 I004 Man 1.00 . . .
O404 I004 Woman 1.00 . . .
O501 I005 Woman 0.82 Statue
0.34 . . .
O601 I006 Woman 0.95 Statue
0.66 . . .
O602 I006 Man 0.79 Chim- 0.62 . . .
panzee
______________________________________
Here, the object pairs O403-O401, and O404-O401 satisfy the link constraint imposed in Step 1, i.e. both of them belong to the same image, 1004. Both pairs also satisfy the spatial constraint:
______________________________________
Image Orientation Objects
______________________________________
I004 horizontal O401 O402 O403 O404
______________________________________
As a result of this step, ix gets bound twice to I004 with confidence value 0.6 (once for O403-O401 and once for the O404-O401 pair). Please note that theoretically the degree of confidence should be equal to or less than 0.6 since we relax is to is.sub.-- a.
______________________________________
Image Orientation Obects
______________________________________
I004 horizontal O401 O402 O403 O404
______________________________________
Step 8: The facilitator 210 removes the duplicates, and it returns the Ids of the images returned by the semantics-based query processor. Since the previous step returned two I004s, the result of the query is as shown in FIG. 15. In FIG. 15 (image I004), the ranking is 0.6. EXAMPLE 3 Retrieve all images in which there is a green tree and there is an object which looks like this drawing. This query can be stated as shown in FIG. 16 (Note that in this example we show not only how the facilitator 210 processes the query, but also how the binding of the variable entities are performed). Step 1: The facilitator 210 uses the semantics-based query processor 218 to find all images that contain a green tree, and creates image entities to hold them.
______________________________________
Sim- Sim-
Ob- Im- Seman- ilar-
Seman-
ilar- Co-
ject age tics ity tics ity . . .
ord. Adj.
______________________________________
O302 I003 Tree 1.00 . . . Green
______________________________________
ip.fwdarw. <I003,1
Here, 1 corresponds to the confidence in the generation of the entity. Since, in our example Image1 definitely contains a green tree (FIG. 6), the confidence value for ip is 1. Step 2: The facilitator 210 uses, again, the semantics-based query processor 218 to find all objects in these images and stores each of these objects in a separate dual entity linked to ip.
______________________________________
Image Orientation Objects
______________________________________
I003 Elegant O301 O302 O303
______________________________________
Sim- Sim-
Ob- Im- Seman- ilar-
Seman-
ilar- Co-
ject age tics ity tics ity . . .
ord. Adj.
______________________________________
O301 I003 TV 0.85 . . .
O302 I003 Tree 1.00 . . . Green
O401 I004 Person 1.00
______________________________________
dy.sub.1 .fwdarw.<ip,<tv.sub.-- image.sub.-- ptr,tv,1>> dy.sub.2 .fwdarw.<ip,Itree.sub.-- image.sub.-- ptr,/green/tree,1>> dy.sub.3 .fwdarw.<ip,<person.sub.-- image.sub.-- ptr,person,1>> Step 3: The facilitator 210 sends the image components of the dys to COIR 212 for matching with the image that the user provided. In our example, COIR 212 returns a similarity value of 0.97 for the image pointed by tv.sub.-- image.sub.-- ptr, and returns value close to 0.0 for the other two. Step 4: If some of the image components stored in dys are matched by COIR 212 to the provided image, then facilitator 210 succeeds, and returns the corresponding ip as a result. result.fwdarw.<I003, 0.97> The image retrieved is as shown in FIG. 14(a). In FIG. 14(a)(image I003), the ranking is 1.0. Those skilled in the art will quickly appreciate that SEMCOG may support a variety of interfaces. One such interface is shown in FIG. 10 where an HTML-based user interface is used for the main menu of image queries. Users can use this interface to specify CSQL directly. The windows for input sketches and specifying images will pop up by pressing familiar buttons or other controls. In a similar manner, FIG. 11 shows the interfaces implemented using Java for input sketches. FIG. 12 show the interface for input images by specifying the file path. Advantageously, SEMCOG also supports a visually richer user query interface called IFQ (In Frame Query) that provides a natural image query user interface. FIG. 13 shows how the query "Retrieve all images in which there is a man to the right of a car and this man looks like this image" is generated using IFQ. With IFQ, the user uses a bullet to represent an object and then associates additional descriptors to describe the entity. In this example, the user uses "i.sub.-- like<image>" and "is Man" to describe the object on the right and "is car" to describe the object on the left. The user then specify relations between these two objects. These queries using CSQL and IFQ are equivalent. As shown on the button on FIG. 13, the CSQL corresponding to IFQ specified by the user is generated by SEMCOG automatically. Please note that the user needs not be aware of the syntax and variable specification in CSQL since these are handled by the IFQ interface. We believe that IFQ provides a more natural way to pose semantics and cognition-based queries. Of course, since our SEMCOG technique is computer implemented, computer systems that are capable of operating our technique are well known. Specifically, a computer, having a data processing system with processor, connected to a display monitor, user interface device such as a mouse, pen or digitizing tablet and keyboard in a conventional manner are appropriate to operate our inventive technique. Operating system, text/graphic interactive user interface and corresponding programs which provide the functionality of our technique, will reside and operate within the data processing system. Clearly, it should now be quite evident to those skilled in the art, that while the invention was shown and described in detail in the context of a preferred embodiment, and with various modifications thereto, a wide variety of other modifications can be made without departing from the scope of the inventive teachings which should only be limited by the following claims.
|
Same subclass Same class Consider this |
||||||||||