Query augmenting and refining (e.g., inexact access)

Information retrieval method with natural language interface

6778951

Abstract

A method for retrieving documents in a knowledge database is disclosed in which a natural language query question is mapped to a plurality of equivalent strings. Both relevancies in meaning and in wording pattern are taken into account in ranking the equivalent strings by introducing respective weight factors. A document coupled to the top ranked equivalent string is retrieved and displayed to the questioner.


Claims

What is claimed:

1. A method of retrieving documents in a database retrieval system having a knowledge database, the method comprising the acts of

a. receiving a query string inputted by a user into a natural language interface of said database retrieval system, said interface being coupled to a string source having a plurality of pre-stored strings, each of said pre-stored strings being coupled to one of said documents;

b. in response to said receiving act, detecting from said strings source a plurality of equivalent strings having essentially the same meaning as said query string;

c. in response to said detecting act, initially ranking said plurality of equivalent strings by a weighing correlation between said query string and each of said equivalent strings;

d. generating a first correlation value for the meaning of each of said plurality of equivalent strings;

e. in response to said act of generating a first correlation value, quantifying the correlation in meaning between said equivalent strings and said query string with a first factor;

f. generating a second correlation value for a wording pattern of each of said plurality of equivalent strings;

g. in response to said act of generating a second correlation value, quantifying the correlation in wording pattern between said equivalent strings and said query string with a second factor;

h. in response to said quantifying acts, ranking said equivalent strings by a combined correlation of meanings and wording patterns for each of said plurality of equivalent strings; and

i. in response to said act of ranking said equivalent strings, retrieving a document coupled to a selected and ranked equivalent string and displaying said document to said user.

2. The method of claim 1 wherein said first and second factors are adjustable by said user.

3. The method of claim 1 wherein said pre-stored strings are sample questions and said documents are answers to each of said sample questions.

4. The method of claim 1 wherein said first factor is larger than said second factor.

5. The method of claim 1 wherein said second factor is larger than said first factor.

6. The method of claim 1 wherein said selected equivalent string is a top ranked equivalent string.

7. The method of claim 1 further comprising a step of displaying one or more highest ranked equivalent strings, and said selected equivalent string is determined by said user by selecting among said displayed equivalent strings.

8. An information retrieval system, comprising:

a knowledge database having a document source comprising a plurality of documents and a string source comprising a plurality of pre-stored strings, each of said pre-stored strings being coupled to at least one of said documents;

a natural language interface for a user to input a query string, said interface being coupled to said string source;

a natural language processor for detecting equivalent strings having essentially the same meaning as said query string input at said natural language interface;

means for weighing a correlation in meaning between said query string and said equivalent strings by a first factor, and weighing a correlation in wording pattern between said query and said equivalent strings by a second factor, so as to obtain a combined correlation;

means for ranking said equivalent strings with said combined correlation; and

means for retrieving said documents coupled to a selected equivalent string.

9. The retrieval system of claim 8 further comprises means for said user to adjust said first and second factors.

10. The retrieval system of claim 8 wherein said selected equivalent string is a top ranked string.

11. The retrieval system of claim 8 further comprising means for displaying to said user one or more highest ranked equivalent strings, and means for said user to select one of them as said selected equivalent string so as to retrieve said document.

12. The retrieval system of claim 8 wherein said wording pattern comprises number of same words used in both said query string and said equivalent strings.


Description

TECHNICAL FIELD

This invention relates to information retrieval technologies, and more particularly to a method for retrieving documents by intelligently matching a query string to one or more pre-stored strings. A novel ranking method is employed for said intelligent matching.

BACKGROUND OF THE INVENTION

Frequently Asked Questions ("FAQs") are commonly presented by customers to a company. Due to the high repetition of FAQs, standard answers are usually pre-stored in a database retrievable by a query inputted into the system. A customer may present the question by dialing into the IVR system of the company, or may input the query at the website of the company.

Natural language queries are more acceptable to common customers as no special searching rules are required to be understood. A questioner can simply input a question (a query string) in natural language into the retrieval system and receive the prestored, correct answer. This is implemented by a mapping technique used inside the retrieval system. Specifically, a group of sample questions are pre-stored in the database, each with a corresponding answer. Upon receiving a query in natural language format, the system intelligently maps, by using a relatively complex, artificial intelligence algorithm, the query question to a pre-stored sample question which is coupled to an answer.

Due to the casual use of words in a natural language query string, it is important to improve the technique in successfully mapping the query string to a sample string. At present, natural language processing techniques are able to detect equivalent strings (strings that have the essentially the same meaning as the query string). They may detect the equivalent strings that are worded very differently from the query string and reject strings that are worded similar to the query string but have a different meaning. Usually more than one equivalent string is mapped to the same query string and ranked by meaning. An answer coupled to the top ranked equivalent string (i.e., that which has a meaning closest to the input string) will be retrieved and displayed to the questioner.

However, there is no technique to further distinguish equivalent strings from each other if they have the same ranking in meaning. Furthermore, the ranking among equivalent strings relies solely on either correlation in meaning or correlation in wording pattern, neither of which may be accurate enough and both of which have their limitations.

Therefore, there exists a need for improved techniques for the retrieval system to map the query strings and the prestored strings more accurately.

SUMMARY OF THE INVENTION

In the novel method of the present invention, both meaning and wording pattern are taken into consideration in ranking equivalent strings. Separate modules are utilized, a first for matching the meaning of an input string to prestored questions, and a second and independently operating module for matching word patterns of an input string to a prestored string. When plural strings are deemed to have an equivalent meaning, the word pattern of each is examined and the word pattern closest to a prestored word pattern is utilized.

In a preferred embodiment, correlation in meaning and correlation in wording pattern are weighted with different factors to obtain a combined correlation for each equivalent string, and the ranking is implemented based on the combined correlation thus obtained.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and further features and advantages of the present invention may be appreciated from the detailed description of preferred embodiments with reference to the accompanying drawings, in which:

FIG. 1 is a schematic illustration of an FAQ retrieval system;

FIG. 2 is an embodiment of the present invention; an.

FIG. 3 is another embodiment of the present invention.

DETAILED DESCRIPTION OF THE INVENTION

A typical FAQs retrieval system is schematically shown in FIG. 1. Database 3 comprises a question source 4 and an answer source 5. Sample questions are pre-stored in the question source 4 and each of the sample questions is coupled to one of the standard answers that is prestored in the answer source 5.

The natural language query questions are input at a natural language interface 1 which transmits the query to a natural language processor 2. The questions may be received via text over a data network, or via an audio signal over a data network or a telephone network. If the questions are received via an audio signal, then a speech recognition algorithm, many of which are commercially available, should be employed.

The natural language processor 2 runs to detect equivalent questions from the question source 4. These equivalent questions are ranked by their correlation in meaning to the query question input at the interface 1. Usually only the answer coupled to the top ranked equivalent question is retrieved and displayed to the user by a proper displaying means such as a monitor or printer. The system may also retrieve answers to some other equivalent questions other than the top ranked one, if necessary.

Natural language processors and recognition programs are available widely, and the details of how such programs are implemented are not critical to the present invention. In the present invention however, the output of such programs is not directly used, but instead is combined with a signal that measures the degree of correlation between the wording of an input string and that of potential matches.

Sometimes there may be more than one top ranked equivalent questions are detected. In such a situation, correlation in wording pattern may be taken into account to rank the equivalent questions, as shown in FIG. 2.

In FIG. 2, equivalent Questions 30 and 42 are found to have the same correlation in meaning to the query question. To further differentiate them, a step is introduced in which the wording pattern is also taken into consideration. For example, if the query question is "When can I get the payment?", the Question 30 is "How can I receive my money", and the Question 42 is "When can I receive the payment", finally the Question 42 is ranked over the Question 30 because of its higher correlation in wording pattern.

The wording pattern may comprise many factors. For example, the system may check how many words are used in both the query question and the equivalent question. Usually the words to be considered will not consider words such as the articles "a", "the" and the connectives such as "because", "therefore", etc. Moreover, the words will be considered in a stemmed form. For example, the words "paying", "payment", "pay" will all be considered as "pay".

A more complicated embodiment is shown in FIG. 3. In this embodiment, both correlation in meaning and correlation in wording pattern are taken into account in determining ranks for all the equivalent questions.

Presumably, a query question results in four equivalent questions Q30, Q42, Q48 and Q56 that are highest ranked ones from the correlation in meaning. The system generates a first correlation value or score for meaning, and a second correlation value or score for wording pattern.

Conceptually, the A scores (i.e., A1-A4) measure the correlation in meaning while the B scores (i.e., B1-B4) measure the correlation in wording pattern. These two score series, however, may not have a same weight in ranking the equivalent questions. In some situations correlation in meaning may be more important than correlation in wording pattern, while in other situations correlation in wording pattern may be more important.

Having this in mind, a weighing system is introduced to compare the relevant importance of the two scores. In particular, a weight factor X is introduced for A scores and a weight factor Y is introduced for B scores. The relative importance of the relevancies in meaning and in wording pattern is quantified by weight factors X and Y. After being weighed, correlation algorithm is made on the two score series to get final combined scores "C1", "C2", "C3", and "C4" respectively, which reflect both relevancies in meaning and wording pattern as well as their relative importance. Finally the equivalent questions are ranked in accordance with these final combined C scores. Thus, the ranking results are more accurate. Factions that may be taken into account include number of words, length of the string, etc.

Usually an answer coupled to the top ranked sample question is retrieved and displayed to the questioner.

As an alternative, the system may first display one or more highest ranked equivalent questions to the questioner who may select one among them to retrieve the answer. This, however, may sometimes be inconvenient to the questioner by introducing an additional step and the questioner has to read through several questions before he can determine which is the best. This may be impractical if the query interaction is implemented over a telephone. Nonetheless, the user can be prompted to select one of several questions as they are read.

In a preferred embodiment, the weight factors X and Y may be changeable by a questioner so as to fine tune the weight factors X and Y. This is advantageous as a questioner is able to interact with the system. If the questioner is not satisfied with a query result, he may change the weight factors X and/or Y to try for a better hit without changing his query question.

For example, if a query question uses more distinguishable keywords, the questioner may increase the weight factor Y so that the final ranking scores will rely more on the correlation in wording pattern than correlation in meaning. If the words used in a query question is less distinguishable, a higher weight factor X may produce a better result.

In the past, the correlation between a query question and the equivalent questions cannot be changed unless the questioner changes the query question. Therefore, in order to get a better hit, the questioner had to try many query questions until he got the right answer. With the present invention, it is much more convenient if the questioner may adjust the ranking by only changing the weight factors.

Though the above takes a FAQs answer retrieval system as an exemplary embodiment, it will be appreciated that the present invention is also applicable in other document or information retrieval systems and that modifications and variations will be possible to those with ordinary skill in the art without departing from the spirit of the invention. The scope of the invention is therefore intended to be solely defined in the claims.

«Previous Next»
Techniques for improved searching of electronically stored information Push content filtering