Management and analysis of document information text

Abstract

An interactive system for analyzing and displaying information contained in a plurality of documents employing both term-based analysis and conceptual-representation analysis. Particulars of the invention are especially effective for analyzing patent texts, such as patent claims, abstracts and other portions of a patent document.

Claims

What is claimed is:

1. A method of analyzing and displaying information regarding a plurality of documents, the method comprising the steps of:

generating a set of N different representations of each document, a given representation being designated the i.sup.th representation where i is an integer in the range of 1 to N inclusive;

for selected pairs of documents, determining N utility measures, a given utility measure being designated the i.sup.th utility measure where i is an integer in the range of 1 to N inclusive, the i.sup.th utility measure being based on the respective i.sup.th representations of the documents in that pair; and displaying a scatter plot in an area bounded by N non-parallel axes, a given axis being designated the i.sup.th axis where i is an integer in the range of 1 to N inclusive, where each selected pair is represented by a point in N-space having a coordinate along the i.sup.th axis equal to the i.sup.th utility measure.

2. The method of claim 1 wherein the set of N different representations comprises:

a first representation being a conceptual-level representation; and

a second representation being a term-based representation.

3. The method of claim 2 wherein the utility measure is a proximity score.

4. A method of analyzing and displaying information regarding a plurality of documents, the method comprising the steps of:

generating first and second different representations of each document;

for selected pairs of documents, determining (a) a first utility measure based on the respective first representations of the documents in that pair, and (b) a second utility measure based on the respective second representations of the documents in that pair; and

displaying a scatter plot in an area bounded by first and second non-parallel axes where each selected pair is represented by a point having a first coordinate along the first axis equal to the first utility measure and a second coordinate along the second axis equal to the second utility measure.

5. The method of claim 4 wherein:

the first representation is a conceptual-level representation; and

the second representation is a term-based representation.

6. The method of claim 4 wherein:

the first representation is a subject vector; and

the second representation is a word vector.

7. The method of claim 4 wherein each of the selected pairs consists of a particular document in the plurality of documents and a different respective one of the remaining documents in the plurality of documents.

8. The method of claim 5 wherein the utility measure is a proximity score.

9. The method of claim 4 wherein the documents are publications.

10. The method of claim 4 wherein the documents are articles from journals.

11. The method of claim 4 wherein the documents are attributable to a product.

12. The method of claim 4 wherein the documents are contained in a split dataset for making comparisons between collections of documents.

13. The method of claim 4 wherein the documents are different parts of patents.

14. The method of claim 13 wherein the different parts of patents include claims.

15. The method of claim 13 wherein the different parts of patents include a detailed description.

16. The method of claim 13 wherein the different parts of patents include an abstract.

17. The method of claim 13 wherein the different parts of patents include a summary.

18. The method of claim 13 wherein the different parts of patents include a Background of Invention.

19. A method of analyzing information regarding a plurality of documents, each having a unique document index, the method comprising the steps of:

parsing each document into a plurality of elements;

generating a first representation of each of said elements; and

for selected pairs of documents, comprised of a first document and a second document, determining a first utility measure based on the respective first representation

of the plurality of elements for the documents in that pair.

20. The method of claim 19, wherein said plurality of elements are in a hierarchical relationship, further comprising the step of:

displaying a representation of each of said plurality of elements reflecting said hierarchical relationship.

21. The method of claim 19 wherein said elements comprise patent claims.

22. The method of claim 20 wherein said representation is a hypertext link.

23. The method of claim 20 wherein said representation is a depiction of a sequence of said plurality of elements organized to reflect said hierarchical relationship.

24. The method of claim 19, wherein said plurality of elements are in a hierarchical relationship, further comprising the step of:

selecting a particular element from said plurality of elements as a basis for further analysis.

25. The method of claim 19 wherein the parsing step produces a transitive closure of said plurality of elements.

26. The method of claim 19 wherein the elements are claims and the parsing step comprises the steps of:

reading in text;

determining whether a new claim has begun;

tokenizing said text to extract a plurality of tokens;

adding said plurality of tokens to a word list for the claim; and

scanning said tokenized text for tokens which indicate a reference to a different claim.

27. The method of claim 19 further comprising the step of displaying a plot in an area bounded by first and second non-parallel axes where each selected pair is represented by a point having a first coordinate along the first axis and a second coordinate along the second axis.

28. The method of claim 27 further comprising the steps of:

generating a second representation of each of said elements;

for the selected pairs of documents, determining a second utility measure based on the respective second representation of the plurality of elements for the documents in that pair; and

wherein in the displaying step, the plot is a scatter plot, the first coordinate is equal to the first utility measure and the second coordinate is equal to the second utility measure.

29. The method of claim 19 further comprising the steps of:

generating a second representation of each of said elements;

for the selected pairs of documents, determining a second utility measure based on the respective second representation of the plurality of elements for the documents in that pair.

30. The method of claim 27 further comprising the steps of:

wherein in the displaying step, the plot is a 2 dimensional visualization, the first coordinate is equal to the unique document index of the first document of a pair of documents and the second coordinate is equal to the unique document index of the second member of a pair of documents, and an icon representing the first utility measure is plotted for each pair of documents.

31. The method of claim 19 further comprising the step of displaying a plot in an area bounded by first, second and third non-parallel axes where each selected pair is represented by a point having a first coordinate along the first axis, a second coordinate along the second axis and a third coordinate along the third axis.

32. The method of claim 31 further comprising the steps of:

wherein in the displaying step, the plot is a 3 dimensional visualization, the first coordinate is equal to the unique document index of the first document of a pair of documents and the second coordinate is equal to the unique document index of the second member of a pair of documents, and the third coordinate is equal to the first utility measure, and an icon representing the first utility measure is plotted for each pair of documents.

33. The method of claim 30 wherein said first utility measure is a combination of N utility measures.

34. The method of claim 32 wherein said first utility measure is a combination of N utility measures.

35. The method of claim 28, for an additional document further comprising:

parsing said additional document into a plurality of elements;

generating a first representation of each of said elements from the parsing step;

for selected pairs of documents drawn such that a first member of the pair is the additional document and a second member of the pair is from said plurality of documents, determining a first utility measure based on the respective first representation of the plurality of elements for the documents in that pair;

generating a second representation of each of said elements from the parsing step;

for selected pairs of documents drawn such that a first member of the pair is the additional document and a second member of the pair is from the plurality of documents, determining a second utility measure based on the respective second representation of the plurality of elements for the documents in that pair; and

wherein in the displaying step, the plot is a scatter plot, generating an overlay plot in contrasting color to the scatter plot, the first coordinate equal to the first utility measure computed on the pairs of documents including the additional document the second coordinate is equal to the second utility measure computed on the pairs of documents including the additional document.

36. The method of claim 35 wherein said additional document is a textual query entered by a user.

37. The method of claim 35 wherein:

the first representation is a conceptual-level representation; and

the second representation is a term-based representation.

38. The method of claim 37 wherein:

the first representation is a subject vector; and

the second representation is a word vector.

39. The method of claim 19 wherein said step of determining a first utility measure further comprises the steps of:

determining a first intermediate utility measure;

determining a second intermediate utility measure;

selecting a particular intermediate utility measure from said first intermediate utility measure and said second intermediate utility measure as said first utility measure.

40. The method of claim 29 wherein said step of determining a second utility measure further comprises the following steps:

determining a third intermediate utility measure;

determining a fourth intermediate utility measure;

selecting a particular intermediate utility measure from said third intermediate utility measure and said fourth intermediate utility measure as said second utility measure.

41. The method of claim 39 wherein:

said first intermediate utility measure is a combination of a first similarity measure for said first document element and said first similarity measure for said second document element and a first normalization constant; and

said second intermediate utility measure is a combination of a first similarity measure for said second document element and said first similarity measure for said first document element and a second normalization constant.

42. The method of claim 40 wherein:

(a) said third intermediate utility measure is a combination of a second similarity measure for said first document element and said second similarity measure for said second document element and a first normalization constant; and

(b) said fourth intermediate utility measure is a combination of said second similarity measure for said second document element and said second similarity measure for said first document element and a second normalization constant.

43. The method of claim 41 wherein said first similarity measure is a word weight vector.

44. The method of claim 42 wherein said second similarity measure is an SFC weight vector.

45. The method of claim 19 wherein:

said pairs of documents further comprises a first document and a second document,

said first document is a dependent claim, x, depending from an independent claim, X, and

said second document is a dependent claim, y, depending from an independent claim, Y,

said determining a first utility measure further comprises the following steps:

determining a first intermediate utility measure;

determining a second intermediate utility measure;

combining said first intermediate utility measure and said second intermediate utility measure.

46. The method of claim 29 wherein:

said pairs of documents further comprises a first document and a second document,

said first document is a dependent claim, x, depending from an independent claim, X, and

said second document is a dependent claim, y, depending from an independent claim, Y,

said determining a second utility measure further comprises the following steps:

determining a third intermediate utility measure;

determining a fourth intermediate utility measure;

combining said third intermediate utility measure and said fourth intermediate utility measure.

47. The method of claim 45 wherein:

(a) said first intermediate utility measure is a combination of a first similarity measure for said first document element, x, and said first similarity measure for said second document element Y; and

(b) said second intermediate utility measure is a combination of said first similarity measure for said first document element, X, and said first similarity measure for said second document element, Y.

48. The method of claim 46 wherein:

(a) said third intermediate utility measure is a combination of said second similarity measure for said first document element, x, and said second similarity measure for said second document element Y; and

(b) said fourth intermediate utility measure is a combination of said second similarity measure for said first document element, X, and said second similarity measure for said second document element, Y.

49. The method of claim 47 wherein said first similarity measure is a word weight vector.

50. The method of claim 48 wherein said second similarity measure an SFC weight vector.

51. The method of claim 45 wherein said step of combining comprises an averaging.

52. The method of claim 46 wherein said step of combining comprises an averaging.

53. A computer program product which analyzes and displays information regarding a plurality of documents comprising:

code for generating first and second representations of each document;

code for determining, for selected pairs of documents;

(a) a first utility score based on the respective first representations of the documents in that pair, and

(b) a second utility score based on the respective second representations of the documents in that pair;

code for displaying a scatter plot in an area bounded by a first and a second non-parallel axes where each selected pair is represented by a point having a first coordinate along the first axis equal to the first utility score and a second coordinate along the second axis equal to the second utility score; and

a computer readable storage medium for storing the codes.

54. A method of analyzing patent documents comprising the steps of:

providing a dataset containing a plurality of patent documents;

identifying within each patent document a portion of said document containing a set of claims;

generating a first representation of each set of claims within said plurality of patent documents; and

determining a first utility measure of at least one claim within at least one set of claims based upon similarity of said at least one claim with a query document.

55. The method of claim 54 wherein said query document is a concept query, patent or claim.

56. The method of claim 54 further comprising the step of displaying on a computer screen a ranking of a plurality of claims contained within said patent documents based upon said first utility measure associated with each of said plurality of claims, said screen including a claim number and rank number for each of said plurality of claims.

57. The method of claim 56 further comprising the step of providing a link at said claim number to a full-text display of an associated claim.

58. The method of claim 57 further comprising the step of providing a link at said rank number to a side-by-side textual display of said associated claim and said query document.

59. The method of claim 54 further comprising the step of parsing each set of claims to identify each individual claim within said each set and all claims referenced by said each individual claim.

60. The method of claim 54 further comprising the steps of:

generating a second representation of each set of claims within said plurality of patent documents; and

determining a second utility measure of said at least one claim within said at least one set of claims based upon similarity of said at least one claim with said query document.

61. The method of claim 60 further comprising the step of displaying on a computer screen a ranking of said plurality of patent documents based upon said first and second utility measures associated with claims of each of said patent documents, said screen including a rank number for each of said plurality of patent documents.

62. The method of claim 61 further comprising the step of providing a link at said rank number to a side-by-side textual display of an associated patent document and said query document.

63. The method of claim 62 further comprising the step of providing a link at a screen icon to a textual display of a ranked listing of matching claims of said associated patent document and said query document.

64. A method of analyzing a patent document comprising the steps of:

providing a dataset containing at least one patent document;

identifying within said at least one patent document a portion of said document containing a set of claims;

parsing said set of claims to identify an individual claim within said set and all claims referenced by said individual claim; and

displaying on a computer screen a link for each claim referenced by said individual claim.

65. The method of claim 64 further comprising the step of displaying on said computer screen at least a portion of said individual claim.

66. The method of claim 65 wherein activation of said link for a particular claim referenced by said individual claim produces a full text display of said particular claim.

67. The method of claim 66 wherein said link is a claim number.

68. The method of claim 67 wherein said full text display of said particular claim comprises a transitive closure of said particular claim.

Description

COPYRIGHT NOTIFICATION

A portion of the disclosure of this patent document contains material which is subject to copyright protection. The copyright owners have no objection to the facsimile reproduction, by anyone, of the patent document or the patent disclosure, as it appears in the patent and trademark office patent file or records, but otherwise reserve all copyright rights whatsoever.

BACKGROUND OF THE INVENTION

The present invention relates to information management and, more particularly to the management and analysis of document information text.

We live in the information age. How prophetic the statement of a major computer manufacturer which said "It was supposed to be the atomic age, instead it has turned out to be the information age." Prophetic both in the impact of the age, as well as its potential for beneficial and deleterious effects on humankind. Faced with an explosion of information fueled by the burgeoning technologies of networking, inter-networking, computing and the trends of globalization and decentralization of power, today's business manager, technical professional and investment manager are faced with the need for careful, accurate and timely analysis of the deluge of information underlying their everyday decisions. Several factors underlie this need for prompt information analysis. First, in an era of ever tighter cost controls and budgetary constraints, companies are faced with a need to increase their operational efficiency. In doing so, they face the need to assimilate large amounts of accounting and financial information, both concerning their internal functioning as well as their position in the market place. Second, the omnipresent factor of litigation which may cost or earn a company billions of dollars. The outcome of such contests is often determined by which side has access to the most accurate information. Third, the drive for greater economies of scale and cost efficiencies spurs mergers and acquisitions, especially in high technology areas. The success of such activity is highly dependent upon who has superior abilities to assimilate information. Fourth, the explosive growth of technology in all areas, especially in biotechnology, computing and finance, brings with it the need to access and comprehend technical trends impacting the individual firm. Fifth, the globalization of the market place in which today's business entities find themselves brings with it the need to master information concerning a multiplicity of market mechanisms in a multiplicity of native languages and legal systems. Sixth, the decentralization of large industrial giants has led to the need for greater cross-licensing of indigenous technologies; requiring that companies discern precisely the quantity and kinds of technology being cross-licensed.

Faced with the increasing importance of successful analysis of a burgeoning information stockpile, today's business professional is faced, as never before, with a need for tools which not only find information, but find the correct information, as well as, assist the user in drawing conclusions and perceiving the meaning behind the information resources discovered.

The most typical information analysis tool available today is a database of text or images which is searched by a rudimentary search engine. The user enters a search query consisting of specific key words encoded in a boolean formalism. Often the notation is so complex that trained librarians are needed to ensure that the formula is correct. The results of database searches are a list of documents containing the key words the user has requested. The user often does not know the closeness of the match until each reference cited by the search engine is studied manually. There is often no way to search different portions of documents. Finally, the output of this process is a flat amalgam of documents which has not been analyzed or understood by the system performing the search.

The user who turns to an automated information analysis system is seeking not merely a collection of related documents, but the answers to critical questions. For example,

"Are there any issued patents that are so close to this invention proposal that they might represent a potential infringement problem?"

"Are the resources of company X complimentary to our own company such that we should consider a merger with company X?"

"Of the court cases decided in California last year, how many of them involved a sexual harassment charge?"

"What companies exist as potential competitors in the market place for our planned product?"

Current analysis tools demonstrate themselves to be ineffective when faced with these types of issues. What is needed is an information analysis tool capable of analyzing, acquiring, comprehending a large amount of information and presenting that information to users in a intelligible way.

SUMMARY OF THE INVENTION

The present invention provides an interactive document-management-and-analysis system and method for analyzing and displaying information contained in a plurality of documents. Particular embodiments of the invention are especially effective for analyzing patent texts, such as patent claims, abstracts, and other portions of the specification.

A method according to one embodiment of the invention includes generating a set of N different representations of each document, and for each of a number of selected pairs of documents, determining N utility measures, a given utility measure being based on one of the N representations of the documents in that pair. In a specific embodiment, this information is displayed as a scatter plot in an area bounded by N non-parallel axes, where each selected pair is represented by a point in N-space having its coordinates along the N axes equal to the N utility measures.

In a specific embodiment, wherein N=2, the first representation is a conceptual-level representation such as a subject vector, and the second representation is a term-based representation such as a word vector.

In one use scenario, the selected pairs include all pair wise combinations of the documents in the plurality. In another scenario, the selected pairs are all pair wise combinations that include a particular document in the plurality.

The use of multiple methods of analysis, such as, for example, word-vector analysis and semantic-thread analysis, creates synergistic benefits by providing multiple independent measures of similarity. A system which uses multiple methods together can discover similar documents that either single method may have overlooked. In the cases where both methods agree, the user has greater confidence in the results because of the built-in "second opinion".

In accordance with another aspect of the invention, a dynamic concept query is performed by treating a user-specified query as a special type of document. The user can enter a list of words ranging from a single keyword to the text of an entire document, which is treated as a new document. A multidimensional array of similarity scores comparing that document to each existing document in the set is calculated. The user can then view the resulting clusters using the visualization techniques described herein.

The invention provides for an innovative analysis tool that assists users in discovering relationships among thousands of documents such as patents. Sophisticated natural language and information retrieval techniques enable the user to analyze claim sets, cluster claims based on similarity, and navigate through the results using graphical and textual visualization.

The invention provides further for a search routine which goes beyond simple keyword search; it understands the structure of documents such as patents and it captures concepts like patent infringement and interference. Users can browse through data visualizations (e.g., range query as described below), inspect quantitative score comparisons, and perform side-by-side textual analysis of matching patent claims. Based on the information gathered, users may analyze competitive patent and acquisition portfolios, develop patent blocking strategies, and find potential patent infringement.

In accordance with another aspect of the invention, the analysis methods described herein may be applied to a set of documents formed by the additional step of filtering a larger set of documents based on a concept query. For example, a user may find it useful to examine only those patents that discuss microelectronic packaging, analyze those patents, and generate a scatter plot display (e.g., run a concept query to pick a claim of interest followed by a claim query based on such claim and generate an overlay plot as described below).

In accordance with another aspect of the invention, recognizing and exploiting the relationship between various document types and "compound documents" each to the other permits multi-faceted analyses of multiple document types. For example, a patent is a compound document with nested sub-document linkages to sub-components, such as claims, background and summary of invention, etc. A claim is also a compound document because it may refer to other claims. Applying this paradigm to document analysis strategies, claims, whether individual, nested or as an amalgam, may be compared to other compound document components. For example, comparing claims to background and summary of invention in a patent. Furthermore, claims and background and summary of invention can be compared to other documents, such as related prior art literature from other sources such as magazines or journals. This enables the patent practitioner to view relevant claims, background and summaries, and other documents (non-patents), and cluster these together by similarity measures.

In accordance with one aspect of the invention, the user may select a metric that captures the essence of the document or documents under analysis. For example, the legal concept of patent infringement may be applied to sets of patents or patent applications. In a particular embodiment, a similarity matching algorithm treats the exemplar part of a patent claim differently from the dependent parts of the claim. Thus, a kind of "cross-comparison" matching is used, wherein the combined scores for (1) patent A, claim X dependent and independent part(s) vs. patent B, claim Y, independent part and (2) patent A, claim X dependent and independent part(s) vs. patent B, claim Y, dependent and independent part(s), generate an aggregate matching (or similarity) score for patent A, claim X vs. patent B, claim Y.

Normalization techniques deal with asymmetries in the matching, especially for documents of different lengths. For example, in the patent context, the situation where there is a short claim on "blue paint" and a long claim containing "blue paint." Looking at the small claim vs. the long claim appears close (since the long one at least contains the small one). But what of the case where it's the long claim vs. the small one? Standard information retrieval techniques would dictate that it's a poor match, since the long claim contains many limitations not in the small one. For patents, the "interference/infringement" match suggests that these are close, because if one "covers" the other, it doesn't matter which is the "query" and which is the "document."

Similarity based on the legal concept of patent infringement and interference serves as the touchstone to analyze, cluster and visualize patents and applications. This enables users to evaluate incoming applications for infringement against existing patents, filter large sets of patents to remove reissued and derivative patents, identify significant claim modifications in a reissued patent and identify related and unrelated patents to compare the intellectual property of two businesses.

A further understanding of the nature and advantages of the present invention may be realized by reference to the remaining portions of the specification and the drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A is a block diagram of a document analysis system embodying the present invention;

FIG. 1B is a more detailed block diagram of the interactions between the user and the system during the processing of document information;

FIGS. 2A-2B depict off line structured document processing steps according to a particular embodiment;

FIG. 3 depicts the preprocess step of off line structured document processing of FIGS. 2A-2B according to a particular embodiment;

FIG. 4A depicts the mapit-process step of off line structured document processing of FIGS. 2A-2B according to a particular embodiment;

FIG. 4B depicts the mapit-sfc step of FIG. 4A according to a particular embodiment;

FIG. 5 depicts the on line concept query processing according to a particular embodiment;

FIGS. 6A-6B depict off line generic document processing steps according to a particular embodiment;

FIG. 7A depicts claim parsing according to a particular embodiment;

FIG. 7B depicts the process-words step of claim parsing of FIG. 7A according to a particular embodiment;

FIG. 8A illustrates a scatter plot visualization technique according to a particular embodiment of the invention;

FIG. 8B illustrates a 2D plot visualization technique according to a particular embodiment of the invention;

FIG. 8C illustrates a 3D plot visualization technique according to a particular embodiment of the invention;

FIG. 8D illustrates an S-curve plot visualization technique according to a particular embodiment of the invention;

FIG. 8E is a flow chart depicting the steps for generating an S-curve plot;

FIG. 9A illustrates a representative sign on screen according to a particular embodiment of the invention;

FIG. 9B illustrates a representative concept query screen according to a particular embodiment of the invention;

FIG. 9C illustrates a representative concept query review screen according to a particular embodiment of the invention;

FIG. 9D illustrates a representative concept query results screen according to a particular embodiment of the invention;

FIG. 9E illustrates a representative concept query results viewer screen according to a particular embodiment of the invention;

FIG. 9F illustrates a representative concept query results viewer screen depicting a side-by-side comparison according to a particular embodiment of the invention;

FIG. 10A illustrates a representative patent query screen according to a particular embodiment of the invention;

FIG. 10B illustrates a representative patent query results screen according to a particular embodiment of the invention;

FIG. 10C illustrates a representative patent query side-by-side comparison screen of claims according to a particular embodiment of the invention;

FIG. 10D illustrates a representative patent query side-by-side comparison screen of patents according to a particular embodiment of the invention;

FIG. 11A illustrates a representative claim query screen according to a particular embodiment of the invention;

FIG. 11B illustrates a representative claim query claim finding screen according to a particular embodiment of the invention;

FIG. 11C illustrates a representative claim query results screen according to a particular embodiment of the invention;

FIG. 11D illustrates a representative overlay plot for a claim query results screen according to a particular embodiment of the invention;

FIG. 11E is a flow chart depicting the steps for generating an overlay plot;

FIG. 12A illustrates a representative range query screen according to a particular embodiment of the invention;

FIG. 12B illustrates a representative range query results screen according to a particular embodiment of the invention; and

FIG. 12C is a flow chart depicting the steps for generating a range query.

DESCRIPTION OF SPECIFIC EMBODIMENTS

A preferable embodiment of a document-management-and-analysis system and method according to the invention applicable to the task of patent search and analysis is reduced to practice and is available under the trade name, MAPIT.TM..

A document search and analysis tool must be both fast enough to handle a voluminous quantity of documents and flexible enough to adapt to different user requirements. Other aspects of the invention are of particular importance to expedient, accurate and efficient document analysis. First, the understanding of the structure and content of documents on multiple levels is useful to provide a much deeper analysis than generic search engines known in the art. Second, the combination of multiple similarity metrics is useful to achieve highly customized results. By contrast, search engines known in the art restrict the user to whatever notion of similarity was incorporated into the system by its designers. Third, the ability to parse structured documents, such as patent claims, is useful to extract their meaning. Fourth, the graphical display of information relevant to the user provides the user with quick access to the product of the analysis.

In accordance with the invention, multiple forms of textual analysis used to compare documents may be combined in any particular embodiment. One textual analysis method, called word-vector (also referred to as "wordvec" and "term-based") analysis, focuses on the co-occurrences of individual words and phrases between documents under analysis. Stemming technology, used in conjunction with word-vector analysis, matches words such as "projected" and "projection". Noun phrases, which are the key building blocks for many documents, such as patents, are identified and isolated. Those technologies are not restricted to English, but can be applied directly to other European languages. Word-vector analysis may be reduced to practice using Term Frequency/Inverse Document Frequency ("TF.IDF") techniques, or other techniques known in the art. TF.IDF techniques are further described in the Salton reference identfied above.

Another form of textual analysis is called semantic thread analysis (also referred to herein as "subject-vector" or "conceptual representation" analysis). Instead of focusing on individual words, this method identifies the general topics and themes in a document. It can determine that a patent, for example, is 35% about engineering physics, 15% about polymer science, 20% about holography, and 30% about manufacturing processes. If two patents cover the same subject areas in the same proportions, it is likely that they are closely related even if they use completely different words to describe their inventions. Semantic thread analysis may be reduced to practice by employing subject field code (SFC) techniques described by Dr. Elizabeth Liddy, et al. in one or more of the references identified above.

Preface on the Format of the Drawings

Embodiments of the invention will be best understood with reference to the drawings included herewith. A note on the format of these drawings is in order. In the drawings, process steps are depicted as squares or rectangles. Data structures internal to the program are depicted as rhomboid type structures. For example, in reference to FIG. 2A, element 10, text format file of patents in a search set is a rhomboid structure. Conventional data or text files are depicted as squares or rectangles with the upper right hand corner turned downward. For example, in FIG. 2A element 50 justclaims is a file which may exist on a hard disk, floppy disk, CD ROM or other form of storage medium. Open ended arrows reflect the flow of information. Tailess arrows indicate the flow of processing.

These drawings depict the processing steps, files and information according to one embodiment of the invention targeted to processing and understanding patents. While this serves as an excellent example of the features of the invention, the reader with ordinary skill in the art will appreciate that the invention's scope encompasses not merely the understanding and analysis of patents, but other documents as well.

Table 1 provides a definitional list of terminology used herein.

                  TABLE 1
    ______________________________________
    Term         Definition
    ______________________________________
    Claim Query  A query against a collection of text
                                    documents compared to a part of a
                                    particular member of the collection.
    Concept Query
                      A query against a collection of text
                                    documents compared to a user input
                 textual
                                    concept.
    Corpus                   A dataset.
    Dataset                 A document database containing documents
                                    upon which search and analysis
                 operations
                                   are conducted.
    Document               A unit of text which is selected for
                                   analysis which may inciude an entire
                                  document or any portion thereof such as a
                                 title, an abstract, or one or more
                                 clauses, sentences, or paragraphs. A
                                 document will typically be a member of a
                                document database containing a large
                                number of documents and may be referred to
                                by the term corpus.
    DR LINK                 Document Retrieval using Linguistic
                                    Knowledge. This is a system for
                                   performing natural language processing.
                                  This system is described in papers by
                                 Dr. Liddy referenced in the cross-
                                reference section herein above.
    Patent Query       A query against a collection of text
                                    documents compared to a particular
                 member
                                   of the collection, identified by the
                 user.
    Polysemy               The ability of a word to have multipie
                                  meanings.
    Query                     Text that is input for the purpose of
                                    selecting a subset of documents from a
                                   document database. While most queries
                                  entered by a user tend to be short
                                 compared to most documents stored in a
                                database this should not be assumed.
    score                     A numerical indicator assigned to a
                                    document indicative of a particular
                                   characteristic, e.g. relevance to a
                 query.
    Searchset             A document database containing documents
                                    upon which search and analysis
                 operations
                                   are conducted.
    SFC                         Subject field coder. A subject field
                                  coder is a process which tags content-
                                 bearing words in a text with a
                                    disambiguated subject code using a
                 lexical
                                   resource of words which are grouped in
                                  subject categories.
    SGML                       Standard Generalized Markup Language.
                                    Standard generalized markup language is
                                   comprised of a set of tags which may be
                                  embedded into a text document to indicate
                                 to a text processor how to process the
                                surrounding or encompassed text.
    Split Dataset
                      A dataset may be split into two distinct
                               components in order to perform comparative
                              analyses between the two sub-datasets.
                             For example, a split datasetof A company
                            patents and B company patents enables the
                           user to discover relationships between the
                          patent portfolios of the two companies.
    Stemming               Stemming is a process whereby nouns are
                         reduced to their most basic form or stem.
                        For example, the words "processing" and
                       "processed" are stemmed to the word
                      "process".
    Stop Word             One of a collection of words which are not
                                    assigned a semantic meaning by the
                 system.
                                   For example, the word "the".
    Stop Word List
                     A list of stop words.
    Term Index           A unique identifier assigned to each stem
                                  by a term indexer.
    Term Indexer       Term indexer is a process which performs
                                 indexing on an input text. Indexing
                                involves extracting terms from the text,
                               checking for stop words, processing
                              hyphenated words, then stemming all
                             inflected terms to a standard form.
                            Finally, a unique term index is assigned
                           to each stem.
    TFIDF                     Term Frequency/Inverse Document Frequency.
                                    This is a score computed by a term
                 indexer
                                   process. This score determines the
                                  relative prominence of a term compared to
                                 its occurrence throughout a document body.
    Token                     A white space delimited sequence of
                                characters having a particuiar meaning.
    Tokenize               A process whereby input text is separated
                               into a collection of tokens.
    Transitive           The transitive closure of a claim is
    Closure                 the claim itself and the transitive
                                    closure of all references within the
                                   particular claim.
    weight                   A numerical indicator assigned to a word
                                  or token indicative of a particular
                                 characteristic, e.g. relevance to a query.
    Word                       A sinqle word, compound word, phrase or
                                multiword construct. Note that the terms
                               "word " and "term" are used
                              interchangeably. Terms and words include,
                             for example, nouns, proper nouns, complex
                            nominals, noun phrases, verbs, and verbs
                           numeric expressions and adjectives. These
                          include stemmed and non-stemmed forms.
    ______________________________________

                  TABLE 2
    ______________________________________
    TTPPPPPPP.sub.-- CCC
    ______________________________________
    1.     TT is a 2-digit number used to identify the
           following patent-related documents:
    (1)    normal patent
    (2)    RE(reissue)
    (3)    PP (plant patent)
    (4)    D (design patent)
    (5)    T (defensive publication number)
    (6)    H (statutory invention registration)
    (7)    HD (design statutory invention registration)
    (8)    HP (plant statutory invention registration)
    2.     PPPPPPP is the original PTO patent number, possibly
           zero-padded on the left to force it to 7 digits.
    3.     CCC is a particular claim number, possibly
           zero-padded on the left to force it to 3 digits. In
           addition to identifying claims, this field is also
           used to identify a document type with select "dummy"
           numbers (e.g., in the "900" range) assigned
           particular meaning by the system:
    (1)      full patent
    (2)      abstract
    (3)      summary
    (4)      detailed description
    (5)      drawings
    (6)      justclaims
    There are two exceptions to this numbering convention:
    1.     the full text of the patent may be specified using
           the patent number with no claim at all (TTPPPPPPP)
    2.     the justclaims  file may be specified using the
           patent number with a ".justclaims" extension
           (TTPPPPPPP.justclaims)
    ______________________________________

                  TABLE 3
    ______________________________________
    VARIABLE      DEFINITION
    ______________________________________
    word.sub.-- count
                  Number of times word has appeared in
                  a document
    num.sub.-- docs
                  Total number of documents processed
    doc.sub.-- count
                  Number of documents containing a word
    average.sub.-- tf
                  Average TF score across corpus
    avg.sub.-- doc.sub.-- length
                  Average number of words per document
    current.sub.-- doc.sub.-- length
                  Number of words in this document
    ______________________________________

                  TABLE 4
    ______________________________________
    >DOC<
    >DOCNO< xxx >/DOCNO<
    >DOCID< yyy >/DOCID<
    >PARA< 1 >/PARA<
    >TEXT<
    text goes here
    >/TEXT<
    ______________________________________
     xxx = the unique document number generated in makedbm.pl
     (first document is #0)
     yyy = the MAPIT claim number TTPPPPPPP.sub.-- CCC

• Inventors
  Snyder, David L.; Calistri-Yeh, Randall J.;
• Assignee
  Manning & Napier Information Services (Rochester, NY)
• Published
  Mar-14-2000
• Current US Classes:
  707/10
  707/2
  707/3
  707/6
  715/522
• Application #
  929603
• International Classes
  G06F 017/30
• Field of Search
  707/9 707/10 707/6 707/1 707/2 707/3 707/4 707/5
• Examiner
  Amsbury; Wayne
• Agent
  Townsend and Townsend and Crew LLP
• US Patent References:
  4736308
  4788538
  5433199
  5442778
  5483650
  5622178
  5623679
  5623681
  5675819
  5696963
  5754840
  5761661
  5787422
  5799325
  5806079
  5809318
  5845301
  5848409