Multilingual electronic transfer dictionary containing topical codes and method of use6490548Abstract A multilingual electronic transfer dictionary provides for automatic topic disambiguation by including one or more topic codes in definitions contained the dictionary. Automatic topic disambiguation is accomplished by determining the frequencies of topic codes within a block of text. Dictionary entries having more frequently occurring topic codes are preferentially selected over those having less frequently occurring topic codes. When the topic codes are members of a hierarchical topical coding system, such as the International Patent Classification system, an iterative method can be used with starts with a coarser level of the coding system and is repeated at finer levels until an ambiguity is resolved. The dictionary is advantageously used for machine translation, e.g. between Japanese and English. Claims I claim: Description COPYRIGHT
TABLE 1
Term in Language 1 (Japanese) "soshiki"
Part of Speech Noun
Term in Language 2 (English) tissue
Part of Speech Noun
Topic Code A61K 47/38
Although the Japanese term is shown in the Tables herein in English characters within quotation marks, in the records of the present invention, a term in a language that uses other than English characters is preferably represented in a customary coding system such as Unicode. Alternatively all terms, optionally including the Topic Code, may for consistency be represented in such a coding system. The electronic dictionary of the present invention is embodied as a data structure in any form of machine-accessible memory, which may be permanent or transient. For example, the data structure may be stored using means known in the art for digital or other discrete encoding that is readable to produce a physical signal responding to the contents of selected memory locations, as by electromagnetic or optical means. Various magnetic memories are well known, including fixed disc drives, removable diskettes, tape, and cards. Integrated circuit memory modules may also be used in the present invention, including those in a self-contained form such as PCMCIA cards. A dictionary of the present invention may desirably be stored in permanent form, as on CD-ROM or like media. In accordance with the present invention, the dictionary may be accessed by a general purpose computer running an operating system (e.g. Windows, Mac OS, Unix, Linux, Pick, etc.) suitable to access the memory on which the dictionary is resident (either permanently or transiently) and including suitable application programming. For purpose of exemplification, programming in the C++ language is disclosed herein, but the reader will appreciate that any of a wide variety of programming languages or database applications may alternatively be employed, including, for example, Pascal, Fortran, COBOL, Eiffel, Java; Access, dBase, FoxPro, Paradox, and the like. A dictionary of the present invention may be incorporated in a standalone, handheld unit, as an enhanced version of translators such as those currently available from Selectronics, Sony EB Electronic Book, and Franklin Computer Corporation. Alternatively, a dictionary and associated programming in accordance with the present invention may be stored on a single general purpose computer or distributed on a CD-ROM; or it may be made available as a service via a network of computers, e.g. an intranet, a wide-area network, or a global communications network such as the Internet. Although the records illustrated in this disclosure include fields for Part of Speech, the reader should understand that an electronic dictionary of the present invention does not require information as to a term's Part of Speech, and so the present invention may optionally be implemented without any such fields. A dictionary record according to this invention may contain more than two languages. For example, it may also contain the term in German and French as shown in Table 2 below.
TABLE 2
Term in Language 1 (Japanese) "soshiki"
Part of Speech Noun
Term in Language 2 (English) tissue
Part of Speech Noun
Term in Language 3 (German) Gewebe
Part of Speech Noun
Term in Language 4 (French) tissu
Part of Speech Noun
Topic Code A61K 47/38
Alternatively, a record representing a term in more than two languages may be structured to include a field for a representation of the language for which each translation is provided. For example: "shoshiki"/Japanese/tissue/English/Gwebe/German/tissu/French/A61K 47/38 A dictionary record according to this invention may contain more than one topic code, as own in the record in Table 3 below.
TABLE 3
Term in Language 1 (Japanese) "maku"
Part of Speech Noun
Term in Language 2 (English) membrane
Part of Speech Noun
Topic Code 1 A61K 47/38
Topic Code 2 B01D 61/36
There are several topical code systems in existence that can be used in this invention. This can be a nonhierarchical system such as that disclosed in Appendix A of U.S. Pat. No. 5,873,056. However, hierarchical topical code systems are preferred. There are several known hierarchical topical code systems that can be used. Examples include the International Patent Classification (IPC) codes, the United States Patent Classifications, the categories of Roget's International Thesaurus.RTM., the Dewey Decimal System, and the Library of Congress Card Catalog Classification system. Other subject codes that may be used include the Longman subject codes disclosed in the Longman Dictionary of Contemporary English published by Longman Group UK Limited, Longman House, Burnt Mill, Harlow, Essex CM22JE, England. For example, the IPC codes contain five levels of classification as illustrated in Table 4 below.
TABLE 4
LEVEL Example
FAMILY A
CLASS A61
SUBCLASS A61K
MAIN GROUP A61K 47
SUBGROUP A61K 47/38
These levels in hierarchical topical code systems present the advantage of granularity. As is disclosed in the Examples below, if an ambiguity cannot be resolved at a shallow level, it may be resolvable at a deeper code level. The topic codes for a given set of terms can be selected by locating a document that has previously been classified by topic and for which the source term and its translations are appropriate. For example, in translating a Japanese patent document into English, the main IPC code for that document can be used for Japanese-English term pairs that are encountered during translation. According to this invention, these topic codes are used to determine a favored subject area within which to translate a particular term. Briefly, to determine the favored subject area, the topic codes for part or all of the terms in a particular block of text are counted. The block can be a sentence, a paragraph, a table, a set of text occurring within a certain number of bytes or words, a subdocument, an entire document or any other definable set of text in the source document. There are several counting methods known to the art. For example, a two-column table comprising topical codes and their corresponding frequencies may be used. The preferred method is a "map" as used in the programming language C++. Descriptions of "map" may be found in Mark Nelson's "C++ Programmer's Guide to the Standard Template Library," IDG Books Worldwide, Inc, Foster City, 1995, or in Microsoft Corporation's "Microsoft.RTM. Visual Studio.TM. 6.0 Development System", 1998 (hereinafter, "MVS"). To quote the latter: "The template class describes an object that controls a varying-length sequence of elements of type pair<const Key, T>. The first element of each pair is the sort key and the second is its associated value. The sequence is represented in a way that permits lookup, insertion, and removal of an arbitrary element with a number of operations proportional to the logarithm of the number of elements in the sequence (logarithmic time). Moreover, inserting an element invalidates no iterators, and removing an element invalidates only those iterators that point at the removed element." In the current invention, the topical code or a substring generated from the topical code is assigned to the Key and inserted into the map. The second member of the pair<const Key, T> may be arbitrary as the map only needs to be used for counting the frequencies of the codes. If a document has been assigned one or more topical codes, for example, IPC codes in the case of patent documents, these assigned codes can optionally be added to the counts of topical codes for the block of text being analyzed. It should be stressed that the use of such document-level assigned codes is optional and that the method of the current invention can be applied to any text. While there are several coding systems that may be used in this invention, the preferred coding system is the Unicode.RTM. wide character set as described in Microsoft Corporation's "Microsoft.RTM. Visual Studio.TM. 6.0 Development System", 1998 as follows: "Unicode: The Wide Character Set A wide character is a 2-byte multilingual character code. Any character in use in modern computing worldwide, including technical symbols and special publishing characters, can be represented according to the Unicode specification as a wide character. Developed and maintained by a large consortium that includes Microsoft, the Unicode standard is now widely accepted. Because every wide character is always represented in a fixed size of 16 bits, using wide characters simplifies programming with international character sets." A complete listing of the Unicode.RTM. codes preferred for this invention, and especially preferred for encoding Japanese, Chinese and Korean terms for this invention, can be found in The Unicode Consortium, "The Unicode Standard: Worldwide Character Encoding, Version 1.0, Vols. 1 and 2", Addison-Wesley, Reading, Mass., 1992. EXAMPLES EXAMPLE I Bilingual Electronic Dictionary Containing Hierarchical Topical Codes And Methods For Its Use Example Ia. Bilingual Electronic Dictionary Containing Hierarchical Topical Codes 1) Structure of Dictionary Record in Storage With the character coding system being Unicode.RTM., codes belonging to the user domain of the Unicode.RTM. system are selected as record and field delimiters as follows:
<record> 0xe000 // start of record
</record> 0xe001 // end of record
<lang1> 0xe011 // start of first language term field
</lang1> 0xe021 // end of first language term field
<lang2> 0xe013 // start of second language term field
</lang2> 0xe023 // end of second language term field
<topic> 0xe0a0 // start of topical code field
</topic> 0xe0a1 // end of topical code field
In the list above, 0xe011 is a hexadecimal representation of the two-byte (16-bit) code consisting of the hexadecimal bytes e0 and 11. A dictionary entry then has the following sequence of 16-bit codes: <record><lang1>Japanese term</lang1><lang2>English term</lang2><topic>IPC code</topic></record> The records in the dictionaries are selected and constructed by manually examining patents in many IPC classifications. For a particular patent in this Example, the language of the patent Japanese is taken as the first language and English is taken as the second language. For a given Japanese term in the patent, the Japanese term is entered into the first language term field and the English translation of that term which is most appropriate for the topic of the patent is entered into the second language term field. The main IPC classification for the patent is entered into the topical code field. The dictionary is entered into storage using Windows dialog methods as disclosed in David J. Kruglinski, "Inside Visual C++," 4.sup.th Ed., Microsoft Press, 1997 and known to persons knowledgeable in the field. A simple parse routine known to persons knowledgeable in the field is used to read the dictionary record into active memory as described below. 2) Dictionary File The dictionary file consists of a sequence in storage of the above dictionary records. It also contains an array of offsets of the dictionary records so that the records can be addressed by the record location number contained in the node elements of the index file. 3) Index File The index file for the aforesaid dictionary file is a multinodal tree made up of nodes and node elements as follows. There is one tree per language in the dictionary. If the dictionary is only to be used in one direction, for example, from the first language to the second language, only one tree for the source language need be present. Within each node, the node elements are arranged in Unicode.RTM. code order according to the key. Ordering of keys is determined by the Unicode.RTM. character order. The first 10 characters of a term are contained in the node element for speed. If the term is less than 10 characters in length, the key is terminated by a NULL. For terms 10 or more characters in length, the dictionary record is parsed into memory so that the full term can be compared. (This is a simple variant of the B-tree method disclosed in Donald E. Knuth, "The Art of Computer Programming, Volume 3, Sorting and Searching", Addison-Wesley, Reading, Mass., 1973, pp. 473-476, hereinafter "Knuth".) The choice of 127 elements makes the size of a node 4096 bytes which is the size of a memory page of the Windows.RTM. NT operating system as implemented on Intel x86 microprocessors. In the index file, the nodes are aligned on page boundaries so that a single file access reads a complete node into physical memory. Persons knowledgeable in the field can modify the parameters stated herein to the operating system and file structures as deemed appropriate. a) Node Class:
class Node
{
DWORD m_ThisNode; // number of this node
DWORD m_NumberOfElements;
// number of node elements currently in node
DWORD m_MaxNumberOfElements;
DWORD m_ParentNode;
DWORD m_ParentElement;
// number of parent element in parent node
DWORD m_FirstElement;
DWORD m_bBottomNode; // true if this is a bottom node
DWORD m_LastElement; // last element in linked list
NodeElement m_Elements[127];
};
b) Node Element Class:
class NodeElement
{
wchar_t key[10]; // hold up to 10 characters from
record
DWORD childNode; // for keys less than this key
DWORD next; // next element in node
DWORD record; // number of record in dictionary
};
4) Structure of Dictionary in Active Memory When associated with a textual object, the aforementioned dictionary record is parsed into an active object, for example a C++ object having the following member variables where "wstring" is a wide character (Unicode.RTM.) string according to MVS:
class Record
{
wstring wsL1Term; // term from first language
wstring wsL2Term; // term from second language
wstring wsTopic; // topical code
}
Example Ib Use of Bilingual Dictionary in a Single Level Method of Selecting Translations of Terms Using Hierarchical Topical Codes 1) Association of Dictionary Entries with a Textual Object in a First Language Given a first language wstring wsLang1String of length lenL1S, at each position m in the wstring, the index file is searched for first language terms that match wsLang1String starting at position m and copies of these dictionary records are then read into active memory as follows: a) A CPtrArray paMaster (as described in MVS), is constructed with length lenL1S and initialized with NULL at each position in the array. In operation, this will become an array of arrays, with a CPtrArray paRecords of dictionary records at each position in this array. Example of paMaster before adding dictionary records, Table 5:
TABLE 5
paMaster[0] NULL
(intervening paMasters) (all NULL)
paMaster[m] NULL
paMaster[m + 1] NULL
(intervening paMasters) (all NULL)
paMaster[lenL1S-1] NULL
b) Staring from position 0 in wsLang1String, the index file is searched following the method described in Knuth. The records in the dictionary for which the first language term matches the substring of wsLang1String of the same length as the first language term are read into active memory as follows: The record is parsed by sequentially reading each character of the record. Upon reading a <record> code, the program allocates memory for an instance of the class Entry. After reading the <lang1> code, the program places subsequent characters in wsL1Term until the </lang1> code is read. Likewise, after reading the <lang2> code, the program places subsequent characters in wsL2Term until the </lang2> code is read. Continuing, after reading the <topic> code, the program places subsequent characters in wsTopic until the </topic> code is read. Upon reading the </record> code, the program appends the pointer to this record to the CPtrArray at position 0. If this is the first record at that position, i.e., there is a NULL at that position, the program allocates a CPtrArray for the record and assigns its pointer to paMaster[0], then appends the pointer of the new dictionary record to this newly allocated CPtrArray. c) The procedure in b) is repeated for each position in wsLang1String. When these iterations are complete each position m of paMaster will contain either a NULL (if no matching dictionary records were found) or a pointer to a CPtrArray of dictionary records. Example of paMaster after adding dictionary records, Table 6:
TABLE 6
paMaster[0] *paRecords (0.sup.th)
(intervening paMasters) (may include NULL)
paMaster[m] *paRecords (m.sup.th)
paMaster[m + 1] NULL
(intervening paMasters) (may include NULL)
paMaster[lenL1S-1] *paRecords ( (lenL1S-1).sup.th )
(may be NULL)
In this example, NULL entries remain where no matching dictionary entries were found. An example of a paRecord after adding dictionary records is shown below. In this case, three matching records were found and copied into active memory, Table 7.
TABLE 7
paRecord[0] Pointer to first record
paRecord[1] Pointer to second record
PaRecord[2] Pointer to third record
2) Inserting Topical Codes Into a Structure that Allows for Counting their Frequencies of Occurrence For each member of the pointer array, a key consisting of the first three characters of the IPC code contained in wsTopic (i.e., the class code) are added to the map defined by: map<wstring, int> STR2INT; to get the map STR2INT smapClassCode; Following the teachings of MVS for the map template class, the following C++ code segment is executed for each member of paMaster as follows: ##EQU1## 2a) Alternate Structure and Method for Counting Topical Codes: Alternatively to using a map structure, a list structure (as disclosed in MVS) can be used by appending each instance of the topical code or the substring derived from the topical code to a list. After all of the topical codes have been appended, this list can be optionally sorted. A vector is then constructed using the following C++ structures: ##EQU2## For each unique topical code in the list, the number of occurrences of said member in the list is counted, the topical code assigned to wsCode and the count assigned to nCount for a new STRINTPAIR and this STRINTPAIR appended to the STRINTARRAY with the STRINTARRAY::push_back function. For this implementation, the most frequently occurring topical code is determined by looking up and comparing nCount for the respective topical codes in STRINTARRAY. The alternative structure described hereinabove can be used as an alternative in any of the following examples as well. 3) Selecting Translations of Terms in Second Language Using Frequencies of Topical Codes For each of the positions m of wsLang1String, if there is more than one matching first language terms from the dictionary, the number of occurrences of the first three characters of the IPC code in smapClassCode for the matching first language terms are compared. The term with the most frequent IPC key is selected. For the particular example of paRecord shown above, the following code segment selects the record with the most frequently occurring IPC code: ##EQU3## Note that in this example, if more than one IPC class code has the same frequency the first of the two records to occur in pRecords is selected. Example Ic. Use of Bilingual Dictionary in a Multilevel Method of Selecting Translations of Terms Using Hierarchical Topical Codes 1) Association of Dictionary Entries with a Textual Object in a First Language This step is the same as for Example Ib above. 2) Inserting Topical Codes into a Structure that Allows for Counting their Frequencies of Occurrence Instead of the single smapClassCode in Example Ib, four maps are constructed, namely, STR2INT smapClassCode; // from IPC code through class level STR2INT smapSubclassCode; // from IPC code through subclass level STR2INT smapG roup // from IPC code through main group level STR2INT smapSubgroup; //PC code through subgroup level In this case, the following C++ code segment is executed for each member of paMaster as follows: ##EQU4## 3) Selecting Translations of Terms in Target Language Using Frequencies of Topical Codes For each of the positions m of wsLang1String, if there is more than one matching first language terms from the dictionary, the number of occurrences of the first three characters of the IPC code in smapClassCode for the matching first language terms are compared. The term with the most frequent IPC key is selected. If there are still two or more terms with the same numbers of IPC class code occurrences, the numbers of occurrences of the subclass codes in smapSubclassCode are compared. This comparison is repeated until either a single term is selected or a repetition at the subgroup level is completed. For the particular example of paRecord shown above, the following C++ code segment selects the record with the most frequently occurring IPC code: ##EQU5## In this example if there is redundancy at all levels, the first record is selected. Example Id. Use of Bilingual Dictionary in a Single Level Method of Selecting Translations of Terms Using Hierarchical Topical Codes, Additionally Using Topical Codes Assigned to the Document as a Whole. This method applies to a patent or patent application for which IPC classifications have been assigned and whose codes are included in the bibliographic portion of the document. 1) Association of Dictionary Entries with a Textual Object in a First Language This step is the same as for Example Ib above. 2) Inserting Topical Codes Into a Structure that Allows for Counting their Frequencies of Occurrence This step is the same as for II above with the exception that the Subclass codes of the bibliographic IPC codes are also added to smapClassCode. This is done by assigning each IPC code for the document to a wstring wsIPCCode and adding these to smapClassCode with the following C++ code segment: ##EQU6## 3) Selecting Translations of Terms in Second Language Using Frequencies of Topical Codes This step is the same as for Example Ib above. EXAMPLE II Multilingual Electronic Dictionary Containing Hierarchical Topical Codes and Method for Its Use Example IIA. Multilingual Electronic Dictionary Containing Hierarchical Topical Codes 1) Structure of Dictionary Record in Storage With the character coding system being Unicode.RTM., codes belonging to the user domain of the Unicode.RTM. system are selected as record and field delimiters as follows:
<record> 0xe000 //start of record
</record> 0xe001 //end of record
<lang1> 0xe010 //start of first language term field
</lang1> 0xe011 //end of first language term field
<lang2> 0xe020 //start of second language term field
</lang2> 0xe021 //end of second language term field
<lang3> 0xe030 //start of third language term field
</lang3> 0xe031 //end of third language term field
<lang4> 0xe040 //start of fourth language term field
</lang4> 0xe041 //end of fourth language term field
<topic> 0xe0a0 //start of topical code field
</topic> 0xe0a1 //end of topical code field
In the list above, 0xE011 is a hexadecimal representation of the two-byte (16-bit) code consisting of the hexadecimal bytes E0 and 11. A dictionary entry then has the following sequence of 16-bit codes: <record><lang1> Japanese term</lang1><lang2> English term </lang2><lang3> German term </lang3><lang4> French term</lang4><topic> IPC code</topic></record> The records in the dictionaries are selected and constructed by manually examining patents in many IPC classifications. For a particular patent in this Example, the language of the patent Japanese is taken as the first language, English as the second language, German as the third language and French as the fourth language. For a given Japanese term in the patent, the Japanese term is entered into the first language term field and the English translation of that term which is most appropriate for the topic of the patent is entered into the second language term field. Likewise, the appropriate German and French terms are entered into their respective fields. The main IPC classification for the patent is entered into the topical code field. A simple parse routine known to persons knowledgeable in the field is used to read the dictionary record into active memory. 2) Dictionary File The dictionary file consists of a sequence in storage of the above dictionary records. It also contains an array of offsets of the dictionary records so that the records can be addressed by the record location number contained in the node elements of the index file. 3) Index File The index file for the aforesaid dictionary file is a multinodal tree made up of nodes and node elements as follows. There is one tree per language in the dictionary. If the dictionary is only to be used in one direction, for example, from the first language to the second, third and fourth languages, only one tree for the first language need be present. Within each node, the node elements are arranged in Unicode.RTM. code order according to the key. Ordering of keys is determined by the Unicode.RTM. character order. The first 10 characters of a term are contained in the node element for speed. If the term is less than 10 characters in length, the key is terminated by a NULL. For terms 10 or more characters in length, the dictionary record is parsed into memory so that the full term can be compared. (This is a simple variant of the B-tree method disclosed in Donald E. Knuth, "The Art of Computer Programming, Volume 3, Sorting and Searching", Addison-Wesley, Reading, Mass., 1973, pp.473-476.) The choice of 127 elements makes the size of a node 4096 bytes which is the size of a memory page of the Windows.RTM. NT operating system as implemented on Intel x86 microprocessors. In the index file, the nodes are aligned on page boundaries so that a single file access reads a complete node into physical memory. a) Node Class
class Node
{
DWORD m_ThisNode: //number of this node
DWORD m_NumberOfElements; //number of node elements
currently in node
DWORD m_MaxNumberOfElements;
DWORD m_ParentNode;
DWORD m_ParentElement; //number of parent element in
parent node
DWORD m_FirstElement;
DWORD m_bBottomNode; //true if this is a bottom node
DWORD m_LastElement; //last element in linked list
NodeElement m_Elements[127];
};
b) Node Element Class:
class NodeElement
{
wchar_t key[10]; //hold up to 10 characters from record
DWORD childNode; //for keys less than this key
DWORD next; //next element in node
DWORD record; //number of record in dictionary
};
4) Structure of Dictionary in Active Memory When associated with a textual object, the aforementioned dictionary record is parsed into an active object, for example a C++ object having the following member variables where "wstring" is a wide character (Unicode.RTM.) string according to MVS:
class Record
}
wstring wsL1Term; //term from first language
wstring wsL2Term; //term from second language
wstring wsL3Term; //term from third language
wstring wsL4Term; //term from fourth language
wstring wsTopic; //topical code
}
Example IIb. Use of Multilingual Dictionary in a Single Level Method of Selecting Translations of Terms Using Hierarchical Topical Codes 1) Association of Dictionary Entries with a Textual Object in a First Language Given a first language wstring wsLang1String of length lenL1S, at each position m in the wstring, the index file is searched for first language terms that match wsLang1String starting at position m and copies of these dictionary records are then read into active memory as follows: a) A CPtrArray paMaster (as described in MVS), is constructed with length lenL1S and initialized with NULL at each position in the array. In operation, this will become an array of arrays, with a CPtrArray paRecords of dictionary records at each position in this array. Example of paMaster before adding dictionary records, Table 8:
TABLE 8
paMaster[0] NULL
(intervening paMasters) (all NULL)
paMaster[m] NULL
paMaster[m + 1] NULL
(intervening paMasters) (all NULL)
paMaster[lenL1S-1] NULL
In this example, NULL entries remain where no matching dictionary entries were found. An example of a paRecord after adding dictionary records is shown below. In this case, three matching records were found and copied into active memory, Table 9.
TABLE 9
paRecord[0] Pointer to first record
paRecord[1] Pointer to second record
paRecord[2] Pointer to third record
b) Starting from position 0 in wsLang1String, the index file is searched following the method described in Knuth. The records in the dictionary for which the first language term matches the substring of wsLang1String of the same length as the first language term are read into active memory as follows: The record is parsed by sequentially reading each character of the record. Upon reading a <record>code, the program allocates memory for an instance of the class Entry. After reading the <lang1>code, the program places subsequent characters in wsL1Term until the </lang1>code is read. Likewise, after reading the <lang2>code, the program places subsequent characters in wsL2Term until the </lang2> code is read. After reading the <lang3> code, the program places subsequent characters in wsL3Term until the </lang3> code is read. After reading the <lang4> code, the program places subsequent characters in wsL4Term until the </lang4> code is read. Continuing, after reading the <topic> code, the program places subsequent characters in wsTopic until the </topic> code is read. Upon reading the </record> code, the program appends the pointer to this record to the CPtrArray at position 0. If this is the first record at that position, i.e., there is a NULL at that position, the program allocates a CPtrArray for the record and assigns its pointer to paMaster[0]. c) The procedure in b) is repeated for each position in wsLang1String. When these iterations are complete each position m of paMaster will contain either a NULL (if not matching dictionary records were found) or a pointer to a CPtrArray of dictionary records. Example of paMaster after adding dictionary records, Table 10:
TABLE 10
paMaster[0] *paRecords (0.sup.th)
(intervening paMasters) (may include NULL)
paMaster[m] *paRecords (m.sup.th)
paMaster[m + 1] NULL
(intervening paMasters) (may include NULL)
paMaster[lenL1S-1] *paRecords ( (lenL1S-1).sup.th)
(may be NULL)
2) Inserting Topical Codes Into a Structure that Allows for Counting their Frequencies of Occurrence For each member of the pointer array, a key consisting of the first three characters of the IPC code contained in wsTopic (i.e., the class code) are added to the map defined by: map<wstring, int> STR2INT; to get the map STR2INT smapClassCode; Following the teachings of MVS, the following C++ code segment is executed for each member of paMaster as follows:
for(int i = 0; i < lenL1S; i++)
{
if( paMaster[i] ! = NULL )
{
int nRecords = paMaster[i].GetLength( ); //number of dictionary
records copied
CPtrArray* pRecords = paMaster [I];
for( int j = 0; j < nRecords; j++)
{
Record* pRecord = pRecords[j];
pair < wstring, int > newPair; //pair for insertion into
smapClassCode
newPair.first = pRecord- > wsTopic.left(3);
newPair.second = 1;
smapClassCode.insert( newPair );
}
}
}
3) Selecting Translations of Terms in Second, Third and Fourth Languages Using Frequencies of Topical Codes For each of the positions m of wsLang1String, if there is more than one matching first language terms from the dictionary, the number of occurrences of the first three characters of the IPC code in smapClassCode for the matching first language terms are compared. The term with the most frequent IPC key is selected. For the particular example of paRecord shown above, the following C++ code segment selects the record with the most frequently occurring IPC code:
int nMostFrequent = 0; //initialize to first record in paRecord
int nMaxCount = 0; //for following the maximum count
for( int i = 0; i < paRecord.GetLength( ); i++)
{
Record* pRecord = pRecords[i];
wstring wsTest = pRecord- > wsTopic.left(3);
int nCount = smapClassCode.count( wsTest );
if( nCount > nMaxCount )
{
nMostFrequent = i;
nMaxCount = nCount;
}
}
Note that in this example, if more than one IPC class code has the same frequency the first of the two records to occur in pRecords is selected. EXAMPLE III Bilingual Electronic Dictionary Containing Multiple Hierarchical Topical Codes and Methods for Its Use Example IIIa. Bilingual Electronic Dictionary Containing Multiple Hierarchical Topical Codes 1) Structure of Dictionary Record in Storage With the character coding system being Unicode.RTM., codes belonging to the user domain of the Unicode.RTM. system are selected as record and field delimiters as follows:
<record> 0xe000 //start of record
</record> 0xe001 //end of record
<lang1> 0xe011 //start of first language term field
</lang1> 0xe021 //end of first language term field
<lang2> 0xe013 //start of second language term field
</lang2> 0xe023 //end of second language term field
<topic> 0xe0a0 //start of topical code field
</topic> 0xe0a1 //end of topical code field
In the list above, 0xE011 is a hexadecimal representation of the two-byte (16-bit) code consisting of the hexadecimal bytes E0 and 11. A dictionary entry then has the following sequence of 16-bit codes: <record><lang1>Japanese term </lang1><lang2>English term </lang2><topic>IPC code1</topic>(repeat n-1 times)<topic>IPC code n</topic></record> The records in the dictionaries are selected and constructed by manually examining patents in many IPC classifications. For a particular patent in this Example, the language of the patent Japanese is taken as the first language and English is taken as the second language. For a given Japanese term in the patent, the Japanese term is entered into the first language term field and the English translation of that term which is most appropriate for the topic of the patent is entered into the second language term field. The inventive IPC classifications for the patent are entered into the topical code fields. A simple parse routine known to persons knowledgeable in the field is used to read the dictionary record into active memory as described in below. 2) Dictionary File The dictionary file consists of a sequence in storage of the above dictionary records. It also contains an array of offsets of the dictionary records so that the records can be addressed by the record location number contained in the node elements of the index file. 3) Index File The index file for the aforesaid dictionary file is a multinodal tree made up of nodes and node elements as follows. There is one tree per language in the dictionary. If the dictionary is only to be used in one direction, for example, from the first language to the second language, only one tree for the source language need be present. Within each node, the node elements are arranged in Unicode.RTM. code order according to the key. Ordering of keys is determined by the Unicode.RTM. character order. The first 10 characters of a term are contained in the node element for speed. If the term is less than 10 characters in length, the key is terminated by a NULL. For terms 10 or more characters in length, the dictionary record is parsed into memory so that the full term can be compared. (This is a simple variant of the B-tree method disclosed in Donald E. Knuth, "The Art of Computer Programming, Volume 3, Sorting and Searching", Addison-Wesley, Reading, Mass., 1973, pp. 473-476, hereinafter "Knuth".) The choice of 127 elements makes the size of a node 4096 bytes which is the size of a memory page of the Windows.RTM. NT operating system as implemented on Intel x86 microprocessors. In the index file, the nodes are aligned on page boundaries so that a single file access reads a complete node into physical memory. a) Node Class:
class Node
{
DWORD m_ThisNode; //number of this node
DWORD m_NumberOfElements; //number of node
elements currently in node
DWORD m_MaxNumberOfElements;
DWORD m_ParentNode;
DWORD m_ParentElement; //number of parent element in
parent node
DWORD m_FirstElement;
DWORD m_bBottomNode; //true if this is a bottom node
DWORD m_LastElement; //last element in linked list
NodeElement m_Elements[127];
};
b) Node Element Class:
class NodeElement
{
wchar_t key[10]; //hold up to 10 characters from record
DWORD childNode; //for keys less than this key
DWORD next; //next element in node
DWORD record; //number of record in dictionary
};
4) Structure of Dictionary in Active Memory When associated with a textual object, the aforementioned dictionary record is parsed into an active object, for example a C++ object having the following member variables where "wstring" is a wide character (Unicode.RTM.) string according to MVS:
class Record
{
wstringwsL1Term; //term from first language
wstringwsL2Term; //term from second language
vector<wstring> vecTopics; //array of topical code
wstrings
}
Example IIIb. Use of bilingual dictionary in a single level method of selecting translations of terms using multiple hierarchical topical codes 1) Association of Dictionary Entries with a Textual Object in a First Language Given a first language wstring wsLang1String of length lenL1S, at each position m in the wstring, the index file is searched for first language terms that match wsLang1String starting at position m and copies of these dictionary records are then read into active memory as follows: a) A CPtrArray paMaster (as described in MVS), is constructed with length lenL1S and initialized with NULL at each position in the array. In operation, this will become an array of arrays, with a CPtrArray paRecords of dictionary records at each position in this array. Example of paMaster before adding dictionary records, Table 11:
TABLE 11
paMaster[0] NULL
(intervening paMasters) (all NULL)
paMaster[m] NULL
paMaster[m + 1] NULL
(intervening paMasters) (all NULL)
paMaster[lenL1S-1] NULL
b) Starting from position 0 in wsLang1String, the index file is searched following the method described in Knuth. The records in the dictionary for which the first language term matches the substring of wsLang1String of the same length as the first language term are read into active memory as follows: The record is parsed by sequentially reading each character of the record. Upon reading a <record>code, the program allocates memory for an instance of the class Entry. After reading the <lang1> code, the program places subsequent characters in wsL1Term until the </lang1> code is to read. Likewise, after reading the <lang2> code, the program places subsequent characters in wsL2Term until the </lang2> code is read. Continuing, after reading the <topic> code, the program places subsequent characters in a wstring wsTopic until the </topic> code is read. This wstring is appended to the vector vecTopics. If there are additional topic codes, these are likewise read into a wstring wsTopic which is then appended to vecTopics. Upon reading the </record> code, the program appends the pointer to this record to the CPtrArray at position 0. If this is the first record at that position, i.e., there is a NULL at that position, the program allocates a CPtrArray for the record and assigns its pointer to paMaster[0]. c) The procedure in b) is repeated for each position in wsLang1String. When these iterations are complete each position m of paMaster will contain either a NULL (if not matching dictionary records were found) or a pointer to a CPtrArray of dictionary records. Example of paMaster after adding dictionary records, Table 12:
TABLE 12
paMaster[0] *paRecords (0.sup.th)
(intervening paMasters) (may include NULL)
paMaster[m] *paRecords (m.sup.th)
paMaster[m + 1] NULL
(intervening paMasters) (may include NULL)
paMaster[lenL1S-1] NULL
In this example, NULL entries remain where no matching dictionary entries were found. An example of a paRecord after adding dictionary records is shown below. In this case, three matching records were found and copied into active memory, Table 13.
TABLE 13
paRecord[0] Pointer to first record
paRecord[1] Pointer to second record
paRecord[2] Pointer to third record
2) Inserting Topical Codes Into a Structure that Allows for Counting their Frequencies of Occurrence For each member of the pointer array, a key consisting of the first three characters of the IPC code contained in wsTopic (i.e., the class code) are added to the map defined by: map<wstring, int> STR2INT; to get the map STR2INT smapClassCode; Following the teachings of MVS, the following C++ code segment is executed for each member of paMaster as follows:
for( int i = 0; i < lenL1S; i++)
{
if( paMaster[i] ! = NULL)
{
int nRecords = paMaster[i].GetLength( ); //number of dictionary
records copied
CPtrArray* pRecords = paMaster[I];
for( int j = 0; j <nRecords; j++)
{
Record* pRecord = pRecords[j];
vector < wstring > & vecTopics = pRecord- > vecTopics;
for(int k = 0; k < vecTopics.size( ) ; k++)
{
wstring& wsTopic = vecTopics[k];
pair < wstring, int > newPair; //pair for insertion into
smapClassCode
newPair.first = pRecord- > wsTopic.left(3);
newPair.second = 1;
smapClassCode.insert( newPair );
}
}
}
}
3) Selecting Translations of Terms in Second Language Using Frequencies of Topical Codes For each of the positions m of wsLang1String, if there is more than one matching first language terms from the dictionary, the number of occurrences of the first three characters of the IPC code in smapClassCode for the matching first language terms are compared. The term with the most frequent IPC key is selected. For the particular example of paRecord shown above, the following C++ code segment selects the record with the most frequently occurring IPC code:
int nMostFrequent = 0; //initialize to first record in paRecord
int nMaxCount = 0; //for following the maximum count
for( int i = 0; i < paRecord.GetLength( ); i++)
{
Record* pRecord = pRecords[i];
wstring wsTest = pRecord- > wsTopic.left(3);
int nCount = smapClassCode.count( wsTest );
if( nCount > nMaxCount)
{
nMostFrequent = i;
nMaxCount = nCount;
}
}
Note that in this example, if more than one IPC class code has the same frequency, the first of the two records to occur in pRecords is selected. EXAMPLE IV Bilingual Electronic Dictionary Containing Nonhierarchical Topical Codes and Methods for Its Use Example IVa. Bilingual Electronic Dictionary Containing Nonhierarchical Topical Codes 1) Structure of Dictionary Record in Storage With the character coding system being Unicode.RTM., codes belonging to the user domain of the Unicode.RTM. system are selected as record and field delimiters as follows:
<record> 0xe000 //start of record
</record> 0xe001 //end of record
<lang1> 0xe011 //start of first language term field
</lang1> 0xe021 //end of first language term field
<lang2> 0xe013 //start of second language term field
</lang2> 0xe023 //end of second language term field
<topic> 0xe0a0 //start of topical code field
</topic> 0xe0a1 //end of topical code field
In the list above, 0xE011 is a hexadecimal representation of the two-byte (16-bit) code consisting of the hexadecimal bytes E0 and 11. A dictionary entry then has the following sequence of 16-bit codes: <record><lang1>Japanese term </lang1><lang2> English term </lang2><topic>topical code</topic></record> The records in the dictionaries are selected and constructed by manually examining patents in many topical areas. Each patent is manually assigned to one of the topic codes listed in Appendix A of U.S. Pat. No. 5,873,056. For a particular patent in this embodiment, the language of the patent Japanese is taken as the first language and English is taken as the second language. For a given Japanese term in the patent, the Japanese term is entered into the first language term field and the English translation of that term which is most appropriate for the topic of the patent is entered into the second language term field. The manually assigned topic code for the patent is entered into the topical code field. A simple parse routine known to persons knowledgeable in the field is used to read the dictionary record into active memory as described below. 2) Dictionary File The dictionary file consists of a sequence in storage of the above dictionary records. It also contains an array of offsets of the dictionary records so that the records can be addressed by the record location number contained in the node elements of the index file. 3) Index File The index file for the aforesaid dictionary file is a multinodal tree made up of nodes and node elements as follows. There is one tree per language in the dictionary. If the dictionary is only to be used in one direction, for example, from the first language to the second language, only one tree for the source language need be present. Within each node, the node elements are arranged in Unicode.RTM. code order according to the key. Ordering of keys is determined by the Unicode.RTM. character order. The first 10 characters of a term are contained in the node element for speed. If the term is less than 10 characters in length, the key is terminated by a NULL. For terms 10 or more characters in length, the dictionary record is parsed into memory so that the full term can be compared. (This is a simple variant of the B-tree method disclosed in Knuth.) The choice of 127 elements makes the size of a node 4096 bytes which is the size of a memory page of the Windows.RTM. NT operating system as implemented on Intel x86 microprocessors. In the index file, the nodes are aligned on page boundaries so that a single file access reads a complete node into physical memory. a) Node Class
class Node
{
DWORD m_ThisNode; //number of this node
DWORD m_NumberOfElements; //number of node elements
currently in node
DWORD m_MacNumberOfElements;
DWORD m_ParentNode;
DWORD m_ParentElement; //number of parent element in parent
node
DWORD m_FirstElement;
DWORD m_bBottomNode; //true if this is a bottom node
DWORD m_LastElement; //last element in linked list
NodeElement m_Elements[127];
};
b) Node Element Class:
class NodeElement
{
wchar_t key[10]; //hold up to 10 characters from record
DWORD childNode; //for keys less than this key
DWORD next; //next element in node
DWORD record; //number of record in dictionary
};
4) Structure of Dictionary in Active Memory When associated with a textual object, the aforementioned dictionary record is parsed into an active object, for example a C++ object having the following member variables where "wstring" is a wide character (Unicode.RTM.) string according to MVS:
class Record
{
wstring wsL1Term; //term from first language
wstring wsL2Term; //term from second language
wstring wsTopic; //topical code
}
Example IVb. Use of Bilingual Dictionary in a Single Level Method of Selecting Translations of Terms Using Nonhierarchical Topical Codes 1) Association of Dictionary Entries with a Textual Object in a First Language. Given a first language wstring wsLang1String of length lenL1S, at each position m in the wstring, the index file is searched for first language terms that match wsLang1String starting at position m and copies of these dictionary records are then read into active memory as follows: a) A CPtrArray paMaster (as described in MVS), is constructed with length lenL1S and initialized with NULL at each position in the array. In operation, this will become an array of arrays, with a CPtrArray paRecords of dictionary records at each position in this array. Example of paMaster before adding dictionary records, Table 14:
TABLE 14
paMaster[0] NULL
(intervening paMasters) (all NULL)
paMaster[m] NULL
paMaster[m + 1] NULL
(intervening paMasters) (all NULL)
paMaster[lenL1S-1] NULL
b) Starting from position 0 in wsLang1String, the index file is searched following the method described in Knuth. The records in the dictionary for which the first language term matches the substring of wsLang1String of the same length as the first language term are read into active memory as follows: The record is parsed by sequentially reading each character of the record. Upon reading a <record> code, the program allocates memory for an instance of the class Entry. After reading the <lang1> code, the program places subsequent characters in wsL1Term until the </lang1> code is read. Likewise, after reading the <lang2> code, the program places subsequent characters in wsL2Term until the </lang2> code is read. Continuing, after reading the <topic> code, the program places subsequent characters in wsTopic until the </topic> code is read. Upon reading the </record> code, the program appends the pointer to this record to the CPtrArray at position 0. If this is the first record at that position, i.e., there is a NULL at that position, the program allocates a CPtrArray for the record and assigns its pointer to paMaster[0]. c) The procedure in b) is repeated for each position in wsLang1String. When these iterations are complete each position m of paMaster will contain either a NULL (if not matching dictionary records were found) or a pointer to a CPtrArray of dictionary records. Example of paMaster after adding dictionary records, Table 15:
TABLE 15
paMaster[0] *paRecords (0.sup.th)
(intervening paMasters) (may include NULL)
paMaster[m] *paRecords (m.sup.th)
paMaster[m + 1] NULL
(intervening paMasters) (may include NULL)
paMaster[lenL1S-1] NULL
In this example, NULL entries remain where no matching dictionary entries were found. An example of a paRecord after adding dictionary records is shown in Table 16 below. In this case, three matching records were found and copied into active memory.
TABLE 16
paRecord[0] Pointer to first record
paRecord[1] Pointer to second record
paRecord[2] Pointer to third record
2) Inserting Topical Codes into a Structure that Allows for Counting their Frequencies of Occurrence For each member of the pointer array, a key consisting of the entire topical code contained in wsTopic is added to the map defined by: map<wstring, int> STR2INT; to get the map STR2INT smapCode; Following the teachings of MVS, the following C++ code segment is executed for each member of paMaster as follows:
for( int i = 0; i < lenL1S; i++)
{
if( paMaster[i] ! = NULL )
{
int nRecords = paMaster[i].GetLength( ); //number of dictionary
records copied
CPtrArray* pRecords = paMaster[I];
for( int j = 0; j < nRecords; j++)
{
Record* pRecord = pRecords[j];
pair < wstring, int > newPair; //pair for insertion into
smapCode
newPair.first = pRecord- > wsTopic; //entire code because
nonhierarchical
newPair.second = 1;
smapCode.insert( newPair );
}
}
3) Selecting Translations of Terms in Second Language Using Frequencies of Topical Codes For each of the positions m of wsLang1String, if there is more than one matching first language terms from the dictionary, the number of occurrences of the first three characters of the IPC code in smapClassCode for the matching first language terms are compared. The term with the most frequent IPC key is selected. For the particular example of paRecord shown above, the following code segment selects the record with the most frequently occurring IPC code:
int nMostFrequent = 0; //initialize to first record in paRecord
int nMaxCount = 0; //for following the maximum count
for( int i = 0; i < paRecord.GetLength( ); i++)
{
Record* pRecord = pRecords[i];
wstring wsTest = pRecord- > wsTopic; //entire code because
nonhierarchical
int nCount = smapCode.count( wsTest );
if( nCount > nMaxCount )
{
nMostFrequent = i;
nMaxCount = nCount;
}
}
Note that in this example, if more than one topical code has the same frequency the first of the two records to occur in pRecords is selected.
|
Same subclass Same class Consider this |
||||||||||
