Hyper text mark up language document to speech converter

Abstract

A system for converting a hyper text markup language (HTML) document to speech includes an HTML parser, an HTML to speech (HTS) control parser, a tag converter, a text normalizer and a TTS converter. The HTML parser receives data of an HTML formatted document and parses out content text, HTML text tags that structure the content text and control rules used only for translating the received data into sound. The HTS control parser parses control rules for converting the received data into sound. The HTS control parser modifies entries in one or more of a tag mapping table, an audio data table, a parameter set table, an enunciation modification table and a terminology translation table depending on each of the parsed control rules. The text normalizer modifies enunciation of each text string of the content text of the HTML document for which the enunciation modification table has an entry, according to an enunciation modification indicated in the respective enunciation table entry. The text normalizer also translates each text string of the content text of the HTML document for which the terminology translation table has an entry, according to a translation indicated in the respective terminology translation table entry. The tag converter modifies an intonation and a speed of audio generated from the content text of the HTML document encapsulated by each text tag for which the tag mapping table has an entry, as specified in corresponding entries of the parameter set table pointed to by pointers in the tag mapping table. The tag converter also inserts audio for each text tag for which the tag mapping table has an entry, as specified in corresponding entries of the audio data table pointed to by entries of the tag mapping table. The TTS converter converts the content text of the HTML document, as modified, translated and appended by the text normalizer and the tag converter, to speech audio.

Claims

What is claimed is:

1. A computer system for converting a hyper text markup language (HTML) document into audio signals comprising:

an HTML parser receiving data of an HTML formatted document for parsing out content text, HTML text tags that structure said content text and control rules used only for translating said received data into sound,

an HTML to speech (HTS) control parser for parsing out of said control rules for converting said received data into sound, said HTS control parser modifying entries in one or more of a tag mapping table, an audio data table, a parameter set table, an enunciation modification table and a terminology translation table depending on each of said parsed control rules,

a text normalizer for modifying enunciation of each text string of said content text for which said enunciation modification table has an entry, according to an enunciation modification indicated in said respective enunciation table entry, and for translating each text string of said content text for which said terminology translation table has an entry, according to a translation indicated in said respective terminology translation table entry,

a tag converter for modifying an intonation and a speed of audio generated from said content text encapsulated by, and for inserting audio data at, each text tag for which said tag mapping table has an entry, as specified in corresponding entries of said parameter set table and said audio data table pointed to by pointers in entries of said tag mapping table indexed by each of said text tags, respectively, and

a text to speech converter for converting said content text, as modified, translated and appended by said text normalizer and said tag converter, to speech audio.

2. In a hyper text markup language (HTML) text to speech (HTS) control parser, a method for converting data of an HTML document to speech comprising the steps of:

parsing one or more intonation/speed modification rules that specify intonation and speed modification parameters for generating speech encapsulated by particular text tags of an HTML document and one or more rules that specify audio data to be inserted for particular text tags of an HTML document, and generating a tag mapping table mapping said text tags to corresponding tag identifiers, a parameter set table of entries containing parameter sets pointed to by pointers in corresponding tagged entries of said tag mapping table, and an audio data table of entries containing audio data pointed to by pointers in corresponding tagged entries of said tag mapping table, according to said parsed intonation/speed modification and audio data rules, respectively, and

parsing one or more rules for modifying enunciation of particular strings of content text of an HTML document and one or more rules for translating particular strings of said content text of an HTML document to terms that can be converted to speech by a text to speech converter, and generating an enunciation modification table mapping particular ones of said particular strings to replacement enunciation strings and a terminology translation table mapping particular ones of said particular strings to replacement terminology strings, according to said parsed enunciation modification and terminology translation rules, respectively.

3. In a parcer and text normalizer, a method for converting data of a hyper text markup language (HTML) document to speech audio comprising the steps of:

parsing one or more HTML to speech (HTS) control rules, including generating a tag mapping table entry indexed by an HTML text tag specified in an audio data rule and containing a tag identifier unique to said HTML text tag, and generating an audio data table entry, pointed to by said entry of said tag mapping table indexed by said tag specified in said audio data rule, and containing audio data indicated by said audio data rule,

replacing each instance of a string of one or more content text characters of an HTML document, for which an enunciation modification table has an entry, with an enunciation replacement string of text characters indicated in said entry, said enunciation replacement string being converted to speech audio of a particular one of multiple permissible enunciations of said replaced string of content text characters, and

replacing each instance of a second string of content text characters of an HTML document, for which a terminology translation table has an entry, with a translation string of text characters in said entry, said translation string of text characters being convertible to speech audio, and at least part of said second replaced string of content text characters being unconvertible to speech audio, by a predetermined text to speech converter.

4. In a tag converter for intonation modification and audio data insertion, a method for converting data of a hyper text markup language (HTML) document comprising the steps of:

modifying the intonation and speed of speech audio generated for content text encapsulated by, and inserting audio data at, each instance of an HTML text tag for which a tag mapping table has an entry, an indication to access a parameter set table, and a first pointer to a particular entry of said parameter set table, according to intonation and speed parameters specified in said entry of said parameter set table pointed to by said first pointer, and

generating a particular audio sound for each instance of an HTML text tag, for which said tag mapping table has an entry, an indication to access an audio data table, and a second pointer to a particular entry of said audio data table, from audio data specified in said entry of said audio table pointed to by said second pointer.

5. A method for converting data of a hyper text markup language (HTML) document to speech comprising the steps of:

parsing one or more HTML to speech (HTS) control rules, said step of parsing comprising the steps of:

in response to an intonation/speed rule, generating a tag mapping table entry indexed by an HTML text tag specified in said intonation/speed rule and containing a tag identifier unique to said HTML text tag, and generating a parameter set table entry, pointed to by said entry of said tag mapping table indexed by said tag specified in said intonation/speed rule, and containing a set of intonation and speed parameters indicated by said intonation/speed rule,

in response to an audio data rule, generating a tag mapping table entry indexed by an HTML text tag specified in said audio data rule and containing a tag identifier unique to said HTML text tag, and generating an audio data table entry, pointed to by said entry of said tag mapping table indexed by said tag specified in said audio data rule, and containing audio data indicated by said audio data rule,

in response to an enunciation rule, generating an enunciation table entry indexed by a text string in an HTML document and containing at least a replacement text string, that is converted to a particular audio sound of one of plural enunciations of said index text string, indicated by said enunciation rule, and

in response to a terminology translation rule, generating a terminology translation table entry indexed by a text string in an HTML document that cannot be converted to an audio sound by a predetermined text to speech converter and containing a replacement text string that can be converted to an audio sound by said predetermined text to speech converter.

6. The method of claim 5 further comprising the step of extracting said HTS control rules from HTML comment text of an HTML document.

7. The method of claim 5 further comprising the step of reading said HTS control rules independently from HTML document data.

8. The method of claim 5 further comprising the steps of:

parsing data of an HTML document,

in response to parsing an HTML text tag, attempting to index one or more entries of said tag mapping table using a particular parsed HTML text tag that encapsulates data yet to be parsed,

using a pointer in each indexed tag mapping table entry, to identify entries of said intonation/speed table and said audio data table indicated by said indexed tag mapping table entries,

modifying an intonation and speed by each set of parameters contained in each identified intonation/speed table entry, and

inserting audio data contained in each identified audio table entry.

9. The method of claim 8 further comprising the steps of:

in response to parsing a start HTML text tag, pushing said start HTML text tag onto a stack, and

in response to parsing an end HTML text tag, popping an HTML text tag from said stack,

wherein said particular parsed HTML text tag used in said step of attempting to index is an HTML text tag at a top of said stack.

10. The method of claim 9 further comprising the steps of:

scanning content text of said HTML document,

replacing each content text string of said HTML document that matches one of said text strings that indexes one of said entries in said terminology translation table with said replacement text string contained in said corresponding terminology translation table entry indexed by said matching text string, and

replacing each content text string of said HTML document that matches one of said text strings that indexes one of said entries of said enunciation table with said replacement text string contained in said corresponding enunciation translation table entry indexed by said matching text string.

11. The method of claim 10 wherein a particular entry of said enunciation table further comprises a candidate text string, and wherein said content text string of said HTML document is only replaced with said replacement text string contained in said particular enunciation table entry if said content text string is contained in a second content text string of said HTML document that matches said candidate text string.

12. The method of claim 11 further comprising the steps of:

generating an audible sound including sound generated from said audio data and speech audio generated by converting content text of said HTML document and said replacement text strings, if any, to speech audio according to said intonation and speed parameters.

13. The method of claim 5 further comprising the steps of:

parsing data of an HTML document,

scanning content text of said HTML document,

replacing each content text string of said HTML document that matches one of said text strings that indexes one of said entries in said terminology translation table with said replacement text string contained in said corresponding terminology translation table entry indexed by said matching text string, and

replacing each content text string of said HTML, document that matches one of said text strings that indexes one of said entries of said enunciation table with said replacement text string contained in said corresponding enunciation translation table entry indexed by said matching text string.

14. The method of claim 13 wherein a particular entry of said enunciation table further comprises a candidate text string, and wherein said content text string of said HTML document is only replaced with said replacement text string contained in said particular enunciation table entry if said content text string is contained in a second content text string of said HTML document that matches said candidate text string.

15. In a text normalizer and tag converter, a method for converting data of a hyper text markup language (HTML) document to speech audio comprising the steps of:

replacing each instance of a string of one or more content text characters of an HTML document, for which an enunciation modification table has an entry, with an enunciation replacement string of text characters indicated in said entry, said enunciation replacement string being converted to speech audio of a particular one of multiple permissible enunciations of said replaced string of content text characters,

replacing each instance of a second string of content text characters of an HTML document, for which a terminology translation table has an entry, with a translation string of text characters in said entry, said translation string of text characters being convertible to speech audio, and at least part of said second replaced string of content text characters being unconvertible to speech audio, by a predetermined text to speech converter, and

inserting audio data at each text tag for which a tag mapping table has an entry, as specified in corresponding entries of an audio data table.

Description

COPYRIGHT NOTICE

A portion of the disclosure of this patent document contains material which is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent disclosure, as it appears in the Patent and Trademark Office Patent files or records, but otherwise reserves all copyright rights whatsoever.

FIELD OF THE INVENTION

This invention pertains to converting text documents to audible speech.

BACKGROUND OF THE INVENTION

Text to speech (TTS) converters are devices that convert a text document to audible speech sounds. Such devices are useful for enabling vision impaired individuals to use visible texts. Alternatively, TTS converters are useful for communicating information to any individual in situations where a visual display is not practical, as when the individual is driving or must focus his or her eyes elsewhere, or where a visual display is not present but an audio device, such as a telephone or radio, is present. Such visible texts may originate in tangible (e.g., paper) form and are converted to electronic digital data form by optical scanners and text recognizers. However, there is a large source of electronic or computer originating visual texts, such as from electronic mail (Email), calendar/schedule programs, news and stock quote services and, most notably, the World Wide Web.

In the case of electronic originating texts, speech data may be separately generated, e.g., by digitizing the voice of a human reader of the text. However, digitized voice data consumes a large fraction of storage space and/or transmission capacity--far in excess of the original text itself. It is thus desirable to employ a TTS converter for electronic originating texts.

Generating speech from an electronic originating text intended for visual display presents certain challenges for the TTS converter designers. Most notably, information is present not only from the content of the text itself but also from the manner in which the text is presented, i.e., by capitalization, bolding, italics, listing, etc. Formatting and typesetting codes of a text normally cannot be pronounced. Punctuation marks, which themselves are not spoken, provide information regarding the text. In addition, the pronunciation of text strings, i.e., sequences of one or more characters, is subject to the context in which text is used. The prior art has proposed solutions in an attempt to overcome these problems.

U.S. Pat. No. 5,555,343 discloses a TTS conversion technique which addresses formatting and typesetting codes in a text, contextual use of certain visible characters and formats and punctuation. A first predetermined table maps formatting and positioning codes, such as codes for generating bold, italics or underlined text, to speech commands for changing the speed or volume of the speech. A second predetermined table maps predetermined patterns of visible text, such as numbers separated by a colon (time) or numbers separated by slashes (date or directory), to replacement text strings. A third predetermined table maps punctuation, such as an exclamation point, to speech commands, such as a change in spoken pitch. An inputted text is scanned and spoken and non-spoken characters are mapped according to the tables prior to inputting the text to a TTS converter.

U.S. Pat. No. 5,634,084 discloses another TTS conversion technique. Inputted text is classified according to the context in which it appears. The classified text is then "expanded" by consultation to one or more tables that translate acronyms, initialisms and abbreviation text strings to replacement text strings. The replacement text strings are converted to speech in much the same way as a human reader would convert the text strings. For example, the abbreviation text string "SF, CA" may be replaced with the text string "San Francisco California", the initialism "NASA" may be left unchanged, and the mixed initialism, acronym "MPEG" may be replaced with "m peg."

The most important source of electronic text is the World Wide Web. Most of the electronic texts available from the World Wide Web are formatted according to the hyper text markup language (HTML) standard. Unlike other electronic texts, HTML "source" documents, from which content text is displayed, contain embedded textual tags. For example, the following is an illustrative example of a segment of an HTML source document:

    ______________________________________
    <!BODY BGCOLOR=#DBFFFF>
      <body bgcolor=white>
      <CENTER>
      <map name="Main">
      <area shape="rect"coords="157,12,257,112"href="Main.html">
      <area shape="rect"coords="293,141,393,241"href="VRML.html">
      <area shape="rect"coords="18,141,118,241"href="VRML.html">
      <area shape="rect"coords="157,266,257,366"href="Main.html">
      </map>
      <img src="Images/Main.gif" usemap="#Main" border=0></img>
      <br><br><br><br>
      <b>
      <font size=3 color=black>
      Welcome to the VR workgroup of our company
      </font>
      <a href=
      "http://www.itri.org.tw"><font size=3 color=blue>ITRI</font></a>
     <font size=3 color=black>/</font>
     <a href=
     "http://www.ccl.itri.org.tw"><font size=3 color=blue>CCL</font></a>
     <font size=3 color=black>. We have been<br>
     developing some advanced technologies as fotlows.<br>
     </b>
     <ul>
     <a href="Main.html">
     <li><font size=3 color=blue>PanoVR</font>
     </a>
     <font size=3>(A panoramic image-based VR)</font><br>
     <a href="VRML.html">
     <li><font size=3 color=blue>CyberVR<font>
     </a>
     <font size=3>(A VRML 1.0 browser)</font><br>
     </ul>
     <br><br><a href="Winner.html"><img src=
     "Images/Winner.gif" border=no></img></a><br>
     <a>
     <br><br>
     <font size=3 color=black>
     <br>You are the <img src="cgi-bin/Count.cgi?df=
     vvr.dat"border=0 align=middle>th
     visitor<br>
     </font>
     <HR SIZE=2 WIDTH=480 ALION=CENTER>
     (C) Copyright 1996 Computer and Communication Laboratory,<BR>
     Industrial Technology Research Institute, Taiwan, R.O.C.
     </BODY>
    ______________________________________

• Inventors
  Chung, Jin-Chin; Hwang, Shaw-Hwa; Chung, Chung-Ping;
• Assignee
  Industrial Technology Research Institute (TW)
• Published
  Sep-5-2000
• Current US Classes:
  704/260
  704/270
  715/513
• Application #
  053629
• International Classes
  G10L 013/00; G06F 017/22
• Field of Search
  704/258 704/260 704/266 704/267 704/269 704/270 704/275 379/88.16 707/513
• Examiner
  Hudspeth; David R.
• Agent
  Proskauer Rose LLP
• US Patent References:
  5555343
  5634084
  5864814
  5884266
  5890123
  5899975
  5983184