System for optically entering, displaying and decoding handwritten symbols

Abstract

An optical data entry and display system for decoding handwritten symbols into a data processing code is disclosed. The system comprises a television monitor for displaying decoded and otherwise derived symbols as well as for displaying a sequence of light spots on which a symbol may be traced by means of a light pen. The positions of the light spots comprising the trace are used in a comparison with a set of stored values defining the predetermined symbols to decode the trace.

Claims

The embodiments of the invention in which an exclusive property or privilege is claimed are defined as follows:

1. An interactive optical data entry and display system for decoding a handwritten trace into a data processing code defined by a set of predetermined symbols, said system comprising:

(a) a control means for generating a signal sequence;

(b) a display responsive to said signal seequence for displaying a corresponding pattern of display elements;

(c) tracing means for making said handwritten trace on said display;

(d) sensing means in association with said control means and said tracing means for identifying, in terms of a first coordinate and a second coordinate, the positions of those display elements passed over by said tracing means while making said handwritten trace; and

(e) computing means responsive to said sensing means and said control means comprising:

i. for each of said predetermined symbols, a stored trace uniquely corresponding thereto;

ii. means for comparing said handwritten trace with each of said stored traces such that the best comparison identifies said written trace as the symbol corresponding to the stored trace which caused said best comparison;

iii. means for tabulating a first ordered list of values corresponding to those of said first coordinates identified by said handwritten trace and a second ordered list of values corresponding to those of said second coordinates identified by said hand-written trace;

iv. a plurality of matrices; and

v. calculating means for multiplying one of said matrices with said first ordered list to derive a first vector and for multiplying one of said matrices with said second ordered list to derive a second vector.

2. A system according to claim 1, wherein said control means comprises a timing circuit and a plurality of registers and wherein said display is responsive to said timing circuit such that each of said display elements is uniquely defined by a particular state of said timing circuit.

3. A system according to claim 2 wherein said display is responsive to said plurality of registers such that the display image produced by said display is defined by the contents of said plurality of registers and wherein said tracing means comprises a lightpen responsive to said display elements and said sensing means comprises a sensing circuit adapted to generate an output in response to inputs from said lightpen.

4. A system according to claim 3, wherein said computing means includes means responsive to said timing circuit and to said output of said sensing circuit wherein the states of said timing circuit corresponding to the generation of said output of said sensing circuit define a first series of values descriptive of said handwritten trace on said display .

5. A system according to claim 4, wherein each of said stored traces comprises a series of stored characteristic values uniquely identifying a corresponding one of said predetermined symbols and wherein said computing means includes means for comparing said first series of values with each of said series of stored characteristic values such that the best value comparison identifies said handwritten trace as that symbol corresponding to the series of stored characteristic values causing said best value comparison.

6. A system according to claim 1, wherein said computing means includes interpolation means for interpolating said first and second ordered lists such that their lengths are equal and wherein said plurality of matrices comprisees a simple matrix having a dimension compatible with said length of said first and second ordered lists.

7. A system according to claim 6, wherein said single matric comprises numbers such that when said single matrix is multiplied with the sampled values of a periodic function the coefficients of the Fourier Series expansion of said periodic function results.

8. A system according to claim 7, wherein said interpolation means includes means for interpolating said first and second lists such that their length is eight and wherein said single matrix is

    ______________________________________
    1.5  1       0       -1    -1.5  -1    0     1
    1    0       -1      0     1     0     -1    0
    0    -1      -1.5    -1    0     1     1.5   1
    ______________________________________

Description

BACKGROUND OF THE INVENTION

The present invention relates in general to the field of data processing and data communication systems and more specifically to man-machine apparatus in these systems.

The field of data processing has experienced rapid growth. A particularly active area in this field is equipment for use by an operator to interact with these data processing systems. Equipment of this type functions in the man-machine interface and includes, for example, tele-typewriters, keyboard-CRT's and cardpunch apparatus. The present invention is an entirely novel means to perform the man-machine interface function.

The present invention requires on the part of man no more than ordinary writing skills, that is, man writes with a pen on the face of a display and a programmed processor decodes the writing into any code commonly used in data processing systems.

The set of symbols to be used includes alpha-numeric characters as well as non-alpha-numeric symbols such as punctuation signs and control signs. The symbols are those presently in common use, but may also include symbols not commonly known, for example for use in control functions.

Input verification is a required function in the man-machine interface. In the present invention verification is facilitated by the display of the decoded symbol in the position where the writing took place. Analogous to the manner in which man writing with a pencil on paper will automatically review his writing, in the present invention input verification is near automatic.

Current man-machine data entry apparatus are, in part, mechanical. An apparatus in accordance with the present invention is all electronic. The electronic components used are the same or substantially similar to the components used in data processing or calculating equipments. The many advances in the technology of these components, which are continuously being made, are directly applicable to the present invention for improvements in use, reliability and economy. By the way of example, for reason of reducing the bulk of current data terminals and television sets, much attention is given to flat panel displays. This advance will allow implementation of this invention in a manner of near complete analogy with notepad and pencil.

The present invention implemented with the same general physical dimensions as a notepad will greatly enhance the use of a man-machine interface. Through standard telephone communication connections, the sophisticated software of central data processing systems becomes available with ease.

Many prior art systems have attempted to take advantage of man's manual dexterity in the manipulation of a stylus to effect the input of data into data processing or data communication systems. Exemplarly of one type of such system is U.S. Pat. No. 3,487,371 issued to Frank. This patent teaches a system in which a stylus is mechanically linked to a position determining means with display effected separate and displaced from the position of writing. Writing with the stylus causes the display to show a decoded symbol. Inability of this system to display the trace of the stylus while writing and the display of the decoded symbol in the place of writing, as well as, the electronic complexity severely hampers the use of this system.

Other systems employing an optically sensitive stylus, commonly known as a lightpen, effect data input by pointing the lightpen at a desired symbol or at a plurality of segments constructing a symbol. Exemplary of such apparatus are U.S. Pat. No. 3,768,073 issued to Rawson and U.S. Pat. No. 3,760,373 issued to Bartz. Both apparatus employ light emitting diodes located in a limited matrix of positions for display. This limited matrix of light point positions severely limits the flexibility and ease of use for data input and writing.

Other systems employing a lightpen and a cathode ray tube display are primarily intended for the input of a graphic data, and the manipulation, with the assistance of computer software, of the graphics. Exemplary of these systems are U.S. Pat. No. 3,534,338 issued to Christensen et al. and U.S. Pat. No. 3,653,001 issued to Ninke. In these systems a small on-site computer performs routine and less sophisticated tasks, such as for example tracking the lightpen, while a central computer is used to perform tasks requiring extensive software. The appeal of these systems is limited by their inability to interpret the graphics.

Other graphics systems are designed to transmit handwritten data or drawings as a facsimile. Exemplary of these systems is U.S. Pat. No. 3,559,182 issued to Floret et al. This system effects the transmission of a handwritten trace to a plurality of television sets, thus providing a message transmission capacity. The message however can only be understood if the receiver is a human capable of reading the writing, an obvious limitation in the usefulness of this system.

Accordingly it is an object of the present invention to provide a man-machine interface apparatus capable of communicating with a remote data processing facility in a data processing code.

It is a further object of the present invention to provide the capacity to interpret and decode handwritten "graphic" data into a data processing code.

It is a further object of the present invention to achieve decoding of handwritten graphics with economical electronic components.

It is a further object of the present invention to provide a data input means with a pen writing the desired data.

It is a further object of the present invention, for ease of use and verification, to provide display of the writing and the decoded symbol where the writing takes place.

SUMMARY OF THE INVENTION

In accordance with these and other useful objects, the present invention discloses an optical data entry and display system for decoding handwritten symbols into a machine code defined by a set of predetermined symbols.

The system includes a television monitor or the like which is interfaced to a digital computing means by a timing circuit and a plurality of registers. Under instruction of the computing means, the plurality of registers, in association with the timing circuit, causes the television monitor to display a defined sequence of display elements or light spots. The position of each of the display elements or light spots is uniquely defined by a particular state of the timing circuit.

A light pen is used to enter a desired symbol by tracing the symbol on the defined sequence of display elements. The light pen, which is sensitive to light emitted from the display elements, activates a sensing circuit each time a light hit is detected. The sensing circuit, in turn, addresses the plurality of registers so as to cause the display element corresponding to the detected light hit to change state whereby visual verification of the traced symbol is achieved. In addition, upon the detection of a light hit, the sensing circuit causes the computing means to sample the output of the timing circuit whose state defines the position of the light hit.

The computing means compares the data representing light hit positions for a traced symbol with stored data uniquely defining each of the predetermined symbols. This comparison is facilitated by means of a Fourier analysis performed on the light hit positions. Finally, the best comparison defines the traced symbol as a particular symbol from the set of predetermined symbols. That is, the comparison decodes the traced handwritten symbol into a data processing code defined by a particular symbol from the set of predetermined symbols.

The data processing code may subsequently be presented to a character generator which provides video signals to the display for showing the decoded symbol.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a general block diagram showing the major functional hardware elements of the preferred embodiment of the present invention.

FIG. 2 is a detailed representation of the manner of display employed in the present invention.

FIG. 3 is a detailed block diagram of the register shown generally in FIG. 1.

FIG. 4 is a detailed block diagram of the timer shown generally in FIG. 1.

FIG. 5 is a detailed block diagram of the sensor shown generally in FIG. 1.

FIG. 6 is a general flow chart illustrating the major functional software modules useful in practicing the present invention.

FIG. 7 shows the relative values of a series of data points and illustrates a group of interpolated values for the series.

FIG. 8 is a flow chart showing the program for implementing the improved decoding algorithm of the present invention.

FIG. 9 is a first section of a flow chart showing a program for decoding input data useful in practicing the present invention.

FIG. 10 is a second section of the flow chart shown in FIG. 9.

FIG. 11 is a general block diagram showing the major functional hardware of another preferred embodiment of the present invention.

FIG. 12 is a detailed block diagram of a digital computing means for implementation in digital large scale integrated circuits.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

The apparatus and methods of the present invention are useful in any of a number of computer based data entry and/or data display systems. Although the decoding and display systems of the present invention are disclosed in terms of a preferred embodiment, this has been done for purposes of an exemplary showing and it will be understood by one skilled in the art that the means described are for purposes of example only, and are not intended to be limiting.

FIG. 1 is a diagrammatic illustration showing the major functional hardware elements of the preferred embodiment of the present invention. In particular, the basic decoding and display functions of the present invention are controlled by a computing means shown generally at 100. The computing means 100 includes a general purpose programmable digital computer 101 and control circuits 102 and 105. Interfacing connections between the digital computer 101 and the control circuits 102 and 105 are generally shown at 106 and 107 respectively. The control circuit 102 serves to interface the digital computer 101 to a register 200 via line 201, to a timing circuit 300 via line 103, and to a sensing circuit 400 via line 104. Similarly, the control circuit 105 serves to interface the digital computer 101, via line 701, to a bank of peripheral equipment shown generally at 700. The peripheral equipment 700 may typically include an interface control circuit 702 and any of a number of well-known peripheral apparatus diagrammatically shown at 703. The control circuit 702 and the peripheral apparatus 703 communicate by means of connection 704. In the foregoing and following discussions, it will be understood by one skilled in the art that a connection identified in the figures by a single line may, in fact, comprise a plurality of individual connecting means each of which may conduct one or a sequential series of data elements or signals. By way of example, in the preferred embodiment of the present invention line 201 includes nine wires, six wires conducting six parallel information data elements and the remaining three wires conducting various control signals.

The basic function of register 200 is the generation of video signals for application to the display 500 overline 501. The display 500, in response to the video signals on line 501 as well as timing signals on line 502 from timer 300, generates a unique image on the face of the display 500. This unique image, which consists of a plurality of individual image elements, will be discussed in further detail hereinafter.

A lightpen 600, which is capable of detecting light within its field of view in a manner well-known in the art, is utilized to detect the presence of an image element on the face of the display 500. The detected light will be transmitted over line 401 to a sensor 400 which, upon determining that a light hit has occurred, will output control signals to the computing means 100 and the register 200 over lines 104 and 203 respectively. The output on line 104 will cause the control circuit 102 to read from the timer 300 two binary words, referred to hereinafter as the first binary word and the second binary word, defining the position of the lightpen 600 on the face of the display 500. The control circuit 102 will then interact with the interrupt hardware and program of the digital computer 101 so as to cause the computer 101 to store the first and second binary words in its memory.

Simultaneously, the output on line 203 from the sensor 400 will cause the register 200 to reflect the position of the lightpen 600 by causing a first change in the video output on line 501, whereby the image element corresponding to the position of lightpen 600 will undergo a change of state from light to dark. This change of state will then provide visual feedback regarding the position of the lightpen 600. By retaining a sequence of such first changes, the register 200 will cause the display 500 to provide visual verification of the trace of the lightpen 600 on the face of the display as the operator traces out a symbol.

In addition, certain of the first and second binary words will cause the computer 101 to detect a first reset condition. Upon the detection of this first reset condition, the computer 101 will cause the control circuit 102 to generate a first reset signal. The first reset signal will be applied over line 201 causing the erasure of the first changes in the register 200 and thereby erasing the lightpen trace.

The timer 300 is utilized to generate the various synchronizing signals necessary to make the computing means 100, the register 200, and the display 500 operate in unison. In this regard, synchronizing signals are applied from the timer 300 to the otherwise free running display 500 over line 502. Register 200 receives timing signals over line 202 which provides for the performance of register shift operations so as to enable the presentation of video information on line 501 in synchronization with the synchronizing signals applied to the display 500 over line 502. Also, by means of connection 402, the timer 300 applies a measurement enable signal to the sensor 400 to enable its circuitry at appropriate times.

In the preferred embodiment of the present invention the register 200 has a capacity to store data for the display of one sentence of 32 symbols, i.e., one row of symbols on the face of the display 500. Therefore, after one sentence of symbols has been displayed on the display 500, the timer 300 must request from the computer 101 the information for the next sentence. This request is made by means of a signal on line 103. The computer 101 when outputs data for the next sentence to the register 200 over line 201 in such a manner that the output is completed before timer 300 causes the register 200 to initiate its video output on line 501 for the next sentence.

As previously mentioned, a computing means 100 interfaces with a bank of computer peripheral equipment 700, by means of line 701. Although various computer peripheral equipment are currently available and within the scope of the present invention, the preferred embodiment described herein contemplates the use of a teletypewriter with a keyboard, a printer, a papertape reader and a papertape punch. Also, the preferred embodiment contemplates the use of a modulator/demodulator system (MODEM) for interfacing the computing means 100 to one or more data communication facilities. Finally, the preferred embodiment also contemplates the use of peripheral equipment for connecting the computing means 100 directly to a data processing facility.

Because of their unique functions in connection with the system of the present invention the display 500, the lightpen 600, and the controlling circuits including the register 200, the timer 300 and the sensor 400, have been singled out for separate presentation and discussion hereinafter. However, before undertaking a detailed discussion of the aforementioned hardware, it is considered useful to initially examine the characteristics of the visual display produced on the face of the display 500.

With regard to the display 500 itself, this may comprise a modified television set whose mode of operation closely resembles the operation of a typical television set. That is, an electronically modulated beam of electrons is caused to sweep across the face of the display causing the coating material of the display face to either emit light or to remain non-illuminated, depending upon the modulation of the beam of electrons. For example, display 500 may typically comprise a Ball Brothers Research Corporation Model TV-5-STD data display.

An example of the function of the present invention is given hereinafter in connection with FIG. 6. An appreciation of one detailed functional objective of this invention and its manner of attainment is considered helpful in understanding the constituent elements of the invention, even though presentation of the example must be deferred until a presentation of the constituent elements has been made. For this reason, this example is referenced for information now.

As previously mentioned, register 200 has the capacity to store data for the display of one sentence of 32 symbols on the face of display 500. Each of the symbols will be displayed in accordance with the general organization shown in FIG. 2. FIG. 2 shows in part, the letter E 510 and adjacent thereto a letterarea 530 comprising a letterborder 511, a letterspace 512 and a letterend 513. In addition, FIG. 2 shows a series of coordinates XC, XF, YC and YF which uniquely identify each video element capable of being displayed on the face of display 500. The coordinates XC, XF, YC and YF are generated by a series of counters included within timer 300, which will be described in further detail hereinafter. In general terms, the counters include an X-coordinate Coarse counter XC, an X-coordinate Fine counter XF, a Y-coordinate Coarse counter YC, and a Y-coordinate Fine counter YF.

This function of display 500 will be explained in connection with the coordinates XC, XF, YC and YF, where it is understood that these coordinates are representations for the states of the coordinate counters XC, XF, YC and YF, respectively. By way of example, coordinates XF 0 in FIG. 2 represents a state called 0 of the XF counter.

Referring again to FIG. 2, a beam of electrons is caused to sweep across the face of display 500 such that the beam will enter at the position identified by 514. Position 514 identifies a point on the face of display 500 with coordinates YF = 17, XF = 0, XC = M-1, and YC = N-1. The beam will then continue for increasing XF and XC, while YF and YC remain constant. When XF and XC have reached the position identified by XC = 32 and XF = 11, the timer 300 will cause the beam of electrons to be modulated off. The state XC = 32 and XF = 11 will cause the next state of YF, in this example YF = 18. As shown in FIG. 2, the beam of electrons is modulated off during YF = 18 and YF = 19.

It will be appreciated that the smallest increment at which the electronic controls are capable of modulating the electron beam on and off correspond to the duration of one stae of counter XF. Therefore, the fastest rate of turning the electron beam on and then off is diagrammatically represented by positions 515 through 520. Each position 515 through 520 represents a lightspot corresponding to counter state YC = N-1, YF = 0, XC = M-1 and XF = 0, 2, 4, 6, 8 and 10.

The generation of symbols such as alphanumeric characters in a matrix of lightspots consisting of five horizontal positions and seven vertical positions in a technique well known in the art. The letter E 510 is drawn on such a 5.times.7 matrix. This 5.times.7 matrix corresponds to the letterspace 512 of letterarea 530. Blocks 521 and 531 are diagrammatic representations of corresponding lightspots in the 5.times.7 matrix. By appropriately modulating the beam of electrons the letter E 510 as well as the letterarea 530 can be made to appear to the face of display 500. Thus, it is clear that each of the images 510, 511, 512 and 513 can be achieved by appropriately modulating the electron beam during a plurality of unique states of counters, XF, XC, YF and YC.

As detailed above, the position of lightpen 600 on the face of display 500 can be determined from the lightspot uniquely identified by a particular set of XF, XC, YF and YC values. In this connection, the previously mentioned first and second binary words will contain the values of the coordinates XC, XF and YF. The first binary word includes the values for XF and XC, while the second binary word includes the values for XC and YF. It will therefore be apparent that the character position in a particular sentence can be identified by either the first or second binary word, whereas the unique identification of a lightspot position requires information from both binary words.

As will be discussed in further detail hereinafter, register 200 includes a letterspace register which contains the requisite information for modulating the electron beam so as to cause the letterspace 512 to be displayed. The initial information stored in the letterspace register will cause all letterspace display elements to be "on", i.e., illuminated. When an operator writes on the letterspace 512 with the lightpen 600 positioned over a particular letterspace display element, the sensor 400 detects the light emitted thereby and causes a change in the information stored in the letterspace register. This change will result in the particular letterspace display element being extinguished, i.e., non-illuminated, giving the operator a visual feedback of his writing. The changes in information content of the letterspace register are retained until the computer 101 causes the first reset signal to be generated whereupon the information in the letterspace register will be returned to its initial or "on" condition.

It will be clear from an inspection of FIG. 2 that many detail design variations can be made in the letterarea 530. For example, the sections of the letterborder 511 can be made to overlap or, in the alternative, they can be designed to only partially enclose the letterspace 512. It is also possible to replace any section of the letterborder 511 with the letterend 513 as well as to extend the lighted area of letterborder 511 to meet letterspace 512 on any or all sides of letterspace 512. These and other variations can be implemented by those skilled in the art of logic design by an appropriate design of the gating networks in the timer 300.

FIG. 13 is a detailed representation of register 200. As generally shown in FIG. 1, line 201 connects the register 200 to the output of the control circuit 102; line 203 connects the register 200 to the output of the sensor 400; line 202 connects the register 200 to the output of the timer 300; and line 501 connects the register 200 to the input of the display 500.

The register 200 includes a sentence register 208 typically comprising a shift register integrated circuit model TM 3112 Hex 32 Bit Static Shift Register manufactured by Texas Instruments, Inc. In response to inputs from the control circuit 102 on lines 201 and 204, the sentence register 208 can be filled with an ordered sequence of thirty-two, six-bit codes. Each of the six-bit codes may represent a symbol in accordance with the U.S.A. Standard Code for Information Interchange USASCII by eliminating the seventh code bit. After being placed in the sentence register 208, the ordered sequence of thirty-two, six-bit codes can be shifted by clock signals derived from the timer 300 over line 202 to present on output line 209 the ordered sequence of codes. The ordered code sequence can be presented on output line 209 repeatedly.

The output 209 of the sentence register 208 connects to a row generator 210. Row generator 210 may typically comprise a Model TM 2501, 2560 Bit Static Read-Only Memory manufactured by Texas Instruments, Inc. The characteristics of row genertor 210 are such that under conditions of six inputs on line 209 and three inputs on line 211, five outputs can be achieved on output line 212. The five outputs on line 212 represents in binary form one row of seven rows of a 5.times.7 binary matrix. This 5.times.7 binary matrix corresponds to the letterspace 512 shown in FIG. 2. By appropriately pulsing the three inputs represented by line 211, all of the seven rows can be output on line 22 sequentially. It will be clear that by changing the input code on line 209, and the inputs on line 211, all symbols defined for the row generator 210 can be caused to appear on line 212 sequentially. Furthermore, by sequentially outputting codes on line 209, while maintaining the three inputs of line 211 in the same state, the same row of all thirty-two symbols contained in sentence register 208 can be caused to appear on output line 212.

Output line 212 of the row generator 210 connects to the input of a row register 213. Row register 213 is designed to accept the five outputs on line 212 from row generator 210 and to subsequently shift out five corresponding information bits on output line 214. The shifting action of row register 213 is controlled by appropriate clock pulses derived from the timer 300 over line 202.

A row select circuit interfaces with the row generator 210 and controls the state of the three inputs to row generator appearing on line 211. Row select 215 is designed to sequence the three inputs on line 211 in a manner such that a first row of all symbols contained in sentence register 208 will appear on output line 212, followed by a second row, etc. Appropriate clock pulses for row select 215 are derived from the timer 300 over line 202.

It will be recognized that the combined operation of sentence register 208, row generator 210, row select 215 and row register 213 will enable the display of symbols on display 500 such as the letter E shown in FIG. 2. Accordingly, any defined symbol may be displayed on the face of display 500 by means of the aforementiond circuitry.

The generation of the letterarea 530 is accomplished by means of the letterspace register 207, the letterborder generator 218, the letterend generator 219 and the letterarea register 217. One unique six-bit code is designated to represent the letterarea 530. When this unique six-bit code appears on line 209 it will be detected by the letterarea register 217. The letterarea register 217, in turn, activates the letterspace register 207, the letterborder generator 218 and the letterend generator 219 through connection 220 and 223. However, since an inherent delay in the circuitry of row generator 210 causes its output on line 212 to appear later than the coresponding input on line 209, the letterarea register 217 must delay the inputs to the letterspace register 207, the letterborder generator 218 and the letterend generator 219 so as to provide synchronization between the outputs appearing on line 212 and lines 221, 222 and 224.

As previously discussed in connection with FIG. 2, every display element of letterborder 511 is uniquely defined by an XF coordinate, represented by the state of an XF counter, and a YF coordinate, represented by the state of a YF counter. The letterborder generator 218 comprises a network of logic gates adapted to generate on output line 222 the signals which will cause display 500 to display the letterborder 511. It will be appreciated by one skilled in the art that the logic network of letterborder generator 218 will implement a logical sum of products operation, in which each product of the form XF "logical and" YF represents a letterborder display element, and in which the sum is the sum of products of all letterborder elements.

The letterspace register 207 includes a register stage for every display element in letterspace 512. The letterspace register 207 is shifted, in response to clock signals appearing on line 202, to sequentially output on line 221 the modulation information contained therein. When lightpen 600 is positioned over letterspace 512 and sensor 400 detects the presence of light, the stage in letterspace register 207 containing the modulation information for that lightspot will change state. In this manner, all "hits" detected by sensor 400 are recorded in the corresponding stage of the letterspace register 207 whereby the modulation information appearing on line 221 will reflect the sequence of detected "hits" and consequently show the trace of lightpen 600. The letterspace register 207 will retain the modulation information indicative of the trace until the first reset signal on line 205 causes the letterspace register 207 to reset to its initial "no trace" condition. For those skilled in the art of logic design the methods of analysis and design for a register that meets the requirement of letterspace register 207 are well known.

As in the case with the letterborder 511, the letterend 513 as depicted in FIG. 2 comprises a series of display elements each uniquely defined by the state of counter XF and the state of counter YF. Therefore, the letterend generator 219 can be reduced by one skilled in the art to a logic network implementing a logical sum of products representing all display elements of the letterend 512. When enabled, letterend generator 219 will then generate an output on line 224 the signals which will cause display 500 to display the letterend 513.

The output gates 225 merge the information signals appearing on line 214 and 221, 222, 224 to provide a composite video output for the display 500 on line 510. The composite video signals appearing on line 501 are then utilized, in conjunction with synchronizing signals appearing on line 502, by the display 500 to generate the visual image. It will be recognized that the output gates 225 are responsive to control signals appearing on line 220 which indicates the presence/absence of a letterarea code on line 209. In addition, the output gates 225 receive synchronizing signals from the timer 300 over line 202. Finally, and as will be described in further detail hereinafter, measurement enable video signals are applied to the output gates 225 from the timer 300 over line 202. The design of a network of logic gates which meet the requirements for output gates 225 is well known to those skilled in the art of logic design.

The timer 300 shown generally in FIG. 1 is shown in further detail in FIG. 4. It will therein be noted that line 103 connecting to the control circuit 102 comprises lines 306, 310, 315 and 320 merging at juncture 328; that line 502 connecting to the display 500 comprises line 324 and 325 merging at juncture 329; and that line 202 connecting to register 200 comprises lines 103, 326, 327, 303 and 333 merging at juncture 330.

The timer 300 derives its basic timing signals from a crystal clock generator 302. The square wave clock signal output from the crystal clock 302 appears on line 303 and, in the preferred embodiment, has a clock pulse period of approximately 0.1357 microseconds. Also included in the timer 300 is the XF counter 304, the XC counter 309, the YF counter 314, and the YC counter 319. One of a series of network gates 307, 311, 316 and 334 is associated with each of the counters. The counters 304, 309, 314 and 319 may typically comprise binary counting integrated circuits of the type manufactured by Texas Instruments, Inc. as SN 74163.

One function of timer 300 is generation of the synchronizing and timing signals required by the display 500. A typical display of the type contemplated by display 500 may require a vertical blanking pulse having a nominal duration of 900 microseconds, a horizontal blanking pulse having a nominal duration of 11 microseconds, a vertical drive pulse having a duration between 300 and 1400 microseconds, and a horizontal drive pulse having a duration of 25 to 30 microseconds. In addition, the duration of a video line across the face of the display 500, the line time, may typically be 63.5 microseconds. Another timing requirement is the symbol time which may be defined as the time required by the electron beam to sweep across the width of a symbol. In the preferred embodiment, the symbol time has been set at twelve clock pulse periods. Therefore, the requirement for horizontal blanking can be fulfilled by counting seven symbol times. And, since the display of 32 symbols in a sentence requires 32 symbol times, the sum of the horizontal blanking and 32 symbol times fulfills the line time requirement of 63.5 microseconds. Finally, the horizontal drive time may be derived from the horizontal blanking plus ten symbol times.

In order to meet the timing requirements set forth above, the XC counter 309 must count to seven (for horizontal blanking) and to thirty-two (one sentence). Block 311 comprises a network of logic gates adapted to detect the terminal counts, seven and 32, of XC counter 309 to appropriately reset the counter.

The time required for the beam of electrons to sweep across the letterarea 530, the letterarea time, is eighteen clock pulse periods. Since a sentence containing the letterarea would exceed the line time, 63.5 microseconds, in the preferred embodiment the time for the last symbol in a sentence is shortened by an amount equal to the difference between a symbol time (twelve clock pulse periods) and a letterarea time (eighteen clock pulse periods). Thus, the XF counter 304 is required to count to twelve, eighteen and six (the difference between twelve and eighteen). Gates 307 comprise a network of logic gates adapted to detect the terminal counts of and reset the XF counter 304.

As will be noted upon examination of FIG. 2, in the present preferred embodiment, 32 video lines are used to display one sentence including one space between sentences. Also, the vertical blanking time can be derived by counting fourteen lines which can also be used to fulfill the vertical drive time requirement. Accordingly, the YF counter 314 must count to 32 and 14. Gates 316 comprise a network of logic gates adapted to detect the terminal counts of and to reset the YF counter 314 accordingly.

The display 500 must be capable of displaying a member of sentences, typically eight. Accordingly, the YC counter 319 must count to eight. The terminal state of the YC counter 319 is detected by AND gate 334 whose output appears on line 323 and is used to reset the YC counter 319 and to indicate to gates 316 that the vertical blanking pulse must be generated.

Finally, the characteristics of the cathode ray tube of the display 500, the optic fiber in lightpen 600 and the sensor 400 are such that in order to determine the position of lightpen 600 the lightpen trace must be momentarily erased. This momentary erasure rendering all display elements in letterspace 512 "on" or illuminated, is controlled by the measurement enable counter diagrammatically represented by block 331. The measurement enable counter 331 counts display frames, and may typically comprise a two stage binary counter. The AND gate diagrammatically represented by block 332 decodes one of the four states of the two stage binary counter 331 and enables display of the trace during this state. During the remaining three states of the two stage binary counter 331 the trace is not enabled whereby measurement of the lightpen position may be accomplished. The output 402 of AND gate 332 is used to enable the sensor 400. The output 333 from AND gate 332 appears on line 202 and is applied to output gates 225, see FIG. 3, to erase the lightpen trace three out of every four frames.

FIG. 5 shows the circuit for the sensor 400. The optic fiber 401 optically connects the lightpen 600 to a photodiode 404 as diagrammatically illustrated by arrow 403. The lightpen 600 is typically a stylus type apparatus suitable for writing and, in the preferred embodiment, comprises a mechanical pencil model 0.5 mm sold under the tradename PENTEL modified by drilling a hole through the eraser. Optic fiber 401 may typically comprise 20 mil Crofon optic fiber manufactured by the DuPont Corp. Photodiode 404 may typically comprise a PIN photodiode model MD-1 manufactured by the Monsanto Commercial Product Corp.

Changes in the conduction characteristics of photodiode 404 due to incident light energy are sensed by amplifier 405, which generates an output to decision flip-flop 406. When the output of amplifier 405 exceeds a preset level, decison flip-flop 406 is set and its output is counted by output counter 407. Output 407 generates an output on line 104 to the control circuit 102 and an output on line 203 to the register 200 when a count of three has been achieved. As previously discussed, the output on line 203 will cause display of the trace and the output on line 104 will cause control circuit 102 to acquire the first and second binary words indicative of the position of the lightpen 600 on the face of the display 500.

The sensor 400 receives various reset signals from the timer 300 over line 402. The reset signals appearing on line 408 occur every time a horizontal blanking signal is generated and are thus utilized to reset the decision flip-flop 406 at the beginning of every T.V. line. The reset signals appearing on line 409 occur every time the vertical blanking signal is generated as well as during each frame in which lightpen position measurement is not enabled (i.e., the one frame out of four during which the trace is displayed). In this manner, the output counter 407 is held at 0 during the display of the lightpen trace, and, in addition, is reset once during every frame. Consequently, the output counter 407 generates output signals on lines 104 and 203 only when the decision flip-flop 406 is set the third time during a frame when the lightpen position measurement is enabled. This is done to select the display element under the lightpen from the large number of display elements seen by the wide field of view of the optic fiber.

As noted previously, the computing means 100 includes a digital computer 101. Digital computer 101 may be of the PDP-8 family of mini or microcomputers pioneered and manufactured primarily by Digital Equipment Corporation and typically may be of the microprocessor type MP-12 manufactured by FabriTek, Inc. In accordance with the methods and apparatus of the preferred embodiment of the present invention, computer 101 is programmed, in general terms, in accordance with the flow chart shown in FIG. 6. It will be appreciated that the main function of computer 101 is to appropriately acquire the first and second binary words and, on the basis thereof, to decode the handwritten symbols on the face of the display 500. In order to properly accomplish this function, various other anciliary functions must also be accomplished.

It will be appreciated that the flow chart is to be implemented in instructions stored in the core memory of computer 101. In addition, the preferred embodiment contemplates the storage of these instructions in the interconnections of a read-only-memory ROM for use with more modern computing means of the PDP-8 family.

The operating program associated with the present invention and shown by the flow chart of FIG. 6 is initially called to perform its functions by an interrupt caused by timer 300 or sensor 400. After performing its function the computer 101 returns to the interrupted program until a next interrupt occurs. The interrupted program is diagrammatically represented in FIG. 6 by block 800, background, whose functions generally may comprise those interfacing the computer 101 to the peripheral equipment 700. Once during the execution of each instruction of the background program, the logic hardware of computer 101 will test the interrupt circuit. In the absence of an interrupt, diagrammatically represented by the no exit from block 801, the next instruction in the background program will be executed. However, in the case of the occurrence of an interrupt, diagrammatically represented by the yes exit from block 801, an Interrupt Subprogram, represented by clock 802, will be executed.

The function of the Interrupt Subprogram 802 is to ascertain the source of the interupt and to award an appropriate priority thereto. The Interrupt Subprogram 802 must also determine the current service status of any prior received interrupts. Upon determining that the interupt must be serviced, the Interrupt Subprogram 802 will prepare the computer for return to the interrupted program 800 after completion of the functions necessary to service the interrupt.

These are various interrupts generated by the circuitry of the present invention which the Interrupt Subprogram 802 must be capable of processing. Initially, since register 200 contains storage for only one sentence, the computer 101 must, in addition to being instructed when to output data for the sentence at the top of the display, also be instructed when to output data for a following sentence. Therefore, the timer 300 generates two types of interrupts. A first type of interrupt is generated by the timer 300 at the beginning of each frame and a second type is generated after the display of a complete sentence except for the last sentence when a first type is generated. Computer 101 will then output data for a first sentence in response to an interrupt of the first type and will output data for subsequent sentences in response to interrupts of the second type. In addition to the first and second types of interrupts generated by the timer 300, the Interrupt Subprogram 802 must be capable of processing a lightpen interrupt from sensor 400. In response to the lightpen interrupt, the computer 101 will place the first and second binary words in its memory.

Returning to the flow chart of FIG. 6, if the interrupt received by Interrupt Subprogram 802 is of the first type, diagrammatically represented by the yes exit from block 804, the instructions represented by the block 805 will reset the output data pointer whereby the instructions represented by block 806 will cause the output of a sequence of six-bit codes representative of the sequence of symbols to be displayed for the first sentence. If the interrupt is of the second type, diagrammatically represented by the no exit from block 804, the instructions represented by block 806 will cause the output of six-bit codes for the next sentence.

If the interrupt was a lightpen interrupt, instructions diagrammatically represented by block 808 will be executed causing the input of the first and second binary words. Since a lightpen interrupt can be caused either by light from a displayed symbol or light from letterarea 530, the condition which caused the interrupt must be determined. This determination is made by instructions represented by blocks 809 through 813. The action taken as a consequence of the decision made in blocks 809 through 813 are represented by blocks 1000, 900 and blocks 814 through 818. For reason of their particular significance to the present invention, the instructions diagrammatically represented by blocks 900 and 1000 will be described in further detail hereinafter. In general, the instructions represented by block 1000 cause the input data to be either entered into a first file or ignored, in the case of repetitious data. The instructions diagrammatically represented by block 900 cause the data stored in the first file to be decoded into a data processing code representative of the traced symbol.

Referring to the blocks 809 through 813, if the lightpen interrupt was caused under the condition Letterarea Not Defined (i.e., no letterarea appearing on the face of the display 500), the no exit from block 809, then a letterarea will be defined in the position of the lightpen 600. This is accomplished by the instructions diagrammatically represented by block 817. Block 817 calls up from a non-displayed storage location the letterarea code and inserts it in place of the code of the symbol the lightpen is pointing at. And, at the same time, block 817 will cause the code for the replaced symbol to be placed in a storage location hereinafter referred to as RETAIN. If, on the other hand, a letterarea is defined and the lightpen 600 is pointed at the letterspace, the yes exit from blocks 809 and 810, then the instructions represented by block 1000 are executed whereby the first and second binary words will be entered into the first file as appropriate.

If the letterarea is defined and the lightpen is not pointed at the letterspace or letterborder for two consecutive interrupts (the yes, no and yes exits from blocks 809, 810, and 811, respectively), block 812 is executed to determine whether the lightpen is positioned over the letterend. The NO exit from blocks 812 indicates that the lightpen is not positioned over the letterend and must therefore be positioned over a symbol. This condition will cause the instructions represented by block 815 to be executed. The instructions represented by block 815 will interchange the symbol code stored in RETAIN with the letterarea code. That is, the symbol originally replaced by the letterarea by the instructions of block 817 in response to a no decision from block 809 will reappear on the display 500. After the instructions of block 815 have been executed, the instructions represented by block 817 are reexecuted, whereby the code of the symbol the lightpen is currently pointing at is placed in RETAIN and the letterarea code is substituted therefor. Therefore, if a yes decision results from execution of block 810, data will be entered into the first file. If a no decision results from execution of block 810, a letterarea will be displayed in a position different from the then existing letterarea position.

The yes exit from block 812 indicates that the lightpen 600 is positioned over a letterend and causes the instructions represented by block 813 to be executed. If the number of entries in the first file is less than three (indicating that a complete symbol has not been written) the yes exit from block 813, the letterarea is shifted one position by a process of interchanging symbols analogous to that described with respect to blocks 815 and 817. Thus, the symbol originally appearing at the position from which the letterarea was shifted will reappear and the symbol (or lack of symbol) to which the letterarea is shifted will be replaced by letterarea. The code of the latter replaced symbol will then be placed in the storage location RETAIN.

If the number of entries in the first file is larger than or equal to three (indicating that a complete symbol has been written), the no exit from block 813, then the instructions represented by block 900 are executed. The instructions represented by block 900 decode the written symbol into a data processing code, here the USASCII code. The code for the decoded symbol will then be acted upon by the application subprogram 818, the details of which will depend upon the end use of the system. For example, in an editing application, the application subprogram 818 may contain instructions equivalent to the instructions represented by blocks 815 and 817 whereby the decoded symbol would be displayed and the letterarea would appear at the adjacent position for further editing.

The following is intended, by way of example, to facilitate the understanding of and to clarify the use of the operating program associated with the present invention by enumerating the specific steps followed in a typical application of the program. The following exemplary recitation describes one function of a class of functions comprising a text editing application. For purposes of clarity, in the following example a text as displayed on the face of the display 500 will be capitalized and the letterarea identified by the symbol #.

Assume that the text output of an automatic transcription system is displayed on the face of the display 500 in part as follows:

1

. . WHO ELSE HANCE OUT THERE BESIDES . . .

The objective of the operator will thus be to change the word "HANCE" to "HANGS". The operator's initial manipulation will be to point the lightpen 600 to the "C" in the word "HANCE". With respect to FIG. 6, the following will occur:

2

    ______________________________________
    yes, Interrupt
    block 801
    yes, Lightpen interrupt
    block 803
    no, Letterarea defined
    block 809
    Define Letterarea in position
    lightpen aimed at
    block 817
    ______________________________________

    ______________________________________
    yes, Interrupt
    Block 801
    yes, Lightpen interrupt
    Block 803
    yes, Letterarea defined
    Block 809
    yes, Lightpen on letterspace
    Block 810
    Data Input to first file
    Block 1000
    ______________________________________

    ______________________________________
    yes, Interrupt
    block 801
    yes, Letterarea defined
    block 809
    no, Lightpen on previous position
    block 811
    Set previous position
    block 814
    ______________________________________

    ______________________________________
    yes, Interrupt
    block 801
    yes, Letterarea defined
    block 809
    yes, Lightpen on previous
    position
    block 811
    yes, Lightpen on letterarea
    block 812
    no, number of entries smaller
    than three
    block 813
    Decode sumbol
    block 900
    ______________________________________

    ______________________________________
    STATE    SUBSTATE   CL       OPERATION
    ______________________________________
    0        0          6        Halt. If start signal 2105 present,
                                 set to state 1 substate 0. If sentence
                                 output request 315, set to state 2
                                 substate 0. If lightpen 600 output 104
                                 present and register 2400 CR1 not set,
                                 set to state 3 substate 0. If output
                                 104 present, and CR1 is set and 3101
                                 is not present, set to state 4 sub-
                                 state 0. If output 104 present, and
                                 CR1 is set and 3101 is present, set to
                                 state 5 substate 0.
    ______________________________________

    ______________________________________
           SUB-
    STATE  STATE    CL     OPERATION
    ______________________________________
    1      0        2      Reset address counter 2900 by pulse
                           on 2902.
    1      0        6      Set to state 1 substate 1.
    1      1        1      Word request to peripheral 700 by
                           gate on 2108.
    1      1        6      If word on 701 (indicated by signal on
                           2104) set to state 1 substate 2.
    1      2        1      Select address counter 2900 output
                           3001
                           by 2801; select 2510 input by 2501.
    1      2        2      Write page memory 2700 by 2702.
    1      2        4      Up address counter 2900 by 2901.
    1      2        6      If address detect 3002 present, set
                           state to 0 and substate to 0; if
                           address detect 3002 not present, set
                           to state 1 substate 1.
    ______________________________________

    ______________________________________
           SUB-
    STATE  STATE    CL     OPERATION
    ______________________________________
    2      0        1      Select address counter 2900 output by
                           2801.
    2      0        2      Read pulse to register 200 on 2603.
    2      0        4      Up address counter 2900 by 2901.
    2      0        6      If detector 3000 output 3003 true, set
                           to state 0 substate 0.
    ______________________________________

    ______________________________________
           SUB-
    STATE  STATE    CL     OPERATION
    ______________________________________
    3      0        2      Set character position counter 3200
                           by 3203.
    3      0        6      Set to state 3 substate 1.
    3      1        1      Select character position counter 3200
                           output 2803 by 2802.
    3      1        2      Set character register 2600 by 2602.
    3      1        3      Select letterarea code by 2502; select
                           character position output 2803 by 2802.
    3      1        4      Write page memory 2700 by 2702.
    3      1        6      Set to state 0 substate 0; set condi-
                           tion register 2400 CR1.
    ______________________________________

    ______________________________________
           SUB-
    STATE  STATE    CL     OPERATION
    ______________________________________
    4      0        2      Up character position counter 3200
                           by 3201.
    4      0        6      If comparator 3100 detects 2803 equal
                           to 3104 by 3101, set to state 6;
                           if 2803 not equal to 3104, set to
                           state 4 substate 1.
    4      1        2      Down character position counter 3200
                           by 3202.
    4      1        3      Select character position counter 3200
                           output 2803 by 2802; select character
                           register 2600 output 2506 by 2505.
    4      1        4      Write page memory 2700 by 2702.
    4      1        5      Set character position counter 3200
                           by 3203.
    4      1        6      Set to state 3 subtate 1; reset
                           condition register 2400 CR1.
    ______________________________________