Method and apparatus for creating a virtual microscope slide

Abstract

A method and apparatus are disclosed for constructing a virtual microscope slide comprised of digitally scanned images from a microscope specimen. The digitally scanned images are arranged in a tiled format convenient for viewing without a microscope, and for transferring the tiled images for viewing by another at a remote location. Several original microscope views at a low magnification are digitized and stored as digitized images coherently seamed together to provide an overall virtual, macro image of the specimen at a lower resolution. Several original microscope views at higher magnifications are digitized and stored as digitized images coherently seamed together to provide virtual micro images at higher resolution. A data structure is formed with these virtual macro and micro digitized images along with their mapping coordinates. Preferably, a generic viewing program is also provided in the data structure that allows remote users to manipulate and interpret the tiled images on the user's monitor. Also, the data structure is formed with significantly compressed data so as to be transmitted over low bandwidth channels, such as the Internet, without loss of resolution that would interfere with the analysis at a remotely-located pathologist receiving the data structure over the Internet. The preferred interactive program allows the pathologist to scroll and view neighboring image areas of interest. A marker on the macro image indicates to the user the location of the micro image and assists the user in selecting areas from the macro image to be viewed at higher resolution and magnification.

Claims

What is claimed is:

1. A method of constructing and using a self-executing data structure of an image of a specimen on a microscope slide comprising:

digitally scanning and storing multiple magnification and multiple resolution images from the specimen on a microscope slide to create a plurality of individual image tiles having multiple magnifications and multiple resolutions;

providing a dynamic, self-executing program on the data structure for viewing, manipulating and reconstructing the image tiles; and

transferring the scanned, digital image tiles with the dynamic, self-executing program to allow viewing of a digital image of substantially larger image area than the area of the individually acquired tiles and at multiple resolutions.

2. The method of claim 1 further comprising the steps of:

displaying a first image comprising a portion of the specimen as an overall macro view; and

displaying a second image comprising higher resolution view from the specimen on the microscope slide at a higher magnification than the magnification of the overall macro view.

3. The method of claim 2 further comprising:

selecting a point on said overall macro image with a marker; and

producing a corresponding higher magnification image at the location of the marker.

4. A method in accordance with claim 3 wherein the step of selecting a point includes:

moving across tile boundaries to a desired point on a selected macro tile image; and

executing a command to product the micro image from the selected macro tile image point.

5. The method of claim 1 further comprising displaying the X-Y coordinates of said point.

6. The method of claim 1 wherein said image tiles are stored as bit-mapped files.

7. The method of claim 6 further comprising converting said bit-mapped files to JPEG files.

8. The method of claim 7, comprising scanning and storing said images of said specimen at at least three magnifications.

9. A method in accordance with claim 1 including the step of transferring the digital images to a web browser.

10. A method in accordance with claim 9 including the step of transferring the data structure having the digital image files and the dynamic, self-executing program over an Internet/intranet communication channel.

11. A method in accordance with claim 9 including the step of retoggling between lower magnification images and higher magnification images stored on the web browser.

12. A method in accordance with claim 1 including the step of:

scrolling a portion of an image being viewed in a direction to cause the image being viewed to shift to include, in a new image, a portion of the image from a neighboring tiled image that was not previously viewed in the last image viewed by a user.

13. An apparatus for creating a data structure comprising:

a computer-controlled microscope imaging system for digitally scanning images from a specimen on a microscope support at a plurality of image magnifications;

a program for recording the scanned digital images in a series of contiguous image tiles; and

a program for linking the series of contiguous image tiles with a dynamic, self-executing program effective for viewing, manipulating and reconstructing the image tiles.

14. An apparatus is accordance with claim 13 wherein a data compressor compresses the data of the digitally scanned significantly to allow the scanned digitized images to be sent over the Internet.

15. An apparatus in accordance with claim 13 wherein an addressable coordinate system provides addresses so that images can be seamed together and higher magnification images can be easily located with respect to the lower magnification images.

16. An apparatus in accordance with claim 13 wherein a program for scrolling allows the user to scroll a portion of a neighboring image into view.

17. An apparatus in accordance with claim 13 including an address display to display the coordinates to assist multiple viewers to identify the same area for analysis and commentary.

18. An apparatus in accordance with claim 13 further comprising:

a browser for storing the dynamic, self-executing program; and

a monitor for viewing the images stored on the monitor and for flipping back and forth between low resolution macro images and high resolution micro images.

19. An apparatus in accordance with claim 18 including a marker program to mark an addressable area on the macro image and to cause the addressed area to appear at a higher resolution, micro image on the monitor.

20. A program for creating a data structure of an image of a specimen on a microscope slide comprising:

a scanning routine for digitally scanning the image of the specimen on a microscope slide at a plurality of image magnifications;

a recording routine for recording the scanned digital image in a series of contiguous image tiles; and

a linking routine for linking the series of contiguous image tiles with a dynamic, self-executing program effective for viewing, manipulating and reconstructing the image tiles.

21. The program of claim 20 further comprising a first display routine for displaying a micro image comprising a portion of the scanned image at a first magnification and a second display routine for displaying a macro second image comprising an overall view from the specimen on the microscope slide.

22. The program of claim 20 further comprising a routine for selecting a point on said macro image and for producing a corresponding micro image at said point.

23. The program of claim 20 further comprising a coordinate display routine for displaying the coordinates of said point to the user.

Description

BACKGROUND OF THE INVENTION

The invention described in the aforesaid application answers a need, a requirement to image and digitally record an object in a relatively flat plane at high resolution/magnification. Today, it is impractical to construct an optical image sensor large enough to cover the entire image area e.g., of a specimen on a microscope slide, at the required resolution. This is because lens size and resolution/magnification issues limit the size of the field of view of magnified objects and their resulting images. Viewing through a microscope is akin to viewing through a periscope in that one sees a very small field of view even at low magnifications, such as 1.25.times.. A pathologist using a microscope often scans a slide to obtain in his mind an overall view or sense of what constitutes the specimen and he remembers the general locations of the diagnostically significant, small pieces of the specimen. Usually, these are the diseased areas, such as malignant or potentially malignant portions of the specimen. To obtain higher resolution and magnification of these suspicious portions, the pathologist switches to a higher magnification objective lens but then the field of view becomes much smaller again. Often, the pathologist switches back and forth between the lower magnification, larger field of view objective lens to orient himself relative to the specimen and the high magnification, smaller field of view to obtain the detailed, high resolution view of the suspicious area on the specimen. Thus, the user never receives a magnified, condensed overall view of the specimen or a portion of the specimen but must remember the series of views taken at low magnification. Likewise, at high resolution, high magnification, the user never receives or views a collection of adjacent images but must interrelate these successive images in the user's mind.

A similar problem exists on the Internet or intranet where a pathologist may receive a single field of view magnified image taken from a specimen over the Internet or the intranet on his browser. The pathologist must be provided with explanations to coordinate the high resolution view with the lower resolution view. The number of views available to the pathologist is very limited, and the pathologist is unable to select other views or to scroll to neighboring views at the areas that are most interesting to the pathologist.

In the aforesaid prior application, there is disclosed a method and apparatus whereby a person may construct a low magnification, digitized overall, image view of the entire specimen on a slide or a selected portion of the specimen on a slide, such as the basal layer of a tissue section. The overall, low magnification digitized image allows the user to understand where the user is presently located in his viewing and where the user may want to make the next observation. That is, the low magnification overall view is generally in color and provides to the experienced user a visual overall or thumbnail view of the slide and shows the possible areas of interest for malignancy or other diseases which manifest themselves at certain locations on the specimen image being viewed. This low magnification overall view enables the user to select thereon the points of interest that the user wants to view at a higher magnification.

The overall view was constructed by taking by a large number of low magnification images of the specimen through a microscopic scanning system and then coherently assembling and coordinating these respective smaller views or images (hereinafter "image tiles") into one coherent, low magnification, macro image from the specimen. Often the digitized macro image is reduced in size by a software system to even a smaller size to be displayed on a local screen or to be transferred over a low bandwidth or a high bandwidth channel to a remote viewing screen.

The prior application teaches how to assemble a large number of image tiles, for example, 35 image tiles for the macro image, and then to take a series of other tiles of a higher magnification or magnifications which will also be viewed by the user. To this end, the user is provided with a marker, such as a cursor or the like, to select the defined area of interest, and by a simple command, to cause the selected, higher magnification digitized images to appear on the screen for viewing by the user. The higher magnification images may be one of several magnifications or resolutions such as 10.times., 20.times. and 40.times..

As disclosed in the aforesaid application, it is preferred to allow the user, such as a pathologist, to quickly flip back and forth between the high resolution micro image and the low macro resolution image or to provide separate split screens whereby the pathologist is shown an overall macro view and a marker showing where the current higher magnification view is located. Because of the multiple magnifications, the user may change to an intermediate magnification such as would be accomplished by switching between intermediate objective lenses. This provides the pathologist with views which correspond to changing back and forth between objective lenses in a microscope, a procedure with which most pathologists are familiar and have been trained.

Additionally, the aforesaid application provides the user with a scrolling feature that allows the user to shift into the viewing screen adjacent, magnified images on the screen so that the pathologist is not limited to only seeing just a full tile view but may see adjacent image material from adjacent, neighboring tile images.

In the aforesaid patent application, there is a disclosure of transmitting the low magnification image over a local area network or over the Internet through various servers and computers. The tiled images that were being transmitted were achieved by use of a fully computer controlled microscope which allowed the user to navigate along a specimen area of interest, such as along a basal area or to other suspicious points spread throughout the specimen to acquire tiled images of selected areas so that the entire specimen would not have to be digitized and stored. As disclosed in the preferred embodiment in the aforesaid application, an Internet browser remotely-controlled, automated microscope could be used by a pathologist from a remote location to view the reconstructed macro image tiles; and, with his manipulation of the microscope, using an intranet or Internet browser, could acquire single images at higher magnifications if desired. While several people could see the particular digitized images being transmitted out over the Internet as they were acquired by a particular pathologist and several people could view the stored images, there was still a problem of control at operation of the microscope by each person viewing the digitized images, and a problem with acquiring and transmitting large areas of higher magnification images using the tiling method.

As stated above in greater detail, the current state of archiving the digital images achieved through a microscope is often by having photographs or by video tapes. The photographs are difficult to use as is a video tape particularly when the user wants to move rapidly back and forth between various images and to scroll through various adjacent parts of the specimen image. Further, current archival methods lack an overall macro image of the specimen, which allows the user to know exactly where the particular high resolution view is from when it is making an analysis of the high resolution image.

While digitized images can be stored magnetically or otherwise digitized and recorded on various recording mediums, no current archival system allows the user to toggle back and forth between high magnification images and low magnification images or between various images at different magnifications such as that achieved by a pathologist switching microscope objective lenses in real time to get the macro and micro images from the same location on the specimen. Heretofore, the practice of pathology has been relatively limited to the use of microscopes and to the pathologist having to use the microscope to review the particular specimen.

There is a need for a dynamic system whereby one or more or several pathologists, including a consulting pathologist, may view the same area simultaneously and interact with one another either in diagnosis or in analysis. Also, it would be best if the images from the specimen could be stored so that a pathologist could easily examine the images at his leisure using an intranet or Internet browser at a later date merely by accessing the particular web site where the images are located.

It will be appreciated that a host of problems need to be solved to allow Internet or intranet users to view on their respective monitors useful, low resolution, macro images and high resolution, micro images of several adjacent, original microscope images. One of the first problems is how to seam together neighboring tile images to form a seamless overall view of these tiles. Heretofore, attempts to seam the tiles used software to combine the pixels at the tile boundaries and have been generally unsuccessful. Another problem is that of mapping of coordinates beginning with the coordinates, usually X and Y coordinates, from and at the microscope stage carrying the slide and then the mapping of coordinates on the scanning screen not only for one magnification but also to coordinate the mapping for the respective multiple resolution images taken typically at 1.25.times., 10.times. and 40.times. or more. These coordinates must be maintained for a large number of tiled images, e.g., 40 tiled images for one macro image. In order for the remote user to view these tile images and to flip back and forth between different resolution, tiled images, the user's computer and monitor not only must receive the addresses and stored parameters for each pixel but must also run them on a generic viewing program.

Another problem with acquiring image tiles and sending them over a low bandwidth Internet channel is that both the storage requirements on the server and the amount of data acquired per slide become high, such as for example, 120 megabytes to one gigabyte. The 120 megabytes is only achieved by not taking image tiles of the entire specimen but only image tiles from the areas selected by the pathologist when tracing at high resolution along basal layers or only at the dispersed, suspicious cancer appearing area in a breast cancer. Even with this selective interaction by a pathologist in constructing the macro and micro digitized images with a vastly reduced amount of image tiles relative to that which would be acquired if the entire specimen where imaged at each of the multiple magnifications, the acquired amount of data is a monstrous problem of transmitting in a reasonable amount of time over a narrow bandwidth channel to an ordinary web browser having limited storage capacity. While rough compression techniques could be used, they cannot be used at the expense of providing the high resolution image that the pathologist must have for diagnosis of the specimen.

SUMMARY OF THE INVENTION

In accordance with the present invention, there is provided a new and improved method and apparatus for constructing digitally scanned images from a microscope specimen, for storing the digitally scanned images in a tiled format convenient for viewing without a microscope, and for transferring the tiled images at multiple magnifications for viewing by another at a remote location. This is achieved by assembling together several adjacent, original microscope views at a first magnification to obtain an overall macro view of the specimen and assembling together several adjacent original microscope views at a higher magnification to create a combined data structure. The data structure may then be transferred to the remote viewer to provide this viewer multiple resolution macro and micro images of areas on the slide specimen. The data structure is constructed by digitally scanning and storing the low magnification images with their mapping coordinates and likewise, digitally scanning and storing higher magnification images with their mapping coordinates. Further, a pathologist may interactively select only those diagnostically significant areas of the specimen for digital scanning and storing to reduce significantly the number of image pixels stored at high resolution.

The data structure can be transmitted over the Internet or intranet to allow multiple users to consult on a particular microscope each using his own virtual images of the specimen. These users each may flip back and forth between different resolution images in a manner similar to that achieved when shifting among objective lens for different resolution views. However, the preferred embodiment of this invention provides a marker on the overall macro view showing the remote user where the higher resolution image is located on the specimen so that the user does not have to remember the location of the higher resolution image. Unlike the single, small optical field of view currently available, the remote user is provided with a series of abutted, tiled images each being substantially equal to one small optical field of view. Thus, the remote user is provided with better and larger macro and micro tiled images than the single, small optical fields of view taken at the same magnifications of a single tiled image.

The preferred data structure is also provided with a generic viewing program that allows the remote user to manipulate and interpret the tiled images on the user's browser. This generic viewing program is self-contained with its own display and the interpretative program is usable with a variety of computers, browsers and monitors. The data structure uses selectively compressed data to reduce the huge amount of acquired data, e.g. 120 megabytes, into a small amount of data, e.g. 1.4 megabytes. Such smaller, more manageable amounts of data can be transmitted over a low bandwidth channel such as the Internet without the loss of resolution that would interfere with the remote pathologist's analysis. Further, the interactive program allows the pathologist to scroll and to view neighboring image areas of neighboring image tiles which were currently unavailable to the pathologist until the inventions set forth in the aforesaid application and in this application.

Turning now in greater detail to aspects of this invention, problems with achieving tileable (i.e. contiguous images which can be seamlessly abutted next to each other to recreate the original image, but at different magnifications) multiple images of a specimen on a microscope slide are overcome by the system of the invention. The system includes a microscope and stage in which digital locations on the stage have been predetermined in accordance with an electromechanical addressable coordinate system (X-Y for convenience). Each point on the stage is assigned an "X" and a "Y" coordinate which uniquely defines its location. The increments in each of the X and Y directions are established at a predefined amount for example in 0.1 micrometer increments. A key factor in achieving superior resolution of the specimen images at higher magnifications is to establish many more physical increments on the stage for each pixel of the image sensor and of the intended display. For example, at 1.25.times. magnification, 64 points on the stage correspond to one pixel on a CCD optical sensor, which corresponds to one pixel on a 640 by 480 monitor (for a VGA display), using the bitmap addressing and scrollable image method described herein.

Once the coordinate system is defined for the microscope stage, when a specimen on a microscope slide is placed on it, each feature of interest on the slide can be uniquely located with reference to the stage. Then the microscope system is used to digitally scan the image. The first scan is done at a relatively small magnification since this image will be used to provide a "macro" image of the entire specimen. In the preferred embodiment, 1.25.times. magnification is used. The microscope system then scans the slide using the 1.25.times. objective. Since the image is detected by rectangular optical sensors, such as the optical sensors in a CCD grid, the stage must be moved in relatively larger increments to place the next adjacent physical part of the slide exactly in the region where that rectangular area will be precisely imaged on the CCD sensor.

Although the area traveled is relatively large, the precision must be high to enable alignment of the image parts within the pixel resolution of the CCD sensor. For example, at the 1.25.times. magnification, 48,143 X steps and 35,800 Y steps are necessary to move the specimen object on the stage to a new, contiguous region for optical imaging on the CCD sensor. The signal produced by the optical sensors in the CCD grid are then transmitted to a computer which stores the image signals in a series of tiled images. Since each image frame is defined by predetermined X-Y coordinates, these can be easily converted into a series of contiguous tiled images.

To view the scanned digital image on a monitor, the computer uses a method of reserving an image bitmap corresponding to the entire size of the tiled image, e.g., in this instance, 10.times.8, 1.25.times. magnified tiled images are acquired. This requires an image bitmap of 7,520.times.3,840 in size, using a 752.times.480 pixel CCD sensor. Since the X-Y coordinates are known for each image tile, and thus for each pixel in each tile, the bitmap can be used to coordinate and display the stored image tiles to present a fused macro view of the image with one-to-one pixel correspondence of the screen pixels with the image pixels. Typically, the screen pixels are fewer in X-Y size than the macro tiled image, (that is, the entire image cannot be viewed on the monitor without some sort of image compression); and in this case, the macro tiled image is scrolled on the viewable window segment of the screen to maintain the one-to-one correspondence. An advantage of the one-to-one correspondence is that significant image detail is available to the user. Further, since the physical X,Y position on the specimen is known through the stage coordinate relationship to the image pixels, the tiled macro images can be used to locate regions, and move the stage to that region from collection of higher magnification tiled images.

Since the nature of optics, i.e. lenses, is that they provide a generally circular image with a sharp central region and with fuzziness around the periphery of the image, the microscope system is designed to step through the various locations on the slide in such a manner to scan only the high resolution image portion in the center of the optical image. The fuzzy outer regions are discarded. This also has the benefit of ensuring a high resolution image once the tiled images are reconstructed for viewing by a user on a monitor.

After the macro image is completed, a trained professional, such as an examining pathologist, views the image of the specimen by viewing the macro image and looking for areas of interest. In general, most specimen slides contain only a few small areas of diagnostic significance. The balance of the slide is generally empty or not significant. When the examining pathologist views the slide, some areas may have been previously marked in the regions of interest for viewing and analysis at higher magnifications. Once these regions are marked, the microscope is set to the desired higher magnification and then only the marked regions are scanned and stored. Alternatively, he may define new areas directly on the macro image. In either case, the regions are outlined using a pointing device, such as a mouse, directly on the viewing window displaying the macro image. As described above with respect to the 1.25.times. images, since the stage has a predefined coordinate system, the scanned higher magnification image portions can be easily located with respect to the macro image, creating a series of micro images.

The fact that a typical microscope specimen slide contains only limited information of interest and the ability of the system embodying the invention to accurately locate such regions enables the system to create a virtual microscope slide, i.e. a data structure which can be used in place of the actual specimen slides. This advantageously enables multiple users to consult on a particular specimen. Additionally, because of the reduced size of the data structures, they can be viewed locally on a personal computer, transmitted over an intranet or via the Internet globally. The created data structures can be stored on a variety of storage or recording media: for example, on a server's hard disk, a Jazz drive, a CD-Rom or the like. Storing the data structure on a portable storage media further enables the transfer and archiving of the microscopic slide data structures by multiple users.

Another feature of the invention is a self-executing data structure. This is achieved by packaging the tiled images with an active, dynamic control program. When an active dynamic control program is used by a viewing program such as a common web browser, the browser can interpret the dynamic control program. This allows the user to interact and control the viewed images seen on the viewer's screen from the recording medium. More specifically, in the preferred embodiment of the invention, a large number of low magnification, digitized tiled images are formed and embedded in a data structure with linking information allowing them to be coherently tiled to each other during viewing to form a macro image, and a series of higher magnification tiled images also similarly constructed into a micro image, and a control program such as a JAVA applet, is provided and transferred with the macro and micro tiled images for use by a remote user. Thus, for example, the macro and micro tiled images with their active control program may be transmitted over an Internet or an intranet to a browser, or other application program for viewing the images, where the user may then access the browser to analyze the images at multiple resolutions and with a macro field of view before the user. This enables the viewing of the images in a manner similar to the use of an optical microscope, but in this case visually the view is of a virtual microscope slide at multiple resolutions.

Also, in accordance with the invention, the constructed, tiled macro and tiled micro images along with the control program can be placed on a web server and can be accessed locally and over a wide area, even globally, by multiple users at various times. For instance, a large number of previously scanned and recorded specimen slides, such as 300 specimen slides, may have their respective micro and macro tiled images put on a server. Medical students or pathology students then may each access the slide or all of the 300 slides and review them on their respective web browsers at their leisure. Likewise, a pathologist may dial up or otherwise connect through an internet service provider to the Internet or other long-level network and access a web server and obtain a particular patient's specimen results. Those results would have been stored as a data structure (including macro and micro tiled images along with the control and interpretative program). The pathologist then may and perform an analysis at his home or in his office without needing to have or to control a microscope or the particular slide. The pathologist may toggle back and forth between the micro and macro images, and then dictate or otherwise prepare his analysis, findings or diagnosis from these stored images. This advantageously enables the pathologist to perform part of his job in the convenience of his home or office and also enables a laboratory to maintain actual specimen slides in a safe and secure location, away from the potential of damage and without the necessity of shipping the slides for microscopic examination at a remote location.

The control program, which in the preferred embodiment of the invention is a dynamic self-executing program such as a JAVA applet, allows the user to manipulate and interpret the images while on a browser. The dynamic, self-executing program is completely self-contained with its own display and interpretative program for operation by the user of the browser.

The present invention is not limited to use on a browser since the tiled, digitized images and the active, control program may be stored on a CD-ROM or other portable storage medium and sent through the mail, or otherwise transferred to the user for review at the user's convenience with dedicated viewers.

Thus, from the foregoing, it will be seen that there is provided a new and improved method of and apparatus for archiving of microscopic slide information on a storage medium with an active control program, which allows the display and interpretation of various micro and macro images.

In accordance with another important aspect of the invention, there is provided with the self-executing data structure (the stored macro images, micro images and dynamic, self-executing program for viewing, reconstructing and manipulating the stored images) the ability to scroll through the displayed images. This allows the user not only to see one image tile at a particular magnification, but also to use a pointer or to otherwise move a point to cause displayed images from adjacent neighboring image tiles which were not previously viewable to be included in the field being viewed by the user. That is, the user may shift the viewing location across tile boundaries from one tile to another, and up or down, or right or left or to other points of interest in a normal two-dimensional scrolling manner. Thus, the user is provided with an archived stored slide at multiple magnifications which can be readily scrolled through in any arbitrarily chosen direction or directions. As in the aforesaid application, the user interactively will go to various areas of selected interest and operate a pointer or a marker to select for high magnification viewing the particular area of interest and also do a scrolling of neighboring areas of interest.

In addition to the Internet browser, the data images can be viewed, reconstructed and manipulated using a dynamic, self-executing program such as, for example, a JAVA applet or an ACTIVE-X applet. An advantage of using a dynamic, self-executing program which is linked with the data images on a data structure is that the data images can be viewed, reconstructed and manipulated independent of the operating system of the users computer. Additionally, the user does not have to acquire the latest version of the dynamic, self-executing program since it is already linked with and provided with the data images on the data structure or on the storage medium. Thus, the user can always view the data images, regardless of different program versions.

The dynamic, self-executing program permits interchanging the image in its entirety simulating the visual effect of changing objectives in a regular, mechanical optical microscope view. Thus, the user can easily switch from one magnification to another and scroll through portions of the image, simulating tracking the image by moving the slide under the microscope lens.

The dynamic, self-executing program permits scrolling the image in a window to enable viewing of the reconstructed large field of view images. The user can use a mouse, or other pointing device, to select a portion of the image on the large field of view image and the program will display that selected portion in another window at the desired magnification.

A method of constructing a record of the digital image of a specimen on a microscope slide using image tiles includes scanning the image at a first low magnification so that substantially all of the specimen is obtained. Then the specimen is scanned at a second higher magnification so that images of selected (or all) sub-portions of the specimen are obtained. The spatial relationships of the first lower magnification image to the second higher magnification images is used to reconstruct the image during viewing. The individual, sub-portions or tiles of the scanned image are seamed together by the dynamic, self-executing program to create a digital image of substantially larger areas than individually acquired image fields of view without tiling.

A data structure according to the invention is created by first digitally scanning the desired specimen at a plurality of image magnifications. The scanned images are then stored in a series of contiguous image tiles. Then the stored images are linked with a dynamic, self-executing program. The data structure can be created using a software program. Images are preferably first stored as bitmap files (.bmp). (Note that storing the resulting image files in the bitmap format is different from the bit mapping method of creating the image files described herein.) An image compression program is used to convert the bitmap files to a JPEG (.jpg) format, which requires less storage space and consequently less time to display on a computer. The person creating the data structure can select how much detail to include in the conversion. JPEG images can be created for example, using 20 to 80% compression ratios of the original image. An advantage of the JPEG format is that essentially empty tiles (tiles with mostly white or black space) compress down to very small files. Detailed files, however, do not compress as much. Additionally, the dynamic, self-executing program may include compression algorithms for displaying the entire image or portions thereof in the viewing window.

After downloading or installation of a data structure on a storage medium, when the user desires to view the data images, he uses a mouse and "clicks" on the icon for the self-executing data structure. The dynamic, self-executing program displays the image in a window. Typically, the program will display a macro or thumbnail view of the entire specimen image at a lower magnification and a smaller window containing a particular image tile or groups of tiles at a higher magnification. The program enables the user to use the mouse or other pointing device to select a point or outline a region on the thumbnail view. The selected view will then be displayed in the smaller window at the second magnification. The user can move the mouse or pointing device and the image in the smaller window will scroll with the selection on the thumbnail view. In this way, the program simulates movement of a microscope slide under the field of view of the mechanical microscope. However, it should be noted that because of the one-to-one correspondence between the CCD pixels and the screen pixels, not all macro images may be able to be displayed on the monitor. The user may scroll through the macro image or select a compression feature to display the entire macro image in the window.

Another feature of the self-executing data structure is that when the image is displayed on the viewing screen, the user can select an image tile or sub-portion of the image and alternately view that portion of the image at each scanned magnification. For example, if the data was scanned at magnifications of 1.25.times., 20.times. and 40.times., the user can "click" and see the same tile at each of those magnifications alternately.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a system according to the invention for creating and transmitting locally, over an intranet or via the Internet data structures of an image of specimen on a microscope slide;

FIG. 1A is representation of a microscope slide which has been arbitrarily assigned to be scanned into eighty tiled images;

FIG. 1B is a representation of the detected signals of the individual pixel sensors in a CCD optical array after detecting a selected image area to tile and the referenced data files containing the information describing the detected signals;

FIG. 2 is a screen view of a system embodying the present invention showing a low magnification image of a specimen on a microscope slide in one window, a high magnification image of a portion of the low magnification image selected by a region marker and a control window;

FIG. 3 is a view of a display screen of the apparatus embodying the present invention showing the control window a low magnification window having a plurality of high magnification micro image regions delineated therein and a high magnification window including one or more of the micro image regions;

FIG. 4 is a view of a macro image of an actual breast cancer specimen displayed at 1.25.times. as seen on a computer monitor;

FIG. 5 is a view of the grid portion of FIG. 4 outlining a region of interest selected by a pathologist displayed at 40.times. magnification;

FIG. 6 is a block diagram of the steps in the mapping of the scanned image from the optical sensor array to computer bit map in memory to the display on a user's monitor;

FIG. 7A is a file listing such as would be seen under Windows 95 file manager showing the data files included in a data structure for a breast cancer specimen;

FIG. 7B is a file listing of a Java applet for controlling a data structure;

FIG. 8 is file listing such as would be seen under Windows 95 file manager showing the data files included in an alternate data structure for a breast cancer specimen;

FIGS. 9A and 9B are a block diagram of the apparatus embodying the present invention;

FIG. 10 is a block diagram of a portion of the apparatus shown in FIG. 9 showing details of a mechanical arrangement of a microscope;

FIG. 11 is a flow diagram related to operation of the apparatus;

FIG. 12 is a flow diagram of details of one of the steps in FIG. 11;

FIG. 13 is a display screen showing control parameters to be manipulated thereon;

FIG. 14 is a flow chart for a region outlying routine;

FIG. 15 is a flow chart for a scanning and analyzing routine;

FIG. 16 is a schematic showing of the limits of travel of the microscope stage with respect to the image tiles;

FIG. 16A is a perspective view of the microscope stage and stepper motors and encoders providing a closed loop drive for the motors;

FIG. 17 is a block diagram of a networked system allowing multiple workstations to obtain access to the microscope and to manipulate the microscope locally at each workstation;

FIG. 17A is a view of the system described in connection with FIG. 10; and

FIG. 18 is a block diagram of a remote networked system for distributing and accessing diagnostic images and data, i.e. virtual microscope slides, through a hypertext transport protocol based server directly or over a packet network.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

FIG. 1 is a block diagram of a system according to the invention for creating, and transmitting over an intranet or via the Internet a virtual microscope slide, i.e. interrelated data structures and display procedures depicting at multiple resolutions, images of a specimen on a microscope slide. The system includes a microscope with a digital platform for supporting the microscope slide. Digital platform or stage 11 has been specially calibrated to include a large number of increments for locating portions of specimen images with high precision. After calibration and initial registration of stage 11 in the microscope setup, a microscope slide or other substrate with a specimen to be scanned is placed on stage 11.

For exemplary purposes, the creation of virtual microscope slide specimen according to the invention will be described with respect to a breast cancer specimen. The first step in creating a data structure according to the invention is to establish a macro image of the entire specimen (or that portion of the specimen desired to be stored as the macro image). The purpose for creating the macro or large area thumbnail image is to enable the viewer to see the entire specimen at once and to use the entire image to choose those significant portions thereon for viewing at greater magnification. In this example, the user has selected 1.25.times. as the magnification to display the entire breast cancer slide. Once specimen 13a has been placed on stage 11, rotating optical assembly 15 are rotated to select lens 17 which corresponds to the 1.25.times. magnification.

In accordance with the teachings of the prior patent application, the computer controlled microscope is moved to scan the entire image of specimen 13a. The focusing system is programmed to step through increments which detect/select only the high resolution center area of the field of view in order to avoid storing the fuzzy areas at the periphery of the field of view. In this example, the macro image will be stored in a 10 by 8 array, for a total of 80 contiguous image tiles, as shown in FIG. 1A.

A typical microscope slide is about 77 mm by 25 mm, where the usable area, without including the label, is about 57 mm by 25 m. Each of the 80 image segments is about 4.8 mm by 3.5 mm in dimension. This means each of the 80 image segments will be scanned separately and stored as a separate image tile.

The precision of the microscope systems is set up so that each step of the motor has a precision of 0.1 micron (micrometer). In this example, the microscope is set up to move 48,143 steps in the X direction and 35,800 steps in the Y direction at 1.25.times. magnification for each of the 80 image areas. At higher magnifications, the image areas to scan are considerably smaller, so the number of steps is corresponding smaller. For each of the 80 image areas, the microscope lens will detect only the high resolution center area of the field of view.

The optical image of the desired image area is then detected by optical array sensor 19 (preferably a CCD sensor array). In this example, each of the 80 scanned areas is sensed by the entire array, which includes 752 pixels by 480 pixels. The optical array sensor sends electrical signals indicative of the detected image to microscope controlled computer 32. Computer 32 stores the scanned images, including the top left X-Y stage coordinates for each of the 80 individual areas of the microscope slide. Each of the 80 scanned image areas' pixel locations are stored in a bit mapped file (i.e., a file which contains a map of the location of each bit in the area) which corresponds to the layout of the individual images thereon. Thus, all of the pixels from the image tile derived from region A on FIG. 1A, which is the seventh from the left and in the top row, are individually assigned unique locations in the computer memory's bit-mapped file (FIG. 6), and are also stored in the data structure image tile file as shown in FIG. 1B.

Each of the stored data image tiles is a standard image file with extension.bmp, and is of the order of one megabyte, i.e. each of the 752.times.480 pixels is stored as 3 bytes of red, green and blue image data (752.times.480.times.32=1,082,880 bytes). Since the location of each image tile is known according to the bitmap, the complete microscope image can be recreated by painting (displaying) each image tile in accordance with its grid location. It should be noted

To display the resulting image, computer 32 calculates the appropriate portion to be displayed from each image tile depending upon the relative-size of the display screen. Since the stored image data is usually greater than the size of the typical monitor, the viewer must scroll through the image on the window to view it entirely. However, an optional compression algorithm can be used to compress the entire image into the viewing window. The X-Y coordinate information is used by the viewing and manipulation program to reconstruct the image tiles into a complete image of the specimen. The resulting image is larger, and with better resolution than would be achieved if optics technology were able to construct a single lens capable of viewing the entire specimen in one field of view. In this example, each of the 80 image tiles has digital resolution of 752.times.480 pixels, with corresponding optical resolution of approximate 0.2 microns at 40.times. to approximately 6.4 microns at 1.25.times..

After the macro or thumbnail images are digitally scanned and stored with their X-Y coordinate information, the user then examines the macro image or original specimen for significant details. Typically, the user will highlight with a marking pen the areas to be viewed at higher magnification. The user then changes the magnification of optics system 15 to the desired higher magnification, moves the scanning system to bring the selected region into view. Computer 32 then repeats the scanning and image tile creation process for the selected region, but at higher magnification and with a new grid system to locate the scanned selected regions.

In the example, the user has selected region B shown on FIG. 1A to perform a second view at a higher magnification. The user selects, for example, a 40.times. magnification. The computer calculates the number of tiles to cover the selected area at 40.times. magnification and sets up a second grid.

It should be noted that region B crosses over several of the larger tiles in FIG. 1A. Because of the extreme precision of the instrument, 0.1 micron resolution, locating such selected regions with high resolution is readily accomplished. As noted above, the compute calculates the size of the image portion, in this case as an example, X=1500 and Y=1200 stepping increments. Each image portion at the 40.times. resolution is detected by the optical sensor array, 752 by 480 pixels. Each resulting data file is stored in a separate, high magnification mapped area of memory so that the computer can easily recall the location of region B, or any of its 200 individual image tiles, when requested by a user.

Once the user has completed selecting and having the computer controlled microscope system scan and store the digital images in image tiles, computer 32 stores the mapped .bmp files along with their coordinate information and creates slide image data structure 31 in FIG. 1. Slide image data structure includes all of the bitmap image tile files at both magnifications (note that similarly, additional images could be stored at further magnifications, if desired), as well as X-Y coordinate information for the location of the various image tiles.

FIG. 7A is a file listing such as would be seen under a Windows 95 file manager showing the data files included in a data structure for a breast cancer specimen. Included in the file listing are FinalScan.ini and SlideScan.ini as well as sixty bitmap data files. Slidescan.ini is a listing of all the original bitmap (.bmp) files. The bitmap files represent the individual image tiles in the scan at, say, 1.25.times. magnifications. Slidescan.ini is set forth below in Table 1 and describes the X-Y coordinates for each image tile file. When the data structure is viewed by a control program, the program uses the X-Y coordinates to display all the image tiles contiguously.

                             TABLE 1
                          Slidescan.ini
                         [Header]
                         x = 278000
                         y = 142500
                         1XStepSize = 48143
                         1YStepSize = 35800
                         iScannedCount = 37
                         [Ss1]
                         x = 181714
                         y = 142500
                         [Ss2]
                         x = 133571
                         y = 142500
                         [Ss3]
                         x = 37285
                         y = 106700
                         [Ss4]
                         x = 85428
                         y = 106700
                         [Ss5]
                         x = 133571
                         y = 106700
                         [Ss6]
                         x = 181714
                         y = 106700
                         [Ss7]
                         x = 229857
                         y = 106700
                         [Ss8]
                         x = 229857
                         y = 70900
                         [Ss9]
                         x = 181714
                         y = 70900
                         [Ss10]
                         x = 133571
                         y = 70900
                         [Ss11]
                         x = 85428
                         y = 70900
                         [Ss12]
                         x = 37285
                         y = 70900
                         [Ss13]
                         x = -10858
                         y = 70900
                         [Ss14]
                         x = -10858
                         y = 35100
                         [Ss15]
                         x = 37285
                         y = 35100
                         [Ss16]
                         x = 85428
                         y = 35100
                         [Ss17]
                         x = 133571
                         y = 35100
                         [Ss18]
                         x = 181714
                         y = 35100
                         [Ss19]
                         x = 229857
                         y = 35100
                         [Ss20]
                         x = 278000
                         y = -700
                         [Ss21]
                         x = 229857
                         y = -700
                         [Ss22]
                         x = 181714
                         y = -700
                         [Ss23]
                         x = 133571
                         x = -700
                         [Ss24]
                         x = 85428
                         y = -700
                         [Ss25]
                         x = 37285
                         y = -700
                         [Ss26]
                         x = -10858
                         y = -700
                         [Ss27]
                         x = -10858
                         y = -36500
                         [Ss28]
                         x = 37285
                         y = -36500
                         [Ss29]
                         x = 85428
                         y = -36500
                         [Ss30]
                         x = 133571
                         Y = -36500
                         [Ss31]
                         x = 181714
                         y = -36500
                         [Ss32]
                         x = 229857
                         y = -36500
                         [Ss33]
                         x = 278000
                         y = -36500
                         [Ss34]
                         x = 278000
                         y = -72300
                         [Ss35]
                         x = 229857
                         y = -72300
                         [Ss36]
                         x = 181714
                         y = -72300
                         [Ss37]
                         x = 133571
                         y = -72300

                             TABLE 2
                          FinalScan.ini
                    [Header]
                    tPatientID = mda027
                    tAccession =
                    tOperatorID = jwb
                    tTimeOfScan = 8/4/97 1:19:56
                    PM
                    1XStageRef = 278000
                    1YStageRef = 142500
                    iImageWidth = 752
                    iImageHeight = 480
                    1XStepSize = 1590
                    1YStepSize = 1190
                    1XOffset = -1900
                    1YOffset = -400
                    dMagnification = 40
                    1AnalysisImageCount = 105
                    1CalibrationImageCount = 0
                    [Da0]
                    x = 214532
                    y = 65584
                    [Da1]
                    x = 212996
                    y = 65584
                    [Da2]
                    x = 211460
                    y = 65584
                    [Da3]
                    x = 209924
                    y = 65584
                    [Da4]
                    x = 208388
                    y = 65584
                    [Da5]
                    x = 206852
                    y = 65584
                    [Da6]
                    x = 205316
                    y = 65584
                    [Da7]
                    x = 203780
                    y = 65584
                    [Da8]
                    x = 214532
                    y = 64400
                    [Da9]
                    x = 212996
                    y = 64400
                    [Da10]
                    x = 211460
                    y = 64400
                    [Da11]
                    x = 209924
                    y = 64400
                    [Da12]
                    x = 208388
                    y = 64400
                    [Da13]
                    x = 206852
                    y = 64400
                    [Da14]
                    x = 205316
                    y = 64400
                    [Da15]
                    x = 203780
                    y = 64400
                    [Da16]
                    x = 214532
                    y = 63216
                    [Da17]
                    x = 212996
                    y = 63216
                    [Da18]
                    x = 211460
                    y = 63216
                    [Da19]
                    x = 209924
                    y = 63216
                    [Da20]
                    x = 208388
                    y = 63216
                    [Da21]
                    x = 206852
                    y = 63216
                    [Da22]
                    x = 205316
                    y = 63216
                    [Da23]
                    x = 203780
                    y = 63216
                    [Da24]
                    x = 214532
                    y = 62032
                    [Da25]
                    x = 212996
                    y = 62032
                    [Da26]
                    x = 211460
                    y = 62032
                    [Da27]
                    x = 209924
                    y = 62032
                    [Da28]
                    x = 208388
                    y = 62032
                    [Da29]
                    x = 206852
                    y = 62032
                    [Da30]
                    x = 205316
                    y = 62032
                    [Da31]
                    x = 203780
                    y = 62032
                    [Da32]
                    x = 214532
                    y = 60848
                    [Da33]
                    x = 212996
                    y = 60848
                    [Da34]
                    x = 211460
                    y = 60848
                    [Da35]
                    x = 209924
                    y = 60848
                    [Da36]
                    x = 208388
                    y = 60848
                    [Da37]
                    x = 206852
                    y = 60848
                    [Da38]
                    x = 205316
                    y = 60848
                    [Da39]
                    x = 203780
                    y = 60848
                    [Da40]
                    x = 214532
                    y = 59664
                    [Da41]
                    x = 212996
                    y = 59664
                    [Da42]
                    x = 211460
                    y = 59664
                    [Da43]
                    x = 209924
                    y = 59664
                    [Da44]
                    x = 208388
                    y = 59664
                    [Da45]
                    x = 206852
                    y = 59664
                    [Da46]
                    x = 205316
                    y = 59664
                    [Da47]
                    x = 203780
                    y = 59664
                    [Da48]
                    x = 214532
                    y = 58480
                    [Da49]
                    x = 212996
                    y = 58480
                    [Da50]
                    x = 211460
                    y = 58480
                    [Da51]
                    x = 209924
                    y = 58480
                    [Da52]
                    x = 208388
                    y = 58480
                    [Da53]
                    x = 206852
                    y = 58480
                    [Da54]
                    x = 205316
                    y = 58480
                    [Da55]
                    x = 203780
                    y = 58480
                    [Da56]
                    x = 180740
                    y = 82160
                    [Da57]
                    x = 179204
                    y = 82160
                    [Da58]
                    x = 177668
                    y = 82160
                    [Da59]
                    x = 176132
                    y = 82160
                    [Da60]
                    x = 174596
                    y = 82160
                    [Da61]
                    x = 173060
                    y = 82160
                    [Da62]
                    x = 171524
                    y = 82160
                    [Da63]
                    x = 180740
                    y = 80976
                    [Da64]
                    x = 179204
                    y = 80976
                    [Da65]
                    x = 177668
                    y = 80976
                    [Da66]
                    x = 176132
                    y = 80976
                    [Da67]
                    x = 174596
                    y = 80976
                    [Da68]
                    x = 173060
                    y = 80976
                    [Da69]
                    x = 171524
                    y = 80976
                    [Da70]
                    x = 180740
                    y = 79792
                    [Da71]
                    x = 179204
                    y = 79792
                    [Da72]
                    x = 177668
                    y = 79792
                    [Da73]
                    x = 176132
                    y = 79792
                    [Da74]
                    x = 174596
                    y = 79792
                    [Da75]
                    x = 173060
                    y = 79792
                    [Da76]
                    x = 171524
                    y = 79792
                    [Da77]
                    x = 180740
                    y = 78608
                    [Da78]



                    x = 179204
                    y = 78608
                    [Da79]
                    x = 177668
                    y = 78608
                    [Da80]
                    x = 176132
                    y = 78608
                    [Da81]
                    x = 174596
                    y = 78608
                    [Da82]
                    x = 173060
                    y = 78608
                    [Da83]
                    x = 171524
                    y = 78608
                    [Da84]
                    x = 180740
                    y = 77424
                    [Da85]
                    x = 179204
                    y = 77424
                    [Da86]
                    x = 177668
                    y = 77424
                    [Da87]
                    x = 176132
                    y = 77424
                    [Da88]
                    x = 174596
                    y = 77424
                    [Da89]
                    x = 173060
                    y = 77424
                    [Da90]
                    x = 171524
                    y = 77424
                    [Da91]
                    x = 180740
                    y = 76240
                    [Da92]
                    x = 179204
                    y = 76240
                    [Da93]
                    x = 177668
                    y = 76240
                    [Da94]
                    x = 176132
                    y = 76240
                    [Da95]
                    x = 174596
                    y = 76240
                    [Da96]
                    x = 173060
                    y = 76240
                    [Da97]
                    x = 171524
                    y = 76240
                    [Da98]
                    x = 180740
                    y = 75056
                    [Da99]
                    x = 179204
                    y = 75056
                    [Da100]
                    x = 177668
                    y = 75056
                    [Da101]
                    x = 176132
                    y = 75056
                    [Da102]
                    x = 174596
                    y = 75056
                    [Da103]
                    x = 173060
                    y = 75056
                    [Da104]
                    x = 171524
                    y = 75056

                             TABLE 3
                            index.html
    <HTML>
    <TITLE>
    DCIS_027 - Web Slide
    </TITLE>
    <BODY>
    <APPLET CODE=WebSlide/BliWebSlide.class NAME=DCIS_027
    WIDTH=3384 HEIGHT=960 HSPACE=0 VSPACE=0 ALIGN=Middle>
    <PARAM NAME = "tPatientID" VALUE = "mda027">
    <PARAM NAME = "tAccession" VALUE = " ">
    <PARAM NAME = "tOperatorID" VALUE = "jwb">
    <PARAM NAME = "tTimeOfScan" VALUE = "8/4/97 1:19:56 PM">
    <PARAM NAME = "1XStageRef" VALUE = "278000">
    <PARAM NAME = "1YStageRef" VALUE = "142500">
    <PARAM NAME = "iImageWidth" VALUE = "752">
    <PARAM NAME = "iImageHeight" VALUE = "480">
    <PARAM NAME = "1XStepSize" VALUE = "1590">
    <PARAM NAME = "1YStepSize" VALUE = "1190">
    <PARAM NAME = "1XOffset" VALUE = "-1900">
    <PARAM NAME = "1YOffset" VALUE = "-400">
    <PARAM NAME = "dMagnification" VALUE = "40">
    <PARAM NAME = "iImageCount" VALUE = "105">
    <PARAM NAME = "1XSsStepSize" VALUE = "48143">
    <PARAM NAME = "1YSsStepSize" VALUE = "35800">
    <PARAM NAME = "iScannedCount" VALUE = "37">
    <PARAM NAME = "1StartX" VALUE = "278000">
    <PARAM NAME = "1StartY" VALUE = "142500">
    <PARAM NAME = "Ss1_X" VALUE = "181714">
    <PARAM NAME = "Ss1_Y" VALUE = "142500">
    <PARAM NAME = "Ss2_X" VALUE = "133571">
    <PARAM NAME = "Ss2_Y" VALUE = "142500">
    <PARAM NAME = "Ss3_X" VALUE = "37285">
    <PARAM NAME = "Ss3_Y" VALUE = "106700">
    <PARAM NAME = "Ss4_X" VALUE = "85428">
    <PARAM NAME = "Ss4_Y" VALUE = "106700">
    <PARAM NAME = "Ss5_X" VALUE = "133571">
    <PARAM NAME = "Ss5_Y" VALUE = "106700">
    <PARAM NAME = "Ss6_X" VALUE = "181714">
    <PARAM NAME = "Ss6_Y" VALUE = "106700">
    <PARAM NAME = "Ss7_X" VALUE = "229857">
    <PARAM NAME = "Ss7_Y" VALUE = "106700">
    <PARAM NAME = "Ss8_X" VALUE = "229857">
    <PARAM NAME = "Ss8_Y" VALUE = "70900">
    <PARAM NAME = "Ss9_X" VALUE = "181714">
    <PARAM NAME = "Ss9_Y" VALUE = "70900">
    <PARAM NAME = "Ss10_X" VALUE = "133571">
    <PARAM NAME = "Ss10_Y" VALUE = "70900">
    <PARAM NAME = "Ss11_X" VALUE = "85428">
    <PARAM NAME = "Ss11_Y" VALUE = "70900">
    <PARAM NAME = "Ss12_X" VALUE = "37285">
    <PARAM NAME = "Ss12_Y" VALUE = "70900">
    <PARAM NAME = "Ss13_X" VALUE = "-10858">
    <PARAM NAME = "Ss13_Y" VALUE = "70900">
    <PARAM NAME = "Ss14_X" VALUE = "-10858">
    <PARAM NAME = "Ss14_Y" VALUE = "35100">
    <PARAM NAME = "Ss15_X" VALUE = "37285">
    <PARAM NAME = "Ss15_Y" VALUE = "35100">
    <PARAM NAME = "Ss16_X" VALUE = "85428">
    <PARAM NAME = "Ss16_Y" VALUE = "35100">
    <PARAM NAME = "Ss17_X" VALUE = "133571">
    <PARAM NAME = "Ss17_Y" VALUE = "35100">
    <PARAM NAME = "Ss18_X" VALUE = "181714">
    <PARAM NAME = "Ss18_Y" VALUE = "35100">
    <PARAM NAME = "Ss19_X" VALUE = "229857">
    <PARAM NAME = "Ss19_Y" VALUE = "35100">
    <PARAM NAME = "Ss20_X" VALUE = "278000">
    <PARAM NAME = "Ss20_Y" VALUE = "-700">
    <PARAM NAME = "Ss21_X" VALUE = "229857">
    <PARAM NAME = "Ss21_Y" VALUE = "-700">
    <PARAM NAME = "Ss22_X" VALUE = "181714">
    <PARAM NAME = "Ss22_Y" VALUE = "-700">
    <PARAM NAME = "Ss23_X" VALUE = "133571">
    <PARAM NAME = "Ss23_Y" VALUE = "-700">
    <PARAM NAME = "Ss24_X" VALUE = "85428">
    <PARAM NAME = "Ss24_Y" VALUE = "-700">
    <PARAM NAME = "Ss25_X" VALUE = "37285">
    <PARAM NAME = "Ss25_Y" VALUE = "-700">
    <PARAM NAME = "Ss26_X" VALUE = "-10858">
    <PARAM NAME = "Ss26_Y" VALUE = "-700">
    <PARAM NAME = "Ss27_X" VALUE = "-10858">
    <PARAM NAME = "Ss27_Y" VALUE = "-36500">
    <PARAM NAME = "Ss28_X" VALUE = "37285">
    <PARAM NAME = "Ss28_Y" VALUE = "-36500">
    <PARAM NAME = "Ss29_X" VALUE = "85428">
    <PARAM NAME = "Ss29_Y" VALUE = "-36500">
    <PARAM NAME = "Ss30_X" VALUE = "133571">
    <PARAM NAME = "Ss30_Y" VALUE = "-36500">
    <PARAM NAME = "Ss31_X" VALUE = "181714">
    <PARAM NAME = "Ss31_Y" VALUE = "-36500">
    <PARAM NAME = "Ss32_X" VALUE = "229857">
    <PARAM NAME = "Ss32_Y" VALUE = "-36500">
    <PARAM NAME = "Ss33_X" VALUE = "278000">
    <PARAM NAME = "Ss33_Y" VALUE = "-36500">
    <PARAM NAME = "Ss34_X" VALUE = "278000">
    <PARAM NAME = "Ss34_Y" VALUE = "-72300">
    <PARAM NAME = "Ss35_X" VALUE = "229857">
    <PARAM NAME = "Ss35_Y" VALUE = "-72300">
    <PARAM NAME = "Ss36_X" VALUE = "181714">
    <PARAM NAME = "Ss36_Y" VALUE = "-72300">
    <PARAM NAME = "Ss37_X" VALUE = "133571">
    <PARAM NAME = "Ss37_Y" VALUE = "-72300">
    <PARAM NAME = "Da0_X" VALUE = "214532">
    <PARAM NAME = "Da0_Y" VALUE = "65584">
    <PARAM NAME = "Da1_X" VALUE = "212996">
    <PARAM NAME = "Da1_Y" VALUE = "65584">
    <PARAM NAME = "Da2_X" VALUE = "211460">
    <PARAM NAME = "Da2_Y" VALUE = "65584">
    <PARAM NAME = "Da3_X" VALUE = "209924">
    <PARAM NAME = "Da3_Y" VALUE = "65584">
    <PARAM NAME = "Da4_X" VALUE = "208388">
    <PARAM NAME = "Da4_Y" VALUE = "65584">
    <PARAM NAME = "Da5_X" VALUE = "206852">
    <PARAM NAME = "Da5_Y" VALUE = "65584">
    <PARAM NAME = "Da6_X" VALUE = "205316">
    <PARAM NAME = "Da6_Y" VALUE = "65584">
    <PARAM NAME = "Da7_X" VALUE = "203780">
    <PARAM NAME = "Da7_Y" VALUE = "65584">
    <PARAM NAME = "Da8_X" VALUE = "214532">
    <PARAM NAME = "Da8_Y" VALUE = "64400">
    <PARAM NAME = "Da9_X" VALUE = "212996">
    <PARAM NAME = "Da9_Y" VALUE = "64400">
    <PARAM NAME = "Da10_X" VALUE = "211460">
    <PARAM NAME = "Da10_Y" VALUE = "64400">
    <PARAM NAME = "Da11_X" VALUE = "209924">
    <PARAM NAME = "Da11_Y" VALUE = "64400">
    <PARAM NAME = "Da12_X" VALUE = "208388">
    <PARAM NAME = "Da12_Y" VALUE = "64400">
    <PARAM NAME = "Da13_X" VALUE = "206852">
    <PARAM NAME = "Da13_Y" VALUE = "64400">
    <PARAM NAME = "Da14_X" VALUE = "205316">
    <PARAM NAME = "Da14_Y" VALUE = "64400">
    <PARAM NAME = "Da15_X" VALUE = "203780">
    <PARAM NAME = "Da15_Y" VALUE = "64400">
    <PARAM NAME = "Da16_X" VALUE = "214532">
    <PARAM NAME = "Da16_Y" VALUE = "63216">
    <PARAM NAME = "Da17_X" VALUE = "212996">
    <PARAM NAME = "Da17_Y" VALUE = "63216">
    <PARAM NAME = "Da18_X" VALUE = "211460">
    <PARAM NAME = "Da18_Y" VALUE = "63216">
    <PARAM NAME = "Da19_X" VALUE = "209924">
    <PARAM NAME = "Da19_Y" VALUE = "63216">
    <PARAM NAME = "Da20_X" VALUE = "208388">
    <PARAM NAME = "Da20_Y" VALUE = "63216">
    <PARAM NAME = "Da21_X" VALUE = "206852">
    <PARAM NAME = "Da21_Y" VALUE = "63216">
    <PARAM NAME = "Da22_X" VALUE = "205316">
    <PARAM NAME = "Da22_Y" VALUE = "63216">
    <PARAM NAME = "Da23_X" VALUE = "203780">
    <PARAM NAME = "Da23_Y" VALUE = "63216">
    <PARAM NAME = "Da24_X" VALUE = "214532">
    <PARAM NAME = "Da24_Y" VALUE = "62032">
    <PARAM NAME = "Da25_X" VALUE = "212996">
    <PARAM NAME = "Da25_Y" VALUE = "62032">
    <PARAM NAME = "Da26_X" VALUE = "211460">
    <PARAM NAME = "Da26_Y" VALUE = "62032">
    <PARAM NAME = "Da27_X" VALUE = "209924">
    <PARAM NAME = "Da27_Y" VALUE = "62032">
    <PARAM NAME = "Da28_X" VALUE = "208388">
    <PARAM NAME = "Da28_Y" VALUE = "62032">
    <PARAM NAME = "Da29_X" VALUE = "206852">
    <PARAM NAME = "Da29_Y" VALUE = "62032">
    <PARAM NAME = "Da30_X" VALUE = "205316">
    <PARAM NAME = "Da30_Y" VALUE = "62032">
    <PARAM NAME = "Da31_X" VALUE = "203780">
    <PARAM NAME = "Da31_Y" VALUE = "62032">
    <PARAM NAME = "Da32_X" VALUE = "214532">
    <PARAM NAME = "Da32_Y" VALUE = "60848">
    <PARAM NAME = "Da33_X" VALUE = "212996">
    <PARAM NAME = "Da33_Y" VALUE = "60848">
    <PARAM NAME = "Da34_X" VALUE = "211460">
    <PARAM NAME = "Da34_Y" VALUE = "60848">
    <PARAM NAME = "Da35_X" VALUE = "209924">
    <PARAM NAME = "Da35_Y" VALUE = "60848">
    <PARAM NAME = "Da36_X" VALUE = "208388">
    <PARAM NAME = "Da36_Y" VALUE = "60848">
    <PARAM NAME = "Da37_X" VALUE = "206852">
    <PARAM NAME = "Da37_Y" VALUE = "60848">
    <PARAM NAME = "Da38_X" VALUE = "205316">
    <PARAM NAME = "Da38_Y" VALUE = "60848">
    <PARAM NAME = "Da39_X" VALUE = "203780">
    <PARAM NAME = "Da39_Y" VALUE = "60848">
    <PARAM NAME = "Da40_X" VALUE = "214532">
    <PARAM NAME = "Da40_Y" VALUE = "59664">
    <PARAM NAME = "Da41_X" VALUE = "212996">
    <PARAM NAME = "Da41_Y" VALUE = "59664">
    <PARAM NAME = "Da42_X" VALUE = "211460">
    <PARAM NAME = "Da42_Y" VALUE = "59664">
    <PARAM NAME = "Da43_X" VALUE = "209924">
    <PARAM NAME = "Da43_Y" VALUE = "59664">
    <PARAM NAME = "Da44_X" VALUE = "208388">
    <PARAM NAME = "Da44_Y" VALUE = "59664">
    <PARAM NAME = "Da45_X" VALUE = "206852">
    <PARAM NAME = "Da45_Y" VALUE = "59664">
    <PARAM NAME = "Da46_X" VALUE = "205316">
    <PARAM NAME = "Da46_Y" VALUE = "59664">
    <PARAM NAME = "Da47_X" VALUE = "203780">
    <PARAM NAME = "Da47_Y" VALUE = "59664">
    <PARAM NAME = "Da48_X" VALUE = "214532">
    <PARAM NAME = "Da48_Y" VALUE = "58480">
    <PARAM NAME = "Da49_X" VALUE = "212996">
    <PARAM NAME = "Da49_Y" VALUE = "58480">
    <PARAM NAME = "Da50_X" VALUE = "211460">
    <PARAM NAME = "Da50_Y" VALUE = "58480">
    <PARAM NAME = "Da51_X" VALUE = "209924">
    <PARAM NAME = "Da51_Y" VALUE = "58480">
    <PARAM NAME = "Da52_X" VALUE = "208388">
    <PARAM NAME = "Da52_Y" VALUE = "58480">
    <PARAM NAME = "Da53_X" VALUE = "206852">
    <PARAM NAME = "Da53_Y" VALUE = "58480">
    <PARAM NAME = "Da54_X" VALUE = "205316">
    <PARAM NAME = "Da54_Y" VALUE = "58480">
    <PARAM NAME = "Da55_X" VALUE = "203780">
    <PARAM NAME = "Da55_Y" VALUE = "58480">
    <PARAM NAME = "Da56_X" VALUE = "180740">
    <PARAM NAME = "Da56_Y" VALUE = "82160">
    <PARAM NAME = "Da57_X" VALUE = "179204">
    <PARAM NAME = "Da57_Y" VALUE = "82160">
    <PARAM NAME = "Da58_X" VALUE = "177668">
    <PARAM NAME = "Da58_Y" VALUE = "82160">
    <PARAM NAME = "Da59_X" VALUE = "176132">
    <PARAM NAME = "Da59_Y" VALUE = "82160">
    <PARAM NAME = "Da60_X" VALUE = "174596">
    <PARAM NAME = "Da60_Y" VALUE = "82160">
    <PARAM NAME = "Da61_X" VALUE = "173060">
    <PARAM NAME = "Da61_Y" VALUE = "82160">
    <PARAM NAME = "Da62_X" VALUE = "171524">
    <PARAM NAME = "Da62_Y" VALUE = "82160">
    <PARAM NAME = "Da63_X" VALUE = "180740">
    <PARAM NAME = "Da63_Y" VALUE = "80976">
    <PARAM NAME = "Da64_X" VALUE = "179204">
    <PARAM NAME = "Da64_Y" VALUE = "80976">
    <PARAM NAME = "Da65_X" VALUE = "177668">
    <PARAM NAME = "Da65_Y" VALUE = "80976">
    <PARAM NAME = "Da66_X" VALUE = "176132">
    <PARAM NAME = "Da66_Y" VALUE = "80976">
    <PARAM NAME = "Da67_X" VALUE = "174596">
    <PARAM NAME = "Da67_Y" VALUE = "80976">
    <PARAM NAME = "Da68_X" VALUE = "173060">
    <PARAM NAME = "Da68_Y" VALUE = "80976">
    <PARAM NAME = "Da69_X" VALUE = "171524">
    <PARAM NAME = "Da69_Y" VALUE = "80976">
    <PARAM NAME = "Da70_X" VALUE = "180740">
    <PARAM NAME = "Da70_Y" VALUE = "79792">
    <PARAM NAME = "Da71_X" VALUE = "179204">
    <PARAM NAME = "Da71_Y" VALUE = "79792">
    <PARAM NAME = "Da72_X" VALUE = "177668">
    <PARAM NAME = "Da72_Y" VALUE = "79792">
    <PARAM NAME = "Da73_X" VALUE = "176132">
    <PARAM NAME = "Da73_Y" VALUE = "79792">
    <PARAM NAME = "Da74_X" VALUE = "174596">
    <PARAM NAME = "Da74_Y" VALUE = "79792">
    <PARAM NAME = "Da75_X" VALUE = "173060">
    <PARAM NAME = "Da75_Y" VALUE = "79792">



    <PARAM NAME = "Da76_X" VALUE = "171524">
    <PARAM NAME = "Da76_Y" VALUE = "79792">
    <PARAM NAME = "Da77_X" VALUE = "180740">
    <PARAM NAME = "Da77_Y" VALUE = "78608">
    <PARAM NAME = "Da78_X" VALUE = "179204">
    <PARAM NAME = "Da78_Y" VALUE = "78608">
    <PARAM NAME = "Da79_X" VALUE = "177668">
    <PARAM NAME = "Da79_Y" VALUE = "78608">
    <PARAM NAME = "Da80_X" VALUE = "176132">
    <PARAM NAME = "Da80_Y" VALUE = "78608">
    <PARAM NAME = "Da81_X" VALUE = "174596">
    <PARAM NAME = "Da81_Y" VALUE = "78608">
    <PARAM NAME = "Da82_X" VALUE = "173060">
    <PARAM NAME = "Da82_Y" VALUE = "78608">
    <PARAM NAME = "Da83_X" VALUE = "171524">
    <PARAM NAME = "Da83_Y" VALUE = "78608">
    <PARAM NAME = "Da84_X" VALUE = "180740">
    <PARAM NAME = "Da84_Y" VALUE = "77424">
    <PARAM NAME = "Da85_X" VALUE = "179204">
    <PARAM NAME = "Da85_Y" VALUE = "77424">
    <PARAM NAME = "Da86_X" VALUE = "177668">
    <PARAM NAME = "Da86_Y" VALUE = "77424">
    <PARAM NAME = "Da87_X" VALUE = "176132">
    <PARAM NAME = "Da87_Y" VALUE = "77424">
    <PARAM NAME = "Da88_X" VALUE = "174596">
    <PARAM NAME = "Da88_Y" VALUE = "77424">
    <PARAM NAME = "Da89_X" VALUE = "173060">
    <PARAM NAME = "Da89_Y" VALUE = "77424">
    <PARAM NAME = "Da90_X" VALUE = "171524">
    <PARAM NAME = "Da90_Y" VALUE = "77424">
    <PARAM NAME = "Da91_X" VALUE = "180740">
    <PARAM NAME = "Da91_Y" VALUE = "76240">
    <PARAM NAME = "Da92_X" VALUE = "179204">
    <PARAM NAME = "Da92_Y" VALUE = "76240">
    <PARAM NAME = "Da93_X" VALUE = "177668">
    <PARAM NAME = "Da93_Y" VALUE = "76240">
    <PARAM NAME = "Da94_X" VALUE = "176132">
    <PARAM NAME = "Da94_Y" VALUE = "76240">
    <PARAM NAME = "Da95_X" VALUE = "174596">
    <PARAM NAME = "Da95_Y" VALUE = "76240">
    <PARAM NAME = "Da96_X" VALUE = "173060">
    <PARAM NAME = "Da96_Y" VALUE = "76240">
    <PARAM NAME = "Da97_X" VALUE = "171524">
    <PARAM NAME = "Da97_Y" VALUE = "76240">
    <PARAM NAME = "Da98_X" VALUE = "180740">
    <PARAM NAME = "Da98_Y" VALUE = "75056">
    <PARAM NAME = "Da99_X" VALUE = "179204">
    <PARAM NAME = "Da99_Y" VALUE = "75056">
    <PARAM NAME = "Da100_X" VALUE = "177668">
    <PARAM NAME = "Da100_Y" VALUE = "75056">
    <PARAM NAME = "Da101_X" VALUE = "176132">
    <PARAM NAME = "Da101_Y" VALUE = "75056">
    <PARAM NAME = "Da102_X" VALUE = "174596">
    <PARAM NAME = "Da102_Y" VALUE = "75056">
    <PARAM NAME = "Da103_X" VALUE = "173060">
    <PARAM NAME = "Da103_Y" VALUE = "75056">
    <PARAM NAME = "Da104_X" VALUE = "171524">
    <PARAM NAME = "Da104_Y" VALUE = "75056">
    </APPLET>
    </BODY>
    </HTML>

• Inventors
  Bacus, James W.; Bacus, James V.;
• Assignee
  Bacus Laboratories, Inc. (Lombard, IL)
• Published
  Aug-10-2004
• Current US Classes:
  382/133
  382/134
  382/172
  382/305
  707/102
  707/104.1
• Application #
  365548
• International Classes
  G06K 009/00
• Field of Search
  382/133 382/128 382/154 382/172 382/257 382/283 382/284 382/134 382/305 600/407 707/102 707/104.1
• Examiner
  Rones; Charles
• Agent
  Fitch, Even, Tabin & Flannery
• US Patent References:
  3999047
  4150360
  4175860
  4199748
  4213036
  4523278
  4742558
  4760385
  4777525
  4887892
  4965725
  5018209
  5068906
  5072382
  5073857
  5099521
  5107422
  5123056
  5163095
  5216500
  5216596
  5218645
  5252487
  5257182
  5260871
  5268966
  5287272
  5297034
  5313532
  5333207
  5428690
  5473706
  5499097
  5505946
  5544650
  5625765
  5655029
  5687251
  5784162
  5793969
  5796861
  5836877
  5838837
  5978804
  5993001
  6078681
  6091842
  6091930
  6148096
  6151405
  6438200
  6469797
  6606413