System and method for storing and displaying ultrasound images5920317Abstract A system for recording and displaying ultrasound images. The system includes a database server, an ultrasound image capture station and an ultrasound image review station. The ultrasound images are captured from an analog video signal stream output by an ultrasound machine and converted into compressed digital data preferably in MPEG format. The compressed digital data is archived to a compact disc recordable medium for permanent storage. The ultrasound images may be annotated by a sonographer as they are captured. The sonographer selects an annotation icon from a predefined list of icons to highlight points of interest in an ultrasound scan. The system also includes a demographic data module for tracking captured ultrasound scans, and a clinical data module for reporting ultrasound findings and diagnoses. The advantages include reduced storage requirements because of the elimination of video tape; more accurate tracking of ultrasound scans; easier review due to perfect still frames, random access, annotated highlights and zoom capability; faster more accurate reporting of findings and diagnosis; and, the ability to share ultrasound images over local and wide area networks. Claims I claim: Description TECHNICAL FIELD
TABLE I
______________________________________
FIELD DESCRIPTION
______________________________________
Admission ID
A pointer (long integer) to the patient admission to
which the scan is related.
Scan No. The file name of the MPEG data file to which the
annotation file is attached.
Annotation Type
A code indicating the annotation type.
Location The MPEG video data frame number at which the
annotation was made.
Scan Data The calendar date on which the scan was made.
Sonographer ID
A pointer (long integer) to the sonographer (or other)
who created the annotation.
______________________________________
Preferably, the annotation file is assigned the same name as the MPEG data file but is assigned a different file extension, though another file naming convention can be used for locating and retrieving the annotation file. As may been seen in Table I, the file includes an annotation type which identifies an icon to be placed on a time line when the scanned images are reviewed. Annotations are used as pointers to direct physicians to points of interest or note in the recorded ultrasound image sequence. The function and use of annotations is described below. After the annotation file is created in step 72, the MPEG encoder is queued in step 74 and the ultrasound image capture module checks to see if a user command is received in step 76. If no command is received, the ultrasound image capture module executes a wait state in step 78 until a user command is received. In step 78, the ultrasound image capture module determines whether a user command received is a stop command. If so, the ultrasound image capture module stops the MPEG encoder in step 80, unloads the MPEG encoder in step 82, closes the MPEG data file in step 84, closes the annotation file in step 86, writes a record to an unviewed scan file in step 88 and ends the program. The unviewed scan file written to in step 88 is a record of all scans which have been recorded but not reviewed and is used in the ultrasound image review module to remind physicians of completed scans which have not been diagnosed, as will be explained below in more detail with reference to the description of the ultrasound image review module. If it is determined in step 78 that the command received is not a stop command, the ultrasound image capture module determines in step 90 whether the command received is a pause or resume MPEG encoder command. If a pause or resume MPEG encoder command is received, the ultrasound image capture module responds by pausing or resuming the MPEG encoder in step 92 and returns to step 76 to await a subsequent user command. If in step 90 it is determined that a pause or resume MPEG encoder command was not received, the ultrasound image capture module determines in step 94 whether an annotate command was received. If an annotate command was not received, the ultrasound image capture module returns to step 76 to await a user command. If an annotate command is received in step 94, the ultrasound image capture module queries the MPEG encoder for the current frame number in step 96 and stores an annotation record in the annotation file created in step 72. In the preferred embodiment of the invention, eight annotation types are accommodated. Each annotation type is indicated by a unique icon which is placed on a display time line when the ultrasound images are reviewed. The annotations are used by the sonographer to indicate points of interest or note in the ultrasound image recorded during the scan. Table II shows the eight annotation types, the icons associated with each annotation type and the indication which the annotation conveys to a skilled person reviewing the ultrasound images.
TABLE II
______________________________________
ANNOTATION TYPE
ICON INDICATION
______________________________________
Start Cine vertical bar
highlights start of a sequence of
interest
End Cine vertical bar
highlights end of a sequence of
interest
Measure ruler measurement
Voice microphone
voice recording
Flag flag indicates a frame of particular
interest
Unknown ? indicates an unfamiliar motion or
structure
View eye indicates a change of view
Summary sheet of text
indicates location of scan summary
______________________________________
The annotation types include "Start Cine" which is used to highlight a short motion sequence in the ultrasound image stream. Normally, a "cine loop", as it is known in the art, is used by the sonographer to highlight a cycle of interest or movement, for instance, the operation of a valve in a scanned human heart, or the like. In practice, if a sonographer observes a scanned image sequence of particular interest while moving an ultrasound transducer over a patient's body in a particular way, the sonographer will select the "start cine" button to create a start cine annotation record and repeat the movement of the transducer to capture the sequence for the physician who will review the recorded scan. When the sequence of interest has been recorded, the sonographer selects an "End Cine" annotation to highlight the end of a sequence of interest. A third annotation type is the "Measure" annotation which is indicated by a ruler icon when the sonographer takes a measurement during an ultrasound scan. Ultrasound machines are enabled to perform a plurality of measurements including a distance between two points, the slope of a doppler curve, the area of a two-dimensional plane defined by a boundary, or the like. Such measurements are made for diagnostic purposes. When a measurement is made, the sonographer may indicate the position of the measurement in the image sequence by selecting the Measure annotation. A fourth annotation type is the "Voice" annotation which is indicated by a microphone icon. This annotation type is selected by the sonographer when the sonographer records a voice comment while performing an ultrasound scan. Most ultrasound machines are enabled for voice recordal, though in practice the voice recordal function is seldom used in most institutions. A fifth annotation type is the "Flag" which is represented by a flag icon and used to indicate one or more frames of particular interest. The flag annotation permits the sonographer to flag any frame(s) in the video which may show interesting or abnormal structure. A sixth annotation type is the "Unknown" annotation type represented by a question mark icon. The Unknown annotation type is used to indicate any motion or structure which is unfamiliar to the sonographer. A seventh annotation type is the "View" annotation type which is represented by an eye icon and is used to indicate a change of view. Normally, any ultrasound scan is performed in a specific sequence of so-called "views". In each view the transducer is moved over a specific part of the body. Diagnosis of the scan is facilitated if the physician has an indication of when the "view" is changed. The View icon provides a visual indication of that change. By noting the point in the scan at which the view changes, the physician can deduce by convention which view is being observed. The last type of annotation is the "Summary" annotation which is indicated by a sheet of text icon. The summary annotation is used to indicate the location of the scan summary in the recorded ultrasound images. Ultrasound machines store all measurements made during a scan and those measurements can be displayed together on a summary screen which is at or near the end of a scan. The Summary icon is used to pinpoint the location of the Summary screen so that a physician can readily locate it. The use of annotation icons, which is more fully described below in the description of the ultrasound image review module, greatly facilitates the review of ultrasound scans by physicians. When combined with the capability for random access provided by digital image data, the improvement over prior analog storage and review methods is dramatic. Compact Disc Archival Module This module is designed to transfer the recorded digital video data from the hard disk of the capture station 22 to a Compact Disc Recordable (CDR). The annotation file is also copied to the CDR. The recorded compact disc therefore provides a complete record of the scan as will be further described below in relation to the ultrasound image review module. FIG. 4 shows a flow diagram of the principal functions of the compact disc archival module. The programs begin by loading a CD Recorder driver in a step 96. The compact disc archival module then requests the user to enter a storage type in step 98. In accordance with the preferred embodiment of the invention, a scan may be saved on a compact disc for each patient, or on any available compact disc. It is preferred that scans be stored on a compact disc per patient basis since this facilitates tracking and does not significantly increase cost. In step 100, the compact disc archival module determines whether the storage type is to be on a compact disc per patient basis. If so, the patient's CDR is requested in step 102 and if not, a CDR is requested in step 104. In either case, the compact disc archival module determines in step 106 whether a CDR has been inserted and if not, waits in step 108 for the CDR to be inserted. After the CDR is inserted, the compact disc archival module examines the available space on the CDR in step 110 to determine whether it is adequate space to store the MPEG video file and the annotation file. If the space is not adequate, the compact disc archival module displays a CDR full message in step 112 and returns to step 100. Otherwise, this compact disc archival module determines in step 114 whether the CDR is labelled. If there is no label on the CDR, the CDR is labelled in step 116 and the compact disc archival module begins writing to the CDR in step 124. If the CDR is labelled, the compact disc archival module checks the CDR per patient flag in step 118. If that flag is set to yes, the compact disc archival module determines whether the label matches the patient ID in step 120. If the label does not match the patient ID, the CDR is ejected in step 122 and the system returns to step 108 to wait for a CDR to be inserted. If the CDR label matches the patient ID in step 118, compact disc archival module begins writing to the CDR in step 124. In step 126, the compact disc archival module determines whether the write is successful. If the write is unsuccessful, an error message is displayed in step 128 and the program ends. If the write is successful, the MPEG video data file and the annotation file written to the CDR are verified in step 129. If the files do not verify correctly in step 130, an error message is displayed in step 128 and the program ends. If the files do verify correctly, both the MPEG data file and the annotation file are deleted from the capture station 22 hard disk in step 132 and a new CDR flag is checked in step 134 to determine whether the CDR was new or had been used before, as indicated by the label writing process in step 116. If the CDR is new, the CDR is ejected and a message is displayed in step 136 to request the user to record an identification number on the CDR. The program then ends and all allocated resources are released. Ultrasound Image Review Module The ultrasound image review module handles the retrieval and display of captured MPEG and annotation files. An MPEG decoding card and MCI command set is used to decode and display the MPEG data on a computer display monitor (or a television monitor, or the like). The MCI commands permit the application to respond to requests from a user to pause, resume, seek to any frame in the video sequence and to set the frame display rate of the ultrasound images. Since digital video, such as MPEG video enables random access to video, a simple way of permitting a user to select a particular point in the ultrasound scan is required. This requirement is fulfilled by constructing a time line which is displayed as a horizontal bar where each horizontal pixel represents one second of video play time at 30 frames per second, the default display rate. For example, an ultrasound scan of 4 minutes and 20 seconds would be represented by a time line measuring 260 pixels in length. When a user clicks on the time line using a mouse, the ultrasound image review module determines the pixel at which the mouse click occurred and seeks a corresponding point in the MPEG data file. For example, if the user clicks on the 64th pixel, the MPEG data is retrieved from the 64th display second in the data file. FIG. 5 shows a preferred configuration for an ultrasound image review screen in accordance with the invention. The screen includes a display area 138 where ultrasound images are displayed. The ultrasound images are displayed at a user definable display rate. The default display rate is the international movie standard of 30 frames per second. The display rate is displayed adjacent a top right hand corner of the ultrasound image display area 138 in a frame rate display area 140. The frame display rate may be adjusted by dragging a vertical display rate indicator 142 along a horizontal display rate bar 144 in order to change the display rate from a minimum of one frame per second to a maximum of 64 frames per second. Spaced below the display rate bar 144 is a display time line 146 which is preferably about 180 pixels in length so that it displays the current three minutes of the ultrasound scan being displayed. A vertical display time indicator 148 shows the position of the current display in the displayed sequence. A scroll bar 150 shows the current position relative to the total length of the ultrasound scan. As in any Windows.TM. application, the scroll button 152 can be dragged to any point in the sequence to move the display to that point. Likewise, the display time indicator 148 can be dragged to a new point on the display time line 146 to move the display to that point in the ultrasound images, or the user may click on the time line to move the display to the point indicated by the mouse pointer. As the ultrasound images are displayed beyond the three minute mark, the numerals 0, 1, 2 and 3 displayed below the display time line 146 scroll to the left and are replaced by the numerals 4, 5, 6, etc. Spaced below the display time line 146 are preferably five annotation time lines 154. As will be explained below in more detail, when an ultrasound scan is loaded for display, the annotation time lines are populated with annotations entered by the sonographer when the ultrasound scan was performed. A plurality of annotation time lines are used in order to provide adequate space for the annotations and to eliminate confusion if several annotations are closely grouped together. The uppermost annotation time line 154 is preferably used for flags (see Table II), the second is preferably used for start and end cine symbols, the third is preferably used for unknown type annotations and voice annotations, the fourth is preferably used for measure annotations and the fifth is used for new view and summary annotations. A person reviewing an ultrasound scan may move to any annotated frame in the scan by clicking on the appropriate annotation icon with the mouse, as will also be explained below. Positioned under the display area 138 are a plurality of clinical data buttons generally indicated by the reference 156. The clinical data buttons are related to the clinical data module which is explained below in detail. Located above the frame rate control bar 144 is a toolbar which includes five buttons. The functions of the five toolbar buttons will be explained below in more detail. FIG. 6 is a flow diagram of the principal operations of the ultrasound image review module. When the program is begun, the software application initializes the MPEG decoder card in a step 158. The application then opens the MPEG data file in step 160. If the file opening is determined not to have been successful in step 162, an error message is displayed in step 164 and the program is terminated. If the file opening is determined to be successful in step 162, the MPEG card is used to retrieve the length of the ultrasound image recording in step 166 and the application constructs the display time line 146 (see FIG. 5) in step 168. The application then opens the annotation file in step 170. If the annotation file opening is determined not to be successful in step 172, an error message is displayed in step 164 and the program is ended. If in step 172 the file opening is determined to be successful, the application constructs the annotation time lines 154 (see FIG. 5) and populates the annotation time lines in step 173 with the annotations indicated by the annotation records in the annotation file opened in step 170. The application then waits for a user display request in step 174 if a request is received in step 176, the application determines whether the request equals stop displaying ultrasound images in step 178. If the request is not equal to stop, the selected user option is performed in step 180 and the program is returned to step 174. If the request is a stop display request, the MPEG data file is closed in step 182, the annotation file is closed in step 184, allocated resources are released in step 186 and the display program ends. Table III shows a list of user options which can be selected as display requests in step 174 and performed in step 180. Column 1 shows the user request, column 2 shows the source of the request, column 3 shows the system response.
TABLE III
______________________________________
User Request
Source of Request
System Response
______________________________________
Start Click on start button
Begin an ultrasound
scan review session
Pause/Resume
Click on pause/resume
Pause/resume playback
icon on toolbar
Maximize display
Click on maximize
Maximizes display
window icon on tool bar
area
Set frame rate
Click on frame rate
Increase/decrease
control bar frame display rate
per request
Seek to position
Click on time line;
Compute new position
click on time line
from time line or
scroll bar; click on an
annotation file and
annotation shown on
send start display
the annotation time
command to decoder
lines
Copy current frame to
Click on camera icon
Create new still image
still window
on toolbar object from current
video frame
Replay still frames in
Click on film icon on
Replay still image
sequence toolbar objects in order
created
Zoom/unzoom window
Click on zoom icon on
Command decoder to
toolbar zoom/unzoom display
area
______________________________________
Referring once again to FIG. 5 at the top of the display control portion of the display screen is a tool bar which contains five icons which are also used to control the display sequence. The user can request action from any of the icons as shown in Table III. The pause/resume icon 188 is indicated by a right arrow which permits a user to pause or resume play of ultrasound images at any point in a review. A maximize display icon 190 is used to maximize the size of the image display area 138 for a larger view of ultrasound images. A capture still icon 192, which looks like a camera, is used to capture a still view of the current frame being displayed in the display area 138. The captured still is stored as a copy of the current contents of the display area of the screen image memory. Any number of still images may be captured during an ultrasound image review session. The captured still images are saved for the duration of the session but are not saved after the session is ended. A zoom/unzoom icon 194, which looks like a magnifying glass, is used to zoom in and zoom out the current image displayed in the image display area 138. A replay stills icon 196, which has the appearance of a strip of film, is used to replay all of the still images captured to that time in the order of their capture. This tool bar enables physicians to manipulate ultrasound images to their advantage in diagnosing an ultrasound scan. In addition, as noted above, a physician may seek to any position in the ultrasound scan system by clicking on the display time line 146 or on any annotation icon displayed on any of the annotation time lines 154. This permits a physician reviewing the ultrasound session to move quickly in any direction to a specific frame or sequence of frames in the ultrasound scan. This eliminates the time consuming and frustrating task of fast forwards and rewinds of analog video tape and greatly facilitates review and diagnosis. Clinical Data Module In order to further facilitate the analysis and the reporting of ultrasound studies, a clinical data module was designed to permit physicians to enter a wide variety of clinical findings using a capture station 22 or a review station 26 (see FIG. 1). A "point and click" reporting system based on a tree-like structure is used to provide a simple way for a physician to navigate to the diagnosis they need. FIG. 7 is a diagram of the tree structure developed for this application. Situated at the top of the tree are buttons which are related to branches. The branches are related to findings, which may be related to descriptors. The tree contents are user definable and adapted to support any clinical discipline that requires ultrasound scans. The clinical data buttons 156 (see FIG. 5) are normally defined as anatomical structures, such as atrium, pericardium, vessels, valves and ventricles, for example. While the buttons shown in FIG. 5 relate to cardiology, as noted above, the tree contents are user specifiable and any clinical discipline may be incorporated into the tree structure including obstetrics, pediatrics, etc. Each button is followed by any number of branches until the required finding is located. A final branch in the tree provides a descriptor which may be used to automatically generate a report of the findings and diagnosis of an ultrasound scan. Shown below is a sample tree transversal for the atrium of a human heart: Button Branch Findings Descriptor
______________________________________
Button Branch Findings Descriptor
______________________________________
Atrium Right Calcified Mild
Left Clot Moderate
Enlarged Severe
Regurgitation
______________________________________
This sequence generates the finding, moderate left atrial enlargement. This tree-like structure is created by the interaction of two database tables, a Sequence table and an Elements table. The Sequence table provides a means of traversing the tree while the Elements table stores the description of a finding. The Sequence table includes the following fields: NodeID, GroupID, DataID and NextGroup. The Elements table includes the following fields: DataID, Label and Text. The two tables are transparent to a user, such as a physician using the tree structure to report a scan diagnosis. The physician uses a computer mouse to point to and select a button relating to the anatomical structure which is a subject of his report. When the physician selects a finding by clicking on the following branches, findings and descriptors, the finding is stored in a database table called the EntryData table. Only a pointer to the selected finding is saved, not the text of the finding. By saving a pointer to the finding, the amount of storage space required in the database is minimized. The EntryData table includes the following fields: EntryID, AdmissionID, DataID, Ultrasound Image Frame and Value. The frame field allows findings to be associated to a particular frame in an ultrasound video. The value field allows the finding to be associated with a measurement, such as a doppler slope, a heart wall thickness, or the like, for example. The purpose of the clinical data module is to permit physicians to enter a wide variety of clinical findings in a simple and intuitive manner. This is accomplished by dividing the diagnostic process into the tree-like structure described above. For example, the major chamber in the human heart is the atrium. The atrium is divided into left and right chambers called the left atrium and the right atrium. Within the left atrium, a physician may make several findings, one of which may be an enlarged left atrium. This finding, the enlarged left atrium may be further qualified by recording the amount of enlargement, such as a "severely enlarged left atrium". Each of the major anatomical structures associated with each ultrasound discipline are displayed as buttons. By clicking on a button, a list of qualifiers or subcategories is displayed. For example, clicking on the atrium button will display a list containing left and right. These branches permit more precise specification of a button. As noted above, the atrium is divided into left and right chambers. Each chamber has certain attributes which may be stored in further subcategories etc. Shown below are several typical entries from a Sequence table created for the human heart:
______________________________________
GroupID DataID NextGroup What is displayed
______________________________________
205 1200 310 Atrium
310 1877 387 right
310 1878 388 left
388 1950 0 normal
388 1952 410 enlarged
388 1953 0 calcific
388 1951 0 thickened
410 2220 0 mild
410 2221 0 moderate
410 2222 0 severe
______________________________________
The DataID field in the Sequence table links the Sequence and the Element tables. The NextGroup field is used to determine the elements to be displayed in the next branch of the tree. For example, if a physician points to and clicks on the "atrium" button (see FIG. 5), the branches "right" and "left" in NextGroup 310 (see Sequence table above) are displayed to the right of the buttons 156 shown in FIG. 5. If the physician then clicks on "left" the NextGroup 388 element of "normal", "enlarged" "calcific" and "thickened" are displayed to the right of the "right" and "left" branches. If "enlarged" is selected from group 388, the NextGroup 410 is displayed to the right of group 388. The elements of group 410 include "mild", "moderate" and "severe". Group 410 has no NextGroup, so no further group is displayed when "moderate" is selected from that group, and the DataID 2221 for the finding "moderately enlarged left atrium" is stored in the EntryData table. If all the branches of a tree will not fit simultaneously on the screen shown in FIG. 5, the display scrolls to the left as new branches are selected until there are no more branches to display and the finding is saved to the EntryData table. The purpose for separating the tree transversal information found in the Sequence table from the textual information found in the Elements table is for language translation. The entire database can be translated by changing the text in the Elements table without rebuilding the traversal information. Some typical records in the Elements table are displayed below:
______________________________________
DataID Label Text
______________________________________
1200 Atrium Atrium
1877 right right atrium
1878 left left atrium
1950 calcified calcified left atrium
1951 clot clot in the left atrium
1952 enlarged enlarged left atrium
1953 regurgitation
left atrial regurgitation
2220 mild mild left atrial enlargement
2221 moderate moderate left atrial enlargement
2222 severe severe left atrial enlargement
______________________________________
When a finding is chosen, the DataID field is stored in a data table associated with the patient's file. For example, when a physician selects "moderate left atrial enlargement" the following record in the data table is added:
______________________________________
AdmissionID
DataID Frame Value
______________________________________
8829 221
______________________________________
The AdmissionID field contains a link to the patient admission for which the finding was entered. The frame field is used to link a finding to a particular ultrasound image frame (if applicable). The value field is used to store information to findings that require numerical data (such as measurements). Saving the patient information in the demographic data module permits a diagnosing physician to review the findings of previous scans for purposes of comparison to assess deterioration/regeneration. This simple tree structure permits physicians to quickly select a diagnosis using point and click techniques without any requirement to dictate diagnosis to tape or use transcriptionists. If the physician wishes to create freeform text aside from the options provided by the clinical data module tree structure, the physician can select the comments buttons from the clinical data buttons 156 (see FIG. 5) which permits the physician to enter freeform text through the keyboard of the review station 26. The physician can also select the "diagrams" which displays a diagram of the human heart, for example, to permit the physician to select appropriate descriptors from an associated list provided with each section of the organ. While the preferred embodiment of the invention has been described in relation to cardiology ultrasound, it should be understood that the invention can be equally well applied to any other discipline which requires ultrasound scans and is in no way limited to studies involving cardiology or any other discipline. Changes and modifications to the preferred embodiment described may be apparent to those skilled in the art. The invention is therefore intended to limited only by the scope of the appended claims.
|
Same subclass Same class Consider this |
||||||||||