System and method for annotating and capturing chart data7036081Abstract A program is disclosed for permitting a user to directly annotate and capture data associated with a Java™ based applet application running within a conventional browser program. The program addresses sand-box local security restrictions otherwise attendant with such applet by exploiting non-local system resources that are accessible to such types of applications. Claims What is claimed is: Description CROSS REFERENCE TO MICROFICHE APPENDIX
In the above described method, the modified data is derived from an initial data file (which may include more than one initial data file) and supplemental data input under control of a user of the remote program. In a further variation, the updated data file is image data compressed using an encoder which translates a pixel stream into a file format usable by the browser program or another program having access to the local file system. Data transfers of the present invention are accomplished through an on-line connection between a local computing system and a remote server preferably using the following steps:
Again, in a preferred embodiment, the modified data is compressed image data derived from dynamically modifying an initial data file. This image data is compressed using an encoder which translates a pixel stream into a file format usable by the second program. Nonetheless, the initial data file can be in a graphics file format, an audio file format, a text format, a video format, or some combination thereof. An interactive, on-line session of the present invention permitting a user to engage in an interactive on-line session with a server using a remote program downloaded from the server but executing on the user's local computing system, and wherein the remote program is restricted from accessing a file system on such local computing system, is accomplished as follows:
In this manner, the browser program can thereafter transfer such modified file to an output and/or storage device in the local computing system, such as a printer, a local file system, etc. Although the inventions are described below in a preferred embodiment, it will be apparent to those skilled in the art the present invention would be beneficially used in many environments where it is necessary to provide security constrained software routines with additional functionality. BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a block diagram illustrating an embodiment of a system implemented in accordance with the teachings of the present invention; FIG. 2 depicts in flow chart form the general method of operation of the system noted in FIG. 1; FIGS. 3A to 3C illustrates the output generated by the present system and method as such would be seen on a display device by a user of such system; DETAILED DESCRIPTION OF THE INVENTION A preferred embodiment of a system 100 constructed in accordance with the present disclosure is shown in block diagram form in FIG. 1. A local client computing system 102 includes a number of components found in any typical contemporary computing system, including a processor 105 operable with a system code/data memory 110 (normally a DRAM), one or more input devices 115 (usually a keyboard, mouse, joystick or the like), a display memory 120 and associated buffer 121) (typically a supplemental fast VRAM or DRAM separate from system memory 110), a display 125 (preferably a CRT monitor, LCD panel etc.), one or more storage devices 130 (ordinarily a hard disk drive or other magnetic/optical based devices), an output device 135 (usually a printer) and an I/O interface 140 (preferably a high speed modem and associated drivers) for establishing an electronic connection 142 to the internet, an intranet, or other computer network. All of the aforementioned components are conventional, and it is understood that such computing system may range in complexity spanning the spectrum from a large multi-user mainframe down to personal electronic assistants. Operating within code/data memory 110 under control of processor 105 is a browser program 150, and an applet 160 executing within such browser. This applet runs in a standard Java virtual machine (JVM) within the browser. This operational relationship is denoted by browser program 150 enclosing applet 160; it will be apparent to those skilled in the art that this visual representation is merely intended for illustrative purposes, and does not purport to describe the entirety of the relationship between the browser and the applet. Moreover, since this relationship is well-known in the art, see e.g., the Kamins reference discussed above, it is not necessary to replicate such details here. Browser program 150 (preferably one of the more robust versions known in the art, such as Netscape Navigator or Microsoft Explorer) is configured to operate within system memory 110, and is capable of interacting/communicating of course with the other components within system 102 in conventional fashion, as well as through internet link 142 with a remote server 180 located at an ISP site. Remote server 180 also includes many of the same components of system 102 (albeit in larger scale to accommodate the needs of several potential data requesters) such as an I/O interface 181, a processor 182, a server code/data memory 183, and a storage system 185. This system is used to store both an initial data file 186, and an modified data file 187, which items are discussed in more detail below. Returning back to applet routine 160, it can be seen in FIG. 1 that this item includes applet code 162, which is written in Java® programming code; a listing of the source code used in the present invention is appended to the end of the present disclosure as Appendix A. This Java® code can be executed by means of a Java® interpreter in browser 160 to carry out operations within system 102. Applet code 162 has access to an applet data buffer 164, which, as shown in FIG. 1, is designed to store incoming/outgoing data through internet link 142. As seen in FIG. 1, initial window data 165, as well as updated window data 166 are located and stored within such applet data buffer 164. Finally, applet code 162 also has access to an associated applet window data buffer 166, which it can use for painting a window on display 125 as explained further below. This buffer can be generated with conventional Java® programming tools, including, for example, the createlrnage( ) command to any desired size for the applet and application in question. The relationship between the various applet routines and resources can be gleaned further from the source code listing attached hereto, which includes all of the routines discussed herein except for the freeware programs noted below. With the exception of the above details pertaining to applet 160, the other circuitry, structures and routines embodied in the block diagram of FIG. 1 are not material to the teachings of the present invention. These items are well-known in the art, and can be implemented by skilled artisans in a variety of ways. Moreover, it will be apparent to those of skill in the art that a number of conventional (but extraneous in this case) features of client system 102 and server system 180 have been omitted to better illustrate the present teachings. As FIG. 1 suggests, to maximize security and privacy for the user's local client computing system 102, applet 160 is operationally restricted and has only limited access to components within system 102. It cannot, for example, directly interact with a storage device 135, or an output device 135 in order to create (or read) a file within local client system 102. Nor can applet 160 operate to transfer a file directly to browser program 160. This is an undesirable situation, since, as mentioned above, it is likely that many users in the future will download files from remote servers, modify such files, and yet not be able to do anything useful with them afterwards. For example, they will not be able to save such modifications for use with other programs on the user's computing system. It will be understood by those skilled in the art that Java language applets are only one form of programming which may be subject to such restrictions, and that the present invention is by no means limited to such specific contexts. To overcome this limitation, applet code 162 of the present invention includes instructions which permit the local client computing system 102 of FIG. 1 to carry out the method of the present invention as shown in flow chart form in FIG. 2. At step 205, a user initiates an online session from his/her local client computing system 102 in conventional fashion, such as by dial-up or always-on access to an ISP. Browser 160 is then activated at step 210 to permit the user to see, interact and access other programs, files, etc. at step 215; such items may be located on the internet, including at a WWW compatible remote server 180 through an http link or the like. When the user desires to interact with a remote program on remote server 180, one or more applets 160 necessary for operating such program will be transmitted downstream to the user's computing system 102 through link 142 at step 220 where they will be loaded into a code/data memory 110 for execution on local computing system 102 during step 225. In a preferred embodiment, the remote program accessed and loaded by the user is located at a website maintained by Prophet Information Services at www.prophetcharts.com. This program is known as "Prophet Charts," and generally includes such functionality as: (1) permitting a user to retrieve historical stock data from a file stored on a remote server; (2) placing such stock data in graphical form in a visible display window at the user's end; (3) permitting the user to annotate the chart with trend lines, labels, etc. to glean trading patterns, potential buy/sell opportunities, etc. Accordingly, in the event the user requests to see data associated with an initial remote data file 186 (FIG. 1)—as for example, in the case of a historical chart showing the price of a particular stock for a particular period of time this file is retrieved by applet code 162 at step 230. In this instance, as described above, applet 160 can access such data in file 186 because the latter is located logically in the same environment from which applets 160 are derived. To ensure that such data is usable by the user within system 102, file 186 must be retrieved from server 180 in a format that is readable or decodable by browser 160; i.e., in text form, or some graphics form that is acceptable, such as GIF, TIFF, JPEG, or the like, depending on the particular browser program 160. In any event, initial data from file 186 is stored within Applet Data Buffer 164 at location 165 (as shown in FIG. 1) where it is processed by applet code 162 and then placed into Applet window data buffer 166. By coordinating with browser 160, a graphical representation of initial data file 186 is then passed through display buffer 121, where it then is placed in display memory 120 as noted in the dotted lines shown in FIG. 1. Subsequently, the applet window image data then also becomes visible to the user on display 125 as noted in step 235 of FIG. 2. Up until this point, the above description has merely described the typical operation of any prior art Java® applet program loaded from a remote site onto a local client computing system. In the present embodiment, nonetheless, a number of additional unique operative steps are implemented to provide such program user with functionality not previously available. Referring again to FIG. 2, the primary features, benefits, etc. of the present invention are based in large part on the operations carried out by applet code 162 beginning at step 240. At that point in the routine, the user is permitted to interact with the data presented in the applet window image on display 125. In particular, in a preferred embodiment, the user is allowed to add annotations to a graphical chart painted in such window. These annotations could include for example, data entered by the user manually from a keyboard/mouse 115, additional data generated automatically by applet code 162 in response to a user command from such input device, or some other authorized and accessible data source for the applet. These modifications (including deletions, supplementations or embellishments) of the window image data are handled at step 245 until the user is finished and wants to do some other task (i.e., saving the modified image data, or perhaps working with another image). Again, with reference to FIG. 1, it is apparent that such additional data entered at step 240 can be passed through to applet 160 and processed by applet code 162 to update applet window data buffer 164. From there it is passed on to display buffer 121 before it is used to update display memory 120 and the user's display 125. Nonetheless, even though the window image data has been updated to include the efforts of the user to include such extra user generated data, it is not easily "capturable," so as to help the user preserve their efforts because as it is stored it is not in a format that is understandable by browser program, and cannot be output by applet 160 to a storage device 130 because of the aforementioned security restrictions. Returning to FIG. 2, the present invention permits a user to retain the results of their efforts, and capture their modifications to the initial data in the following manner. First, at step 250, the user is prompted to determine if they would like to capture and preserve the modified window image data, i.e., such as by printing in hard copy form, or storing in some electronic form. If not, the user can retrieve a different file at step 230 or perform another operation of their choosing. Should the user desire to capture the updated window image, however, applet code 162 then proceeds at step 255 to convert such window image data into modified window data, which is stored in location 166 (FIG. 1). First, the window image data is captured using a built-in standard Java® programming tool known as pixelGrabber. This routine captures the pixels from the Java® image and renders them into a one-dimensional array of integers. Each integer contains the red green and blue components of the color of that pixel. Then, this array is passed through a converter, where it is translated or converted into a standard graphics file format. This conversion is preferably performed by a pixel-to-gif converter to convert the pixels of the applet window image data into a Graphics Interchange Format (GIF) file. This conversion can be accomplished, for example, by a number of prior art programs, including by a freeware software routine authored by Jef Poskanzer entitled GifEncoderjava and available at any number of file sites on the Internet including at www.acme.com. It should be noted that the selection of the particular file format, and/or particular converter is not material to the present teachings, and the present invention is by no means restricted to any particular combination thereof. The only important parameter, as explained below, is that the modified applet window image data must be changed into a format that is compatible with browser program 160 as noted below. Alternatively, such format need only be readable by browser program 160 so that it can be downloaded and used by another program on the local computing system 102. In any event, after such conversion, at step 260 modified window data 166 is then uploaded by applet 160 to remote server 180, where it can be written to remote data storage device 185 in a location 187 as a modified data file. This happens by opening a connection to a programming script running on web server 180 at the site the user is connected to. In the preferred embodiment, this is a script is implemented in "perl" which is a conventional, well-known scripting language commonly used by those skilled in the art. Basically, the perl script running on server 180 merely fetches the next available image number and writes the data from modified window data file 166 (containing the compressed array) to it, with a designation such as chartxxx.gif, where xxx is the next available number available on server 180. Because this file is in compressed format, the transfer time between local computing system 102 and remote server 180 should be relatively reasonable. Furthermore, it is expected that improvements in high speed internet access will likely obviate any concerns about such types of relatively small scale data transfers. At this point the captured user modified image data is now located on a server which can be accessed by browser program 150. The perl script on server 180, therefore, replies to applet 160 at step 265 with a URL indicating the location of the new file, such as "www.prophetcharts.corn/graphs/chartXXX.gif" and applet 160 then opens a new browser window, which can be displayed to the user on display 125, which points to that image file. Because such file is also retrieved in a format usable by browser 160, it can be manipulated and treated like any other file so that, for example, at step 270 it can now be saved in electronic form on local storage device 130, or printed on a local output device 135. The user can then proceed to retrieve additional data files at step 230, or as noted above, simply go on to another operation. As a housekeeping matter, and to save storage space, any such additional files stored on server 180 can be deleted after the user's session is over. It can be seen from the above description, therefore, that a server 180 is configured to permit a user remote from such server to engage in an interactive on-line session with such server using the aforementioned applet 160. In this way, server 180 can interact and exchange data with local computing system 102 as described above, and preserve any contributions made during such session for the user's benefit. While the above process appears somewhat circuitous, applicants have confirmed that it in fact proceeds very rapidly, and from the perspective of the user, does not involve an inordinate delay. Furthermore this process retains the integrity of the intended framework for Java® applets by ensuring that they do not access local resources. In this manner, users are guaranteed that the applets are kept in the Java® "sandbox," and not permitted to perform operations that might compromise the user's privacy or security The present invention is extremely useful, therefore, for preserving and capturing the user's edits, modification, supplementations of an applet window's image data during an interactive session with a remote server, which, absent the present teachings, would be otherwise lost resulting in reduced productivity, utility, user satisfaction, etc. It is expected that the present teachings can be used in any number of Java® applet based programs, therefore, where it is desirable to permit the user to interact with applet data and yet still permit the user to exploit the use of local resources without the restrictions normally imposed on applet code. The present invention opens up a new realm of possibilities for WWW programmers, web-site operators, on-line vendors and users of the since it now allows data to be freely transferred and then modified under the user's control in a manner that preserves the user's contributions. FIGS. 3A to 3C are a series of screen shots illustrating the operation of the present invention as perceived by a user of a remote program which is used to launch locally running applets on the user's local computing system. In FIG. 3A, a display 300 presents a user with a graphical interface 310 (generated by underlying browser 160) and an applet generated window image 320, which window contains applet window image data 325 (derived from initial data file 165) as well as command buttons/activators 328 at the top of window 320. The latter are used, as explained above, to permit a user to interact with the image presented in window 320, so that, as seen in FIG. 3B, such user can add additional information to such window data, remove information from the window, modify the existing data, etc. In this instance, the user has added a trend line 330 notation to the chart data image to reflect some additional perception by such user about such data that is not represented in the original data file 166; for example, an indicator showing the general trend of a stock price over time. Other image data annotations are of course possible, including descriptive labels, applet generated calculations, etc. A second data file containing data for a second stock could be subsequently loaded, for example, so that a composite image having data from both the first and second stocks could be seen in window 320. Assuming the user desires to capture the image data including the added trend line 330, the conversion steps described above beginning with step 255 (FIG. 2) are then performed as noted. Looking at FIG. 3C, the returned modified data file 166 is then displayed in a graphics image format in a separate window 350 where, as noted earlier, it can be printed, saved, etc. by the user. As alluded to briefly above, in another variation of the present invention, applet 160 can also permit such user to import an additional data file 166′ from server 180, for example, representing data for a different stock so that a combined data image utilizing data from two different files 166 and 166′ can be observed simultaneously in an overlay fashion and manipulated by the user. Such data from the different files can be represented in different colors in applet data image window 320 to help the user differentiate the behavior of the two superimposed data sets for the selected two stocks. Such combined data from the two files (or more if desired) can then be converted into a suitable file format as explained above for the single image case, and then handled subsequently in the same manner. In yet another embodiment, the modified data file can include information other than graphics data. For example, the user can download a standard audio file from the remote serve, edit such audio file using any conventional audio file editor, and then save any modifications in the same manner as performed above for the image data. Then, instead of receiving back a GIP file, a WAV file, MIDI file, etc., could be returned by the server. Standard encoders for compressing audio information can be used for generating the modified file at the local computing system before sending on this file to the server for later retrieval. Similar editing sessions with other types of files retrieved during an on-line session and manipulated by the user could be captured as well. In this manner, a user's contributions associated with an audio file, a text file, a video file, or any kind of multimedia file can be preserved and stored on the local computing system, even when using restricted code such as Java applets. Although the present invention has been described in terms of a preferred embodiment, it will be apparent to those skilled in the art that many alterations and modifications may be made to such embodiments without departing from the teachings of the present invention. Accordingly, it is intended that the all such alterations and modifications be included within the scope and spirit of the invention as defined by the following claims. Attached hereto is an appendix of the source code used in the various applet routines described above.
|
Same subclass Same class Consider this |
||||||||||