Method for computer-assisted media processing5553281Abstract A system for computer-assisted processing of media by an end-user is disclosed. The system preferably includes a relational database management system (RDBMS) containing media content. The RDBMS structure allows users to effectively search an extensive database and identify desired content. At the same time, the adaptive designation of "in-fame" and "out-frame" pointers allows users to efficiently view, modify, annotate, store, and/or distribute to other users only the specific segments of content that are of interest. This highly desirable combination of features reflects the untraditional, open architecture of the system disclosed in accordance with the present invention. Claims We claim: Description FIELD OF THE INVENTION
TABLE 1
______________________________________
Show:
##STR1## Episode:
##STR2##
Scene:
##STR3## Take:
##STR4##
{Motion Picture or Television?}
{Short, Opening, Closing, Trailer, Commercial, or Series?}
{Animation or Live Footage?}
{Background, Prop, or Character?}
{Motion or Still?}
Keywords:
##STR5##
##STR6##
Groups:
##STR7##
##STR8##
Executive producer:
##STR9##
Producer:
##STR10##
Producer:
##STR11## Director:
##STR12##
Artist:
##STR13## Creator:
##STR14##
Writer:
##STR15## FX Supervisor:
##STR16##
Worker:
##STR17## Actor:
##STR18##
______________________________________
One especially noteworthy aspect of the preferred search criteria is the notion of "groups." As discussed further below, when the end-user has identified a particular collection of media content that he or she would like to be able to collectively reference in the future, the end-user is permitted to enter a group name for the collection. The entered name is then stored on the "group" attribute list for each record corresponding to the selected collection of content. From that point on, the defined group name may be included among search criteria, to identify the particular content so designated. Thus, the group attribute facility effectively allows the end-user to adaptively modify the structure of the database in desired fashion. Browser 4 preferably controls workstation 2 so as to provide a convenient graphical interface for input by the end-user of the foregoing search criteria; an on-screen form or menu of items to be interactively completed or selected by the end-user works well for this purpose, as will be apparent to those of skill in the art. At step 22, browser 4 invokes RDBMS engine 10, causing engine 10 to perform a search of media database 6 based on the specified search criteria. RDBMS implementation and operation are well-known in the art. For example, a detailed and comprehensive discussion of RDBMS theory and application is provided in C. J. Date, "An Introduction To Database Systems," Volumes 1 and 2, 4th Edition, Addison Wesley (1986). Therefore, only a brief description of pertinent points is provided herein. As noted briefly above, each clip or portion of content in media database 6 is associated with certain management information such as key words and attribute values. This association is illustrated in FIG. 3. Content Table 50 contains a plurality of content records; the exemplary structure of such records is illustrated in Content Record 58. Content Record 58 contains a plurality of fields: one such field (say, field 58a) contains a "key" or pointer to a portion or clip of content in media database 6 (stored somewhere in devices 8a-n) that is associated with this record; the key preferably incorporates not only the address of the clip, but also any permission information that may be necessary to obtain access to the clip. Another field (say, field 58b) points to a compressed, "preview" version of the clip; digital implementation of such compressed previews is well-known in the art (e.g., using video compression standards such as Apple Quicktimer.TM.). Another field (say, field 58c) points to storage of a representative frame (or "key frame") selected in advance from the clip. The function and purpose of the preview and key frame information will be explained below. Other fields (say, fields 58d, 58e, and possibly others) store or identify the various keywords and other attribute values associated with the corresponding clip. In addition, an index is created, in the form of a table or B-tree, linking each keyword or attribute value to the specific content records matching that value. For example, Keyword Index 54 is shown in FIG. 3 as containing a plurality of keyword entries 56a-n; each entry contains pointers to all of the content records in Content Table 50 that have a matching keyword. Similar indices are built and stored for each of the various attributes used to index the database. In this way, a "many-to-many" relationship between content and attributes is established. In other words, many different keywords (e.g., "forest") may correspond to a given portion of content, while many portions of content (e.g., various clips which all involve forest settings) may correspond to the same keyword. The implementation and operation of a relational database using B-tree structures is very well-known in the art. We have found the library of RDBMS B-tree routines and structures licensed by the C-Tree company, Cambridge, Mass. to offer good performance and functionality for these purposes. An important feature of media database 6 according to the present invention is the clear distinction between management information and underlying content. RDBMS engine 10, for example, deals with management information; underlying content is generally accessed by interested applications directly through keys according to the invention, not through the RDBMS engine. Because of this distinction, certain aspects of the system, such as indexed searching for content based on criteria, focus on management information, and see content only through the structure of Content Table 50 and related indices. Yet, at the same time, other aspects of the system, such as the designation and processing of selected content, access content directly via keys, and can therefore define selected content in a more flexible and adaptive manner--as discussed in the sections that follow. C. Selection of Content After the database search has been performed, the results of that search are communicated to the end-user in step 24. Browser 4 controls workstation 2 so as to display a list of the portions of media content that satisfy the user's specified search criteria. FIG. 4 depicts an exemplary graphical user interface screen display 60 for this purpose. Key-frame grid 62 is a rectangular grid; each element of the grid contains a single key-frame chosen to represent a portion of the media content. As noted above, the management information associated with each portion of content in media database 6 includes a predefined key-frame. Thus, each portion of content identified in the search of step 22 is represented at step 24 by the display of a corresponding key-frame within grid 62. At step 26, the end-user selects the particular media content that he or she wishes to process further. FIG. 4 depicts an exemplary graphical user interface screen display 60 for the selection process, and FIG. 5 describes the process in greater detail. At step 70, the end-user preferably investigates portions of content that may be of interest by previewing such content. This is accomplished by using a mouse or similar interactive pointing device to select any key-frame in grid 62 corresponding to content that the end-user is tentatively interested in pursuing further. In the case of full-motion media, a relatively low-resolution, compressed version of the corresponding media content is then immediately played on-screen in a pop-up window for the end-user. As noted previously, the management information associated with each portion of content preferably includes such a compressed preview version. Suppose the end-user decides, at step 72, that he or she would like to process some or all of the previewed content further. In that case, at step 74 the end-user again uses a mouse or similar interactive pointing device to point and drag the appropriate key-frame from grid 62 to accumulator 64. Next, at step 76, in an important aspect of the present invention, the end-user designates the specific segment within the previewed content that is to be selected for further processing. More specifically, the end-user identifies the individual starting and ending frames ("in-frame" and "out-frame") of the segment within the chosen clip of content that is of interest. The designation of in and out frames may preferably and conveniently be accomplished as follows. After a chosen clip has been added to the accumulator, and in response to the end-user's command, a preview of the clip is again played on-screen in a pop-up window. However, this time the preview is played more slowly, in "flip-book" fashion, so that the user can click on displayed in-frame and out-frame buttons to interactively identify the beginning and end points of the desired segment of content. As stated previously, browser 4 is responsible for controlling the graphical user interaction. When the end-user chooses a particular clip and designates an in-frame and out-frame, browser 4 examines the database management information record corresponding to the clip, and formulates and stores the key necessary to address and access that selected segment of content. In this way, the end-user is allowed to adaptively designate any segment of media content that is contained within any item of content included in the list. In other words, the end-user can specify the particular media content that he or she is interested in at a finer level of granularity than is reflected by the fixed structure of the database. As shown at step 78, the process of selecting content is iterative in nature; the user may select and deposit in accumulator 64 as many portions of content (represented by key-frames in grid 62) as desired for further processing. D. Processing of Selected Content At step 28, the end-user chooses a particular function to be performed with the media content that has been selected. Again, browser 4 preferably controls workstation 2 so as to permit the end-user to interactively indicate this choice. Graphical user interface 60 shown in FIG. 4 preferably includes an array of function buttons or icons 66 corresponding to each available processing function; to select a particular function, the user may simply click on the appropriate button or icon. The function chosen by the end-user may typically involve such media production tasks as printing content on an output device (step 30), modifying or editing a particular portion of content (step 32), or collaborating with another end-user (step 38). According to the present invention, the end-user may also create and store an electronic "bookmark" designating a selected segment of content (step 40) for future reference, or may create a new attribute to be associated in database 6 with a group of selected media content so that the selected content may be retrieved in the future as a group (step 42). The details of each such function or task are discussed below, in this section and the subsequent sections. FIG. 6 illustrates the carrying out of production tasks involving printing selected content on an output device. At step 82, browser 4 identifies the appropriate hardware device (and associated driver routine) 12 for carrying out the requested function. Next, at step 84, browser 4 invokes the identified driver 12, providing it with keys to access the selected content. Browser 4 also provides the driver with any additional database management information that may be necessary, such as format specifications for the selected content. At step 86, driver 12 uses the keys to access the selected content, for processing by the associated device. If the content is in a data format that is not compatible with target device 12, then at steps 88-90 driver 12 converts the format of the selected content as needed. For this purpose, driver 12 preferably includes Host Independent Image Protocol ("HIIP") interface software licensed by ASDG Incorporated. In any case, at step 92, driver 12 actually sends the selected content to associated device 12, for printing as requested. Similarly, FIG. 7 illustrates the carrying out of production tasks such as modifying or editing a particular portion of content according to the principles of the current invention. First, at step 98, browser 4 interrogates RDBMS engine 10 to find out if the selected content has already been "checked out" and is currently being modified by a second user of the system. If the content has been checked out, then our first user cannot be permitted to modify the content at this time, and at step 99 he or she is returned to the choice point of FIG. 2, step 44, to select an alternative course of action. As is well known in the art, this precaution is necessary to avoid corruption of database content. If the selected content is not currently checked out, however, then at step 100, browser 4 identifies the appropriate application program (and associated driver routine) 14 for carrying out the requested function. Next, at step 102, browser 4 invokes the identified driver 14, providing it with keys to access the selected content. Browser 4 also provides the driver with any additional database management information that may be necessary, such as format specifications for the selected content. Note that browser 4 does not itself actually access the underlying content. This eliminates any redundant copying or transmission of content data to or from browser 4; browser 4 can work entirely in terms of keys and management information, which improves efficiency greatly. At step 104, driver 14 uses the keys to access the selected content, for processing by the associated application. By accessing content directly by means of keys, instead of requesting content through RDBMS engine 10, driver 14 can access the precise segment of content needed (i.e., as defined in part by the in-frames and out-frames selected by the end-user), as reflected in the key, rather than automatically accessing the entire clip as organized in the database. Once again, if the selected content is in a data format that is not compatible with target application 14, then at steps 106-108 driver 14 converts the format of the selected content as needed, as discussed above with regard to FIG. 6. In any case, at step 110 driver 14 actually invokes associated application 14, which modifies the content as requested. Typically, application 14 will itself be an interactive program (such as a "paint" utility, or an on-line editing tool), and so the end-user will directly control the processing performed at step 110. After the desired modifications are complete, the end-user is preferably given the opportunity, at step 34 in FIG. 2, to decide whether or not to permanently save the content as modified. If the end-user does elect to save the modifications, then step 36 updates database 6 appropriately. This update should include an opportunity for the end-user to modify the keyword or other attribute values associated with the modified content, since the modifications may have directly changed some of those attributes. Browser 4 preferably controls this interaction, again by providing a suitable, on-screen, interactive menu or form. E. Collaboration (E-mail) In a preferred embodiment of the invention, a given end-user may collaborate with one or more other users in processing the contents of media database 6 (step 38). The details of this technique are now explained with reference to FIG. 8. At step 120, browser 4 controls workstation 2 to enable first end-user ("user1") to input textual instructions corresponding to each portion of selected content, and intended for a second end-user ("user2"). At step 122, browser 4 (preferably using the underlying operating system electronic mail routines) transmits an electronic message ("e-mail") to user2; the message contains pointers to the management information and keys associated with each portion of selected content defined by user1, and also contains user1's textual instructions for each portion of selected content. After sending the message, user1 can then proceed to other work (step 124). Meanwhile, at step 126, browser 16 (associated with user2 and workstation 18) notifies user2 that he or she has been sent an e-mail, and user2 elects to receive the e-mail. At step 128, browser 16 controls workstation 18 so as to display a list of the selected content identified in the e-mail message. Preferably, this display is provided by browser 16 in the same fashion as the display of search results described earlier in connection with FIG. 4: i.e., using the grid of key frames. Under control of user2 (preferably using a mouse to point and click on key frames of interest), user1's instructions for each portion of content are also displayed. At that point, user2 can proceed to select and process specific content in the same fashion described earlier for the single-user embodiment: i.e., at step 132, user2 selects specific content to be processed according to the steps of FIG. 5, and then continues on at step 28 of FIG. 2. In this way, user1 and user2 can conveniently and efficiently collaborate by exchanging information and instructions that are highly specific to particular portions of content that are of joint interest. F. Reference Functions Another application of the present invention is that an end-user may preferably define and store an electronic "bookmark" designating a selected segment of content (step 40) for future reference. The details of this technique are now explained with reference to FIG. 9. If this function is chosen, then at step 150, browser 4 controls workstation 18 (or a separate monitor) to play the selected content for the end-user. As shown in decision step 152, the end-user may continue viewing as long as desired. If the end-user desires to mark a particular point in the media content (e.g., so that the end-user may temporarily stop viewing the current content and resume viewing later on at the same point, or for other reference purposes), he or she may elect to insert an electronic bookmark. In that case, at step 154, browser 4 creates a modified key for the content being viewed, with a new beginning frame corresponding to the position in the content that the end-user wishes to mark. Browser 4 stores the key, at step 156, for future reference by the end-user. As shown in step 158, the end-user may then preferably elect to either stop viewing the current content--in which case, the system returns to the main choice point of FIG. 2, step 44--or to continue further. One special function that may be requested by the end-user with respect to selected content is the "group" function, which creates a new attribute to be permanently associated in the database with the selected content. The group function corresponds to step 49, and is a relatively straightforward step: if the end-user selects the "group" function button, then browser 4 controls workstation 2 so as to graphically prompt the end-user for a group name, which could preferably be any text string. The database "group" index (i.e., the table or tree for groups) is then updated to contain the newly created group item, and pointers to the selected content in that group. In this way, selected content may conveniently and efficiently be retrieved in the future by the end-user as a group, simply by specifying the new group name as part of the end-user's search criteria. G. Iteration It is envisioned that the system will be used in an iterative manner. Thus, as shown in FIG. 2, after a particular chosen task has been performed, the end-user may decide at step 44 whether to perform additional processing functions with the selected content (i.e., return to step 28), whether to select different media content for processing (i.e., return to step 26), or whether to define and commence a new search (i.e., return to step 20). The invention has been described in an exemplary and preferred manner, but is not limited thereto. Those skilled in the art will recognize that a number of modifications and improvements can be made to various aspects of the invention without departure from the essential spirit and scope. The scope of the invention is limited only as described in the following claims:
|
Same subclass Same class Consider this |
||||||||||