Method and apparatus for content repository with versioning and data modeling

Abstract

The present invention is related to method and apparatus for versioning. Versioning information is stored as metadata associated with data entities. A hierarchical relationship between the entities allows the definition of an editorial sequence and separate revisions within that sequence as well as relationships between sequences in a self-contained format.

Claims

1. A content repository computer system comprising a storage device for a plurality of persistent data entities including metadata each entity having a predetermined level of scope such that within a set of related data entities, the scope of an entity at a higher level encompasses the scope of related entities at a lower level of scope, wherein at least one entity includes metadata that identifies a sequential relationship between one or more entities within the scope of said one entity, each of said entities including metadata defining a position within said sequential relationship.

2. The system according to claim 1, wherein a further entity includes, in addition to metadata identifying a sequential relationship between one or more entities within the scope of said further entity, metadata identifying as a source of said sequential relationship, one of said entities within the scope of said at least one entity.

3. The system according to claim 1, wherein said storage device is connected to a network.

4. The system according to claim 1, including a plurality of said storage devices.

5. A content repository computer system comprising a storage device for a plurality of persistent data entities including metadata each entity having a predetermined level of scope such that within a set of related data entities, the scope of an entity at a higher level encompasses the scope of related entities at a lower level of scope, wherein within the scope of an entity at a higher level there exists at least one entity at an intermediate level whose metadata identifies a sequential relationship between one or more entities of a lower level and within the scope of said one entity, each of said entities at said lower level including metadata defining a position within said sequential relationship.

6. The system according to claim 5, wherein a further entity at said intermediate level includes, in addition to metadata identifying a sequential relationship between one or more entities of a lower level and within the scope of said further entity, metadata identifying as a source of said sequential relationship, one of said entities at a lower level within the scope of said one entity at said intermediate level.

7. The system according to claim 5, wherein said storage device is connected to a network.

8. The system according to claim 5, including a plurality of said storage devices.

Description

BACKGROUND

1. Field of the Invention

This invention relates to management and distribution of electronic media, and more specifically to versioning and data modeling.

2. Discussion of the Related Art

With the advent of the computer and particularly the networking of computers, the ability of organizations and individuals to rapidly generate, store, access and process data has increased dramatically. In the case of many organizations, the ability to manage and leverage data has become a central aspect of their business.

Not surprisingly, considerable effort and development has occurred in those computational and software fields related to the generation, storage, accessibility and processing of data. Nevertheless, it has been the case that as organizations have moved to a distributed architecture paralleling the development of the Internet, the complexity involved in providing solutions across different platforms and operating systems has become ever more challenging. Consequently, developers have tended to concentrate on limited solutions for preferred platforms and operating systems. Similarly, organizations have sought to standardize the tools they use to leverage data.

Unfortunately, the pull exerted by those distributed computing models currently finding favor is in direct contradiction to the solutions adopted by the majority of developers and those responsible within organizations for the selection of tools. Consequently, the management and distribution of data, particular of high value media content remains problematic. In particular, present techniques do not successfully address the need to be able to manage and track the editorial lifecycle of data entities and be able to differentiate between distinct yet related variants of a common body of information.

SUMMARY OF THE INVENTION

Thus, according to one aspect of the invention, there is provided a content repository system comprising a storage device for a plurality of persistent data entities including metadata each entity having a predetermined level of scope such that within a set of related data entities, the scope of an entity at a higher level encompasses the scope of related entities at a lower level of scope, wherein at least one entity includes metadata that identifies a sequential relationship between one or more entities within the scope of said one entity, each of said entities including metadata defining a position within said sequential relationship.

Advantageously, the identification provided by the metadata of the at least one entity corresponds to an indication of an editorial sequence or release comprising those entities within its scope each of which include metadata defining a position or version with the sequence. Preferably, where a relationship exists between one or more such editorial sequences, then a further entity indicative of a different release will contain within its metadata an indication of the source of that release. Such an indication may identify a particular revision within another release. Thus, the system seeks to overcome a difficult present in known tree-based versioning models namely their inability to explicitly define relationships between different releases.

According to a further aspect of the invention, there is provided a versioning method for an object oriented programming environment comprising a set of persistent data entities each having a predetermined level of scope wherein a related set of data entities comprises a hierarchical plurality of levels such that an entity at a higher level of scope encompasses the scope of related entities at a lower level of scope, the method comprising associating metadata with an entity indicative of a position within a sequence of one or more entities of corresponding scope said sequence being identified by associating further metadata with an entity whose scope encompasses said sequence of one or more entities.

Such a method may be implemented on any suitable platform with any suitable environment including a network comprising mobile and/or fixed elements. By defining versioning information within metadata, it permits the generation of a versioning model suited to a particular agent or user request. Thus, by way of example, a tree-based versioning model may be generated from the metadata albeit with explicit definition of the relationships between releases. It will, of course, be apparent to those skilled in the art that other versioning models may be generated.

Furthermore, according to yet another aspect of the invention, there is provided a data modeling tool for an object oriented programming environment comprising a set of persistent data entities each having a predetermined level of scope wherein a related set of data entities comprises a hierarchical plurality of levels such that an entity at a higher level of scope encompasses the scope of related entities at a lower level of scope, the tool comprising an interface operable to receive a request specifying a data model in terms of relationships between sequences of entities and a processor operable in response to said received request to generate a data model utilizing metadata associated with said entities, wherein at least one entity includes metadata indicative of a position within a sequence of one or more entities of corresponding scope said sequence being identified by further metadata associated with an entity whose scope encompasses said sequence of one or more entities.

Such a tool may be implemented within an agent that, because the physical data is abstracted to metadata and manipulations of said metadata are carried out through a predetermined language and semantics, permits the agent to operate in different environments perhaps containing further agents. Clearly, such a tool is equally applicable to environments made up of a single machine or a distributed network.

Furthermore, because the agents within the system interact utilizing the predetermined language and semantics there is a reduced requirement for knowledge of specialized configuration and implementation details of each agent.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention is further described in the detailed description which follows in reference to the noted plurality of drawings by way of non-limiting examples of embodiments of the present invention in which like reference numerals represent similar parts throughout the several views of the drawings and wherein:

FIG. 1 is a diagram of a system for information delivery according to an example embodiment of the present invention;

FIG. 2 is a diagram of a conceptual level showing the relationships between framework elements according to an example embodiment of the present invention;

FIG. 3 is a diagram of an identity architecture defined by a framework according to an example embodiment of the present invention; and

FIG. 4 is a diagram of a Registry Service architecture according to an example embodiment of the present invention.

DETAILED DESCRIPTION

The particulars shown herein are by way of example and for purposes of illustrative discussion of the embodiments of the present invention. The description taken with the drawings make it apparent to those skilled in the art how the present invention may be embodied in practice.

Further, arrangements may be shown in block diagram form in order to avoid obscuring the invention, and also in view of the fact that specifics with respect to implementation of such block diagram arrangements is highly dependent upon the platform within which the present invention is to be implemented, i.e., specifics should be well within purview of one skilled in the art. Where specific details (e.g., circuits, flowcharts) are set forth in order to describe example embodiments of the invention, it should be apparent to one skilled in the art that the invention can be practiced without these specific details. Finally, it should be apparent that any combination of hard-wired circuitry and software instructions can be used to implement embodiments of the present invention, i.e., the present invention is not limited to any specific combination of hardware circuitry and software instructions.

Although example embodiments of the present invention may be described using an example system block diagram in an example host unit environment, practice of the invention is not limited thereto, i.e., the invention may be able to be practiced with other types of systems, and in other types of environments.

Reference in the specification to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the invention. The appearances of the phrase "in one embodiment" in various places in the specification are not necessarily all referring to the same embodiment.

The present invention relates to a Metia Framework that defines a set of standard, open and portable models, interfaces, and protocols facilitating the construction of tools and environments optimized for the management, referencing, distribution, storage, and retrieval of electronic media; as well as a set of core software components (agents) providing functions and services relating to archival, versioning, access control, search, retrieval, conversion, navigation, and metadata management.

A Metia Framework according to the present invention may serve as the foundation for the realization of corporate documentation strategy, upon which company wide tools and services operate. A Metia Framework according to the present invention addresses the common requirements of all corporate business units, while also allowing custom extensibility by specific business units for special needs.

A Metia Framework architecture according to the present invention may be based on a standard HTTP 2 web server and is media neutral, such that the particular encoding of any data is not relevant to storage by or interchange between agents. This does not mean that specific encoding or other media constraints may not exist for any given environment implementing the framework, depending on the operating system(s), tools, and processes used, only that the framework itself aims not to impose any such constraints itself. Non-agent systems, processes, tools, or services that are utilized by an agent can still be accessed via proprietary means if necessary or useful for any operations or processes outside of the scope of the framework. Thus, framework based tools and services can co-exist freely with other tools and services utilizing the same resources. A Metia Framework according to the present invention brings together both existing, legacy systems as well as new solutions into a common, interoperable environment; maximizing the investment in current systems while reducing the cost and risk of evolving and/or new solutions.

A Metia Framework according to the present invention may be comprised of a number of components, each defining a core area of functionality needed in the construction of a complete production and distribution environment. Each framework component is defined separately by its own specification, in addition to a top level framework specification. The top level specification will be referred to as Metia Framework for Electronic Media. The other framework components include Media Attribution and Reference Semantics (MARS), Generalized Media Archive (GMA), Portable Media Archive (PMA), and Registry Service Architecture (REGS).

MARS is a metadata specification framework and core standard vocabulary and semantics facilitating the portable management, referencing, distribution, storage and retrieval of electronic media. MARS is designed specifically for the definition of metadata for use by automated systems and for the consistent, platform independent communication between software components storing, exchanging, modifying, accessing, searching, and/or displaying various types of information such as documentation, images, video, etc. It is designed with considerations for automated processing and storage by computer systems in mind, not particularly for direct consumption by humans; though mechanisms are provided for associating with any given metadata property one or more presentation labels for use in user interfaces, reports, forms, etc.

The GMA defines an abstract archival model for the storage and management of data based solely on Media Attribution and Reference Semantics (MARS) metadata; providing a uniform, consistent, and implementation independent model for information storage and retrieval, versioning, and access control. The GMA is a central component of the Metia Framework and serves as the common archival model for all managed media objects controlled, accessed, transferred or otherwise manipulated by Metia Framework agencies.

The PMA is a physical organization model of a file system based data repository conforming to and suitable for implementations of the Generalized Media Archive (GMA) abstract archival model. The PMA defines an explicit yet highly portable file system organization for the storage and retrieval of information based on Media Attribution and Reference Semantics (MARS) metadata. The PMA uses the MARS Identity metadata property values themselves as directory and/or file names, avoiding the need for a secondary referencing mechanism and thereby simplifying the implementation, maximizing efficiency, and producing a mnemonic organizational structure.

REGS is a generic architecture for dynamic query resolution agencies based on the Metia Framework and Media Attribution and Reference Semantics (MARS), providing a unified interface model for a broad range of search and retrieval tools. REGS provides a generic means to interact with any number of specialized search and retrieval tools using a common set of protocols and interfaces based on the Metia Framework; namely MARS metadata semantics and either a POSIX or CGI compliant interface. As with other Metia Framework components, this allows for much greater flexibility in the implementation and evolution of particular solutions while minimizing the interdependencies between the tools and their users (human or otherwise).

Initially, it should be noted that in order to improve the readability of the specification, sections that describe in detail all aspects of a particular component and that relate to the description of the embodiments described below, have been included at the end of the specification. When appropriate, reference has been made in the description to these sections by a title, name, or function of the section. These sections include Metia Framework for Electronic Media, Media Attribution and Reference Semantics (MARS), Portable Media Archive (PMA), Generalized Media Archive (GMA), and Registry Service Architecture (REGS).

FIG. 1 shows a diagram of a system for information delivery according to an example embodiment of the present invention. A network 1 includes an Hypertext Transfer Protocol (HTTP) web server 3 that may be accessible 4 by production clients 5 operating a number of operating systems on various platforms, and a set of on-line distribution clients 7. The on-line distribution clients 7 may include a wireless terminal 9 utilizing Wireless Mark-up Language (WML). As such, the terminal 9 may accesses 6 the HTTP web server 3 indirectly via a WAP server 1, which provides the necessary translation 8 between HTTP and WML. The HTUP web server 3 may further provide a Common Gateway Interface (CGI).

In addition to these physical elements of the network 1, data exchanged with the HTTP web server 1 may also be exchangeable 10 with an Agent pool 13 made up of a number of core software components or agents 13a, 13b, 13c, 13d providing services which will be elaborated upon below. Data exchanged 10 with the HTTP web server 3 by the Agent pool 13 may be transferred 12 between agents 13a-13d. The Agent pool 13 may have additional connections. A connection 14 may exist to a customer documentation server 15 capable of providing both on-line 17 and hard media 19 access to users. Moreover, a connection 16 may exist to a set of one or more archives 21 which themselves may be monitored and managed through an on-line connection 18 to a remote terminal 23.

FIG. 2 is a diagram of a conceptual level showing the relationships between framework elements according to an example embodiment of the present invention. A Media Attributions and Reference Semantics (MARS) 25 provides a core standard vocabulary and semantics utilizing metadata for facilitating the portable management, referencing, distribution, storage and retrieval of electronic media. As will be further described below, MARS 25 is the common language by which different elements of embodiments of the present invention communicate. A Generalized Media Archive (GMA) 27 provides an abstract archival model for the storage and management of data based on metadata defined by MARS 25. At a physical level, a Portable Media Archive (PMA) 29 provides an organizational model of a file system based data repository conforming to and suitable for implementations of the Generalized Media Archive (GMA) abstract archival model. A Registry Service Architecture (REGS) 31 may be provided which permits dynamic query resolution by agencies including users and software components or agents utilizing MARS 25, thereby providing a unified interface model for a broad range of search and retrieval tools.

As noted previously, a Framework according to the present invention may be based on a web server 3 running on a platform that provides basic command line and standard input/output stream functionality. An agent 13 may provide two interfaces, a combined Hypertext Transfer Protocol (HTTP) and Common Gateway Interface (CCL), HTTP+CGI, and a Portable Operating System Interface (POSIX) command line+standard input/output/error. In addition to these interfaces, the agent may provide further interfaces based on Java method invocation and/or Common Object Request Broker Architecture (CORBA) method invocation. An agent (or other user, client, or process) is free to choose among the available interfaces with which to communicate including communication with another such agent 13. In addition, a framework according to the present invention allows non-agent systems, processes, tools, or services that are utilized by an agent 13 to be accessed via proprietary means if necessary or useful for any operations or processes outside of the scope of the architecture. Thus, tools and services intended for the architecture can co-exist freely with other tools and services utilizing the same resources.

Specifically, the protocols on which a framework according to the present invention may be based include HTTP which is an application-level protocol for distributed, collaborative, hypermedia information systems. As a generic, stateless, protocol HTTP can be used for many tasks beyond hypertext. Thus, it may also be used with name servers and distributed object management systems, through extension of its request methods, error codes and headers. A particularly useful feature of HTTP is the typing and negotiation of data representation, allowing systems to be built independently of the data being transferred.

CGI is a standard for interfacing external applications with information servers, such as Web servers. CGI may serve as the primary communication mechanism between networked clients and software agents within a framework according to the present invention.

POSIX is a set of standard operating system interfaces based on the UNIX operating system. The POSIX interfaces were developed under the auspices of the 1EEE (Institute of Electrical and Electronics Engineers). A framework according to the present invention adopts the POSIX models for command line arguments, standard input streams, standard output streams, and standard error streams.

CORBA specifies a system that provides interoperability between objects in a heterogeneous, distributed environment that is transparent to a database programmer. Its design is based on the Object Management Group (OMG) Object Model. Framework agents may utilize CORBA as one of several means of agent intercommunication.

Java™ is both a programming language and a platform. Java is a high-level programming language intended to be architecture-neutral, object-oriented, portable, distributed, high-performance, interpreted, multithreaded, robust, dynamic, and secure. The Java platform is a "virtual machine" which is able to run any Java program on any machine for which an implementation of the Java virtual machine (JVM) exists. Most operating systems commonly in use today are able to support an implementation of the JVM. The core software components and agents provided by a framework according to the present invention may be implemented in Java.

Metadata is held within a framework according to the present invention using a naming scheme which is compatible across a broad range of encoding schemes including, but not limited to the following programming, scripting and command languages: C, C++, Objective C, Java, Visual BASIC, Ada, Smalltalk, LISP, Emacs Lisp, Scheme, Prolog, JavaScript/ECMASCriPt, Pen, Python, TCL, Bourne Shell, C Shell, Z Shell, Bash, Korn Shell, POSIX, Win32, REXX, and SQL.

The naming scheme according to the present invention may also be compatible with, but not limited to, the following mark-up and typesetting Languages: SGML, XML, HTML, XI-ITML, DSSSL, CSS, PostScript, and PDF. Equally, the naming scheme may be also compatible with but not limited to the following file systems: FAT (MS-DOS), VFAT (Windows 95/98), NTFS (Windows NT/2000), HFS (Macintosh), HPFS (OS/2), HP/UX, UFS (Solaris), ext2 (Linux), ODS-2 (VMS), NFS, ISO 9660 (CDROM), UDF (CDRIW, DVD).

In order to provide such compatibility, the naming scheme may utilize an explicit, bound, and typically ordinal set of values referred to hereinafter as a token. The token may comprise any sequence of characters beginning with a lowercase alphabetic character followed by zero or more lowercase alphanumeric characters with optional single intervening underscore characters. More specifically, any string matching the following POSIX regular expression:

Some examples may include: Abcd, ab_{—cd, a}123, x2345, and here_is_a_very_long_token_value.

By defining MARS metadata properties in a token format, an agent 13 or other tool is able to operate more efficiently as a result of its processes being based on controlled sets of explicitly defined values rather than those based on arbitrary values.

A token provides the structure through which a framework according to the present invention is able to define metadata in the form of a property. This property is representative of a quality or attribute assigned or related to an identifiable body of information. The property thus includes an ordered collection of one or more values sharing a common name. The name of the property represents the name of the collection and the value(s) represent the realization of that property. In accordance with the token structure adopted in the framework, constraints are placed on the values that may serve as the realization of a given property. A property set is thus any set of MARS 25 properties.

Further details of the property types allowed under MARS 25 are to be found in the MARS section following. Certain property values are also defined under MARS 25 and may also be found in the MARS section following. These include the property value of count that may be a single meaning that at most there may be one value for a given property or multiple meaning that there may be one or more values for a given property. Another property value is range which for any given property may be bounded or unbounded. In addition, the property value of ranking provides, for any given property, the set of allowed values for that property may be ordered by an implicit or explicit ordinal ranking, either presumed by all applications operating on or referencing those values or defined. Some property value types are ranked implicitly due to their type and subsequently the value ranges of all properties of such types are automatically ranked. Examples of such property types include Integer, Count, Date, Time and the like. Most properties with ranked value ranges are token types having a controlled set of allowed values which have a significant sequential ordering such as status, release, milestone and the like.

Ranking, if it is applied, may be either strict or partial. With strict ranking, no two values for a given property may share the same ranking. With partial ranking, multiple values may share the same rank, or may be unspecified for rank, having the implicit default rank of zero.

Ranked properties may only have single values. This is a special constraint which follows logically from the fact that ranking defines a relationship between objects having ranked values, and comparisons between ranked values becomes potentially ambiguous if multiple values are allowed. For example, if the values x, y, and z for property P have the ranking 1, 2, and 3 respectively, and object ‘foo’ has the property P(y) and object ‘bar’ has the property P(x,z), then a boolean query such as "foo.P <bar.P?" cannot be resolved to a single boolean result, as y is both less than z and greater than x. Thus the query is both true and false, depending on which value is chosen for bar.P (i.e. foo.P(y)<bar.P(x)=False, while foo.P(y)<bar.P(z)=True).

Ranking for all property types other than token are defined implicitly by the data type, usually conforming to fundamental mathematical or industry standard conventions. Ranking for token property values are specified using Ranking. In either case and as has already been stated, ranking may be strict in the sense that the set of allowed values for the given property corresponds to a strict ordering, and each value is associated with a unique ranking within that ordering. Alternatively, ranking may be partial in the sense that the set of allowed values for the given property corresponds to a partial ordering, and each value is associated with a ranking within that ordering, defaulting to zero if not otherwise specified. Finally, ranking may not be applied such that the set of allowed values for the given property corresponds to a free ordering, and any ranking specified for any value is disregarded.

FIG. 3 shows a diagram of an identity architecture defined by a framework according to an example embodiment of the present invention. The Identity architecture 33 may have a set of nested pre-determined definitions of specific scope each utilizing tokens to hold information. At the lowest level of scope, a Storage Item 35 corresponds to what would typically be stored in a single file or database record, and is the physical representation of the data that the framework is capable of manipulating. Thus, Items 35 are the discrete computational objects which are passed from process to process, and which form the building blocks from which the information space and the environment used to manage, navigate, and manipulate it are formed. Hence, an Item 35 may embody content, content fragments, metadata, revision deltas, or other information.

At the next highest level of scope, a Media Component 37 defines a particular realization of a defined token value. Thus, the Component 37 defines at an abstract level properties and characteristics of one of the following non-exhaustive content types, namely data, metadata, table of contents, index or glossary. A data content type might include a language, area of coverage, release or method of encoding. A component 37 is linked to one or more storage item 35 that relates to the content at a physical level.

Immediately, above the level of scope of the Media Component 37 is a Media Instance 39. The media instance 39 is made up of a number of media components 37 each of which relate to a particular property of an identifiable body of information. Thus, a particular Media Instance 39 may comprise a set of properties 37 namely a specific release, language, area of coverage and encoding method.

Finally, the highest level of scope is a Media Object 41 which represents an body of information corresponding to a common organizational concept such as a document, book, manual, chapter, section, sidebar, table, image, chart, diagram, graph, photograph, video segment, audio stream or the like.

However, the body of information is abstract to the extent that no specification is made of any particular language, coverage, encoding or indeed release. Thus, depending on the presence, or otherwise of information at the lower levels of scope, dictated ultimately by the existence or otherwise of a relevant Storage Item 35, it may be possible to realize some, if not all, particular media instances 39 corresponding to that media object 41.

In order to allow for referencing of specific content, namely a fragment within a given item, component, instance, or object, MARS 25 adopts the Worldwide Web Consortium (W3C) proposal for the XPointer standard for encoding such content specific references in SGML, HTML, or XML content. A fragment will be understood by those skilled in the art to be an identifiable linear sub-sequence of the data content of a component 37, either static or reproducible, which is normally provided where the full content is either too large in volume for a particular application or not specifically relevant. Those skilled in the art will also be aware of the W3C Xpointer proposal, however further details may be found from the W3C website which is presently located at www.w3c.org. XPointer is based on the XML Path Language (XPath). Through the selection of various properties, such as element types, attribute values, character content, and relative position, XPointer supports addressing within internal structures of XML documents and allows for traversals of a document tree. Thus, in place of structural references to data, the framework may provide that explicit element ID values are used for all pointer references thereby avoiding specific references to structural paths and data content. As a result, a framework according to the present invention ensures the maximal validity of pointer values to all realizations of a given media object, irrespective of language, coverage, encoding, or partitioning. In addition to the Xpointer standard proposal, other alternative/additional internal pointer mechanisms for other encodings may be utilized.

In addition to the above-described architecture, a framework according to the present invention provides rules that relate to the inheritance and versioning of the scoped definitions. Thus, the framework provides that metadata defined at higher scopes is inherited by lower scopes by ensuring that two rules are applied. Firstly, all metadata properties defined in higher scopes are fully visible, applicable, and meaningful in all lower scopes, without exception. Secondly, any property defined in a lower scope completely supplants any definition of the same property that might exist in a higher scope. Consequently, all metadata properties defined for a media object 41 may be inherited by all instances 39 of that object; and all metadata properties defined for a media instance 39 or media object 41 may be inherited by all of its components 37.

In relation to versioning, MARS 25 defines a versioning model using two levels of distinction. A first level is defined as a release, namely a published version of a media instance that is maintained and/or distributed in parallel to other releases. By way of example, a release could be viewed as a branch in a prior art tree based versioning model. A second level is defined as a revision corresponding to a milestone in the editorial lifecycle of a given release; or by way of example, a node on a branch of the prior art tree based model. MARS 25 defines and maintains versioning for ‘data’ storage item 35, only.

In addition to the Identity architecture described above, MARS 25 provides a management architecture that permits control of processes such as retrieval, storage, and version management. Details of the properties defined to provide such functionality might be found in the MARS section following. MARS 25 also provides affiliation properties that define an organizational environment or scope where data is corrected and maintained. Examples of such properties can also be found in the MARS section following.

MARS 25 further provides content properties that allow definition of data characteristics independent of the production, application or realization of that Data. Again, examples of such properties can be found in the MARS section following. MARS 25 also provides encoding properties defining special qualities relating to the format, structure or general serialization of data streams. These properties are, of course, of significance to tools and processes operating on that data. Yet again, examples of such properties can be found in the MARS section following. MARS 25 also provides association properties that define relationships relating to the origin, scope or focus of the content in relation to other data. Examples of such properties may be found in the MARS section following. Finally, MARS 25 provides role properties that specify one or more actors who have a relationship with the data. An actor may be a real user or a software application such as an agent. Examples of such properties may be found in the MARS section following.

As has been previously mentioned, a Generalized Media Archive (GMA) 27, based on Media Attribution and Reference Semantics (MARS) 25 metadata provides a uniform, consistent, and implementation independent model for the storage, retrieval, versioning, and access control of electronic media. Further details of the GMA may be found in the GMA section following. The GMA 27 and serves as the common archival model for all managed media objects controlled, accessed, transferred or otherwise manipulated by agencies operating with a framework according to the present invention. Hence, the GMA 27 may serve as a functional interface to wide range of archive implementations whilst remaining independent of operating system, file system, repository organization, versioning, mechanisms, or other implementation details. This abstraction facilitates the creation of tools, processes, and methodologies based on this generic model and interface which are insulated from the internals of the GMA 27 compliant repositories with which they interact.

The GMA 27 defines specific behavior for basic storage and retrieval, access control based on user identity, versioning, automated generation of variant instances, and event processing. The identity of individual storage items 35 is based on MARS metadata semantics and all interaction between a client and a GMA implementation must be expressed as MARS 25 metadata property sets.

The GMA manages media objects 41 via media components 37 and is made up of storage items 35. The GMA manages the operations of versioning, storage, retrieval, access control, generation and events as will be further described below. Examples of pseudo code corresponding to the above and other managed operations carried out by the GMA may be found in the GMA section following.

The GMA 27 operates on the basis of MARS 25 metadata and as a result of its operation the GMA 27 acts on that same metadata. The metadata operated on by the GMA 27 may be restricted to management metadata rather than content metadata. The former being metadata concerned with the history of the physical data, such as retrieval and modification history, creation history, modification and revision status, whereas the latter is concerned with the qualities and characteristics of the information content as a whole, independent of its management. Content metadata is stored as a separate ‘meta’ component 37, not a ‘meta’ item 35, such that the actual specification of the content metadata is managed by the GMA 27 just as any other media component 37. The metadata that is of primary concern to a GMA 27, and which a GMA accesses, updates, and stores persistently, is the metadata associated with each component 37.

A GMA 27 manages media components 37, and the management metadata for each media component 37 is stored persistently in the ‘meta’ storage item of the media component 37. A special case exists with regards to management metadata which might be defined at the media instance 39 or media object 41 scope, where that metadata is inherited by all sub-components 37 of the higher scope(s) in accordance with the inheritance rules set out above.

In order to provide the necessary functionality, the GMA 27 requires that the certain metadata properties are defined in an input query and/or in respect of any target data depending on the action being performed and which functional units are implemented. These properties are set out in the GMA section, Section 4.1.2-4, following. In accordance with inheritance rules defined in MARS 25, retrieval of metadata for a given media component scope includes all inherited metadata from media object and media instance scopes. In addition, the GMA 27 will assume the default values as defined by the MARS 25 specification for all properties which it requires but that are not specified explicitly. It is an error for a required property to have neither a default MARS 25 value nor an explicitly specified value. In addition to relying on existing metadata definitions, the GMA 27 is responsible for defining, updating, and maintaining the management metadata relevant for the ‘data’ item 35 of each media component 37, which is stored persistently as the ‘meta’ item 35 of the component 37.

The GMA 27 stores ‘meta’ item 35, containing management metadata, in any internal format; however the GMA must accept and return ‘meta’ storage items as XML (eXtensible Mark-up Language) instances. However, content metadata constituting the data content of a ‘meta’ component 37 and stored as the ‘data’ item 35 of the ‘meta’ component 37, must always be a valid XML instance.

These two constraints ensure that an agent interacting with the GMA 27 is able to retrieve from or store to the GMA 27 both content and management metadata as needed. The GMA 27 is also able, as a consequence of these constraints to resolve inherited management metadata from meta components at higher scopes in a generic fashion.

In order to store and retrieve items, the GMA 27 associates electronic media data streams to MARS 25 storage item identities and makes persistent, retrievable copies of those data streams indexed by their MARS 25 identity. The GMA 27 also manages the corresponding creation and modification of time stamps in relation to those items. The GMA 27 organizes both the repository 21 of storage items 35 as well as the mapping mechanisms relating MARS identity metadata to locations within that repository 21. The GMA 27 may be implemented in any particular technology including, but not limited to common relational or object oriented database technology, direct file system storage, or any number of custom and/or proprietary technologies.

In addition to the core storage and retrieval actions provided by the GMA 27, the GMA 27 is capable of providing the functionality necessary to permit operations by agents in relation to versioning, access control, generation, and/or events. The GMA 27 will exhibit a pre-defined behavior, to the extent that such functionality is provided by it.

Thus, if the GMA 27 implements access control, then access control of media 15 components 37 is based on several controlling criteria as defined for the environment in which the GMA resides and as stored in the metadata of individual components managed by the GMA. Access control is defined for entire components and not for individual items within a component. Access control may also be defined for media objects 41 and media instances 39, in which case subordinate media components 37 inherit the access configuration from the higher scope(s) in the case that it is not defined specifically for the component. The four controlling criteria for media access are User identity, Group membership(s) of user, Read permission for user or group and Write permission for user or group.

Accordingly, every user must have a unique identifier within the environment in which the GMA operates, and the permissions must be defined according to the set of all users and groups within that environment.

A user may be a human, but also can be a software application, process, or system typically referred to as an agent 13. This is especially important for both licensing as well as tracking operations performed on data by automated software agents 13 operating within the GMA 27 environment. Furthermore, any user may belong to one or more groups, and permissions may be defined for an entire group, and thus for every member of that group. Consequently, the maintenance overhead in environments with large numbers of users and/or high user turnover many users coming and going is reduced. In a manner similar to the inheritance rules applied by MARS 25, permissions defined for explicit user override permissions defined for a group of which the user is a member. For example, if a group is allowed write permission to a component 37, but a particular user is explicitly denied write permission for that component 37, then the user may not modify the component 37.

The GMA 27 may also provide read permission such that a user or group may retrieve a copy of the data. Where a lock marker is placed in relation to data, it does not prohibit retrieval of data, merely modification of that data. If access control is not implemented, and/or unless otherwise specified globally for the GMA 27 environment or for a particular archive, or explicitly defined in the metadata for any relevant scope, a GMA 27 must assume that all users have read permission to all content.

Similarly, the GMA 27 may also provide Write permission that means that the user or group may modify the data by storing a new version thereof. The GMA 27 provides that write permission equates to read permission such that every user or group which has write permission to particular content also has read permission. This overrides the situation where the user or group is otherwise explicitly denied read permission.

As in the case of read permission, the presence of a lock marker prohibits modification by any user other than the owner of the lock, including the owner of the component 32 if the lock owner and component owner are different. Optionally, the GMA 27 provides a means to defeat locking as a reserved action unavailable to general users. Should locking be defeated in this manner then the GMA 27 logs the event and notifies the lock owner accordingly.

Where access control is not implemented, then the GMA 27 applies the rule that all users have write permission to all content. If access control is implemented, and unless otherwise specified globally for the GMA 27 environment or for a particular archive or explicitly defined in the metadata for any relevant scope, the GMA 27 must assume that no users have write permission to any content. Regardless of any other metadata defined access specifications not including settings defined globally for the archive, the owner of a component 37 always has write access to that component 32.

In addition to blanket access control, the GMA 27 may, if access control is enabled provide a set of access levels which serve as convenience terms when defining, specifying, or discussing the "functional mode" of a particular GMA 27 with regard to read and write access control.

Access levels can be used as configuration values by GMA 27 implementations to specify global access behavior for a given GMA 27 where the implementation is capable of providing multiple access levels. At each level the read and write capability may be predefined subject to the overriding rule that a read right may never fall below the corresponding write right.

The GMA 27 may implement versioning. Through the implementation of versioning, the GMA 27 facilitates the identification, preservation, and retrieval of particular revisions in the editorial lifecycle of a particular discrete body of 30 data.

The versioning model used by the GMA 27 and further description in the GMA section, section 4.5 following, in particular defines a release as a series of separately managed and independently accessible sequences of revisions. Revisions are defined as ‘snapshots’ along a particular release. Where a release is derived from another release then the GMA 27 updates a MARS 25 source property to identify from what release and revision the new release stems. Within the above rules, the GMA 27 is responsible for linear sequence of revisions within a particular release. The GMA 27 is responsive to external agent 13 activities that are themselves responsible for the automated or semi-automated creation or specification of new instances 39 relating to distinct releases. The GMA is also responsive to agent 13 activities relating to the retrieval of revisions not unique to a particular release. Typically, a human editor manually performs the creation of new releases, including the specification of ‘source’ and any other relevant metadata values. Other tools, external to the GMA 27 may also exist to aid users in performing such operations.

A GMA 27 performs versioning for the ‘data’ item 35 of a media component 37 only and that sequence of revisions constitutes the editorial history of the data content of the media component 37. The GMA 27 is also responsible for general management and updating of creation, modification and other time stamp metadata. Storage or update of items other than the ‘data’ item 35 neither effect the status of management metadata stored in the ‘meta’ item 35 of the component 37 unless the item 35 in question is in fact the ‘meta’ 35 item of the component 37, nor are reflected in the revision history of the component 37. If a revision history or particular metadata must be maintained for any MARS 25 identifiable body of content, then that content must be identified and managed as a separate media component 37, possibly belonging to a separate media instance 39.

Revisions are identified by positive integer values utilizing MARS 25 property type Count values. The scope of each media component 37 is unique and revision values have significance only within the scope of each particular media component 32. Revision sequences should begin with the value ‘1’ and proceed linearly and sequentially. The GMA 27 implementation is free to internally organize and store past revisions in any fashion it chooses.

The GMA 27 may implement one or both of the following described methods for storing past revisions of the content of a media component. However, regardless of its internal organization and operations, the GMA 27 must return any requested revision as a complete copy.

One method that the GMA 27 may employ to store past revisions is to generate snapshots. A snapshot is a complete copy of a given revision at a particular point in time. As such snapshotting is straightforward to implement, and possibly time consuming regeneration operations are not needed to retrieve past revisions. The latter can be very important in an environment where there is heavy usage and retrieval times are a concern.

Alternatively or in conjunction with snapshots, the GMA 27 may store past revisions through a reverse delta methodology. A delta is set of one or more editorial operations that can be applied to a body of data to consistently derive another body of data. A reverse delta is a delta that allows one to derive a previous revision from a former revision. Rather than store the complete and total content of each revision, the GMA 27 stores the modifications necessary to derive each past revision from the immediately succeeding later revision. To obtain a specific past revision, the GMA 27 begins at the current revision, and then applies the reverse deltas in sequence for each previous revision until the desired revision is reached.

In a variant of the above, the GMA 27 utilizes a forward delta methodology where each delta defines the operations needed to derive the more recent revision from the preceding revision.

The GMA 27 may also implement generation through the dynamically creating data streams from one or more existing storage items 35. By way of example, this includes conversions from one encoding or format to another, extraction of portions of a component's content, auto-generation of indices, tables of contents, bibliographies, glossaries, and the like as new components 37 of a media instance 39, generation of usage, history, and/or dependency reports based on metadata values, generation of metadata profiles for use by one or more registry services.

The GMA 27 also provides dynamic partitioning whereby a fragment of the data content is returned in place of the entire ‘data’ item, optionally including automatically generated hypertext links to preceding and succeeding content, and/or information about the structural/contextual qualities of the omitted content, depending on the media encoding. The GMA 27 may implement dynamic partitioning irrespective of whether static fragments exist. Dynamic partitioning is controlled by one or possibly two metadata properties, in addition to those defining the identity of the source data item. The required property is size that determines the maximum number of bytes which the fragment can contain starting at the beginning of the data item. Whereas the second and optional property is pointer that defines the point within the data item from which the fragment is extracted. Thus, the GMA 27 extracts the requested fragment, starting either at the beginning of the data item, where no pointer is defined or at the point specified by the pointer value that may be at the start of the data item if the pointer value is zero. The GMA 27 collects the largest coherent and meaningful sequence of content up to but not exceeding the specified number of content bytes. What constitutes a coherent and meaningful sequence will depend on the media encoding of the data and possibly interpretations inherent in the GMA 27 implementation itself.

A GMA 27 may implement event handling. Accordingly, for each storage item, media component 37, media instance 39, or media object 41, a set of one or more MARS 25 property sets defining some operation(s) can be associated with each MARS 25 action, such that when that action is successfully performed on that item 35, component 37, instance 41, or object, the associated operations are executed. Automated operations are thus defined for the source data and not for any target data that might be automatically generated as a result of an event triggered operation. Each operation property set must specify the necessary metadata properties to be executed correctly, such as the action(s) to perform and possibly including the CGI URL of the agency that is to perform the action. The GMA 27 determines how a given operation is to be performed, and by which software component or agent 13 if otherwise unspecified in the property set(s).

In the case of a remove action, which will result in the removal of any events defined at the same scope as the removed data, the GMA 27 will execute any operations associated with the remove action defined at that scope, after successful removal of the data, even though the operations themselves are part of the data removed and will never be executed again in that context.

The most common type of operation for events is a compound ‘generate store’ action which generates a new target item from an input item and stores it persistently in the GMA 27, taking into account all versioning and access controls in force. By this operation, it is possible to automatically update components such as the toc (Table of Contents) or index when a data component 37 is modified, or generate static fragments of an updated data component 37.

The GMA 27 may associate automated operations globally for any given action provided the automated operations are defined in terms of MARS 25 property sets. Automated operation may also be applied within the scope of the data being acted upon. The GMA 25 may also associate automated operations with triggers other than MARS 25 actions, such as reoccurring times or days of the week, for the purpose of removing expired data such as via a ‘locate remove’ compound action.

The GMA 27 must also apply the following rules relating to the serialization and encoding of certain storage items. Thus, the GMA 27 provides that every ‘meta’ storage item that is presented to a GMA 27 for storage or returned by a GMA 27 on retrieval must be a valid XML instance. Metadata property values "contained" within ‘meta’ storage items 35 need not be stored or managed internally in the GMA 27 using XML, but every GMA 27 implementation must accept and return ‘meta’ items as valid XML instances. In the case of ‘data’ Storage Items 35 within ‘meta’ Media Components 37, the serialization of ‘meta’ storage items 35 is also used to encode all ‘data’ storage items 35 for all ‘meta’ components 37. Although the GMA 27 persistently stores all ‘data’ storage items 35 literally, it may also choose to parse and extract a copy of the metadata property values defined within meta component data items to more efficiently determine inherited metadata properties at specific scopes within the archive 27.

Every ‘idmap’ storage item which is presented to a GMA 27 for storage or returned by a GMA 27 on retrieval should be encoded as a Comma Separated Value (CSV) data stream defining a table with two columns where each row is a single mapping and where the first column/field contains the value of the ‘pointer’ property defining the symbolic reference and the second column/field contains the value of the ‘fragment’ property specifying the data content fragment containing the target of the reference, for example: