Method and apparatus for device interaction by format

Abstract

A method of passing information between two or more information handling devices is described. Such information handling devices might be a printer 342, a personal computer 343, or a scanner 344. Means for communication of information between information handling devices in the form of a network 341 and network connection means 345 exist. The information transmitted comprises a data format hierarchy, wherein a device intended to receive transmitted data evaluates the data format hierarchy and determines the format in which the data is then received thereby. Advantageously, the receiving device determines the format in which the data is then received by a response to the transmitting device comprising a path through the data format hierarchy, and all data formats comprise one or more of a plurality of data format types, and wherein for each data format type, there exists a data format receivable by all information handling devices supporting that data format type. It is advantageous in implementation if the method of passing information further comprises requests for content data for a chosen path through the data format hierarchy and responses to such requests.

Claims

What is claimed is:

1. A method of passing information between two or more information handling devices, there being a means for communication of information between information handling devices, wherein said information transmitted comprises a data format hierarchy and determines the format in which the data is then received thereby, and wherein a receiving device determines the format in which the data is then received by a response to a transmitting device comprising a path through the data format hierarchy.

2. A method as claimed in claim 1, wherein all data formats comprise one or more of a plurality of data format types, and wherein for each data format type, there exists a data format receivable by all information handling devices supporting that data format type.

3. A method as claimed in claim 1, wherein said data format hierarchy is comprised in a description of a surface, wherein the surface is a representation of an internal state of one of the information handling devices.

4. A method as claimed in claim 1, wherein transmitted together with said data format hierarchy is content data for one or more of the choices available in said data format hierarchy.

5. A method as claimed in claim 1, wherein all the data format hierarchy is not contained in a single message.

6. A method as claimed in claim 5, wherein the method of passing information further comprises requests for additional parts of the data format hierarchy and responses to such requests.

7. A method as claimed in claim 1, wherein a data format hierarchy comprises a hierarchical structure, a set of keywords relating to properties of information transmitted and values associated with each of said keywords.

8. A method as claimed in claim 7, wherein each data format hierarchy comprises a single encoding, where each encoding represents a different basic form in which information can be presented.

9. A method as claimed in claim 8, wherein one choice of encoding is an image.

10. A method as claimed in claim 8, wherein one choice of encoding is as text.

11. A method as claimed in claim 8, wherein one choice of encoding is as a binary file.

12. A method as claimed in claim 8, wherein one choice of encoding is as a plane, wherein a surface having a plane encoding may have child surfaces comprising information for rendering on that plane.

13. A method as claimed in claim 12, wherein the data format hierarchy contains information determining the ordering of child surface information on the plane.

14. A method as claimed in claim 12, wherein said plane possesses two sides and information may be rendered on either side of the plane.

15. A method as claimed in claim 14, wherein the plane possesses variable opacity, said opacity determining the degree to which information on one side of the plane can be perceived from the other side of the plane.

16. A method as claimed in claim 12, wherein the data format hierarchy determines whether the position of child surfaces in a plane or association encoding is controlled by the device providing the data format hierarchy or may be modified by the device receiving the data format hierarchy.

17. A method as claimed in claim 8, wherein one choice of encoding is an association of child surfaces each comprising information to be processed together.

18. A method as claimed in claim 17, wherein plane or association encodings may be nested within other plane or association encodings.

19. A method as claimed in claim 7, wherein the data format hierarchy contains reference to child surfaces, the order in which the child surfaces are provided may be chosen to be fixed or determinable by the device receiving the data format hierarchy.

20. An information handling device adapted to send information to or receive information from another information handling device according to the method claimed in claim 1.

21. A method of passing information between two or more information handling devices, there being a means for communication of information between handling devices,

wherein said information transmitted comprises a data format hierarchy, wherein a device intended to receive transmitted data evaluates the data format hierarchy and determines the format in which the data is then received thereby and wherein the method of passing information further comprises requests for content data for a chosen data path through the data format hierarchy and responses to such requests.

22. A method of passing information to an information processing appliance in a data format suitable for processing by the information processing appliance, comprising:

sending a tree structure of data format choices to the information processing appliance;

receiving from the information processing appliance a selected data format choice; and

providing the information in the data format of the selected data format choice to the information processing appliance.

23. The method as claimed in claim 22, wherein said tree structure further comprises at each node thereof a set of keywords relating to properties of information transmitted and values associated with each of said keywords.

24. The method as claimed in claim 23, wherein the tree structure does not contain all available data format choices, and wherein the method comprises after sending the tree structure of data format choices;

receiving from the information processing appliance a request for further data format choices; and

sending a further tree structure of further data format choices to the information processing appliance;

wherein the selected data format choice is one of said further data format choices.

25. The method as claimed in claim 22, wherein the tree structure contains a node which defines a basic data type of the information to be provided.

26. The method as claimed in claim 25, wherein the basic data type is an image.

27. The method as claimed in claim 25, wherein the basic data type is text.

28. The method as claimed in claim 25, wherein the basic data type is an assemblage of discrete elements of information, wherein each discrete element of information has its own data format.

29. The method as claimed in claim 28, wherein the order in which the discrete elements of information are provided may be chosen to be fixed or determinable by the information processing device.

30. The method as claimed in claim 28, wherein one of said discrete elements of information is itself an assemblage of discrete elements of information.

31. The method as claimed in claim 28, wherein the basic data type is an association of said discrete elements of information.

32. The method as claimed in claim 28, wherein the basic data type is a page, wherein said discrete elements of information are to be rendered on the page.

33. The method as claimed in claim 32, wherein said page possesses two sides and information may be rendered on either side of the page.

34. The method as claimed in claim 33, wherein the page possesses variable opacity, said opacity determining the degree to which information rendered on one side of the page can be perceived from the other side of the page.

35. The method as claimed in claim 32, wherein the tree structure of data format choices contains information determining the ordering of said discrete elements of information on the page.

36. The method as claimed in claim 32, wherein the tree structure of data format choices determines whether the position of discrete elements of information on a page may be modified by the information processing device.

37. A method as claimed in claim 22, wherein the data format choices each represent a format for display or rendering of data by the information processing appliance.

38. The method as claimed in claim 37, wherein one of said data format choices in the tree structure is a default data format adapted to be displayable or renderable by all information processing appliances adapted to select a data format choice.

39. A method of passing information to an information processing appliance in a data format suitable for processing by the information processing appliance, comprising:

sending a tree structure of data format choices to the information processing appliance, wherein at least one of the data format choices is represented by information provided in an associated data format;

receiving from the information processing appliance a selected data format choice; and

providing the information in the data format of the selected data format choice to the information processing appliance.

40. The method as claimed in claim 39, wherein said tree structure further comprises at each node thereof either a set of keywords relating to properties of information transmitted and values associated with each of said keywords or an identification of a data format and the information provided in said data format.

41. The method as claimed in claim 39, wherein the data format choices each represent a format for display or rendering of data by the information processing appliance.

42. A method by which an information processing appliance receives information from an information source, the method comprising:

receiving a tree structure of data format choices and selecting a data format choice suitable for processing by the information processing appliance;

sending the selected data format choice to the information source; and

receiving the information in the selected data format from the information source.

43. The method as claimed in claim 42, wherein the tree structure does not contain all available data format choices, and wherein the method comprises after receiving the tree structure of data format choices;

determining that further data format choices are required, and sending a request for further data format choices to the information source; and

receiving a further tree structure of further data format choices to the information processing appliance;

wherein the selected data format choice is one of said further data format choices.

44. The method as claimed in claim 42, wherein the data format choices each represent a format for display or rendering of data by the information processing appliance.

45. An information processing appliance adapted to process information in one or more data formats, the appliance having a communications interface and a processor programmed to select a data format choice suitable for processing by the information processing appliance from a tree structure of data format choices received from an information source through the communications interface, the processor being further programmed to request the selected data format choice from the information source wherein the information processing appliance is adapted to process the information when received in the selected data format.

46. The information processing appliance as claimed in claim 45, wherein the information processing appliance is a printer, and wherein the information processing appliance processes the information by rendering it for printing.

47. The information processing appliance as claimed in claim 45, wherein the information processing appliance comprises a display apparatus, and wherein the information processing appliance processes the information by rendering it for display.

48. The method as claimed in claim 1, wherein said data format hierarchy comprises status information.

49. The method as claimed in claim 1, wherein each data format hierarchy comprises a single encoding, where each encoding represents a different basic form in which information can be presented.

50. The method as claimed in claim 49, wherein one choice of encoding is an image.

51. The method as claimed in claim 49, wherein one choice of encoding is as text.

52. The method as claimed in claim 49, wherein one choice of encoding is as a binary file.

53. The method as claimed in claim 49, wherein one choice of encoding is as a two-dimensional planar object, wherein the two-dimensional planar encoding may have children comprising information for rendering on the two-dimensional planar object.

54. The method as claimed in claim 53, wherein said two-dimensional planar object possesses two sides and information may be rendered on either side of the two-dimensional planar object.

55. The method as claimed in claim 53, wherein said two-dimensional planar object possesses variable opacity, said opacity determining the degree to which information on one side of the two-dimensional planar object can be perceived from the other side of the two-dimensional planar object.

56. The method as claimed in claim 53, wherein where a data format hierarchy determines whether the position of children in a two-dimensional planar encoding is controlled by the device providing the data format hierarchy or may be modified by the device receiving the data format hierarchy.

57. The method of claim 1, wherein where a data format hierarchy contains reference to children, the order in which the children are provided may be chosen to be fixed or determinable by the device receiving the data format hierarchy.

58. The method as claimed in claim 49, wherein one choice of encoding is an association of children, each child comprising information to be processed with information from the other children in the association.

59. The method as claimed in claim 58, wherein two-dimensional planar encodings or association encodings may be nested within other two-dimensional planar encodings or association encodings.

60. The method as claimed in claim 58, wherein where a data format hierarchy determines whether the position of children in an association encoding is controlled by the device providing the data format hierarchy or may be modified by the device receiving the data format hierarchy.

Description

The present invention relates to methods and apparatus for device interaction, in particular for passing of information between devices.

The requirement for efficient and successful passage of information between devices is pervasive. For communication between electronic appliances, such as a scanner and a printer, a storage device and a viewing device, or a camcorder and a television set, an application specific protocol is required. In the first two cases, it will also be normal that a computing device such as a personal computer mediates communication. A personal computer requires device specific drivers for each electronic appliance (normally those considered as "peripherals" for a personal computer) to allow this communication to occur.

In the physical world, interaction with physical devices does not require special protocols. A small set of generic instructions are used, and the effect of these interactions is determined by the device interacted with. For example, the interaction of a user with a telephone and a photocopier is (typically) essentially the same: the pressing of a series of buttons. In the telephone, this results in the establishment of a connection with a remote telephone and an open communication channel, whereas in the photocopier the result is the reproduction of a number of images. In each case, the fundamental interaction (pressing buttons) is the same, but the result is very different.

The present invention seeks to achieve the passing of information between devices with the same ease and effectiveness as is achieved in the physical world. In particular, the present invention seeks to enable rich and effective communication to be achieved with generic instructions.

Accordingly, the invention provides a method of passing information between two or more information handling devices, there being a means for communication of information between information handling devices, wherein said information transmitted comprises a data format hierarchy, wherein a device intended to receive transmitted data evaluates the data format hierarchy and determines the format in which the data is then received thereby.

Advantageously, the receiving device determines the format in which the data is then received by a response to the transmitting device comprising a path through the data format hierarchy, and all data formats comprise one or more of a plurality of data format types, and wherein for each data format type, there exists a data format receivable by all information handling devices supporting that data format type. It is advantageous in implementation if the method of passing information further comprises requests for content data for a chosen path through the data format hierarchy and responses to such requests.

A particularly advantageous context for implementation of the invention is where the data format hierarchy is comprised in a description of a surface, wherein the surface is a representation of an internal state of one of the information handling devices.

A particular effective further feature is for each data format hierarchy to comprises single encoding, where each encoding represents a different basic form in which information can be presented Such encodings may be image, text, binary file, plane or association. Preferably, such a plane possesses two sides and information may be rendered on either side of the plane. Both plane and association encodings may advantageously possess child surfaces (indicating further encodings to be rendered on the plane, or to be processed together, respectively).

Application of the invention enables device independent information exchange. Fully rich information exchange can be achieved with the communication medium provided by the invention. The medium can in embodiments be exploited in a transport independent, device independent manner, and the ability to communicate through this medium can be built into any device which has a potential need to connect to other devices regardless of its function. This could extend from obvious communication and rendering appliances (such as fax machines, printers, whiteboards) through to domestic appliances such as thermostats, washing machines and toasters. In preferred embodiments, the same medium is used for communication between any appliance, and it is guaranteed that devices able to handle the same type of information can pass information of some form.

Appropriate use of the invention allows guaranteed exchange of information with existing and future devices. It is also possible for devices to communicate even if they have no knowledge of each other--no element of preprogramming is required, as is the case for host-peripheral environments. Further advantages are that system components can be developed separately (for example, a printer can be developed without consideration of creation of drivers for a personal computer). Moreover, preferred implementations of the invention are such that product features are defined by the developer, not the protocol: the protocol is merely a pipe through which information is exchanged and has no knowledge of devices or implementations.

Specific embodiments of the invention will be described below, by way of example, with reference to the accompanying drawings, of which:

FIG. 1 shows the creation of a surface impression on one device from the surface expression of another;

FIG. 2 shows a depiction of a two-page document in a visual and a surface representation;

FIG. 3 shows the components of the JetSend architecture and their logical relationship to each other;

FIG. 4 shows a transcript of messages exchanged by two devices when they establish a session;

FIG. 5 shows a schematic depiction of a session between two JetSend devices;

FIG. 6 shows a schematic illustration of the states in a JetSend Session Protocol session;

FIG. 7 shows a schematic illustration of starting a JetSend Session Protocol session;

FIG. 8 shows a schematic illustration of the opening and closing of a channel in JetSend Session Protocol;

FIG. 9 shows a schematic illustration of partial message fragmentation in JetSend Session Protocol;

FIG. 10 shows user of a timer in scheduling gateway broadcast messages using JetSend Session Protocol;

FIG. 11 shows the generic format of a JetSend Session Protocol Message Header;

FIGS. 12a to 12k show the use of messages of the JetSend Interaction Protocol in exchange of information between appliances;

FIG. 13 shows content provision on a message channel;

FIG. 14 shows content provision on a stream channel;

FIG. 15 shows a graphical representation of a self-surface as an example of a well-known surface;

FIG. 16 shows a sample document transmission according to the job policy;

FIG. 17 shows a graphical representation of a well-known surface;

FIG. 18 shows the structure of a generic self-surface;

FIGS. 19a to 19e show graphical representations of a self-surface and its sub-surfaces;

FIG. 20 shows the structure of a generic status-surface;

FIGS. 21a to 21e show graphical representations of a status-surface and its sub-surfaces;

FIG. 22a shows a graphical representation of an address in-surface; and

FIG. 22b shows the structure of an address card surface interacting with an address in-surface;

FIG. 23 shows a hierarchy of attributes for the vImage encoding;

FIG. 24 shows a hierarchy of attributes for the vText encoding;

FIG. 25 shows a hierarchy of attributes for the vFile encoding;

FIG. 26 shows a hierarchy of attributes for the vPlane encoding;

FIG. 27 illustrates schematically the assembly of elements for a vPlane encoding;

FIGS. 28a to 28d illustrate derivation of values for vBackCoord in the vPlane encoding;

FIG. 29 illustrates an example of a document for encoding with vPlane incorporating four child elements;

FIG. 30 shows a hierarchy of attributes for the vAssociation encoding;

FIG. 31 shows a schematic diagram for a generic e-material block;

FIG. 32 shows a pictorial representation of a specific e-material block;

FIG. 33 shows a schematic representation of a surface description encoding hierarchy; and

FIG. 34 illustrates communication between devices to which the present invention is applicable.

The present invention is embodied in the JetSend architecture of Hewlett-Packard Company. An introduction to this architecture is provided on the World Wide Web at http://www.jetsend.hp.com/, and a full description is provided in the HP JetSend Communications Technology Protocol Specification, available from Hewlett-Packard Company and incorporated herein by reference.

The basic elements of the JetSend architecture will be discussed below. The individual elements will then be discussed individually in greater detail to the extent necessary to illustrate the invention and advantageous features thereof. Reference to preferred and required features below relates to preferred and required features of the JetSend architecture, rather than to preferred and required feature of the invention itself.

Fundamental to the JetSend architecture are four principles which define how the system operates. These are as follows:

Peer to peer interaction--When network transport allows it, devices must be able to address other devices directly. Devices should not require the presence of a PC or any other intermediary device to enable an information exchange between two non-PC devices. Users can thus connect with a minimum configuration and can perform ad hoc information transfers.

Device and Device Type Independence--No device specific information about other devices should be preprogrammed into any device. A device should not be required to behave differently for a specific device or set of devices it may encounter. In the JetSend architecture, a device can perform generic operations, such as sending information or observing status, which do not in themselves have any device specific aspect. This allows any two JetSend enabled devices to interact without using drivers that are specific to a device or device type.

Negotiation--Devices are to negotiate data encodings to the highest common denominator that the sender and receiver support. There must be a default encoding for each data type that all senders and receivers using that data type must support. The ability to negotiate ensures that devices can implement high end and even proprietary data types, while the default encoding ensures that meaningful data exchange will always take place. Negotiation should be flexible and open ended.

Consistency--The same protocol is used regardless of whether the devices are exchanging control, transferring data, exchanging status information or passing any other form of information.

FIG. 34 illustrates the environment in which JetSend devices operate. A network 341 of some kind exists between devices such as a printer 342, a personal computer 343, and a scanner 344. Each of these devices must possess a processor 346 of some kind, together of course with a connection means 345 to allow interface with the network 341. It is necessary in this implementation for each of the devices to have some measure of processing capacity, as such capacity is necessary for processes integral to JetSend.

The basis for communication between devices in JetSend is the surface interaction model. A surface is a representation of some aspect of internal state possessed by a device. The representation is universal, and is not determined by the functions performed by the device. A surface in this context is analagous to the surface provided by a physical object (such as a telephone, or a brick). The way in which the object operates is determined by how the "functional" parts of the object connect to the surface, but the surface itself can be described in a simple universal way, irrespective of function, and provides the medium through which the object is connectable to other physical objects in the world--the nature of the connection between physical objects being independent of any device function. In JetSend, the surface is the fundamental unit of information exchange, and images, documents, status messages, device labels and the like are all transferred through use of one or more surfaces. A surface consists of a number of elements: description, content, properties and class--these will be discussed further below.

The surface interaction model defines mechanisms for creating, sharing, modifying and deleting surfaces. Only a fixed number of generic operations can be performed on a surface, using the messages defined by the JetSend Interaction Protocol (JIP), which will be discussed further below.

The original copy of a surface is here termed an expression. There is one expression involved in any surface interaction. Devices that have information to share with other devices create one or more expressions to represent that information.

Surface expressions can be impressed on other devices. This creates an impression of that surface--also known as sharing the surface. Impressions are copies of other device's expressions, and connections are maintained between the devices to ensure that impressions are up-to-date with their corresponding expression. Typically, two devices will share several surfaces at a given time; surfaces that represent the elements of a job, status information from each device, security information, and so forth.

FIG. 1 shows how an expression 11 on one device (Device A) is shared with another device (Device B), thereby creating a new impression 12. The same expression can be impressed multiple times on multiple devices. This creates multiple impressions of the same surface expression. When the expression changes, or is deleted, all impressions will be notified. This is here termed change propagation.

Change propagation enables interesting dynamic features, such as status information, to be implemented for devices in the architecture. For example, a device with status information creates a surface expression containing a status string which is impressed on a number of connected devices. The device with the status changes the surface depending on its state. All the JetSend devices that have an impression of the status surface will receive update messages with the current status. This functionality is built into the architecture and its use by device designers is optional.

From an implementation standpoint, an expression is defined by the state that is kept about a given surface (its name, its description, its content and the set of impressions that have been made on remote devices). It will be shown later that from a protocol standpoint, an expression is defined by the surface handle used to exchange JIP messages about it.

A surface comprises a description of the surface, and the content contained within the surface. The distinction is fundamental to JetSend and of significance in aspects of the present invention, because it provides a mechanism for content negotiation.

The surface description establishes the full range of possible forms in which information associated with a surface (this information being provided as surface content data) can be shared with another device. The description consists of a data format hierarchy, and can be represented as a tree structure with a number of nodes each representing a definable aspect of the data format (referred to as an attribute, and discussed in greater detail elsewhere). A specific data format for the information associated with the surface is a path through this tree structure from a root node down to a terminal node (a leaf node of the tree). Such specific paths are reached through a process of negotiation. The process of negotiation, in this context, comprises the surface owner sharing the surface description, or a part of the surface description, with another device. The other device then chooses the options it prefers (generally such options will be selected so that the receiving device can use its own functionality in the richest manner available given the choices offered, but this is essentially a matter for the designer of any given device) and this process continues until a specific path, and thus a specific data format, is chosen.

The surface content is the information associated with the surface, and is provided in a format determined by negotiation as indicated above. The content data can be provided in all the formats indicated by the data format hierarchy embodied in the surface description, but for a given surface interaction will generally only be provided in one of the formats available. There does exist the possibility of providing information in more than one format for a single impression: a case where this may prove useful is in retrieval of information from a storage device, where initially, for example, an image may be provided initially as a thumbnail at low resolution, and later, after confirmation that the image is desired, provided at full size and high resolution without change of impression. Request for further content would be normal in such a situtation, however. Content data need not exist prior to completion of the negotiation: for some or all of the choices in the data format hierarchy the data may only be generated after the format choice is known. Alternatively, for some choices the content data may be provided directly together with the description identifying a particular point in the data format hierarchy as a part of the negotiation process (so no separate step of provision of content is required for that format choice). This is termed providing content data in-line.

As indicated above, descriptions are sent along with the original surface impression, but in the general case content is requested separately. For example, a surface representing a raster image might define a surface description with choices about color spaces, resolutions, and compressions for the image. The receiver of the impression will request the actual pixel data (the content), specifying a preferred encoding for the receiver from the choices available.

A surface description can also contain references to other surfaces. These surfaces are known as sub-surfaces (or child surfaces) and they must be requested separately from the owner of the parent surface. Each sub-surface contain its own description and content, independent of its parent. A document for example, can be encoded as multiple surfaces each representing a separate page. The document itself is represented as a surface containing references to these page surfaces.

A document, in this context, has a broader meaning than a collection of pages bound or otherwise held in a single unit. Here, a document is any collection of information held in such a form that it can be provided by one device for processing by another device. In the specific context of JetSend, a document comprises one or more surfaces where each surface is connected to all the other surfaces either directly by a single parent-child relationship or indirectly through a chain of parent-child or child-parent relationships.

A surface reference, or a surface name, is simply an ASCII string--like a URL. All surfaces are identified by their name, and may belong to a particular class. All surfaces possess a class. Class is a property indicating the purpose for which the surface is used: in the present implementation, each surface must have one, and only one, class. Classes of surfaces include the self-surface, in-surface, status-surface and address-surface. Specific rules exist, or may be devised, for use of surfaces of a particular class. Classes are used particularly in the context of specific policies. For example, the job policy employs one particular class of surface, the in-surface, as a means to provide jobs to a job processing device. A printer has an in-surface, and surfaces that are impressed onto the in-surface will be treated as new jobs and printed. A PC, a camera, or some other sending device may be impressing surfaces on to this in-surface. Where a particular class of surface plays a fundamental role in execution of a given policy, surfaces of that class are termed "well-known surfaces" for that policy.

A policy, or interaction policy, is a set of conventions for use of generic surface interactions to achieve a particular task. The advantage of using a policy for a particular task is that it enables all JetSend devices adapted to perform, or require another device to perform, the particular task to interact successfully and in a predictable way with other devices in order to achieve that particular task. This does not exclude the possibility of, for example, a pair of devices adapted to perform such a task in a different way not open to other JetSend devices using a different class of surface to do so. However, it is desirable that such devices still support the relevant JetSend policy so that they can achieve this task with other JetSend devices too.

All surface interaction between two devices are done with the following messages:

SurfaceRequestMsg--request a surface of a particular name (and class)

SurfaceMsg--impress a surface (send its description)

ContentRequestMsg and ContentReplyMsg--transfer surface content

DescriptionRequestMsg and DescriptionReplyMsg--transfer additional description

SurfaceChangeMsg--notify and request changes to a surface

SurfaceReleaseMsg--remove the connection between an impression and its expression

The format of these messages and their implementation is described at a later point.

E-material a term coined as a shortened version of "electronic material", is the form taken by information through which surfaces are expressed and shared. It comprises description and content. The description indicates a hierarchy of choices of format in which the associated content data can be provided. The content is the data associated with the surface itself. The description indicates the ways in which the data can be presented, whereas the content is the information itself, presented in a form chosen to be suitable for the device sending it and the device receiving it. The existence of this choice is key to the concept of e-material, and the mechanism by which this choice is made is discussed further below.

Surface interaction and e-material are at the heart of the JetSend architecture and aspects of the present invention are of particular relevance to the implemented combination of surface interaction and e-material. The two concepts are separate, but closely related. Surface interaction is the protocol for exchanging information encoded as e-material. It is the combination of surfaces and e-material that allow JetSend devices to negotiate data types, and exchange information in a device independent fashion.

E-material is not a file format. E-material defines the format of the data that devices exchange, but it does not define how devices process or store that data. Devices that consume e-material do so on a manner that is specific to that device. For example, a receiving device such as a printer will process e-material differently than a receiving device such as a PC. The PC may put the e-material into some kind of a file format, while the printer may convert it to PCL or PostScript.TM., and eventually to dots on a page. Likewise, devices that produce e-material will do so in a manner that is specific to that device.

The most fundamental division of e-material types is into encodings. An encoding is an e-material representation of a fundamental form in which information can be presented in the world. The idea of a "fundamental form" can be illustrated by the different encodings so far defined.

vText relates to presentation of information in the form of written text. A string of ASCII or Unicode text that fills a rectangular plane (no font or spacing information). The simple text encoding is used mainly for status information, but can be used to mark pages as well. The symbol set is negotiable.

vImage relates to presentation of information in the form of graphical images. The encoding relates to a raster image, for which there is the possibility of negotiating qualities such as color space, pixel depth, resolution, pixel encoding, and compression.

vPlane represents to presentation of information on a planar object, of which the commonest form in life is a sheet of paper--as in real life, vPlane relates to planar objects with two sides. More specifically, this encoding provides a two-dimensional rectangular plane with a background colour, and an ordered "layer" of child elements on its front and back. A page is a plane containing other encodings such as images or text.

vFile relates to presentation of information in the form of a binary file. Binary files include such things as documents in application specific formats, executables, etc. A file name, mime-type and an icon can be associated with the data. The mime-type can be negotiated.

vAssociation relates to presentation of information in the form of an assembly of information presented in one or more formats (which will each be represented by their own encoding). Essentially, an association is an ordered sequence of child elements. A conventional document might be represented as an association of pages.

Further encodings can readily be devised in accordance with the present invention, and will be appropriate for device types and functions not addressed in the existing JetSend specification: for example, voice or sound encodings may be devised to represent information in the form of human voice or sound more generally. It should also be noted that the term "encoding" is used here not only to signify fundamental format types, but also to any specific data format hierarchy (for example, one reached at the end of a particular negotiation process). Such uses of the term "encoding" are however readily distinguishable by context.

For each encoding (vImage, vText etc.) there is defined a single default encoding. This ensures that all devices which are able to exchange data of a particular type (for example, all devices which can represent images in some form--personal computers, printers, scanners, but not a microphone) are able to exchange data in some form In the case of an image, the base encoding in the present implementation is vImage.vGray.1.(300,300).vRLE: all devices which are capable of using image information can exchange information in this format.

There will generally be a better format available for the exchange of data than the default encoding--which must, of course, be available for use by the devices with the least rich function set. It is nonetheless desirable to have standard formats which will generally be available for exchange by devices of higher functionality. Such encodings are termed base encodings: base encodings will not be supported by all devices, but it would be expected that all devices with sufficiently rich functionality to do so would support exchange of data according to a base encoding.

An attribute is a feature of a piece of e-material that is defined in the e-material description (or defined for the piece of e-material after the completion of the negotiation process). Essentially, each node in the data format hierarchy provided in the e-material description represents an attribute. From this equivalence between nodes and attributes there is derived the concept of an "attribute level": essentially, an attribute level is shared by all nodes equally distant from the root node of the data format hierarchy. Attributes comprise a quality that can be provided for the e-material content, and a choice or value for that quality: this can range from the size in which an image is provided, or the language in which text is provided, to somewhat less tangible qualities (such as the order in which elements are rendered on a plane), and the choice or value may itself represent a further attribute (a quality requiring a choice or value). The quality itself is represented by an attribute name (sometimes shortened to "attribute" in this description), whereas the choice or value is represented by an "attribute value" (sometimes shortened to value in this description): consequently the nodes of the data format hierarchy can be considered as providing a specific quality and either a specific choice or a range of choices for that quality. Attribute names are keywords, for which the available values or choices are predefined, so the options offered in a given surface description for a given keyword must be some selection from these predefined choices. For different encodings, certain attributes will be required, others will be optional, and others not used at all.

E-material thus expresses the encoding choices of a surface with a number of attribute-value pairs arranged in a hierarchy. All surface descriptions contain the attribute vEncoding (all attribute and value names are prefixed with a lower case v). As indicated above, this determines the overall encoding of the surface--for example, if it is raster image, vEncoding will be vImage. It can also be a list of choices, where the owner of this surface can provide its contents in more than one encoding.

Each encoding has a number of standard attribute-value pairs associated with it. For example, vImage defines attributes such as resolution, pixel size, colour space, and compression. Some of these attributes can also contain choices. In fact, the owner of the surface can express quite complex choices by adding options to many of the attributes in the surface description.

The attributes are arranged in a hierarchy, which allows different choices for a certain attribute depending on previous choices. For example, if an image encoding is offered for a surface, it might contain a choice of JPEG or JPEG-LS compression if SRGB is chosen as the colour space, but RLE or Group3 fax compression if monochrome is the colour space.

Surface descriptions that list encoding choices are written in a tabular form, termed a description table. Examples of description tables will be shown later in this specification. Once a device receives a surface description, it is faced with the decision as to which encoding it prefers for that surface. Once made, the choice is sent to the owner of the surface, with a request for the actual content data.

The description table lists the attributes in three columns: the level, the attribute and its value. The level specifies the values of other attributes that a given attribute applies to. For example, the attribute "vResolution" with the value "(150,150)" may only apply if the selections "vImage.vSRGB.24" have been made for previous attributes listed in the same description.

All encodings can be negotiated with respect to the vLanguage attribute. Any surface can be offered in multiple languages for localization purposes. For each language, different choices about the encoding can be given. It is even possible to offer, say, text for one language and image for another.

The JetSend specification here described is focused on modeling static two-dimensional visual information. It therefore specifies encodings that are useful for representing information that can be contained on physical paper. E-material is not limited to this type of information, and new encodings can readily be devised within the context of the present invention applicable to, in principle, any form of information that can be exchanged between devices. It is also possible to invent proprietary encodings as long as certain rules are followed. This allows effective communication between pairs or groups of devices.

FIG. 2 depicts a two-page document. A visual representation 21 is shown on the left, and its surface representation on the right. In this case, it is represented using seven surfaces, each with its own encoding. The rectangles are surfaces that contain references to other surfaces. The top-level surface 22 is an association with a reference to each of the two pages of the document. Each page is represented as a plane encoding 23, 24. Each plane contains two references to surfaces that represent the information on the pages. The first plane 23 points to an image surface 25 and a text surface 26. The second plane 24 points to two image surfaces 27, 28.

Each image surface is negotiated separately, and might contain different encoding offerings. This is just one way that this document can be represented using surfaces and e-material. In this case it is broken down into the elements that are visible on the page--i.e., the three images and the text segment. Another possibility is to encode the information as two planes, each with one large image region covering the entire page. It is a design choice of the owner of a document how to encode its information.

As indicated above, incorporated into this architecture is the principle of provision of a default encoding. For each encoding there is a default set of attributes that all JetSend devices are required to support. The default encoding ensures that any combination of devices that exchange the same class of information will be able to communicate. It also ensures forwards compatibility: new devices can implement smarter encodings as long as they support the default one.

All devices are required to support the association encoding, which is simply a container for surfaces with other encodings. Also, all devices that deal with static two-dimensional information are required to support the plane encoding.

The default encoding for imaging devices is 300.times.300 dpi, monochrome, RLE compressed raster. That means that all Jetsend image devices (printers, scanners, copiers, cameras, whiteboards, etc.) must be able to send and/or receive this encoding. Text and File encodings are optional for such devices. As devices supporting other classes of data are implemented, default encodings for those data types will also be introduced. This would include such data types as audio and video. The default is intended as the lowest common denominator for a given encoding.

As further indicated above, in addition to default encodings, an optional but advantageous feature is the provision of base encodings. These are recommended, higher-level encodings that ensure better results for devices with higher capabilities. For instance, for colour imaging devices there is a base encoding for colour images that these devices should support, in addition to the default encodings. A device is free to offer additional encodings as long as the default encodings are offered as well. The device may provide better level of performance or quality by also implementing the base encodings best suited to that particular device type.

The JetSend protocols are designed to be independent of device type, platform and transport. A brief overview of each of the functional components of Jetsend appliances. Appliance is a term widely used throughout this specification, and is given a more specific meaning than device. An appliance is a device with a dedicated function, which is capable of substantially independent handling of the data provided to it to achieve that dedicated function. It thus has a broader utility than a pure peripheral to a personal computer, for example. An appliance is generally not reprogrammable, and will generally require only a minimum quantity of configuration to operate. Devices such as scanners and printers will desirably function as appliances when adapted to support the present implementation of the present invention.

There are three primary areas of functionality that make up a JetSend appliance; the transports in the device, the JetSend protocols themselves, and device specific code. FIG. 3 identifies the components of the JetSend architecture and their logical relationships to each other. This is followed by an overview of each of the components. Details for each component are provided at a later point. It should be noted that FIG. 3 is not an implementation diagram. It shows the relationship between the protocol, not between software components. Actual implementations can have similar components, or combine the implementation of multiple protocols into a single module.

The JetSend architecture is applicable independent of transport. JetSend devices can address each other directly over any bi-directional transport 36 using unique addressing. It is necessary for the transport to be reliable: therefore for an unreliable transport such as UDP, a further protocol layer must be added to make transmissions over the transport reliable (a further protocol here termed Reliable Message Transport Protocol (RMTP) 37 is used for this purpose). Possible transports include TCP/IP, SPX/IPX, IrDA, IEEE1284, IEEE1394, and others. A device can implement one or more transports, allowing it to communicate with other devices using those same transports.

Communication between JetSend appliances occurs using a number of layered protocols, as can be seen from FIG. 3. These layers are similar to most networking systems, where each protocol layer communicates with the equivalent layer in the other device. The layers that comprise the JetSend protocol are the Interaction Policies 35, the Interaction Protocol 33, the Session Protocol 34 and the RMTP Protocol 37.

The Interaction Policies define various typical sequences of interactions that devices will follow. The interaction policies are used in as generic building blocks to accomplish more specific information exchanges between devices. The following interaction policies have been defined and are discussed further below:

    Job Policy    How to send documents between senders and receivers.
    Self Policy   How to exchange global information about a device such
                  as label, icon and passwords.
    Status Policy How to give status about a device and about jobs.
    Address Policy How to program devices with new destination addresses.

                             TABLE 1
                        JSP Message Types
     SYN   ACK   NAK   RST   NUL   CHN  Name    Meaning
      1     0     0     0     0     0   SYN     Request for session
      1     1     0     0     0     0   ASYN    Accept session
                                                request
      1     0     1     0     0     0   NSYN    Reject session
                                                request
      0     0     0     1     0     0   RST     Close session
      0     0     0     0     1     0   NUL     Keep-alive
      0     0     0     0     0     1   CHN     Data for a channel
      1     0     0     0     0     1   CSYN    Request for channel
      1     1     0     0     0     1   CASYN   Accept channel
                                                request
      1     0     1     0     0     1   CNSYN   Reject channel
                                                request
      0     0     0     1     0     1   CRST    Close channel


                         TABLE 2
                       JSP Gateway messages
         ACK       CHN      Name        Meaning
          0         1       GWY         Gateway broadcast
          1         1       GACK        Gateway broadcast reply

                                        TABLE 3
                               SurfaceMsg message fields
    Field                  Type                 Use
    Message type           Unsigned 8-bit integer Type of message
    Protocol version       Unsigned 8-bit integer Interaction-Protocol version
     number
    Source surface identifier Unsigned 32-bit integer Identifier of expression
    Source surface class   Unsigned 32-bit integer Class of expression
    Source surface version Unsigned 32-bit integer The current version number
     of the surface
    Source surface properties Unsigned 32-bit integer The properties of the
     surface. A bit mask.
    Target surface identifier Unsigned 32-bit integer Identifier of target
     expression
    Reserved               2-byte               Reserved for later use. Must be
     set to NULL.
    Status                 Unsigned 16-bit integer The status for impresses and
     impress
                                                requests
    Request identifier     Unsigned 16-bit integer The request identifier if
     this impress resulted
                                                from a request
    Reserved               2-byte               Reserved for later use. Must be
     set to NULL.
    Impress identifier     Unsigned 32-bit integer A unique identifier for this
     impress message
    Source surface name length Unsigned 16-bit integer Length of expression
     name in bytes
    Source surface name    Sequence of bytes    Name of expression
    Target address length  Unsigned 16-bit integer Length of the target address
     in bytes
    Target address         Session Protocol     Address of target device
                           address (sequence of
                           bytes)

                             TABLE 4
                    Class values for surfaces
           Value         Symbol          Meaning
             1           OTHER           Other surface
             2           SELF            Self surface
             3           IN              In surface
             4           STAT            Status surface
             5           ADDR            Address surface

                             TABLE 5
         Properties of surface impressed with SurfaceMsg
    Value Meaning
      1   The expressive device will respond to a SurfaceMsg on this surface
      2   The expressive device will accept a SurfaceChangeMsg from an
          impressive device

                             TABLE 6
               Defined values for SurfaceMsg status
    Value Meaning
      0   This is an unsolicited SurfaceMsg
      1   This is an impress in response to a SurfaceRequestMsg. The
          request identifier field is set to the corresponding request
     identifier
      2   This is a rejection of a previous SurfaceRequestMsg. The request
          identifier field is set to the corresponding request identifier