Method, system, and computer program product for a data propagation platform and applications of same

Abstract

A method, system, and computer program product for a data propagation platform and applications of same is described. The data propagation platform provides various storage functions. According to an embodiment, the factories of the data propagation platform create and implement device modules. Device modules perform operations on encapsulated messages formed from data and commands, such as device operation commands and data from storage device read/write operations. Device modules are formed into device chains. Device chains are able to perform various tasks as a sum of device modules. In embodiments, device chains are layered and built on each other.

Claims

What is claimed is:

1. A system for a data propagation platform, comprising:

one or more factories for the creation of at least one device module, wherein said one or more factories contain configuration information and feature implementation parameters;

a database for storing at least one configuration event and event handler;

one or more storelets for encapsulating procedural information, wherein said procedural information includes at least one control parameter for the operation of one or more servers, storage devices, or network devices;

one or more triggers for identifying at least one of a type, location, or specific data relating to said one or more servers, storage devices, or network devices; and

a configuration manager for receiving said at least one configuration event, wherein said configuration manager forwards said at least one configuration event to the proper event handler.

2. A system of claim 1, wherein said one or more storelets contain one or more bullets that can be attached to said one or more triggers; wherein said bullets contain at least one executable code segment to process data provided by said one or more triggers.

3. A system of claim 1, further comprising:

a message processing module for receiving one or more changes to one or more of said one or more factories, said database, said one or more storelets, said one or more triggers, and said configuration manager.

4. A system of claim 1, wherein said at least one event handler includes an event removal handler.

5. A system of claim 1, wherein said at least one event handler includes a notification handler.

6. A system of claim 1, wherein said at least one event handler includes a report handler.

7. A system of claim 1, further comprising:

a mapping module for removing data-flow operations from kernel memory space, wherein said data-flow operations are maintained in user memory space.

8. A method for processing commands in a data propagation platform, comprising the steps:

(1) receiving one or more commands from one or more servers, storage devices, or network devices;

(2) building a message from said one or more commands;

(3) accessing one or more triggers with said message;

(4) accessing one or more device chains with said message;

(5) querying one or more device modules in each of said one or more device chains about said message;

(6) copying said message to said one or more device modules, wherein only said one or more device modules that are designed to respond to said message receive a copy of said message;

(7) querying said one or more device modules that are designed to respond to said message for replies; wherein said replies include configuration information; and

(8) obtaining said replies from said one or more device modules that are designed to respond to said message.

9. A method of claim 8, wherein said command is a small computer serial interface (SCSI) command.

10. A method of claim 8, wherein said configuration information includes current operating information about said one or more servers, storage devices, or network devices.

11. A method of claim 8, wherein said one or more device chains are operating in user memory space.

12. A computer program product comprising a computer usable medium having computer readable program code means embodied in said medium for causing an application program to execute on a computer that processes commands in a data propagation platform, said computer readable program code means comprising:

a first computer readable program code means for causing a computer to receive one or more commands from one or more servers, storage devices, or network devices;

a second computer readable program code means for causing a computer to build a message from said one or more commands;

a third computer readable program code means for causing a computer to access one or more triggers with said message;

a fourth computer readable program code means for causing a computer to access one or more device chains with said message;

a fifth computer readable program code means for causing a computer to query one or more device module in each of said one or more device chains about said message;

a sixth computer readable program code means for causing a computer to copy said message to said one or more device modules, wherein only said one or more device modules that are designed to respond to said message receive a copy of said message;

a seventh computer readable program code means for causing a computer to query said one or more device modules that are designed to respond to said message for replies; wherein said replies include configuration information, and

an eighth computer readable program code means for causing a computer to obtain said replies from said one or more device modules that are designed to respond to said message.

13. A computer program product of claim 12, wherein said command is a small computer serial interface (SCSI) command.

14. A computer program product of claim 12, wherein said configuration information includes current operating information about said one or more servers, storage devices, or network devices.

15. A computer program product of claim 12, wherein said one or more device chains are operating in user memory space.

Description

BACKGROUND OF THE INVENTION

1. Field of the Invention

The invention relates generally to the field of storage area networks, and more particularly to data-flow processing through a layered framework.

2. Related Art

Traditional approaches exist for data path architectures in storage networks. These approaches generally follow a stream oriented configuration for data transfer with flow control.

Stream oriented approaches typically focus on using a data path to communicate between processes. For example, a first process may write data to a data path for a second process. The second process must wait for the data to arrive from the first process. These processes must wait for the data to be pushed to them along the path. Thus, stream oriented architectures have a drawback based on their intrinsic nature.

Specifically, stream oriented architectures suffer from stability issues. If one process fails, then other processes may not receive any data. Additionally, other difficulties include: managing the paths between and among processes for each storage device, server (host) and network device; monitoring the processes so users of the various processes can be alerted to faulty processes; and managing inter-host dependencies when processes are altered because the functionality is altered.

Therefore, in view of the above, what is needed is a system, method and computer program product for a data propagation platform. Furthermore, what is needed is a system, method and computer program product for producing applications capable of being added and/or layered without changing, affecting, or disrupting the basic processing architecture. Still further, what is needed is a system, method and computer program product for a data propagation platform that provides flexibility in that processes can be instantiated as modules linked in data paths. Yet still further, what is needed is a system, method and computer program product for a data propagation platform that is capable of providing a stable architecture for data paths and processes.

SUMMARY OF THE INVENTION

The present invention is directed to a method, system, and computer program product for a data propagation platform that satisfies the above-stated needs. Device modules are constructed to form the basic elements of the data path. Device modules manipulate passing objects by processing only those objects that the device modules are designed to understand. In one embodiment, objects which are not understood are passed without modification. Objects include messages, which are encapsulated data and commands. Thus, device modules execute commands in the data path.

The present invention can be implemented in a storage area network (SAN) and operated from a SAN appliance that interfaces between the hosts and the storage subsystems of the SAN. Furthermore, the present invention can be implemented in more complex network topologies as described herein in embodiments.

Further aspects of the invention, and further features and benefits thereof, are described below. The accompanying drawings, which are incorporated herein and form a part of the specification, illustrate the invention and, together with the description, further serve to explain the principles of the invention and to enable a person skilled in the pertinent art to make and use the invention.

BRIEF DESCRIPTION OF THE FIGURES

In the drawings, like reference numbers indicate identical or fuctionally similar elements. Additionally, the left-most digit(s) of a reference number identifies the drawing in which the reference number first appears.

FIG. 1A illustrates a block diagram of an example data path, according to an embodiment of the invention;

FIG. 1B illustrates a block diagram of an example mirrored data path, according to an embodiment of the invention;

FIG. 1C illustrates a block diagram of an example redundant data path, according to an embodiment of the invention;

FIG. 2 illustrates a block diagram of an example system using a data propagation framework, according to an embodiment of the invention;

FIG. 3 illustrates a block diagram of an example device chain, according to an embodiment of the invention;

FIG. 4 illustrates a block diagram of example device chains using layering, according to an embodiment of the invention;

FIG. 5 illustrates a block diagram of an example of fibre channel protocol (FCP) mirroring with device chains, according to an embodiment of the invention;

FIG. 6 illustrates a block diagram of an example of copy-on-write or snapshot with device chains, according to an embodiment of the invention;

FIG. 7 illustrates a block diagram of an example single partition device chain, according to an embodiment of the invention;

FIG. 8 illustrates a block diagram of an example multiple partition device chains, according to an embodiment of the invention;

FIG. 9 illustrates a block diagram of an example expansion device chain, according to an embodiment of the invention;

FIG. 10 illustrates a block diagram of an example of a small computer serial interface (SCSI) read command on an access-controlled logical unit number (LUN) with device chains, according to an embodiment of the invention;

FIG. 11 illustrates an example operating environment, according to an embodiment of the invention;

FIG. 12 shows a flowchart providing detailed operational steps for handling commands in user space, according to an embodiment of the invention;

FIG. 13 shows a flowchart providing detailed operational steps for adding configuration events, according to an embodiment of the invention;

FIG. 14 shows a flowchart providing detailed operational steps for creating a device module, according to an embodiment of the invention;

FIG. 15 shows a flowchart providing detailed operational steps for instantiating an event handler, according to an embodiment of the invention;

FIG. 16 shows a flowchart providing detailed operational steps of an event removal handler, according to an embodiment of the invention;

FIG. 17 shows a flowchart providing detailed operational steps of a notification handler, according to an embodiment of the invention;

FIG. 18 shows a flowchart providing detailed operational steps of a report handler, according to an embodiment of the invention;

FIG. 19 shows a flowchart providing detailed operational steps for processing commands, according to an embodiment of the invention;

FIG. 20 illustrates an example data communication network, according to an embodiment of the invention;

FIG. 21 shows a simplified five-layered communication model, based on an Open System Interconnection (OSI) reference model; and

FIG. 22 shows an example of a computer system for implementing the invention.

The invention will now be described with reference to the accompanying drawings.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Applications of a Data Propagation Platform and Embodiments

The present invention is directed to a method, system, and computer program product for a data propagation platform that provides an object oriented framework for applications. The data propagation platform or data propagation framework (hereinafter referred to as "DPF") provides flexibility in that device modules, which perform various functions, can be linked and re-linked to form different data paths. Additionally, new device modules can be linked in data paths to provide new functionality. The invention manages the creation of chains of device modules to form data paths. In an embodiment, the invention also allows for the implementation of data paths in the user space of computing environment, thereby separating the processing of the device modules from the kernel of the computing environment.

FIG. 1A illustrates a block diagram 100A of an example data path, according to an embodiment of the invention. Diagram 100A includes host 102 and storage 104. The arrow going from host 102 to storage 104 and the arrow going from storage 104 back to host 102 both illustrate a straightforward data path without the use of any network devices.

FIG. 1B illustrates a block diagram 100B of an example mirrored data path, according to an embodiment of the invention. Diagram 100B includes host 110 connected to appliance 112. In turn, appliance 112 is connected to each of storage 114 and storage 116. Storage 114 or storage 116 are accessible to host 110 through appliance 112. In one embodiment, data is mirrored to both storage 114 and storage 116 in order to provide a backup of the data in the event of storage device failure.

FIG. 1C illustrates a block diagram 100C of an example redundant data path, according to an embodiment of the invention. Diagram 100C includes host 120 connected to each of appliance 122, appliance 124, and appliance 126. Each of appliances 122-126 is connected to storage 128. In one embodiment, host 120 is able to connect to storage 128 through any of appliances 122-126. In the event of a network outage involving one or two of the appliances, host 120 can still reach storage 128 using the third appliance.

By changing the data path, one changes the way the storage operates and performs. Further combinations of the topologies of diagrams 100A, 100B, and 100C are possible as one skilled in the relevant art would recognize based on the teachings described herein. For example, each of appliances 122-126 can be connected to overlapping storage devices to provide redundancy and reliability at the appliance and storage level.

FIG. 2 illustrates a block diagram of an example system 200 using a data propagation framework, according to an embodiment of the invention. DPF system 200 includes device chains 202a-202n, wherein the device chains comprise device modules.

According to embodiments of the invention, device modules are objects or run-time entities constructed to perform operations on the data flowing past them in the data path. In one embodiment, a device module contains an algorithm for checking for the presence of a specific storage device, the algorithm sending a reply (or alert notice) is the device is not present. This example is not intended to limit the present invention. One skilled in the relevant art(s) would realize that, based on the teachings described herein, more complicated device modules can be constructed.

Device chains contain device modules that are linked together to form a data path. In one embodiment, the device chain is constructed as the data-past. Data and commands from devices on the network will therefore enter the device chain. Data and commands are passed along from device module to device module. Once the end of the device chain is reached, replies are passed back along the device chain by some or all of the device modules. In embodiments described below, each device module can also be viewed as a device chain, which can be linked with other chains, depending on the complexity of the operations being performed.

Device modules, which are the basic elements of the data path, manipulate passing objects by processing only those objects that they are designed to understand. In one embodiment, other objects are passed though the device chain(s) without modification. In another embodiment, other objects are time-stamped before being passed through the device chain(s). Some example device modules are listed in Table 1 below.

                             TABLE 1
                      Example Device Modules
    DPFCdbResizeDevice   Module for the modification of SCSI CDBs
                         (Command Descriptor Blocks) from the 6-byte
                         versions to the 10-byte versions,
                         allowing expanded disk addressing
    DPFClusterHealthDevice Module for the monitoring of redundant pairs,
                         allowing functionality to be shut down during
                         failure scenarios
    DPFContainerDevice   Module to contain other modules, with
                         commands passing in only one direction at a
                         time
    DPFDeviceChain       Module to contain other modules, with
                         commands passing both directions
    DPFDeviceResizer     Module to modify the reported size of the
                         Appliance virtual LUN
    DPFEmptyDevice       Empty placeholder module, with no other
                         purpose than to serve as a reference point
    DPFExpanderDevice    Module to combine two underlying virtual
                         LUNs into a single virtual LUN image
    DPFFailoverDevice    Module to handle alternate pathing and
                         resnchronization of data images for data
                         replication
    DPFForwardDevice     Module to pass all commands to a different
                         chain
    DPFGeometryFilter    Module to modify reported disk geometry
                         when a virtual LUN's geometry is changed
    DPFHealthDevice      Module to monitor the accessibility of a
                         virtual LUN
    DPFInquiryDevice     Module to modify the data contained in a
                         SCSI inquiry command
    DPFJournalDevice     Module to store location information about IO
                         failures
    DPFLunMasterDevice   Module to present virtual LUN "up" to the
                         rest of the world: serves as a starting point for
                         all IO requests
    DPFMirrorDevice      Module to implement simple data replication
                         between virtual LUNs
    DPFModeSenseDevice   Module to modify SCSI mode sense
                         commands on virtual LUNs
    DPFMultiPartitionDevice Module for creation of a large number of
                         virtual partitions based on a single virtual
                         LUN
    DPFNTDevice          Module for communication with the back-end
                         NT storage device
    DPFReadAheadDevice   Module for implementing read-ahead caching
    DPFReadOnlyDevice    Module for denying all write access to a
                         virtual LUN
    DPFReadWriteAccess   Module for partially restricting a particular
                         host's access to a virtual LUN
    DPFRedundantDevice   Module for the communication of IO
                         information across the Fibre Channel link in a
                         redundant Appliance pair
    DPFRemovableDevice   Module for the simulation of a removable
                         device on a virtual LUN
    DPFReserveAccess     Alternative module for partially restricting a
                         particular host's access to a virtual LUN based
                         on the SCSI reserve/release protocol
    DPFReserveDevice     Module for implementing the SCSI
                         reserve/release protocol on a virtual LUN
    DPFRPICDevice        Module for the discovery and monitoring of a
                         primary Appliance in a redundant Appliance
                         pair
    DPFScsiBusDevice     Module for discovery and communication of
                         back-end SCSI host bus adapters
    DPFScsiForwardDevice Module for forwarding SCSI commands to a
                         different chain
    DPFShareDevice       Module for sharing a single physical device
                         between multiple virtual LUNS
    DPFShareTarget       Module for presenting a virtual LUN based on
                         a ShareDevice
    DPFSnapshotSource    Module for using a virtual LUN as the source
                         of a snapshot
    DPFSnapshotTarget    Module for allocating a virtual LUN as a
                         target of a snapshot
    DPFSubLunDevice      Module for the partitioning of a single virtual
                         LUN into multiple
    DPFUnknownDevice     Module for handling commands directed at an
                         unknown virtual LUN

                             TABLE 2
                         Example Triggers
    RESYNCH_START                Mirror resynchronization process
                                 started
    RESYNCH_COMPLETE             Mirror resynchronization process
                                 ended
    SNAP_ALLOCATED               Snapshot target device allocated
    SNAP_DEALLOC                 Snapshot target device deallocated
    SNAP_TAKEN                   Snapshot taken from source to target
    SNAP_BROKEN                  Snapshot to target removed
    IO_TIMEOUT                   IO request to back-end storage timed
                                 out
    IO_FAILURE                   IO request to back-end storage failed
    DEVICE_FAILURE               IO request to back-end storage failed
                                 due to hardware error
    DEVICE_RECOVERY              IO request to previously failed
                                 hardware succeeded
    NODE_DOWN                    Network request to remote node
                                 failed
    NODE_TIMEOUT                 Network request to remote node
                                 timed out
    NODE_UP                      Network request to previously
                                 unreachable remote node succeeded
    FCLINK_FOUND                 Fibre channel link between
                                 Appliances in a redundant pair
                                 discovered
    FCLINK_DOWN                  Fibre channel link request between
                                 Appliances in a redundant pair failed
    FCLINK_UP                    Fibre channel link request between
                                 Appliances in a redundant pair
                                 succeeded
    GET_REPORT                   Received a request for information
    LUNMAP_ENABLED               Added a new mapping between a
                                 Appliance virtual LUN and a LUN
                                 on an attached host
    LUNMAP_DISABLED              Removed a mapping between a
                                 Appliance virtual LUN and a LUN
                                 on an attached host
    HOST_ADDED                   New host has been attached to the
                                 Appliance
    HOST_NAMED                   Host name has been registered with
                                 the Appliance
    HOST_REMOVED                 Host name and identifier has been
                                 removed from the Appliance
    HOST_HBA_ADDED               Known host has become visible
                                 from a new target HBA
    WAITING_FOR_BUFFER           Memory buffer pool exhausted and a
                                 thread is waiting for a buffer to be
                                 freed
    ADDING_CONFIGURATION         A configuration change is being
                                 initiated (adding new configuration)
    ADDED_CONFIGURATION          A configuration change was
                                 successfully made
                                 (added new configuration)
    ADD_CONFIG_FAILED            A configuration change did not
                                 complete successfully
    REMOVING_CONFIGURATION       A configuration change is being
                                 initiated (removing old
                                 configuration)
    REMOVED_CONFIGURATION        A configuration change was
                                 successfully made
                                 (removed old configuration)
    REMOVE_CONFIG_FAILED         A configuration change did not
                                 complete successfully
    NOTIFICATION_RECEIVED        Notification of underlying system
                                 change received
    NOTIFICATION_SCAN            Notification of underlying system
                                 configuration change received
    DEVICE_HEALTH_UP             Appliance virtual LUN marked
                                 online
    DEVICE_HEALTH_DOWN           Appliance virtual LUN marked
                                 offline
    SHARE_USAGE                  Shared device block usage passed
                                 notification threshold
    SHARE_FREE                   Shared device block mapping
                                 deallocated

    Arbitrated Loop   A shared 100 MBps Fibre Channel transport
                      supporting up to 126 devices and 1 fabric
                      attachment.
    Fabric            One or more Fibre Channel switches in a
                      networked topology.
    GBIC              Gigabit interface converter; a removable transceiver
                      module for Fibre Channel and Gigabit Ethernet
                      physical-layer transport.
    GLM               Gigabit link module; a semipermanent transceiver
                      that incorporates serializing/deserializing functions.
    HBA               Host bus adapter; an interface between a server or
                      workstation bus and a Fibre Channel network.
    Hub               In Fibre Channel, a wiring concentrator that
                      collapses a loop topology into a physical star
                      topology.
    Initiator         On a Fibre Channel network, typically a server or a
                      workstation that initiates transactions to disk or
                      tape targets.
    JBOD              Just a bunch of disks; typically configured as an
                      Arbitrated Loop segment in a single chassis.
    LAN               Local area network; a network linking multiple
                      devices in a single geographical location.
    Point-to-point    A dedicated Fibre Channel connection between two
                      devices.
    Private loop      A free-standing Arbitrated Loop with no fabric
                      attachment.
    Private loop device An Arbitrated Loop device that does not support
                      fabric login.
    Public loop       An Arbitrated Loop attached to a fabric switch.
    Public loop device An Arbitrated Loop device that supports fabric
                      login and device services.
    RAID              Redundant Array of Independent Disks.
    SCSI              Small Computer Systems Interface; both a
                      protocol for transmitting large blocks of data
                      and a parallel bus architecture.
    SCSI-3            A SCSI standard that defines transmission of SCSI
                      protocol over serial links.
    Storage           Any device used to store data; typically, magnetic
                      disk media or tape.
    Switch            A device providing full bandwidth per port and
                      high-speed routing of data via link-level
                      addressing.
    Target            Typically a disk array or a tape Subsystem on
                      a Fibre Channel network.
    Topology          The physical or logical arrangement of devices in
                      a networked configuration.
    WAN               Wide area network; a network linking
                      geographically remote sites.

• Inventors
  Bates, John W.; Vekiarides, Nicos A.;
• Assignee
  Hewlett-Packard Development Company, L.P. (Houston, TX)
• Published
  Oct-12-2004
• Current US Classes:
  709/220
  709/223
  719/318
• Application #
  665583
• International Classes
  G06F 003/00; G06F 009/44; G06F 009/46; G06F 013/00
• Field of Search
  709/220-230 709/313 709/314 709/315 709/318 719/318
• Examiner
  An; Meng-Al T.
• US Patent References:
  4274139
  4404647
  4611298
  4654819
  4688221
  4701915
  4942579
  4996687
  5051887
  5129088
  5136523
  5138710
  5155845
  5157663
  5167011
  5175849
  5185884
  5206939
  5212784
  5212789
  5235601
  5257367
  5263154
  5276867
  5278838
  5317731
  5325505
  5347653
  5357509
  5367682
  5375232
  5390313
  5404361
  5430866
  5430885
  5432922
  5435004
  5452420
  5479615
  5481701
  5495601
  5497483
  5504882
  5519844
  5528759
  5530845
  5537533
  5544347
  5546535
  5546556
  5561815
  5566316
  5568628
  5568629
  5583994
  5596736
  5611069
  5619694
  5625818
  5659787
  5664144
  5671439
  5673322
  5673382
  5680580
  5684945
  5692155
  5710885
  5710918
  5715393
  5719942
  5720027
  5732238
  5737743
  5740397
  5742792
  5745789
  5748897
  5761410
  5761507
  5764913
  5765186
  5765204
  5768623
  5771367
  5774655
  5774680
  5784703
  5787304
  5787459
  5787470
  5787485
  5790774
  5805919
  5809332
  5812754
  5812784
  5819020
  5828475
  5828820
  5832088
  5835718
  5835764
  5842224
  5848251
  5850522
  5852715
  5857208
  5857213
  5873125
  5889935
  5893919
  5894554
  5897661
  5901280
  5901327
  5909692
  5911150
  5911779
  6105122
  6209023
  6425036
  6470382
  6470397
  6574696
  6584502
  6622163
  6640278
  6654502