Access augmentation or optimizing

Regulating file access rates according to file type

6907421

Abstract

File access rates of processes are regulated according to file type. An association table stores entries associating processes to be regulated with specific access rates for various file types. System calls that access files are intercepted, and a system call wrapper executes. The system call wrapper determines the type of file that is being accessed by the process. The system call wrapper examines the association table in order to determine if the calling process is associated with an access rate for the file type being accessed. If so, the system call wrapper regulates access to the file according to the appropriate rate.


Claims

1. A computer-implemented method for regulating file access rates of processes according to file type, the computer-implemented method comprising:

intercepting a system call attempts to access a file;

determining whether a process that made the intercepted system call is associated with an access rate corresponding to a type of the file being accessed;

in response to the attempt to access the file by the process, determining the associated access rate for the type of the file being accessed; and

regulating the process to access of the file at the determined rate.

2. The method of claim 1 further comprising:

storing an association between a selected process the an access rate for at least one file type.

3. The method of claim 2 wherein:

storing the association is performed by a process executing in user address space.

4. The method of claim 1 further comprising:

selecting a process to associate with an access rate for at least one file type;

loading the selected process into process address space by a modified loader program; and

storing, by the modified loader program, an association between the selected process and the access rate for at least one file type.

5. The method of claim 1 further comprising:

associating at least two different access rates with a single process, each access rate corresponding to a different file type.

6. The method of claim 1 further comprising:

associating at least two different access rates with at least two different processes.

7. The method of claim 1 wherein:

in response to the determination that the process is associated with the access rate, regulating the process to access of the file at the associated rate.

8. The method of claim 1, further comprising:

determining the type of the file being accessed.

9. The method of claim 1, further comprising:

associating at least one process with the access rate for at least one file type.

10. A computer-readable program product for regulating file rates of processes according to file type, the computer-readable program product comprising:

program code for intercepting a system call attempts to access a file;

program code for determining whether a process that made the intercepted system call is associated with an access rate corresponding to a type of the file being accessed;

program code for, in response to the attempt to access the file by the process, determining the associated access rate for the type of the file being accessed;

program code for regulating the process to access of the file at the determined rate; and

a computer readable medium on which the program codes are stored.

11. The computer-readable program product of claim 10 father comprising:

program code for storing an association between a selected process and an access rate for at least one file type.

12. The computer-readable program product of claim 10 further comprising:

program code for selecting a process to associate with the access rate for at least one file type;

program code for loading the selected process into process address space; and

program code for storing an association between the selected process and the access rate for at least one file type.

13. The computer-readable program product of claim 10 further comprising:

program code for associating at least two different access rates with a single process, each access rate corresponding to a different file type.

14. The computer-readable program product of claim 10 further comprising:

program code for associating at least two different access rates with at least two different processes.

15. The computer-readable program product of claim 10 further comprising:

program code for, in response to the determination that the process is associated with the access rate, regulating the process to access of the file at the associated rate.

16. The computer-readable program product of claim 10 further comprising:

program code for determining the type of the file being accessed.

17. The compute-readable program product of claim 10 further comprising:

program code for associating at least one process with the access rate for at least one file type.

18. A computer implemented system for regulating file access rates of processes according to a file type, the system comprising:

an interception module, for intercepting system calls that access files, the interception module being coupled to a determination module;

at least one system call wrapper, for determining whether a process that made an intercepted system call is associated with an access rate corresponding to a type of the file being accessed, the system call wrapper being coupled to the interception module;

the determination module for, in response to an attempt to access a file by the process, determining the associated access rate for the type of the file being accessed; and

coupled to the determination module, a file access rate regulation module, for regulating the process to access of the file at the determined rate.

19. The computer system of claim 18 further comprising:

a selection module for selecting the process to associate with the access rate for at least one file type, the selection module being coupled to the determination module; and

a modified loader program for loading the selected process into process address space, and for storing the association between the selected process and the access rate for at least one file type, the modified loader program being coupled to the selection module.

20. The computer system of claim 18 further comprising:

a storage module, for storing the association between a selected process and the access rate for at least one file type, the storage module being coupled to the determination module.

21. A computer-implemented method for regulating file access rates of processes according to file type, the method comprising:

associating processes with rates for accessing files of specific types;

detecting attempts by processes to access files by intercepting system calls that access files;

determining associated access rates for the types of files being accessed; and

regulating processes to access of the files at the associated rates.


Description

BACKGROUND

1. Field of Invention

The present invention relates generally to regulating access rates in a computer system, and specifically to regulating file access rates of software processes according to file type.

2. Background of Invention

Multitasking operating systems such as UNIX® and Microsoft WINDOWS NT®are widely utilized in commercial computing systems. Among their many commercial uses, these operating systems are commonly deployed on Internet and other network server computers. With the popularity and success of the Internet, server computer operating systems are currently of great commercial importance.

One function of a multitasking operating system is to allocate system resources to the multiple software processes that simultaneously execute under the control of the operating system. Control over the allocation of system resources by an operating system is commercially useful for a number of reasons. Multitasking operating system are commonly used on Internet web servers by Internet Service Providers (ISP's). Where an ISP provides host services to multiple customers on a single physical computer, it is desirable to allot to each virtual host a specific amount of computer resources appropriate to the needs of the customer, and preferably based upon the amount paid for the services.

For example, suppose two customers purchase host services from an ISP. The first customer is a large corporation providing financial services to thousands of clients internationally. The financial services host requires fast file access, as well as prompt response time to all client requests. Of course, the first customer is willing to compensate the ISP appropriately for providing such a level of host services. The second customer is a sole proprietorship that sells floral arrangements locally. The second customer has a very limited budget, but only requires minimal computer resources. Clearly, it is desirable for the ISP to allocate different percentages of the system resources to the two separate virtual hosts provided by the ISP for the two separate customers.

In the example above, the ISP may wish to provide the financial services host with the ability to access files at the rate of 1,000 bytes per second, but to allow the florist to access files at the rate of only 150 bytes per second. These different access rates would be based upon the different needs of the two customers, and the corresponding different compensation schemes of each. As multitasking operating systems operate today, it would be impossible for the ISP to regulate the file access rates of the different customers. Each process associated with either virtual host simply accesses the file system at the same unregulated rate, and thus it is impossible for the ISP to guarantee or restrict access rates based upon customer need and corresponding financial arrangement. What is needed is a method that facilitates the regulation of the rate at which individual processes access the file system of a multitasking operating system.

Many commercially popular operating systems such as UNIX® and Microsoft WINDOWS NT® treat communication channels as files. In such an operating system, when a process instructs the operating system to create a communication channel, the operating system returns a file descriptor. The communication channel is subsequently accessed via the file descriptor, in a similar manner as a file stored on media.

Communication channels and files stored on media are inherently different, despite the fact that both are accessed via file descriptors. It is often desirable to for an ISP to allow a single process to access files stored on media at one rate, and to access communication channels at another. For example, a customer of the ISP may need to receive and respond to client requests very quickly, but need only an average access time for files stored on the system storage device(s). Such a customer would require (and be willing to pay for) a fast communication channel access rate, but only need (and be willing to pay for) a slower access rate to files stored on media.

Multitasking operating systems today are not capable of regulating the file access rates of different processes generally, much less facilitating different access rates for specific processes based on file type. Currently, each process accesses both files stored on media and communication channels at unregulated rates, not controlled by the operating system. Thus, it is impossible for the ISP to provide customers with different access rates for files stored on media and for communication channels, based upon customer need and corresponding financial arrangement. What is further needed is a method that not only facilitates the regulation of the rate at which individual processes access the file system of a multitasking operating system, but which also allows processes to be regulated to different access rates for different file types.

It is further desirable to not only be able to set separate access rates for communication channels and files stored on media, but to be able set separate access rates for file types generally. Under some operating systems, entities other than communication channels and files stored on media are treated as files, and hence need separate access rates. For example, under the UNIX® operating system, hardware devices are treated as files. For the same reasons that it is desirable to set separate access rates for communication channels and files stored on media, it is further desirable to be able to set a separate access rate for any type of file. Accordingly, what is needed is a method to set separate access rates for individual processes according to file type.

SUMMARY OF INVENTION

The present invention allows regulation of the file access rates of processes according to file type. An association data structure stores associations between processes to be regulated and specific access rates for various file types. In order to regulate processes to their associated access rates, system calls that access files are intercepted. When a process to be regulated makes a system call that accesses a file, the system call is intercepted, and a system call wrapper executes instead. The system call wrapper determines the type of file that is being accessed by the process. The system call wrapper examines the association table in order to determine if the calling process is associated with an access rate for that file type. If not, then the process is not to be regulated for that file type, in which case the system call wrapper simply allows the file access to proceed. Otherwise, the system call wrapper regulates the rate of the access to the file according to the appropriate rate.

The features and advantages described in this summary and the following detailed description are not all-inclusive, and particularly, many additional features and advantages will be apparent to one of ordinary skill in the art in view of the drawings, specification, and claims hereof. Moreover, it should be noted that the language used in the specification has been principally selected for readability and instructional purposes, and may not have been selected to delineate or circumscribe the inventive subject matter, resort to the claims being necessary to determine such inventive subject matter.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram providing a high level overview of a system for regulating file access rates of processes based upon file type, according to one embodiment of the present invention.

FIG. 2 is a block diagram illustrating a system for managing an association table by an access rate regulator program, according to one embodiment of the present invention.

FIG. 3 is a block diagram illustrating a system for managing an association table by a modified loader program, according to another embodiment of the present invention.

FIG. 4 is a block diagram illustrating a system for regulating file access rate, according to one embodiment of the present invention.

FIG. 5 is a block diagram illustrating a leaky-bucket regulator according to one embodiment of the present invention.

The figures depict embodiments of the present invention for purposes of illustration only. One skilled in the art will readily recognize from the following discussion that alternative embodiments of the structures and methods illustrated herein may be employed without departing from the principles of the invention described herein.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

I. System Overview

FIG. 1 illustrates a high level overview of a system 100 for regulating file access rates of processes 107 based upon file type according to one embodiment of the present invention. A computer memory 101 includes user address space 103 and operating system address space 105. A process 107 executes in user address space 103. Although FIG. 1 illustrates only a single process 107 executing in user address space 103, it is to be understood that within a given computer memory 101, multiple processes 107 can execute simultaneously.

Preferably, a data structure for storing associations 129 between processes 107 and access rates for file types is inserted into the operating system 117. In one embodiment, the data structure is an association table 127, but in other embodiments other data structures are utilized, for example a linked list. In one embodiment, the association table 127 (or other data structure) is dynamically loaded into the operating system kernel 109, while the kernel 109 is active. In another embodiment, the association table 127 is stored in user address space 103. The maintenance and use of the association table 127 is discussed in detail below.

In order to regulate file access rates, system calls 115 that access files are intercepted. A system call wrapper 111 is utilized in order to intercept system calls 115. In one embodiment, the system call wrapper 111 is dynamically loaded into the operating system kernel 109, while the kernel 109 is active. In another embodiment, the system call wrapper is loaded in user address space 103. The system call wrapper 111 is preferably in the form of object code, the functional features of which are described in detail below.

Pointers 114 to system calls 115 are located in an operating system call vector table 113. It is to be understood that the term "system call vector table" as used herein denotes an area in operating system address space 105 in which there are stored the addresses of system calls. In the UNIX® operating system, this part of the operating system is called the "system call vector table," and that term is used in this specification. Other operating systems employ different terminology to denote the sane system component. A system call vector table by any other name is still within the scope of the present invention.

A copy 116 is made of a pointer 114 to each system call 115 to be intercepted. These copies 116 of pointers 114 are preferably stored in operating system address space 105, but in an alternative embodiment are stored in user address space 103. Once the copies 116 have been made and saved, the pointers 114 in the system call vector table 113 to the system calls 115 to be intercepted are replaced with pointers 118 to the system call wrapper 111, such that when a system call 115 to be intercepted is made, the system call wrapper 111 executes instead. In one embodiment, this copying, storing, and replacing of pointers is performed by the system call wrapper 111. In other embodiments, copying, storing, and replacing of pointers is performed by a pointer management module executing in either operating system address space 105 or user address space 103 as desired. The pointer management module can either be a stand alone program, or a component of a larger application program as desired.

Executing alternative code when a system call 115 is made comprises intercepting the system call 115. The steps of inserting a system call wrapper 111 into the operating system 117, making a copy 116 of an operating system pointer 114 to a system call 115, and replacing the operating system pointer 114 with a pointer 118 to the system call wrapper 111 facilitate interception of a system call 115. When a system call 115 to be intercepted is made, the operating system 117 uses the pointer 118 in the system call vector table 113 to the system call wrapper 111 to execute the system call wrapper 111.

It is to be understood that only system calls 115 that access files need be intercepted, and thus only pointers 114 to system calls 115 to be intercepted are replaced with pointers 118 to the system call wrapper 111. Pointers 114 to system calls 115 which are not to be intercepted are not replaced. Thus, when a non-intercepted system call 115 is made, the system call 115 executes, not the system call wrapper 111.

In one embodiment, a single system call wrapper 111 is loaded into operating system address space 105. Thus, whenever any process 107 makes a system call 115 to be intercepted, the system call wrapper 111 executes. In another embodiment, techniques described in the "Selective Interception of System Calls" application are used to associate a specific system call wrapper 111 with each process to be regulated.

Processes 107 execute in user address space 103 under control of the operating system 117, and make system calls 115. When a process makes a system call 115 that accesses a file, the system call wrapper 111 determines the type of the file being accessed. The system call wrapper 111 also examines the association table 127 to determine whether the process 107 that made the system call 115 is associated with an access rate for that file type. If so, the system call wrapper 111 regulates the file access rate appropriately. Otherwise, the access is allowed to proceed normally. Note that even in the embodiment in which processes 107 to be regulated are associated with individual system call wrappers 111, the system call wrapper still determines if the process 107 is associated with a rate for the file type being accessed, because an individual process 107 may be regulated for some file types and not for others.

II. Storing Associations by a Regulator Program

FIG. 2 illustrates one embodiment of a system 200 for regulating file access rates of processes 107 based upon file type. In the embodiment illustrated by FIG. 2, the association table 127 is managed by an access rate regulator program 201.

The access rate regulator program 201 modifies the operating system 117 of the computer to include the association table 127. Preferably, the regulator program 201 loads the association table 127 into the kernel 109 of the operating system 117 while the kernel is active.

For each process 107 to be regulated, the regulator program 201 stores, in the association table 127, an association 129 between the process 107 and an access rate for at least one file type. For example, suppose a process 107 associated with a financial services host is to be regulated to 1,000 bytes per second for files stored on media, and to 1,400 bytes per second for communication channels. The regulator program 201 would store two associations for the process 107, the first indicating the access rate for files stored on media (1,000 bytes per second), and the second the access rate for communication channels (1,400 bytes per second). Of course, the regulator program 201 can add, modify, and delete associations 129 from the association table 127, as desired.

III. Storing Associations by a Loader Program

FIG. 3 illustrates another embodiment of a system 300 for regulating file access rates of processes 107 based upon file type. In the embodiment illustrated by FIG. 3, processes 107 to be regulated are loaded by the modified loader program 301, which also stores the associations 129.

A loader program is an operating system utility that is used to execute computer programs that are stored on static media. Typically, a loader program loads an executable image from static media into user address space 103 of computer memory 101, and then initiates execution of the loaded image by transferring execution to the first instruction thereof.

Like a standard loader program, the modified loader 301 loads executable images from static media into user address space 103. Additionally, the modified loader program 301 stores, in the association table 127, at least one association 129 between the process 107 and a file access rate. Thus, whenever a process 107 to be regulated is loaded, the association table 127 is updated as needed.

In other embodiments, in addition to storing associations, the modified loader program 301 uses methodology described in detail in the "Selective Interception of System Calls" application in order to associate an individual system call wrapper 111 with the loaded process 107. In different embodiments, the system call wrapper 111 associated with the process is loaded into process address space of the process, user address space 103, or operating system address space 105 as described in the "Selective Interception of System Calls" application. In each of these embodiments, whenever the process 107 makes a system call 115 to be intercepted (i.e., a system call 115 that accesses a file) the system call wrapper 111 associated with the process 107 executes, and proceeds to manage the regulation of the file access rate. Regulation of file access rate by system call wrappers 111 generally is discussed in detail below.

IV. Determining the File Type

When a system call wrapper 111 executes, the system call wrapper determines the type of the file being accessed. Once the file type has been determined, the system call wrapper 111 examines the association table 127 to determine whether the process 107 that made the system call 115 is associated with an access rate for that file type. If so, the system call wrapper 111 regulates the file access rate appropriately. Otherwise, access of files of that type by the process 107 is not to be regulated, so the access is allowed to proceed normally. In one embodiment, the determination is made using methodology described in the "Disambiguating File Descriptors" application. Other techniques such as file extension or file name can also be used as desired.

V. Regulating the Access Rate

FIG. 4 illustrates one embodiment of a system 400 for regulating file access rate. After determining the file type being accessed, the system call wrapper 111 executes a file access rate regulation module 401 in order to regulate the access rate of the file 403 by the process 107. In one embodiment, the file access rate regulation module 401 executes in operating system address space 105, as illustrated in FIG. 4. In another embodiment, the file access rate regulation module 401 executes in user address space 103. The file access rate regulation module 401 can either be a section of the system call wrapper (as illustrated), a stand alone program, or a component of a larger application program as desired. In any case, the file access rate regulation module 401 applies a known rate regulation technique to the access of the file 403 by the process 107. More specifically, data being written to the file 403 by the process (input to the file 405) is routed through the file access rate regulation module 401. The file access rate regulation module 401 controls the rate at which the input 405 is sent to the file 403 by the process. Likewise, data read by the process 107 from the file (output from the file 407) is also routed through the file access rate regulation module 401, which controls the rate at which the output 407 is sent to the process 107.

Many techniques for regulating access rate are known. In one embodiment, the file access rate regulation module 401 uses a leaky-bucket regulator to regulate the access rate. A leaky-bucket regulator is one specific example of a technique for regulating access rates. A leaky-bucket regulator is described in An Engineering Approach to Computer Networking, Srinivasan Keshav, 1997, Addison Wesley, Chapter 7, at pp 403-405, which is incorporated herein by reference.

FIG. 5 illustrates a leaky-bucket regulator 501. As illustrated in FIG. 5, the leaky-bucket regulator 501 stores fixed sized tokens 503 in a data structure known as a token (or leaky) bucket 505. Each token 503 constitutes permission for the regulator 501 to send a certain number of bits to a destination. The regulator 501 adds tokens 503 to the bucket 505 at a fixed rate, R. The bucket 505 overflows if the number of tokens 503 crosses some threshold, called its depth, D.

Data arrives at the regulator 501 as input 507 from a source. A test module 509 examines the bucket 505, to determine the sum of the sizes of the tokens 503 therein. The regulator 501 only sends the data as output to a destination 511 if the sum equals or exceeds the size of the data. When data is sent as output 511, the regulator 501 removes tokens 503 corresponding to the output 511 size from the token bucket 505.

If the sum is less than the size of the input 507, the input 507 is held as stored data 513 in a data buffer 515. Once the bucket 505 contains sufficient tokens 503, the stored data 513 is sent as output 511.

A leaky-bucket regulator 501 limits the size of a burst of output 511 to a little more D. The size can be slightly greater than D because tokens 503 may be added to the bucket 505 while output 511 equal in size to D is being sent. Over the long term, the rate at which output 511 is sent by the regulator 501 is limited by the rate R at which tokens 503 are added to the bucket 505. Thus, the leaky-bucket regulator 501 can be used to regulate a process 107 to an access rate for a file type by setting R to that access rate.

For example, suppose the system call wrapper 111 determines that the file access write for a process 107 is 1,000 bytes per second. The system call wrapper passes the rate to the leaky-bucket regulator 501, which proceeds to set R to the rate. Thus, the regulator 501 adds 1,000 byte tokens 503 to a bucket 505 at a rate of once per second. The system call wrapper 111 then routes the communication between the process 107 and the file through the regulator 501, which regulates the access rate to 1,000 bytes per second.

Note that if the process 107 is writing data to a file 403, the input to the file 405 becomes the input 507 to the leaky bucket regulator 501, and the output 511 from the leaky bucket regulator 501 is sent to the file 403. On the other hand, if the process 107 is reading data from a file 403, the output from the file 407 becomes the input 507 to the leaky bucket regulator 501, and the output 511 from the leaky bucket regulator 501 is sent to the process 107.

In other embodiments, other access rate techniques are employed by the file access rate regulation module 401, for example moving average, jumping average, or peak rate.

As will be understood by those familiar with the art, the invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. Likewise, the particular capitalization or naming of the modules, protocols, features, attributes or any other aspect is not mandatory or significant, and the mechanisms that implement the invention or its features may have different names or formats. Accordingly, the disclosure of the present invention is intended to be illustrative, but not limiting, of the scope of the invention, which is set forth in the following claims.

«Previous Next»
Method and system for access and display of data from large data sets On-demand service expanding system and method for providing services