Internet profiling

Abstract

A system, method, and various software products provide for consistent identification of web users across multiple web sites, servers and domains, monitoring and capture of data describing the users' web activities, categorization of the web activity data, aggregation of the data into time dependent models describing interest of users and groups over time. Categorization is made with respect to a category tree which may be standardized or customized for each web site. User groups may be defined based on membership rules for category interest information and demographics. Individual user profiles are then created for users automatically based on satisfaction of the user group membership rules. As new data is collected on a user over time, the category interest information extracted from the user's web activity is updated to form a current model of the user's interests relative to the various categories. This information is also used to automatically update group membership and user profile information. Identification of users across multiple sites is provided by a global service that recognizes each user and provides a globally unique identifier to a requesting web server, which can use the identifier to accumulate activity data for the user. Client side user identification is provided to track user activity data on web servers that do not communicate with the global service and do not process activity for category information. User profiles may be shared among web sites that form alliances. User activity data may be aggregated along various dimensions including users/user groups, categorization, and time to provide robust models of interest at any desired time scale.

Claims

We claim:

1. A system for profiling users of online information systems, comprising:

a first web server that receives requests from a user for web content items and records web events for selected web content items,

wherein each web event comprises content identification information identifying a web content item, time information describing an amount of time the user interacted with the web content item, and category relevance information for a plurality of categories about the web content item;

an aggregation service that aggregates the plurality of web events recorded by the first web server with at least one web event obtained from a source remote from the first web server into aggregated information along at least one of a plurality of categorization dimensions;

a user group service for defining a plurality of user groups, each user group having a definable membership rule which can be evaluated with respect to at least one category based on category relevance information from web events aggregated by the aggregation service;

a profile service that determines for each user a user profile, the user profile specifying at least one user group, defined by the user group service, of which the user is member;

an alliance service for sharing user profiles with one or more remote web servers participating in an alliance;

a client application resident on a computer used by the user that records web events for selected items of web content requested from a second web server that does not itself record such web events, each web event recorded by the client application containing information identifying the web content item and time information describing an amount of time the user interacted with the web content, where the client application uploads the recorded web events periodically to a global upload service; and

a global upload service, remote from the first web server, that maintains information for each user indicating one or more web servers that subscribe to receive web events for the user, receives the web events from the client application of a user, and provides the web events to the web servers that subscribe to the user's web events.

2. A system for profiling users of online information systems, comprising:

a first web server that receives requests from a user for web content items and records web events for selected web content items,

wherein each web event comprises content identification information identifying a web content item, time information describing an amount of time the user interacted with the web content item, and category relevance information for a plurality of categories about the web content item;

an aggregation service that:

an aggregates the plurality of web events recorded by the first web server with at least one web event obtained from a source remote from the first web server into aggregated information along at least one of a plurality of categorization dimensions;

scales the category relevance information of a web content item as a function of the amount of time the user interacted with the web content; and

transforms the category relevance information for each category into a weight, the weight being a function of a category score, the amount of time the user interacted with the web content item, a scaling factor, and a total time scaled category score for all categories;

a user group service for defining a plurality of user groups, each user group having a definable membership rule which can be evaluated with respect to at least one category based on category relevance information from web events aggregated by the aggregation service;

a profile service that determines for each user a user profile, the user profile specifying at least one user group, defined by the user group service, of which the user is member;

an alliance service for sharing user profiles with one or more remote web servers participating in an alliance;

wherein each category receives a scaled category score:

NewScore.sub.i =Category Score.sub.i *Duration.sub.i *Constant

where Duration.sub.i is the amount of time the user interacted with the web content item, and Constant is a scaling factor; and

wherein each category receives a Weight: ##EQU2##

Description

BACKGROUND

1. Field of the Invention

The present invention relates to the analysis of the behavior and interests of users of online networks, and more particularly to the analysis and modeling of user's interests for users of the Internet and World Wide Web.

2. Background of the Invention

In any market, customer behavior is important. This is true of traditional retail businesses, where there are well developed mechanisms for determining customer's interests. In brick-and-mortar businesses, the customers of the business can be observed by watching those customers walk through a store. Customer behavior can also be observed by tracking their purchases (e.g., through credit card purchases.) Customer observation is, in fact, an important technique used by many retail businesses. It is so important that major databases of customer behavior exist and are in continuous usage. For example, many supermarket chains have vast databases of customer behavior. Analysis of the data in such databases can be used for many purposes (e.g., inventory control, product placement, new product analysis).

Understanding customer behavior is also necessary for electronic commerce, but the techniques of observing the customer in this medium are necessarily different. The way that customers interact with an e-commerce web site is radically different from the experience of walking into a business in person and making a purchase, but many things remain the same. When Web visitors browse a web site, sometimes they buy, and sometimes they do not. Businesses are very interested in knowing why visitors buy and why they don't. So these new electronic merchants want to understand their prospects and their customers. These businesses must observe their web visitors. This observation leads to the need for modeling the interests of customers over time, the need for managing the tremendous amount of data that such modeling would entail, and the need for categorizing web content to providing for meaningful models of user interests.

Conventionally, observation of users in online systems has typically involved using user-provided information about users interests, such as surveys or forms that allow the user the identify the categories of information that are important to them. Examples of this approach include the various customizable home pages offered by search portals such as Yahoo and Excite. In these portals, users can select various predefined categories of interest, and relevant news and related data is then provided to the user. If however the user's interests change over time, the user must manually change the specified categories of interest; this is not done automatically. These sites also allow users to specify their interests with simple keywords, but again, if the interests change, the user must manually change these keywords.

Other web sites more systematically track user behavior in terms of clickthroughs and page views, and then assemble information about these activities. As the user's activity changes on this particular web site, the assembled information is updated. This approach, while capturing some aspects of change in user behavior, it typically limited to only identifying interests relative to a single web site. User behavior on other web sites does not effect the particular site's assembled information, even though such remote behavior may most accurately express the user's interests. More particularly, the analysis of user behavior is typically limited to the particular Internet domain of the server that tracks the usage. User activity at another domain is not tracked.

Further, the assembled information on such a server only expresses the user's interest without respect to potential future or past interests. That is, it does not model changing user interests over time. However, it is the change in user interest over time that is of significant value to web marketers and others attempting to deliver content to web visitors.

SUMMARY OF THE INVENTION

The present invention overcomes the limitations in the prior art by providing a system and methodology, and various software products that tracks user activity across multiple domains, and from such activity develops a time based model that describes the user's interests over time. The changing user interests are also used to determine each user's membership in any number of defined user groups. Each user's time based model of interests and group memberships forms a detailed profile of the Internet activity that can be used to market information and products to the user, to customize web content dynamically, or for other marketing purposes.

Thus the present invention fulfills an important need: to identify web visitors and understand their interests over time. The present invention, sometimes referred to herein as "ProReach" or "ProReach system" is a software system that tracks and analyzes web visitors on the World Wide Web. In short, it helps turn web visitors into web customers. The present invention has the following features and aspects.

First, the present invention can identify and monitor a web visitor as he visits a web site. Of course, on the internet there are many web sites, and there would many web visitors. Whether two web sites or thousands of web sites are involved, or there are millions of web visitors, the present invention provides a system which can identify many visitors across many web sites. Thus, in this aspect, the present invention identifies each visitor to a web site, with unique identification information. This allows the visitor to be consistently identified, during both multiple visits to the same web site, and during visits to other web sites.

ProReach combines data from many web activities to get a more complete picture of a web visitor. ProReach is able to combine the data from these different web sites because the visitor identification process works across the web. This simply means that when a web visitor goes from place to place on the world wide web, ProReach can repeatedly and consistently identify the web visitor typically. More specifically, in contrast to other web tracking products, the ProReach System collects data on both the web server and the web client. ProReach does the latter by providing downloadable software that web clients can install on their systems. Once installed, this software tracks the web user's actions from his machine. Each time he visits a web site, his actions are recorded. Periodically, a compact version of this data is uploaded to ProReach, and then distributed to other web sites which maintain profiles and user group information relative to the user.

Accordingly, the user's activity at each web site there is monitored to identify items of web content with which the user interacts, such as page views, purchases, and so forth. The monitoring may be done by the web server itself, or by the client side software. This monitoring includes identifying each item of web content, such as with its URL or URI, along with information about how long the user viewed the content. This is beneficial because web activities that take longer --such as reading a web page --reflect a higher of interest by the user. The data of a user's specific interaction with an item of content is stored in a web event record. (Certain web activities, e.g., simple, fast clickthroughs may not be tracked in a web event record because they do not useful reflect a user's interest.) This process of identifying web visitors and monitoring the web content they interact with occurs automatically and continuously. Over time then, a large number of web event records will be generated resulting from the activities of many web users at many web sites.

Once data of a web visitor visit to a web site is gathered, this data is not yet in a form that is particularly helpful to making business decisions. For example, it is not particularly helpful to know that some web visitor has viewed hundreds of web pages at a dozen web sites. Rather, it is more useful to understand what kinds of things did the web visitor look at: Motorcycles? Cosmetics? News? Technical information? Music CDs? Books?

Ideally, every document on the World Wide Web would be associated with a description that would describe briefly what that document was about.

That is, this description would categorize that document, much in the way in which books are categorized in a library. Such an ideal is never going to be a reality any time soon, if ever. So there needs to be a way to automatically categorize the documents that a web visitor sees. This categorization technique should be robust, accurate and maintainable.

The ProReach system provides just this capability. It uses a content recognition engine to do this. A content recognition engine is a software component that can take a document and a set of categories and compute how closely the document matches up with these categories. Using the content recognition engine, the ProReach system can categorize various kinds of web document, and provide a ranked list of categories, including hierarchical categories that pertain to the document. The basic idea is that the content recognition engine evaluates some number of categories that may or may not match up with a given document. The content recognition engine tests the document and returns a score as to how closely it matches with each category. During this process, the document gets tested against many categories, so the resulting categorization is really a vector of categorization scores. Each categorization score of that vector shows how well that document matches up with a given category, such as sports, news or computers.

Accordingly, each web event record is processed to determine its relevance to various defined categories. The categories are maintained in a category tree which covers a wide range of categories and topics. Preferably the web content is scored with respect to each category to indicate to the degree to which the content may be said to be about category. This categorization takes place automatically, without requiring action by a webmaster or system administrator.

The categories themselves used as part of the categorization process are part of the data that are provided to the content recognition engine. ProReach preferably provides turnkey categories, allowing the system to categorize web content as soon as ProReach is installed and running on a particular web site. In one embodiment, the turnkey categories are provided from a central host system that is in communication with a particular local ProReach system installation The host ProReach system provides a comprehensive set of categories that target the practical information needs of e-businesses, and it provides sample data for these categories.

As an optional capability, ProReach system users can modify categories, or create their own. In this way, a web site using the ProReach system can categorize the viewing habits of its prospects and customers in a custom fashion. They can create new kinds of categories. This customization is optional. They are not required to do this. ProReach is a turnkey system that is customizable. It is not a system that requires customization to be used. ProReach also provides other tools to assist in the process of category creation and maintenance.

The data about a web visitor's activities is valuable, but ProReach can distill more meaning from this data. Electronic commerce decision makers are interested in the psychographic and demographic profile of the user. They do not want every single detail of the user's activities, but rather a summary of the user's interests which is abstracted from the details of the user's activities. It therefore becomes very desirable that all the detailed data of the user's activities can be compressed into a highly meaningful summary. Accordingly, the present invention further processes this information to develop detailed Internet profiles of each user, and of different user groups and categories of information.

The ProReach system of the present invention creates summaries of a web visitor's activities via a process of web activity aggregation. Through this process, the ProReach system automatically takes the previous history of a visitor's activities and integrates this with data collected from new visits. This process of taking new visits and integrating them with previous visits is performed on an as-needed basis. In this way, the profile of a web visitor is always kept up to date, reflecting that web visitor's interests.

More specifically, ProReach aggregates web visitor's web activity data on three dimensions --on who they are (identity), what they did (content categorization) and when they did it (time). This process is called dimensional combining. Along these three dimensions, ProReach provides sophisticated, statistical-based aggregation.

Another strength of the ProReach system is its flexible approach to aggregating a visitor's activities. Different kinds of e-commerce businesses will want to summarize their visitor's activities in different ways. This is because different companies have different needs for understanding the nature of their customers. Accordingly, aggregation may be tuned to the needs of a particular business.

Hence the ProReach system provides excellent aggregation capabilities that can then be tuned by ProReach system administrators. It allows parameters to be set that control the aggregation process. Power and flexibility are combined. These parameters control what information is maintained and the amount of storage allowed for its maintenance.

In this aspect of the invention then, the web event records accumulated at a given web server are first aggregated into a set of aggregated results for each web user at the site, preferably on a periodic, fixed basis, such as a daily basis. Thus, a user may visit a particular web site several times a day, each time generating dozens of individual web event records. The same is true for many different users. Accordingly, for each user, the web event records are combined to collect all of the categorization information for that user together. In addition, the category score information in each web event record is processed to reflect the duration of the web activity. This processing results in a set of category weights.

The combined category weighting information for the collected period, such as a day, describes in detail the user's degree of interest across a number of categories. However, further processing is beneficial to obtain a more summarized model of the user's interests. Thus, from the weighted category information various statistical measures are derived such as the mean category weight over the period, maximum and minimum weights, standard deviation, and the like. In addition, a trend pattern is also extracted which described whether the user's interest in the category is increasing, decreasing, or constant, or some combination of these, over the time period. This summarized representation of the category weights for the time period can be stored, and best captures the changes in the user's interest, across a number of categories, over the time period. As a result, the underlying raw data of the web event records deleted, so that storage efficiency is achieved.

First, the period information may be aggregated for each user with respect to each of the categories across a longer time period. For example, the daily aggregated information for a user may be further aggregated for a week's time period, a month, a quarter, a year and so forth. This forms what is termed a user-category complex, wherein the statistical information for a single category from many different days is combined by an aggregation function. One exemplary aggregation function is mean, and thus the mean of the category weights for this particular category over the time period is obtain, along with trend pattern and other statistical measures.

Second, dimensional combining may be used to form category complexes. A category complex summarizes a large number of users' interests in a particular category over a selected time period. This complex describes the level of interest, over time, for a population of users in a particular category.

Another type of dimensional combining now makes use the user-category complexes. First, the many user-category complexes for an individual user may be combined for a selected time period, to form an aggregated view of the user's overall interests. That is, the category information from many different categories is aggregated and describes the user's interests overall.

Additionally, the user-category complexes may be combined for an individual category and across selected users who form a user group, to create user group-category complexes. The user group members are selected by having meet certain membership tests based on their category interests and optionally demographics. This gives a summary of the user group's interest in that category over time.

The user complexes can be further combined into user group complexes to describe overall group interest across all categories. Finally, the group complexes may be aggregated to form an overall total complex which describes the total population's interest across all categories for the selected time period.

In addition to the various complexes that may be aggregated, individual profiles of the users can be further augmented with the user group information. A number of user groups may be defined, each having particular membership criteria. Marketers can define groups of users that share interests, buying propensities or demographics. The criteria are preferably based on a user having (or not having) particular levels or ranges of category weights for one or more categories. A user may be member of multiple user groups. The group membership is automatically updated, as the users interacts with web content over time, and as their interests change as expressed by the changing levels of categories weights. The ProReach system will automatically classify a user into the right user groups based on his or her profile. If the definition of the user groups changes, then the ProReach system will automatically re-classify users into the right user groups. Similarly, as the interests of user change, they will automatically be put into the right visitor segments based on their new interests. In this way, a marketer has immediate access to market segments on demand, and can swiftly apply electronic sales campaigns.

The visitor profile information that ProReach systems generate can be retained for the sole use and benefit of the web site that created it. It also possible for ProReach systems to share their user profile information. To facilitate this sharing, ProReach provides a centralized service that helps ProReach systems define policies for the transfer of information between each other. For ProReach customers that want a deeper relationship with each other, the present invention provides for an alliances. An alliance is a group of ProReach systems who have decided to contribute their user profiles into a database of profiles. All members of the alliance contribute profiles, and all members of the alliance benefit by getting a degree of access to the alliance profiles. In particular, alliances are useful to vertical markets where companies may want to work together on the world wide web. Such groups of businesses may benefit from combining their information, but they need the infrastructure to facilitate this sharing, regulate it and make it safe. ProReach provides this enabling infrastructure. In an alliance, each member contributes visitor profiles created for visitors to the member's web sites. These contributed profiles are aggregated together in a database of profiles maintained by the alliance. All members to the alliance get controlled access to these profiles. A system of sharing rules controls this whole sharing process, so that companies only share selected information. ProReach supports the formation of multiple alliances. An ProReach-enabled system can belong to more than one alliance.

A very large amount of visitor activity data will be generated by web sites using ProReach systems. The existence of this data raises privacy concerns. It also raises issues about how ProReach Systems themselves share data amongst themselves. ProReach has an architecture that addresses privacy concerns. ProReach ensures the privacy of web visitors via what it calls an identity firewall. The purpose of an identity firewall is to establish a boundary. Inside the boundary of the identity firewall, the identity of a web visitor is accessible to authorized personnel or processes. Other personal information is also available, such as e-mail address, home address and age.

Outside the boundary of the identity firewall, no data is provided that could be used to identify a web visitor. Instead, any person or process requesting information outside an identity firewall, only gets an opaque visitor identifier. The ProReach System that issues the opaque visitor identifier can use it to uniquely identify the web visitor. Hence, an opaque visitor identifier is an externalizable reference to ProReach visitors.

A person or process with an opaque visitor identifier can present the opaque visitor identifier to that ProReach System. The ProReach System can then map that opaque visitor identifier back to the actual visitor. Using this method, it is possible for a web marketer, for example, to be given a large amount of information about the interests of a web visitor but the marketer doesn't know the visitor's identity or contact information. The web marketer is simply given an opaque visitor identifier (or a set of such identifiers). The marketer gets the data he needs, but the privacy of the visitor's data is maintained. So outside the identity firewall, a web visitor being tracked by ProReach is anonymous.

The web marketer may have the ProReach system contact the web visitor on his behalf using IPro's Visitor Contact Service. Given an opaque visitor identifier and a message, the Visitor Contract Service looks up the e-mail address (or other necessary information). It then sends the message to the web visitor. The web marketer gets his message delivered to the web visitor, but the web marketer does not know the web visitor's identity.

Identity firewalls can be flexibly configured. They can be configured so that the identity firewall encloses a single ProReach System. They can be configured so that an identity firewall encloses a group of ProReach systems. The latter configuration would make sense when there are multiple ProReach servers working as a group (e.g., for a portal with multiple servers) and data should be shared between the servers.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates the features of a web event.

FIG. 2 illustrates data flow in the process of aggregating web events and creating user profiles.

FIG. 3 illustrates a top level system architecture of various ProReach systems.

FIG. 4 illustrates the hub and spoke architecture of a ProReach system.

FIG. 5 illustrates an embodiment of a ProReach system operating with a firewall.

FIG. 6 illustrates the Global Services server.

FIGS. 7a-7f illustrate the overall processing flow of a ProReach system.

FIG. 8 illustrates an alliance of ProReach systems.

FIG. 9 illustrates the aggregator queue used to store web event records.

FIG. 10 illustrates the features of the aggregator service.

FIG. 11 illustrates the processing function of the parser.

FIG. 12 illustrates the concept of a category interest curve.

FIG. 13 illustrates the root portion of central category tree.

FIG. 14 describes the process of updating the standard category tree.

FIG. 15 illustrates the operation of the content recognition engine.

FIG. 16 illustrates the process of customizing content based on a user profile.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

TABLE OF CONTENTS

I. WEB EVENTS AND AGGREGATION

A. WEB EVENT RECORDS

II. OVERVIEW OF PROREACH SYSTEM ARCHITECTURE

A. GLOBAL SERVICES

III. BASIC SYSTEM PROCESSING

A. PROREACH FUNCTIONAL OVERVIEW

B. CATEGORY DISCOVERY AND MAINTENANCE

1. Category Discovery

2. Category Maintenance

IV. PROREACH SYSTEMS WITH ALLIANCES

V. AGGREGATION

A. AGGREGATING DAILY WEB EVENTS

1. Transform Category Scores to Weights

2. Restructure Web Event Records to Collate Category Weights by User

3. Create Category Interest Time Model Information

B. DIMENSIONAL COMBINING

C. USER GROUP SYSTEM

D. DAILY AGGREGATION

E. AFFINITY GROUP MANAGER

F. THE UPDATE OBJECT

G. SCHEDULER

H. EVENT DISPATCHER

I. PROFILE SYSTEM

J. AQL SYSTEM

1. AQL Language

2. AQL Interpreter

VI. CATEGORIES AND CATEGORIZATION

A. OVERVIEW OF CATEGORIZATION

B. CATEGORIES AND HIERARCHIES ORGANIZE DATA

1. Building and Maintaining Category Hierarchies

C. CATEGORY NAMES AND ID'S

1. Default Unalterable User Category Structure

2. Similarities and Differences Between Categories and Groups

D. USING SOURCE OR LOCATION IN CATEGORIZATION

E. THE CONTENT CATEGORY LIFECYCLE:FORMATION, TUNING, AND CHANGE

1. The Standard Category Tree and Additions by ProReach System Administrators

a) Adding Categories At ProReach systems

b) Updating the Standard Category Tree

c) Building the Standard Category Tree

d) Discovery, Refinement, and Editing of Categories

F. CATEGORIZATION MODEL OF THE CONTENT RECOGNITION ENGINE

1. Category Creation

2. Document Categorization

3. Multiple Dictionary Categorization

4. Category Cache

VII. GLOBAL SERVICES

A. GLOBAL IDENTIFIER SERVICE

1. Requests For GIDs.

2. Individual Identification via PIDs

B. GLOBAL UPLOAD SERVICE

C. GLOBAL CLIENT MANAGEMENT SERVICE

D. YELLOW PAGES

E. GLOBAL EXCHANGE POLICY

VIII. PROREACH CLIENT SIDE WEB USAGE DATA COLLECTION

A. WEB ACTIVITY MONITORING

B. PROREACH CLIENT WEB USAGE DATA FILTRATION AND AGGREGATION

1. Time-based consolidation

a) Adjust web event record time stamps

b) Ignore short-term activities

c) Aggregate Web activities

2. Other Filtration of Data

3. Privacy Control

C. FILTRATION BASED ON PRIVACY SETTINGS (USER MODIFIABLE)

1. URL pattern-based filtration

2. Keyword-based filtration

D. DEFAULT PRIVACY-RELATED FILTRATION

E. PROREACH CLIENT DATA UPLOAD

1. ProReach client upload queue

2. ProReach Upload Stream and Upload Record

3. Data upload

a) Web Event Record upload

b) Homepage URL upload

4. Upload time and upload stages

a) Pre-upload stage

b) Upload stage

c) Post-upload stage

5. ProReach Upload Service and upload

IX. CONTENT TARGETING

A. ACCESS TO PROFILE BY A CGI

1. Access to page Metadata by CGI

a) Handling dynamic content categorization of multipart pages at runtime

I. Web Events and Aggregation

Referring now to FIG. 1, there is shown an illustration of the concept of a web event, which is used as a basic modeling unit for measuring the interests of web visitors in web content. A web event 101 is the combination of three different types of information. First, a web event 101 contains information which uniquely identifies the particular web visitor 103, or generically a "user." This user can be an individual person, or any group of persons to which the user is deemed to belong. Second, a web event 101 includes information which describes or measure 107 the amount of time spent by the web visitor in viewing or interacting with the web content.

Finally, the web event 101 includes information that identify one or more categories 105 into which the web content visited by the web visitor belongs and a measure of the user's interest in each of the one or more category. The categories used to describe the web content preferably form a hierarchy of categories, with parent categories (e.g., "Sports") having multiple child categories (e.g., "Soccer" and "Golf").

These three pieces of data are used model the basic idea that a user viewing or interacting with an item of web content is expressing an "interest" in whatever category or categories that web content is about. The longer the visitor views or interacts with the content, the greater the visitor's interest is presumed to be (other factors may also be used to scale the level of interest, such as the type of interaction, e.g., a simple viewing of a page versus a purchase).

This measure of interest in of a user in a category at a particular time or duration is expressed as a weight. A weight is a function of the amount of time spent by the visitor interacting with an item of web content, and the degree to which the category is deemed to describe the content. In a preferred embodiment where there are a number of categories available, a web event includes a weight for each category. This reflects the fact that a given item of web content may relate to many different categories in different degrees.

To provide a meaningful scale of interpretation of these weights, and hence a level of interest in a category, the weights are scaled to a standard unit called an interaction unit. An interaction unit is interpreted to mean 1 minute of attention paid by a user to an item of content. By scaling web events using interaction units, it becomes possible to meaningfully compare the interests of any variety of different users and categories of web content.

These three types of information are collected for each item of web content viewed by a web visitor at a particular web site, and by extension by multiple different visitors across many web sites. For example, as the visitor moves from one web page to another on a given web site, a web event is generated which encapsulates the information identifying the visitor, the category description of the page, and the amount of time spent by the visitor on the page. As the same visitor visits different web sites, they are identified and web events which capture the category of content and time spent viewing such content are generated.

In themselves, web events are merely individual data items, and do not directly describe the overall patterns of interest of any individual user or groups of users, or patterns relatives to categories or time. This level of abstraction is provided by a second aspect of the present invention, aggregation. Most generally, aggregation is the process of summarizing the weights of different groups of web events to establish patterns of interest. Generally, web events can be combined with respect to time periods, individuals users, groups of users, categories, or groups of categories, or any combinations of these. When considered together, there are six different ways to combine web events:

1) Combine all web events between two dates: This combination approach combines web events related to all categories and all users over a given time period to provide a model of the global interests of the population of users.

2) Combine all web events for a category between two dates: This combination combines the web events for a specific category (or group of categories) for all users over a given time period to provide a model of the user's level and pattern of interest in the specified category.

3) Combine web events for a user and a category between two dates: This combination combines the web events of specific user and a specific category over a time period to provide a model of the user's level and pattern of interest in the specified category.

4) Combine web events for a user group and a category between two dates: This combination provides a model of the group's interest in the specified category.

5) Combine web events for a specific user between two dates, across all categories. This combination provides a description of how the overall distribution of a user's interests for all categories, whether narrowly interested in one or a few categories maintained a web site narrow or broadly interested in many of the categories at the web site.

6) Combine web events for a user group between two dates, across all categories.

In one embodiment, when performing these various types of combinations, the events selected during a given time period are thus which start during the time period, even if they end after the selected time period.

We call the process of combining web events in these various ways "dimensional combining", since there are six "dimensions" in the data along which web events may be combined. These possible combinations can be used to provide an analysis of any user's or group's interest in any category or categories over any time period. Referring now to FIG. 2, there is shown an illustration of these various ways of combining web events.

In FIG. 2 there is shown a number of discrete web events 101, occurring over some period of time, such as a number of days. All of the web events 101 for each day are aggregated into user specific, daily aggregated results 201. These daily aggregated results 201 form what is labeled as Level 0 of the figure. To obtain an understanding of the web visitors' interests, the web events over some number of days (e.g., week, month, quarter, year, etc.) are combined in different ways, as discussed above.

First, in Level 1, the daily aggregates can be combined per (3) above into "UC" or User-Category complexes 203, or per (2) above into individual "C" or Category complexes 205. Note that a single daily aggregated result 101 may contribute to either of these complexes; that is, the results of a particular web visitor's web activity contributes to both the Category complexes 205 for all categories effected by that visitor's activity, as well as to that user's specific user-category complexes 203 describing that user's level of interest in the various categories.

Next in Level 2, the individual UC complexes can themselves be combined. First, per (4) above, the particular UC complex for certain users who form a user group can be combined into "GC" or Group-Category complex 206. This complex 206 describes the group's interest in the particular category for the data. Second, per (5) above, all of the User-Category complexes for a particular user can be combined to form a single "U" or User complex 207, summarizing the user's interests across all of the categories. The User complex 207 is particularly useful to gauge the breadth or narrowness of user's interest. For example, a web site may have a limited number of categories of content. For one user of this web site, the user complex 207 may show a high level of interest in a just one or two categories, whereas another user's user complex 207 may show a high level of interest in a majority of the categories; this second user is like to be more valuable to the web site for purposes of marketing or other value driven activities.

Next in Level 3, the complexes 207 for individual users can be combined per (6) into "G" or Group complex 209 across all categories.

Finally, in Level 4, the complexes 209 from the many groups can be combined per (1) above into Total complex 211, describing the interests of all users across all categories.

This web event modeling and aggregation framework provides many advantageous features. First, it allows a system administrator (or a member of ProReach System) to arbitrarily select the time period over which any of these aggregations to obtain broader or narrow analyses of the time pattern of the users' interests. This is useful to identify very short term interest trends or longer term trends in users' interest. Second, because each level of aggregation fully captures the information of the level below it, the underlying web event data may be selectively discarded to improve storage efficiency. For example, web events for categories which have a very low level of interest (identified by a low weight) may be discarded after their data has been summarized into UC or C data. Web events with greater weight may be stored longer to allow them to be used for more analysis or marketing.

A. Web Event Records

When a web visitor performs a web activity, such as viewing the contents of a uniform resource locator, or clicking on a submit button that initiates a web transaction, this web activity is recorded by client-side or server-side trackers, which record this web event. The data of each web event is stored in a web event record. Web event records are then aggregated into the daily aggregated results 101, and from there into the various complexes. The basic features of a web event record are as follows:

                         Web Event Record
    Field         Explanation
    User ID       Uniquely identifies the visitor
    Location      The URL or URI of the web content.
    Start time    Onset of activity in Greenwich Mean Time for a single
                  event. If there are multiple events at this URL, then the
                  time of the earliest download.
    End time      Last recorded activity in Greenwich Mean Time for a
                  single event. If there are multiple events at this URL,
                  then the time of the last download. If unknown, a default
                  1 minute from the start time is used.
    Event type    Stores a value indicating the type of web activity, such as
                  view, clickthrough, purchase, and so forth.
    Event count   The number of times this URL/URI was downloaded
    Category Score The category scores for the content.
    For example, assume that a user's web activity is as follows:
      Activity    Start Time-End Time  URL         Duration
          1       10:05 am-10:10 am    <URL A>  5 min
          2       10:10 am-10:12 am    <URL B>  2 min
          3       10:12 am-10:14 am    idle
          4       10:14 am-10:15 am    <URL C>  1 min
          5       10:15 am-10:15:03 am <URL B>  3 sec
          6       10:15:03 am-10:16 am <URL A> 57 sec
          7       10:16 am-10:16:06 am <URL D>  6 sec
          8       10:16:06 am-         <URL A>  4 sec
                  10:16:10 am
          9       10:16:10 am-         <URL E>  6 min 20 sec
                  10:22:30 am
         10       10:22:30 am-10:30 am idle

    URL       Start-time  End-time    Duration          Occurence
    <URL A> 10:05 am    10:16:10 am 5 min 57 sec          2
    <URL B> 10:10 am    10:12 am    4 min *(see Note 2)     1
    <URL C> 10:14 am    10:15 am    1 min                 1
    <URL E> 10:16:10 am 10:22:30 am 5 min *(see Note 3)     1

                 Sports   Conser-            Recent   Planning For
     INTEREST   Enthusiast vative  Hobbyist  Divorcee  Retirement
      RESULT       YES      YES      NO        NO         YES

                                          Number of
                  Category                Documents
                Storage device                500
                   CD Rom                     80
                 Hard drives                 200
                 Zip drives                   40
                Floppy drives                100