Dynamically assigning a survey to a respondent6993495Abstract A method and apparatus for assigning a survey to a respondent are provided. According to one technique, a set of one or more screening questions are generated and provided to the respondent. Input data is received in response to the respondent answering the set of one or more screening questions. Based on how the respondent answered the set of one or more screening questions, it is determined which survey should be presented to the respondent. According to another aspect, the set of one or more screening questions is generated based on the target participation criteria that is associated with one or more active surveys. Claims What is claimed is: Description FIELD OF THE INVENTION
Tables enable network users to efficiently review and answer multiple items, reducing the amount of time and effort required to complete a set of questions or survey. By selecting the value in box 324, clients may choose the presentation format of a question. For example, selecting a value of "Radio Button" in box 324, the following question format is presented to the network user. Using the scale below, do you enjoy reading? Yes No The same question is presented to network users in the following format if the client selected box 324 to be "Drop Down." Using the scale below, do you enjoy reading?
Enabling clients to select presentation formats makes surveys more versatile, user-friendly, and interesting for network users. In certain embodiments, the client may include "branching" in defining the survey. The inclusion of branching in the survey allows for a different survey question to be presented based on previous response(s). For example, FIG. 4 illustrates example questions (402, 406, 410 and 414) that are associated with responses (404, 408, 412 and 416) respectively. By including branching in the survey, if in responding to question 402 a network user taking the survey selects the response option of "I do not read any newspapers" in response 404, then network user may next be presented with question 410 and with response 412. However, if the network user selects a response other than "I do not read any newspapers" then the network user may instead be presented with question 406 and response 408. At any point while the client is defining the survey, the client has the option of editing, changing, adding, deleting, or reordering any of the previously entered questions, question types, or responses. Reordering enables the client to alter the sequence of questions within the survey, potentially providing a more logical flow within the survey or improving the quality of data provided by network users to eliminate bias or other influences that may adversely affect the survey results. Clients have the ability to add multiple questions, one after the other, to define the survey. To help the client correctly define the survey, a "View Survey" option is contained on an interface that is provided to the client which allows the client to view how the completed survey will appear to a network user. In certain embodiments, the client can include graphical representations in defining the survey. For example, to enhance the appearance of the survey, the client may include pictures, drawings, sound, and animations. In certain embodiments, clients may include sound, video, or other "objects" that may engage network users or facilitate communication and interaction with network users. Clients may develop and save multiple surveys. Each survey may be identified and named by the client. The definition and content of each survey may be maintained persistently across client invocations or log-ins. Furthermore, existing surveys or parts of surveys may be copied, renamed, or modified to create new surveys—expediting the survey development process for experienced clients. Selecting a Target Group Once the client is satisfied with the developed survey, a target group of network users must be selected for fielding the survey. In one embodiment, to aid the client in selecting the appropriate target group, the client is provided with a list of predefined target or network user groups. For example, a list of predefined target groups may include: (i) children between the ages of 13 and 16 (ii) bicyclists (iii) women who are pregnant (iv) men over 45 (v) people who own computers In addition to the above examples, demographic, lifestyle, behavioral, and interest groups are provided to clients for selection and survey targeting. Clients may also define their own target group or select a representative sample from a myriad of populations. For the survey to be fielded, the client must specify the number of completed surveys they require (i.e. the number of network users who complete the survey) and the timeframe for obtaining those completed surveys. In certain embodiments, the client is provided with a default or suggested number of completed surveys and timeframes. In one embodiment, the default or suggested number of completed surveys is based on the number of completed surveys that is typically requested or required in the field of market research. In certain embodiments, the client is able to define a particular duration that the survey is to be "in the field." For example, the client may specify that the survey is to be fielded for two weeks. In another embodiment, the client may use a combination of completed surveys and duration to define how long the survey is to field. For example, the client may request that the survey field for six months or until 10,000 responses are received. As described in detail below, the number of completed surveys requested and/or the duration that the survey is required to field (among other factors), will typically affect the price of the survey. Providing Survey Result Data to the Client When a survey is fielded to a particular target group, the results are gathered, processed, and analyzed to generate survey data that reflects the survey results. To enable the delivery of the survey data, the interface allows the client to specify an address for receiving the survey data. Using the specified address, the survey data may be provided to the client in a variety of formats. For example, in certain embodiments the survey data is provided to the client as a comma separated value file (CSVF) that is attached to an e-mail message and sent over the network system to the specified email addressee. In another embodiment, the survey results are provided to the client in spreadsheet formats that include automated data processing, analyzing, and charting scripts, software, or computer instructions. In addition to delivering results, in certain embodiments, the client is provided with or allowed to access real-time results (i.e. information about the completed surveys received thus far). In one embodiment, real-time survey data is posted on a site connected to the network system that is accessible by the client. Pricing Information Prior to launching a survey, the client is presented with an interface indicating a price for fielding the survey and performing the requested service. Should the price be unacceptable, the client has the ability to iteratively alter the parameters of the survey to obtain an acceptable price. For example, FIG. 5 illustrates sample interface 500 that may be presented to a client. In this example, interface 500 contains a survey price field 502 which indicates the price of the survey as currently defined. Interface 500 also contains a change question button 504, a change target button 506, a change complete button 508 and a survey wizard button 510. By selecting the change button 504, the client is presented with an interface that allows them to add, delete and modify questions that are used in defining the survey. Selecting the change button 506 provides the client with an interface that allows them to change or modify the target group that has been associated with the survey. Selecting the change complete button 508 provides the client with an interface that allows them to change the number of completed surveys previously requested by the client. Selecting the survey wizard button 510 provides the client with a guided, step-by-step sequence that can be used to create a survey. Payment Information As previously stated, the interface provides the client with the ability to securely enter payment information. Once the client defines the survey, selects a target group and determines the number of completed surveys and/or a duration for fielding the survey, a survey price is calculated and provided to the client through the interface. Once a client accepts the price of the survey, the client selects a "pay for survey" button and enters pertinent payment information. For example, when a client chooses to proceed with the survey, an interface is provided to the client allowing the client to select a particular form of payment (i.e. credit card, debt card, etc.). In certain embodiments, a client is allowed to enter a customer account number that corresponds to an account or credit line that was previously established. Screening Requested Surveys Prior to launching a survey, the automatic survey mechanism has the ability to review and screen surveys to ensure their appropriateness for fielding to network users. In certain embodiments, an automated review of the survey is conducted by comparing components of the survey with a database of prohibited words, phrases, pictures, sounds, or themes. In one embodiment, surveys are automatically provided to an individual who is responsible for reviewing the appropriateness of the survey contents. Once a survey is approved, banners are launched and the survey is fielded. Generating the Survey After a client selects the pay button, enters valid payment information, and agrees to the terms and conditions associated with using the automatic survey mechanism, the automated survey mechanism determines the particular location (controlled by the automated survey system) where the survey will reside on the network. The automated survey mechanism then generates a survey based on the information (survey definition) supplied by the client and places it at the identified location. In certain embodiments, the generated survey consists of one or more HTML pages that are accessible over the Internet or Intranets to network users wanting to take the survey. In addition to generating and storing the survey, the automated survey mechanism generates response validation rules that are automatically enforced. These rules provide a mechanism for validating the responses from network users input as they participate in a survey. In certain embodiments, the response validation rules are linked to the survey to ensure the data provided by a network user is logically valid and consistent with questions in the survey. The response validation rules can be used to increase the quality of the survey data that is provided to the client. In certain embodiments, errors or logical inconsistencies that are identified are reported to the network user thus enabling them to take correct action (i.e. enter valid information). Additionally, the automated survey mechanism identifies and designates a specific storage unit for storing survey results. A storage unit is linked to each survey and is used to automatically capture survey results from network users who participate in the survey. In certain embodiments a unique storage unit is used to store the results for a particular survey. For example, when a survey is generated, a unique storage unit is identified and attached to the survey such that the results from a network user completing the survey are automatically stored into the unique storage unit. In certain embodiments, the storage unit is a database that can be used to store the survey results that were generated by participating network users. Fielding the Survey and Launching a Banner To attract network users of the desired target group to participate in the survey, the automated survey mechanism causes a banner or other recruitment device to be placed, launched, or made available for viewing on a one or more sites on the network system. In certain embodiments, the particular sites for viewing a banner associated with a particular survey are selected based on the target group information that was previously supplied by the client. For example, if the client specified that the target group for a particular survey would be kids 13-18 years of age, then sites associated with this target group are selected as banner locations for the particular survey. In one embodiment, the automated survey mechanism maintains a static mapping between banner locations and targeted groups. The automated survey mechanism uses this static mapping to identify one or more locations for launching a banner on the network system. For example, a static mapping may exist between a target group of kids 13-18 years of age and specific portions of the Yahoo! site or other "kid-related" locations. Based on this static mapping, those specific portions of the Yahoo! site may be selected as banner locations for a survey targeted for kids 13-18 years of age. In certain embodiments, a dynamic mapping between target groups and network users is performed by (1) acquiring data that identifies a network user (i.e., name, social security number, etc.), (2) locating within an information repository or database prior information that was obtain about the network user, (3) determining a target group to which the network user belongs based on the information in the repository, and (4) delivering to the network user a Web page that has a banner for a survey targeted to a group to which the network user belongs. For example, when a network user requests a Web page, the request may contain data (e.g. from a cookie) that identifies the user as John Smith. The host or a process that is linked to the Web page may respond by searching a database for previously gathered information about John Smith. Assume that the database contains information indicating that John Smith is male and 47 years old. Based on this information, it is determined that John Smith falls into the target group of a survey for men over the age of 45. Therefore, a banner for that particular survey is placed in the Web page that is returned to John Smith in response to his request. Banners can either be statically or dynamically created. In one embodiment, the automatic survey mechanism dynamically creates a particular banner using the target group information that was supplied by the client while defining the survey. The automated survey mechanism then requests the owner of the identified site(s) to launch the dynamically created banner. In one embodiment, the automated survey mechanism causes a static or predefined banner to be launched at the identified site(s). The static or predefined banner may be maintained by either the site owner(s) or by the automatic survey mechanism. In other embodiments, banners are launched by sending a message to the owner or representative of the identified site(s), requesting that a banner be launched. Each banner that is launched contains a reference (e.g. hypertext link) to a corresponding survey. Thus, by selecting a particular banner, network users are automatically presented with a corresponding survey. For example, by selecting a particular banner, a network user may be presented with the sample survey as previously described in FIG. 4. In certain embodiments, network users are requested to enter information about themselves (i.e., age, occupation, etc.) within the banner or recruiting document itself. Using this information, a survey is dynamically identified and presented to the network user. By dynamically identifying a particular survey based on the attributes of a network user, an appropriate survey can be presented to the network user, thus potentially increasing the efficiency of the overall system. For example, if a network user "Tom" selects a particular banner, Tom is prompted to enter certain information about himself (i.e. age and gender). If Tom enters an age of 25 and a gender of male, he is requested to complete a survey that requires males that are 25 or older. In certain embodiments, banners for a survey that correspond to a specific target group are placed on a general-purpose site (i.e. Yahoo!, AOL, Excite, USA Today, etc.). These banners contain information that specifies the desired target group of network users that are requested to complete the survey. (e.g. "if you are a male over 45, click here") In certain embodiments, the banner contains information that attempts to attract users of the network system to participate in the survey. For example, a particular banner may state that by participating in the survey, network users are automatically entered into a drawing or sweepstakes for a vacation to Hawaii. Similarly, the banner may also state that network users who complete the survey may receive cash or other forms of incentives. To reward a network user for completing a survey, the network user may be required to enter certain pertinent information (participant information). For example, if a banner offers Internet account credits for completing a survey, the network user may be required to provide their Internet account number so that they may be credited for participating in the survey. Because meaningful survey results typically require opinions from a non-biased population, the automatic survey mechanism attempts to prohibit network users from participating in a particular survey multiple times. In one embodiment, the automatic survey mechanism uses cookie information to identify computers that were previously used to complete a particular survey. For example, when a network user completes a survey, cookie information is stored on the computer that identifies the computer as one that has previously been used to participate in the particular survey. Thereafter, when a network user attempts to participate in a particular survey using the same computer, the network user is denied access to the survey. In an alternative embodiment, the automatic survey mechanism maintains previously entered participant information that can be used to identify network users who have previously completed a particular survey. After the required number of completed surveys is obtained, or the duration for fielding the survey has expired, access to the survey is disabled. The owners of the sites where the corresponding banner was launched are notified that the banner should be removed. Upon receiving the notice, the owner removes the banner from the site or prevents the banner from being displayed, thus eliminating access to the survey by network users. Gathering and Analyzing Survey Results After a banner is launched, network users may click on the banner to participate in the survey. When a network user completes a survey, the results are automatically sent over the network system where they are validated and stored in a corresponding storage unit (e.g. database) using the validation and insertion commands that are attached to the survey. After each response for a particular survey is inserted into the appropriate database, a counter for that particular survey is incremented. If the counter for the survey equals the number of responses requested by the client, the owner of the site on which the banner was launched is notified that the banner should be removed. The results are then processed and analyzed to generate survey data and charts reflecting the gathered survey results. Processing and analysis of the collected data may include, for example, the extraction of the survey data from the database and the creation of cross-tabulations and graphical charts. Once the processing is complete, the survey data is sent to the address that was previously specified by the client while defining the survey. In certain embodiments, survey results are analyzed on a periodic basis to generate real-time survey data. For example, in one embodiment, the automated survey mechanism generates real-time survey data by causing the survey results for a particular survey to be analyzed on a daily basis. The real-time survey data is then posted at a particular site on the network that can be accessed by the client. This allows the client to obtain survey data while the survey is still being fielded to users of the network system. The Automated Survey Process FIG. 6 illustrates a flow diagram for automating a survey over a network in accordance with an embodiment of the invention. At step 602, an automatic survey system generates interface data that defines an interface. The automatic survey system then causes the interface to be displayed on a client's display unit. At step 604, the client interacts with the interface to define a survey. By interacting with the interface, the client can define questions, select responses, edit, reorder, and view the survey. At step 606, the client selects a target group of network users who are to take and complete the survey. During this step, the client also specifies the number of desired completed surveys and/or the duration the survey is to field. At step 608, the client interacts with the interface to identify the price of the survey based upon the parameters provided by the client in previous steps. At step 608, the client has the capability of returning to steps 602 through 606 to modify the requested survey parameters to produce a survey price that is acceptable to the client. At step 610, the client uses the interface to provide an address to define where the survey results are to be delivered. At step 612, the client uses the interface to securely provide billing information and to provide final approval to proceed with survey fielding. At step 614, the survey created by the client is reviewed and screened for propriety. At step 616, the automated survey system generates a survey based on the parameters and data provided by the client. At this step, response validation rules and insertion commands are created and attached to the survey. At step 618, an information repository such as a database is created to store survey results from network users. At step 620, quotas or the number of completed surveys requested by the client are attached to the database. The automatic survey mechanism then fields the survey by causing one or more banners to be launched, activated, or displayed on one or more sites on the network system. At step 622, survey responses from network users participating in the survey are captured and inserted into the information repository or storage unit. At step 624, compensation or incentives are managed for network users who complete the survey. At step 626, the banners are removed from the sites on the network system once the number of desired completed surveys has been achieved. At this step, the automatic survey system sends messages to all appropriate sites indicating the banner should no longer be displayed. At step 628, the survey results provided by network users are extracted from the storage device. At step 630, the extracted survey results are processed, analyzed, and formatted using common file formats. At step 632, graphical representations of the survey results are generated and inserted into the survey data file. At step 634, the file containing the results of the survey are sent to the address that was previously provided by the client. At step 636, a receipt confirmation is received from the client indicating the file containing the survey results has been received. Dynamically Assigning a Survey to a Respondent A drawback with statically linking or associating a banner or URL to a specific survey is that it provides no mechanism for selecting a different survey that may be better suited or more desirable for participation by the respondent. The same drawback exists when inviting individuals to participate in surveys via e-mail, pop-up surveys, well-know URLs, or other forms. For example, in systems where each banner is linked to a specific survey, upon selecting a banner, the respondent is presented with the survey that is linked to that particular banner even if the respondent may actually possess certain characteristics or attributes that make them better suited for participation in a different active survey. For example, assume that the set of active surveys includes a first survey for all males over 20, and a second survey for all males over 70 that have never worn glasses. If a banner is linked to the first survey, then some respondents that would qualify for the second survey may be given the first survey. If the banner is linked to the second survey, then a high percentage of the respondents will not qualify to take it. Additionally, by statically linking a banner or URL to a specific survey, the survey system itself has no mechanism for reassigning an individual to a different survey based on the current survey participation data that has already been collected for the active surveys. FIG. 7 is a block diagram of a system 700 upon which certain embodiments of the invention may be implemented. Generally, system 700 includes one or more request sources 702, a client 714, a survey assigning server 716, one or more active surveys 718, and a set of participation criteria 756 that is used, at least in part, to dynamically identify and select a particular survey that is to be provided to a respondent. Request sources 702 represent a variety of mechanisms that may be used to send a message to survey assigning server 716 to request participation in a survey. For example, request sources 702 may include an online advertisement banner 704 that includes a URL associated with survey assigning server 716; an email message 706 that has an embedded URL associated with survey assigning server 716; a pop-up browser window 708 that includes a link to survey assigning server 716; a well published URL associated with survey assigning server 716; or a variety of other online recruiting methods 712 that are configured to send messages to survey assigning server 716 for requesting participation in a survey. Client 714 is a personal computer, workstation, cellular phone, Palm Pilot, or a variety of other device types that are capable of communicating with survey assigning server 716. Client 714 is used by or associated with a respondent 760. Although a single client 714 is shown in FIG. 7 by way of example, any number of clients can be included in the system 700, and multiple connections may be simultaneously established between different client devices and survey assigning server 716. In certain embodiments, client 714 is configured to execute a browser type application, such as Netscape Navigator®, Microsoft Internet Explorer® or other similar type of WAP or HTML browser application that has been developed for use in high constrained devices. Respondent 760 can use the browser application to communicate with survey assigning server 716 over a network system. Survey assigning server 716 is a computer, or one or more hardware or software components or processes that cooperate or execute in one or more computer systems. In certain embodiments, survey assigning server 716 is a Web server that is configured to receive requests from respondents and to select and forward a particular survey in response to receiving the requests. For example, survey assigning server 716 may include a servlet process that is configured to respond to clients requests that are initiated by respondents desiring to participate in a survey. As depicted, survey assigning server 716 may include among other things, one or more active surveys 718, an access point process 720, a screening question generator process 722, a screening question presenter process 724 and an assignment and distribution process 726. In this example, one or more active surveys 718 consist of a plurality of active surveys 732, 734, 736 and 738. Respectively associated with each of the active surveys 732, 734, 736 and 738 may be a set of target participation criteria (740, 742, 744, 746) and/or a set of current survey participation data (748, 750, 752, 754). The target participation criteria identifies a specific set of criteria or characteristics that is desirous of those individuals who are to participate in a particular survey. For example, target participation criteria 740 indicates that for a respondent to participate in survey 732 the respondent should be "Male." Conversely target participation criteria 744 indicates that for a respondent to participate in survey 736 the respondent should be "Female", an "Investor" and of age "25" or over. The current survey participation data identifies a variety of participation information that is associated with each of the surveys. For example, current survey participation data 748 indicates that "101" completes have currently been acquired out of a desired number of "200." Current survey participation data 748 further indicates that the survey became "active" on "Jun. 1, 2000" and is scheduled to become "inactive" on "Jun. 2, 2000," whether or not the desired number of completions has been obtained by that date. As is explained in further detail below, in certain embodiments, the target participation criteria (740, 742, 744, 746) and/or the current survey participation data (748, 750, 752, 754) may be used in determining the set of screening questions that are to be presented to a respondent and in dynamically selecting the specific survey that is presented to the respondent for their participation. Access point 720 is configured to receive requests for participation in a survey. For example, by selecting online ad banner 704 or an embedded URL within an email 706 that is associated with survey assigning server 716, a participation request message may be sent to and received by access point 720. In certain embodiments, the participation request message may include "supplemental" information about the requesting respondent and/or the client device that was used to send the participation request message. For example, an embedded URL within an email message may include additional parameter information that describes certain characteristics or features of the respondent. Additionally, the participation request message may be information stored on the requesting client device, such as cookie or other browser information, that is also sent to and received by access point 720. In certain embodiments, the supplemental information is used in determining the screening questions that are provided to the respondent. For example, attributes associated with a particular URL (for example, attributes that indicates that the user has selected a banner associated with men's running shoes or that the respondent came from a particular Web site or location), may indicate that the respondent is most likely a man and thus may help to determine the set of screening questions that are most likely relevant to the respondent. In one embodiment, the supplemental information is used alone, or in combination with a set of screening questions, to identify the particular survey that is to be present to the respondent. For example, if survey assigning server 716 receives cookie information that provides the identity of the requesting respondent and/or specific characteristics about the requesting respondent, the survey assigning server 716 may use the supplemental information to select a particular survey without presenting a set of screening questions to the respondent. As shown by this example, in certain embodiments, survey assigning server 716 may assign a specific survey to a respondent based solely on the supplemental information, thus without requiring any additional input from the respondent. In response to receiving a participation request message, access point 720 forwards the information to screening question generator 722. Thereafter, screening question generator 722 generates an appropriate set of one or more screening questions for determining the appropriate survey that should be presented to the respondent. How the set of screening questions is determined is explained in greater detail below. Once the set of one or more screening questions is generated, the screening question presenter 724 transmits interface data to client 714 to cause client 714 to display the set of one or more questions for viewing by respondent 760. Once the respondent completes the set of one or more questions, the respondent's answers are then forwarded back to screening question presenter 724 by client 714. Thereafter, assignment and distribution process 726 uses the respondent's answers, possibly in combination with the target participation criteria and/or set of current survey participation data of database 729, to determine the particular survey that should be presented to respondent 760. The identified survey is then forwarded to client 714 to allow respondent 760 to participate in the particular survey. As will be explained in greater detail below, assignment and distribution process 726 may use a variety of different target and/or current survey participation data in determining the particular survey that should be presented to respondent 760. Generating Screening Questions for Respondents As previously indicated, associated with each survey is a set of zero or more target participation criteria that an individual must possess to qualify and be invited to participate in the particular survey. The following are examples of sets of target participation criteria: (i) Males (one criterion) (ii) Females between the ages of 25 and 55 (two criteria) (iii) Individuals who have visited a particular Web site in the past six months and use a 33.5K or a 56K dial-up modem (three criteria) (iv) Sports enthusiasts (one criterion) (v) Anyone (no criteria) In certain embodiments, one or more databases 756 may be used to store information associated with the one or more active surveys. For example, a database of active surveys 728 may be used to store the URLs, required access codes, survey access restrictions, or other parameters that may be required to access any individual survey. Additionally, in certain embodiments, a database of target participation criteria and current survey participation data 729 is used to store the target participation criteria and the current survey participation data that is associated with the active surveys 718. Still further, in certain embodiments, a screening question mapping mechanism 730 is used by survey assigning server 716 to maintain a mapping between the target participation criteria that is associated with the currently active surveys and a set of corresponding screening questions that will allow assigning server 716 to determine whether an individual actually meets the target participation criteria that is associated with a particular survey. To determine whether a respondent possesses specific criteria or characteristics, questions are formulated and presented to respondents. Questions may be dynamically generated in response to receiving a participation request. Questions may also be statically defined and linked to corresponding target participation criteria prior to receiving a respondent's participation request. Examples questions include:
In Example 1 above, a single question is used to determine if a respondent is either male or female. In Example 2, a single question is also used to determine an individual's areas of interest across multiple categories. If both Example 1 and Example 2 are presented to respondents, the gender and interests of the respondent can be easily acquired. In other embodiments (see Examples 3, 4 and 5 below), specific questions (such as Yes/No) can be statically configured or dynamically formulated for each set of target participation criteria that is associated with a particular survey. Examples include:
Selecting Questions to Present to a Respondent When a respondent expresses a desire to participate in a survey, (for example, clicks on a banner, activates a hyperlink, views a pop-up invitation, or arrives through any other recruiting mechanism via a computer network), the survey assigning server 716 dynamically determines which screening questions to present to the respondent based upon current target participation criteria. Below are example criteria that can be used independently or in combination to generate the set of screening questions presented to the respondent. In one embodiment, to potentially increase the number of respondents who are willing to answer a set of screening questions, it may be desirable, and/or necessary, to reduce the number of screening questions that would normally be presented to a respondent at a single time or on a single Web page. To limit the number of screening questions that are presented a respondent, survey assigning server 716 may be required to select a reduced set of screening questions. A variety of factors may influence the particular selection of the screening questions within the reduced set, some of which are identified below. Also, in certain embodiments, to potentially increase the likelihood that a respondent will qualify to participate in at least one active survey, the incidence (probability that an individual possesses the required characteristics) associated with the target participation criteria of active surveys are used to determine which screening questions are presented to the respondent. By definition, to locate qualified respondents for target participation criteria possessing a low incidence requires the associated screening questions to be presented to respondents more frequently than high incidence target participation criteria. If many low incidence surveys (each possessing different target participation criteria) are active, it is often not possible nor desirable to present all screening questions to the respondent (due to the maximum number of screening questions per page mentioned above). Consequently, the system must determine which screening questions are to be included to acquire the desired number of completions from low incidence surveys while balancing the probability that a respondent will qualify for at least one survey based upon the generated set of screening questions. As previously indicated above, in certain embodiments, supplemental information is received from a client and used to determine the set of one or more screening questions that will be presented to a respondent. For example, supplemental information that is passed as parameters within a URL (i.e., http://www.insightexpress.com/ix/screener.asp?target=1 where the parameter is "target" and the value is "1") can be used to determine which questions are to be included or excluded from the set of screening questions. This information is particularly useful when it is known or expected that the respondent possesses or does not possess specific characteristics. Additionally, supplemental information (such as a referring URL) or information stored within a cookie (i.e., information regarding surveys previously completed by the respondent, etc.), can also be used to determine which questions should be included or excluded from the set of generated screening questions. In certain embodiments, the current number of "completes" that have been acquired for a particular survey and/or the number of "completes" that are still required for a particular survey may impact which screening questions are presented to a respondent. For example, active surveys that possess fewer completions may be assigned a higher priority, causing questions directed to them to be included within the generated set of screening questions. Similarly, the time remaining until a survey is no longer considered active may also impact which screening questions are selected. For example, surveys that will become inactive within the next one hour may result in a higher probability that their associated screening questions will be included in the generated set of screening questions. Furthermore, the elapsed time of a survey (time since becoming active) and/or the rate of acquired survey completions (number of completions over a specified period of time) can also be used to determine which questions are to be included or excluded within the set of generated screening questions. This approach is particularly helpful in "pacing," throttling, or managing the elapsed time required to obtain the desired number of target completions for a particular survey. Additionally, derived calculations such as the survey completion rate (the number of respondents who successfully complete the assigned survey divided by the number of respondents who were assigned to the survey) for each active survey can also be used to determine which questions are included or excluded from presentation to the respondent. For example, screening questions associated with surveys with low completion rates may be presented more frequently to respondents to acquire the desired number of completions over a specified time period (compensating for a survey's lower completion rate). In addition, in certain embodiments, the determination of which screening questions are to be presented to the respondents can be performed on a periodic basis or in response to the occurrence of a specific event (i.e., activation of a new survey, termination of an active survey, the number of times a screening question has been presented to requesting respondents, etc.), in addition to or instead of dynamically generating the set of screening questions upon receiving a respondent's participation request. As previously indicated, once the set of screening questions is generated, they are then presented to a respondent via a user interface that is provided to a client over a computer network. The screening questions may be presented to a respondent using a single interface window (i.e., single HTML page), or through multiple interface windows (i.e. multiple HTML pages). Additionally, in certain embodiments, a set of cascading screening questions are presented to a respondent based on the respondent's prior answers. For example, survey assigning server 716 may present an initial question to respondent 760 asking whether they are "male" or "female." If respondent 760 indicates that she is a female, survey assigning server 716 may present one or more questions that are directed to females. Alternatively, if respondent 760 indicates that he is a male, survey assigning server 716 may present one or more questions that are directed to males. The cascading of questions is not limited to any particular number of questions. The participation criteria identified above is provided as an example of some of the types of criteria that may be used in determining a set of one or more screening questions to present to a respondent. Thus, other types of target participation criteria and/or current survey participation data may be used to generate a particular set of screening questions. For example, a country-of-residence question may be included in a set of screening questions to enable the adherence to country-specific privacy laws. In one embodiment, an age question is included within each set of screening question to identify and screen individuals that are under a specific age. In certain embodiments, if a respondent does not meet certain requirements (i.e. is not thirteen years of age or older) the respondents answers to the screening questions are discarded and the respondent is denied participation in any of the active surveys. Selecting a Particular Survey for a Respondent Once the set of screening questions has been formulated, it is presented to the respondent. Upon providing answers to the screening questions, the survey assigning server 716 dynamically selects and presents the respondents with a selected survey. In one embodiment, the survey that is selected by survey assigning server 716 is based, at least in part, on the respondent's answers to the screening questions. In certain embodiments, the survey that is selected by survey assigning server 716 is also based, at least in part, on the current survey participation data that is associated with the active surveys. For example, some or all of the criteria that was previously described above for selecting questions to present to a respondent may also be used to select the particular survey that is to be presented to a respondent. In certain situations, survey assigning server 716 may identify two or more "candidate" surveys for the respondent. In one embodiment, if survey assigning server 716 identifies two or more "candidate" surveys for presenting to the respondent, survey assigning server 716 uses current survey participation data to select a particular survey for presenting to the respondent. For example, if survey assigning server 716 determines that the respondent qualifies for surveys 732, 734 and 736, survey assigning server 716 may (1) select survey 732 because it is scheduled to become inactive before surveys 734 and 744; (2) select survey 734 because it currently has the lowest percentage of completions to desired completions (i.e. 1/500); or (3) select survey 734 because a respondent that has the necessary target participation criteria 744 (female, investor, age 25+) is harder to find than respondents with the necessary target participation criteria for surveys 732 (male) and 734 (investor). Similarly if the survey assigning server 716 identifies zero "candidate" surveys, in certain embodiments the survey assigning server 716 may assign the respondent to a "default" survey. In another embodiment, upon determining that a respondent is equally eligible for two or more surveys, the system randomly assigns the respondent to one of the candidate surveys. This randomization process has been shown in certain instances to produce high quality research results. Sequence for Assigning a Survey to a Respondent FIG. 8 illustrates a flow diagram for assigning a survey to a respondent in accordance with certain embodiments of the invention. For explanation purposes, FIG. 8 will be described in reference to the components of FIG. 7. At block 802, a participation request is received from a respondent for participation in a survey. As previously indicated, requests to participate in a survey may be initiated using a variety of mechanisms. For example, by selecting an online ad banner 704, or by selecting an embedded URL within an email message which are linked to survey assigning server 716, a respondent may send a message to survey assigning server 716 to request participation in a survey. At block 804, in response to receiving the participation request, target participation criteria for the active surveys is retrieved to determine a set of one or more screening questions that are to be presented to the respondent. For example, in response to receiving a participation request from client 714, screening question generator 722 interfaces with the database of active surveys 728 to identify a list of active surveys. Based on the list of active surveys, screening question generator 722 retrieves target participation criteria data from database 729. FIG. 9 illustrates an example set of active survey data 900 that may be maintained for the active surveys. At block 806, a set of one or more screening questions is generated. In one embodiment, the set of screening questions is based at least in part on the target participation criteria data that was received in block 804. For example, based on the retrieved target participation criteria data, screening question generator 722 interfaces with screening question mapping mechanism 730 to generate a set of one or more screening questions. In one embodiment, the set of one or more screening questions are generated "on-the-fly," based on the target participation criteria that currently exists for the active surveys. At block 808, the set of one or more screening questions is sent to the respondent for answering by the respondent. For example, once the set of screening questions are generated, question presenter process 724 transmits interface data to client 714 to present the set of screening questions to respondent 760. FIG. 10 illustrates an example set of screening questions 1000 that maybe displayed on client 714 for respondent 760. At block 810, response data is received based on the respondent's answers to the set of screening questions. For example, in response to the respondent 760 completing the set of one or more screening questions, client 714 transmits response data to question presenter process 724. At block 812, a set of zero or more candidate surveys is identified based on the respondent's answers to the set of screening questions. For example, if the respondent's answers indicate that the respondent 760 is both a "male" and an "investor," assignment and distribution process 726 may select active surveys 732 and 734 as the set of zero or more candidate surveys. Alternatively, if the respondent's answers indicate that the respondent 760 is a "female" that is of age "18," assignment and distribution process 726 may select no active surveys (i.e. empty set) to be the set of zero or more candidate surveys. At block 814, a specific survey is selected based on the identified set of zero or more candidate surveys. For example, if the set of zero or more candidate surveys includes a single active survey, that survey or a "default" survey will generally be selected as the specific active survey. Alternatively, if the set of zero or more candidate surveys includes two or more active surveys, assignment and distribution process 726 determines which of the two or more active surveys should be presented to the respondent 760. In one embodiment, assignment and distribution process 726 selects the specific survey based at least in part on the current survey participation data. Additionally, if the set of zero or more candidate surveys includes zero active surveys, assignment and distribution process 726 determines which of the active surveys should be presented to the respondent 760. Here again, the assignment and distribution process 726 may select the specific survey based at least in part on the current survey participation data. At block 816, the selected survey is transmitted to the requesting client for participation by the respondent. For example, after selecting a particular survey, assignment and distribution process 726 transmits interface data to client 714 to cause the selected survey to be presented to respondent 760. As indicated above, the set of screening questions may be provided to a respondent in a cascading manner so as to allow survey assigning server 716 to base a subsequent question, at least in part, on the answer that was given to a previously presented question. For example, in certain embodiments, in response to receiving the response data at block 810, the sequence may loop back to block 806 to generate one or more screening questions that are based, at least in part, on prior responses previously received from the respondent. Alternatives, Extensions In describing certain embodiments of the invention, several drawing figures have been used for explanation purposes. However, the invention is not limited to any particular. The invention includes other contexts and applications in which the mechanisms and processes described herein is available to other mechanisms, methods, programs, and processes. Thus, the specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense. For example, although example were described in which the selected active survey was transmitted to the same client that received the set of one or more screening questions, in certain embodiments, the selected active survey and the set of one or more screening questions may actually be sent to different clients, and possibly for different respondents. Likewise, the source of the initial participation request may be different than the client that receives the set of one or more screening questions and/or the selected active survey. Additionally, although FIG. 7 depicts a single survey assigning server 716, embodiments of the invention are not limited to any particular number of survey assigning servers. In addition, although survey assigning server 716 is depicted as a single component, it may actually consist of multiple computing and/or storage units that are configured to perform the functions described herein. In addition, in this disclosure, certain process steps are set forth in a particular order, and alphabetic and alphanumeric labels are used to identify certain steps. Unless specifically stated in the disclosure, embodiments of the invention are not limited to any particular order of carrying out such steps. In particular, the labels are used merely for convenient identification of steps, and are not intended to imply, specify or require a particular order of carrying out such steps.
|
Same subclass Same class Consider this |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||