Automatic selection of statistical quality control procedures

Abstract

Disclosed is a computer implemented method for automating the selection of an analytical statistical quality control (QC) procedure. Inputs of a quality requirement and observed performance characteristics of a method of measurement are initially provided, together with a database of theoretical operating characteristics of statistical QC procedures. An appropriate quality-planning algorithm (either analytical or clinical) which relates the quality requirement, the observed performance characteristics of the method of measurement, and the theoretical operating characteristics of statistical QC procedures is also provided. Using predetermined QC selection criteria and selection logic, a computer is programmed to automatically choosing, based on the quality requirement, observed method performance, and operating characteristic of candidate QC procedures, a subset of control rules and numbers of control measurements which satisfy selection criteria and selection logic. Also disclosed is a dynamic QC process for automatically changing the statistical QC procedure when there are changes in performance of a method of measurement. In this embodiment, real-time statistical estimates of the performance characteristics of a method are provided through on-line sampling with an analytical instrument and this data is used to automatically select a QC procedure.

Claims

We claim:

1. A computer implemented method for automating the selection of an analytical statistical quality control (QC) procedure, the method comprising:

a) providing inputs of a quality requirement and observed performance characteristics of a method of measurement;

b) providing a database of theoretical operating characteristics of statistical QC procedures;

c) providing an analytical quality-planning algorithm which relates the quality requirement, the observed performance characteristics of the method of measurement, and the theoretical operating characteristics of statistical QC procedures;

d) providing QC selection criteria and selection logic;

e) automatically choosing, based on the quality requirement, observed method performance, and operating characteristic of candidate QC procedures, a subset of control rules and numbers of control measurements which satisfy selection criteria and selection logic.

2. The method of claim 1 wherein the statistical QC procedure can be characterized by a control rule, control statistic, or control chart whose operating characteristics can be described in terms of probabilities of acceptance or rejection as a function of the size of systematic and random errors.

3. The method of claim 2 wherein the control statistic represents the average of normal patient test results used to monitor errors or changes.

4. The method of claim 1 wherein the quality specification is an analytical quality requirement for the allowable total error, TEa.

5. The method of claim 1 wherein the input parameters necessary for the analytical quality-planning algorithm are the stable imprecision observed (S.sub.meas); stable inaccuracy (bias.sub.meas); matrix inaccuracy (bias.sub.matx); number of replicate samples (n.sub.samp); and z-value for maximum defect rate (z).

6. The method of claim 1 further comprising automatically selecting from among the subset of control rules in step e) the control rule which best satisfies the selection criteria and selection logic for type of control rule or control chart, total numbers of control measurements per analytical run, error detection rate, and false rejection rate.

7. The method of claim 6 wherein the database of operating characteristics of statistical QC procedures comprises types of control rules and control charts selected from the group consisting of single-rule constant limits, single-rule variable limits, multi-rule with specified maximum number of runs, and mean/range rules.

8. The method of claim 6 wherein the selection criteria and selection logic for the total number of control measurements per analytical run (N) includes 1, 2, 3, 4, 6, and 8 control measurements per run.

9. The method of claim 6 wherein the selection criteria and selection logic for the rate of error detection are 90% or more, 50% or more, or 25% or more.

10. The method of claim 9 wherein the selection criteria and selection logic for error detection depend on the input for the observed instability of the measurement procedure.

11. The method of claim 6 wherein the selection criteria and selection logic for the rate of false rejection are 10% or less, 5% or less, or 1% or less.

12. A computer implemented method for automating the selection of a clinical statistical quality control (QC) procedure, the method comprising:

a) providing inputs of a quality requirement and observed performance characteristics of a method of measurement;

b) providing a database of theoretical operating characteristics of statistical QC procedures;

c) providing a clinical quality-planning algorithm which relates the quality requirement, the observed performance characteristics of the method of measurement, and the theoretical operating characteristics of statistical QC procedures;

d) providing QC selection criteria and selection logic;

e) automatically choosing, based on the quality requirement, observed method performance, and operating characteristic of candidate QC procedures, a subset of control rules and numbers of control measurements which satisfy selection criteria and selection logic.

13. The method of claim 12 wherein the statistical QC procedure can be characterized by a control rule, control statistic, or control chart whose operating characteristics can be described in terms of probabilities of acceptance or rejection as a function of the size of systematic and random errors.

14. The method of claim 13 wherein the control statistic represents the average of normal patient test results used to monitor errors or changes.

15. The method of claim 12 wherein the quality specification is a clinical quality requirement for a medically important change in the form of a decision interval, DInt.

16. The method of claim 12 wherein the input parameters necessary for the clinical quality-planning algorithm are the stable imprecision observed (S.sub.meas); stable inaccuracy (bias.sub.meas); matrix inaccuracy (bias.sub.matx); specimen bias (bias.sub.spec); within-subject biological variation (S.sub.wsub); sampling variation (S.sub.spec); number of tests (n.sub.test); number of specimens (n.sub.spec); number of replicate samples (n.sub.samp); and z-value for maximum defect rate (z).

17. The method of claim 12 further comprising automatically selecting from among the subset of control rules in step e) the control rule which best satisfies the selection criteria and selection logic for type of control rule or control chart, total number of control measurements per analytical run, error detection rate, and false rejection rate.

18. The method of claim 17 wherein the database of operating characteristics of statistical QC procedures comprises types of control rules and control charts selected from the group consisting of single-rule constant limits, single-rule variable limits, multi-rule with specified maximum number of runs, and mean/range rules.

19. The method of claim 17 wherein the selection criteria and selection logic for the total number of control measurements per analytical run (N) includes 1, 2, 3, 4, 6, and 8 control measurements per run.

20. The method of claim 19 wherein the selection criteria and selection logic for the rate of error detection are 90% or more, 50% or more, or 25% or more.

21. The method of claim 20 wherein the selection criteria and selection logic for error detection depends on the input for the observed instability of the measurement procedure.

22. The method of claim 16 wherein the selection criteria for the rate of false rejection are 10% or less, 5% or less, or 1% or less.

23. A dynamic QC process for automatically changing the statistical QC procedure when there are changes in performance of the method of measurement, the method comprising:

a) providing inputs of a quality requirement and initial observed performance characteristics of a method of measurement;

b) providing a database of theoretical operating characteristics of statistical QC procedures;

c) providing quality-planning algorithms which relate the quality requirement, the observed performance characteristics of the method of measurement, and the theoretical operating characteristics of statistical QC procedures;

d) providing QC selection criteria and selection logic;

e) automatically choosing, based on the quality requirement, observed method performance, and operating characteristic of candidate QC procedures, a subset of control rules and numbers of control measurements which satisfy selection criteria and selection logic;

f) providing real time statistical estimates of method performance characteristics through on-line sampling with an analytical instrument;

g) repeating steps b)-f) above for a sequence of runs thereby automatically selecting a higher number of control measurements or more sensitive control rules when method performance deteriorates and decreasing the number of control measurements or changing to less sensitive control rules when method performance improves;

h) implementing the selected statistical QC procedure through calculations and control charts;

i) providing an assessment of the state of control and a decision about the acceptance or rejection of individual analytical runs based on the selected statistical QC procedure.

Description

BACKGROUND OF THE INVENTION

Health care providers in the United States have entered a new competitive era. Changes in government reimbursement policies have limited the amount of money that is paid for medical care. Payment of flat fees, based on Diagnostic Related Groups (DRGs), has effectively limited hospital and laboratory charges. Support of Health Maintenance Organizations and similar prepaid medical plans has created new health-care organizations to compete with traditional providers. The new competitive marketplace puts pressure on a provider not only to reduce the costs of products and services but also, and at the same time, to improve their quality, to maintain or increase their share of the market. The effect on clinical laboratories is that performing more tests no longer guarantees more revenue or increased resources to support new tests and services. In addition, the quality of the laboratory's testing services that employ complex analytical processes, and their usefulness to the physician and patient must be at least maintained and, in many cases, improved.

In general, an analytical process includes a measurement procedure and a control procedure. The measurement procedure refers to the analytical method per se, i.e., the reagents, the instrument, and the step-by-step directions for producing an analytical result. The control procedure refers to that part of the process concerned with testing the validity of the analytical result to determine whether it is reliable and can be reported. The development and theoretical aspects of a control procedure are discussed, for example, by Westgard and Barry (Westgard, J. O. and Barry, P. L., Cost-Effective Quality Control: Manage the quality and productivity of analytical processes, AACC Press, Washington, DC (1986)).

Improvements in an analytical process translate into reduced costs, higher productivity and improved quality of analysis for an implementing laboratory. In today's competitive environment, laboratories which focus on improvements in such analytical processes are at a competitive advantage relative to those which do not.

SUMMARY OF THE INVENTION

The subject invention relates to a computer implemented method for automating the selection of an analytical statistical quality control (QC) procedure. Inputs of a quality requirement and observed performance characteristics of a method of measurement are initially provided, together with a database of theoretical operating characteristics of statistical QC procedures. An appropriate quality-planning algorithm (either analytical or clinical) which relates the quality requirement, the observed performance characteristics of the method of measurement, and the theoretical operating characteristics of statistical QC procedures is also provided. Using predetermined QC selection criteria and selection logic, a computer is programmed to automatically choosing, based on the quality requirement, observed method performance, and operating characteristic of candidate QC procedures, a subset of control rules and numbers of control measurements which satisfy selection criteria and selection logic.

In another embodiment, the subject invention relates to a dynamic QC process for automatically changing the statistical QC procedure when there are changes in performance of a method of measurement. In this embodiment, real-time statistical estimates of the performance characteristics of a method are provided through on-line sampling with an analytical instrument and this data is used to automatically select a QC procedure.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 represents a logic diagram for automatic quality control procedure selection in both clinical and analytical formats.

FIG. 2 represents a logic diagram for the dynamic, real-time application of automatic quality control procedure selection.

FIG. 3 represents a quality control procedure selection criteria window.

FIG. 4 represents a quality control procedure selection logic window.

DESCRIPTION OF PREFERRED EMBODIMENTS

The present invention relates to a computer implemented method for automating the selection of a statistical quality control procedure. The discussion which follows relates primarily to the application of the invention in the field of clinical diagnosis for human or animal healthcare. However, the invention has applications in other fields of use where comparatively small numbers of controls plotted on a statistical control chart (e.g., the Levey-Jennings chart), are used to determine the acceptability of much larger numbers of samples tested by the same analytical method within the same time period.

The selection of a statistical quality control procedure is an essential element in the development of an analytical process. A statistical quality control procedure is necessary to alert an analyst when medically important errors occur. The control procedure should be selected on the basis of the needed capabilities for error detection, as well as practical considerations related to its rate of false alarms, ease of use, training requirements, etc. The performance of the control procedure should be documented in quantitative terms, so the chance of detecting the medically important errors is known.

A direct way to check the quality of an analytical sample is by comparison to control samples having predetermined characteristics (e.g., concentration of an analyte) which behave similar to patient test samples. Control sample, as used herein, refers to a control material or solution which can be purchased, or produced by dissolving appropriate concentrations of a solute in an appropriate solvent (typically buffered saline). Such materials are widely available as controls for clinical chemistry analytes.

The analytical results obtained for control solutions (analyzed for purposes of quality control) are called control measurements or control observations. Such measurements are made using an analytical instrument. In some control procedures, the individual results are plotted and interpreted directly. Many procedures, however, require calculations to be performed on the control measurements before the data can be used to test control status. These derived or calculated values are called control statistics; for example, the mean of a group of six control measurements could be used as a control statistic.

Statistical techniques can be used to determine whether the control measurements are different from the known values. Such procedures are known as statistical quality-control procedures, or simply, quality-control procedures. Control procedures are commonly implemented by plotting control measurements (or statistics calculated from those measurements) on control charts; a control chart, therefore, is a statistical control model that provides a visual representation to aid interpretation. The American Society for Quality Control (ASQC) states that a control chart is "a graphical method for evaluating whether a process is, or is not, in a `state of statistical control`".

By using control charts, analysts make decisions about the acceptability of the analytical results from individual analytical runs. Aliquots or samples from one or more stable control solutions are analyzed by the measurement procedure during each analytical run. The results are plotted on a control chart and compared with the control limits drawn on the chart; thus the analyst determines whether an analytical run is in control or out-of-control and whether the results for the patients' samples in that run are to be reported.

According to the ASQC, control limits are the "limits on a control chart which are used as criteria for signaling the need for action, or for judging whether a set of data does, or does not, indicate a `state of control`". Control limits consist of upper and lower limits, which define a range of acceptable values. In general, an analytical run is judged to be in control when the control measurements fall within the control limits, and out of control when the control limits are exceeded.

An analytical run can be defined as a period of time or series of measurements, within which the accuracy and precision of the measuring system is expected to be stable; between analytical runs, events may occur causing the measurement process to be susceptible to variations which are important to detect (see also NCCLS Document C24-A, Internal Quality Control Testing: Principles and Definitions, Vol 11 (No. 6), National Committee for Clinical Laboratory Standards, 940 West Valley Road, Suite 1400, Wayne, Pa. 19087-1898). The length of an analytical run must be defined appropriately for the specific analytical system and specific laboratory application. The manufacturer should recommend run length for the analytical system and the user should define run length for the specific application. Thus, the length of an analytical run depends on the stability of the measurement procedure and its susceptibility to errors.

However defined, an analytical run will include a certain number of patients' and control samples. An analytical run is that group of samples for which a decision is to be made concerning the validity of the measurements. The number of control measurements per run, N, is the actual number of control measurements available when judging the control status of the measurement procedure. N is critical for determining the performance characteristics of a control procedure.

i. Analytical Quality Control

QC requirements can be defined in either clinical or analytical formats (or terms) and the methods and compositions of the present invention relate to both categories. The logic for both the analytical and clinical embodiments are shown in FIG. 1. Referring first to the analytical embodiment, inputs of a quality requirement are essential. Analytical quality requirements have been defined by the CLIA-88 proficiency testing (PT) criteria for acceptable performance (Federal Register 57(40): 7002 (Feb. 28, 1992)). These criteria are presented in three different ways:

as absolute concentration limits, e.g., target value.+-.1 mg/dL for calcium;

as a percentage, e.g., target value.+-.10% for albumin, cholesterol, and total protein; and

as the distribution of a survey group, e.g., target value.+-.3 standard deviations (SD) for thyroid stimulating hormone.

In a few cases, two sets of limits are given, e.g., the glucose requirement is given as the target value.+-.6 mg/dL or .+-.10% (whichever is greater).

The CLIA PT criteria specify the total errors that are allowable. The total error format is implicit because the CLIA-88 rules specify that only a single test is to be performed for each PT specimen. Under such conditions, the observed analytical error will be the total error due to both inaccuracy and imprecision. Following is a list of the CLIA proficiency testing criteria for the currently regulated analytes, based on the Feb. 28, 1992, Federal Register.

    ______________________________________
    Test or Analyte   Acceptable Performance
    ______________________________________
    Routine Chemistry
    Alanine aminotransferase (ALT)
                      Target value .+-. 20%
    Albumin           Target value .+-. 10%
    Alkaline phosphatase
                      Target value .+-. 30%
    Amylase           Target value .+-. 30%
    Aspartate aminotransferase (AST)
                      Target value .+-. 20%
    Bilirubin, total  Target value .+-. 0.4 mg/dL or .+-.
                      20% (greater)
    Blood gas pO.sub.2
                      Target value .+-. 3 SD
    Blood gas pCO.sub.2
                      Target value .+-. 5 mm Hg or .+-.
                      8% (greater)
    Blood gas pH      Target value .+-. 0.04
    Calcium, total    Target value .+-. 1.0 mg/dL
    Chloride          Target value .+-. 5%
    Cholesterol, total
                      Target value .+-. 10%
    Cholesterol (HDL) Target value .+-. 30%
    Creatine kinase   Target value .+-. 30%
    Creatine kinase isoenzymes
                      MB elevated (present or absent)
                      or Target value .+-. 3 SD
    Creatinine        Target value .+-. 0.3 mg/dL
                      or .+-. 15% (greater)
    Glucose           Target value .+-. 6 mg/dL
                      or .+-. 10% (greater)
    Iron, Total       Target value .+-. 20%
    Lactate dehydrogenase (LDH)
                      Target value .+-. 20%
    LDH isoenzymes    LDH1/LDH2 (+ or -)
                      or Target value .+-. 30%
    Magnesium         Target value .+-. 25%
    Potassium         Target value .+-. 0.5 mmol/L
    Sodium            Target value .+-. 4 mmol/L
    Total protein     Tarqet value .+-. 10%
    Triglycerides     Target value .+-. 25%
    Urea Nitrogen     Target value .+-. 2 mg/dL
                      or .+-. 9% (greater)
    Uric acid         Target value .+-. 17%
    Toxicology
    Alcohol, blood    Target value .+-. 25%
    Blood lead        Target value .+-. 10%
                      or .+-. 4 mcg/dL (greater)
    Carbamazepine     Target value .+-. 25
    Digoxin           Target value .+-. 20%
                      or 0.2 ng/mL (greater)
    Ethosuximide      Target value .+-. 20%
    Gentamicin        Target value .+-. 25%
    Lithium           Target value .+-. 0.3 mmol/L
                      or .+-. 20% (greater)
    Phenobarbital     Target value .+-. 20%
    Phenytoin         Target value .+-. 25%
    Primidone         Target value .+-. 25%
    Procainamide (and metabolite)
                      Target value .+-. 25%
    Quinidine         Target value .+-. 25%
    Theophylline      Target value .+-. 25%
    Tobramycin        Target value .+-. 25%
    Valproic acid     Target value .+-. 25%
    Hematology
    Cell identification
                      90% or greater consensus on
                      identification
    White cell differentiation
                      Target .+-. 3 SD based on percentage
                      of different types of white cells
    Erythrocyte count Target .+-. 6%
    Hematocrit        Target .+-. 6%
    Hemoglobin        Target .+-. 7%
    Leukocyte count   Target .+-. 15%
    Platelet count    Target .+-. 25%
    Fibrinogen        Target .+-. 20%
    Partial thromboplastin time
                      Target .+-. 15%
    Prothrombin time  Target .+-. 15%
    Endocrinology
    Cortisol          Target value .+-. 25%
    Free thyroxine    Target value .+-. 3 SD
    Human chorionic gonadotropin
                      Target value .+-. 3 SD
                      or (positive or negative)
    T.sub.3 uptake    Target value .+-. 3 SD by method
    Triiodothyronine  Target value .+-. 3 SD
    Thyroid stimulating hormone
                      Target value .+-. 3 SD
    Thyroxine         Target value .+-. 20%
                      or 1.0 mcg/dL (greater)
    General Immunology
    Alpha-1 antitrypsin
                      Target value .+-. 3 SD
    Alpha-fetoprotein Target value .+-. 3 SD
    Antinuclear antibody
                      Target value .+-. 2 dilution
                      or (positive or negative)
    Antistreptolysin O
                      Target value .+-. 2 dilution
                      or (positive or negative)
    Anti-Human Immunodeficiency virus
                      Reaction or nonreactive
    Complement C3     Target value .+-. 3 SD
    Complement C4     Target value .+-. 3 SD
    Hepatitis (HBsAg, anti-HBc,
                      Reactive (positive) or
    HBeAg)            nonreactive (negative)
    IgA               Target value .+-. 3 SD
    IgE               Target value .+-. 3 SD
    IgG               Target value .+-. 25%
    IgM               Target value .+-. 3 SD
    Infectious mononucleosis
                      Target value .+-. 2 dilution
                      or (positive or negative)
    Rheumatoid factor Target value .+-. 2 dilution
                      or (positive or negative)
    Rubella           Target value .+-. 2 dilution
                      or (positive or negative)
    ______________________________________

    ______________________________________
    Factors for calculating control limits
    Rule    Number of control measurements (N)
    ______________________________________
    1.sub.0.05
            2.24      2.39   2.50     2.64 2.74
    1.sub.0.01
            2.81      2.93   3.01     3.13 3.21
    1.sub.0.002
            3.27      3.36   3.44     3.52 3.59
    X.sub.0.05
            1.39      1.13   0.98     0.80 0.60
    X.sub.0.01
            1.82      1.49   1.29     1.05 0.91
    X.sub.0.002
            2.19      1.78   1.54     1.26 1.09
    R.sub.0.05
            2.77      3.31   3.63     4.03 4.29
    R.sub.0.01
            3.64      3.63   4.40     4.76 4.99
    R.sub.0.002
            4.37      4.80   5.05     5.37 5.58
    ______________________________________

• Inventors
  Westgard, James O.; Stein, Bernard;
• Assignee
  WesTgard Quality Corporation (Madison, WI)
• Published
  Aug-10-1999
• Current US Classes:
  702/81
  702/82
  702/83
  702/89
  705/21
  705/7
• Application #
  643993
• International Classes
  G01N 021/25; G06F 015/18; G06F 017/50; G06F 019/00
• Field of Search
  705/7 705/21 436/8 364/151 364/468.15 364/468.16 364/468.17 702/81 702/82 702/83 702/84
• Examiner
  MacDonald; Allen R.
• Agent
  Farrell; Kevin M.