CKVi.C^i/3 Quality Assurance of Chemical IVIeasurements John K. Taylor, Ph.D. Center for Analytical Chemistry National Bureau of Standards Gaithersburg, MD 20899 TABLE OF CONTENTS I INTRODUCTION II MEASUREMENT AS A PROCESS - CHEMICAL ANALYSIS AS A SYSTEM III STATISTICAL CONCEPTS IV MODELING V PLANNING AND SAMPLING VI METHODOLOGY VII CALIBRATION VIII QUALITY ASSURANCE - GENERAL ASPECTS IX QUALITY CONTROL X CONTROL CHARTS XI QUALITY ASSESSMENT XII CORRECTION OF ERRORS AND/OR IMPROVING PRECISION AND ACCURACY XIII MEASUREMENT CAPABILITY - THE NATIONAL MEASUREMENT SYSTEM - TRACEABILITY XIV REFERENCE MATERIALS XV REPORTING DATA XVI VALIDATION XVII LABORATORY CERTIFICATION/EVALUATION XVIII PLANNING QUALITY ASSURANCE PROGRAMS XIX APPENDICES A (Reprint) - Quality Assurance of Chemical Measurements B (Reprint) - Sampling for Chemical Analysis C Guidelines for Evaluating the Blank Correction D Validation of Analytical Methods E Selected references ' August 1984 INTRODUCTION Digitized by the Internet Archive in 2012 with funding from LYRASIS IVIembers and Sloan Foundation http://archive.org/details/qualityassuranceOOtayl MEASUREMENT "If you can measure what you speak of and can express it by a number, you know something about your subject; but if you cannot measure it, your knowledge is meagre and unsatisfactory . . . ' Lord Kelvin "The ability to measure is one of man's great capabilites" F. D. Rossini "The trouble with measurement is its seeming simplicity" Anon "Measure what is measurable and render measurable that which is not yet measurable" Galileo Galilei 1564-1642 1-1 MEASUREMENTS NEEDED FOR Basic Science Identification of Problems Solving Problems Control of Production Evaluation of Products and Services in Virtually Every Area of Human Life and Welfare MEASUREMENTS ALLOW HUMANS To: Describe Predict Communicate Decide Control React The Questions WHEN USED FOR DECISIONS How Good? How Sure? Are rightfully asked and must be properly answered A GOOD METROLOGIST MUST BE ABLE TO: Make Good Measurements Define His/Her Measurements Defend the Measurements Technically Sound Legally Defensible Interpret His/Her Measurements Supportable Value Judgments Need to be Made Quality 1-2 TWO KINDS OF QUALITY Design Quality Design developed to meet a need Conformance Quality The actual production of a product that meets the need Quality is the Absence of Defects Deficient - General Inadequacy Defective - Wanting in some essential property, faulty QUALITY PARAMETERS QUALITATIVE IDENTIFICATION "Certainty" QUANTITATIVE ACCURACY Probability Confidence Limits Positive identification is the first requirement for reporting data Limits of uncertainty are needed to judge the confidence associated with the numerical result. A LABORATORY IS EXPECTED TO BE ABLE TO SPECIFY THE QUALITY OF ITS DATA IN QUANTITATIVE TERMS. THIS REQUIRES THE EXISTENCE OF SOME DEGREE OF QUALITY ASSURANCE. 1-3 DEFINITIONS Measure - to ascertain the extent, degree, quantity, dimensions, or capability with respect to a standard, hence to estimate. Measurement A Process An Operation A Piece of Data 1 . Act or result of measuring something 2. The extent, size, capacity, amount or quantity ascertained by measuring 3. A system of measures 4. Math - the correlation with numbers of entities that are other than members of aggregates CHEMICAL MEASUREMENTS Measurements of chemical substances Measurements which chemists make to obtain information on chemical substances and chemical systems The data obtained as a result of measurements on chemical substances or chemical systems QUALITY ASSURANCE OF CHEMICAL MEASUREMENTS OBJECTIVES To Assess Limits of Error in Measurements To Reduce Analytical Errors to Acceptable Levels To Reduce Amount of Work Needed to Obtain Reliable Data To Provide Basis for Intercomparison of Data TWO CONCEPTS Quality Control Quality Assessment BASIC REQUIREMENTS Understand the Nature of Errors Understand the Measurement System Used Develop Techniques and plans to Minimize Error I-n DEFINITIONS QUALITY CONTROL - The overall system of activites whose purpose is to provide a quality of product or service that meets the needs of users; also, the use of such a system. The aim of quality control is to provide quality that is satisfactory, adequate, dependable, and economic. The overall system involves integrating the quality aspects of several related steps including: (a) the proper specification of what is wanted; (b) production to meet the full intent of the specification; (c) inspection to determine whether the resulting product or service is in accord with the specifications; and (d) review of usage to provide for revision of specifications. QUALITY CONTROL - A system of inspections, testing, and remedial actions applied to a process or operation so that, by inspecting a small portion (a sample) of the product currently produced, an estimate can be made of its quality and whether or not, or what if any, changes need to be made to achieve or maintain a predetermined or required level of quality. QUALITY ASSESSMENT - A system of activities whose purpose is to provide assurance that the overall quality control job is in fact being done effectively. The system involves a continuing evaluation of the adequacy and effectiveness of the overall quality control program (see quality control) with a view to having corrective measures initiated where necessary. For a specific product or service, this involves vertif ications , audits, and the evaluation of the quality factors that affect the specification, production, inspection, and use of the product or service. QUALITY ASSURANCE QUALITY CONTROL/QUALITY ASSESSMENT QUALITY CONTROL - Those procedures and activities developed and implemented to produce a product/measurement of required quality. QUALITY ASSESSMENT - Those procedures and activities utilized to verify that the quality control system is operating within acceptable limits. 1-5 QUALITY ASSURANCE TECHNICALLY SOUND AND LEGALLY DEFENSIBLE There is a difference and both requirements need to be met. The first is necessary but not sufficient. It includes all those things that a careful competent analytical chemist would do. The second includes the proof that sound work was done. Quality assurance practices are central to both. They stress documentation which takes the burden off memory and can remove any suspicion or shadow of doubt of details of what was done. Such measures cannot give credence to poor technical work. In fact they can help to identify poor quality when it exists. But lack of documentation can make defense deficient if not impossible. While legal defense is often the issue, technical reliablity is always at stake. Quality assurance practices can promote this, as well. QUALITY ASSURANCE is the name given to procedures by which one ASCERTAINS that INDIVIDUAL MEASUREMENTS are GOOD ENOUGH for their INTENDED PURPOSE. The SHADOW OF DOUBT should be SUITABLY SMALL to permit VALID SCIENTIFIC INFERENCE EFFECTIVE PROCESS CONTROL QUALITY PRODUCTS INTELLIGENT ACTIONS QUALITY ASSURANCE vs QUALITY CONTROL Quality Control - Old Concept 1 Widely endorsed All good workmen strive for quality outputs Quality recognized as necessary Quality Assurance - Recent Concept Statistical control of quality Resistance to QA Based on o Improve technical methods so that no important quality variations remain o Statistics have no proper place among scientific production methods o If product is good, no inspection is needed; if not, inspection will not help 1-6 QUALITY ASSURANCE OF A PRODUCT o For Conformance with Specifications o Involves o Sampling a Defined Population o Measurement of a Distinctive Property o Variance of Measurement System often Considered Negligible OF A MEASUREMENT o Data Output may be Considered as the Product o Typical Classes Include o Calibrations o Chemical Analysis o Involves o Establishing statistical control o Evaluating precision by repetition o Evaluating bias by reference materials PRODUCTION PROCESS QUALITY ASSUR-AHCE f Quality "^^ V^ Control J rCASUREMEMT PROCESS QUALITY ASSURANCE 1-7 APPROACHES TO QUALITY ASSURANCE Craftsman/Artisan Approach Responsibility - Craftsman Effectiveness Depends on Craftsman's Knowledge, Skill, Dedication Requirements Highly Skilled/Dedicated Craftsman Atmosphere which Encourages Excellence Outstanding Characteristics Accuracy, Redundancy Works Best for Complex Investigations Controls Peer Review Formal Quality Assurance Program Responibility - Management Effectiveness Depends on Defined Protocols, Trained Operators, Strict Compliance Requirements Infallible Protocols, Competent Staff Outstanding Characteristics High Precision, Efficiency Works Best For Routine, Recurring, Well-Defined Problems Controls Quality Assurance Program/Office 'The English Language is the most important scientific instrument at your disposal. Learn to use it with precision." C. W. Foulk 1-8 QUALITY ASSURANCE OF LARGE AND SMALL OPERATIONS COMPARISONS LARGE PRODUCTION FACILITY vs. JOB SHOP MASS PRODUCTION vs. CUSTOM PRODUCTION LARGE OPERATIONS SYSTEM CAN BE WELL DEVELOPED FINE-TUNING JUSTIFIED COMMON CAUSES IDENTIFIABLE/CONTROLLABLE SPECIFIC QA POSSIBLE SPECIAL CAUSES RECOGNIZABLE SMALL OPERATIONS EXPEDIENCY INHIBITS SYSTEM DEVELOPMENT MINIMAL FINE-TUNING JUSTIFIED COMMON CAUSES/SPECIAL CAUSES INDISTINGUISHABLE GENERAL QA MORE APPLICABLE SMALL OPERATIONS DO NOTHING vs. DO SOMETHING SPECIFICS HOT SPOT APPROACH KEEP RECORDS OF OFFENDERS/PROBLEMS IDENTIFY HOT SPOTS BY PARETO ANALYSIS MAGNITUDE COST/RISK EASE OF SOLUTION CONCENTRATION ON HOT SPOTS METHODOLOGY EQUIPMENT MATERIALS PERSONNNEL 1-9 GENERAL IDENTIFY NON-SPECIFIC PROBLEMS INDEPENDENT OF WHAT ANALYZED? WHO ANALYZES? HOW ANALYZED? EXAMPLES - COMMON OPERATIONS CHEMICAL TREATMENTS REAGENTS/SOLVENTS/ WATER EXTRACTIONS STANDARDS CONTAMINATION HOUSEKEEPING SAMPLE PREPARATION FACILITIES DOCUMENTATICiJ/RECORDS/REPORTS MAINTENANCE SUPERVISION CONTROL CHARTS GENERAL CATEGORIZATION APPROACH FIND COMMONALITIES MATERIALS METHODOLOGY OPERATIONS OPERATORS POOL EXPERIENCE CONTROL ON BASIS OF COMMONALITIES GLP'S GMP'S MINI -SOP'S QUALITY ASSESSMENT ON BASIS OF COMMONALITIES TYPICAL REFERENCE MATERIALS TYPICAL CONTROL CHARTS I —10 II MEASUREMENT AS A PROCESS CHEMICAL ANALYSIS AS A SYSTEM MEASUREMENT AS A PROCESS PRODUCT IS NUMBERS MEASUREMENT PROCESS CAN BE IN CONTROL STATE OF STATISTICAL CONTROL RANDOMNESS ACCURACY AND PRECISION ARE CHARACTERISTICS OF THE PROCESS CAN BE APPLIED TO THE NUMBERS PRODUCED THE BODY OF DATA SUPPORTS INDIVIDUAL MEASUREMENTS (and vice versa) PROPERTIES OF MEASUREMENT PROCESSES Repeated measurements will disagree Means of repeated measurements will disagree Measurements at different times, places, by different operators/ apparatus/methods will disagree Some questions that need answers: Within what limits would an additional measurement by the same instrument agree when measuring some stable quantity? Would the agreement be poorer if the time interval between repetitions were increased? What if different instruments from the same manufacturer were used? If two or more types (or manufacturers) were used, how much disagreement would be expected? What effect does geometry (orientation, etc.) have on the measurement? What about environmental conditions-temperature, moisture, etc.? Is the result dependent on the procedure used? Do different operators show persistent differences in values? Are there instrumental biases or differences due to reference standards or calibrations? The answers to such questions serve to define the measurement process-the process whose "output" we seek to characterize. Likewise, the inability to answer such questions indicates weakness in knowledge of the measurement processs. II-1 MEASUREMENT as a PROCESS CHEMICAL ANALYSIS AS A MEASUREMENT SYSTEM PROCESS A progressive action, or a series of acts, especially in the regular course of performing, producing, or making something. SYSTEM An aggregation or assemblage of objects or processes united by some form of regular interaction or interdependence. Science Environrr.ent Health Material Production Solution Data Compliance Treatment Use Product II-2 CHEMICAL MEASUREMENT SITUATIONS MEASUREMENT VARIANCE SIGNIGICANT - Property Variance Not Significant Measurement Variance Known Estimated MEASUREMENT VARIANCE SIGNIFICANT - Property Variance Significant Measurement Variance Known Estimated Property Variance Estimated MEASUREMENT VARIANCE NOT SIGNIFICANT - Property Variance Significant Property Variance Estimated ANALYTICAL PROBLEM SOLVING ««^JTP'JT ' PROBLEM O'JTP'JT >^ MODEL I ►! ► ±±. PLAN SOLUnON : + "wHSURE\'E7n- CAUBRATION QUALITY CO.^ROL QUALITY ASSESS,VE.\ PLANNING PRIMARY SECONDARY • • • • DATA FLOW II-3 QUESTIONS ANALYTICAL CHEMISTS MUST OFTEN ANSWER 1. Mean of n measurements III-16 2. Standard deviation of the measurements III-16, III-17, III-18 3. Confidence interval for mean of n measurements III-20, III-21 4. Mean of property measured III-16 5. Standard deviation of property III-16, III-18 6. How does measured property compare with that from another laboratory III-23 7. Is the precision of measurement within specification/expectation 111-24 8. What confidence do I have in my precision III-21 9. How do precisions of two laboratories/methods compare III-24 Differ? Larger? Smaller? 10. Do measurements indicate product/value is within specification/ expectations III-23 Differ? Larger? Smaller? 1 1 . How to combine data from one/several sources to improve estimate of mean III-36 estimate of variance III-15 12. How to identify outliers III-30 13. How to evaluate measurement and property variances when both are or may be present III-18 14. How to estimate number of measurements/samples to estimate mean with prescribed precision V-4 estimate variance(s) with prescribed precision III-21 15. Limits for a given percentage of population of measurements/samples - statistical tolerance limits III-22 II-4 Ill STATISTICAL CONCEPTS SELECTED STATISTICAL CONCEPTS USEFUL WHEN EVALUATING MEASUREMENTS AND MEASUREMENT PROCESSES Basic Reference M. G. Natrella, NBS Handbook 91, Experimental Statistics QUANTIFICATION - Getting a Number UNCERTAINTY - Putting ± Limits on the Number MEASUREMENT AS A PROCESS o Product is Numbers o Process Stays in Lab, Numbers Go Out MEASUREMENT PROCESS CAN BE "IN CONTROL" ACCURACY AND PRECISION ARE CHARACTERISTICS OF THE PROCESS Can be Applied to the Numbers Produced STATE OF STATISTICAL CONTROL Measurements Behave Like Random Samples from a Probability Distribution POPULATION - All SAMPLE - Some STATISTICAL TECHNIQUES are TOOLS Rather Than ENDS AoaiyticaJ Error CONSEQUENCES OF ERROR iii-i SOURCES OF UNCERTAINTY ERRORS Systematic Known Unknown Random Accidential Random Component of Systematic Chance Causes Assignable Causes BLUNDERS AXIOMS A measurement is . . . ACCURATE when the value reported does not differ from the true value. UNBIASED when the error of the limiting mean is zero; freedom from systematic error. BIASED when the error of the limiting mean is not zero; influenced by systematic error. COROLLARIES ERROR in reported values occurs as a result of bias and imprecision. ACCURATE METHOD - One capable of providing precise and unbiased results (within acceptable limits). ACCURATE and PRECISE are relative terms III-2 PRECISION AND ACCl'RACV Limit im flean True Value Limit inn Mean True Value ACCURAC'' = PRECISION' + BIAS III-3 PRECISION - BIAS - ACCURACY 10 12 9.8 10.5 14 14 10.1 10.6 6 9 10.0 10.8 11 11 9.9 10.6 13 10 10.1 10.4 8 15 10.3 10.8 9 13 9.9 10.4 7 8 10.3 10.5 12 16 9.8 10.6 10 12 10.1 10.5 X 10. 12. 10.03 10.57 s 2. 6 2. 6 .18 .14 c.l. ±1. 9 ±1. 9 ±.13 ±.10 TEST SAMPLE VALUE 10.15 ± 0.05 PRECISION IS THE FIRST REQUIREMENT o Necessary to Define Operational Characteristics o Good Precision Necessary to Detect Small Constant Errors ".... A measurement process must have attained a state of statistical control; until a measurement operation has been "debugged" to the extent that it has attained a state of statistical control, it cannot be regarded in any logical sense as measuring anything at all." C. Eisenhart - Ref. 13 "Reproducibility is desirable, but it should not be forgotten that it may be achieved just as easily by insensitivity as by an increase in precision. Example: All men are two meters tall - give or take a meter." Anon III-4 PROBABILITY PLOTS RANK DATA according to value n = total number of data points i = rank = 1 to n CALCULATE PROBABILITY PLOT POSITION 100 (i-0.5) n PLOT on PROBABILITY PAPER Scale chosen for distribution postulated Fj^ on X axis Value on Y axis INTERPRET on BASIS OF FIT TO A STRAIGHT LINE See Reference 38 "Everybody believes in the exponential law of errors; the experimenters because they think it can be proved by mathematics; and the mathemati- cians because they believe it has been established by observation." Lippman, quoted by Poincare "In applying statistical theory, the main consideration is not what shape the universe is, but whether there is any universe at all. No universe can be assumed nor statistical theory applied unless the observations show statistical control Very often, the experimenter instead of rushing in to apply statistics and statistical methods, should be more concerned about attaining statistical control and asking himself whether any predictions at all (the only purpose of his experiment) by statisical theory or otherwise can be made." W. Edwards Deming Some Theory of Sampling J. Wiley, pp. 502-3 (1950) III-5 INPUT OUTPUT A N RAW MATERIAL A SOLVENTS L Y S CATALYSTS I CARRIERS S RECOVERY BY PRODUCTS A N \ A L PRODUCTS / Y S BY PRODUCTS I s III-6 POLLUTION PATHWAYS AIR / PLANTS / 1 AQUATIC ANIMALS MAN / TERRESTRIAL ANIMALS PLANKTON POTABLE WATER SOIL — \ / / f f ■ \ \ \ , / f SEA WATER INLAND WATER MARINE SEDIMENTS FLUVIAL SEDIMENTS III-7 MORr^AL UNIFORM LOG NORMAL DISTRIBUTIONS III-8 EXAMPLES OF DISTRIBUTIONS NORMAL 1 1 MODAL MULTIMODAL DETECTION-LIMIT DATA NOMINAL- VALUE DATA OUTLYING DATA III-9 • . i-A. i _._ #-- A # I f 10 50 50 TO ^O ^9 FEKCENT 0.1974 0.3521 1 0-3829 0.3011 0.5467 1,1, ■ i i 1- i i i :— 1 1 1 ■'.; ! 1 ' 1 I 0.2420 0.6827 0.1826 0.7887 0.1295 0.6664 ■1; ■ 1 1 ; ,1: :.! , — 0.0863 0.9199 0.0540 0.9545 0.0317 0.9756 U- ' i i : ; -j- ■•::-l; - : ■ ~ - - , :.. :.:.j::.: ■\- ::.,:. .j ..,.., 0.0175 ! 0.9876 0.0091 j 0.9940 0.0044 • 0.9973 ■ •-:■ .}:- '" ]'.'■■ --;■-•:-:!;::: :A\' .: 3 .00 ' 1 ^:^r-|^::f^:^ -■:-- ";:lj:::7::r~ -■-!■■■- I. ' . ' ■■ " ■ - 1 1 :■ : : i:- : i - ■ • ■ - - - -- • : 1 1 : ■ • J-J -:--"".::. i ■ ■" ■ 1 : . 1 1 -^f^- :^^ ■;:-.:- 1 I ^:r:;:^ :^^(:::: .:.; ;:■ -.i-]-: a-- ,.- ■ :::;! "v -: --:-^ - n--; -1- — --■■- !■ :...: .i... ■:•:-■ :--f;d-^ 1 ■ ^t - .1 ■: 1 '-■' •- i 1 — - - — ._:[:.:[:„ :-^'[r~\" ,-r,^l^:- : -- - .!.:: ^^/;::-if^^i-;; -A-\''-:: ^^Lr= iiiji;-. - 1 1 . : i- .-;:_. ::.:[::- :_:..:: ;:::;:... r: — ;. - 1 ' . • -.:[:-: :•::•-: :\i-.\^\ ^£J^l: ■ :;:•_!- -_::;:::. ::; : : i ' - = - 1- •■■i-— -_ i.:r. r .-:■ . \ ~, -;: ' ,-.. - :-. -.-- 1 -.-■ T--; r-rrri.- 1.-- ■-l-:- -.1^ u^dir' i:^ ^^ - ..1. ,_ i- ■ ^: ■::-: ^ ^..::^\ - !-^ -H-4^' :::- - ; 1 ;: •■•:-: ■ ■. ■ :: ;:::^ - ■"■[--. ::-.:;-- :- ;;_ ^-^ -■ ■ ■■- ^; !^-i .i r^'-^-- : . '■ ^ -'r_ ^-^ • t :-:,■■. ;.:-|:;:- -''}-■-': -i|:- -: rv d . ^ i : ! --;■ - : y ■■ ■ , " 1 -,{:;■ , ^ CN 1 • ■ !>:. ::-:h:: -■^;i:;^: 5 0- i . 1 : , ;. . ■ :.:..: 1 j 1 ■ ■:■; ■ P— ly i ^ '.;. " 1 .:.|:. J:::. ■;;.;i:. oo t ; : : . .;■:■■ 1 . |.- ■I- :: j:- — _. - - 00 i ' - : : ; _ 1 ■ 1^- • y '' :.-|-: ■:.i: 1 1- i - - - - - - - i ■ ; : ; ' ■ ij:- — t-: M- : i- . - - - - — - - - - - - '■-■ H- : I ::-..| •i-r;:: -::!-: -. j: :::^|-- :::t- ■ 1 j 1 j y> - - - -- - - _ I ^ I ^ i 1 ;"• T : r A: •O ! ■ : i : i . i" f ■ : 1 i ■ ■ .■I.:., ■ - - ._. - - ! [ : i i i 1 i ..| :■ 1 -■ ! • I : : i • - ^~ 1 ■ ! -: i;. O >• ^ < ■ - > ._. - .._ >3 — 1 : N : j i...;L.:. [■ •i ■ ! -!• -l-- 1 —'■ — 1 ^ - - - ^ — , • . ' ! ! ^ ^71— — V.i " J r I ^{ JL T IP (. e S \ \ P 5- > i i > 3 III-14 BASIC REQUIREMENTS STABLE INDEPENDENT RANDOM SOME GROSS DEVIATIONS FROM RANDOMNESS Problem TREND Diagnostic Short-Term Trend Jumps or Shifts Plots Fits Control Chart Runs Up and Down * Run Above and Below the Mean* Control Charts t-Tests Control Charts * See e.g., Acheson J. Duncan, Quality Control and Applied Statistics , 4th Edition, Irwin, Homewood, ILL, 1974. III-15 Population All measurements POPULATION VALUES AND SAMPLE ESTIMATES Sample Mean Standard Deviation o Variance o^ Sample of n measurements XI, X2. ^n ZXi n = /^:^^ 2 ^(Xj - x)2 ^ " n - 1 Alternative Computing Formula; n(n - 1) or PUSH BUTTON df = V = degrees of freedom, often n - 1 '■^= t standard error of mean — = coefficient of variation X f X 100 = Relative standard deviation X where POOLING ESTIMATES OF VARIANCE GENERAL CASE 2 _ VI Sf ^ V2 S2 ^ ' Vk Sg v^ + V2 +••••+ Vk ^k = ni, - 1 df= Z vi (ni-1) sf ->■ (n2-1) s| ("k-D sg n-) + n2 nk - k df = Z ni III-16 DUPLICATE MEASUREMENTS 1 k S2 = — Z d2 P 2k 1 where k = number of sets of duplicates dl = difference of duplicate measurement Sr) has k degrees of freedom USE OF RANGE TO ESTIMATE VARIABILITY Number Size of Sample of Samples k 2 3 4 5 6 1 d| 1.41 1.91 2.24 2.48 2.67 V 1.00 1.98 2.93 3.83 4.68 3 d| 1.23 1.77 2.12 2.38 2.58 V 2.83 5.86 8.44 11.1 13.6 5 4 1.19 1.74 2.10 2.36 2.56 V 4.59 9.31 13.9 18.4 22.6 10 dp 1.16 1.72 2.08 2.34 2.55 V 8.99 18.4 27.6 36.5 44.9 20 d| 1.14 1 .70 2.07 2.33 2.54 V 17.8 36.5 55.0 72.7 89.6 dp 1.13 1.69 2.06 2.33 2.53 R/dp ^ Adapted from Lloyd S. Nelson, J. Qual . Tech. 7 No. 1, January 1975. III-17 WITHIN AND BETWEEN STANDARD DEVIATION I. GENERAL CASE n Replicates of k samples (e.g., homogeneity) n Replicates on h occasions (e.g., long-/short-term s) n Replicates by k laboratories (e.g., Methodology) where 26 Tabulate R2 Highest-Lowest ^k 1 . Calculate R R2 _LRK 2. Calculate S^ = R/d2 (d2 for appropriate k) V = see Table III-17 3. Calculate s; XI ^ ^2 ^k E(x-[-x)' k-1 2 2 4. Calculate Sg = / (Sx - S;^) V = k - 1 II. SPECIAL CASE - S^ known n > 1 Omit steps 1 and 2 III-l TABLE 2-3. PROPAGATION OF ERROR FORMULAS FOR SOME SIMPLE FUNCTIONS (X and Y are assumed to be independent.) Function form Approximate formula for j^ nty, = Arrtx + Bnty A'si + ^*4 —=: (f)*(l-#) -^'k 1 "^ m. + m. (f)'(;'4 + --'^') "■"-rf^ 4 *m« = /iVn, 3 See Chapter 2, Handbook 91, also III-36. III-20 CONFIDENCE INTERVAL FOR THE MEAN (o Not Known, Using s From Sample) Form: s X ± tc -r ►^n Where: X = sample mean n = no-, of measurements in sample s = calculated from sample tc from t-table (e.g., Table pIII-38) depends on: confidence level and degrees of freedom associated with _s (D.F. = n - 1 in simple series). For example, if s_ has 4. D.F. (n = 5), For 905s conf. int., t^ = 2.132 95% conf. int., tc = 2.776 995^ conf. int., t^ =4.604 See Chapter 2, Handbook 9''. CONFIDENCE INTERVAL FOR Lower limit: B^s Upper Limit: Bys Interval: B^s to B^s Bl and By from table (e.g., Table pIII-39) depend on confidence level and degrees of freedom associated with s_, For example, if s has 4 D. F., 95% conf. int., .599 s to 2.567 s 995s conf. int., .4865 s to 3.892 s Another example, if s has 30 D.F., 955^ conf. int., .791 s to 1.321 s 99? conf. int., .740 s to 1.457 s III-21 STATISTICAL TOLERANCE INTERVALS (m and o Known) Form: m ± zo where: z depends only on P, the percentage of the Individual measurements to be included in the interval. For example, To include 50?, z = .67 905J, z = 1.6M5 955S, z = 1.96 > 2 9956, z = 2.576 99.8$, z = 3.09 — > 3 See Chapter 2, Handbook 91. STATISTICAL TOLERANCE INTERVALS (Using X and s from Sample) Form: X ± k s k depends on three things; P, the proportion or percentage of individuals to be included; Y, the confidence coefficient to be associated with the interval; D.F., the degrees of freedom associated with £. For P = 90% For P = 99? Y = .90 Y = .99 n = 2 (D.F. = 1 ) k = 15.98 k = 2M2. n = 5 (D.F. = 4) k = 3.il94 k = 10.26 n = 10 (D.F. = 9) k = 2.535 k = 5.59 See Chapter 2, Handbook 91, See also III-i<1 . REMINDER Confidence Interval: is expected to include the mean or the standard deviation. Tolerance Interval: is expected to include ?% of the individual measurements. The Interpretation of "Expected" is the Same in Each Case III-22 COMPARING A SET OF MEASUREMENTS WITH A STANDARD MEAN VALUE (nio) Given: (tiq Set of Measurements: x-] , X2 ••• x^^ 1. Calculate: x, s (or use o known) 2. Calculate: confidence interval for mean ts X ± z —r- or X ± — - /n /n 3. mQ is given. If confidence interval includes mQ, set is consistent with standard mean value, mQ. See Chapter 3, Handbook 91. COMPARING TWO SETS WITH REGARD TO THEIR MEANS (Using Confidence Intervals) Have: two sets of measurements, A and B Calculate: xp^ xg SA SB (df = n^ - 1) (df = ng - 1) Confidence Interval for Difference of Means: (XA - xg) ± te sp / -^p^ where tQ depends on: - confidence level, D.F. for Sp - D.F. for Sp = n^ + ng - 2 here - from Table (e.g., A-4 in Handbook 91), or page III-38. and _ ^ ("A - 1)3^A - (ng - 1)s-B If confidence interval includes zero, consider sets consistent with regard to mean . See Chapter 3, Handbook 91. Note: If Sa and Sg are considered to be different, calculate individual confidence intervals and check for overlap. III-23 COMPARING A SET OF MEASUREMENTS WITH A STANDARD VALUE FOR VARIABILITY (oq) Given: Oq Set of Measurements: x-|, X2, ••• x^ 1 . Calculate: s 2. Calculate: confidence interval for o B^s to Bys 3. Given Oq. If confidence interval includes Oq, set is consistent with this standard value. See Chapter 4, Handbook 91. COMPARING TWO SETS WITH REGARD TO VARIABILITY (F Test) Have: Set A Set B s^A s2b "A "B (D.F. = n;^ - 1 ) (D.F. = nB - 1 ) 2 Sfl Calculate: F = — ^ S2 B Compare with table value of F (e.g., Table A-5, Handbook 91) or p. Ill- depends on: - significance level of test - degrees of freedom of numerator and denominator See Chapter 4, Handbook 91; see also III-40. III-24 IS LINEAR ASSUMPTION JUSTIFIED Grapical Approach Plot data and attempt to draw "best" straight line If straight line is "easy" to draw, linearity is probably justified Fit function, y = Bq + Bi x, by least squares Compute 60 and sg^ 6^ and sg^ Judge significance of Bq "^d Bi Compute Ay = yobs " Ycalc. Tabulate sign changes and sign follows Should not be large differences between them Examine lengths of runs Plot A vs. X and look for absence of functional trends. CORRELATION COEFFICIENT APPROACH COMPUTE CORRELATION COEFFICIENT, r n Zxy - E X Zy / [nZx2 - (Zx)2] [nZy2 - (Zy)2)] r = , no correlation r = +1 , perfect positive correlation r = -1 , perfect negative correlation r = immediate values, usual case, use Table A17 to interpret, see p III-29. Note: Correlation coefficient is often misused. See H. T. Arm, "The Significance of the Correlation Coefficient for Analyzing Engi- neering Data," Materials Research and Standards Vol. 11, No. 5, pp. 16-19 (1971). III-25 LINEAR RELATIONSHIPS t (preselected ) t of individuals selected height red beforehand; values of X are -h to measure Y. 1 1 (M U3 -^ exist in the r computed experiment a distorted correlation. W> 00 S II Correlation maj population, but from such an would provide estimate of the i X = Heigh values Y = Weigh of pre X is measu only selected used at whic c i £ + II ll id on a ran- f individuals, elected but sample unit. « £ ^ 1 Sj K X = Height Y = Weight Both measur dom sample X is not £ "comes with" Ordinarily ne pared to van individuals. + + .0 is" 1 f 2 §^« £ atical formu not observ in one or bo ant of a spri -known weig anding elong 1 "0 CO •a depends ptions can aragraph 5-4 1 athem liich is errors const rately orresp "O 1 5^^ III 1 ■3 5 z ed by a m + P[y, w bances or of elastic X = accu value pf c is 1 m ^^ c 3 ■S '<«f 3 § 5- ^ S •^ al l£ arly rel »r X = of dist •minati oke's la leasure ei ^ 1 s: X and y are line exactly because variables. Example: Detei which obeys Ho applied, Y = n tion y. c .0 0. g OS 3 i? 1^ -1- a c5 "^ q" C Distinctive Features and Example c 1 fl II c ^ .2 c 11 Chapter 5. Handbook 91 III-26 BASIC WORKSHEET FOR ALL TYPES OF LINEAR RELATIONSHIPS X fipnofiis y Hpnntps T.Y = ^y = t » v^ = Nnrnhpr nf pninh 4 III-31 PROBLEM OF REJEOING OBSERVATIONS 17-3.1 WHEN EXTREME OBSERVATIONS IN EITHER DIRECTION ARE CONSIDERED REJECTABLE 17-3 1.1 Population Meon and Standard Deviation Unknown — Sampl« in Hand is th« Only Source of Information. (Tho Dixon Crtterion] Procodur* Rank Data In Order of Increasing Size Xj ^2 X^ X£| .X n-1. Choose or, the probability or risk we are willing to take of rejecting an observation that reaib belongs ip the group. If: 3 < n < 7 8 < n < 10 11 < n < 13 14 < n < 25 where r,; is computed as follows: rif If X„ is Suspect Tin iX, iX. X._,)/(J^, - A-,) X..,)/(X. - X,) X...,)/{X„ - X,) X,_,)/(X, -A-,) Compute r,,, Compute Til Compute r^ Compute r.;, If Xi_is Suspect (X, -'Xi)/{X. - X,) (X, - Z,)/(X,-. - X,) (X, - A,)'(X,.. - X.) (.Y:, - X,)'(X„_, - X,) Look up r,_. , for the r,, from Step (2), in Table A-U. If r,v > ri_„ ,, reject the suspect observation; otherwise, retain it. III-32 TABLE A-14. CRITERIA FOR REJECTION OF OUTLYING OBSERVATIONS Number of Upper Percentiles Statistic Obser- vations, n .70 .80 .90 .95 .98 .99 .995 3 .684 .781 .886 .941 .976 .988 .994 4 .471 .560 .679 .765 .846 .889 .926 Tw 5 .373 .451 .557 .642 .729 .780 .821 6 .318 .386 .482 .560 .644 .698 .740 7 .281 .344 .434 .507 .586 .637 .680 8 .318 .385 .479 .554 .631 .683 .725 rii 9 .288 .352 .441 .512 .587 .635 .677 10 .265 .825 .409 .477 .551 .597 .639 11 .391 .442 .517 .576 .638 .679 .713 rn 12 .370 .419 .490 .546 .605 .642 .675 13 .351 .399 .467 .521 .578 .615 .649 14 .370 .421 .492 .546 .602 .641 .674 15 .353 .402 .472 .525 .579 .616 .647 16 .338 .386 .454 .507 .559 .595 .624 17 .325 .373 .438 .490 .542 .577 .605 18 .314 .361 .424 .475 .527 .561 .589 19 .304 .350 .412 .462 .514 .547 .575 Tn 20 .295 .340 .401 .450 .502 .535 .562 21 .287 .331 .391 .440 .491 .524 .551 22 .280 .323 .382 .430 .481 .514 .541 23 .274 .316 .374 .421 .472 .505 .532 24 .268 .310 .367 .413 .464 .497 .524 25 .262 .304 .360 .406 .457 .489 .516 III-33 RANKING TEST TO IDENTIFY OUTLIERS Table 4 App roximate 5% two -tail limits for ranking scores No. of Number of Material* UU. 3 4 5 8 7 8 9 10 u 12 13 14 15 4 5 7 8 10 12 13 15 17 19 20 22 3 12 15 17 20 22 24 27 29 31 33 36 38 4 6 8 10 12 14 16 18 20 22 24 26 4 16 19 22 25 28 31 34 37 40 43 46 49 5 7 9 11 13 16 18 21 23 26 28 31 5 19 23 27 31 35 38 42 45 49 52 56 59 3 5 7 10 12 15 18 21 23 26 29 32 35 6 18 23 28 32 37 41 45 49 54 58 62 66 70 3 5 8 11 14 17 20 23 26 29 32 36 39 7 21 27 32 37 42 47 52 57 62 67 72 76 81 3 6 9 12 15 18 22 25 29 32 36 39 43 8 24 30 36 42 48 54 59 65 70 76 81 87 92 3 6 9 13 16 20 24 27 31 35 39 43 47 9 27 34 41 47 54 60 66 73 79 85 91 97 103 4 7 10 14 17 21 26 30 34 38 43 47 51 10 29 37 45 52 60 67 73 80 87 94 100 107 114 4 7 11 15 19 23 27 32 36 41 46 51 55 11 32 41 49 57 65 73 81 88 96 103 110 117 125 4 7 11 15 20 24 29 34 39 44 49 54 59 12 35 45 54 63 71 80 88 96 104 112 120 128 136 4 8 12 16 21 26 31 36 42 47 52 58 63 13 38 48 58 68 77 86 95 104 112 121 130 138 147 4 8 12 17 22 27 33 38 44 50 56 61 67 14 41 52 63 73 83 93 102 112 121 130 139 149 158 4 8 13 18 23 29 35 41 47 53 59 65 71 15 44 56 67 78 89 99 109 119 129 139 149 159 169 W. J. Youden. in NBS Special Publication 300. H. Ku. Editor. Ref. 26 111-34 COMPUTING MEAN OF DATA SETS CASE I All Sets Have Same Precision Case lA Same Number of Measurements in Each Set = ^ XI ^ X2 + Xk X X ''^ ^ 7k ^ 7nk s- -^^ X /nk n = no. of meas. in each set. Case IB Unequal Number of Measurements in Each Set Assign wts., W, equivalent to no. of meas. in set _ x^Wt . X2W2 ..... X^W), X - Wi + W2 Wk Case II Data Sets Have Different Precisions Due to Number and Imprecision 1 Compute Wts., W = _ _0X_ X /n _ 1L_ X /n III-35 X • 2 + X • 2 + X • 2 1 0- 2 0- k 0- XX X 1 2 k X = 1 1 1 2 + 2 + 2 0- 0- X X X 12 k Ordinarily, s will be known, rather than o X = above with s substituted for o 1 Wi + W2 + Wk 1 1 where W = — W = — 2 2 s Z-factors for 2-sided confidence interval Confidence Level Z Factor 5055 0.67 67 1.00 75 1.15 90 1.645 95 1.960 95.28 2.000 99.00 2.575 99.74 3 99.9934 4 99.99995 5 10-9 6 10-''2 7 10-15 8 10-18.9 9 10-23 10 III-36 Combining Data Sets REPORT WEIGHTED MEAN AND RATIONALE REPORT REJECTED VALUES AND RATIONAL REPORT INDIVIDUAL VALUES AND RATIONAL REJECT ALL SUCH DATA AS "QUALITY UNKNOWN" III-37 TABLE A-4. PERCENTILES OF THE t DISTRIBUTION df /.«. ^70 t.so t.^ t^ t.,:. tn t^ 1 .325 .727 1.376 3.078 6.314 12.706 31.821 63.657 2 .289 .617 1.061 1.886 2.920 4.303 6.965 9.925 3 .277 .584 .978 1.638 2.353 3.182 4.541 5.841 4 .271 .569 .941 1.533 2.132 2.776 3.747 4.604 5 .267 .559 .920 1.476 2.015 2.571 3.365 4.032 6 .265 .553 .906 1.440 1.943 2.447 3.143 3.707 7 .263 .549 .896 1.415 1.895 2.365 2.998 3.499 8 .262 .546 .889 1.397 1.860 2.306 2.896 3.355 9 .261 .543 .883 1.383 1.833 2.262 2.821 3.250 10 .260 .542 .879 1.372 1.812 2.228 2.764 3.169 11 .260 .540 .876 1.363 1.796 2.201 2.718 3.106 12 .259 .539 .873 1.356 1.782 2.179 2.681 3.055 13 .259 .538 .870 1.350 1.771 2.160 2.650 3.012 14 .258 .537 .868 1.345 1.761 2.145 2.624 2.977 15 .258 .536 .866 1.341 1.753 2.131 2.602 2.947 16 .258 .535 .865 1.337 1.746 2.120 2.583 2.921 17 .257 .534 .863 1.333 1.740 2.110 2.567 2.898 18 .257 .534 .862 1.330 1.734 2.101 2.552 2.878 19 .257 .533 .861 1.328 1.729 2.093 2.539 2.861 20 .257 .533 .860 1.325 1.725 2.086 2.528 2.845 21 .257 .532 .859 1.323 1.721 2.080 2.518 2.831 22 .256 .532 .858 1.321 1.717 2.074 2.508 2.819 23 .256 .532 .858 1.319 1.714 2.069 2.500 2.807 24 .256 .531 .857 1.318 1.711 2.064 2.492 2.797 25 .256 .531 .856 1.316 1.708 2.060 2.485 2.787 26 .256 .531 .856 1.315 1.706 2.056 2.479 2.779 27 .256 .531 .855 1.314 1.703 2.052 2.473 2.771 28 .256 .530 .855 1.313 1.701 2.048 2.467 2.763 29 .256 .530 .854 1.311 1.699 2.045 2.462 2.756 30 .256 .530 .854 1.310 1.697 2.042 2.457 2.750 40 .255 .529 .851 1.303 1.684 2.021 2.423 2.704 60 .254 .527 .848 1.296 1.671 2.000 2.390 2.660 120 .254 .526 .845 1.2S9 1.658 1.980 2.358 2.617 00 .253 .524 .842 1.282 1.645 1.960 2.326 2.576 III-38 TABLE A-20. FACTORS FOR COMPUTING TWO-SIDED CONFIDENCE LIMITS FOR — ©oot~tc cgcJ — — — o CO c- t~ t' CO -H ^ «J ■JOC5ao eococoe^evi to 00 CM «=^ c~ CO to uo in CM CM CM CM CM tOCMOTt6cO CMCMegrJci eg eg eg eg CM eMCM — — « •o a» ■«? !>J ^o «duo-^-»eo VCCO — oc^ COC5 WCOM CM CM CM eg CM CM eg eg eg eg CM eg eg eg' eg' — ooto-^eo CO — ©oo eg CM eg — — 2 P3* iocotficjoo «eu3-»Trco <0 -^ CM >-• CO CO CO CO CO 05 00 cot- c- CM CM CM CM CM (N eg CM eg CM CM CM eg eg CM TTCM— ©0> CM CM eg CM -< o to U5 rr ■*■ CO e^ CO so CO C3 © 05 C7> 00 00 CO CM ei CM cm' r- CO © t~ -^ C~f-C-tOtO CM CM eg eg cm' eg CM CM CM CM U0?5CM-© CM cm' eg CM CM o- pS^ usCooocoq CO CO CO CO CO CM Ui 00 CO 06 — © CT> CX> 00 CO CO ej CM CM CMCMCMeJCM CM CM CM CM CM vn -a- CO ^o — eg eg ei eg eg' m t— - 10 TT -.T -W eoeococrieo CM —i © cr> CO CO' CO CO CM CM CM CM eg eg' c-c~c-toto CM CM CM eg e^i CM eg tM eg cm K CMCCOO oca>- TT CO CO CO CO CO CO C^l CM — — ' © eoK^coeoco CO eg eg CM CM CM CJ CM CM CM t>toocoeM CM CM eg CM eg uo uo -oo CM CM eg eg CM OOC-tOiOTji eg CM CM eg eg' n t-: iri uo ■»■ TT "cr Tji CO CO CO cococoeoeo CS CO CO CO CO — — ©©© CO CO CO eo CO ©Ot-tOUO eo eg eg CM CM - ta CM — to cocMi/:oc- t-^ «o 10 Lo ■«; ■^ CM — 00 "9^ •>»■ -OT ■«r ■>» -w CO ci cocieocico to to to to to coco CO CO CO .n^coCM-H cococoeoeo " crioitbd •» C^J i-o t- odt^«j«JkO -■COCOuOiO -a; TT CO CO CO cj CM rj eg CM i^sgs ■«r •<»■ CO eo CO - t-'odt-^cvi •^ CO <-■ •-• CD C3obadc-^c^ «j <£> «5 <£> «£> CM— i©a>a> tocotoiraio ooocr-r-c- U5 ui irt iri to to to CO in ininiriuiio uiuoouiio S / -«n* in«Ka»' o-ixn* ■1'0k«(» — r<.-n.«» oooOg jO|Duiuiouap joj uiopaaij jo saejQsp = ^w III-MO TABLE A-6 (Continued). FACTORS FOR TWO-SJDED TOLERANCE LIMITS FOR NORMAL DISTRIBUTIONS 7 = 0.95 7 = 0.99 X 0.75 0.90 0.95 0.99 0.999 0.75 0.90 0.95 0.99 0.999 2 22.858 32.019 37.674 48.430 60.573 114.363 160.193 188.491 242.300 303.054 3 5.922 8.380 9.916 12.861 16.208 13.378 18.930 22.401 29.055 36.616 4 3.779 5.369 6.370 8.299 10.502 6.614 9.398 11.150 14.527 18.383 3.002 4.275 5.079 6.634 8.415 4.643 6.612 7.855 10.260 13.015 6 2.604 3.712 4.414 5.775 .7.337 3.743 5.337 6.a45 8.301 10.548 y 2.361 3.369 4.007 5.248 6.676 3.233 4.613 5.488 7.187 9.142 2.197 3.136 3.732 4.891 6.226 2.905 4.147 4.936 6.468 8.2:34 2.078 2.967 3.532 4.631 5.899 2.677 3.822 4.550 5.966 7.600 1.987 2.839 3.379 4.433 5.649 2.508 3.582 4.265 5.594 7.129 1.916 2.737 3.259 4.277 5.452 2.378' 3.397 4.045 5.308 6.766 1.858 2.655 3.162 4.150 5.291 2.274 3.250 3.870 5.079 6.477 1.810 2.587 3.081 4.044 5.158 2.190 3.130 3.727 4.893 6.240 1.770 2.529 3.012 3.955 5.045 2.120 3.029 3.608 4.737 6.043 1.735 2.480 2.954 3.878 4.949 2.060 2.945 3.507 4.605 5.876 1.705 2.437 2.903 3.812 4.865 2.009 2.872 3.421 4.492 5.732 1.679 2.400 2.858 3.754 4.791 1.965 2.808 3.345 4.393 5.607 1.655 2.366 2.819 3.702 4.725 1.926 2.753 a. 279 4.307 5.497 1.635 2.337 2.784 3.656 4.667 1.891 2.703 3.221 4.230 5.399 20 1.616 2.310 2.752 3.615 4.614 1.860 2.659 3.168 4.161 5.312 21 1.599 2.286 2.723 3.577 4.567 1.833 2.620 3.121 4.100 5.234 22 l..'->84 2.264 2.697 3 . 54.1 4.523 1.808 2.584 3.078 4.044 s-ie.-? 23 1.570 2.244 2.G73 3.51-J 4.484 1.785 2.551 3.040 3.993 5.098 24 1.557 2.225 2.651 3.483 4.447 1.764 2.522 3.004 3.947 5.039 25 1.545 2.208 2.631 3.457 4.413 1.745 2.494 2.972 3.904 4.985 26 1.534 2.193 2.612 3.432 4.382 1.727 2.469 2.941 3.865 4.935 27 1.523 2.178 2.595 3.409 4.353 1.711 2.446 2.914 3.828 4.888 Ill-i^l TABLE A-36, SHORT TAHLE OF RANDOM NUM3ERS 46 96 85 77 27 92 iiS 2G <5 21 89 91 Tl 4J 64 64 53 22 75 31 74 91 48 46 13 44 19 15 32 63 55 S7 77 33 29 45 00 31 34 54 05 72 90 44 27 78 tl 07 62 17 34 59 80 62 24 33 SI 67 2S li 34 79 26 25 34 23 O'J 94 00 SO 55 31 63 :27 91 74 97 80 30 65 07 71 30 01 94 47 45 b9 70 74 13 04 90 51 27 61 34 6.J 8.7 44 22 14 CI 60 S5 3i 33 71 13 S3 72 08 16 13 50 56 48 51 29 43 30 9J 45 65 23 40 03 &6 40 03 47 24 50 09 21 21 18 00 05 86 52 85 40 73 73 57 6.S .-.G 33 91 52 33 76 44 56 15 47 75 73 73 78 19 87 06 98 47 4S 02 62 03 42 0". 32 55 02 87 59 20 40 93 17 £2 24 19 ?0 GO S7 32 74 59 S4 24 49 79 17 23 7> 83 42 Go 11 02 55 57 43 84 74 36 22 67 19 20 15 02 53 37 13 75 54 89 5:3 73 23 39 07 10 23 79 26 54 54 71 S3 S9 74 C8 48 23 17 49 18 81 05 52 85 70 05 73 U 17 C? 59 2S 25 47 89 11 65 65 20 42 23 9G 41 64 20 30 89 87 64 37 93 36 SG 35 93 50 75 20 09 13 54 34 63 02 54 57 23 05 43 3C 93 29 57 93 S7 08 20 92 9S 24 43 23 72 80 C4 34 27 23 43 15 36 10 r.3 21 59 63 76 02 62 31 62 47 t;0 C4 39 91 63 18 38 27 10 78 ?8 84 42 32 00 97 92 00 04 94 50 05 75 82 70 J-0 35 74 62 19 67 54 IS 25 92 33 69 93 96 74 35 72 11 Cs 25 03 95 31 7'> 11 75 5-! 91 03 35 CO 61 16 CI 97 25 14 78 21 22 05 25 47 26 37 60 39 19 C6 41 02 00 42 57 66 7G 72 91 03 63 48 46 44 01 33 53 62 28 80 59 55 05 02 16 13 17 51 OS iS 03 06 15 03 72 28 01 S3 25 37 C6 48 55 19 56 41 29 28 76 49 74 39 50 92 70 95 70 C9 SO 87 14 25 49 25 94 C2 78 26 15 41 39 48 75 64 69 61 Oo 3S 91 08 83 53 52 13 04 «2 23 00 26 35 47 41 04 08 84 80 07 44 73 51 52 41 i/j (A 85 97 74 47 53 90 05 90 84 87 48 25 01 11 05 45 11 43 15 60 40 31 8J 59 59 54 13 0.1 13 fO 42 29 63 03 24 64 12 43 28 10 01 65 62 07 79 83 05 50 51 39 IS 32 b"9 S3 46 58 19 34 03 59 28 97 81 02 C5 47 47 70 29 74 17 30 22 65 67 43 SI 03 12 CO 19 57 C3 7S 11 SO 10 57 15 70 04 89 81 78 54 84 87 83 42 61 75 37 19 £6 90 75 39 03 5G 49 92 72 95 27 52 87 47 12 52 54 62 43 23 13 78 10 21 n 00 63 19 63 74 £8 69 03 51 38 60 30 53 5G 77 06 69 C3 89 91 24 93 23 71 58 09 78 08 03 07 71 79 32 25 19 CI 04 40 S3 12 06 78 91 97 88 95 37 55 <8 82 63 89 52 59 M 72 19 17 22 61 90 20 03 64 96 60 48 01 95 41 S4 U 13 11 71 17 23 29 25 13 85 33 35 07 63 25 68 67 92 57 11 84 44 01 33 60 29 89 97 47 03 13 20 86 22 45 69 98 64 M 89 C4 94 81 65 87 73 81 58 40 42 J6 94 85 82 89 07 17 SO 29 89 89 80 98 36 25 36 53 02 49 14 34 03 52 09 20 04 93 10 59 75 12 93 84 60 S3 68 16 87 60 11 50 46 56 58 45 88 72 50 -IS 11 95 71 43 63 97 IS 65 17 13 08 CO 50 77 50 46 92 45 26 97 21 43 22 23 OS :52 86 05 33 14 35 48 68 18 36 57 09 62 40 28 87 08 74 79 91 08 27 12 43 Z'l 03 £9 30 eO 10 41 31 00 C9 63 77 01 89 9^ 60 19 02 70 88 72 33 38 S3 20 60 83 05 45 S5 40 54 03 S3 96 76 27 77 84 80 03 64 60 44 S4 54 24 85 20 S5 77 32 71 85 17 74 66 27 85 19 55 56 51 35 48 92 32 44 40 47 10 38 22 52 42 2Ji SG 80 20 32 80 98 00 40 52 57 51 52 83 14 55 31 99 73 23 40 07 64 54 44 99 21 13 50 78 02 73 39 66 82 01 28 67 51 75 66 33 97 47 58 42 44 88 09 2S 58 CS 67 92 65 41 45 SS 77 96 4C 21 14 39 56 SG 70 15 74 43 62 69 S2 30 77 28 77 72 56 73 44 26 04 62 81 15 35 79 26 S9 57 28 22 25 91 SO 62 95 48 98 23 86 28 86 85 64 94 11 53 78 45 36 34 45 91 3S 51 10 63 36 87 81 16 77 30 15 36 C9 57 40 80 44 94 60 82 94 93 OS 01 48 50 57 69 60 77 69 CO 74 22 05 77 17 71 20 03 30 79 25 74 17 78 84 54 45 04 77 42 59 75 78 64 99 37 03 IS 03 35 89 98 55 93 22 45 12 49 82 71 67 33 23 69 50 59 15 09 25 79 39 42 84 IS 70 58 74 82 81 14 02 01 05 77 94 65 57 70 39 42 48 56 84 31 59 18 70 41 74 CO 50 54 73 81 91 07 81 26 25 <5 49 61 22 83 41 20 00 15 59 93 51 CO 65 fj 63 49 as 72 90 10 20 65 28 44 63 95 86 75 78 69 24 41 65 86 10 S4 10 32 00 93 n 85 01 43 65 02 83 69 56 88 34 29 M 35 48 15 70 U 77 83 01 34 82 91 04 84 22 46 41 84 74 27 02 57 77 47 93 72 02 95 63 75 74 69 69 61 34 31 92 13 III-42 IV MODELING MODELING More or Less Idealized Representation of an Often Complex Reality One Can Model Objects Phenomena Processes Qualities of a Satisfactory Model o Undistorted, Unbiased picture (A group of models may be needed) o Simplicity (as far as possible) to allow interpretation (sets of simple logical elements may be best) o Coverage of all essential parts (no significant gaps) o Interlinkage defined for all related components EXAMPLES OF MODELS Automobile Emissions Emissions During Driving Cycle Simulated Cycle Composite Sample Relation to Other Models o Air Dispersion Model Stack Concentration Dispersed Concentration Plume Considerations Fumigations o Dissolved Matter o TSP o Respirable Fraction o Biological Availability o BOD - TOC - TOD o Volatile Organics o Flammability IV-1 CO CO LU (_> o 1 • fi. iJ « c — s < >. (J *< M > A 1 < in • • • '^-. IT +_. Z3 •— c 00 I — 1 i_ . +_i cn O M— e -1—' •0 CO c 4— ' cn -(-' i_ Q. .,— I CD E E 4—' 4-' i- CD CD ^ -1—' CJ1 CD . p: c i_ 4-J •, — 1 •4— +-> -M CD ■— ( c= 4-' =3 CD JZl > cr CD ^ -c: 4-J ■0 -1—' 4-' a ••—1 S_ 5 2 CD 4—' XT CD "O 4—' sz CD i_ i- ■i-> D-. CD =3 i- XI v+— D 4—' 4-> SZ L_ u Z3 4-' i_ M— O CD CD ■D 00 i— a I— 1 D 4—' C= D D C ■—1 D D CD CD 4-J ^ 00 X3 cr 4-' CD CD CD E XZ 00 £= +-" -M ,—1 CD i- D -I-' .— 1 +-' D ^^ > XT . — 1 c: Q +-> .nit samples n^ = total number of measurements DRYING Moisture as a "Foreign" Object Free Water Occluded Water Bound Water Options Ignore Moisture Dry Before Analysis Correct for Moisture Water by Drying Water by Analysis REPORTING RESULTS Sample Uncertainty Must be Stated CHAIN OF CUSTODY Sample Safeguards Sample Subdivision Laboratory Records Sample Custodian V-6 VI METHODOLOGY INTRODUCTION GOAL OF MEASUREMENT COST-EFFECTIVE USEFUL DATA METHODOLOGY CHOSEN TO MINIMIZE ERROR COST-EFFECTIVE APPROACH MINIMIZE MEASUREMENTS OPTIMIZE SAMPLES INFORMATION REQUIREMENTS HOW GOOD IS METHODOLOGY ADEQUACY FOR GIVEN USE MERITS OF COMPETITIVE METHODS HOW GOOD IS THE DATA HOW GOOD IS LABORATORY PERFORMANCE VI-1 NOMENCLATURE Technique - Physical or chemical principle for characterizing materials of chemical systems. Method - An assemblage of techniques; implies reduction to practice. Procedure - Detailed instructions to permit replication of a method. Protocol - Methodology specified in regulatory, authoritative, or contractual situations. Absolute Method - Method in which characterization is based entirely on physical (absolute) defined standards. Comparative Method - Method in which characerization is based on chemical standards (i.e., comparison with such standards). Reference Method - A method of known and demonstrated accuracy. Standard Method - A method of known and demonstrated precision issued by an organization generally recognized as competent to do so. Standard Reference Method - A standard method of demonstrated accuracy. Routine Method - Method used in routine measurement of a measurand. It must be qualified by other adjectives since no degree of reliability is implied. Field Method - Method applicable to non-laboratory situations. Trace Method - Method applicable to ppm range. Ultra Trace Method - Method applicable below trace levels. Macro Method - Method requiring more than milligram amounts of sample. Micro Method - Method requiring milligram or smaller amounts of sample VI-2 CLASSES OF METHODS Class Precision/Accur 355t Qualitative Note: For trace analysis, move classes down one step. For Ultra-trace analysis, move classes down two steps. The class must always be designated, since no other terminology defines or implies this characteristic. NOMENCLATURE Technique Pyhsical or chemical principle applied to analytical measurement. Method Adaptation of a technique to a specific measurement problem. Procedure Detailed steps for application of method. Precise Language Critical Steps Identified Calibration Detailed Principles Explained Detailed References Protocol Mandated Methodology Standard methods are really standard procedures VI-3 EXAMPLES OF ANALYTICAL NOMENCLATURE Technique For example, Spectrophotometry Method For example, Pararosaniline Method (West-Gaeke) Procedure For example, ASTM-D291 4 Protocol For example, EPA Method 625 HOW ACQUIRED Technique Existing Technology- Transfer of Technology New Technology Method Revision Adaptation Novel Procedure Modification Original Application Utilization Standardization VI-4 MEANINGFUL MEASUREMENT METHODOLOGY Essential Characteristics Sensitivity Specificity Precision Repeatability Reproducibility Range Desirable Characteristics Large Dynamic Range Ease of Operation Speed Low Cost Portability Ruggedness riGoi^es or oaEr »t J PERFORMANCE PARAMETERS Type of Sample Forms Determined Range of Application Limit of Detection Biases Interferences Special Requirements Calibration Limitations Time Requirement Precision Other Limitations VI-5 METHODOLOGY SELECTION 1 A on ^ 1 Jl >- a: o 1— zr LU > >- CJ o — 1 o Q O 3: 1— LU ^— LU r— ^« e > o <: Related Experience Indications of Feasibility Some R&D Required iquipment Obtainable k • k f 1 • • • 1 ill 1 i 1 J 1 ' 1 1 J 1)111 1 e OJ (U la (tl (U ^ Lu Prior Related Experience Know-How Available Equipment Available 1 1 , , ) 1 1 1 1 1 i J 1 1 ' ^ 1 1 1 1 1 1 1 1 A ' 1 C CQ OJ >>^ — -Q •r- n3 ■O OJ ro CO Previous Experience Equipment Available 1 1 1 W : Mill 1 1 1 1 1 1 1 1 1 1 i 1 1 1 1 1 1 2 o 1— O) .— CL Previous Experience Equipment on Standby Basis SOP on File . 1 1 1 1 1 1 i 1 I > I 1 c- E •-- •r- ■,- Q. s- ^ ;- -r- n3 VI-6 STANDARDS Prepared by Organizations Recognized as Capable to Do So Only as Reliable as the Organizations that Produce Them Usually Result from Consensus of Opinion Developed By Local - Special Organizations National Organizations International Organizations Kinds of Standards Standard Methods Standard Practices Product Standards All Developed as the Result of a Recognized Need CHARACTERISTICS OF A RELIABLE STANDARD Developed by a Reliable Organization Developed by Representative Group Developed by Consensus Tested before Adoption Typical Process Organization Technical Committee 4-f Subcommittee Working Group Advantages Based on Wide Experience, Pre-Tested, Weil-Defined Means for Communication, can Produce "Standard Data." Disadvantages May Hinder Innovation, Freeze Technology, May Induce False Confidence, May be Misused, May be Expensive — Too Much Time to Develop. VI-7 PREREQUISITES FOR A TEST METHOD Backed by research to define its characteristics Validity supported by prior use Ruggedness known or tested Format chosen with End use/user in mind Degree of detail Self-contained For use with other information Question of adequacy of other information REQUIREMENTS FOR COLLABORATIVE TEST MATERIAL Matrix Match Stability Homogeneity Method of Use Whole sample Sub-sample Spike Shipability Quantity For pre-testing For use in test For post-testing For post-use Question of Blindness Blind Double-blind VI-8 COLLABORATIVE TEST OPERATIONS Familiarization Establishing statistical control* Test measurements Re-evaluation of Test Sample * This is best shown by control charts based on pre-test measurements and maintained during the test. RESULT OF COLLABORATIVE TEST Test of a Method Precision Single operator/laboratory Between operator/laboratory Level dependency Proficiency Test Accuracy Evaluation Based on true value "Accuracy" Evalaution Based on consensus values Based on peer-group performance CONCLUSIONS Collaborative Test Results are Often Misunderstood Misinterpreted Poorly utilized Precision/Accuracy of Method vs. Test of Method Proficiency of Analyst/Laboratory vs. at Time of Test Role of Standard/Validated Method Importance of Statistical Control Emphasis Should be on Identification and Minimization of Interlaboratory Bias VI-9 STANDARDIZATION OF A "METHOD" TECHNIQUE ^ METHOD ^ PROCEDURE ^ PROTOCOL 1 . Individual Laboratory Ruggedeness Test Procedure Single Operator Precision 2. Several Labs (3*9) Can Use Own Materials Evaluate - Comment 3. Comments -* Revisions Devise Protocol for Testing M. Collaborative Test on Same Material Unknown Composition Within Lab Precision Between Lab Precision Known Composition Bias Within Lab Precision Between Lab Precision VI-10 Method Standardization Publication VI-11 COLLABORATIVE TEST PROCESS Stable Homogeneous Reference Sample Precision Evaluation Candidate Method Reference Method of Known Accuracy (Unknown ] f Known ^ Composition / V^ Compos itiory Precision and Bias Evaluation Vl-12 SUBCOMMITTEE RECEIVES SUMMARY ^ REPORT TESTING A STANDARD METHOD ■> TASK GROUP ACCEPTS REJECTS DEVELOPS TEST PLAN PREPARES TEST SAMPLES RECEIVES REPORTS V ANALYZES DATA PREPARES SUMMARY REPORT LABORATORIES FAMILIARIZATION TESTS RECEIVES TEST SAMPLES ANALYZES SAMPLES PREPARES REPORT -> STANDARD ADOPTED ■> STANDARD REVISED RETESTED STANDARD REJECTED > NEW METHOD SOUGHT Vl-13 COLLABORATIVE TESTING COmiTTEE COORD. LAB PART. LAB Selects Methods Sets Goals Selects Coord Lob Reviews Reports- Disseminates Info. ■^Designs Exp. Selects Port Lobs. Prepares Instruct (Prelim. Study) Prepares Test Materials ■> Receives Instructs. Receives Reports^ Analyzes Data Prepares Summary Report Conducts Familiz. Studies •Receives Materials Conducts Measurement Prepares Report VI-14 SIMPLIFYING COLLABORATIVE TESTING System Concept of Measurement Identify critical steps Test performance of critical steps Improve methodology to reduce criticality Matrix Concept of Measurement Test for matrix effects Chemical IVIeasurement Parameters TtCHHIQUE PROPERTY WTERIAL CHEMICAL MEASUREMENT PROCESS VI-15 ANALYTICAL METHOD Property X^ , Technique Z, Property X^ , MaterUl Y^ Material Y SAMPLE Step 1 Step 2 KaterUl Y, SAMPLE Step 1 Step 2 Step 3 Step 3 1 Step 4 Step 4 1 Step N Step N Technique Z^ SAMPLE i Step 1 1 Step 2 1 Step 3 1 step 4 1 Step N Technique Z^ SAMPLE 1 Step 1 1 St.() t 1 step 3 1 Step 4 step U MATRIX MANAGEMENT CONCEPT SIGNIFICANCE OF P AND A STATEMENTS OF STANDARD METHODS Summarize the Collaborative Test Only as Good as the Test Influenced By: Design of Test Plan Kind and Number of Participants Prior Experience with Method Fidelity in Following Plan Fidelity in Following Procedure Quality of Test Materials Give General Idea of Overall Performance Characteristics of Method Indicate What Improvements can be Expected Basis for Deciding Usefulness Basis for Choice Between Competitive Methods Basis for Comparing Your Performance with That of Others Useful for Initial Setting of Control Limits Useful for Initial Comparison of Isolated Results Remember Standard Method Implies a Defined Situation for Its Use P&A Statement Applies Only to That Use VI-17 YOUDEN'S RUGGEDNESS TEST Table 1. Eight combiiLations of •even factors used to test the ruggcdnc of an anulytical procedure CombinAtioo or OetenninAtioa Number FMtor Vftlutt 1 2 3 4 5 « 7 8 A or a A A A A a a a a Borb B B b b B B b b Core C e C c C c C Dord D D d d d d D D Eore E e E e e E e E Forf F f f F F f f F Gorg G e g G S G G K Observed result s t u V w X y s PROCEDURE 1. Chose 4 minus 4 combinations of £ to z_ to get M caps minus 4 i.e. of desired letter. s + t + u + V _ w + X + V + z e.g. A-a 4 4 2. Rank the seven differences to identify problems 3. Calculate s from the eight results, conventionally, or by s = / y Z d^ where d = A-a, e.g. TABLE ni.— SCHEDULE FOR TWELVE COMBINATIONS OF ANY NUMBER UP TO ELEVEN CONDITIONS. 1 2 3 4 5 6 7 8 9 10 11 12 A A A. A A a A a a B b B B a b b B b b B C C C C e e C c c C c D D D d d d D d D d D B E f f E E t E E F f / f P f f F f F F F G G G G G H h H h h H k H H H K A I I I I J J J J J } J J ] J I J K k K k K K K k k K k W. J. Youden. in NBS Special Publication 300. H, Ku> Editor VI-1^ DETECTION LIMITS IDL - Instrument Detection Limit Smallest "signal" instrument can reliably detect. MDL - Method Detection Limit Smallest concentration/amount of analyte method can reliably detect, wherever located. LOD - Limit of Detection Smallest concentration/amount of analyte that can be rel iably reported as found/detected in a material/sample. ACS GUIDELINES LIMITS OF DETECTION/QUANTITATION .00 ^OT DETECTeP>^(?EGIOKJ cP DETECTIOM ^ T lo IS" XO fV\ E ASUl^^t^ » ^VO B IN UNITS OF ^S" Measured Value in Units of o VI-20 METHOD DETECTION LIMIT o ■a: > - { ■t - - _ — "J. Q OO CONCENTRATION MDL = 3 S, USEFUL RANGE OF ."lETHODOLOGY LOD LOO Concentration USEFUL RANGE FPOfl LOQ TO LOL VI-21 MAXIMUM HOLDING-TIME ESTIMATION M*X HOLDING TIME 1. PLOT DATA AS FUNCTION OF TIME 2. DRAW BEST GRAPHICAL FIT (OR FIT BY REGRESSION) CALCULATE R •• n d = dif. of dupl, 4. CALCULATE s = R/dg '^ 0.85R 5. CALCULATE LOWER LIMIT = C - 3s -^^ C - 2.55R ^ 6. PLOT LOWER LIMIT AND DETERMINE TIME OF ITS INTERSECTION WITH DATA-FIT LINE 7. INTERSECTION TIME IS "MAXIMUM ACCEPTABLE HOLDING TIME" BY DEFINITION VI-22 VII CALIBRATION CALIBRATION Physical Standards Used Frequency Slow Drift Followed by Abrupt Changes at Each Recalibration Chemical Standards Used May Need to be Prepared by Experimenter Frequency Drift and Abrupt Changes as Above New Standards May Show Differences from Predecessors May Need Intercomparison Calibration vs. Tolerance Testing MOST IMPORTANT ASPECT OF CALIBRATION MATRIX MATCH VII-1 Y = mx + b Theoretical Data LINEAR FUNCTIONAL RELATIONSHIP, ONLY Y AFFECTED BY MEASUREMENT ERRORS Y = mx + b Experimental Data LINEAR FUNCTIONAL RELATIONSHIP. ONLY Y AFFECTED BY HEASUREMENT ERRORS Least Squares Regression Fit VII-2 y = mx + b LINEAR FUNCTIONAL RELATIONSHIP. BOTH X AND Y AFFECTED BY MEASUREMENT ERRORS PROPAGATION OF CALIBRATION UNCERTAINTY VII-3 JOINT CONFIDENCE ELIPSE FOR SLOPE AND INTERCEPT APPROACHES TO CALIBRATION I. Matrix Independence A. Calibration by spiking/standard addition B. Calibration by representative matrix II. Matrix Dependence A. Calibration by spiking/standard addition B. Calibration by matrix modification C. Calibration by matrix removal In Case II. Calibration must match methodology Vll-i^ MEASUREMENT IS A COMPARISON PROCESS Measurement Consists of Comparison of Unknown with an Unknown DIRECT COMPARISON INDIRECT COMPARISON ,g. Fixing the Value of a Scale (Calibration) for Direct Reading Instruments or Analytical Response Function for Others Direct Comparison Can Be Considered As Continuous Calibration Indirect Comparison is Usually Intermittent Calibration CALIBRATION INTERVAL IMPORTANT All Comparisons Require Analogy of Things Compared Calibrate n Standards Must be Analogous to Samples Tested in Level and Matrix VII-5 SOURCES OF ERROR IN CALIBRATION Standards Used Linear Fit Matrix Effects Can be Most Important Aspect of Calibration More Biased Measurements than Methods Biased Results from Unbiased Methods Statistical Control Must be Demonstrated VII- VIII QUALITY ASSURANCE GENERAL WHAT IS QUALITY? Knowing Client's Needs Designing to Meet Them Faultless Implementation Reliable Subcontractors Punctual Delivery Comprehensive/Understandable Reports /Interpret at ions Back-up Service from Defect Detection to Defect Prevention NEVER ENDING IMPROVEMENT OF QUALITY AND PRODUCTIVITY OUTPUTS OF INCREASING QUALITY WITH INCREASING PRODUCTIVITY Dynamic vs Static DETECTION TOLERATES AN ACCEPTABLE LEVEL OF DEFECT PREVENTION AVOIDS DEFECTS FUNDAMENTAL PHILOSOPHY Current level of performance can be improved Mental desire to do so Room for improvement Doing your best The goals of yesterday are the commonplace occurrences of today and the outmoded, practices of tomorrow VIII-1 QUALITY ASSURANCE PROGRAM WHY NEEDED To discharge management's responsibility for quality of laboratory's ouputs To satisfy analyst's concerns for quality work To provide records and documentation for present and future use, and to protect all interests QUALITY ASSURANCE BASIC INGREDIENTS QUALITY ASSESSMENT REFERENCE MATERIALS REPLICATES SPLITS SPIKES SURROGATES COLLABO- RATIVE TESTS QUALITY ASSURANCE STATISTICAL ANALYSIS QUALITY CONTROL VIII-2 THE QUALITY ASSURANCE SYSTEM VIII-3 ''If ^customer) expectations are not met — then all of the debates, the round- robin tests, the committee and task group work, and lofty statements of in- volvement with quality and commitiTient to excellence have not been productive." VIII - 4 IX QUALITY CONTROL "AN OBSTACLE THAT ENSURES DISAPPOINTMENT IS THE SUPPOSITION, ALL TOO PREVALENT, THAT QUALITY CONTROL IS SOMETHING YOU INSTALL ACTUALLY, QUALITY CONTROL TO BE SUCCESSFUL IN ANY COMPANY, MUST BE A LEARNING PROCESS WITH ACCUMULATION OF KNOWLEDGE AND EXPERIENCE UNDER COMPETENT TUTELAGE." W. Edwards Deming "On Statistical Aids Toward Economic Production" Interfaces Vol. 5, No. 4, August 1975. THE FOUNDATIONS FOR CHEMICAL ANALYSIS IX-1 A MEASUREMENT SYSTEM MUST HAVE ATTAINED A STATE OF STATISTICAL CONTROL Randomness Limiting Mean Stable Variance Fixed Distribution UNTIL A MEASUREMENT SYSTEM HAS ATTAINED A STATE OF STATISTICAL CONTROL IT CANNOT BE CONSIDERED AS MEASURING ANYTHING. QUALITY CONTROL Basic Elements Good Laboratory Practices (GLP's) Good Measurement Practices (GMP's) Standard Operations Procedures (SOP's) Protocols for Specific Purposes (PSP's) Education/Training Formal Courses Seminars Informal Discussions Readings One-on-One Hands On QUALITY CONTROL PROGRAM Basic Elements Use of Qualified Personnel/Operators Use of Reliable Equipment Use of Appropriate Methodology Use of SOP'S Strict Adherence to GLP's and GMP's Use of Control Charts Use of Appropriate Calibrations and Standards Close Supervision of All Operations by Management/Senior Personnel Protocols for All Critical Steps/Operations Protocols for Special Purposes IX-2 SOME FACTORS THAT INFLUENCE PRECISION Operational Skill Instrument Stability Environmental Fluctuations Reagent Control Failure to Identify Critical Procedural Tolerances Failure to Maintain Tolerances Variable Recoveries Variability in Control of Biases SOME FACTORS THAT INFLUENCE BIAS Interferences Calibration Inefficiencies Losses Contamination Matrix Effects Instrumental Shifts/Drifts Resolution Insensitivity Operator-Related Methodology Related Application Related GOOD LABORATORY PRACTICES Address Such General Subjects Laboratory Facilities Cleaning, Housekeeping Services-Temperature, Humidity Cleaning Glassware Chemicals Grades, Storage, Handling, Disposal, Labeling, Shelf-Life, Water Samples Custody, Documentation, Routing, Storage, Preparation, Retention General Operations Weighing, pH General Purpose Equipment Responsibilities, Use, Maintenance, Calibration Data Reporting, Format, Release, Documentation Statistical Procedures Safety IX- 3 GOOD MEASUREMENTS PRACTICES For Each Measurement Technique, Address Such Subject? as Maintenance of Equipment Records Calibration General Operations Special Requirements Not Adequately Addressed by GLP's Instruction Manuals Storage, Unkeep Special Control Charts Precautions SOP'S HOW BASIC OPERATIONS are to be DONE IN THE LABORATORY SOP'S for SAMPLING MEASUREMENT CALIBRATION DATA PROCESSING STANDARD FORMAT ENCOURAGED see XVIII - 9 EDUCATION AND TRAINING Education - General Educational Deficiencies Advances in Technology New Technology Training - Specific Initial and Continuing Job-Assignment Related Organizational Orientation Safety Quality Assurance Change Related One-on-One Serial Training Discouraged IX- 4 QUALJTY CONTROL BY IPJTERPRETATION — «► SAMPLING INSPECTION -^ DISCARD MEASUREMENT -«- INSPECTION >■ DATA INSPECTION ■*► REJECT ACCEPT INSPECTION LOOP YOU HAVE ALREADY FAILED IF YOU NEED A LC : OF INSPECTORS YOU DON'T INSPECT QUALITY IN, YOU MUST BUILD IT IN IX-5 COMPUTERIZED INFORMATION MANAGEMENT Measurement Sampling Numbering - Log-In Automatic Test Assignment/Work Orders Automatic Production of Labels Computer Filing of Methodology Easy Inspection of Previous Data Comparison of Results with Acceptable Run^ Control Charts/Graphics Calibration Data - Checks Easy Inspection of Results Automated Report Generation Reduction of Paperwork/Errors Filing/Archiving Data Management Sample/Test Status Report Sample Location/Tracking Rapid Response of Inquiries Facilitation of Audits Records Management Statistics on Outputs Cost Accounting Chain of Custody Legal Def ensibility MAXIMS The analyst should go over the data with the same care exercised in doing the analytical work. ♦ The use of a computer does not relieve the chemist of the need to examine the data for suspect items. o o o o o Just glancing at data does not give one the familiarity that comes from working with the data. o o o o o Doing one's own computations whenever feasible is a great help to successful evaluation of analytical data. IX-6 GLP NO Method for Cleaning Plastic Containers for Trace Element Samples* Use: For containment and manipulation of aqueous solutions for inor- ganic trace analysis. Materials: Containers should be constructed of conventional polyethylene (CPE), TeflonR FEP or Teflon^ PFA. TeflonR TFE is satisfactory but less desirable because of high surface porosity. Certain other materials are suitable but those cited here are the most commonly available through commercial sources. Containers should be free of visible occlusions within 1 mm of the surface. Closures should be fabricated of similar polymers (polypropylene, or Teflon^) without cardboard liners or ring gaskets. Cleaning Procedure Polyethylene: New -(polypropylene closure) 1) Using a clean room towel, wipe the outside of the container with solvent (hexane, an alcohol or ketone) and then distilled water to remove surface dirt. Use squeeze bottles to rinse the inside of the container with solvent and distilled water to remove as much surface dirt as possible. 2) Fill the container with a 1 + 1 dilution of reagent grade HCl. If facilities permit, submerging the container in a glass acid bath is preferable since it affords better cleaning of the closure threads and the outside of the bottle. 3) Allow to stand for one week at room temperature. 4) Empty container, rinse with distilled water and fill (as in (2)) with 1 + 1 reagent grade HNO3. 5) Allow to stand for one week at room temperature. 6) Empty container, rinse several times with distilled water. Fill with highest quality distilled water and store until needed. 7) To use, empty, rinse with distilled water and dry under a clean (preferably Class 100) environment. « Prepared by John R. Moody, National Bureau of Standards IX - 7 Polyethylene - Used 1) The re-use of polyethylene bottles Is not recommended unless they will be re-used for essentially identical samples. Any insoluble material in the bottle should preclude its re-use. 2) Repeat steps I-3 plus 6-7. The leaching time in step 2 may be reduced to one day or less depending upon the degree of contamination of the bottle. 3) Note that polyethylene does not have a great tolerance for strong acid solutions and repeated cleanings may cause yellowing and embrittlement. Fluoropolymers: New 1) Repeat steps I-7. 2) Change temperature in steps 3 and 5 to 80 °C or more. Fluoropolymer surfaces when new have more surface impurities and require a more vigorous cleaning. Fluoropolymers: Used 1) Unlike polyethylene and other plastics, the fluoropolymers are essentially inert and can be cleaned vigorously to remove contaminating traces of old samples. 2) Rinse or dissolve as appropriate any remaining sample material. 3) Repeat steps I-7, changing the time and temperature in steps 3 and 5 to 8 hours or more at temperatures just below the boiling point. 1. Reference - J. R. Moody and R. M. Lindstrom, Anal. Chem. _49, 226^1 (1977). IX - 8 CONTROL CHARTS CONTROL CHARTS Purpose Criterion for Confirming Statistical Control or Its Lack Method of Identifying Assignable Causes Basis for Assigning Confidence limits for Data Emphasis Order Sequence Time Grouping Place Source Test Conditions Measurement Variables Equipment Operators TYPICAL CONTROL CHART SEQUENCE/TIME X-.1 in X CONTROL CHART UCL ~ - UWL LWL LCL Advantages Useful For Non-Normal Distributions (means of non-normal distributions are essentially normally distributed) Takes Pressure Off Single Measurement Disadvantages Increased Number of Measurements Needed Chance of Computational Error CONTROL LIMITS No Given Standard Available Based Entirely on Test Data Detect Inconsistencies Changes in Precision Changes in Accuracy Trends Cycles Detect Assignable Causes Given Standard Available Based on Known x, o, R Standard Values Based on Representative Prior Data Desired Aimed-At Values Mandated Values Legal - Extrinsic Real - Intrinsic Combination X-2 CONTROL LIMITS X-CHART Central Line X WL ± 2 Sg CL ± 3 Sg X-CHART Central Line X, or known property WL ±2 Sg//n CL ±3 Sg//n R-CHART - DUPLICATES Central Line R UWL 2.512 R UCL 3.267 R LWL = LCL SETTING CONTROL LIMITS No Given Standard SEQUENCE X-3 DIAGNOSTIC CONTROL CHART Assignable Cause T 1 1 r DAY NIGHT DAY NIGHT DAY NIGHT VE ^ CORRECTI ACTION •!-. A ,. • , • • • t ■ 1^ ^ ASSIGNABLE CORRECTIVE CAUSE ACTION X-4 CONTROL CHART ^ _^ — _ _ __ ^ — — 10-90 SRM I -^ — 10.70 10.50 . 10.20 f^[v| I ! : 10.00 • • • « LU TIME X-5 RANGE/DIFFERENCE CONTROL CHART Idea R oc standard Deviation Approach _ R = (Ri + R2 + ... Rk)/K For Duplicates UCL = 3.267 R UWL = 2.512 R LCL = Population of Concern All Measurements with Similar Character Relative Range, R/X, Permits Greater Inclusions Advantages Applicable to Diverse Measurements Based on Performance of Actual Samples Disadvantages Need for Replicate Measurements Possible Inclusion of Imcompatibles DUPLICATE CONTROL CHART CONCENTRATION FOR DUPLICATES " = 0,886 R R = 1.129 " UCL = 3.257 R UWL = 2.512 R X-6 RANGE RATIO CONTROL CHART Range as a Function of Concentration R = fx R = k R = b + mx '.■'-? - = Ro = observed ran R For R Central Line = R Control Limits = D3R and Dj^R For Rp Central Line = R/k Control Limits = D^R/k and Di^R/k which leads to Central Line = R/R = 1 Control Limits = DoR/R and D^ R/R = Do and D4 Thus Central Line . 1 UWL = 2 512 UCL = 3 267 LWL = LCL = X-7 RANGE-RATIO CONTROL CHART 3.267 2.512 Time (or sequence) R = fx = 0.886 "r = 0.886 fx c.i.95 = 1.252 fx (mean of duplicates) c.i.95 = 1.772 fx (single measurement) ACCEPT RUN OUT- OF- CONTROL REJECT RUN LOGIC DIAGRAM FOR ATLYING A SERIES OF DECISION CRITERIA (CONTROL RULES) IN THE MULTI-RULE SHEWHART PROCEDURE. (Westqard) (See reference 25) X-8 STRA-^EGY OF USE Test Result Lies Outside Control Limits 3 = 3/1000 Strategy of Runs n Expected 2 Ln 4 3 1 Ln 8 i\ in 16 5 1 m 32 6 in 64 7 1 ] in 128 8 1 ] Ln 256 9 1 in 512 10 1 ] in 1024 Strategy of Seven Seven consecutive values show rinsing tendency Seven consecutive values show falling tendency Seven consecutive values lie on one side of mean Western Electric Recommends Zones Concept 2 out of 3 in zone A 4 out of 5 in Zone B 1 out of 20 in Zone C None out of Zone C 8 in a row in some pattern Ascending Descending One side of Central Line X-9 EXAMPLES OF CONTROL CHARTS Selected SRM IQA Sample Typical Test "Solution" Surrogates Spikes Duplicate Samples Operator Charts Instrument Operational Characteristics Spectrophotometer Filter Slope of Calibration Curve Selected Calibration Point(s) Extraction Recoveries "Dummy" Test Object Test Weight Blank Second Buffer k\ nvtUlCTM W4 run Control chart for a spcctropholomete r showing the variation of tran^>mtlt■'tnr o as a function of time for a neutral flas* filter at 635. nm (a) and i'/O. nm (b) and 24.0 C. I I I I I I 1 T X-10 XI QUALITY ASSESSMENT QUALITY ASSESSMENT Internal Internal Test Samples Control Charts Interchange of Operators Interchange of Equipment Repeat Measurements Independent Measurement External Collaborative Tests Exchange of Samples Reference Samples SRM INTERNAL TEST SAMPLES Internal QA Samples (IQA) Replicates Splits Spikes Surrogates Blind vs. Double Blinds ALL BEST USED IN CONTROL CHART MODE FREQUENCY OF QA SAMPLES "Length of Run" Concept aI B; ! i CI I J 1 Di ! 1 ^ ; r3: \ 1 J I • 1 Sequence of Tests Analyze Measurement Process for Variability of Sub-Steps Estimate Variance of Sub-Steps Use "Assignable Cause" Concept as Possible Include at least two QA Samples. Preferably Three, in Each Run or Possible Run XI-1 QUALITY ASSESSMENT USING IQA SAMPLES Daily/Event Schedule Calibration - Full Expected Range IQAo Test Samples - Group 1 IQAi Test Samples - Group 2 IQ/\o Test Samples - group N-1 * IQAn_i Test Samples - Group N * IQAm * Calibration - Midpoint NOTES * - Decision Point 1 . Maintain Control Charts X - Control Chart, IQA R - Control Chart, AIQA 2. System must be in Control at Decision Points 3. At Least 2 Groups: Maximum of 10 Samples in Each Group IQAj^ = occasion that a given IQA is measured. XI-2 QUALITY ASSESSMENT USING DUPLICATES/SPLITS FREQUENCE SCHEDULE Full Calibration * Calibration Check - Midpoint Sample 1 » Sample 1 D/S Sample 2-9 Sample 10 * Sample 10 D/S Sample 11-19 Sample 20 * Sample 20 D/S * Calibration Check - Midpoint * Calibration Check - Midpoint/Duplicate NOTES Decision Point Maintain R-Control Chart a. Duplicate Midrange Calibration b. Duplicate/Split Sample System Must be in Control at Decision Points If More than 20 Samples, Repeat Sequence If Less than 20 Samples, Divide into two groups and follow similar plan. SPIKES Internal Standards Carried through analysis Added before measurement Surrogates Analyte Caution Standard Addition Blank Correction See Appendix C Xl-3 SPIKES SLOPES SHOULD CHECK CALIBRATION CURVE SLOPE SPIKE/INTERNAL STANDARDS Level Recommendations Sample Concentration Expected < 10 MDL Spike at 20 x MDL increments Sample Concentration Expected > 10 MDL Routine Work Spike at 2X expected concentration High Accuracy Work Iterative approach Run sample Spike at sample level, or Two spikes .90X and 1.1 OX sample level, using blank matrix XI-4 RECOVERIES Add Known Amount and Measure Found % Recovery = x 100 Added Use Efficiency Checks Control chart use recommended Corrections Coupled with control chart Cautions Spike may not simulate sample Efficiency may be coupled to concentration level Variable recoveries indicative of trouble - lack of control Low recoveries signal losses High recoveries may indicate variable blanks, contamination Use of surrogates may not be definitive REPLICATES What is Replicated? S> ^ = I s steps Samples Entire Process S R"^ S sample "^ ^ processing "^ ^ measurement "^ Splits ■R ~ 2 processing "^ ^ measurement "•■ Aliquots ^ R ^ s measurement ■*■ Should check with calibration replication Suitable combination to isolate individual components See "blank correction" for propagation of errors XI-5 10 15 20 NUMBER OF MEASUREMENTS XI-11 QUALITATIVE IDENTIFICATION Inductive Reasoning Known Selectivity Knowledge of Absence of Possible Interferents Experimental Demonstration Confirmation by Independent Technique Procedural Variations of Methodology Combination XI-6 SYSTEM AUDITS Internal and External Spot Checks of Facilities Equipment Records Calibrations Spot Checks of Selected Tests Some in detail for critical elements Some in general for entire process Identify defects classify as critical/non critical Take corrective action immediate remedies long-term actions modification of QA program Internal Audits Minimize Surprises from External Audits QUALITY AUDIT Evaluation and Detection Standards Calibration Methodology Measurement- sampling-computations Reports Records Assessments Discrepancies Significant-affect conclusions Minor-all others Categories Equipment Measurements Reports Records Feed Back Personnel Review Supervisory Review Management Review Corrective Actions XI-7 PERFORMANCE AUDITS (Proficiency Tests) Circulation of Test Samples Monitoring Networks by Regulatory Agencies Subscription Services Information They Provide Typical Performance, or Best Performance Either only if in a state of statistical control DEMONSTRATING ACCURACY REPLICATE \ MEASUREMENTS / PRECISION CONTROL CHARTS INTERNAL STANDARDS INDEPENDENT METHODS CERTIFIED REFERENCE MATERIALS ROUND ROBINS SPIKED SAMPLES ACCURACY XI-8 r-« eo © o ooc-s oeoo- ooc o ee oeoooooo««o ooeeet- o •o o o CO eo o o o f' ^ ^^h I o eo« ••eooo eoeoo • CO ooo XI-9 ANALYTICAL RESULTS y( \( — *— >« ^ — A)( >i y ( TRUE VALUE XI-10 TABLE I — Data and Calculations on Porctnt IntolubU ItMidu* in Ctmcnt R«portcd by 29 Laboratoriii Labor- PrrrrnI Rrilduo A — B (A— B) — •.•*> atory A B 1 031 022 Ol/9 —0 005 2 00« 012 —0 04 —0 135 3 024 0.14 0.10 0005 4 014 007 0.07 —0 025 i 052 0J7 t 038 01* 019 0095 1 0.08 -0015 0.46 023 * 021 0.115 10 028 0.14 0.14 0.045 11 0,10 0.18 -0.08 -0175 12 0,20 009 0.11 0015 13 0^6 010 0.16 0065 14 028 014 014 0045 IS 025 0.13 0.12 0.025 16 025 Oil 014 0045 17 0.2S 0.17 009 —0 005 18 026 018 008 —0015 012 005 007 -0 025 20 029 0.14 0.15 0.055 21 022 0.11 0.11 0.015 22 013 0.10 0.03 -«.06J 23 0.56 0.42 24 0.30 030 0.00 -0.095 25 0.24 006 018 0.0S5 26 0.25 0.35 27 024 009 015 0055 28 028 023 005 -0 045 29 0.14 0.10 0.04 -0055 Average 0229 0134 0095 0053 0.4 _ • • 0.3 " • / • az ■( • • f?' Y 0.1 n ■•i 1 • 1 / 1 1 0.1 02 0.3 0.4 0.5 Figure 3 — Percent of Inioluble Rctidu* TABLE II— Probability Table for Circular Normal DistribuHoa Percent of the Points Within Circle Multiple b of the Standard DevUUoa 10 20 s s 70 90 SS 99 0459 0.66a B 11 is 2.146 2.448 3035 Note: Percent - 100 il — txp(—b>/2}] Directions for Calculations 1. Tabulate Data Using Format of Table I. 2. Calculate A-B and The Average, A^. 3. Calculate [(A-B) - (A^)] = R. 4. Calculate R" = Average of Absolute Values of R. 5. Estimate a, i.e., s by s = 0.886 R. 6. Calculate 95% Confidence Circle Radius = 2.448s, W. J. Youden, Ind. Qual . Control XV, No. 11 (1959) XI-11 HOW MANY TEST SAMPLES ■=3 CO PQ SAMPLE MULTISAMPLE • Reduces Burden on Sample • Identifies Range Dependencies • Identifies Useful Range • Detection Limit Information XI-12 S. E en c o c o O —I -r^ en u en 4^ o o —I O >- o xn XI^13 '"''^TIIT-^l ^^d s^JEj UT uoiarj:ius3uo3 - ,.g„ aidmcs* XI-14 XII CORRECTION OF ERRORS AND/OR IMPROVING PRECISION AND ACCURACY MUST DISTINUGISH THE TWO SOURCES OF ERROR METHODOLOGY - INHERENT and APPLICATION - RELATED TYPES OF CAUSES CHANCES CAUSES ASSIGNABLE CAUSES COMMON CAUSES SPECIAL CAUSES SYSTEM CAUSES PERFORMANCE CAUSES THE DEMING DOCTRINE Identify Defects Tally Defects Analyze Defects Trace Defects to their Source Make Corrections Keep A Record of What Happens Continue Until Defect is Eliminated WHAT TO LOOK FOR The Picket Fence Effect Using each preceding picket to measure the next, rather than a single picket as a model. Think Small Don't overlook the small errors. They happen more frequently than large ones. XII-1 PARETO DIAGRAM 100 y' / / / / / / / A C E B i n 1 Problem Problem Occurrence XOccurrance % of \ A 198 9.1 47.5 B 25 1.2 5.0 C 103 4.8 24.7 D 18 .8 4.3 c 72 3.3 17.3 Total 415 19.2 99.9 2155 objects Inspected ISHIKAWA'S CAUSE-EFFECT DIAGRAM CIDCTJCTJ CZJCOCZJ XII-2 When DEFICIENCY CORRECTION A. In control. Results Questioned or Questionabl< B. Out of Control ick List A B Changes / / Sampling / Sample Handling / Analytical Procedure Calculations / / Data / / Reagents / Equipment / Calibration / Maintenance / Methodology / Blunder / ASSIGNABLE CAUSES Based on Control Chart Nature Bias Imprecision Uncertain Occurrence Permanent Transitory Attack Imprecision First XII-3 ERROR ANALYSIS ERROR SYSTEMATIC ^ RANDOM 1 ^-.PARAMETER OPERATION^ A B N a b c n 1 2 3 4 5 i xu-n AHALYSIS OF SOURCES OF ERROR IN A TTPICAL CHEfllCAL MEASUREMENT ! TOlUf« TEWEMTVRf Tl« WkSS CHEMICAL WRJTT LEWITH K*TRIX EFFECTS rtcwacAL LOSS SMllNG JT X X inRACIlOfJ V X 0RT1N6 ¥■ X CONCENTRATION X X CHEKICU X X X K RASUREHENT X X >c X X CAL18RA11CN X X X X i( X CONFIRMATIM X X CALCUUTION KPORTINC XII-5 PRELIMINARIES Familiarization Dry Runs Use of Reference Materials Analyst/Operator Check-Out General Proficiency Special Proficiency Training No Serial Training Pilot Runs Performance Checks Availability of All Needs Stabilize Equipment Tune Equipment Optimize Verify System Performance Use of Test Sample All Systems Go! "No quality assurance program, whether it be voluntary or imposed, can correct frequent mistakes and unreliable performance introduced by insufficient training, inadequate laboratory environment, and poor administrative practices." William Horwitz Quality Assurance Practices for Health Laboratories, p. 5^7 APHA, 1978 KINDS OF PEOPLE No concern for quality. Just want to get the job done without too many complaints. Those in Between Overconcerned about quality. Care so much the job never gets done. Always trying to do better. "Just a few more measurements." Quality Assurance Practices Help All Of These XII-6 LABORATORY PERSONNEL The Most Important Aspect of Quality Control* Laboratory Staff All Those Who Can Influence the Correctness of the Informat^:on Management Supervisors Analysts Technicians Support Quality of Output Highly Dependent On Motivation Performance Number of Mistakes/Analyst/Year 34 Class A made 175^ of Total 19 Class B made 5855 of Total 3 Class C made 2555 of Total Training is Essential Technical Training to Provide Competence/Skills Supplemental Training/Introduction Personnel Must Have Appreciation of Interested Client Appreciation of Critical Aspects of the Work Appreciation of Personal Responsibilities * J. A. Lott, Med. Instrumentation 8: 22-25; 197^. XII-7 WHAT INFLUENCES THE ANALYST BOREDOM^ \ / STRESS ANALYST ^ MOTIVATION \ \ TRAINING V 1 ^ \ \ TEST REPORT ILLNESS SURROUNDINGS INSTRUMENTATION Training - To Provide Competence/Skills Motivation - Influences How Carefully One Works Boredom - Challenge Must Be Present Stress - Work-Pressure Contributions Equipment Contributions Delicate, Difficult to Adjust, Drift, Tempermental , Unsuited Illness - Physical/Mental Surroundings - Space/Clutter/Cleanliness Instrument - Feedback In Case of Malfunction GLP/SOP Should Describe Good Performance and Trouble Shooting/Corrective Actions XII-8 QUALITY CIRCLES What They Are Small Groups with Kindred Interests/Responsibilities Meet at Regular Intervals on Company Time Meet With or Without Discussion Leaders What They Do Think Quality Think About Sources of Error Identify Problems/Potential or Real Solve or Seek Help to Solve Problems Suggest Remedial Actions Preventive Maincenance Benefits Grass Roots Input Increase Morale Promote Teamwork Create Problem-Prevention Attitudes Solutions Gain Acceptance Resulting from Concensus Approach Boost Quality/Productivity XII-9 QUALITY CIRCLE DISCUSSION TOPICS Program/Project Management Problem Identification Defining Project Objectives Experimental Design Simple Models Factorial Design Project Organization Quality Assurance Program Development General Aspects Chain of Custody OA Responsibilities Whats Right/Wrong with our QA Program Quality Control Techniques Good Laboratory Measurement Practices Methods of Sample Preparation Contamination Control Analytical Factors Influencing Data Control Charts Quality Assessment Techniques Quality Assessment Samples SRM's Collaborative Tests Plotting Data Inspection for Quality Performance Audits System Audits Statistical Techniques Precision/Accuracy Concepts Regression Techniques Fitting Equations to Data Statistical Tests Analysis of Variance Statistical Reporting of Data General Nomenclature Definitions Critical Consideration of Specific Techniques Sampling Maintenance of Equipment Good Housekeeping Book Reviews Safety Basic Laboratory Precautions Chemical Hazards Physical Hazards Waste Disposal Storage of Chemicals Space Considerations NOTE: The above topics can also provide the basis for a short course to indoctrinate new employees. XII-10 MANAGEMENT RELATED PROBLEMS o Inadequate Inspection/Evaluation Measurer Supervisor o Undef ined/Illdef ined Limits o Corrective Actions Illdefined or not Followed o Multiple/Simultaneous System Changes o Inadequate Training or Trainee Evaluation o Failure to Follow Instructions or Initiation of Unauthorized Procedural Changes o Workload Pressures o Quality Assurance Treated as a Step-Child Adapted, in part, from A. Mainline, Jr., Laboratory Management , pp. 27-9 (October 197^). xii-n QUALITY CIRCLES John K. Taylor ORGANIZATION A quality circle consists of a small group (ten to twelve maximum) of employees with similar interests/involvements in an activity/process which may benefit from quality improvements. Since almost any activity/ process should be concerned with the quality of its outputs, there is virtually no situation for which a Circle is not applicable. The group should be reasonably homogenous but not identical in its interest/ involvement to provide a sound basis for attack of problems. An organization planning for several quality circles usually will appoint a facilitator who will coordinate and direct circle activities. Each circle will have a leader, usually appointed, but possibly elected by the group. The leader is responsible for the smooth operation of the Circle and must involve the participation of all members. Participation of the quiet members is encouraged by asking questions, seeking opinions, etc. Over-participation of the exuberant is discouraged by the idea-writing approach which will be discussed later. OBJECTIVES The objective of a Circle is both to prevent and solve problems related to quality of output. While their own outputs are of major concern, outputs of others related to theirs can also be considered. There is also the possibility of colaboration with others on such problems. The idea is to reduce errors and to enhance quality. Cooperation is also encouraged along with participation and motivation. Encouragement of ownership of change and grass roots inputs are additional objectives. Circles can become an organizational resource, consisting of teams experienced in trouble-shooting. But more than this, they are an excellent way to train. Circles, by virtue of an intensive look at a measurement process can provide a mechanism for education on that process (and even research in some instances). Thus ; serendipic benefit is improved understanding of the process. OPERATION The recommended mode of operation for a Quality Circle is shown diagramatically in the figure. The mode is the same whether the objective is to identify problems to solve or to solve problems already identified. In the case of both, an activity or a process is studied to decide the most beneficial course of action. This may require study and often will require critical consideration to define the problem area more clearly. XII - 12 J Ordinarily several aspects of the problem will be identified, in which case the options need to be generated. From the thinking of the Circle, a number of ideas will surface and idea generation should be encouraged. Even trivial and loosely related ideas should be accepted since these, if not directly useful, may stimulate other ideas, thus broadening the basis for selection and action. The recommended procedure is via idea writing which minimizes the direction of thinking by dominant individuals. The written ideas are collected in an idea box, for example (writer's name not included). After reading by each member, the writing process may be continued for additional cycles to the point of diminishing returns. The selection process consists of two activities: categorization ; and prioritization .' Categorization, or clustering consists in grouping by similarities i.e. to go from "local" to "global". This is by way of discussion, in which recorded ideas are classified by group concensus (but not rejected). During the discussion, the ideas may be clarified and edited, as needed. The process can be carried out by intercompar ison , i.e. to distinguish between similarities and differences, on a one-to-one basis. This process will result in several groups (clusters ) with similar characteristics, the general characteristic of which can be identified. While all clusters will have merit, some will be perceived to be more urgent (or important) than others. Prioritization can then be established by group preferential rating. The actions that should be taken will fall into three classes: Iteration - further study may be required to define a problem more clearly for specific action; the problem may need collaborative effort with other circles/groups. Implementation - the circle may have the authority to act on some of its own recommendations with little or no approval required by others. Recommendation - most problem solving will fall in this class in that the solution may need approval by management or interaction with others which may require external approval. In such cases the circle will decide on the best course of action to follow in presenting its recommendation to management. PRESENTATION Recommendations may go to management by two routes. Written reports provide a record of a circle's outputs and are always needed. In addition, oral presentations may be made, as well. The circle should be well-prepared in such a situation, calling for dry runs. If a dry-run XII - 13 does not appear to go well, it may mean that something is lacking in the subject matter. Hence, oral presentations, whether or not used as a final means of communication, are useful for deciding the merits of any recommendations made by a circle. QUALITY CIRCLE OPERATION IDENTIFICATION DEFINE GENERATION IDEA WRITING IDEA COLLECTION ;election CATEGORIZATION PRIORITIZATION ACTION ITERATION IMPLEMENTATION RECOMMENDATION FURTHER STUDY COLLATERAL ACTION GLPs GMPs SOPs TRAINING FACILITIES EQUIPMENT QA PLAN REVISIONS INTERJECTIONS XII XIII MEASUREMENT COMPATIBILITY THE NATIONAL MEASUREMENT SYSTEM TRACEABILITY THE TROUBLE WITH THE IDEA OF MEASUREMENT IS ITS SEEMING SIMPLICITY Problem o o o o o How many dots? What is the distance between them? Needed Unit of length What to measure Where to measure Measurement device What affects measurement? Conclusions Counting is exact Calculation can be as exact as one chooses Measurement is inexact MEASUREMENT PROCESS Conceptual Foundation o Phenomenon Definitions o Concepts o Quantities o Units Basic Technical Infrastructure o Knowledge o Documentation ■'0 Specifications o Reference Data o Reference Materials Realized Measurement Capabilities o Instruments o Skill o Ranges o Precision Accuracy Dissemination/Enforcement Network o Education/Professional Societies o Standards/Testing Laboratories o Regulatory Agencies o States Weights & Measures Labs o NBS End-Use Measurements o Process Control o Commerce o Evaluate Materials o Develop New Technology o Accumulate Knowledge XIII-1 TYPICAL MEASURE.nENT SITUATIONr. XIII-2 KflOWLEOGE publications preoicTTTm^thcos fUNOAHENTAL CCNST. EOUCAllOft PEOPLE PROt SOLIETIES MEASUSEMENT PHENOMENA qUAflTITIES UNITS DEFINITIONS DOCU.'^NTAR SPEClFlCAli SfSTEH 'HYSiLJi s:ns , ■ I . REALIZED iEAsij:;E;-E:ir CAPABILITIES RANGES PMECISlOri ACCURACY ENVIRONMJNI 1 STATE 4 LOCAL OFFICES OF WEIGHTS i HEASJRES S OTHEK CENTRAL SIANBARDS AUTHORITY INSTITUTIO:;^ BASIC IICIINICAL INFRASTRUCTURE REALIZED HtASUREMENT CAPABILITIES DISSEMINATION 4 ENFORCEMENT NETWORK FLOW DIAGRAM OF THE NATIONAL MEASUREMENT SYSTEM GENEALOGY OF A MEASURING SYSTEM (RELATIONSHIPS SHOWK ARE TTPICAL. MANY IMPORTANT QUANTITIES SUCH AS ANGLE. INDUCTANCE . CAPACITY, VISCOSITY, ETC., ARE OMITTED TO KEEP CHART SIMPLE.) I TEMPERATURE | rp?['ssL» I •. •'?°°°°°^ l:$^ [RE^isuiicr] 1; V I i.mui I |«tOKOOSE I I KEAI FLOi I XIII-3 10 IO-« 10-2 |02 LOAD IN KILOGRAMS 10^ 2Kg BALANCE (NBST-I) 6- lb BALANCE lOKg BALANCE (NBSB-I) [251^9 BALANCE (NBSS-1) 150- lb BALANCE QTZ -FIB ULTRA- MICROBALANCE ASSAY BALANCE CORV/IN BALANCE A 200g BALANCE (NBSA-I) KG RUEPRECHT I ^ BALANCE HR RUSSELL BALANCE, 2.5 K-lb P PLATFORM SCALE, lOK- lb MSMASTERSCALE, 150 K-lb, SUB- STITUTION WEIGHING MD MASTER SCALE, DIRECT READING XIII-4 PHYSICAL MEASUREMENTS Characteristics o Ordinarily made on well characterized defined systems o Made by relatively simple measuring systems with limited number of sources of variance o Sources of variance readily identifiable o Systems stable or vary in a predictable manner o Limited problem of interferences o Quality assurance need recognized o Measurement assurance programs established o Calibration networks in place TYPICAL PHYSICAL MEASUREMENT XIII-5 CHEMICAL MEASUREMENTS Characteristics o Often made on ill-defined materials o Measurement errors may approach or exceed "product" tolerances o Made by complex, multicomponent measurement systems with multiple sources of variance o Difficult to identify and/or isolate sources of variance o Measurements often made on fugitive or rapidly-varying populations with little or no opportunity to check results o Users (and often practioners) fail to recognize complexities of problems o Measurements may be made with insufficient regard for quality assurance procedures TYPICAL CMfHICAL MEASUnCMEHT (Measurement of i Pesticide in Water) XIII-6 n^ INCOMPATIBLE ^ / y \ X y -^X \. ^ ^ T^ ^--ir~s COMt'ATIBLE COMPATIBLE n^ ^^.^^ — ^^ --^ ^ COMPATIBLE -T^ ^ -^V \. / ^X n^ ^ ^ ^T COMPATIBILITY IS WITH THE STATION THAT MEASURED THE MEASURAND NBS NATIONAL STANDARDS NBS MEASUREMENTS ^ :: USER'S MEASUREMENTS XIII-7 NATIONAL PRIMARY PHYSICAL STANDARDS Modifi- cation Manu- factur- Construe tion Adapta- tion er Calibration NBS (Physical ^ Standards ^/*^^*v^ Sample (Chemical A^*^^ Standards^/ Measurement 1 Data End Use Preparation Purifi- cation ('Quality N Assurance^ Manufac- turer of Chemi- cals Synthesis SRM's (National Primary Chemical Standards) C Physico- Chemical Constants ) C Standard Methods Inter- national Stds. Bodies Natl./ Internatl Stds. Orgs. (Science/ "\ Technology^ NATIONAL MEASUREMENT SYSTEM FOR ANALYTICAL CHEMISTRY Scientific Technical Research XIII-8 TRACEABILITY DEFINITIONS Trace - To discover or uncover by going backwards over the evidence, step by step; to ascertain, establish, attribute as a result of such retracing or reviewing, as to trace the cause of an epidemic, one's descent from the pilgrims, etc. Traceability n - Able to be traced. Webster's New International Dictionary, Second Edition TRACEABILITY is GENEALOGY Pathways must be defined Uncertainties must be defined Credibility depends on Validity of Pathway Reliability of Measurements Quality Assurance Ancillary Phenomena Stability Chain of Custody XIII-9 TEN CARDINAL GUIDELINES FOR TRACEABILITY 1. A measurement system must be known to be in control to insure confidence in any measurement. The objective is to maximize confidence and minimize number of measurements. 2. Measurements essentially involve inter comparison of an unknown with a standard and control can be demonstrated by replicates on the standard, the unknown, or a combination of such measurements. 3. Quality control procedures may pool measurements to interrelate them on a time-sequence so that any particular measurement is supported by others made at the same or different times. k. When a measurement standard has been certified by NBS, intercomparisons with that standard constitutes direct traceability , with the uncertainty of the measurement process. 5. When a measurement is made with respect to a secondary standard certified with respect to an NBS standard, indirect traceability may exist within uncertainty of the various measurements. 6. Comparison of a measured property with that of a certified material may provide traceability by inference, depending on the validity of the inference. 7. Only the property measured is traceable. Any property inferred from such a measurement must be supported by evidence, which may not be traceable. 8. In any traceability situation, NBS responsibility is only for the standards it certifies. The Measurement Laboratory has the responsibility for its own measurements and all claims related to them. 9. Any measurement is valid only at the time of measurement. Any extension of the measur^ed value on a time basis must be supported by other evidence. 10. Any measurement is only as reliable as the measurer. XIII-10 XIV REFERENCE MATERIALS NOMENCLATURE Reference Material (RM) - Substance of which one or more properties are established for use to calibrate or verify a measurement. Internal Reference Material (IRM) - A reference material developed by a laboratory for its own internal use. External Reference Material (ERM) - A reference material provided by someone other than the end-user laboratory. Certified Reference Material (CRM) - A reference material accompanied by a certificate issued by an organization generally accepted as technically capable to do so. Standard Reference Material (SRM) - National Bureau of Standards certified reference material. CLASSES OF REFERENCE MATERIALS Grade A - Atomic weight standard Grade B - Ultimate standard - a substance which can be purified to virtually Grade A. Grade C - Primary standard - commercially purified to a purity of 100 ± 0.02?, Grade D - Working standard - commercially available, purity of 100 ± 0.05?. Grade E - Secondary standard - of lower purity, standardized against Grade C material . Reagent Water - Not defined as above. ACS and ASTM have specifications. ASTM D1193 - Reagent Water ASTM E200 - Standard Solutions, Preparation Standardization, Storage Reagent Chemicals - Sixth Edition, (1981) American Chemical Society, 1155 16th St. N.W., Washington, DC 20016 XIV-1 USE OF REFERENCE MATERIALS Used to Assess Accuracy of Systems in Statistical Control Reference Material must be Identical to or Simulate Test Material Interpretation of Reference Material Data is by Inference ADDITIONAL USES OF RM'S To Validate Test Methods for a Specific Use To Test Proficiency of Analyst To Measure Performance of Methods Under Development IRM's - Monitor Precision ; Accuracy in Special Cases SRM's - Monitor Accuracy; Precision in Special Cases XIV-2 rect Traoeability SRM U SRM Measurement "S.M=U, Case 1 Materia! Measurement U = k U SRM M XIV-3 indirect TraceabiUty Case 2 SRM Material SRM U M Measurement UcoM - k U^^ SRM Phys2ca5 TracoabiHtv Materia! XIV-i4 SRM Well-Characterized Material For: Calibration of Measurement System Production of Scientific Data Basis of Measurement Comparison Control of Production Process Material Must Have: Homogeneity- Stability Availability SRM Actual Synthetic Simulated Foreign STEPS TO DEVLEOP AN SRM Establish Need Develop Material Develop Measurement Method Study Stability of Material Obtain SRM Material Process SRM Material if Necessary Measure Homogeneity Measure for Certification Package Material Prepare Certificate Announce Its Availability Distribute Material Do Follow-up Studies to Insure Its Reliability CERTIFICATION Measure Homogeneity by at Least One Method Measure Property by Either o Well-Established Reference Method or o At Least Two Independent Techniques Both made by Pre-Established Measurement Plan Resolve all Problems Statistically Analyze Data Establish Confidence Limits Review all Data before Release Retain Records for Future Use XIV-5 PROBABLY MORE NONSENSE IS TALKED ABOUT MEASUREMENT THAN ABOUT ANY OTHER PART OF PHYSICS (OR CHEMISTRY). MEASUREMENT DOES NOT PRODUCE A VALUE, BUT A RANGE OF VALUES THE REAL UNCERTAINTY OF ANY MEASUREMENT IS NOT WHOLLY THE ERROR COMMITTED IN MAKING THE FINAL MEASUREMENT, THAT IS TO SAY THE FINAL SCALE READING. TWO ATTEMPTS AT MEASURING THE SAME THING WILL PRODUCE TWO RANGES THAT MAY NOT OVERLAP AND WE CANNOT BE SURE THAT THE TRUE VALUE LIES WITHIN EITHER OR BOTH OF THEM. OFTEN THE TRUE ERROR IS SO LARGE THAT THE ERROR OF THE FINAL READING MAY BE IGNORED. LET US KEEP SIGNIFICANT FIGURES WHERE THEY BE LONG, AS A CONVENIENT CONVENTION FOR WRITING ANSWERS IN PURE ARITHMETIC. THEY HAVE NO OTHER USE. D. P. DELURY - "COMPUTATION WITH APPROXIMATE NUMBERS", NBS SP 300, p. 392 (1969). XIV XV REPORTING DATA NO MEASUREMENT IS SCIENTIFIC UNLESS IT'S UNCERTAINTY IS KNOWN NO MEASURED VALUE IS SCIENTIFICALLY STATED UNLESS IT'S UNCERTAINTY IS EXPLICITLY STATED Error bars must be associated with every data point so that the strengths and weaknesses of every decision based thereon may be evident. This will support valid interpretation and prevent overinterpretation of data. MINIMUM REQUIREMENTS FOR REPORTING DATA Sample Documentation Methodology Number Location - Time - Space Measurement Documentation Methodology Calibration Quality Control/Quality Assessment Inter call brat ion Precision Accuracy Data Documentation Confidence Limits Measurement Sample XV- 1 DATA LIMITATIONS Precision and Accuracy Confirmation Independent Method Independent Conditions Recovery Natural vs. Spiked Samples Interpretation Limit of Detection Limit of Quantitation UNCERTAINTY LIMITS One Approach Use of Significant Figures Round off so that only last figure is uncertain, i.e., one- half unit in last figure reported or ± 5 units in next unreported place. Problem: What is Uncertain? Need Rules to Decide. Better Approach Use of Statistical Limits Measurement Precision Bounds to Systematic Error State in Sentence Format State Separately Report Above Limits to Two Figures Round Off Data to be Consistent with the Limits XV-2 STATISTICAL CONFIDENCE LIMITS Width of Confidence Interval for the Mean _ zo _ ts X ± — X ± — /n /n Statistical Tolerance Limits For Fixed Percentage of Population X - Ks to X + Ks Bounds to Systematic Error Determination of Reasonable Limits Involves an Element of Judgment Limits Cannot be Set in Exactitude BOUNDS FOR BIAS Physical Factors that Affect Measurement Factors that Affect Samples Chemical Inteferences Blanks Calibration Correction for Known Sources of Error Error analysis made; corrections evaluated and applied All corrections adequately described Documentation All of above documented by: Experimental Data Literature References Detailed Explanations ASSIGNED LIMITS OF UNCERTAINTY Based upon standard deviation or its estimate, and number of measurements of analate of concern Must distinguish between measurement and sample variance in most cases Unacceptable limits may be reduced by: Improving measurement precision Improving sample homogeneity (also compositing, etc.) Increasing number of measurements Increasing number of samples Cost/Benefit Considerations Involved in Decisions of Acceptability XV-3 CONFIDENCE/TOLERANCE INTERVALS df/n ^.025 t ^95/95 1 12.706 8.954 — 2 4.303 2.484 37.674 3 3.182 1.591 9.916 i\ 2.776 1 .241 6.370 5 2.571 1.050 5.079 6 2.447 0.925 4.414 7 2.365 0.893 4.007 8 2.306 0.769 3.732 9 2.262 0.715 3.532 10 2.228 0.672 3.379 15 2.135 0.533 2.954 20 2.086 0.455 2.752 25 2.060 0.403 2.631 30 2.042 0.367 2.549 100 1.99 0.199 2.233 w 1.960 1.960 95^ Confidence Interval of Mean t *T- • s or 1 .96o /n Statistical Tolerance Interval Ks XV-4 CONFIDENCE/TOLERANCE Intervals of a Mean (Unit Standard Deviation) 1 - Tolerance Limits 2 - Confidence Limits of Mean, Based on s 3 - Confidence Limits of Mean, Based on o XV-5 REPORTING NUMERICAL RESULTS Calculate All Means To At Least One More Significant Figure Than Is In Data Calculate Standard Deviation Estimates To At Least Two Significant Figures Calculate Confidence Interval and then Round Off to Two Significant Figures Round Off Mean Consistent with Confidence Interval Report Mean and Its Confidence Interval, Stating What It Is Don't Round-Off Too Early . Keep As Many Figures As Possible When Recording Data and In Calculations, Let the Data Decide Its Significance ANALYTICAL REPORT Title Client Problem Objectives What, Why Done Content Sample(s) Identification History Serial Numbers Client Laboratory Description of What Was Done Procedure Methodology Reference or Description Data Summary, Uncertainty Reference to Laboratory Records (May be Blind) Interpretation (s) With Respect to Problem Recommendation (s) Attestation Analyst; Supervisor; Management * (Some of this can be the reference.) XV-6 ACS GUIDELINES FOR DATA ACQUISITION AND DATA QUALITY EVALUATION IN ENVIRONMENTAL CHEMISTRY Anal. Chem. 52_; 22^12-^19 (1980) PRINCIPLES OF ENVIRONMENTAL ANALYSIS, Anal. Chem., 55_, 2210 (1983) SUMMARY o Well-designed, Carefully Executed Measurement Process o Use of Sensitive, specific, validated measurement methods o Use of Reliable Protocols for Sampling Measurement o Use of Quality Assurance Procedures Demonstration of Statistical Control via. Control Charts o Validation of Samples Measurements/ Data o Assignment of Uncertainties to Data Measurement /Sample o Specifics: LOD, LOQ, Qualitative Confirmation Recovery Verification Use of Reference Materials XV-7 TERTIARY STANDARD LOWER HIERARCHY STANDARD I <^ UBORATORY r T 'A , I WORKING ( , STANDARD ' I A. CALIBRATION CHAIN B. PROPAGATION OF UNCERTAINTY C. ALTERNATE CALIBRATION OF CHAIN UNCERTAINTY FIGURE PROPAGATION OF CALIBRATION UNCERTAINTY XVI VALIDATION VALIDATION Valid (definition) Founded on Truth or Fact Capable of Being Justified, Supported, or Defended Not Weak or Defective Well Grounded Sound Accomplishing What is Claimed or Intended Supported by Truth Having Legal Efficacy or Force Valxd^.tion (definition) The Act or Process of Validating (See Appendix D) SAMPLE VALIDATION Purpose To Accept Individual as a Member of Population Under Study To Admit Sample for Analytical Measurement To Minimize Later Questions on Sample Authenticity To Provide Opportunity for Resampling When Needed Criteria for Acceptance Positive Identification Meets Physical/Chemical Specifications Valid Chain of Custody Criteria for Rejection Sampling System Not in Control Erroneous/Conflicting Data on Identity/Character Questions on Sample that Cannot be Removed or Clarified Sample Cannot be Unequivocally Considered Member of Population A VALID MEASUREMENT PROCESS Should Produce: A Useful Measured Value An Estimate of the Uncertainty of that Value A Reported Value Should . nclude: Beet Estimate of the Value, and Its Estimated Uncertainty XVI-- WHAT IS A VALID METHOD? Performance Parameters Acceptable Definition of Acceptable Limits must Precede Selection Required Performance Parameters Detectability Sensitivity Selectivity Accuracy Adequate Precision Bias Experimental Demonstration of Above Using Evaluation Samples Equivalent to Test Samples Interefences Cost Availability of Equipment Experience Sample Load Calibration Skill Requirements Portability Time Requirements Ruggedness Down Time Other Options OTHER CONSIDERATIONS DATA REQUIREMENTS PROBLEM PERFORMANCE CHARACTERISTICS METHOD 7X" XVI-2 VALIDATION PROCESS General Validation Technique - Research of Scientific Community Method - Research of Individual Scientists; Applied Research Procedure - Users; Standardization Organizations Protocol - Fiat ised on Decision Process Result Demonstration of General Validity Definition of Areas of Applicability Delineation of Some Specific Uses METHODOLOGY MAY BE DEVELOPED: For General Purposes For a Class of Uses For a Highly Specific Use If carefully and properly done, a method which is valid for some purpose should result. UNIVERSE POPULATION SAMPLE I SAMPLING SAMPLING PROCESSING 1 PROCESSING MEASUREMENT 1 MEASUREMENT DATA XVI-3 END-USE VALIDATION User Must Demonstrate/Confirm Validity for a Specific Use by: o Use of "Identical" Reference Samples o Use of "Analogous" Reference Samples o Comparison with a Known Trusted Method o Independent Method Confirmation o Spikes/Surrogates Demonstration of Attainment of Statistical Control Mandatory in Each of Above Cases. DATA VALIDATION (The Bottom Line) Valid Method Necessary But Not Sufficient Other Require - ^ients Demonstration of Competence of Analyst Demonstration of Valid and Operational Quality Assurance Plan Attainment of State of Statistical Control Together With Validity of Model Validity of Sample DATA VALIDATION The process whereby data are filtered and accepted or rejected, based on a set of criteria. A systematic procedure of reviewing a body of data against a set of criteria to provide assurance of its validity prior to its intended use. Data Validation is After the Fact Applied to Body of Data Systematically and Uniformly Applied XVI-4 Data Validation Must be Close to its Origin Independent Objective Criteria Checks for Internal Consistency Checks for Temporal and Spatial Consistency- Checks for Proper Identification Checks for Transmittal Errors Checks for Blunders Techniques Intercomparisons Reasonableness vs. A Priori Limits vs. A Posteriori Limits Data Plots Regression Analysis Test for Outliers Data Flagged or Rejected VALID DATA Raw Data Produced By a Valid Process Sample Method Calibration Quality Assurance Finished Data Screened for Consistency Elimination of Outliers (to the Extent Possible) XVI-5 XVI-6 XVII LABORATORY CERTIFICATION/EVALUATION CRITERIA FOR EVALUATING LABORATORIES ASTM E-548 Generic Criteria for Use in the Evaluation of Testing and Inspection Agencies (6) ASTM D-3856 Evaluating Laboratories Engaged in Sampling and Analysis of Water and Waste Water (3) ASTM D-361 4 Evaluating Laboratories Engaged in Sampling and Analysis of Atmospheres and Emissions (2) ISO Guide 25 Guidelines for Assessing the Technical Competence of Testing Laboratories (23) ASTM E-7^3 Spectrochemical Laboratory Quality Assurance (7) ACIL Quality Control Systems Requirements (38) LABORATORY ACCREDITATION Evaluation of the Capability of a Testing/Inspection Laboratory in Specific Fields of Activity U.S. Accreditation Systems 44 Private 26 Governmental Some Voluntary Some Mandatory Benefits Increased Confidence/Acceptance Identification of Competent Labs Focus on Good Practices Provide Goals and Criteria Disadvantages Retards/Freezes Technology Discourages Innovation/Initiative Petty Annoyances Costs Fees Administration Questionnaires On-Site Inspections Proficiency Testing XVII-1 BASIC SYSTEMS Product Focus Ability to Test Products (Specified) Using Specified Technology (e.g., NVLAP) Discipline Focus Ability to Use Test Technology in Specified Test Areas (e.g., AALA, SCC NATP) Similarities Both Set Criteria Both Evaluate Both Accredit Differences P-F Highly Specific D-F Considerable Generality P-F Requires Multiple Accreditation D-F Requires Only Multi-Area (Discipline) Accreditation AALA - American Association for Laboratory Accreditation NATA - National Association of Testing Authorities (Australia/New Zealand) NVLAP - National Voluntary Laboratory Accreditation Program SCC - Standards Council of Canada PRODUCT FOCUS Identifies Needs Identifies Product Identifies Test Method(s) Identifies Criteria Test Specific Human Resources Space/Equipment Quality Assurance On-Site Inspection Proficiency Testing Major Factor Continuing Activity XVII-2 DISCIPLINE FOCUS Acts on Individual Requests Identifies Disciplines Identifies Criteria Discipline Oriented Organization Human Resources Space/Equipment Method Manuals (Emphasis on use of Standard Methods where possible) Sample Control Data Control Traceability Quality Assurance Documentation Report Requirements On-site Inspections Comprehensive - General Proficiency Testing Minor PRINCIPLES IN COMMON Organization Well Organized Duties/Responsibilities Defined Supervision/Inspection/Audit/Self Appraisal Staff Technical Competence Qualifications Documented Training/Maintenance of Competence Sufficient Supervision Adequate Support Equipment Adequate in Kind/Quality Maintained Calibration/Reference Standards Test Methods/Procedures Environment Space Physical/Chemical Control Housekeeping Test Items Handling Storage Chain of Custody Records Test Reports Quality Assurance Program XVII-3 SELF APPRAISAL Qualifications of Staff Education Experience - General Experience - Specific Facilities and Equipment Methodology Performance A Laboratory Offering Services Should Have: Experience Methodology Facilities Equipment - Standards Knowledgable Personnel ALL SHOULD BE A MATTER OF RECORD AND DEMONSTRATED RECOMMENDED MINIMUM REQUIREMENTS FOR PERSONNEL General Management General Knowledge Consitient with Policy Decisions Technical Director BS Degree License or Certificate as Required Five Years Experience in One or More Fields He Directs Affiliations with Technical/Professional Societies Technical Supervisor BS Degree Five Years General, Two Years Special Experience Affiliations with Technical/Professional Societies Pertinent to Field Up-To-Date Knowledge of Field Scientific Staff BS Degree On Job Training Demonstrated Competence Technical Staff H.S./Some Technical Training On Job Training Familiarity with Tes'.: Methods Support Staff General Competence Consultant (s ) XVII-i^ XVIII PLANNING QUALITY ASSURANCE PROGRAMS It is axiomatic that if you need an outsider to tell you that you have a quality problem, you have a problem. QUALITY ASSURANCE FACT OR FICTION SENSE OR NONSENSE Common Comments "Its only common sense." "If you don't care, nothing will happen." "If you do care, you don't need it." "It only works for routine situations" "We always do special work." "We do research." "Unnecessary" "Its only needed when you have lots of technicians." "Just hire good people, don't bug them, and they'll turn out good work." "We've been in this business for years without any quality problems." "We subscribe to, and informally practice Q.A. There's no need to formalize." PRODUCTION OF QUALITY DATA is a COMMON GOAL BUT IT IS PURSUED DIFFERENTLY o In various organizations o Within laboratories of the sa:me organization o Within groups in the same laboratory o From time-to-time by a given individual PREMISE A commonly accepted set of guidelines will promote greater uniformity and Harmonize on-going practices Enhance compatibility of data XVIII-1 COST/BENEFITS OF QUALITY ASSURANCE Costs Direct Test Materials Standards Quality Assurance Equipment - Test Instruments Analysis of Quality Assurance Samples Time of Personnel Time of Supervision Quality Assurance Official Committee Work Round Robin Costs Travel/Attendance at Meetings Indirect Training Extra Cost for Quality People Extra Quality Equipment Extra Quality Supplies Relaxed Work Schedules Benefits More Efficient Outputs Less Replicates for Same Reliability Less Do-Overs Greater Confidence Of: Staff Laboratory Customers INTANGIBLE Benefits of a QA Program o Promote External Image o Improve Internal Image o Promote Client Confidence o Add Credence to Results o Prevent Hasty Disclosures o Minimize Indecision o Eliminate Unnecessary Redundancies o Promote Continuity of Effort o Provide for Retention of Vital Records o Set Forth Goals and Objectives o Provide Guidance to Staff o Provide Basis for Training The nature of analytical work — production of numerical data — engenders expectation of objective evidence to demonstrate the degree of confidence in the results. XVIII-2 LEVELS OF QUALITY ASSURANCE Individual Laboratory Consumer National International QUALITY ASSURANCE Responsiblities Top Management Policy Climate Resources Supervision Guidance Direction Surveilance Individuals Quality of Outputs-Technical Competence Quality Assurance Officer Consultation Advice Oversight QUALITY ASSURANCE Implementation Policy Established by Top Management Procedures Developed by Concensus Action of All Participants Supervision Exercised by Management Chain Oversight By Quality Assurance Officer, Reporting to Top Management Independent of Normal Management Chain XVIII-3 QUALITY ASSURANCE HEIRARCHY PROGRAM GLP'S IMPLEMENTATION XVIII-4 WHO DOES WHAT Management Decision to go QA Commits Resources Designates Leader Approves Appropriate Stages Leadership Develops Plans Gets Cooperation/Involvement Gets Consensus Approval Staff Provides Expertise Technical Advice/Guidance Writes or Reviews Appropriate Plans QA PROGRAM/PLAN DEVELOPMENT 1 . Identify Goals Motivation Internal External 2. Identify On-Going QA Describe in Writing 3. Review for Adequacy Compare with any Requirements Compare with Experience of Others 4. Revise as Needed 5. Get Approvals Consensus of Users Concurrence of Management Approval of Client/Agency 6. Implementation 7. On-going Review XVIII-5 LABORATORY QA PROGRAM DOCUMENT OUTLINE 1 . Policy o Dedication to quality outputs o Commitment of adequate resources o Requirement for QA protocols for special purposes 2. Purpose o To satisfy concerns of analyst/management/clients o To inform on QA aspects 3. General Aspects o Requires planned work/experimentation o Requires GLP's/GMP's, o Requires Use of SOP's o Requires safe practices, o Requires safe disposal 4. Sampling o Stresses careful attention to samples and sampling o Stresses close identification of analytical data with sample considerations 5. Analytical Methodology o Stresses use of care in selection of methodology o Requires written procedures prior to their use in generating analytical data o Recommends written SOP's for recurring steps and methods o Stresses careful attention to calibration o Requires careful treatment of data 6. Laboratory Records o Emphasizes documentation of measurement details/data, maintenance records of equipment, filing of instruction manuals. 7. Control Charts o Affirms the importance of control charts for laboratory data operations 8. Quality Assessment o Affirms policy of validation by: multiple techniques, multiple operators, replicate measurements, use of SRM or other QC samples 9. Data Review/Reporting o Describes policy of data review and release o Describes general minimum content of all reports 0. Research Investigations o Recognizes the relation of quality data to quality research outputs o Affirms that research outpus require QA practices similar to analytical data outputs o Defines responsibilities 1 . Implementation o Defines the respective responsibilities of individuals, management, and quality assurance coordinator for effective implementation of the QA program. XVIII-6 QUALITY ASSURANCE MANUAL The Quality Assurance Manual is an Instruction kit that you have written for yourself to guide your laboratory operations in the production of QUALITY WORK QUALITY ASSURANCE MANUAL Content QA Program Document GLP's/GMP's SOP'S Implementation Directives Two Purposes Internal Guidance External Requirement Should Be Brief Simple Plusses Show Coordination or Need for Such Show Parallelisms or Lack Thereof RELATING PERSONNEL TO QUALITY ASSURANCE Quality Assurance Must Be a Way of Life Quality Assurance Must Be Personal Objective Quality Assurance Must Be Desired, not Accepted Quality Assurance Must Be Looked Upon As a Necessary Part of Investigation Quality Assurance Must Be Rewarding Not Punitive But, Recognition of Good Work Praise for Identifying Problems Personnel Must Be Involved Consensus Development of GLP's/GMP's Concensus Development of SOP's Educate - Skepticism Encouraged XVIII-7 PSYCHOLOGY OF QUALITY ASSURANCE Analysts Should Develop Own Quality Assurance as far as Possible Allotment of Time to Learn and Demonstrate Mastery of New Methods and on New Types of Samples Cultivate Awareness and Appreciation of Statistical Concepts Reward on the Basis of Quality Role of Quality Circle Keywords: o Knowledge o Ownership o Pride o Recognition XVIII- MODEL Standard Operating Procedure For 1 . Introduction 1.1 Purpose of Measurement/Test A brief description of why the measurement/test is needed and typical end-use(s) of the results/test report. 1.2 Pre-Requisities 1.2.1 Verification of Calibrations A brief statement of what calibrations must have been made previous to, or at the time of the measurement, and the checks required to verify that they have, indeed, been made and are valid at the time of use. This could include a check-list on the data sheet. 1.2.2 Vertif ication of Equipment A statement of the checks, including trial measurements, that must be made, prior to a measurement sequence, to verify that the equipment is operating properly. 1.2.3 Verification of Ability to Test A statement of the qualifying experience required before technical staff is permitted to make definitive measurement with, or apply the procedure to a specific test. The supervision that is required should be specified. Note: The data sheet may include a check-list to indicate that the various verifications have been made. 2. Methodology 2.1 Standard Method Reference to a Standard Method as available. A copy should be appended. 2.2 Laboratory Method When a standard method is not available, the method developed in the laboratory and previously tested for reliability should be included. This should follow the format of a standard method (such as ASTM format, for example). The following layout is considered to be minimal . Title Scope, Precision, Accuracy Summary Special Training Calibration/Standardization Procedure (with explanatory notes as needed) Critical Tolerances Calculations (include sample) Report Reference XVIII-9 MODEL SOP OUTLINE Calibration Purpose Summary Description of Item Calibrated Measurement Principle References to Manual Calibration Interval Equipment Needed Standards Needed Source Preparation Preliminary Operations Procedure Reference to Standard Method When Applicable Calculations Reduction of Data Uncertainty Limits Report Format Labeling/ Approval Appendices Sample Report, References, etc. MODEL SOP OUTLINE Sampling Procedures Introduction Critical Importance of the Sample Sampling Policy General Instructions Sample Requirements Sampling Responsibilities Sample Preservation/Storage Sample Validation Sample Retention Procedure Records Labeling Logging Scheduling Chain of Custody Appendix References to Standard Practices Specific SOP'S XVIII-10 PROTOCOLS FOR SPECIAL PURPOSES* What They Are Protocols to Define What is to be Done in a Specific Case How Prepared By Responsible Authority/Analyst May Concern Recurring Problems May be Tailored to a Specific Problem Approval by Management/Client Desirable in this Case Content Specification of Principle Investigator (Study Director) Specification of Problem Specification of Model Specification of Sample Specification of Data Base Specification of Methodology Specification of any Deviations or Exceptions from GLP's/GMP's References to GLP's, GMP's, SOP's,. etc., as Appropriate Specification of Controls Control Charts to be Maintained Quality Assessment Procedures Release of Data * Sometimes called Project QA Plans XVIII-11 QUALITY ASSURANCE COORDINATOR Basic Function The Quality Assurance Coordinator is responsible for the conduct of the quality assurance program and for taking or recommending corrective measures. Responsibilities and Authority 1. Develops and carries out quality control programs, including statistical procedures and techniques, which will help meet desired quality standards at minimum cost. 2. Monitors quality assurance activities to determine conformance with policy and procedures and with sound practice; and makes appropriate recommendations for correction and improvement as may be necessary. 3. Seeks out and evaluates new ideas and current developments in the field of quality assurance and recommends means for their application wherever advisable. U. Advises management in reviewing technology, methods, and equipment, with respect to quality assurance aspects. 5. Coordinates schedules for measurement system functional check calibrations, and other checking procedures. 6. Evaluates data quality and maintains records on related quality control charts, calibration records, and other pertinent information. 7. Coordinates and/or conducts quality- problem investigations. XVIII-12 Dr. John K. Taylor September 18, 1984 PERSONAL QA PROFILE Self Appraisal This check list la designed for use of individuals to check their own QA profile by the self appraisal process. Each item should be considered and the score for the statement best describing the expertise/knowledge/performance should be entered in the box. Intermediate values may be chosen as appropriate. The level determinants are meant to be suggestive and are open to interpretation. Tabulate the average score as indicated. An average score of 3.8 is acceptable but not laudatory. A score of 2.5 or lower is considered to be unacceptable and indi- cates major QA deficiencies. Intermediate scores indicate the need for immediate remedial actions. No matter what average score Is obtained, individuals should examine the scores for individual Items to identify QA areas that need improvement. Any item rated at 3 or below should be so considered. PERSONAL QA PROFILE Self Appraisal 1. Knowledge of Field / 7 State-of-the-art knowledge of ray field 5 Good practical knowledge of field 3 Some gaps/limited understanding 1 2. General Understanding of Methodology Used / / Excellent comprehensive knowledge of methodology, including basic theory 5 Make point to understand methodology, before use 3 Practical understanding but some gaps in basic comprehension 1 3. Mastery of Specific Technology / / State-of-the-art accuracy and precision always attained 5 Average accuracy and precision attained 3 Accuracy and precision needs improvement 1 i\. Use of Written SOP's / / Use SOP'S and/or develop written procedures for all methods used 5 Use SOP's/written methods for all critical analyses 3 Seldom or little use of written methods 1 5. Pre-check of Methodology Prior to Use / / Extensive checks of new methods/pre-analysis check of all others, before use 5 Pre-checks confined to new methodology 3 Little or no pre-checking 1 6. Adherence to GLP's/GMP's / / GLP's/GMP's developed and used regularly 5 GLP's/GMP's for most critical operations 3 GLP's/GMP's little/not used 1 XVIII - 13 7. Laboratory Notebooks / / Neat, indexed lab notebooks maintained, readily understandable to others 5 Some deficiencies in notebooks but generally acceptable 3 Notebooks need considerable improvement 1 8. Knowledge of Statistics / / Full working knowledge and extensive use of statistics in all decision processes 5 Good understanding and occasional use of statistics 3 Limited knowledge of statistics 1 9. Control Chart Usage / / Good understanding, extensive use of control charts 5 Occasional use of control charts 3 Limited/little use of control charts 1 10. Participation in Technical Activities / / Active/leadership role in technical organizations 5 Passive Role 3 Little participation 1 1 1 . Training Courses / / Training course(s) taken during past 18 months 5 Training course(s) taken during past 3 years 3 No recent training taken 1 12. Technical Books/Informal Training / / Informal training/tech books read during past year 5 Some informal training during past 2 years 3 No recent informal training 1 13- Experimental Planning / / Experimental planning understood/used extensively in all major activities 5 Work generally well planned before starting 3 Planning needs considerable improvement 1 14. Use of Randomization / / Understand/always use randomization in work plans/execution of work I Some use of randomization in work plans 3 Limited/little use of randomization concepts 1 15. Housekeeping Practices / / Work space always tidy consistent with activities in progress 5 No major problems in housekeeping 3 Housekeeping not a strong point 1 Average Score / / XVIII - 14 Dr. John K. Taylor September 17, 1984 l.aboratory QA Profile Self Appraisal This check sheet is designed for use by laboratory management to appraise the QA program of the laboratory. Each item should be considered individually and the appropriate score entered in the box. Intermediate values may be chosen. The level determinants are meant to be suggestive and are open to interpretation. An average score of 3.8 is acceptable but not laudatory. A score of 2.5 or lower is considered to be unacceptable and indicates that serious risk exists in laboratory operations. Intermediate scores will require a review of the QA program to identify and rectify major deficiencies. Even in the case of an acceptable average score, a low score for any item ($3) should be considered for possible corrective actions. LABORATORY QA PROFILE Self Appraisal 1 . Laboratory QA Program / / Written plan adopted/implemented/in use 5 Definite but informal program 3 Informal/variable program 1 2. Use of Written (Before Use) Methodology / / Exclusively 5 Majority of time/for all critical data 3 Few or none used 1 3. Control Chart Use / / Maintained for all critical operations 5 Variable but significant use in organization 3 Little or no use 1 4. Uncertainty Limits for Data / / Limits for all data outputs/policy/enforced 5 Most of the time/at least where critical 3 Minority of cases 1 5. Reports/Proposals / / Pre- and post-screened for QA aspects 5 Those deemed critical are screened 3 Variable/seldom done 1 6. Facilities Maintenance / / Excellent (showplace condition) 5 Good (passes muster) 3 Poor (reservations, no-no areas) 1 7. Equipment Maintenance / / Regular maintenance with records kept, control charts as appropriate 5 Good maintenance, documentation of such has some deficiencies 3 Irregular maintenance practices 1 8. Records / / Laboratory records judged to be excellent by any standards 5 Some reservations; could be difficulties in spots 3 Variable, need considerable improvement 1 XVIII - 15 Training New Employees Formal QA indoctrination Informal QA indoctrination Assumed not needed/not done 10. Personnel/Staff QA Consciousness* / Staff average for personnel QA audit ^ ^ Staff average for personnel QA audit 3 to Staff average for personnel QA audit ^2.5 ^Alternatively, average sta ff QA audit values may be inserted in / / 11. Professional Interations / / Majority and all key staff active in some professional organization 5 Reasonable level of activity 3 Little or don't know 1 12. Management and Statistics / / High level of knowledge and ability to use at supervisory level and above 5 General awareness and reasonable usage 3 Variable comprehension/use 1 3. Internal QA Audits / / Regular program/feedback/corrective actions Occasional audits Few or none m. External QA Audits / / Regular external appraisal of QA policy/practices 5 QA appraisal definite part of other reviews 3 None 1 15. Overall Opinion of QA Status / / No known weaknesses that are not subject of corrective actions Known QA weaknesses but less than vigourous action to correct them No or little basis for judgement 1 Average Score / XVIII - 16 APPENDIX A ■ APPENDIX analytical ^ chemistry Quality Assurance of Chemical Measurements John K. Taylor Center for Analytical Chemistry National Bureau of Standards Washington, D.C. 20234 Volume 53, Number 14 Page 1588A-1596A Copyright 1981 by the American Chemical Society and reprinted by permission of the cop>Tight owner A-1 John K. Taylor Center for Analytical Chemistry National Bureau of Standards Washington, D.C. 20234 Quality Assurance of confidence limits for x, the mean of n replicate measurements, are: Cm=± V^J Figure 1. Measurement tolerances and errors The objective of quality assurance programs for analytical measurements is to reduce measurement errors to tolerable limits and to provide a means of ensuring that the measure- ments generated have a high probabil- ity of being of acceptable quality. Two concept^s are involved. Quality control is the mechanism established to con- trol errors, while quality assessment is the mechanism to verify that the system is operating within acceptable limits. General handbooks that dis- cuss quality assurance in more detail are given in References 1-3. Quality is a subjective term. What is high quality in one situation may be low or unacceptable quality in another case. Clearly the tolerable limits of error must be established for each. Along with this there must be a clear understanding of the measurement process and its capability to provide the results desired. The tolerance limits for the proper- ty to be measured are the first condi- tions to be determined. These are based upon the considered judgment of the end user of the data and repre- sent the best estimate of the limits within which the measured property must be known, to be useful for its in- tended purpose. The limits must be realistic and defined on the basis of cost-benefit considerations. It is bet- ter to err on the side of too-narrow limits. Yet, measurement costs nor- mally increase as tolerances are de- creased, so that the number of mea- surements possible for a fixed budget may be inadequate when coupled with material- variability considerations. Once one has determined the toler- ance limits for the measured property, the permissible tolerances in measure- ment error may be established. The basis for this is shown in Figure 1. The tolerance limits for the measured property are indicated by Lp. Uncer- tainties in the measurement, based on the experience and judgment of the analyst, are indicated by C„. These include estimates of the bounds for the biases (systematic errors), B, and the random errors as indicated by 5, the estimate of the standard devia- tion. Obviously, Cn must be less than Lp if the data are to be useful. The in which t is the so-called student fac- tor. While the effect of random error is minimized by replication of measure- ments, there are practical limitations, and any measurement process that re- quires a large number of replicates has a serious disadvantage. Well-designed and well-implement- ed quality assurance programs provide the means to operate a measurement system in a state of statistical control, thereby providing the basis for estab- lishing reliable confidence limits for the data output. Until a measurement operation . . . has attained a state of statistical con- trol, it cannot be regarded in any logi- cal sense as measuring anything at ail. C. E. Eisenhart The Analytical System Analytical measurements are made because it is believed that composi- tional information is needed for some end use in problem solving. Explicitly or imphcitly, a measurement system such as that depicted in Figure 2 is in- volved. One must have full under- standing of the measurement system for each specific situation in order to generate quality data. The conceptualization of the prob- lem, including the data requirements and their application, constitutes the model. The plan, based on the model, includes details of sampling, measure- ment, calibration, and quality assur- ance. Various constraints such as time, resources, and the availability of sam- ples may necessitate compromises in the plan. .Adequate planning will re- quire the collaboration of the analyst, the statistician, and the end user of the data in all but the most routine Published in .A.nalytical Chemistry, December 1981, pp. 1588A-1596A, by the .American Chemical Society A-2 Report Chemical Measurements cases. In complex situations, planning may be an iterative process in which the actual data output may require re- consideration of the model and revi- sion of the plan. Sampling has been discussed in a recent paper {4). Obviously, the sam- ple is one of the critical elements of the measurement process. Closely re- lated IS the measurement methodolo- gy to be used. The method used must be adequate for the intended purpose and it must be properly utilized. The necessary characteristics of a suitable method include: adequate sensitivity, selectivity, accuracy, and precision. It is desirable that it also have the fol- lowing characteristics: large dynamic measurement range; ease of operation: multiconstituent applicability: low cost: ruggedness; portability. To judge its suitability, the following informa- tion must be known about it: type of sample; forms determined: range of applicability: limit of detection; bias- es: interferences: calibration require- ments; operational skills required: precision: and accuracy. Obviously all of the above characteristics must match the measurement require- ments. In case of doubt, trial measure- ments must be made to demonstrate appiicaoility to a given problem. .-X ■jost-benefit analysis may be needed to determine which of several candi- date methods is to be selected. A method, once adopted, must be used in a reliable and consistent manner, in order to provide reproducible data. This is best accomplished by following detailed written procedures called Standard Operating Procedures (SOPs) in quality assurance terminol- ogy. Standard methods developed by voluntary standardization organiza- tions are often good candidates for SOPs, when they are available. Two kinds of calibrations are re- quired in most cases. Physical calibra- tions may be needed for the measure- ment equipment itself and for ancil- lary measurements such as time, tem- perature, volume, and mass. The mea- surement apparatus may include built-in or auxiliary tests such as volt- age checks, which may need periodic verification of their stability if not of their absolute values. But especially, most analytical equipment requires some kind of chemical calibration, often called standardization, to estab- lish the analytical function (i.e , the relation of instrument response to chemical quantification). Obviously, the analyst must thoroughly under- stand each of the calibrations required for a particular measurement. This in- cludes a knowledge of the standards needed and their relation to the mea- surement process, the frequency of calibration, the effect on a measure- ment system due to lack of calibra- tion, and even the shock to the system resulting from recalibration. Quality Control Quality control encompasses all of the techniques used to encourage re- producibility of the output of the mea- surement system. It consists of the use of a series of protocols developed in advance and based on an intimate un- derstanding of the measurement pro- cess and the definite requirements of the specific measurement situation. Protocols, i.e.. procedures that must be rigorously followed, should be es- tablished for sampling, measurement, calibration, and data handling. Some of these, or at least selected portions. may be applicable to most or all of the measurements of a particular labora- tory and become the basis of a good laboratory practices manual (GLPM). Planning Primary — — — — Secondary Data Flow ~~~^~" Figure 2. Analytical measurement system A-3 Figure 3. Quality control by inspection In fact, the GLPM should cover the generalities, if not the specifics, of all measurement practices of the labora- tory. The protocols for a specific mea- surement process include the GLPs together with any requirements of the specific situation. The GLPM and protocols should be developed collaboratively by all of those involved in the measurements, and this development process may be the most important aspect of their function. It encourages a keen consid- eration of the measurement process and creates an awareness of potential problems that GLPs attempt to avert. Protocols are of little use unless they are followed rigorously, and the attitudes of laboratory personnel are certainly key factors in this regard. Analysts must aspire to produce high quality data and must be their own most severe critics. Notwithstanding, good quality control systems should include provisions for inspection, both periodically and aperiodically (unan- nounced) to ascertain how well they are functioning. Large laboratories may have a quality control officer or group, independent of the laboratory management, that oversees the opera- tion of the quality control system. Quality Control by Inspection An informal kind of quality control involves the frequent if not constant inspection of certain aspects of the measurement system for real or ap- parent problems (5). The essential features of such a system are depicted in Figure 3. Based on an intimate knowledge of the measurement pro- cess, samples may be casually inspect- ed for their adequacy. The rejection and possible replacement of obviously unsuitable ones can eliminate not only extra work but also erroneous data that might be difficult to identify later. Difficulties in the actual mea- surement may often be identified as they occur and remedial measures, in- cluding remeasuiement, may be taken either to save data that might other- wise be lost or at least to provide valid reasons for any rejections. Likewise, data inspection can identify problems and initiate remedial actions, includ- ing new measurements, while it is still possible to do so. Control Charts The performance of a measurement system can be demonstrated by the measurement of homogeneous and stable control samples in a planned re- petitive process. The data so generat- ed may be plotted as a control chart in a manner to indicate whether the measurement system is in a state of statistical control. Either the result of a single measurement on the control sample, the difference between dupli- cate measurements, or both may be plotted sequentially. The first mode may be an indicator of both precision and bias, while the second monitors precision only. To effectively use such a chart, the standard deviation of a single mea- surement of the control sample must be known. This may be obtained by a series of measurements of the control sample, or it may be obtained from the experience of the laboratory on measuring similar samples. Control limits, i.e., the extreme values believed to be credible, are computed from the standard deviation. For example, the 2a limit represents those within which the values are expected to lie 95% of the time. The 2>a limit represents the 99.7% confidence level. Departures from the former are warnings of possi- ble trouble, while exceeding the Latter usually means corrective action is needed. In the event that the standard deviation cannot be estimated with sufficient confidence initially, the con- trol chart may be drawn using the best estimate, and the limits may be modi- fied on the basis of increasing mea- surement experience. The development of a control chart must include the rationale for its use. There must be a definite relation be- tween the control measurements and the process they are designed to con- trol. While the control chart only sig- nifies the degree of replication of mea- surements of the control sample, its purpose is to provide confidence in the measurement process. To do this, the control measurements must simulate the measurements normally made. In chemical measurements, this means simulation of matrix, simulation of concentration levels, and simulation of sampling. The latter objective may be difficult if not impossible to achieve. It must be further emphasized that the control measurements should be random members of the measurement routine, or at least they should not oc- cupy biased positions in any measure- ment sequence. To the extent that control samples are representative of the test samples, and to the extent that measurements of them are representative of the mea- surement process, the existence of sta- tistical control for these samples can imply such control of the measure- ment process and likewise of the re- sults obtained for the test samples. No specific statements can be made about the frequency of use of control samples. Until a measurement pro- cess is well understood, control sam- ples may need to be measured fre- quently. As it is demonstrated to be in control, the need may become less and the incentive to do "extra" work may diminish. Along with the decision on how much effort should be devoted to quality control the risks and conse- quences of undetected loss of control must be weighed. Many laboratories consider that the 5-15% extra effort ordinarily required for all aspects of quality control is a small price to pay for the quality assurance it provides. When measurements are made on a frequently recurring schedule, internal controls, such as duplicate measure- ments of test samples, can provide evi- dence of reproducibility so that con- trol samples may be used largely to identify systematic errors, drifts, or other types of problems. When laboratories are engaged in a variety of measurements, the use of representative control samples may be difficult if not impossible. In such cases, often only the measurement methodology can be tested, and evalu- ation of the quality of the measure- ment output requires considerable judgment. In such cases, the experi- ence of the lab becomes a key factor. In some complex measurement sys- tems, certain steps or subsystems are more critical than others, and hence it may be more important to develop control charts for them than for the entire system. The control of such steps may indeed prevent propagation of error into the end result. An e.icam- ple is the sampling step, which may be very critical with respect to the end result. In such a case, the records of periodic inspections may be adaptable to the control chart technique of qual- ity control. Quality Assessment Procedures used to evaluate the ef- fectiveness of the quality control sys- tem may be classified according to whether the evidence arises from m- ternal or external sources. Internal procedures, useful largely for estimat- mg precision, include the use of inter- nal reference samples and control charts to monitor the overall perfor- mance of the measurement system as described in an earlier section. Repli- cate measurements on replicate or split samples can provide valuable in- sight into the reproducibility of both the measurement and sampling pro- cesses. Comparison of the results ob- tained as a consequence of inter- change of analysts, equipment, or combinations of these can attest to op- erational stability as well as identify malfunctions. Measurements made on split samples using a completely inde- pendent method can lend confidence to the method normally in use or indi- cate the presence of measurement bias. External quality assessment is al- ways needed since it can detect prob- lems of bias that are difficult to iden- tify by internal procedures. Participa- tion in collaborative tests, exchange of samples with other laboratories, and the use of certified reference materials are time-honored assessment devices. N'BS Standard Reference Materials (SRMsi >6) are especially useful for quality assessment in cases where they are available and applicable. The in- formation that can be obtained or in- ferred by their use is described in a later section. Operators of monitoring networks may provide proficiency testing or audit samples to assess labo- ratory performance. Ordinary prac- tices should be used here, so that nor- mal rather than optimum perfor- mance is measured. .\ laboratory should diligently use the information obtained in the quali- ty asses.sment process. .Adverse data should not be treated in a defensive manner but the reason I'or it should be investigated objectively and thorough- ly. When laboratory records are reli- ably and faithfully kept, the task of identifying causes of problem.s is made easier. This is an important reason for developing data handling protocols and ensuring that ill protocols are strictly followed. Systematic Errors Systematic errors or biases are of two kinds— concentration-level inde- pendent iconstani). and concentation- level related. The former are some- times called additive while the latter are called multiplicative. Both kinds may 'oe present simultaneously in a given measurement system. .An exam- ple of the first kind is the reagent blank often present in measurements involving chemical processing steps. The second kind can result from, for example, use of an inaccurately certi- fied calibrant. Systematic errors may arise from such sources as faulty calibrations, the use of erroneous physical constants, incorrect computational procedures, improper units for measurement or re- porting data, and matrix effects on'the measurement. Some of these can be eliminated or minimized by applying corrections or by modification of the measurement technique. Others may be related to fundamental aspects of the measurement process. The most insidious sources of error are those un- known or unsuspected of being present. One of the most important sources of error in modern instrumental mea- surements concerns uncertainties in the calibrants used to define the ana- lytical function of the instrument. The measurement step essentially consists of the comparison of an unknown with a known (calibrant) so that any error in the latter results in a proportional error in the former. The need to use calibrants of the highest reliability is obvious. The measurement protocol should include a detailed analysis of the sources of error and correction for them to the extent possible. The uncertainties, B. referred to earlier, represent the uncertainties in the cor- rections for the systematic errors. In making such an estimate, the 9n% con- fidence limits should be assigned to the extent possible. The magnitudes of these uncertainties can be estimat- ed from those assigned by others in the case of such factors as calibration standards and physical constants. Other constant sources of error may be more subtle both to identify and to evaluate, and the judgment and even intuition of the experimenter may be the only sources o( information. The effectiveness of elimination ot. or correction for, systematic errors is best evaluated from external quality assessment procedures. Differences found between known and measured values of test samples, such as SRMs, need to be reconciled with the labora- tory's own estimates of bounds for its random and systematic errors. When the random error is well established, as bv the quality control process, sig- nificant discrepancies can be attrib- uted to unsuspected or incorrectly es- timated systematic errors. The Use of SRMs for Quality Assessment .An SRM is a material for which the properties and composition are certi- fied by the National Bureau of Stan- dards \6. 7'. To the extent that its compositional properties simulate A-5 Figure 4. Typical analytical systematic errors (bias), (a) = unbiased: (b) = measurement-level related: (c) = con- stant error: and (d) = comoinatlon of b and c those of the sample ordinarily mea- sured, its "correct" measurement can imply "correct" measurement o( the usual samples. Such a conclusion re- quires that the protocol of measure- ment was the same in each case. Hence it is necessary that no special care be exercised in measuring the SRM, other than that ordinarily used. Analysis of SRMs has been recom- mended as a means of providing "trace- ability" to national measurement standards. However, a word of caution is appropriate on this point. Measure- ment processes are seldom identical, so that traceability is most often based on inference. .\lso, the fact that an ac- ceptable result is or is not obtained for an SRM provides no unique explana- tion for such a result. The use of an SRM should never be attempted until the analvtical system has been demonstrated to be in a state of statistical control. .An SRM is not needed for such a purpose and such use is discouraged. Ordinarily, the SRM will be available in limited amount so that the statistics of the measurement process should be dem- onstrated by measurements on other materials. Only under such a situation can the results of an SRM measure- ment be considered as representative o{ the measurement system. .A consideration of the nature of an- alytical errors, shown in Figure 4. will clarify why the measurement of a sin- gle SRM may not be fully informative. It will be noted that errors may be constant, measurement-level related, or a combination of these, and a single right or wrong result will not indicate on which of several possible curves it might lie. Measurement of a series of SR.Ms may clarify the nature of the measurement process and this should be done whenever possible. .An inti- mate understanding of the operation of a particular measurement .-ystem may also make it possible to eliminate some of the possible sources of error and to better interpret the data from measurement of SRMs. Record Keeping Adequate record keeping in an easi- ly retrievable manner is an essential part of the quality assurance program. Records needed include the descrip- tion of test samples, experimental pro- cedures, and data on calibration and testing. Quality control charts should be diligently prepared and stored. A chain of custody of test materials should be operative and such materi- als should be retained and safe- guarded until there is no doubt about their future use or need. Data Control The evaluation, review, and release of analytical data is an important part of the quality assurance process. No data should be released for external use until it has been carefully evalu- ated. Guidelines for data evaluation, applicable to almost every analytical situation, have been developed by the ACS Committee on Environmental Improvement (8). A prerequisite for release of any data should be the as- signment of uncertainty limits, which requires the operation of some kind of a quality assurance program. Formal release should be made by a profes- sional analytical chemist who certifies that the work was done with reason- able care and that assigned Umits of uncertainty are applicable. Laboratory Accreditation Laboratory accreditation is one form of quality assurance for the data output of certified laboratories. Ac- creditation is based on criteria that are considered essential to generate valid data and is a formal recognition that the laboratory is competent to carry out a specific test or specific type of test (9,10). The certification is as meaningful as the care exercised in developing certification criteria and evaluating laboratory compliance. Generic criteria developed by national and international standardization or- ganizations have been influential in this respect (77). These criteria are well conceived and provide general guidance for the sound operation of analytical laboratories, whether or not certification is involved. Implementation Detailed quality assurance plans are ineffective unless there is commitment to quality by all concerned. This com- mitment must be total, from manage- ment to technical staff. The former must provide the resources, training, facilities, equipment, and encourage- ment required to do quality work. The latter must have the technical ability and motivation to produce quality data. Some may argue that if there is such commitment, there is no need for a formal quality assurance program. A-6 However, the experience of many lab- oratories has demonstrated that a for- mal quality assurance program pro- vides constant gxoidance for the attain- ment of the qujdity goals desired. References (1) "Quality Assurance Handbook for Air Pollution Measurement Systems: Princi- ples"; VoL 1, E.P.A. PubUcation No. 600/9-76-005. (2) "Handbook for Analytical Quality Control in Water and Wastewater Labo- ratories"; EPA Publication No. 600/4- 79-019. (3) Juran, J. M. "Quality Control Hand- book," 3rd ed.; McGraw-HUl: New York, 1974. (4) Kratochvil, B. G.; Taylor, J. K. Anal. Chem. 1981, 53, 924 A. (5) Brewers, J. M., et al. "Data Are for Looking At, or Quality Control by Inter- pretation." In "Water Quality Param- ters"; American Society for Testing and Materials: Philadelphia, 1975, ASTM STP573. (6) "Catalogue of NBS Standard Refer- ence Materials"; National Bureau of Standards: Washington, NBS Special Publication 260. (7) Cali, J. P., et al. "The Role of Standard Reference Materials in Measiuement Systems"; National Bureau of Stan- dards: Washington, January 1975, NBS Monograph 148. (8) ACS Subcommittee on Environmental Analytical Chemistry. "Guidelines for Data Acquisition and Data Quality Eval- uation in Analytical Chemistry"; Anal. Chem. 1980, 52, 2242. (9) "Quality Control System-Require- ments for a Testing and Inspections Lab- oratory"; American Council of Indepen- dent Laboratories: Washington. (10) "Testing Laboratory Performance: Evaluation and Accreditation"; Berman, G. A., Ed.; National Bureau of Stan- dards: Washington, NBS Special Publi- cation 591. (11) "Standard Recommended Practice for Generic Criteria for Use in Evaluation of Testing and/or Inspection Agencies"; American Society for Testing and Mate- rials: Philadelphia, ASTM Publication No. E-548. John K. Taylor, coordinator for qual- ity assurance and voluntary stan- dardization activities at the National Bureau of Standards Center for .Ana- lytical Chemistry, received his BS from George Washington University, and his MS and PhD degrees from the University of Maryland. His research interests include electrochemical analysis, refractometry, isotope sepa- rations, standard reference materials, and the application of physical meth- ods to chemical analysis. APPENDIX B APPENDIX Sampling for Chemicol Analysis j^-- .j^^ >*.^~-'«' A major consideration in the reli- ability of any analytical measurement is that of sample quality. Too little at- tention is directed to this matter. The analyst often can only report results obtained on the particular test speci- men at the moment of analysis, which may not provide the information de- sired or needed. This may be because of uncertainties in the sampling pro- cess, or in sample storage, preserva- tion, or pretreatment prior to analysis. The sampling plan itself is often so poorly corisidered as to make relation of the analytical results to the popula- tion from which the sample was drawn uncertain, or even impossible to inter- pret. All of the above aspects of sampling merit full consideration and should be addressed in every analytical determi- nation. Because the scope is so broad, we will limit the present discussion to a small segment of the total problem, that of sampling bulk materials. For such materials the major steps in sam- pling are: • identification of the population from which the sample is to be ob- tained, • selection and withdrawal of vaild gross samples of this population, and • reduction of each gross sample to a laboratory sample suitabl" for the an- alytical techniques to be used. The analysis of bulk materials is one of the major areas of analytical ac- tivity. Included are such problems as the analysis of minerals, foodstuffs, environmentally important sub- stances, and many industrial products. We shall discuss the major consider- ations in designing sampling programs for such materials. While our discus- sion is specil cally directed toward solid materials, extension to other materials will often fee obvious. A brief list of definitions commonly used in bulk sampling is provided in the glossary. Preliminary Considerations in Sampling Poor anal>1:ical results may be caused in many ways — contaminated reagents, biased methods, operator er- rors in procedure or data handling, and so on. Most of these sources of error can be controlled by proper use of blanks, sta'"d:;fds, and reference samples. The pr.blem of an invalid sample, however, is special; neither control nor blank will avail. Accord- ingly, sampling vncertainty is often treated separately from other uncer- tainties in an analysis. For random er- rors the overall standard deviation, So, is related to the standard deviation for the sampling operation, Sj, and to that for the remaining analytical opera- tions, Sa, by the expression: si = si + Sj. Whenever possible, measurements should be conducted in such a way that the components of variance aris- ing from sample variability and mea- surement variabi!i*^y can be separately evaluated. If > .le, measurement process is demonstrated to be in a state of sta- tistical control so that Sa is already known, Sj can be evaluated from So, found by analysis of the samples. Oth- erwise, an appropriate series of repli- cate measurements or replicate sam- ples can be devised to permit evalua- tion of both standard deviations. Youden has pointed out that once the analytical uncertainty is reduced to a third or less of the sampling un- certainty, furthei reduction in the an- alyticaHincertainty is of little impor- tance (7). Therefore, if the sampling uncertainty is large and cannot be re- duced, a rapid, approximate analytical method may be sufficient, and further refinements in the measurement step may be of negligible aid in improving the overall results. In fact, in such cases a rapid method of low precision that permits more samples to be ex- amined may be the best route to re- ducing the uncertainty in the average value of the bulk material under test. An excellent example of the impor- tance of sampling is given in the deter- Published in Analytical Chemistry, July L, pp. 924A-938.'\, by the American Chemical Society B-1 Report Byron Kratochvil Department of Chemistry University of Alberta Edmonton, Alberta. Canada T6G 2G2 Figure 1. Relative standard deviation associated with the sannpling and analysis operations in testing peanuts for aflatoxins (aHer T. B. Whittalt i table of random numbers is recom- mended as an aid to sample selection. The bulk material is divided into a number of real or imaginary segments. For example, a body of water can be conceptually subdivided into cells, both horizontally and vertically, and the cells to be sampled selected ran- domly. To do this each segment is as- signed a number, and selection of seg- ments from which sample increments are to be taken is made by starting in an arbitrary place in a randi^m num- ber table and choosing numbers ac- cording to a predecided pattern For example, one could choose adjacent, alternate, or nih entries and sample those segments whose numbers occur until all of the samples decided upon have been obtained. The results obtained for these and other random samples can be analyzed by some model or plan to identify whether systematic relations exist. This is important because of the possi- ble introduction of apparent correla- tions due to systematic trends or bias- es in the measurement process. .Ac- cordingly, measurement plans should always be designed to identify and minimize such problems. Despite the disadvantages, sam- pling at evenly spaced intervals over the bulk is still often used in place of random sampling owing to its simplic- B-2 4 Define goals. Select analytical procedures, number of --lalyses. and sampling sites on basis of (;oals, time and cost constraints, and personnel and apparatus available. Collect samples; reduce to suitable test o.iions. ;:arry out preliminary operations (dissolve, iQjust conditions, separate interferences); ;-quire data on test portions. select best value from data, estimate eiiability of value, assess validity of model, evise model and repeat if necessary. Figure 2. The place of sampling in the overall analytical process ity. Because this procedure is more subject to bias than random sampling, it is not recommended. If it is used, the results must be closely monitored to ensure that errors from periodicity in the material are not introduced. Systematic Samples. Frequently, samples are obtained and analyzed to reflect or test some systematic hy- pothesis, such as changes in composi- tion with time, temperature, or spatial location. Such samples, if collected in a systematic manner, may each be considered to represent a separate dis- crete population under the existing conditions. However, the results may still be statistically tested for the sig- nificance of any apparent differences. In a carefully designed sampling plan, consideration should be given to the possible concurrence of unantici- pated events or phenomena that could prejudice' the information on the sam- ple measured. Fur e.\ample. measure- ments to 1)0 taken at time intervals are somi-tinus made with a random start or otiur sui>eriniposed random time element. Needless to say, the less known about a given process, the more randomness is merited. Conversely, as a process is more fully understood, systematic approaches can provide ma.ximum efficiency of data acquisi- tion. Representative Samples. The term "representative sample" is fre- quently used in analytical discussions to connote a single sample of a uni- verse or population (e.g., waste pile, lagoon, ground water) that can be ex- pected to exhibit average properties of the population (see glossary). Ob- viously, such a sample cannot be se- lected by a random process. And even if it could, to ascertain the validity of its representativeness would require considerable effort. The concept of a truly representa- tive sample would appear to be valid in only two cases. The first case in- volves samples defined a priori as rep- resentative for a specific purpose. For example, the Hazardous Waste Man- agement .System prescribes seven pro- tocols for sarnijling wastes — ranging from viscous liquids, solids, or con- tainerized liquids to reservoirs — to provide samples that "will be consid- ered by the Agency (EI'A) to be repre- sentative of the waste" (■j). The sec- ond case involves the sampling of truly homogeneous materials. While ihe measurement of samples defined as representative may reduce analytical costs, the information so obtained ordinarily does not enjoy the status of that (obtained from valid ran- dom samples of the population. An ex- ception is when effort has been vigor- ously exerted to homogenize the poi)u- lation prior to sampling. Such pro- cesses are difficult and are ordinarily only justified when the objective is to produce a number of subsarnples of essentially similar properties. Because of the difficulties of se- lecting or producing a "representative sample" it is recommended that this concept be discouraged for genera! purposes and reserved only for cases where the effort required to prepare such a sample is justified. An appre- ciation of the compositional informa- tion that is lost as a result is a further reason to discourage the practice. With a properly designed and execut- ed random sampling plan, the valu- able characteristics of sample mean and variation between members can be ascertained, neither of which can be obtained by measurement of one "representative sample." Composite Samples. A composite sample (see glossary) may be consid- ered as a special way of attempting to produce a representative sample. Many sampling procedures are based on the assumption that average com- position is the only information de- sired. Such averages may be bulk av- erages, time-weighted averages, and flow-proportional averages, for exam- ple, and may be obtained by measure- ment of a composite, suitably pre- pared or collected. Elaborate proce- dures involving crushing, grinding, . mixing, and blending have been devel- oped and even standardized for the preparation of solid composites, while sampling systems for liquids (especial- ly water) have been developed to ob- tain various composite samples. Analysis of a number of individual samples permits determination of the average (at the expense of extra ana- lytical effort) and the distribution of samples within the population (be- tween-sample variability). In some cases, it may be of interest to isolate the within-sample variability as well. All this information is necessary for collaborative test samples and in ref- erence material usage, especially when apparent differences in analytical re- sults within and between lal)oratories need to be evaluated. Because of the limited information provided by a composite sample, full consideration should be given to the consequences before deciding between this approach and the analysis of indi- vidual samples. Subsampling. Usually, the sample received by the analytical laboratory will be larger than that required for a single measurement, so some sub- sampling (see glossary) will be re- quired. Often, test portions (see glos- sary) must be taken for replicate mea- surements or for measurement of dif- ferent constituents by several tech- niques. Obviously, such test portions must be sufficiently alike that the re- sults are compatible. Frequently it is necessary to reduce particle size, mix, or otherwise process the laboratory sample (see glossary) before with- drawing portions (subsamples) for analysis. The effort necessary at this stage depends on the degree of homo- geneity of the original sample. In gen- eral, the subsampling standard devia- tion should not exceed one-third of the sampling standard deviation. Al- though this may sound appreciable, it is wasteful of time and effort to de- crease it below this level. But this does not mean care is unnecessary in sub- sampling. If a sample is already homo- geneous, care may be needed to avoid introducing segregation during sub- sampling. Even though analysts may not be involved with sample collec- tion, they should have sufficient knowledge of sampling theory to sub- sample properly. They should also be provided with any available informa- tion on the homogeneity of the sam- ples received so that they can subsam- ple adequately and efficiently. Model of the Sampling Operation Before sampling is begun, a model of the overall operation should be es- tablished (Figure 2). The model should consider the population to be studied, the substance(s) to be mea- sured, the extent to which speciation is to be determined, the precision re- quired, and the extent to which the distribution of the substance within the population is to be obtained. The model should identify all as- sumptions made about the population under study. Once the model is com- plete, a sampling plan can be estab- lished. The Sampling Plan The plan should include the size, number, and location of the sample in- crements and, if applicable, the extent of compositing to be done. Procedures for reduction of the gross sample isee glossary) to a laboratory sample, and to the test portions, should be speci- fied. .All of this should be written as a Table i. Confidence Intervals and Statistical Tolerance Limits^ n t" -p K" Ks 2 12.70 ±18 37.67 ±75 4 3.18 ±3.2 6,37 ±12.9 8 2.36 ±1.7 3.73 ±7.4 16 2.13 ±1.1 2.90 ±5.8 32 2.04 ±0.7 2.50 ±5.0 100 1.98 ±0.4 2.23 ±4.4 a. 1.96 1.96 ±4.0 ' Calculated for s = 2, based on measurement of n samples ' 95% confidence limits for t)ie mean of n samples - Based on a 95% confidence tfiat the interval will contain 95% of the samples detailed protocol before work is begun. The protocol should include procedures for all steps, from sam- pling through sample treatment, mea- surement, and data evaluation: it should be revised as necessary during execution as new information is ob- tained. The guidelines for data acqui- sition and quality evaluation in envi- ronmental chemistry set out by the ACS Subcommittee on Environmental Analytical Chemistry are sufficiently general to be recommended reading for workers in all fields (-/). The sampling protocol should in- clude details of when, where, and how the sample increments are to be taken. On-site criteria for collection of a valid sample should be established before- hand. Frequently, decisions must be made at the time of sampling as to components likely to appear in the sample that may be considered for- eign, that is, not part of the popula- tion. For example, a portion of dredged sediment in which the mercu- ry content is to be determined might contain cans, discarded shoes, rocks or other extraneous material. For the in- formation sought these items might be considered foreign and therefore legit- imately rejected. Decisions as to rejec- tion become less clear with smaller items. Should smaller stones be reject- ed'' How small? And what about bits of metal, glass, leather, and so on'' Cri- teria for such decisions should be made logically and systematically, if possible before sampling is initiated. The type of container, cleaning pro- cedure, and protection from contami- nation before and after sampling must he specified. The question of sample preservation, including possible addi- tion of preservative.s and refrigeration, should be addressed. Some sampling plans call for field blanks and/or field- spiked samples. The critical nature of the latter and the difficulties possible under field conditions require the ut- most care in planning and execution of the sampling operation if the results are to be meaningful. Whenever possible, the analyst should perform or directly supervise the sampling operation. If this is not feasible, a written protocol should be provided and the analyst should en- sure that those collecting the samples are well-trained in the procedures and in use of the sampling equipment, so that bias and contamination are mini- mized. No less important is careful la- beling and recording oi samples. .A chain of custody should be established such that the integrity • the samples from source to measure- --nt is en- sured. Often auxiliary daia must be recorded at the time the sample is taken: temperature, position of the collecting probe in the sample stream, flow velocity of the stream, and so on. Omission or loss of such information may greatly decrease the value of a sample, or even render it worthless. Sampling Bulk .Materials. Once the substances to be determined, to- gether with the precision desired, have been specified, the sampling plan can be designed. In designing the plan, one must consider: • How many samples should be taken? • How large should each be'' • From where in the bulk material (population) should they be taken'^ • Should individual samples be ana- lyzed, or should a composite be pre- pared? These questions cannot be an- swered accurately without some knowledge of the relative homogeneity of the system. Gross samples should be unbiased with respect to the differ- ent sizes and types of particles present in the bulk material. The size of the gross sample is often a compromise based on the heterogeneity of the bulk material on the one hard, and the cost of the sampling operation on the other. When the properties of a material B-iJ Figure 3. Sampling diagram of sodium-24 in liuman liver homogenate (from Refer- ence 7) to be sampled are unknown, a good approach is to collect a small number of samples, using experience and intu- ition as a guide to making them as representative of the population as possible, and analyze for the compo- nent of interest. From these prelimi- nary analyses, the standard deviation Ss of the individual samples can be calculated, and confidence limits for the average composition can be estab- lished using the relation ^i=x ±tsJ^/n (1) where n is the true mean value of the population, x is the average of the an- alytical measurements, and t is ob- tained from statistical tables for n measurements (often given as n — 1 degrees of freedom) at the desired level of confidence, usually 95%. Table I lists some t values; more extensive tables are provided in books on quan- titative analysis and statistics (5). On the basis of this preliminary in- formation, a more refined sampling plan can be devised, as described in the following sections. After one or two cycles the parameters should be known with sufficient confidence that the optimum size and number of the samples can be estimated with a high level of confidence. The savings in sampling and analytical time and costs by optimizing the sampling pro- gram can be considerable. Minimum Size of Individual In- crements. Several methods have been developed for estimation of the amount of sample that should be taken in a given increment so as not to exceed a predetermined level of sam- pling uncertainty. One approach is through use of Ingamells's sampling constant (6). Based on the knowledge that the between-sample standard de- viation Ss (Equation 1), decreases as the sample size is increased, Ingamells has shown that the relation WR2 = x, (2) is valid in many situations. In Equa- tion 2, W represents the weight of sample analyzed, R is the relative standard deviation (in percent) of sample composition, and Kg is the sampling constant, corresponding to the weight of sample required to limit the sampling uncertainty to 1% with 68% confidence. The magnitude of K^ may be determined by estimating Sj from a series of measurements of sam- ples of weight W. Once A's is evaluated for a given sample, the minimum weight W re- quired for a maximum relative stan- dard deviation of R percent can be readily calculated. An example of an Ingamells sam- pling constant diagram is shown in Figure 3 for a human liver sample under study in the National Environ- mental Specimen Bank Pilot Program at the National Bureau of Standards (NBS) in conjunction with the Envi- ronmental Protection Agency (7). A major goal of the program is to evalu- ate specimen storage under different conditions. This requires analysis of small test portions of individual liver specimens. The material must be suf- ficiently homogeneous that variability between test portions does not mask small variations in composition owing to changes during storage. The homo- geneity of a liver sample for sodium was assessed by a radiotracer study in which a portion was irradiated, added to the remainder of the specimen, and the material homogenized. Several test portions were then taken and the activity of ^'*Na measured as an indi- cator of the distribution of sodium in the samples. From Figure 3 it can be seen that the weight of sample re- quired to yield an inhomogeneity of 1% (±2.4 counts g-'s-i) is about 35 g. For a subsample of one gram, a sam- pling uncertainty of about 5% can be expected. Minimum Number of Individual Increments. Unless the population is known to be homogeneous, or unless a representative sample is mandated by some analytical problem, sufficient replicate samples (increments) must be analyzed. To determine the mini- mum number of sample increments, a sampling variance is first obtained, ei- ther from previous information on the bulk material or from measurements made on the samples. The number of samples necessary to achieve a given level of confidence can be estimated from the relation f2c2 (3) t^s R2J2 where t is the student's f -table value for the level of confidence desired, s, and X are estimated from preliminary measurements on or from previous knowledge of the bulk material, and R is the percent relative standard devia- tion acceptable in the average. Initial- ly t can be set at 1.96 for 95% confi- dence limits and a preliminary value of n calculated. The t value for this n can then be substituted and the sys- tem iterated to constant n. This ex- pression is applicable if the sought-for component is distributed in a positive binomial, or a Gaussian, distribution. Such distributions are characterized by having an average, |i, that is larger than the variance, a;. Remember that values of Og (and sj may depend greatly on the size of the individual samples. Two other distributions that may be encountered, particularly in biological materials, should be mentioned. One is the Poisson distribution, in which the sought-for substance is distributed randomly in the bulk material such that a; is approximately equal to n. In this case The other is the negative binominal distribution, in which the sought-for substance occurs in clumps or patches, and (t; is larger than ^. This pattern often occurs in the spread of contami- nation or contagion from single B-5 sources, and is characterized by two factors, the average, x. and a term, k, called the index of clumping. For this system m (5) Here k must be estimated, along with X, from preliminary measurements on the system. Sometimes, what is wanted is not an estimate of the mean but instead the two outer values or limits that contain nearly all of the population values. If we know the mean and standard de- viation, then the intervals ^l ± 2(t and ^l ±3a contain 95% and 99.7%, respec- tively, of all samples in the popula- tion. Ordinarily, the standard devia- tion (T is not known but only its esti- mate s. based onji observations. In this case we may calculate statistical tolerance limits of the form x + Ks and X — Ks, with the factor K chosen so that we may expect the limits to in- clude at least a fraction P of the sam- ples with a stated degree of confi- dence. Values for the factor K {8) de- pend upon the probability 7 of includ- ing the proportion P of the popula- tion, and the sample size, n. Some val- ues of K are given in Table I. For ex- ample, when 7 = 0.95 and P = 0.95, then K = 3.38 when n = 10, and K = 37.67 for duplicates in = 2). Sampling a Segregated (Strati- fied) Material. Special care must be taken when assessing the average amount of a substance distributed throughout a bulk material in a non- random way. Such materials are said to be segregated. Segregation may be found, for example, in ore bodies, in different production batches in a plant, or in samples where settling is caused by differences in particle size or density. The procedure for obtaining a valid sample of a stratified material is as follows (9): • Based on the known or suspected pattern of segregation, divide the ma- terial to be sampled into real or imagi- nary segments (strata). • Further divide the major strata into real or imaginary subsections and se- lect the required number of samples by chance (preferably with the aid of a table of random numbers). • If the major strata are not equal in size, the number of samples taken from each stratum should be propor- tional to the size of the stratum. In general, it is better to use strati- fied random sampling rather than un- restricted random sampling, provided the number of strata selected is not so large that only one or two samples can be analyzed from each stratum. By keeping the number of strata suffi- ciently small that several samples can be taken from each, possible varia- Figure 4. Relation between minimum sample size and fraction of the richer parti- cles in a mixture of two types of spherical particles (diameter 0. 1 mm and density 1 ) for a sampling |tandard deviation (R) of (a) 0. 1 % and (b) 1 % . Richer panicles contain 10% of substance of interest, and leaner ones contain 0, 1, 5, or 9% (after Reference 12, p 554) tions within the parent population can be detected and assessed without in- creasing the standard deviation of the sampling step. Minimum Number of Individual Increments. When a bulk material is highly segregated, a large number of samples must be taken from different segments. A useful guide to estimating the number of samples to be collected is given by Visman (10). who proposed that the variance in sample composi- tion depends on the degree of homoge- neity within a given sample increment and the degree of segregation between sample increments according to the relation A/W + B/r (6) where s; is the variance of the average of n samples using a total weight W of sample, and A and B are constants for a given bulk material. .4 is called a ho- mogeneity constant, and can be calcu- lated from Ingamells's sampling con- stant and the average composition by .4 = 10-'7-K, (7) Sampling Materials in Discrete Units. If the lot of material under study occurs in discrete units, such as truckloads, drums, bottles, tank cars, or the like, the variance of the analyti- cal result is the sum of three contribu- tions: ( 1) that from the variance be- tween units in the lot, i2) that from the average variance of sets of samples taken from within one unit, and i3) that from the variance of the analyti- cal operations. The contribution from each depends upon the number of units in the lot and the number of samples taken according to the fol- lowing relation (.91: , _ (T^- f.V - n»,i , 'T,, - rih .V ni,n.,, (8> B-6 Glossary Bulk sainpling — sampling of a materlai that does not consist of discrete, identlflat>le, constant units, but rattier of arbitrary, irregular units. Composite — a sample composed of two or more increments. (kocs sample (also called buSt sample, lot sample)— one or more increments of material taken from a larger quantity (lot) of material tor assay or record pirposes. Homogeneity— the degree to wttlch a property or i throughout a n^aterial. Homogeneity depervte on the size of the units uiKler consider- ation. Thus a mixture of two minerals n«iy be inhomogeneous at the molecular or atomic level, but homogeneous at ttie particulate level. m cr emeiH — an individual portion of material collected by a single operation of a sampling devica, from parts of a lot separated in time or space. Increments may be eittier tested individually or combined (composited) and tested as a unit. Indlvkluals — conceivable constituent parts of the population. Laboratory sample — a sample, intended for testing or analysis, prepared from a gross sample or oltierwisa obtained. The laboratory sample must retain tfie composition of the gross sample. Often reduction in particle size is necessary in ttie course of re- ducing ttie quantity. Lot — a quantity of bulk material of similar composition whose properties are uivjer study. Population — a generic term denoting any finite or Infinite collection of Individual things, objects, or events in ttie broadest concept an aggregate determined by some property ttiat distinguishes things tfiat do and do not belong. Reduction — the process of preparing one or more subsamples from a sample. Sample — a portion of a population or kJt. It may consist of an individual or groups of in- dividuals. I specifically demar1sample — a portion taken from a sample. A latx>ratory sample may be a subsample of a gross sample; similarly, a test portion may be a subsample of a laboratory sample. Test portion (also called specimen, test specimen, test unit, aliquot) — That quantity of a material of proper size for measurement of ttie property of interest. Test portions may be taken from ttie gross sample directly, but often preliminary operations, such as mixing or furttier reduction in parUc\e size, are necessary. fT, - = variance ot the mean. iTh- = variance ot the units in the lot. T,, - = average variance of the samples taken from a segment. (T, - = variance of the analytical operations. .V = number of units in the lot, rih = number of randomly selected units sampled, n„ = number of randomly drawn samples from each unit selected for sampling, and n, = total number of analyses. including replicates, run on all samples. If stratification is known to be ih- sent. then much measurement time and effort can be -saved bv combining all the samples and mixing thoroughly to produce a composite sample for analysis. Equation S is applicable to this situation also. If the units vary significantly in weiyht or volume, the results for those units should be weighted accordingly. For homogeneous .materials t , - is zero, and the second term on the right-hand side of Equation S drops out. This IS the case with many liquids or sases. .-Mso. if all units are sampled, then rih, = .V and the first term on the right-hand side of Equation S also drops out. Particle Size in Sampling Particulate l^^ixtures Random -amplmg error may occur even in well -mixed paniculate mix- tures if the particles differ appreciably in composition and the test portion contains too few of them. The problem is particularly important in trace anal- ysis, where sampling standard devia- tions may quickly become unaccepta- bly large. The sampling constant di- agram of Ingamells and the Visman expression are useful aids for estimat- ing sample size when preliminary in- formation is available. .Another ap- proach that can often provide insight is to consider the bulk material as a two-component particulate mixture, with each component containing a dif- ferent percentage of the analyte of in- terest ill). To determine the weight of sample required to hold the sam- pling standard deviation to a prese- lected level, the first step is to deter- mine the number of particles n. The value of n may be calculated from the relation (9) where d, and ^2 are the densities of the two kinds of particles, d is the density of the sample. P; and P2 are the percentage compositions of the component o_f interest in the two kinds of particles. P is the overall average composition in percent of the component of interest in the sample, R is the percent relative standard deviation (sampling error) of the sampling operation, and p and 1 - p are the fractions of *he two kinds o'^ particles in the bulk material. With knowledge of the density, particle diameter, and n. the weight of sample required for a given level of sampling uncertaintv can be obtained through the expression, weight = (4/3)7rr'a'n (assuming spherical particles). Figure 4 shows the relation between the minimum weight of sample that should be taken and the composition of mixtures containing two kinds of particles. (3ne containing 10% o( the sought-for substance and the other 9. 5. 1. or O'^c. A density of 1. applicable in the case of many biological materi- als, is used, along with a particle diam- eter of 0. 1 mm. If half the particles in a mixture contain 10'^'- and the other half 9^-: of the substance of interest, then a sample of 0.0015 g is required if the sampling standard deviation is to be held to a part per thousand. If the second half contains .5^o, a sample of 0.06 g is necessary; if l'~r. 0.3.5 g would be needed. In such mixtures it is rf^e relative difference in composition that is important. The same sample weights would be required if the com- positiems were 100'~c and dO. 50. or 10^7, or if they were 0.1'7 and 0.09. 0.05. or 0.0 1'l:. The same cur\es can be used for any relative composition by substitution ot .r for iO'T;, ana O.I .t. B-7 0.5 X, and 0.9 x for the curves corre- sponding to 1, 5, and 9% in Figure 4. If a standard deviation of 1% is accept- able, the samples can be 100 times smaller than for 0.1%. An important point illustrated by the figure is that if the fraction of richer particles is small, and the leaner ones contain little or none of the sub- stance of interest, large test portions are required. If a sample of gold ore containing 0.01% gold when ground to 140 mesh (0.1 mm in diameter) con- sists, say, of only particles of gangue and of pure gold, test portions of 30 g would be required to hold the sam- pling standard deviation to 1%. (An ore density of 3 is assumed.) Concluding Comments Sampling is not simple. It is most important in the worst situations. If the quantities x,s,Ka,A, and B are known exactly, then calculation of the statistical sampling uncertainty is easy, and the number and size of the samples that should be collected to provide a given precision can be readi- ly determined. But if, as is more usual, these quantities are known only ap- proximately, or perhaps not at all, then preliminary samples and mea- surements must be taken and on the basis of the results more precise sam- pling procedures developed. These procedures will ultimately yield a sampling plan that optimizes the qual- ity of the results while holding down time and costs. Sampling theory cannot replace ex- perience and common sense. Used in concert with these qualities, however, it can yield the most information about the population being sampled with the least cost and effort. All ana- lytical chemists should know enough sampling theory to be able to ask in- telligent questions about the samples provided, to take subsamples without introducing additional uncertainty in the results and, if necessary, to plan and perform uncomplicated sampling operations. It is the capability of un- derstanding and executing all phases of analysis that ultimately character- izes the true analytical chemist, even though he or she may possess special expertise in a particular separation or measurement technique. References (1) W. J. Youden, J. Assoc. Off. Anal C/jem, 50, 1007(1967). (2) T. B, Whitaker, J. W. Dickens, and R. J. Monroe, J Am. Oil Chem. Soc..5\, 214 (1974); T. B. Whitaker, Pure Appl. Chem., 49, 1709(1977), (3) Hazardous Waste Monitoring System, General, Fed Regist.. Vol. 45, No'. 98, pp 3307.5-33127 (May 19, 1980). (4) Staff, ACS Subcommittee on Environ- mental Analytical Chemistry, Anal. r/ipm. 52, 2242 (1980). (5) See, for example, W. J. Dixon and F. J. Massey, Jr., "Introduction to Statistical Analysis," 3rd ed., McGraw-Hill, New York, 1969; M. G. Natrella, "Experimen- tal Statistics," National Bureau of Stan- dards Handbook 91, August 1963, U.S. Government Printing Office. (6) C. 0. Ingamells and P. Switzer, Talan- ta, 20, .547 (1973); C. 0. Ingamells, Ta- lanta, 21, 141 (1974); 23, 263 (1976). (7) S. H. Harrison and R. Zeisler, NBS In- ternal Report 80-2164, C. W. Reimann, R. A. Velapoldi, L. B. Hagan, and J. K. Taylor, Eds., U.S. National Bureau of Standards, Washington, D.C., 1980, p 66. (8) M. G. Natrella, "Experimental Statis- tics," National Bureau of Standards Handbook 91, August 1963, U.S. Govern- ment Printing Office, pp 2-13 and Table A6. (9) ASTM E-300 Standard Recommended Practice for Sampling Industrial Chemi- cals, American Society for Testing and Materials, Philadelphia, 1973 (reap- proved 1979). (10) J. Visman, Materials Research and Standards, November, p 8 (1969). (11) A. Benedetti-Pichler, in "Physical Methods of Chemical Analysis," W. M. Berl, Ed., Academic Press, New York, 1956, Vol. 3, p 183; W. E. Harris and B. Kratochvil, Anal. Chem., 46, 313 (1974). (12) W. E. Harris and B. Kratochvil, "In- troduction to Chemical Analysis," Saun- ders, Philadelphia, 1981, Chapter 21. Kratochvil Byron Kratochvil, professor of chem- istry at the University of Alberta, re- ceived his BS, MS, and PhD degrees from Iowa State University. His re- search interests include solvent ef- fects on solute properties and reac- tions, applications of nonaqueous systems to chemical analysis, and methods for determining ionic so- lutes. John K. Taylor, coordinator for qual- ity assurance and voluntary stan- dardization activities at the National Bureau of Standards Center for Ana- lytical Chemistry, received his BS from George Washington i'niversity. and his MS and PhD degrees from the University of Maryland. His research interests include electrochemical analysis, refractometry, isotope sepa- rations, standard reference materials, and the application of physical meth- ods to chemical analysis. B-8 APPENDIX C Guidelines for Evaluaf-ng APPLiXDTX C The "Blank Correction" John K. Taylor National Bureau of Standards The reagent blank, including contributions du;? to water as a solvent or diluent may significantly affect both the accuracy of •chcinical nieasuroi.'ionts and also the lowest concentration that may be reliably rr.easured. Tectiniques to mininize and control the analytical blank have been disc:i_ ' ' "urphy [I]. The present pacer discusses tne statistical considerations in , the "blank correction" Ordinarily, a measurement is made with all constituents present except the sarr^ple and the measured value of the determinand is considered to be the reagent (sonei:iP'es called chemical) blank. This is subtracted from the value measured for a sapiple, to obtain the "true" concentration of the sample. It is obvious that the blank measurement must be properly made so that the resulting corrections are meaningful. The following should clarify the nature of the OTors resulting from the presence of a reagent blank. Let C"^ = mean of m measurements of the concentration of the measurand in the sample, with standard deviation s m C. = mean of b measurements of the concentration of the measurand in the blank, with standard deviation s, C = best estimate of the concentration of the measurement in the sample, corrected for the blank. The statistical uncertainty of C" is given by where t = t- .^ is the t value for m-1 degrees of freedom for the lOO(l-a)?' confidence 1 evel . Likewise the statistical uncertainty of T, is given by /r The uncertainty of "C is obtained by quadratu^X' to give C-1 if Cs*t's = ^^.. -S' * M/ 4^ ' 4" ^' where t = t, .^^ is the t value for f* degrees of freedom (see equation 3) at the 100(l-a)% confidence level. In the case where the measurement system is demonstrated to be in a state of statistical control and the respective standard deviations are known, equation (1) becomes r~2 T Where Z = Z,_ ,2 i . 1.96 for the 95 percent confidence interval (a = 0.05). A special case exists when C" '^' C", . In this case s, '^^ s = s so that '^ m b b m C c = C„ " C . ± ts if^^ (2) s m b y mb ^ ' where t = t, .^ is the t value for m+b-2 degrees of freedom for the 100(l-a)/i confidence level and .. / (m-1) s^^^ (b-1) s\ in the case of statistical quality control with o = o. = a^ one may use •^s = K-%^ * ^1-0/2 ° Apsr (2a) The expressions (2) and (2a) are based on measurement of the blank and sample by the same method and apply even if the measurand is not detected in the blank. If the blank and sample are measured by different methods, then the equations (1) or (la) apply. Appropriate values of t, based upon the effective degrees of freedom, f*, must be used in equation (1). These n.ay be computed from the equation [2]. f* =1 —J^ 1_ 1-2 (3) 2 In ecjuation (3) , the variance, V, sicpnif ics s . Whenever t]ie blank correction lL>:x:on^:^£ significant, it is ncc* ssary to ircasiire it with sufficient care. It is clear fran die above thsxt blank ni:}asurai>?j-its nviy ne^-vi to be nude witli the sanv2 amouiit of effort as the siLTiple, itself, as ^^ *" ^5- ^^^^ fact is often overlooked by experimenters who may make a limited number of iricasurcmcntf of the blank while devoting most of their effort to measurement of the sample. C-2 Blank corrections become increasingly iaiportant in the case of nicasuromcnts close to the limit of detection. The effect of small variability of the blank is niannificci in this case. Likev/ise even small constant blanks can result in the differencing of two quantities approaching each other in magnitude. The question of acceptable limits for the blank will now be addressed. The absolute value of the blank would appear to be less important than its accurate evaluation. However, it is a necessary correction and good measurement practice dictates that it should be kept within reasonable limits. An empirical rule in the case of trace analysis is to limit the blank correction to no more than ten times the acceptable limit of error for the measurement and furthermore that it should never exceed the concentration level expected in the sample. The logic behind the first condition is that up to a ten percent error in estimation of the blank would cause no serious difficulties. The second condition is to prevent minor errors in the two measured quantities from introducing large errors in the difference which is the quantity of practical interest. In the preceding discussion, the significance of the blank was considered on the basis of its contribution as a concentration factor under the final conditions of measurement. Furthermore, the term C, contains the sum of the contributions from each source of blank. If C, is excessive, and if several sources (reagents) are involved, measurements must be carried out in a suitable program to identify each source and the magnitude of its contribution in order to tate corrective actions. Obviously, the magnitude of each source of blank and the final conditions of nieasureiiient (e.g., final volume of a solution) must be considered in establishing a permissible level for the blank for each reagent used. In all of the above discussions, it was assumed that the blank measurement simulates the sample measurement process so that the value of C, is meaningful. In some cases it may be difficult or even impossible to fully simulate the sample measurement process unless the sample matrix is present in critical steps of the procedure. If matrix simulation is necessary and cannot be achieved, it may be necessary to indepen- dently analyze each reagent for its measurand content and calculate its contribution to the measurement blank. A related question is the uncertainty of the measured values resulting from uncer- tainties in the analytical function. Most measurements involve the use of such a function to relate the measured quantity (signal) to the concentration of the sample. Uncertainties in this function must be considered as a measurement uncertainty. The uncertainty in the analytical function is not a significant consideration in the blank correction, provided both measurements use the same function. However, it must be considered in evaluating the final measured value. C-3 References Ilj Thomas J. Murphy, "The Role of the Analytical Blank in Accurate Trace Analysis", MBS Special Publication 422, Accuracy in Trace Analysis: Sampling, Sample Handling, Analysis (1976). [2J Mary G. Natrella, "Ex-porimental Statistics", NBS Handbook 91, p. 3-28, (1963). C-4 APPENDIX D Report iiirilMWlii John K. Taylor Center (or Analytical Chemistry National Bureau o( Standards Washington, D.C. 20234 AeaSy tncal Mstihi€)ds Validation of analytical methods is a subject of considerable interest. Documents such as the "ACS Guide- lines for Data Acquisition and Data Quality Evaluation" (I) recommend the use of validated methods. The promulgation of federal environmen- tal regulations requires the inclusion of validated reference methods. Stan- dards-writing organizations spend considerable time in collaborative testing of methods they prepare, vali- dating them in typical applications and determining their performance characteristics. Nevertheless, ques- tions about the appropriateness of methods and the \alidit\ of their use in specific situations often arise. Some of these questions may be due to dif- ferences in understanding both what a method really is and what the signifi- cance of the validation process is. This paper attempts to clarity the nomen- clature of analytical methodology and to define the process of validating methods for use in specific situations. Hierarchy of Methodology The hierarchy of methodology, pro- ceeding from the general to the specif- ic, may be considered as follows: technique -* method -* procedure — protocol. A technique is a scientific principle that has been found to be useful for providing conipositii>nal information: spectrophotometry is an example. An- alytical ohtMnists historically have in- vestigated new measurement tech- niques for their ability to provide novel measurement capability, or to replace or supplement existing meth- odology. As a result of innovative ap plications, analysts can now analyze Tl>i~ KKI'UU I' l> Ims.'.l .HI ;.S.|>l IJ 17, I'.tSJ. KniKiisfilv. M.>. for myriad substances in exceedingly complex mixtures at ever lower trace levels, with precision and accuracy un- dreamed of only a few years ago (2). A method is a distinct adaptation of a technique for a selected measure- ment purpose. The pararosaniline method for ineasurement of sulfur dioxide is an example. It involves mea- suring the intensity of a specific dye, the color of which is "bleached" by the gas. Several procedures for carrymg out this method may be found in the literature. Modern methodology is complex and may incorporate several measurement techniques; a method may thus be interdisciplinary'. A procedure consists of the written directions necessary to utilize a meth- od. The "standard methods" de\el- oped by ASTM and AOAC are. in re ality, standardized procedures. AST.M D2914— Standard Test Method for the Sulfur Dioxide Content of the .At- mosphere (West-Gaeke Method)— is an example (3). While a precise de- scription is the aim, it is difficult, if not impossible, to describe every de- Hierarchy of Analytical Methodology Definition Example Technique Scientific principle useful for providing compositional infor- mation Spectrophotometry Method Distinct adaptation of Pararosaniline method a technique for a se- for measurement of lected measurement sulfur dioxide purpose Procedure Written directions ASTM D291 4— Standard necessary to use a Test Method for the method Sulfur Dioxide Content of the Atmosphere (West- Gaeke Method) Protocol Set of definitive EPA Reference Method directions that must be for the Determination followed, without excep- of Sulfur Dioxide tion, if the analytical In the Atmosphere results are to be (Paiarosaniline Method) accepted for a given purpose Pul>lishi-d in Analytical Chcmi.stry, May 1983, G00A-G08A, by the American Chemical Society D-1 tail of every operation in a procedure. Accordingly, some level of sophistica- tion is presumed for the user of every published procedure; if very sophisti- cated users are contemplated, only a minimum of detail will be provided and vice versa. However, it should be noted that any omission in the de- scription of critical steps is a potential source of variance or bias, even in the hands of knowledgeable analysts. Be- cause of the flexibility intentionally or unintentionally provided to the ana- lyst, or because of differences in inter- pretation, it is fair to say that minor- to-major differences of application occur in the use of even the most pre- cisely defined procedures. Such differ- ences often account for the interlabo- ralorv variability observed in many collaborative tes's. Further, at some point of departure from a published procedure, a new method results that mav need its own validation. 'rhe term protocol is the most spe- cific name for a method. A protocol is a set of definitive directions that must be followed, without exception, if the analytical results are to be accepted for a given purpose. Protocols may consist of existing methods or proce- dures, modifications of such, or they mav be developed especially for spe- cific purposes. Typically, they are pre- scribed by an official body for use in a given situation such as a regulatory process. The VIPA Reference Method lor the Determination of Sulfur Diox- ide in the Atmosphere (Pararosaniline Method) is an example of a protocol (■/), The test method specified as part of a contractual arrangement for the acceptance of data or a pnjduct or ma- t.-rial is another cxam|)le of a protocol, although it may not be called that in the contract. A plethora of methods, procedures. Figure 1. Basic concept of the validation process and protocols based on the same mea- surement principle can arise for a given analytical determination. Usual- \y, they are worded differently, and they may contain subtle or major dif- ferences in technical details. The ex- tent to which each needs to be individ- ually validated is a matter of profes- sional judgment. It is evident that some validation tests could be merely a matter of experimentally tc'^i t ' ' '-> clarity of the written word. Goals for Validation Validation is the process of deter- Diinin',' the suitability of iuethodology ior providing useful andytical data. This i> a value judgment in which the performance paramt'ters of the meth- od an comi)ared with the require- ment.-. f. contributed i as a prime measurenu'iil leilinique. Methods arive as the result of ai>- plied research, typically bv individu- nls. that often involves tioth n compre- hensive understanding of me.isiire- ineiU techniques and .1 liigl. clev!ree of ingenuitv and innovation in llieir ap- plication. Testing of the methods in typical praclic.i' silnalions plays a key and in validation. While ordinarily limited in scope, validation at the re- search stage can be comi)rehensive and can apply to a wide variety of end uses. Procedures are develo[)ed for the end use of methods in practical ana- Ivtical situations. The user laboratory ordin.irily needs more experimental details than are contained in a pub- lished research report of a method to use it in practical measurements. Fre- quently, as a method gains widespread use, procedures evolve that the users may decide need to be standardized. This is often done by consensus in a standards organization forum. During this process, the resulting standard procedure is examined both technical- ly and editorially. A thorough review- process includes collaborative testing in which typical stable test materials are analyzed to verify the procedure's usefulness and to identify both techni- cal and editorial weaknesses. The pro- cess is illustrated in Figure 2. If the composition of the reference samples is known, precision and bias, both intra- and interlaboratorv', can be evaluated: otherwise, only precision can l)e evaluated. If a metb.od of known accuracy is available, the col- laborative test may consist of its com- parison with the candidate method, in which case both precision and bias can be evaluated. The performance pa- rameters of the procedure so evalu- ated are for the conditions of the col- laborative test that are considered typical. Any extension of them to other kinds of samples is by inference only, and may need to be justified. Al- though it can be time-consuming, the development of a standard method is one of the best ways to validate a pro- cedure because of the breadth of ex- amination that is involved. A protocol is prescribed by fiat of an organization reciuiring a specific kind of measurement. Presumably it result.s from an intelligent decision based on the organization's validation process or that of others. This inav consist of an extensive collaborative test or pub- lication of a proposed protoctil for public comment, llnfortunaiely, expe- diency has overruled sound siientific judgment in some cases, resulting in the promulgation of unvalidated and scienlil'icallv defective prolocls iti). Protocols that are specilied in a con- tractu. il arrangement m.iv be seleiteil arbil rarilv or lhrou;.;h a well -conceived selection process. \ frilicalion of their validity lor the specific use should be a prime consideration. Validation for Specific I'sc. 'i'he D-3 tail of even- operation in a procedure. Accordingly, some level of sophistica- tion is presumed for the user of every published procedure; if very sophisti- cated users are contemplated, only a minimum of detail will be provided and vice versa. However, it should be noted that any omission in the de- scription of critical steps is a potential source of variance or bias, even in the hands of knowledgeable analysts. Be- cause of the ne.xibility intentionally or unintentionally provided to the ana- lyst, or because of differences in inter- pretation, it is fair to say that minor- to-major differences of application occur in the use of even the most pre- ciselv defined procedures. Such differ- ences often account for the interlabo- ratorv variability observed in many collaborative tes's. Further, at some point of departure from a published procedure, a new method results that mav need its own validation. The term protocol is the most spe- cific name for a method. A protocol is a .set of definitive directions that must be followed, without exception, if the analytical results are to be accepted for a given purpose. Protocols may consist of existing methods or proce- dures, niiidifications of such, or they mav l)e developed especially for spe- cific purposes. Typically, they are pre- scribed by an official body for use in a given situation such as a regulatory process. The P:PA Hefercnce Method for the Determinalion of Sulfur Diox- ide in the Atmosphere (Pararosaniline Method) is an example of a protocol (•/ ). The test method specified as part of a contractual arrangement for the accept.ince of data or a product or ma- tpri.il is .mother example of a protocol, although it may not be called that in the contract. A plethora of methods, procedures, Figure 1. Basic concept of the validation process and protocols based on the same mea- surement principle can arise for a given analvtical determination. Usual- Fy, they are worded differently, and they may contain subtle or major dif- ferences in technical details. The ex- tent to which each needs to be individ- uallv validated is a matter of profes- sional judgment. It is evident that some validation tests could be merely a matter of experimentally testing the clarity of the written word. Goals Jor Validation Validat ion is the procejjs of deter- mining the suitability of iuethodology for providing useful analytical data. This i> a value juilgmenl in which the performance param.-lcrs of the meth- od are comf):!red with the require- menl^ for the analytical data, as illus- trated in Figure 1. Obviously a method that is valid in one situation could be invalid in another. Accordingly, the establishment of firm requiremenU for the data is a prerequisite for meth- od selection and \alidation. When data requirements are ill-considered, analytical measurement can be unnec- essarilv expensive if the method cho- sen is more accurate than required, in- adequate if the method is less accurate than re«iuired. or utterly futile if the accuracv of the method is unknown. Fortunately, typical and even sUn- dard measurement [)roblems often exist. Kxamples include a wide variety of clinical analyses, environmental de- terminations, and recurring measure- ments for the characterization of in- dustrial products. The kinds of sam- ples for which methods have been val- D-2 idatt.Uhouldlx-.lfarlvclcsrrihcd. and UMTS sh. mid lu- .iv.arc ..I lli. need todi-rudiislraU- lluar own .d)!!!!!!-. lo use ihf niilhixl m ttu'ir own laliDraio- rie.s. Statcnu-nlh o( prttasion and accura- cy a ri' ol'lfii a rcMill of a validation process, especially m (he case o|.,a col- laborative lest exercise. Such state- ments are ollen misinterpreted; ihey merely describe the results of the ex- ercise and are. at best, estimaUs o! typical perlornKince expectations lor the method. 'Ihev should not be con- strued tube perlor'iKUUe iJaranielers nor should they be used to cstuiiale the uncertainly ot an>- future data ob- tained by usiii;,' the method. However, information on [irecision and accuracy should be obtained to the extent pos- sible since it pro\ides a quantitative basis lor judi;ini,' {general performance capability. Other information useful for char- acterizing methi'dolo^'y or for judgin<< its suitability lor n i;i\en use includes: sensitivity to intcrlerences, limits of detection, and useful ranire of mea- surement. The specific details for evaluating methodolh a compre- hensive understanding of nir.i: rnenl techniques and .1 bigl. di pi, -n.T: :f the meihodsin role in both the devclot>meni [irocess' and invalidation. While ordinarily limited m sco|>e. validation at the re- search stage can be coniprehensive and can apply to a wide variety of end uses. I'rocedures are developed for the end use of methods in practical ana- lytical situations. The user laboratory ordinarily needs more ex|)erimental details than are contained in a pub- lished research report of a method to use it in practical measurements. Fre- quently, as a method gains widespread use, procedures evobe that the users may decide need to be standardized. This is often done by consensus in a standards organization forum. During this process, the resulting standard procedure is examined both technical- ly and editorially. A thorough review process includes collaborative testing in which typical stable test materials are analyzed to verify the procedure's usefulness and to identify both techni- cal and editorial weaknesses. The pro- cess is illustrated in Figure 2. If the composition of the reference samples is known, precision and bias, both intra- and interlaboratorv, can be evaluated: otherwise, only precision can be evaluated, if a metb.od of known accuracy is available, the col- laborative test may consist of its com- parison with the candidate method, in which case both precision and bias can be evaluated. The performance pa- rameters of the procedure so evalu- ated are for the conditions of the col- laborative test that are considered typical. .Any extension of them to other kinds of samples is by irderence only, and may need to be justified. Al- though it can be time-consuming, the development of a standard method is one of the best ways to validate a pro- cedure because of the breadth of ex- amination that is involved. A protocol is prescribed by fiat of an organization re). The model repre- sents the conceptualization of the problem to be solved, describes the samples that should be analyzed, the data base required, and the way the model will be utilized. Obviously, even flawless measurement data will be of little value if the basic concepts are faulty. Likewise the samples analyzed must be valid if the results obtained for them are to be intelligently inter- preted. The key role of reliable reference materials in the validation of analyti- cal measurements cannot be overem- phasized. Their use in validating the methodology has already been dis- cussed. A planned sequential analysis of relerence materials in a quality as- surance program can assess the quali- ty of the data output and thus validate the overall aspects of the analytical measurement system (7). References ( 1 ) ACS Subcommittee on Environmental Analvticnl Chemistry. Anal. Chem. 1980, 52. •2-242. (2) Taylor, John K. CHKMTECH 1982. 12. ■nh. (3) Annual Rook of ASTM Standards, Part 'J(). American Society for Testing and Materials, Philadelphia, Pa. 1910;i. liiMl. (4) National Primary and Secondary Am- l)U'nt .-XirguMliIv Stanclirds, h\d R.-^ist. April :W. 1!I71. Vol. :l(;. Part SI. p 8187. C)) Cla.ser, J. A.: Kin^rst. 1). I... NU Kee, ('.. 1)., tjuave, S. A.; Miidile. \V. L. Eni't- run Sit. 'I'l-chnol I9S1, 7.5. MJU-M.S. (G) Purdue, I.. T. el al. t:nvin„i. Sci. Tcch- nol. 1972,6,152-54. (7) Taylor, John K. "Reference Materi- als—How They Are or How Thev Should Be Used," ASTM Journal of Testing and Technology, in press. (8) Taylor. JohnK. Anal. Chem 1981,53, 158g-96 A. (9) Kratochvil, Byron; Tavlor. John K. Anal. Chem. 1981, 53, 924-38 A. John K. Taylor is coordinator for chemical measurement assurance and voluntary standardization at the Xa- tional Bureau of Standards' Center for Analytical Chemistry. He recened his BS degree from George Washing- ton University and his MS and Phi) degrees from the I'nirersity of Mary- land. His research interests include electrochemical analysis, refractome- try. isotope separations, standard reference materials, and the applica- tion of physical methods to chenucnt analysis. D-5 APPENDIX E E. Selected References 1. ASTM D2777, ASTM Philadelphia, Pa 19103. Standard Practice for determination of the precision and bias of methods of committee D19 on water. 2. ASTM D3614, ASTM Philadelphia, Pa 19103. Evaluating laboratories engaged in sampling and analysis of atmospheres and emissions. 3. ASTM D3856, ASTM Philadelphia, Pa 19103. Evaluating laboratories engaged in sampling and analysis of water and waste-waters. 4. ASTM El 78, ASTM Philadelphia, Pa 19103. Standard recommended practice for dealing with outlying observations. 5. ASTM E305-83, ASTM Philadelphia, PA 19103. Standard practice for establishing and controlling spectrochemical analytical curves. 6. ASTM E-548, ASTM Philadelphia, PA 19103. Standard recommended practice for generic criteria for use in evaluation of testing and/or inspection agencies. 7. ASTM E-748, ASTM Philadelphia, PA 19103. Quality assurance procedures for spectrographic laboratories. 8. G. A. Berman, Ed., Testing laboratory performance: Evaluation and accreditation. NBS special publication 591. 9. R. Bordner and J. Winter, Microbiological methods for monitoring the environment, water, and wastewater. EPA-600/8-78-017 , Dec. 1978. 10. J. Bryson, et al , Bibliography on laboratory accreditation, NBSIR 82-2523 (1982) National Bureau of Standards, Washington, D.C. 20234. 11. J. P. Gal i , et al , The role of standard reference materials in measurement systems, NBS monograph 148. E- 12. ASTM Manual on presentation of data and control chart analysis. STP 15D, ASTM, Philadelphia, PA 19103. 13. C. Eisenhart, Realistic evaluation of the precision and accuracy of instrument calibration systems, in ref. 26, pp 21-47. 14. J. J. Filliben, Testing basic assumptions in the measurement process, in "Validation of the Measurement Process", J. R. Devoe Ed., ACS Symposium Series No. 63 (1977). 15. L. C. Friedman and D. E. Erdmann, Quality assurance practices for the chemical and biological analyses of water and fluvial sediments- Chapter A6 in "Techniques of Water-Resources Investigations of USGS" (1982). 15. S. Gaft and F. D. Richards, Quality assurance at Ford Motor Company Central Laboratory-A dynamic approch to laboratory quality, in ASTM STP/ 814. 16A. J. A. Glaser, et al . , E. S. & T. 15, 1426-35 (1981). 17. Trace analysis for wastewaters - Method detection limit, J. A. Glaser et al . E. S. & T. 15, 1426-35 (1981). 18. Guidelines for data acquisition and data quality evaluation in environmental chemistry, American Chemical Society, Analytical chemistry 52: 2242-2249; 1980. 19. Handbook for analytical quality control in water and wastewater laboratories EPA 600/4-79-019, March 1979. 20. C. D. Hendrix, What every technologist should know about experimental design. CHEMTECH, March 1979, pp. 167-174. 21. H. S. Hertz and C. N. Chesler, Eds. Trace organic analysis: A new frontier in analytical chemistry. NBS special publication 519. 22. J. Stuart Hunter, Calibration and the straight line: Current statistical practices. AOAC journal 6^, No. 3, 574-83 (1982). E-2 23. ISO Guide 25, Guidelines for assessing the technical competence of testing laboratories, American National Standards Institute, 1430 Broadway, New York, NY 10018. 24. B. G. Kratochvil and J. K. Taylor, Sampling for chemical analysis, et. al. , chem. 53 924A-3&4 (1981). 25. B. G. Kratochvil and J. K. Taylor, A survey of the recent literature on sampling for chemical analysis, NBS technical note 1153, January 1982. National Bureau of Standards, Washington, D.C. 20234. 26. H. S. Ku, Editor, Precision Measurement and Calibration: Statistical Concepts and Procedures, NBS Special Publication 300, Vol. 1. Stock No. 003-003-00072-8 Superintendent of Documents, U.S. Govt. Printing Office, Washington, D.C. 20402 ($11.00). 27. P. D. LaFleur, Ed., Accuracy in Trace Analysis: Sampling, Sample Handling, Analysis, NBS Special Publication 422. 28. J. Mandel and T. W. Lashof - Interpretation and Generalization of Youden's Two-Sample Diagram. J. Quality Technology 5 pp. 22-36 (1974). 29. A. G. McNish, Dimensions, Units, and Standards, Physics Today, 10^, pp. 19-25, April 1957. 30. W. W. Meinke, Ed., Trace Characterization, Chemical and Physical, NBS Monograph 100. National Bureau of Standards, Washington, D.C. 20234. 31. M. G. Natrella, Experimental Statistics, NBS Handbook 91, Stock No. 003-003-00135-0, Superintendent of Documents, U.S. Govt. Printing Office, Washington, D.C. 20402 ($18.00). 32. L. S. Nelson, Use of Range to Estimate Variability, J. Qual . Technology, Vol . 7 No. 1 , Jan. 1975. 33. W. Nelson, How to Analyze Data with Simple Plots. ASQC Basic References in Quality Control - Statistical Techniques. 34. Applied Linear Statistical Models, J. Nether and W. Wasserman, p. 140-145 (1974). Published by Richard D. Irwin, Inc., Homewood, Illinois 60430. 35. W. E. Oatess - Establishment of Accreditation Programs for Environmental Labs. Env. Sci . Tech. 1^1124-27 (1978). 36. Precision Measurement and Calibration, NBS Handbook 77. 37. Principles of Environmental Measurement, American Chemical Society, Anal. Chem. 55 2210-18 (1983). 38. Quality Control System - Requirements for a Testing and Inspection Laboratory, American Council of Independent Laboratories, 1725 K Street N.W. , Washington, D.C. 20006. 39. Standard Reference Materials and Meaningful Measurement, NBS Special Publication 408. 40. Standard Reference Materials Catalogue, NBS Special Publication 260. National Bureau of Standards Washington, D.C. 20234. 41. J. K. Taylor, Importance of Inter-calibration in Marine Analysis, Thai. Jugo. 24 221-29 (1978). 42. J. K. Taylor, Quality Assurance of Chemical Measurements, Anal. Chem. 53 1588A-96A (1981). 43. J. K. Taylor, Quality Assurance Measures for Environmental Data, in "Lead in the Marine Environment", M. Branica and Z. Konrad Eds., Pergamon Press (1980) pp. 1-7. 44. J. K. Taylor, Validation of Analytical Methods, Anal. Chem. 55, 600A (1983). 45. J. K. Taylor, Reference Materials - What they are and how they should be used. J. Testing and Evaluation Vl_, 355-7 (1983). 46. G. Wernimount - Ruggedness Evaluation of Test Procedures, ASTM Standardization News - pp. 13-16, March 1977. E-^ 47. J. 0. Westgard and T. Groth, A multi-rule Shewhart Chart for Quality Control in Clinical Chemistry, Clinical Chemistry 27, 495*501 (1981). 48. William J. Youden, Collection of Papers on Statistical Treatment of Data. Journal of Quality Technology, Vol. 4, No. 1, pp. 1-57, January 1972. 49. Ranking Laboratories by Round Robin Tests, W. J. Youden, Materials Research and Standards, Jaunary 1963. 50. W. J. Youden and E. H. Steiner, Statistical Manual of the Association of Official Analytical Chemists, AGAC, PC Box 540, Washington, DC 20044. 51 . ASTM E882 Accountability and Quality Control in the Chemical Analysis Laboratory. 52. ASTM El 73 Conducting Interlaboratory Studies of Methods of Chemical Analysis of Metals. 53. ASTM CI 009 Establishing a Quality Assurance Program for Analytical Chemistry Laboratories Within the Nuclear Industry. 54. CFR Title 21, Food and Drugs, Chapter 1, F&DA, Part 38, GLP's for Non-Clinical Laboratory Studies. 55. CFR Title 40, Part 792, Toxic Substances Control. Fed. Reg. 48, No. 230, November 28, 1983, pp. 53922. 56. CFR Title 42, Public Health, Chapter 1, Public Health Service, Part 74, Clinical Laboratories. 57. QAMS-005/80, Interim Guidelines and Specifications for Preparing Quality Assurance Project Plans (EPA). 58. L. P. Provost, "Statistical Methods in Environmental Sampling" in Environmental Sampling for Hazardous Wastes, ACS Symposium Series 267 (1984) American Chemical Society, Washington, DC 20036. E-5 QUALITY ASSURANCE CODE OF ETHICS With full appeciation of my responsibilities as an analytical chemist, I subscribe to all apsects of The Chemist's Creed and to the ethical practice of the profession of analytical chemistry and particularly will strive to: 1 . Acquire a full understanding and develop peer technical expertise in every area of analytical chemistry in which my professional services are offered. 2. Comprehend, to the extent possible, all problems for which my analytical services are required; assure myself of the validity of the approach selected; understand any limitations on the measurements and discuss such with clients, as appropriate. 3. Use validated methodology, exclusively. 4. Demonstrate statistical control of the measurement system before definitive measurements are made. 5. Calibrate, to the extent necessary and possible; engage in intercalibration activities as appropriate to minimize chance of internal laboratory bias. 6. Utilize recognized Good Laboratory Practices (GLP's) and Good Measurement Practices (GMP's) throughout all aspects of sampling and measurement processes. 7. Utilize documented procedures and record all significant experimental details in such a way that the measurements could be reproduced by myself or a competent analyst at a later date as necessary. 8. Provide or have available limits of uncertainty of all data reported including that due to sample and to measurement, supported by statistical inference and/or professional judgment, as pertinent; state clearly the basis for any interpretations provided of the measurement data. 9. Confirm the qualitative identification of all parameters measured and provide supporting evidence as necessary. 10. Retain all samples, data and documentary evidence as necessary for a period of time commensurate with its importance. E-6