Verification and Validation Toolkit
This toolkit will help laboratories plan, implement and analyze data associated with new testing methods and technology implementation, determine the difference between a validation and a verification, when each should be performed and how to perform them.
Contact Quality Systems and Analytics: [email protected]
Using the Toolkit
This toolkit can help laboratories plan, implement and analyze data associated with new testing methods (FDA-approved tests and FDA-modified or laboratory developed tests) and technology implementation. In addition, the toolkit serves as a guide to help determine if the quality requirements for establishment of a new test system have been met. These recommendations should be followed to ensure a new or modified method, analyte or instrumentation (the "test method") has acceptable performance.
This toolkit is the result of collective knowledge of subject matter experts and contains guidance for performing verifications and validations that will be compliant with requirements defined by the Clinical Laboratory Improvement Amendments (CLIA). Many of the described processes may be useful for other regulatory requirements.
The examples in the toolkit are primarily focused on microbiology; the toolkit will be expanded to other testing areas in the future.
Verification and Validation Tools
The toolkit is broken down into eight main sections, plus additional reference documents. Download a complete version of the toolkit or download individual sections and resources from within the dropdowns below.
Complete Toolkit
Toolkit Sections
-
Verification and Validation 101
This section provides an overview of the process, required performance characteristics and general steps of a plan.
-
Checklist for the Verification or Validation Process
This section provides an overview of the plan development, initiation of the plan, creation of the testing and summary report and test implementation.
-
Obtaining Appropriate Test Samples
This section provides information on sample types, sample numbers and sample volume.
-
Qualitative Assays
This section provides information on determining performance characteristics, qualitative controls, accuracy, precision, analytical sensitivity, specificity, and reportable and reference ranges.
-
Quantitative Assays
This section provides information on calibration and statistical calculations, quantitative controls, accuracy, precision, analytical sensitivity and specificity, reportable and reference ranges, and ongoing verification of quantitative assays.
-
QSA-VV-Toolkit-Related
This section provides information on stability and integrity, bridging studies and addendums, instrument verification or validation, reagent lot-to-lot comparison, defining when to perform comparisons, reagent comparison failures, and software upgrades. Also included in this section are related templates and examples.
-
Safety Considerations and Risk Assessments
This section provides information on biological and chemical risk assessment considerations, risk assessment tables, reviewing a risk assessment and hazardous waste management.
-
Cost Analysis and Budget
This section provides information on labor, materials, site and maintenance costs.
Reference Materials
Checklists and Plans
- Printable Checklist for the Verification or Validation Process
- Method Verification Validation Plan Approval Checklist
- Verification Plan Template
- Verification Summary Report Template
- Validation Plan Template
- Validation Summary Report Template
Qualitative Assays
- NGS Method Validation Plan Template (NGS Quality Initiative)
- NGS Method Validation Summary Report Template (NGS Quality Initiative)
Quantitative Assays
- CLIA-compliant Analytical Method Validation Plan and Template FOR LRN-C Laboratories (APHL)
- Breakpoint Implementation Toolkit, CRO 2021 (APHL)
- Breakpoint Implementation Toolkit, MIC 2023 (CLSI)
Related Processes
- Bridging, Addendum and Extension Studies Template
- Reagent Comparison QC Studies Template
- Software Update SOP and Template
Safety Considerations and Risk Assessments
Checklists and Plans
- Employee Training Verification Checklist Example
- Guidelines for Verification and Validation of Laboratory Methods (MN)
- Method Validation-Verification Summary Report Example (Fairfax County, VA)
- Method Verification Template Example (IN)
- Validation and Verification SOP Example (TX)
- Validation Plan Example (WA)
- Validation Summary Report Example (WA)
- Verification Plan Example (WA)
Qualitative Assays
- Microbiology MALDI-TOF Validation Supplemental Checklist (New York CLEP)
- Microbiology NAAT Checklist (New York CLEP)
Related Processes
CLSI Harmonized Terminology Database
The Clinical and Laboratory Standards Institute (CLSI) has complied a Harmonized Terminology Database of internationally accepted terminology used in laboratory sciences and related health organizations. This tool is publicly available to encourage broad acceptance and usage of internationally accepted terminology in the laboratory community.
Glossary
Accuracy: An analytical performance measurement that assesses the ability of a method to produce correct results, as compared to a reference standard. Diagnostic sensitivity (known absence for target analyte) and diagnostic specificity (known presence of target analyte) are used.
Analytical Measurement Range: The range of analyte values that a method can directly measure on the specimen without any dilution, concentration, or other pretreatment not part of the usual assay process.
Analytical Sensitivity (Limit of Detection): The lowest concentration, or amount of an analyte, that can be measured and distinguished from a blank (i.e., minimum detection limits).
Analytical Specificity (Interfering Substances): The ability of an instrument or test system to measure only the intended organism or substance. Tests the ability to discriminate between the target analyze and other related, but non-target analytes (i.e., cross-reactivity, interfering substances).
Coefficient of Variation (CV): A measure of relative precision. It is calculated as 100 times the standard deviation, divided by the mean, and expressed as a percentage.
Control—High Positive: A sample, preferably matrix-matched, to evaluate the ability of a laboratory to identify the target of interest near the upper limit of the reportable range, if applicable.
Control—Low Positive: A sample, preferably matrix-matched, to evaluate the ability of a laboratory test to identify the target of interest near the lower limit of the reportable range or near the cutoff.
Control—Negative: A sample, preferably matrix-matched, that lacks the target of interest and used to evaluate the ability of a test not to detect the target when it is not present.
Cutoff Value: In qualitative assays, the cutoff is defined as : the threshold above which the result is reported as positive and below which the result is reported as negative.
Direct Costs: Specific costs traceable to the test produced. These can be fixed or variable. Examples include supplies, reagents, consumables, labor, instrument costs, standards and controls.
Emergency Use Authorization (EUA): An EUA is a mechanism that the enables the FDA to facilitate the availability and use of medical countermeasures during declared public health emergencies.
False Negative: A negative result incorrectly ascribed to a positive sample.
False Positive: A positive result incorrectly ascribed to a negative sample.
FDA Approved: The device has been approved through the Premarket Approval process.
FDA Authorized: The device has been reviewed by FDA through the EUA mechanism.
FDA Cleared: The device has been cleared as a substantially equivalent device through Section 510(k) of the Food, Drug and Cosmetic Act.
FDA Modified: Any modification to an FDA approved or cleared test. The modification should be handled as a laboratory developed test.
Fixed Costs: A cost that remains constant regardless of workload and within a specific range of activity. Examples include Labor, rent, equipment depreciation, equipment/instrumentation.
Gold Standard: Any standardized clinical assessment, method, procedure, intervention or measurement of known validity and reliability which is generally taken to be the best available, against which new tests or results and protocols are compared.
High Complexity: The most complex testing category assigned to a test by the FDA, based on seven scored criteria. CLIA requirements for laboratories will vary based on the assigned complexity of a test with more stringent requirements for high complexity testing.
Indirect Costs: Any cost that is not assigned to the direct production of a test but contributes to the adequate provision of the work environment. Examples include supervisory salaries, quality assurance, education, travel, administrative costs, building maintenance, security and training.
Individualized Quality Control Plan (IQCP): The Clinical Laboratory Improvement Amendments (CLIA) Quality Control (QC) procedure for an alternate QC option allowed by 42CFR493.1250. The guidance and concepts for IQCP are a formal representation and compilation of many things laboratories already do to ensure quality test results. IQCP permits the laboratory to customize its QC plan according to test method and use, environment, and personnel competency while providing for equivalent quality testing.
Laboratory-developed Test: Test developed wholly, or in part, by the performing laboratory. This may include analyte specific reagents (ASR) or adoption of another laboratory's LDT or non-cleared or approved test.
Limit of Detection (LoD): In quantitative and qualitative measurement procedures, the lowest concentration of analyte that can be consistently detected (typically, in ≥ 95% of samples tested under routine medical laboratory conditions and in a defined type of sample).
Matrix Effect: The influence of sample property independent of analyte presence.
Negative Predictive Value (NPP): The probability that a negative result accurately indicates the absence of the analyte or specific disease.
Positive Predictive Value (PPV): The probability that a positive result accurately indicates that the analyte or specific disease is present.
Precision: An analytical performance measurement that assesses the closeness of agreement between independent results of measurements obtained under stipulated conditions. Assesses the inherent random error of a test system to determine how close two or more repeated measurements are to each other, regardless of accuracy.
Premarket Approval (PMA) Process: This is the FDA process of scientific and regulatory review to evaluation the safety and effectiveness of Class III Medical Devices.
Premarket Notification 510(k): Before marketing a device in the US intended for human use that does not require a PMA must submit a 510(k) to FDA (unless otherwise exempt).
Reference Range: The typical result (qualitative) or range of values (quantitative) expected in a non-diseased population that do not have the condition for which the test is performed, including variation due to type of specimen and demographic variables such as age and sex, as applicable.
Reportable Range: The span of test result values over which the laboratory can establish or verify the accuracy of the instrument or test system measurement response. For a qualitative test, the reportable range could be the limit of detection, the cutoff value, or the 95% confidence interval.
True Negative: A negative result that correctly reflects the condition of a sample.
True Positive: A positive result that correctly reflects the condition of a sample.
Validation: The process used to confirm with objective evidence that a laboratory-developed test (LDT) or modified FDA-cleared or approved test method or instrument system delivers reliable results for the intended application.
Variable Costs: A cost that varies with changes in test volume. Examples include reagents and consumables. Labor costs can sometimes be variable when significant increases or decreases in test volume occur, but typically labor will be a fixed cost.
Verification: The one-time process by which a laboratory determines that an unmodified FDA-cleared or approved test performs according to the manufacturer's specifications when used as directed.
Acronyms
AMR: Analytical Measurement Range
ANSI: American National Standards Institute
ASR: Analyte Specific Reagents
CAP: College of American Pathologists
CFU: Colony Forming Units
CLIA: Clinical Laboratory Improvement Amendments
CLSI: Clinical and Laboratory Standards Institute
CoA: CLIA Certificate of Accreditation
CoC: CLIA Certificate of Compliance
CV: Coefficient of Variation
EUA: Emergency Use Authorization
FDA: US Food and Drug Administration
FSIS: USDA Food Safety and Inspection Service
HL7: Health Level 7 standards for electronic transfer of data
IFU: Instructions for Use
IQCP: Individualized Quality Control Plan
ISO: International Organization for Standardization
LIS: Laboratory Information System
LoD: Limit of Detection
MSDS: Material Safety Data Sheet
PPE: Personal Protective Equipment
PPV: Positive Predictive Value
PT: Proficiency Testing
QAO: Quality Assurance Officer
QC: Quality Control
RF: Relative Fluorescence
TCID50: Median Tissue Culture Infectious Dose
TNI: The NELAC Institute