Joint Research Conference
June 24-26, 2014
Design and Analysis of Measurement System Comparison StudiesThe comparison of two measurement systems is important in both industrial and medical contexts. In general, the goal is to decide if a new measurement system agrees suitably with an existing one, and hence decide whether the two can be used interchangeably. A variety of methods for comparison have been proposed in both contexts; in the industrial context measurement system comparison relies largely on a comparison of repeatabilities, and no consideration of bias. Bias is handled, to a degree, in the medical context with the ‘limits of agreement’approach proposed by Martin Bland and Douglas Altman. However, this technique also has its challenges. In this talk we review the drawbacks of existing comparison techniques and propose a likelihood-based framework for analyzing measurement system comparison studies, that overcomes the challenges of other techniques. We propose a simple metric, the probability of agreement, and a simple plot, the probability of agreement plot, which can be used to summarize the results of the comparison. We also propose an optimal study design that facilitates accurate estimation of the probability of agreement.
Keywords: measurement system; bias; repeatability; interchangeability; probability of agreement