9.5 Measurement reliability

Diagnostic tests must be accurate and precise. Accuracy represents how closely the test provides a value which is close to the true value for the patient. Precision tells us how repeatable a test result is. That is, if we repeat the test, under the same conditions using the same samples, we should get similar results.

For example, in the blood test above, how do we know that the value obtained for the measurement of sodium concentration in serum is actually the concentration of sodium in the patient’s serum? And if we were to take blood tests multiple times from the same patient how close would the measurements for serum concentration be to each other?

For determining accuracy, how could we know what the true value is? One way this is done in biomedical diagnostics is to compare the results of the test with results obtained using another method (or methods). Quite often these other methods are ‘gold standard’ techniques which are known to be highly accurate and precise for a certain measurement but are not widely used because they are less convenient or more expensive. The other alternative to estimate the accuracy of a test is to use a known standard in the measurement. For example, in a blood test where the concentration of a particular analyte is being measured then a measurement could also be made of serum which has had a known amount of the analyte added. In this way a standard has been produced to provide a theoretically true value.

How accurate a measurement is can be described by the percentage error of the measurement. This is given by the formula:

Percentage error (%) = × 100

You will notice from this formula that it is possible to get a negative error depending on the magnitude of the measured value. This can be useful to know but quite often percentage error is calculated and reported using the absolute value:

Percentage error (%) = × 100

Also, it is worth noting that you will occasionally see the same information expressed as relative percentage accuracy:

Relative accuracy (%) = × 100

Precision provides us with an indication of how repeatable or reliable our measurement is. If we only take one measurement, the reliability or certainty surrounding that measurement is zero – we have no idea how a precise the measurement is. So, in order to determine precision, we need to take multiple measurements and then need a way of expressing how close the measured values are to each other.

This is known as variability and there are several ways of expressing variability (see the earlier section on describing variations in data). The most useful measure of variability in most cases is the standard deviation. So, the magnitude of the standard deviation for a set of repeated measures provides an indication of the precision of the measurement. The higher the standard deviation the greater the variability of the data and therefore the less precise the measurement. Once we have an estimate of the precision of a particular measurement, we have an estimate of how reliable our data is.

What if we want to compare the precision of two different datasets, perhaps from two different measurement methods? It is not reasonable to simply compare standard deviations as the magnitude of the standard deviation changes with the mean. (Recall that standard deviation is calculated by determining the square root of the variance which in turn is calculated by determining the average of the squared differences between the measurements from the mean.) Therefore, we need a way to normalise standard deviation against the mean to be able to make comparisons. The way to do this is to use the coefficient of variation.

The coefficient of variation is calculated using the following formula:

cv = × 100

where

cv = coefficient of variation

s = sample standard deviation

x̅ = mean.

With the coefficient of variation you can compare the precision of two diagnostic tests or compare the precision of the high and low ranges for a particular diagnostic test.

For example, there are two methods for determining glucose concentrations in plasma. One uses the enzyme hexokinase and the other uses the glucose oxidase.

Which of these methods is more precise?

Hexokinase:

Glucose oxidase:

The glucose oxidase assay is more precise.