Category: Basic QC Concepts | Estimated reading time: 9 minutes
Rejections That Turn Out to Be Non-Issues
That day, the QC was rejected. The instrument was checked, reagents were replaced, calibration was repeated, and it turned out nothing was wrong. The QC was repeated, and then it passed. Two weeks later, it happened again.
If this situation happens often in your laboratory, there are two possibilities. It is either random fluctuation that is indeed unavoidable, or there is something wrong with the numbers that make up the chart (TV or SD). The article "QC Always Green" discusses the opposite side, which is QC that is too quiet. This article discusses QC that is too noisy.
What Is a False Rejection?
A false rejection is a run that is rejected even though the process is actually in control. The metric is called Pfr (probability of false rejection): the chance that QC declares "trouble" when there is no problem. Its partner is Ped (probability of detecting real error), which is discussed in the "Monte Carlo Simulation" article.
Both are in a tug-of-war. More sensitive rules catch more real errors, but they also accuse incorrectly more often. Westgard designed the multirule system specifically to balance the two.
Why False Alarms Cannot Be Completely Eliminated
In a truly controlled process with a normal distribution:
- About 4.5% of points fall outside ±2SD.
- About 0.27% of points fall outside ±3SD.
That is not a sign of a problem, but rather the nature of random variation. With two levels of control per run, the chance of at least one point passing ±2SD is about 1 in 11 runs. That is why 1₂s only has a warning status (see the "1₂s Warning" article).
False alarms also accumulate. Every additional rule, every additional control level, and every additional run provides one more opportunity for a false alarm to appear. The probability per run is small, but in a month, the total can be significant.
How Often Is It Normal?
From a simple simulation (two control levels per run, independent normal data, accurate TV and SD), the percentage of runs flagged as rejected without any real problems is approximately:
- 1₃s only: about 0.5% per run
- 1₃s + 2₂s: about 0.8% per run
- Multirule (1₃s, 2₂s, R₄s, 4₁s, 10x): about 1.9% per run
If you perform one run per day (30 runs per month), a multirule will produce an average of about 0.6 false rejections per month, or once every one to two months. So, an occasional rejection without an identifiable cause is normal and does not need to be suspected.
What is not normal is a frequency far above that. Often-cited design guidelines target a Pfr of no more than about 5% per run, and a multirule with the correct numbers is far below that.
Four Causes of Excessive False Alarms
1. SD is too narrow
This is the most common cause. An SD calculated from one short session only captures short-term variation, whereas inter-day variation (calibration, operators, reagents) has not been factored in. Westgard QC notes that a period that is too short results in an SD that is too small.
The impact from the same simulation (percentage of runs rejected without real problems):
- SD used is 1.25 times smaller than the actual SD: about 4% per run (1₃s+2₂s) and about 8% per run (multirule). That is equivalent to 1 to 2 false rejections per month, or 2 to 3 for multirule.
- SD used is 1.5 times smaller: about 11% per run (1₃s+2₂s) and about 18% per run (multirule), or 3 to 5 false rejections per month.
2. TV is off
If the target mean is not the same as the process mean, points tend to stay on one side, causing 4₁s and 10x to be triggered. A mean that is off by just 0.5 SD increases multirule false rejections from about 1.9% to about 7% per run. This often happens after a lot change (see the "Changing Control Material Lot" article) or when the mean is taken from the manufacturer's insert.
3. Rule combination is too sensitive for the method
The more rules used, the larger the Pfr. Methods with high Sigma actually do not need such a complete set of rules (see the "Sigma Metrics and Westgard Multirule" article). Using full multirules on a very stable method increases false alarms without adding meaningful protection. The calculator in the "Monte Carlo Simulation" article helps compare rule combinations.
4. Statistical assumptions are not met
All the numbers above assume data are normally distributed and independent. Real QC data can deviate: they may be rounded, correlated from day to day, or have heavier tails. In such conditions, false alarms are generally more frequent than theoretical calculations suggest.
Note: Rejections due to problematic control material (contaminated vial, degraded, improper mixing) are not statistically false alarms, as the control is indeed deviating. However, for patient results, the cause is not an analytical error. It still needs to be investigated and documented; only the conclusion is different.
How to Distinguish Random Fluctuation from Real Problems
- Calculate the frequency. What percentage of runs were rejected in the last 2 to 3 months, and how many had an identified cause? Rejections without a cause that are well above the reasonable figures above are a signal that the numbers are the problem, not the process.
- Look at the distribution of points. In a stable process with the correct SD, about 68% of points are within ±1SD and about 95% within ±2SD. If there are many points outside ±2SD without a cause, suspect that the SD is too narrow.
- Look at the direction. Rejections that tend toward one side of the mean point to the TV. Rejections scattered to both sides point to random variation or the SD.
- Check the history of the numbers. When were the TV and SD last established, from how much data, and from what period of time?
- Check the control material. Does the rejection occur at one level only, or in a specific vial?
What Should Not Be Done
- Widening the SD so that rejections stop. This is "chasing the data" and makes QC blind (see the "Target Value & SD" article).
- Repeating QC until it passes without investigation. Repeating once after checking for the cause is not automatically wrong. What is dangerous is repeating many times until it passes, then assuming the problem is solved (see the "QC Reject" article).
- Turning off rules that are felt to be annoying without data-based justification and approval.
What Should Be Done
- Improve the SD based on cumulative data from a stable process, with documented reasoning and approval.
- Improve the TV with the correct lot or period data.
- Choose a combination of rules based on the Sigma metric and test with simulation.
- Record all rejections along with their causes, including those with "no cause found." These records are what allow for the frequency analysis above.
Summary
- False rejection is a run that is rejected even though the process is in control. Pfr cannot be zero.
- With the right numbers, a multirule produces about 2% false rejections per run, or about once every one to two months for one run per day.
- Rejections without a cause that are much more frequent than that usually point to an SD that is too narrow, a TV that is off, or a combination of rules that is too sensitive.
- Do not widen the SD and do not repeat until it passes. Fix the numbers from stable data.
- Record every rejection so its frequency can be analyzed.
Closing
QC that never rejects is suspicious, and QC that rejects too often is as well. Both often stem from the same root: two numbers, TV and SD. Understanding how many false alarms are reasonable helps the laboratory professional distinguish "this is just a coincidence" from "this is something that needs to be fixed," without panic and without normalizing the problem.