Electronic health record data is unable to effectively characterize measurement error from pulse oximetry: a simulation study

Abstract: Large data sets from electronic health records (EHR) have been used in journal articles to demonstrate race-based imprecision in pulse oximetry (SpO) measurements. These articles do not appear to recognize the impact of the variability of the SpO values with respect to time (“deviation time”). This manuscript seeks to demonstrate that due to this variability, EHR data should not be used to quantify SpO error. Using the MIMIC-IV Waveform dataset, SpO values are sampled from 198 patients admitted to an intensive care unit and used as reference samples. The error derived from the EHR data is simulated using a set of deviation times. The laboratory oxygen saturation measurements are also simulated such that the performance of three simulated pulse oximeter devices will produce an average root mean squared (A) error of 2%. An analysis is then undertaken to reproduce a medical device submission to a regulatory body by quantifying the mean error, the standard deviation of the error, and the A error. Bland-Altman plots were also generated with their Limits of Agreements. Each analysis was repeated to evaluate whether the measurement errors were affected by increasing the deviation time. All error values increased linearly with respect to the logarithm of the time deviation. At 10 min, the A error increased from a baseline of 2% to over 4%. EHR data cannot be reliably used to quantify SpO error. Caution should be used in interpreting prior manuscripts that rely on EHR data.