Direct Accuracy Performance Comparison of Cerebral Oximeter Devices

Highlighting data reported in: Bickler PE, Feiner JR, Eilers H, Rollins M. Performance of 5 cerebral oximeters during hypoxia in healthy volunteers. Proceedings of the 2011 Annual Meeting of the American Society of Anesthesiologists; Abstract LBT07.

Background Reported accuracy values of various cerebral oximeters are difficult to compare without identical patient cohorts and testing methodology.

Objective The purpose of this study is to measure the accuracy of commercially available cerebral oximeter monitors tested on identical subjects to eliminate selection bias and ensure uniform data analysis.

Methods Controlled oxygen desaturation sequences were performed on 23 healthy volunteer subjects of mixed gender and ethnicity. Values reported by four cerebral oxygenation monitors were compared against simultaneous invasive weighted CO-oximetry jugular bulb and arterial oxygen saturation reference measurements.

Results Precision for the cerebral oximeters tested ranged from 3.90% to 9.72% (1SD) and Arms accuracy from 4.26% to 9.69%.

Conclusions Performance among the tested monitors varied considerably. The FORE-SIGHT Oximeter demonstrated the greatest precision and accuracy.

Introduction

An imbalance between cerebral oxygen supply and demand can lead to cerebral hypoxic/ischemic injury and has been reported across a wide spectrum of disorders. Lower levels of cerebral oxygenation have been associated with increased cognitive dysfunction, a higher incidence of major organ morbidity, worse outcomes, and extended hospital length of stay.1 Difficulties in detecting cerebral ischemia as it occurs may cause missed opportunities to prevent or minimize permanent ischemic brain and other organ injury.2

Measurements of cerebral oxygenation are useful in assessing the balance between cerebral metabolic supply and demand. Direct invasive methods of measurement such as jugular venous oximetry and brain tissue oxygen tension are expensive and less than ideal. In contrast, cerebral oximetry with near infrared spectroscopy (NIRS) offers a noninvasive, continuous and easy to use measurement of cerebral tissue oxygen saturation and continues to gain acceptance as a crucial monitoring tool in critical care environments.3

Cerebral oximetry has been available as a monitoring device for over a decade. Recent refinements to NIRS technology have resulted in more reliable oximeters with increased accuracy performance.4 Currently, at least three manufacturers have FDA clearance to market NIRS based cerebral oximeters in the United States, but direct comparisons of accuracy among these monitors has been difficult. Some studies have been published where a single monitor brand has been compared to an invasive standard to determine accuracy.5,6 Individual studies, however, provide limited guidance for clinicians to compare the accuracy of devices due to the possibility of patient selection bias or variability in methodologies. Until now, only one published study compared two branded monitors against a mixed venous and arterial reference standard within the same patient cohort.4 While head-to-head comparisons in clinical settings have been reported,7 objective evaluation of the accuracy of the devices without an invasive standard is difficult.

To provide a direct comparison of accuracy performance among commercially available cerebral oximeter monitors without selection bias, this comparative study evaluated the precision of four commercially available cerebral oximeters on the same subjects. Results are also compared with previously published studies conducted with similar reference methods to determine consistency.

Devices

Simultaneous measurements from four commercially available near-infrared spectroscopy (NIRS) cerebral oximeters: FORE-SIGHT® (CAS Medical Systems, Branford, CT, USA), INVOS® 5100C (Covidien, Boulder, CO, USA), EQUANOX™ Model 7600 (Nonin Medical, Plymouth, MN, USA) with both EQUANOX Advance™ and EQUANOX Classic™ sensors, and NIRO®-200NX (Hamamatsu Photonics, Hamamatsu City, Japan) on adult volunteers. The reported cerebral oxygenation values (variously identified as SctO2, rSO2, and TOI) were compared against the commonly recognized invasive standard of weighted CO-oximetry jugular bulb and arterial oxygen saturation values during episodes of deliberate oxygen desaturation. All of these monitors have FDA clearance with the exception of the NIRO-200NX which has only CE Mark.

Methods

Healthy adult volunteers were enrolled in this IRB-approved volunteer study after obtaining written informed consent at the Hypoxia Research Laboratory at the University of California, San Francisco. Sensors from different manufacturers were randomly placed on alternating right or left sides of each subject’s forehead for each hypoxia run.a An internal jugular vein catheter was placed at the position
of the subject’s superior jugular bulb and an arterial catheter was placed in the subject’s radial artery. During the study period, allsubjects remained in a semi-Fowler’s position to ease breathing. Before affixing a mouthpiece and noseclip, and with subjects breathing room air, an initial blood sample was withdrawn from the two catheters.

Subjects then breathed gas mixtures through the mouthpiece and were instructed to breathe two to three times deeper and faster than normal to speed alveolar gas equilibration. Inspired CO2 was adjusted to maintain an end-tidal CO2 value of 40 ±2 mmHg. Inspired oxygen was adjusted to target SpO2 plateaus in approximately equal increments from 98% down to approximately 68%. Subjects were then given pure oxygen to return to normal state.

Blood samples (1.5 -2.0 ml) from the jugular bulb and radial artery catheters were simultaneously withdrawn during the target plateaus
into heparinized syringes. The cerebral oximetry values reported by each monitor were recorded synchronously on a laptop every two seconds and averaged in ten-second moving windows. Readings from each monitor were identified at the moment of each blood draw. Each blood sample was analyzed for functional oximetry in an OSM3 Hemoximeter (Radiometer Medical ApS, Bronshoj, Denmark) for either radial (SaO2) or jugular (SjvO2) data. The subjects underwent three induced
hypoxia sessions with 7 to 8 blood samples collected each run.
Weighted CO-oximeter reference values were calculated based on the manufacturer’s claims:

For FORE-SIGHT, EQUANOX, and NIRO:
CX(70:30) = [0.70 x SjvO2] + [0.30 x SaO2]
For INVOS:
CX(75:25) = [0.75 x SjvO2] + [0.25 x SaO2]

Composite precision (±1SD), bias (mean of [CX – oximeter] error value), and Arms (root-mean-square difference between the displayed cerebral oximeter readings and CX)8 values were determined for each monitor including all subjects. Data from this study were then compared to previously published studies based upon similar testing methods on adult volunteers with monitor values measured against weighted CO-oximeter values.

Results

Accuracy data from 23 patients (14 Male / 9 Female; 13 White / 3 Hispanic White / 3 Asian / 2 Black / 2 Multiracial; 20-40 years; 54.5 - 97.7 kg) comprising 175-180 data points for each monitor are reported (Table 1, Figures 1-5). Data from an additional two subjects were excluded from the analysis: on one subject all monitors failed to provide a value and in another both the INVOS monitor and EQUANOX monitor with Advance sensors failed to read continuously. In those events where a monitor failed to provide a value, corrective actions such as checking cable connections, replacing hardware, or turning off the opposing sensor were taken, but were unsuccessful in obtaining a measurement from the monitor.

Reference CX values were calculated based upon blood draws when the subjects were breathing room air prior to the hypoxia run and were separated into decade increments and compared with the corresponding cerebral oximeter values (Figure 6-10).

Discussion

This study represents the first detailed comparative performance analysis of cerebral oximeters using an invasive reference standard and a common cohort, testing methodology, and data analysis. Large differences were observed in the precision of commercially available monitors in healthy adult volunteers undergoing induced hypoxia events with precisions ranging from ±3.90% to ±9.72% (1SD). The FORE-SIGHT Oximeter monitor measured cerebral oxygenation with the greatest precision (3.90%) and Arms accuracy (4.26%). Data from this study is consistent with several earlier validation studies. Two reports of the FORE-SIGHT Oximeter demonstrated similar precisions of 3.70%5 and 3.12%4, respectively. Furthermore, one of those studies also included a comparison to an INVOS monitor and measured an INVOS precision of 9.62%4, consistent with the precision of 9.69% reported in this study. The current results, however, do not agree with another INVOS reported volunteer hypoxia study showing a precision of 5.0%.9

The EQUANOX Classic sensor has FDA clearance as a trending sensor, but absolute accuracy data published earlier indicated a Arms of 8.3%10, similar to the Arms of 8.47% determined in this study. In contrast with a previous study where EQUANOX Advance sensors reported an Arms value of 4.10%6, this investigation measured an Arms accuracy for those sensors of 6.86%. No standard deviation was
available in the previous study.

The variations in measured accuracy across investigations highlight the difficulty of comparing performance across studies employing different subjects and methodology. This study controlled experimental and analysis conditions to ensure traceability of truly comparative data.

Room air measurement values are important because they often are used as a baseline for determining treatment interventions based upon decrements or trends in that value.11 The FORE-SIGHT Oximeter reported all room air values (Figure 6) in the expected range for healthy adults of 60% to 80%, with a distribution mirroring the reference values. Other manufacturer’s oximeters reported a wide variety of subject room air saturation values ranging from less than 40% to greater than 80%. Large variability in baseline room air values may undermine treatment confidence when intervention at fixed oxygenation levels (e.g., 60%1) are prescribed or when the values are used to stratify risk for cardiac patients prior to on-pump surgery.11

Clinical Implications

Clinicians should be aware of the limitations of certain cerebral oximeters as accuracy varies considerably among brands. Advancements
in NIRS technology as found in the FORE-SIGHT Oximeter provide improved accuracy affording clinicians increased confidence to intervene appropriately in the care of their patients.

Table 1: Precision, Bias, and Arms of different manufacturer’s device. Bias is presented as [Reference CX - Measured Value].

Fig 1: Scatter Plot Graph Comparing
FORE-SIGHT with Reference CX

Fig 2: Scatter Plot Graph Comparing
INVOS with Reference CX

Fig 3: Scatter Plot Graph Comparing
NIRO-200NX with Reference CX

Fig 4: Scatter Plot Graph Comparing
EQUANOX Classic with Reference CX

Fig 5: Scatter Plot Graph Comparing
EQUANOX Advance with Reference CX

Fig 6: FORE-SIGHT room air values
compared to Reference CX values

Fig 7: INVOS room air values
compared to Reference CX values

Fig 8: NIRO-200NX room air values
compared to Reference CX values

Fig 9: EQUANOX Classic room air values
compared to Reference CX values

Fig 10: EQUANOX Advance room air values
compared to Reference CX values

References

1. Fischer GW, Lin HM, Krol M, Galati MF, Di Luozzo G, Griepp RB, Reich DL. Noninvasive
cerebral oxygenation may predict outcome in patients undergoing aortic arch surgery. J Thorac Cardiovasc Surg 2011;141(3):815-21.
2. Highton D, Elwell C, Smith M. Noninvasive cerebral oximetry: is there light at the end of the tunnel? Curr Opin Anaesthesiol 2010;23(5):576-81.
3. Fischer GW. Recent advances in application of cerebral oximetry in adult cardiovascular
surgery. Semin Cardiothorac Vasc Anesth 2008;12(1):60-9.
4. MacLeod DB, Ikeda K, Vacchiano C. Absolute and trending accuracy of FORE-SIGHT and INVOS cerebral oximeters in healthy volunteers. Proceedings of the 2009 Annual
Meeting of the American Society of Anesthesiologists; Abstract A298.
5. MacLeod DB, Ikeda K, Keifer JC, Moretti J, Ames W. Validation of the CAS adult cerebral
oximeter during hypoxia in healthy volunteers. Anesth Analg 2006;102(2S):S162.
6. MacLeod D, Ikeda K. Nonin Equanox 8004CA Advance Cerebral Oximeter Sensor Provides Valid Assessment of True Tissue Oxygenation. Anesth Analg 2011;112(5):S-235.
7. Yoshitani K, Kawaguchi M, Tatsumi K, Kitaguchi K, Furuya H. A comparison of the INVOS 4100 and the NIRO 300 near-infrared spectrophotometers. Anesth Analg 2002;94(3):586-90.
8. Batchelder PB, Raley DM. Maximizing the laboratory setting for testing devices and understanding statistical output in pulse oximetry. Anesth Analg 2007;105(6 Suppl):S85-94.
9. K051274. Somanetics INVOS 5100B FDA 510(K) premarket notification.
10. Nonin Medical, Inc. Model 7600 regional oximeter with Equanox™ technology with Bluetooth® wireless technology for display software revision 8 and higher. Operator’s
Manual 2009: 46.
11. Heringlake M, Garbers C, Kabler JH, Anderson I, Heinze H, Schon J, Berger KU, Dibbelt
L, Sievers HH, Hanke T. Preoperative cerebral oxygen saturation and clinical outcomes in cardiac surgery. Anesthesiology 2011;114(1):58-69.