Assessing the Performance of Emerging and Existing Continuous Monitoring Solutions under a Single-blind Controlled Testing Protocol

22 November 2024, Version 1
This content is a preprint and has not undergone peer review at the time of posting.

Abstract

Continuous Monitoring (CM) solutions have been identified as one method to address methane emissions at oil and gas (O&G) facilities, as these solutions may facilitate faster emission detection and repair than traditional survey methods. This study tested 13 CM solutions over 12 weeks using single-blind controlled testing. Controlled release rates ranged from 0.08 to 6.75 kg CH4/hr and lasted 18 minutes to 8 hours. Six solutions demonstrated 90% detection limits (DLs) within the range of controlled releases, 0.5 [0.3, 0.6] kg CH4/hr to 6.8 [5.9, 8.0] kg CH4/hr. Of the six solutions, 4 had False Positive (FP) rates of 7.8-14%, and four had False Negative rates (FN) of 8-34.1%. Compared with Ilonze et al., these results show retested solutions balancing method sensitivity and FP and FN rates. All scanning/imaging solutions had high localization (≥ 40%) precision and accuracy to the equipment unit. Eleven of 13 solutions were tested for quantification; three had a mean relative quantification error ranging from 33.4 [0.85, 66.0] % to 96.1 [63.4, 128.9] %, 95% CI for leaks between 0.1 – 1 kg CH4/hr. For larger emissions (> 1 kg CH4/hr), nine solutions had a mean relative quantification error ranging from 2.8 [-20.3, 25.9] % to 32.8 [17.2, 48.5] %, 95% CI. Relative to previous studies, uncertainties in quantification estimates decreased, as did FN and FP rates, with improved detection limits for 2 of the four retested solutions. These findings highlight that continuous, rigorous testing enhances solution performance, with notable improvements observed across multiple test programs using the same test protocol.

Keywords

Methane
Detection
Detection Limit
Quantification
Continuous Monitoring
Probability of Detection

Supplementary materials

Title
Description
Actions
Title
Supplementary Information for Assessing the Performance of Emerging and Existing Continuous Monitoring Solutions under a Single-blind Controlled Testing Protocol
Description
Additional information about the test facility and supplementary results are in PDF form. The final reports on each solution’s performance can be found on METEC’s ADED Results website.
Actions

Comments

Comments are not moderated before they are posted, but they can be removed by the site moderators if they are found to be in contravention of our Commenting Policy [opens in a new tab] - please read this policy before you post. Comments should be used for scholarly discussion of the content in question. You can find more information about how to use the commenting feature here [opens in a new tab] .
This site is protected by reCAPTCHA and the Google Privacy Policy [opens in a new tab] and Terms of Service [opens in a new tab] apply.
Comment number 2, James Maslanik: Dec 24, 2024, 02:27

Page 14 of the text tags solution C as "the solution with the largest 90% DL" (DL meaning detection level). It's not apparent why the authors chose to point out the 90% POD value for solution C as predicted by the solution's power curve fit (Figure 2), rather than, say, doing the same for solution R. For solution R, the predicted 90% POD based on it's power fit would be around 1350 kg/hr. Or why not consider the several solutions that had an estimated 90% POD of infinity as being solutions with the largest 90% DL? Perhaps it's because solution C's predicted 90% POD of 76.5 kg/hr isn't quite as ridiculous as Solution R's 1350 kg/hr (or predictions of infinity for other solutions) and therefore was easier to target as something to focus on in the manuscript.

Comment number 1, James Maslanik: Dec 20, 2024, 00:21

To extend on my comment referring to the last line on pg. 18 regarding Solution C's 90% POD results for 2024 vs. 2022, I would like to point out that the authors themselves say the following in Table 2's footnote for Solution C: "Solutions with no observable POD trend with emission rates or whose 90% DL is significantly outside the range of tested rates. These POD curves and DLs should be used with caution." On page 18, despite Solution C's POD curve having minimal POD trend, they violate their own caution by using the POD curve to project out to some unrealistic POD value and then using that value to make an unsupported assertion about Solution C's capabilities. As the authors themselves note, this is an inappropriate use of the data. Underlying this issue of flattening of the POD curve, the authors assume that release rate is the dominant factor in controlling the detection of probability, but there are other factors they do not consider. In fact, when digging deeper into the results, one finds that Solution C appears to be sensitive enough that release rate was not a substantial limitation on detection frequency. Instead, the factor affecting detection frequency, and thus the POD curve, was the number of releases per experiment. This again points to aspects of the ADED protocol that ought to be discussed or at least acknowledged by the authors.