Energy

Performance of continuous emission monitoring solutions under single-blind controlled testing protocol.

Authors

  • Clay Bell Energy Institute, Colorado State University & BPX Energy ,
  • Chiemezie Ilonze Department of Mechanical Engineering, Colorado State University ,
  • Aidan Duggan Energy Institute, Colorado State University ,
  • Daniel Zimmerle Energy Institute, Colorado State University

Abstract

Continuous emission monitoring (CM) solutions promise to accelerate fugitive methane emission mitigation in natural gas infrastructure by detecting large sources sooner than they would be detected in traditional leak surveys. Quantification by CM solutions has also been proposed as the foundation of measurement-based inventories. Herein we present the first published results of CM solutions tested using a consensus protocol. Single-blind testing was performed with controlled releases at a dedicated facility, using test conditions that were challenging but less complex than typical field conditions. Eleven solutions were tested, including point sensor networks and scanning/imaging solutions. Results indicate detection limits (emission rate with 90% probability of detection) of 3-30 kg CH4/h, with 6 of 11 solutions in the range of 3-6 kg CH4/h, with high uncertainty. Of the 11 solutions tested, 6 provided emission rates estimates and for emission rate of 0.1-1 kg/h, the solutions' mean relative errors ranged from -44% to +586% with single estimates between -90% and +300% (all uncertainties stated as empirical 95% confidence intervals) with most solutions' upper uncertainty exceeding +900%. Solutions' mean relative error for emission rates >1 kg/h is -40% to +93% with two solutions achieving mean relative errors below 20%. In this range, all solutions exhibited single-estimate relative error from below -80% to values above +240%. When test results were simulated using emissions data from a portfolio of facilities, two solutions estimated emissions to within ±20%, while 4 estimated emissions at least double the true emission rate. The large variability in performance between CM solutions, coupled with highly uncertain detection, detection limit, and quantification results, indicate that the performance of individual CM solutions should be well understood before relying on results for internal emissions mitigation programs or regulatory reporting. Additionally, given the rapid development of these solutions, additional implementation standards and regular testing will also be necessary.

Content

Thumbnail image of EST_Manuscript.pdf

Supplementary material

Thumbnail image of SI.pdf
Performance of continuous emission monitoring solutions under single-blind controlled testing protocol - Supplementary Information
The document contains relevant additional and supplementary information that was not captured nor discussed in the main paper.
Thumbnail image of DataHeaderDescriptions.xlsx
Data column header definition
This document describes the meaning of the information captured in each column of the data. This file is applicable in the interpretation of the columns of each solution's classified data (.i.e. Classified data for solution A, Classified data for solution B, etc.).
Thumbnail image of Solution A Report.pdf
A comprehensive report on the performance of solution A
The report summarizes the demography of experiments performed to test solution A, the meteorological conditions during the experiments, and the performance of the solution. To assess performance, metrics such as probability of detection, emission source localization, and emissions quantification established by the testing protocol were evaluated provided the data needed for the evaluation was reported by the solution.
Thumbnail image of Solution A Data.csv
Classified data for solution A
This data contains both controlled releases and detection reports by solution A paired to form various classifications. Each row of the data represents either a true positive, false positive, or false negative detection classification. A reported detection successfully paired with a controlled release was classified as a true positive detection. A reported detection which remained unpaired was classified as a false positive detection, while a controlled release which remained unpaired was classified as a false negative. Controlled releases and detection reports excluded from the analysis for varying reasons make up the remaining rows in the data.
Thumbnail image of Solution B Report.pdf
A comprehensive report on the performance of solution B
The report summarizes the demography of experiments performed to test solution B, the meteorological conditions during the experiments, and the performance of the solution. To assess performance, metrics such as probability of detection, emission source localization, and emissions quantification established by the testing protocol were evaluated provided the data needed for the evaluation was reported by the solution.
Thumbnail image of Solution B Data.csv
Classified data for solution B
This data contains both controlled releases and detection reports by solution B paired to form various classifications. Each row of the data represents either a true positive, false positive, or false negative detection classification. A reported detection successfully paired with a controlled release was classified as a true positive detection. A reported detection which remained unpaired was classified as a false positive detection, while a controlled release which remained unpaired was classified as a false negative. Controlled releases and detection reports excluded from the analysis for varying reasons make up the remaining rows in the data.
Thumbnail image of Solution C Report.pdf
A comprehensive report on the performance of solution C
The report summarizes the demography of experiments performed to test solution C, the meteorological conditions during the experiments, and the performance of the solution. To assess performance, metrics such as probability of detection, emission source localization, and emissions quantification established by the testing protocol were evaluated provided the data needed for the evaluation was reported by the solution.
Thumbnail image of Solution C Data.csv
Classified data for solution C
This data contains both controlled releases and detection reports by solution C paired to form various classifications. Each row of the data represents either a true positive, false positive, or false negative detection classification. A reported detection successfully paired with a controlled release was classified as a true positive detection. A reported detection which remained unpaired was classified as a false positive detection, while a controlled release which remained unpaired was classified as a false negative. Controlled releases and detection reports excluded from the analysis for varying reasons make up the remaining rows in the data.
Thumbnail image of Solution D Report.pdf
A comprehensive report on the performance of solution D
The report summarizes the demography of experiments performed to test solution D the meteorological conditions during the experiments, and the performance of the solution. To assess performance, metrics such as probability of detection, emission source localization, and emissions quantification established by the testing protocol were evaluated provided the data needed for the evaluation was reported by the solution.
Thumbnail image of Solution D Data.csv
Classified data for solution D
This data contains both controlled releases and detection reports by solution D paired to form various classifications. Each row of the data represents either a true positive, false positive, or false negative detection classification. A reported detection successfully paired with a controlled release was classified as a true positive detection. A reported detection which remained unpaired was classified as a false positive detection, while a controlled release which remained unpaired was classified as a false negative. Controlled releases and detection reports excluded from the analysis for varying reasons make up the remaining rows in the data.
Thumbnail image of Solution E Report.pdf
A comprehensive report on the performance of solution E
The report summarizes the demography of experiments performed to test solution E, the meteorological conditions during the experiments, and the performance of the solution. To assess performance, metrics such as probability of detection, emission source localization, and emissions quantification established by the testing protocol were evaluated provided the data needed for the evaluation was reported by the solution.
Thumbnail image of Solution E Data.csv
Classified data for solution E
This data contains both controlled releases and detection reports by solution E paired to form various classifications. Each row of the data represents either a true positive, false positive, or false negative detection classification. A reported detection successfully paired with a controlled release was classified as a true positive detection. A reported detection which remained unpaired was classified as a false positive detection, while a controlled release which remained unpaired was classified as a false negative. Controlled releases and detection reports excluded from the analysis for varying reasons make up the remaining rows in the data.
Thumbnail image of Solution F Report.pdf
A comprehensive report on the performance of solution F
The report summarizes the demography of experiments performed to test solution F, the meteorological conditions during the experiments, and the performance of the solution. To assess performance, metrics such as probability of detection, emission source localization, and emissions quantification established by the testing protocol were evaluated provided the data needed for the evaluation was reported by the solution.
Thumbnail image of Solution F Data.csv
Classified data for solution F
This data contains both controlled releases and detection reports by solution F paired to form various classifications. Each row of the data represents either a true positive, false positive, or false negative detection classification. A reported detection successfully paired with a controlled release was classified as a true positive detection. A reported detection which remained unpaired was classified as a false positive detection, while a controlled release which remained unpaired was classified as a false negative. Controlled releases and detection reports excluded from the analysis for varying reasons make up the remaining rows in the data.
Thumbnail image of Solution G Report.pdf
A comprehensive report on the performance of solution G
The report summarizes the demography of experiments performed to test solution G, the meteorological conditions during the experiments, and the performance of the solution. To assess performance, metrics such as probability of detection, emission source localization, and emissions quantification established by the testing protocol were evaluated provided the data needed for the evaluation was reported by the solution.
Thumbnail image of Solution G Data.csv
Classified data for solution G
This data contains both controlled releases and detection reports by solution G paired to form various classifications. Each row of the data represents either a true positive, false positive, or false negative detection classification. A reported detection successfully paired with a controlled release was classified as a true positive detection. A reported detection which remained unpaired was classified as a false positive detection, while a controlled release which remained unpaired was classified as a false negative. Controlled releases and detection reports excluded from the analysis for varying reasons make up the remaining rows in the data.
Thumbnail image of Solution H Report.pdf
A comprehensive report on the performance of solution H
The report summarizes the demography of experiments performed to test solution H, the meteorological conditions during the experiments, and the performance of the solution. To assess performance, metrics such as probability of detection, emission source localization, and emissions quantification established by the testing protocol were evaluated provided the data needed for the evaluation was reported by the solution.
Thumbnail image of Solution H Data.csv
Classified data for solution H
This data contains both controlled releases and detection reports by solution H paired to form various classifications. Each row of the data represents either a true positive, false positive, or false negative detection classification. A reported detection successfully paired with a controlled release was classified as a true positive detection. A reported detection which remained unpaired was classified as a false positive detection, while a controlled release which remained unpaired was classified as a false negative. Controlled releases and detection reports excluded from the analysis for varying reasons make up the remaining rows in the data.
Thumbnail image of Solution I Report.pdf
A comprehensive report on the performance of solution I
The report summarizes the demography of experiments performed to test solution I, the meteorological conditions during the experiments, and the performance of the solution. To assess performance, metrics such as probability of detection, emission source localization, and emissions quantification established by the testing protocol were evaluated provided the data needed for the evaluation was reported by the solution.
Thumbnail image of Solution I Data.csv
Classified data for solution I
This data contains both controlled releases and detection reports by solution I paired to form various classifications. Each row of the data represents either a true positive, false positive, or false negative detection classification. A reported detection successfully paired with a controlled release was classified as a true positive detection. A reported detection which remained unpaired was classified as a false positive detection, while a controlled release which remained unpaired was classified as a false negative. Controlled releases and detection reports excluded from the analysis for varying reasons make up the remaining rows in the data.
Thumbnail image of Solution J Report.pdf
A comprehensive report on the performance of solution J
The report summarizes the demography of experiments performed to test solution J, the meteorological conditions during the experiments, and the performance of the solution. To assess performance, metrics such as probability of detection, emission source localization, and emissions quantification established by the testing protocol were evaluated provided the data needed for the evaluation was reported by the solution.
Thumbnail image of Solution J Data.csv
Classified data for solution J
This data contains both controlled releases and detection reports by solution J paired to form various classifications. Each row of the data represents either a true positive, false positive, or false negative detection classification. A reported detection successfully paired with a controlled release was classified as a true positive detection. A reported detection which remained unpaired was classified as a false positive detection, while a controlled release which remained unpaired was classified as a false negative. Controlled releases and detection reports excluded from the analysis for varying reasons make up the remaining rows in the data.
Thumbnail image of Solution K Report.pdf
A comprehensive report on the performance of solution K
The report summarizes the demography of experiments performed to test solution K, the meteorological conditions during the experiments, and the performance of the solution. To assess performance, metrics such as probability of detection, emission source localization, and emissions quantification established by the testing protocol were evaluated provided the data needed for the evaluation was reported by the solution.
Thumbnail image of Solution K Data.csv
Classified data for solution K
This data contains both controlled releases and detection reports by solution K paired to form various classifications. Each row of the data represents either a true positive, false positive, or false negative detection classification. A reported detection successfully paired with a controlled release was classified as a true positive detection. A reported detection which remained unpaired was classified as a false positive detection, while a controlled release which remained unpaired was classified as a false negative. Controlled releases and detection reports excluded from the analysis for varying reasons make up the remaining rows in the data.