详情

原文传递 Separation Management: Automation Reliability Meta-Analysis and Conflict Probe Reliability Analysis.

题名：	Separation Management: Automation Reliability Meta-Analysis and Conflict Probe Reliability Analysis.
作者：	Masalonis, A. J.; Messina, J.; Rein, J. R.; Willems, B.
关键词：	Accuracy; Air Traffic Control; Aircraft; Automation; Conflict; Human Factors; Meta-analysis; Modernization; Performance Evaluation; Separation Management; Signal Detection Theory; System Reliability
摘要：	The purpose of this study is to establish whether a valid performance criterion exists to determine the acceptability of the En Route Automation Modernization (ERAM) Conflict Probes conflict-detection accuracy and to evaluate observed accuracy against this criterion. The Conflict Probe can exhibit very high or very low accuracy depending on the analysis technique. It is necessary to establish both an empirically backed criterion for accuracy and the appropriate accuracy analyses and metrics. A meta-analysis was conducted on Human Factors automation reliability literature; an additional analysis was done on the results of probe reliability studies by the Federal Aviation Administration (FAA) Concept Analysis branch to derive various accuracy metrics. The results were compared to determine the acceptability of the conflict probe accuracy. The meta-analysis produced an estimated criterion of 65% correct responses for automation to improve performance, but this estimate is subject to a broad confidence interval due to variability in the source data from the literature. The probe performance exceeded the 65% value when giving credit for all correct rejections, but it fell short when not giving credit for correct rejections. Another metric, Positive Predictive Value (PPV, the percent of alerts that are valid), is operationally meaningful and its values demonstrated large accuracy improvements over baseline with the FAA Concept Analysis parametric adjustments, but a PPV cutoff criterion could not be established from the meta-analysis. The present results provide insight on several fronts, but operational input is essential to determine (1) a more justifiable air traffic control-specific accuracy criterion and (2) which aircraft encounters are appropriate to include in a test set for accuracy assessment. Automation responses should only increase the value of an accuracy metric to the extent that the responses add informational value for the controller. The present results w
报告类型：	科技报告