Limits...
Using random forests to diagnose aviation turbulence.

Williams JK - Mach Learn (2013)

Bottom Line: This paper describes a methodology for fusing data from diverse sources and producing a real-time diagnosis of turbulence associated with thunderstorms, a significant cause of weather delays and turbulence encounters that is not well-addressed by current turbulence forecasts.It is evaluated on an independent test set using several performance metrics including receiver operating characteristic curves, which are used for FAA turbulence product evaluations prior to their deployment.A prototype implementation fuses data from Doppler radar, geostationary satellites, a lightning detection network and a numerical weather prediction model to produce deterministic and probabilistic turbulence assessments suitable for use by air traffic managers, dispatchers and pilots.

View Article: PubMed Central - PubMed

Affiliation: Research Applications Laboratory, National Center for Atmospheric Research, P.O. Box 3000, Boulder, CO USA.

ABSTRACT

Atmospheric turbulence poses a significant hazard to aviation, with severe encounters costing airlines millions of dollars per year in compensation, aircraft damage, and delays due to required post-event inspections and repairs. Moreover, attempts to avoid turbulent airspace cause flight delays and en route deviations that increase air traffic controller workload, disrupt schedules of air crews and passengers and use extra fuel. For these reasons, the Federal Aviation Administration and the National Aeronautics and Space Administration have funded the development of automated turbulence detection, diagnosis and forecasting products. This paper describes a methodology for fusing data from diverse sources and producing a real-time diagnosis of turbulence associated with thunderstorms, a significant cause of weather delays and turbulence encounters that is not well-addressed by current turbulence forecasts. The data fusion algorithm is trained using a retrospective dataset that includes objective turbulence reports from commercial aircraft and collocated predictor data. It is evaluated on an independent test set using several performance metrics including receiver operating characteristic curves, which are used for FAA turbulence product evaluations prior to their deployment. A prototype implementation fuses data from Doppler radar, geostationary satellites, a lightning detection network and a numerical weather prediction model to produce deterministic and probabilistic turbulence assessments suitable for use by air traffic managers, dispatchers and pilots. The algorithm is scheduled to be operationally implemented at the National Weather Service's Aviation Weather Center in 2014.

No MeSH data available.


Related in: MedlinePlus

(Left) ROC curves from the 32 DAL upper-level cross-validation experiments for the RF (blue), GTG 3 prototype (green), and distance to storms as indicated by NSSL echo top>10,000 ft (magenta). (Right) Votes to probability calibration curve for the DAL upper-level RF (Color figure online)
© Copyright Policy - OpenAccess
Related In: Results  -  Collection


getmorefigures.php?uid=PMC4627188&req=5

Fig1: (Left) ROC curves from the 32 DAL upper-level cross-validation experiments for the RF (blue), GTG 3 prototype (green), and distance to storms as indicated by NSSL echo top>10,000 ft (magenta). (Right) Votes to probability calibration curve for the DAL upper-level RF (Color figure online)

Mentions: The variable selection process ultimately resulted in identification of 39 predictors that contributed significantly to one or more of the upper or lower-level DAL or UAL datasets. These were used to perform 32 cross-validation experiments, training RFs on samples from even Julian days and testing them on samples from odd Julian days and vice-versa, using a 90/10 split that sensitivity studies found ideal for 200 trees. For comparison, identical cross-validation experiments were performed using k-nearest neighbors (KNN) with k=100 and logistic regression (LR); the skill of the GTG 3 1-hr forecast and “storm distance” were also evaluated. Storm distance was defined as the distance to NSSL echo tops exceeding 10,000 ft, i.e., the top predictor in the RF variable selection (Table 2) and the best of several observation-derived fields that were evaluated. Table 3 shows the skill statistics including AUC, max CSI, and max TSS for each of these. Figure 1 (left) shows the ROC curves for all 32 cross-validation experiments. The RF ROC curves are quite stable, but those for GTG 3 and storm distance vary based on whether the testing dataset is from odd or even Julian days, yielding two distinct sets of curves. The ROC curves and statistics show that the RF provides better performance in diagnosing CIT than the other methods. Note that the prototype GTG 3 was tuned using PIREPs and EDR data collected over 2010 and 2011, so this is not an unfair comparison. Also, although the variable selection process utilized both training and testing sets, the danger of overtraining is minimal because numerous selection experiments were aggregated, the number of selected predictors is relatively large, and the cross-validation experiments were performed independently. Fig. 1


Using random forests to diagnose aviation turbulence.

Williams JK - Mach Learn (2013)

(Left) ROC curves from the 32 DAL upper-level cross-validation experiments for the RF (blue), GTG 3 prototype (green), and distance to storms as indicated by NSSL echo top>10,000 ft (magenta). (Right) Votes to probability calibration curve for the DAL upper-level RF (Color figure online)
© Copyright Policy - OpenAccess
Related In: Results  -  Collection

Show All Figures
getmorefigures.php?uid=PMC4627188&req=5

Fig1: (Left) ROC curves from the 32 DAL upper-level cross-validation experiments for the RF (blue), GTG 3 prototype (green), and distance to storms as indicated by NSSL echo top>10,000 ft (magenta). (Right) Votes to probability calibration curve for the DAL upper-level RF (Color figure online)
Mentions: The variable selection process ultimately resulted in identification of 39 predictors that contributed significantly to one or more of the upper or lower-level DAL or UAL datasets. These were used to perform 32 cross-validation experiments, training RFs on samples from even Julian days and testing them on samples from odd Julian days and vice-versa, using a 90/10 split that sensitivity studies found ideal for 200 trees. For comparison, identical cross-validation experiments were performed using k-nearest neighbors (KNN) with k=100 and logistic regression (LR); the skill of the GTG 3 1-hr forecast and “storm distance” were also evaluated. Storm distance was defined as the distance to NSSL echo tops exceeding 10,000 ft, i.e., the top predictor in the RF variable selection (Table 2) and the best of several observation-derived fields that were evaluated. Table 3 shows the skill statistics including AUC, max CSI, and max TSS for each of these. Figure 1 (left) shows the ROC curves for all 32 cross-validation experiments. The RF ROC curves are quite stable, but those for GTG 3 and storm distance vary based on whether the testing dataset is from odd or even Julian days, yielding two distinct sets of curves. The ROC curves and statistics show that the RF provides better performance in diagnosing CIT than the other methods. Note that the prototype GTG 3 was tuned using PIREPs and EDR data collected over 2010 and 2011, so this is not an unfair comparison. Also, although the variable selection process utilized both training and testing sets, the danger of overtraining is minimal because numerous selection experiments were aggregated, the number of selected predictors is relatively large, and the cross-validation experiments were performed independently. Fig. 1

Bottom Line: This paper describes a methodology for fusing data from diverse sources and producing a real-time diagnosis of turbulence associated with thunderstorms, a significant cause of weather delays and turbulence encounters that is not well-addressed by current turbulence forecasts.It is evaluated on an independent test set using several performance metrics including receiver operating characteristic curves, which are used for FAA turbulence product evaluations prior to their deployment.A prototype implementation fuses data from Doppler radar, geostationary satellites, a lightning detection network and a numerical weather prediction model to produce deterministic and probabilistic turbulence assessments suitable for use by air traffic managers, dispatchers and pilots.

View Article: PubMed Central - PubMed

Affiliation: Research Applications Laboratory, National Center for Atmospheric Research, P.O. Box 3000, Boulder, CO USA.

ABSTRACT

Atmospheric turbulence poses a significant hazard to aviation, with severe encounters costing airlines millions of dollars per year in compensation, aircraft damage, and delays due to required post-event inspections and repairs. Moreover, attempts to avoid turbulent airspace cause flight delays and en route deviations that increase air traffic controller workload, disrupt schedules of air crews and passengers and use extra fuel. For these reasons, the Federal Aviation Administration and the National Aeronautics and Space Administration have funded the development of automated turbulence detection, diagnosis and forecasting products. This paper describes a methodology for fusing data from diverse sources and producing a real-time diagnosis of turbulence associated with thunderstorms, a significant cause of weather delays and turbulence encounters that is not well-addressed by current turbulence forecasts. The data fusion algorithm is trained using a retrospective dataset that includes objective turbulence reports from commercial aircraft and collocated predictor data. It is evaluated on an independent test set using several performance metrics including receiver operating characteristic curves, which are used for FAA turbulence product evaluations prior to their deployment. A prototype implementation fuses data from Doppler radar, geostationary satellites, a lightning detection network and a numerical weather prediction model to produce deterministic and probabilistic turbulence assessments suitable for use by air traffic managers, dispatchers and pilots. The algorithm is scheduled to be operationally implemented at the National Weather Service's Aviation Weather Center in 2014.

No MeSH data available.


Related in: MedlinePlus