Limits...
MAG4 versus alternative techniques for forecasting active region flare productivity.

Falconer DA, Moore RL, Barghouty AF, Khazanov I - Space Weather (2014)

Bottom Line: We present a statistical method of measuring the difference in performance between MAG4 and comparable alternative techniques that forecast an active region's major-flare productivity from alternative observed aspects of the active region.We find that (1) Present MAG4 far outperforms both McIntosh Active-Region Class and Total Magnetic Flux, (2) Next MAG4 significantly outperforms Present MAG4, (3) the performance of Next MAG4 is insensitive to the forward and backward temporal windows used, in the range of one to a few days, and (4) forecasting from the free-energy proxy in combination with either any broad category of McIntosh active-region classes or any Mount Wilson active-region class gives no significant performance improvement over forecasting from the free-energy proxy alone (Present MAG4).Quantitative comparison of performance of pairs of forecasting techniques Next MAG4 forecasts major flares more accurately than Present MAG4 Present MAG4 forecast outperforms McIntosh AR Class and total magnetic flux.

View Article: PubMed Central - PubMed

Affiliation: Heliophysics and Planetary Science Office ZP13 MSFC/NASA Huntsville, Alabama, USA ; Center for Space Plasma and Aeronomic Research, University of Alabama in Huntsville Huntsville, Alabama, USA.

ABSTRACT

: MAG4 is a technique of forecasting an active region's rate of production of major flares in the coming few days from a free magnetic energy proxy. We present a statistical method of measuring the difference in performance between MAG4 and comparable alternative techniques that forecast an active region's major-flare productivity from alternative observed aspects of the active region. We demonstrate the method by measuring the difference in performance between the "Present MAG4" technique and each of three alternative techniques, called "McIntosh Active-Region Class," "Total Magnetic Flux," and "Next MAG4." We do this by using (1) the MAG4 database of magnetograms and major flare histories of sunspot active regions, (2) the NOAA table of the major-flare productivity of each of 60 McIntosh active-region classes of sunspot active regions, and (3) five technique performance metrics (Heidke Skill Score, True Skill Score, Percent Correct, Probability of Detection, and False Alarm Rate) evaluated from 2000 random two-by-two contingency tables obtained from the databases. We find that (1) Present MAG4 far outperforms both McIntosh Active-Region Class and Total Magnetic Flux, (2) Next MAG4 significantly outperforms Present MAG4, (3) the performance of Next MAG4 is insensitive to the forward and backward temporal windows used, in the range of one to a few days, and (4) forecasting from the free-energy proxy in combination with either any broad category of McIntosh active-region classes or any Mount Wilson active-region class gives no significant performance improvement over forecasting from the free-energy proxy alone (Present MAG4).

Key points: Quantitative comparison of performance of pairs of forecasting techniques Next MAG4 forecasts major flares more accurately than Present MAG4 Present MAG4 forecast outperforms McIntosh AR Class and total magnetic flux.

No MeSH data available.


Related in: MedlinePlus

Comparison of the distributions and average values of the same-run differences between each performance metric for MAG4 and the same performance metric for each of the other three forecasting techniques. Each metric-difference distribution from 2000 runs is shown in the five panels, each panel for a separate metric. In each panel, McIntosh AR Class (blue), Total Magnetic Flux (red), and Next MAG4 (green) are compared to Present MAG4. The blue, red, and green vertical lines each show the average of the difference of that metric, while the horizontal line shows the standard deviation of the distribution. For any given metric, a technique that performed equally well as MAG4 would have its metric-difference mean at 0. Except for ΔFAR, a technique that performed significantly worse (or better) than Present MAG4 has a metric-difference mean that is more than 1 standard deviation to the left (or right) of 0. A mean farther to the left is better for ΔFAR. Next MAG4 makes more accurate forecasts than Present MAG4, which in turn is more accurate than Total Magnetic Flux or McIntosh AR Class.
© Copyright Policy - open-access
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC4508933&req=5

fig04: Comparison of the distributions and average values of the same-run differences between each performance metric for MAG4 and the same performance metric for each of the other three forecasting techniques. Each metric-difference distribution from 2000 runs is shown in the five panels, each panel for a separate metric. In each panel, McIntosh AR Class (blue), Total Magnetic Flux (red), and Next MAG4 (green) are compared to Present MAG4. The blue, red, and green vertical lines each show the average of the difference of that metric, while the horizontal line shows the standard deviation of the distribution. For any given metric, a technique that performed equally well as MAG4 would have its metric-difference mean at 0. Except for ΔFAR, a technique that performed significantly worse (or better) than Present MAG4 has a metric-difference mean that is more than 1 standard deviation to the left (or right) of 0. A mean farther to the left is better for ΔFAR. Next MAG4 makes more accurate forecasts than Present MAG4, which in turn is more accurate than Total Magnetic Flux or McIntosh AR Class.

Mentions: Notes: Value in parentheses is the mean difference divided by the variance of the distribution of the difference using sign convention of positive for improvement over Present MAG4. We have reversed the sign of ΔFAR in order for it to be the same as the other four metrics, in that a positive value indicates the forecast is better than free-energy proxy only, and negative indicates the forecast is worse. Note that we have not reversed the sign of ΔFAR in Figure 4. ΔPC, etc. stand for ΔMj(k1,k2), comparing the performance of the forecasting technique in the left column to that of Present MAG4.


MAG4 versus alternative techniques for forecasting active region flare productivity.

Falconer DA, Moore RL, Barghouty AF, Khazanov I - Space Weather (2014)

Comparison of the distributions and average values of the same-run differences between each performance metric for MAG4 and the same performance metric for each of the other three forecasting techniques. Each metric-difference distribution from 2000 runs is shown in the five panels, each panel for a separate metric. In each panel, McIntosh AR Class (blue), Total Magnetic Flux (red), and Next MAG4 (green) are compared to Present MAG4. The blue, red, and green vertical lines each show the average of the difference of that metric, while the horizontal line shows the standard deviation of the distribution. For any given metric, a technique that performed equally well as MAG4 would have its metric-difference mean at 0. Except for ΔFAR, a technique that performed significantly worse (or better) than Present MAG4 has a metric-difference mean that is more than 1 standard deviation to the left (or right) of 0. A mean farther to the left is better for ΔFAR. Next MAG4 makes more accurate forecasts than Present MAG4, which in turn is more accurate than Total Magnetic Flux or McIntosh AR Class.
© Copyright Policy - open-access
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC4508933&req=5

fig04: Comparison of the distributions and average values of the same-run differences between each performance metric for MAG4 and the same performance metric for each of the other three forecasting techniques. Each metric-difference distribution from 2000 runs is shown in the five panels, each panel for a separate metric. In each panel, McIntosh AR Class (blue), Total Magnetic Flux (red), and Next MAG4 (green) are compared to Present MAG4. The blue, red, and green vertical lines each show the average of the difference of that metric, while the horizontal line shows the standard deviation of the distribution. For any given metric, a technique that performed equally well as MAG4 would have its metric-difference mean at 0. Except for ΔFAR, a technique that performed significantly worse (or better) than Present MAG4 has a metric-difference mean that is more than 1 standard deviation to the left (or right) of 0. A mean farther to the left is better for ΔFAR. Next MAG4 makes more accurate forecasts than Present MAG4, which in turn is more accurate than Total Magnetic Flux or McIntosh AR Class.
Mentions: Notes: Value in parentheses is the mean difference divided by the variance of the distribution of the difference using sign convention of positive for improvement over Present MAG4. We have reversed the sign of ΔFAR in order for it to be the same as the other four metrics, in that a positive value indicates the forecast is better than free-energy proxy only, and negative indicates the forecast is worse. Note that we have not reversed the sign of ΔFAR in Figure 4. ΔPC, etc. stand for ΔMj(k1,k2), comparing the performance of the forecasting technique in the left column to that of Present MAG4.

Bottom Line: We present a statistical method of measuring the difference in performance between MAG4 and comparable alternative techniques that forecast an active region's major-flare productivity from alternative observed aspects of the active region.We find that (1) Present MAG4 far outperforms both McIntosh Active-Region Class and Total Magnetic Flux, (2) Next MAG4 significantly outperforms Present MAG4, (3) the performance of Next MAG4 is insensitive to the forward and backward temporal windows used, in the range of one to a few days, and (4) forecasting from the free-energy proxy in combination with either any broad category of McIntosh active-region classes or any Mount Wilson active-region class gives no significant performance improvement over forecasting from the free-energy proxy alone (Present MAG4).Quantitative comparison of performance of pairs of forecasting techniques Next MAG4 forecasts major flares more accurately than Present MAG4 Present MAG4 forecast outperforms McIntosh AR Class and total magnetic flux.

View Article: PubMed Central - PubMed

Affiliation: Heliophysics and Planetary Science Office ZP13 MSFC/NASA Huntsville, Alabama, USA ; Center for Space Plasma and Aeronomic Research, University of Alabama in Huntsville Huntsville, Alabama, USA.

ABSTRACT

: MAG4 is a technique of forecasting an active region's rate of production of major flares in the coming few days from a free magnetic energy proxy. We present a statistical method of measuring the difference in performance between MAG4 and comparable alternative techniques that forecast an active region's major-flare productivity from alternative observed aspects of the active region. We demonstrate the method by measuring the difference in performance between the "Present MAG4" technique and each of three alternative techniques, called "McIntosh Active-Region Class," "Total Magnetic Flux," and "Next MAG4." We do this by using (1) the MAG4 database of magnetograms and major flare histories of sunspot active regions, (2) the NOAA table of the major-flare productivity of each of 60 McIntosh active-region classes of sunspot active regions, and (3) five technique performance metrics (Heidke Skill Score, True Skill Score, Percent Correct, Probability of Detection, and False Alarm Rate) evaluated from 2000 random two-by-two contingency tables obtained from the databases. We find that (1) Present MAG4 far outperforms both McIntosh Active-Region Class and Total Magnetic Flux, (2) Next MAG4 significantly outperforms Present MAG4, (3) the performance of Next MAG4 is insensitive to the forward and backward temporal windows used, in the range of one to a few days, and (4) forecasting from the free-energy proxy in combination with either any broad category of McIntosh active-region classes or any Mount Wilson active-region class gives no significant performance improvement over forecasting from the free-energy proxy alone (Present MAG4).

Key points: Quantitative comparison of performance of pairs of forecasting techniques Next MAG4 forecasts major flares more accurately than Present MAG4 Present MAG4 forecast outperforms McIntosh AR Class and total magnetic flux.

No MeSH data available.


Related in: MedlinePlus