NOAA / Space Weather Prediction Center
Notice: August 3, 2009 the Wang Sheeley Predictions were removed from the Model. These 2 to 8 days predictions may resume at a later date.
It is well established that high-energy (relativistic) electrons can cause deep dielectric charging of satellites, which can in turn lead to spacecraft upsets and/or complete satellite failures [Reagan et al, 1983; Baker et al, 1987]. These relativistic electrons are monitored at geo-synchronous orbit by GOES satellite instruments; plots of the data are available on SWPC’s web page. To warn customers of possible damaging effects, SWPC issues alerts when the >2 MeV electron flux at GOES exceeds 103 particle flux units (pfu) for more than three consecutive five-minute periods. An even more significant threshold is when the cumulative fluence exceeds 109 pfu over a 72-hour period.
As the data plots show, the flux of relativistic electrons at geo-synchronous orbit undergoes significant diurnal variation. The cumulative daily fluence can also show a day to day variation of many orders of magnitude. Yet even with this wide variation, scientists have found a statistical link between the daily electron fluence and average solar wind speed. Baker et al  showed that the daily >2 MeV electron fluence measured at geo-synchronous orbit could be predicted one day in advance using a linear filter technique with solar wind speed as the input. SWPC has made further refinements to this technique, improving the forecast skill and extending the lead-time, resulting in the Relativistic Electron Forecast Model (REFM).
REFM forecasts are intended to supplement current SWPC alerts by providing guidance as to the future likelihood of damaging relativistic electron events. Since the REFM output is a daily fluence forecast, not flux, it applies most directly to the high fluence threshold. In fact the graphical model output displays a warning message during periods when the 72-hour fluence exceeds or is forecast to exceed 109 pfu. High 24-hour fluence values can also be used as a proxy for high flux levels. A study of data measured between 1995 and 2000 showed that when the 24-hour fluence exceeds 3 × 107 pfu the SWPC flux alert criteria is nearly always met (false alarm ratio = 0.07, miss frequency = 0.06). With this threshold, REFM fluence forecasts can be used to predict SWPC flux alerts.
Output from the REFM, which runs once at the start of each UT day, is made available as both graphical plots and text files on the main web page. The REFM web page presents output from two different operational modes. The first mode, ACE, uses real-time solar wind speeds measured by NASA's Advanced Composition Explorer satellite to create forecasts with 1 to 3 days lead-time. The second mode, WS, takes advantage of predicted solar wind values calculated using the Wang-Sheeley technique to extend the predicted electron fluence up to eight days in advance.
Return to Top
Opening the main REFM web page loads the most recent model run. A plot of the output dominates the page. In addition to the plot, the page contains links to text files of model output and recent performance statistics for both the ACE and Wang-Sheeley (WS) modes. There is also a link to a verification plot.
The standard REFM plot displays roughly 30 days of observed and forecast data. Output from both the ACE and WS modes are shown on the same plot; previous forecast values are kept on screen for comparison with observed data. An example from a period of high electron fluence in 1996 is given below, along with a description of the plot's features.
The main difference between the standard and verification plots is that the Verification Plot displays all of the previous forecast values in order to compare them to the observed data. While useful, this makes the plot more difficult to read than the standard plot.
Return to Top
The Relativistic Electron Forecast Model (REFM) predicts the >2 MeV 24-hour electron fluence at geo-synchronous orbit. It is based on a linear prediction filter (Baker, 1990) that uses average solar wind speed as its input. An offset is employed to help account for additional physical processes that can dramatically affect the electron fluence (the fluence can drop 2-3 decades in 24 hours without a corresponding change in solar wind speed).
The linear prediction filter is a relatively simple statistical technique in which correlation coefficients are created from historical data by solving a set of linear equations (Levinson, 1947). These coefficients are then applied to real-time data. The REFM uses 30 days of solar wind speed data (currently from the ACE spacecraft) to predict the 24-hour fluence with +1 to +8 days of lead-time. In the ACE mode the +1, +2, and +3 day forecasts each use a different set of coefficients. In the Wang-Sheeley mode, forecast solar wind speed values are substituted for observed data to extend the electron fluence forecast out to +8 days.
As mentioned earlier, the electron fluence can exhibit dramatic day-to-day variability, typically associated with strong shocks and/or geomagnetic storming. This is believed to be due to both magnetic field stretching, in which the electron population is temporarily shifted into or out of geo-synchronous orbit, and sudden electron loss to either the magnetopause or upper atmosphere. The basic linear filter is unable to account for these processes; given high solar wind speeds it will forecast high electron fluences, even if the electrons actually experience a sudden decrease or increase.
To help account for this short-term variability as well as longer-term drifts, REFM adjusts the output of the basic linear prediction filter with a flux offset. REFM compares previous forecasts to the observed values, computes an offset, and applies it as a correction to the current forecast. The number of days used to compute the offset varies; for example, the +1 Day ACE mode uses only the most recent forecast/observed pair, while the +2 and +3 Day ACE mode and WS mode all use the past 20 days of forecast/observed pairs. These numbers were chosen to maximize the performance statistics of each mode.
Although the offset improves the statistics, at best the flux offset's response lags a day behind the environment's variability. This causes an associated rise in model errors during solar maximum when coronal mass ejections (CMEs) and subsequent strong geomagnetic storms are prevalent. For this reason, the REFM performs best during solar minimum. In addition to the problems associated with electron variability, the WS mode is limited by the capability of the Wang-Sheeley technique. The WS forecasts of solar wind speed assume a "quiet sun", i.e. no CMEs, further increasing the problems for the WS mode during solar max. Fortunately the largest electron events occur during solar minimum when both the ACE and WS modes perform at their best.
Both the electron sensor on GOES and the SWEPAM sensor on ACE are subject to contamination during proton events. To avoid this contamination, the REFM will not use these data when the >30 MeV protons exceed 50 pfu (a fairly conservative threshold). Without solar wind speed data for the current day, the REFM (ACE mode) will not produce a forecast. An outage in ACE coverage or a satellite maneuver can also cause a data gap resulting in no forecast. Because the Wang-Sheeley mode uses forecast wind speeds, it is not as reliant on ACE data, but the WS model requires observations from ground based observatories, and can therefore have gaps due to clouds or maintenance.
Return to Top
Verification was done using standard measures of skill. The Skill Score (SS) is a relative measure in which performance of the target forecast is compared to a reference forecast. The SS is defined as
The most commonly used reference forecast is simply the average observation (i.e. sample mean); when this is used, the SS is referred to as the Prediction Efficiency (PE). Other common reference forecasts include Persistence (using the most recent observation as a forecast) and Recurrence (using the observation from 27 days prior). Perfect predictions result in an SS = 1.0, positive and negative values respectively indicate performance better or worse than the reference forecast.
The Skill Score given above is appropriate for gauging model performance as a whole. To measure the model's ability to forecast rare events, such as 72-hour fluence exceeding 109 pfu, we used the True Skill Statistic (TSS) which is an absolute measure based on contingency tables (Doswell, 1990). The TSS ranges from –1.0 (no correct forecasts) to 1.0 (perfectly correct), with positive values indicating a forecast skill exceeding that expected from pure chance.
As explained in Model Specifics, the REFM performs best during solar minimum conditions, when large geomagnetic storms are less frequent. Therefore, in order to test the model under optimal conditions, scores were tabulated using data from the fall of Cycle 22 into the rise of Cycle 23, 1996 through 1999. Model coefficients for this test were generated using data from 1995, guaranteeing independent results. Due to the large day-to-day variations in electron fluence, the Skill Scores were calculated using the logarithm (base 10) of the observed and forecast values.
The following table lists the Skill Scores for each forecast period, +1 to +8 days, and each forecast mode, ACE or WS. Three standard reference forecasts were used: sample mean (Prediction Efficiency), persistence, and recurrence.
|Skill Score versus|
|+1 Day (ACE)||0.71||0.13||0.79|
|+2 Day (ACE)||0.39||0.15||0.56|
|+3 Day (ACE)||0.24||0.26||0.45|
|+2 Day (WS)||0.32||0.00||0.48|
|+3 Day (WS)||0.19||0.18||0.39|
|+4 Day (WS)||0.09||0.26||0.31|
|+5 Day (WS)||0.06||0.33||0.29|
|+6 Day (WS)||0.03||0.35||0.26|
|+7 Day (WS)||-0.01||0.36||0.23|
|+8 Day (WS)||-0.02||0.37||0.22|
The results show that the ACE mode has good forecast skill, especially on day one, but that in the short-term the results are only slightly better than using persistence. The WS mode shows lower skill (PE) that continues to drop as the lead-time increases. This is not too surprising since the WS mode relies on other model output to create its forecast. In fact, for the +2 and +3 day forecasts, the ACE mode out-performs the WS mode in all three categories. In both modes the Prediction Efficiencies drop with increasing lead-time, while the skill relative to Persistence increases.
The next table shows the True Skill Statistics for the model's ability to predict 72-hour fluences that exceed 109 pfu. For comparison, scores are also shown when persistence is used to forecast these events.
|Forecast||True Skill Statistic|
|Lead||ACE mode||WS mode||Persistence|
These results show that in the short-term (1-2 days) Persistence has higher skill scores than either REFM mode. This is due to the fact that when the 72-hour fluence exceeds 109 pfu, it tends to stay elevated for a few days. Of course as the lead-time increases, the model's performance quickly overtakes persistence.
Return to Top
Baker, D.N., et al, "Deep Dielectric Charging Effects Due to High Energy Electrons in the Earth’s Outer Magnetosphere", J. Electrost., 20, 3, 1987.
Baker, D.N., et al, "Linear Prediction Filter Analysis of Relativistic Electron Properties at 6.6 Re", J. Geophysical Res., 95, A9, 1990.
Doswell, et al, "On Summary Measures of Skill in Rare Event Forecasting Based on Contingency Tables", Weather and Forecasting, 5, December 1990.
Reagan, J.B., et al, "Space Charging Currents and their Effects on Spacecraft Systems", IEEE Trans. Electr. Insul., EI-18, 345, 1983.
Return to To
Space Weather Topics:
Alerts / Warnings, Space Weather Now, Today's Space Wx, Data and Products, About Us ,
Email Products, Space Wx Workshop , Education/Outreach, Disclaimer, Customer Services, Contact Us