Statistical Computation of Maximum Likelihood Estimates Using R - Math Problem Example

Statistical calculations of maximum likelihood estimations using the statistical software R. Each question answered directly not in report style R-code Explanatory comments Rainfall for January was collected over a period of 80 years in a coastal town. Variables recorded were (Xi, Yi). Xi(j): rain or no rain. Yi(j): amount of rain in mm. Si: Total Monthly amount of rain in January in year (i). I year, J day. mean amount of rain in wet days p probability of Rain falls independently on each day L likelihood for p and F density 1. Inter Annual-statistics: Write R-Code to calculate A. Inter-annual mean of S B. Inter-annual variance of S 2. First Model A. Calculate maximum likelihood estimators (p and ) B. Write R-code to evaluate MLE p and C. Compare mean and variance for S (total monthly precipitations) with observed values. Any problems 3. Atmospheric Effect Model Rainfall is affected by changes in the atmospheric "mode". Model used in Part 2 is modified Suppose that the atmosphere has two dominant modes, 0 and 1. Zik = 1 if the atmosphere is in mode k in year i, and set Zik = 0 otherwise. Behavior of the atmosphere is assumed to be independent across years 0 + 1 = 1. P(Zik = 1) = k for each i. p and do not depend on the atmospheric mode. However, due to lack of understanding about how the atmospheric modes arise, the variables Zik are hidden, and so it is suggested that the MLEs for the parameters are computed using an EM algorithm. The 6-parameter complete log-likelihood f(ao,p(O), f.L(O), al,p(l), f.L(1);x, y, z) is given by Write R function to compute Data Source: Meteorological Data for ALDERGROVE during February 2002 obtained on 5 November 2007 from http://ngfl.northumberland.gov.uk/keystage3ictstrategy/Handling%20data/Met%20office%20data/Aldergrove0202.xls Mean Absolute Deviation (MAD): MAD is the difference between prediction and real number. It is calculated as the sum of the absolute differences between the forecasted value and the real value for all forecasted values. The following equation is used to calculate MAD: (Havil 2003). Where: x actual value forecasted value absolute frequency N sample size op of Form Bottom of Form Simple Moving Average (SMA) A simple moving average is used to average a time series by removing the random fluctuation for forecasting future values. SMA do not account for seasonal changes. The duration of the moving average can best be determined according to the type of application data to forecast. Long time periods gives smoother response by removing random variations but react slower to changes in the data as it lags the trend. Short time periods produce more oscillation but closely follow the trend. SMA is calculated by averaging the most recent number of actual values. SMA is calculated by using the following equation (Chase & Jacobs 2006): Where Ft Forecast for coming period At-1 Actual value in the past At-2, At-3, Actual values two, three, periods ago. N Number of periods to be averaged In the attached excel document, SMA is calculated for three periods: three, four, and five. Different n time periods will produce different results of data values. The values of MAD corresponding to each period are shown in the following table: Table 1: MAD values for different periods of SMA Time Period (n) MAD 3 4.36 4 3.10 5 3.95 Table one demonstrates that the smallest value of MAD exists for the period of n=4. This indicates that the type of data being analyzed is best estimated using a period of four. Figure 1: SMA for periods of 3,4, and 5. Figure one confirms the results of MAD analysis from table one. The best fit trend line is the SMA for n=4. This line follows the actual data curve specially on the 15th, 22, and 25 where major change occurred in wind speed. The period that best fits the actual data is dependent on the type of data analyzed which is the wind speed. Weighted Simple Moving Average (WSMA): A weighted moving average puts different weights to each element, providing that the sum of all weights equals 1. Weights are chosen by experience and trial and error. A general rule applies that recent past is more indicative of the future and should get higher weighting. However, if the data are seasonal weights should be established accordingly. The weighted moving average advantage over the simple moving average is the ability to vary the effects of past data. WSMA is computed using the following equation: Where w1 weight given to actual occurrence for the period of t-1 w2 weight given to actual occurrence for the period of t-2 wn weight given to actual occurrence for the period of t-n n total number of periods used in the forecast. In the excel document, in the Weighted SMA sheet, the weights of the moving average are determined by trial and error to produce the least value of MAD since there is no expert opinion as to guide the setup of the weights and determining the optimal period. The following table demonstrates the different weight that provided the least values of MAD for the periods of three, four, and five. Table 2: Weights for different periods of WSMA Weights for n=3 0.1 0.3 0.6 Weights for n=4 0.05 0.1 0.15 0.7 Weights for n=5 0.05 0.05 0.05 0.2 0.65 In the attached excel document, WSMA is calculated for three periods: three, four, and five using weights that provide the least values of MAD. The values of MAD corresponding to each period are shown in the following table: Table 3: MAD values for different periods of WSMA Time Period (n) MAD 3 3.96 4 3.06 5 3.95 Table three demonstrates that the smallest value of MAD exists for the period of n=4. This indicates that wind speed prediction can best be analyzed using a moving average of 4. When comparing the respective periods in SMA and WSMA, it is found that the use of weights improved improved the MAD values in the periods of n=3, and less improvement was achieved in the periods of four and five. Figure 2: Forecast values of wind speed using WSMA for periods of n=3,4, & 5 Figure two demonstrates the close fit of WSMA for the period of two. The sharp changes in wind speed during the days of 15, 21, 24, and 28 are smoothed by the WSMA for n=4. Exponential Smoothing (ES) Exponential smoothing (ES) is the most used of all forecasting techniques. ES is a technique of moving averages that is more convenient and less costly to use from the data point of view. Simple weighted moving averages carry a large amount of historical data. In most applications, most recent occurrences are more indicative of the future than those in the more distant past. The importance of data diminishes as the past becomes more distant then the exponential smoothing may be the most logical and easiest method to use. ES is computed by using only three pieces of data; the most recent forecast, the actual observed value for the forecast period and a smoothing constant represented by the symbol . Each increment in the past is decreased by (1 ). The value for the constant is determined both by the nature of the product and the expert sense. The higher the value of alpha, the more closely the forecast follows the actual. In stable environments small reaction is expected and a small constant is used. If the environment of data is experiencing growth, higher alpha value is used to give greater importance to recent growth experience. ES is computed using the following equation: Where: Ft+1 = the new forecast value; Xt = actual value observed at period t; Ft = period t forecast In the attached excel document, exponential smoothing is calculated using different values of alpha. The value of alpha that produced the smallest value of MAD was alpha = 0.28. The least value of MAD obtained using exponential smoothing was equal to 3.7. This value of MAD is higher than the value obtained from forecasting wind speed using WSMA of period n=4 which was 3.06. The results are contrary to what was expected since exponential smoothing is more popular than WSMA. The following graph demonstrates the fit of the exponential smoothing method forecasted values to the actual wind speed values. Figure 3: Exponential Smoothing vs. Wind Speed Figure three demonstrates that the forecasted values ES lags the actual data curve. Adjusted Exponential Smoothing (AES) To better improve the accuracy of prediction of ES a second smoothing constant is introduced. The value of is determined between 0.0 and 1.0. The adjusted forecast is calculated by adding a trend factor to the initial forecast. The following equation is used to calculate AES: AES = Ft+1 + Tt+1 Where Tt+1 : exponentially smoothed trend factor for t + 1. F t+1 : exponentially smoothed forecast for t + 1. The exponentially smoothed trend factor for t+1 is calculated using the following equation: Where Tt : the trend factor of last period B : second smoothing constant In the attached excel document, trial and error has been applied to different values of alpha and beta to find the values that lead to the smallest value of MAD. The combination values of and that provided the smallest MAD of 3.64 was = 0.59 and = 0.16. The value of MAD produced from the AES method is little less than the value of MAD produced by the ES method (MAD = 3.7). The following figure demonstrates the fitting of the forecasted values of AES in comparison to the actual values of wind speed. Figure 4: Adjusted Exponential Smoothing vs Actual Wind Speed The value of MAD (3.69) for AES is larger than the value obtained from forecasting wind speed using WSMA of period n=4 which was 3.06. However, while comparing the curves of AES and WSMA (n=4), it is found that the curve of AES best represent and fit the actual data of wind speed while the curve of WSMA smoothes the high variations during the days of 22, 25, and 28. Summary and Conclusion: The time series data selected for this study was the average daily wind speed values for city of Aldergrove, U.K. during the month of February 2002. Four different time series forecasting methods was used during this study to forecast future wind speed using most recent data. The four methods used are: Simple Moving Average (SMA), Weighted Simple Moving Average (WSMA), Exponential Smoothing (ES), and Adjusted Exponential Smoothing (AES). The accuracy of each forecasting technique is analyzed using the test of fit Mean Absolute Deviation (MAD). In Excel, MAD for SMA and WSMA were calculated for three periods of three, four and five. MAD for ES and AES were also computed for forecasting wind speeds. It was found that SMA is the least accurate time series predicting method to predict wind speed data. It was also found that WSMA for a period of n=4 provided the smallest value of MAD equal to 3.06. However, the curve of WSMA smoothes and gets rid of the high variations in wind speed as displayed in the days of 22, 25, and 28. Further inspection of the curves that best fit actual wind speed data demonstrates that the curve of AES best fit the actual data and lags the actual data curve in the smallest matter. It is therefore concluded that even though WSMA provides the smallest MAD values but the most-fit curve of the AES method proves that AES is the most accurate time series forecasting method to predict future wind speeds in Aldergrove, U.K. during February 2002. It is suggested to include more wind speed data as to further test and distinguish between the accuracy of different time series methods in predicting future wind speed. References Chase, R. & Jacobs R. (2006). Operation Management for Competitive Advantage. McGraw Hill Eleventh Edition. Havil, J. (2003). "Ways of Means." 13.1 in Gamma: Exploring Euler's Constant. Princeton, NJ: Princeton University Press, pp. 119-121, 2003. Read More