The Ann Arbor weather data provides an annual time series of the low temperature in the month of January from 1900 to 2019. We aim to conduct an exploratory analysis of this time series and apply ARMA techniqures.

Overview of Annual Temperatures in Ann Arbor

Originally, temperature from 1955 was missing, so we interpolated between 1954 and 1956 in order to fill in the missing data. We show the time series plot in figure 1 and fit a linear regression line to the plot. The regression line looks almost flat, indicating that there is almost no trend in the time series.

Plot of Ann Arbor Low Temperature

Figure 1: Plot of Ann Arbor Low Temperature

From table 1 we see that that the mean level is -2.87 °F and ranges from -22 to 19 °F, a fairly wide range. An ARMA model whether there dependence over time or if what we are seeing is pure white-noise.

Table 1: Low Temperature Summary Table
type variables
numeric Low

Fitting ARMA Models

We seek to fit a stationary Gaussian \(\mathrm{ARMA}(p,q)\) model with parameter vector \(\theta=\left(\phi_{1 ; p}, \psi_{1: q}, \mu, \sigma^{2}\right)\) given by \[ \phi(B)\left(Y_{n}-\mu\right)=\psi(B) \epsilon_{n} \] where \[ \begin{aligned} \mu &=\mathbb{E}\left[Y_{n}\right] \\ \phi(x) &=1-\phi_{1} x-\cdots-\phi_{p} x^{p} \\ \psi(x) &=1+\psi_{1} x+\cdots+\psi_{q} x^{q} \\ \epsilon_{n} &\sim \text{iid } N\left[0, \sigma^{2}\right] \end{aligned} \]

We tabulate values of AIC in table 2 according to different values of \(p\) and \(q\), where we take \(p,q\in\{0,\ldots,4\}\). Recall that Akaike’s information criterion (AIC) is given by, \[ A I C=-2 \times \ell\left(\theta^{*}\right)+2 D \]

Table 2: ARIMA AIC table
MA0 MA1 MA2 MA3 MA4
AR0 825.07 826.75 828.72 830.27 832.24
AR1 826.74 828.16 830.14 831.72 833.71
AR2 828.69 830.14 832.09 833.73 835.70
AR3 830.15 831.83 833.76 830.64 831.95
AR4 832.14 833.83 835.76 832.00 824.47
AR5 832.87 834.77 827.00 828.79 829.70

From table 2 we see that ARMA(0,0) is the model with the second lowest AIC. For our analysis we will focus on this model and three neighboring models ARMA(0,1), ARMA(1,0), and ARMA(1,1). We disregard ARMA(4,4), because this model is likely to overfit the data.

Next in table 3 we show our parameter estimates of each of the four models we described. We can see that all the models have nearly the same estimate of the intercept, however the intercept of the ARMA(1,1) model has slightly higher s.e. The ARMA(0,1) and ARMA(1,0) models, both have small s.e’s of their estimates, however their estimates of the AR and MA coefficients are also quite small. Moving forward, we will focus on the ARMA(0,0) model and ARMA(1,1).

Table 3: Regression Coefficients and estimates of Various ARMA(p,q) models.
model Coefficients AR1 MA1 Intercept
ARMA(0,0) coef. NA NA -2.87
ARMA(0,0) s.e. NA NA 0.68
ARMA(0,1) coef. NA 0.05 -2.87
ARMA(0,1) s.e. NA 0.09 0.71
ARMA(1,0) coef. 0.05 NA -2.87
ARMA(1,0) s.e. 0.09 NA 0.71
ARMA(1,1) coef. 0.78 -0.72 -2.92
ARMA(1,1) s.e. 0.09 0.31 0.84

Comparing ARMA Models

Checking the roots of the AR and MA polynomials in the ARMA(1,1) model, the AR root is 1.28 and the MA root is 1.38, wich are both outside the unit circle. Thus, we conclude that our ARMA(1,1) model is causal and invertible, which is desirable. However, the roots are also close, indicating that there is potential cancellation. If this is the case, then ARMA(0,0) model would become our candidate model. Let’s explore the ARMA(0,0) and ARMA(1,1) models further.

Testing Normality Assumption

First we check the assumption that \(\epsilon_{n} \sim \text{iid } N\left[0, \sigma^{2}\right]\) for both ARMA(0,0) and ARMA(1,1) models. One way to check the independence assumption is to plot autocorrelation function of the residuals. In figure 2 we see that the ACFs of both models are similar. However, we see that the short term lags of the ARMA(1,1) model are slightly smaller than those of the ARMA(0,0) model. This is desirable when selecting a proper model. Furthermore we see that the residuals display some cyclical behavior in both plots, which can be remedied by fitting a model with trend.

ACF plots of residuals for for ARMA(0,0) and ARMA(1,1) models.

Figure 2: ACF plots of residuals for for ARMA(0,0) and ARMA(1,1) models.

To test the normality assumption, we create QQ plots of the residuals. The QQ plots are shown in figure 3. Both plots are nearly identical and seem to satisfy the normality assumption.

QQ Plots for the residuals of ARMA(0,0) and ARMA(1,1) models

Figure 3: QQ Plots for the residuals of ARMA(0,0) and ARMA(1,1) models

First Simulation Study

Next, we perform a simulation study on the coefficients of the ARMA(1,1) model to determine if our estimates and errors of the coefficients agree with our simulation. In figure 4 we see that the densities have a bimodal structure, both of which are close to the boundaries of invertibility. This gives us less confidence about the estimated coefficients of our ARMA(1,1) model.

Simulation study showing the densities of the profile likelihoods for our estimates of AR1 and MA1 coefficients.

Figure 4: Simulation study showing the densities of the profile likelihoods for our estimates of AR1 and MA1 coefficients.

Hypothesis Testing

To further test whether an ARMA(1,1) model is appropriate, we can construct a hypothesis test where the null model is ARMA(0,0) and the alternative model is ARMA(1,1). Suppose we have nested hypotheses where,

\[ \begin{aligned} H^{(0)}: \theta \in \Theta^{(0)}\\ H^{(1)}: \theta \in \Theta^{(1)} \end{aligned} \] defined via two nested parameter subspaces, \(\Theta^{(0)} \subset \Theta^{(1)},\) with respective dimensions \(D^{(0)}<D^{(1)} \leq D\). We consider the log likelihood maximized over each of the hypotheses, \[ \begin{aligned} \ell^{(0)} &=\sup _{\theta \in \Theta^{(0)}} \ell(\theta) \\ \ell^{\langle 1\rangle} &=\sup _{\theta \in \Theta^{(1)}} \ell(\theta) \end{aligned} \] Due to Wilks’ aapproximation, we can use the following likelihood ratio test,

\[\Lambda = 2(\mathcal{l}_1 - \mathcal{l}_0) \approx \chi^2_{D^{\left\langle 1\right\rangle} -D^{\left\langle 0\right\rangle}}\]

We find that \(\Lambda=\) 0.916 and \(\chi^2_{2}\) = 5.991, thus we fail to reject the null hypothesis that the data can be modeled by ARMA(0,0).

Second Simulation Study

It is helpful to do another simulation study for which we fit ARMA(1,1) when the true model is ARMA(0,0). In figure 5 we see that when the true model is ARMA(0,0) and fit an ARMA(1,1) model, we obtain a model with estimated MA1 and AR1 coefficients close to the boundary of invertibility.

This adds more evidence that that we cannot reject an ARMA(0,0) hypothesis, even though our previous analysis provided evidence that the data should be modeled by non-zero AR1 and MA1 coefficients.

Densities of the profile likelihoods for our estimates of AR1 and MA1 coefficients given ARMA(0,0) is the true model.

Figure 5: Densities of the profile likelihoods for our estimates of AR1 and MA1 coefficients given ARMA(0,0) is the true model.

Conclusion

We looked at various approaches for comparing ARMA models and our analysis points in the direction of modeling temperature data as an ARMA(0,0) white-noise model. Even though the ARMA(1,1) model may be a potential candidate, we do not find sufficient evidence to support the use of ARMA(1,1) model over ARMA(0,0).

Resources

I relied purely on lecture examples for this homework