📊 Econometrics & OLS Regression Analysis

Econometrics provides the statistical machinery to estimate causal economic relationships, test theoretical hypotheses, and forecast economic time series using empirical observational data.

---

1. 📐 The Classical Linear Regression Model (CLRM)

Consider the population regression function with $k$ regressors and $N$ observations:

$$y_i={\beta }_0+{\beta }_1{x}_{1i}+{\beta }_2{x}_{2i}+\cdots +{\beta }_k{x}_{ki}+u_i⟺\mathbf{y}=\mathbf{X}\mathit{\beta }+\mathbf{u}$$

1.1 Ordinary Least Squares (OLS) Estimator

OLS minimizes the Sum of Squared Residuals ($SSR$):

$$\underset{\hat{\mathit{\beta }}}{min}SSR(\hat{\mathit{\beta }})=\sum _{i=1}^N{\hat{u}}_i^2=(\mathbf{y}-\mathbf{X}\hat{\mathit{\beta }}{)}^{\prime }(\mathbf{y}-\mathbf{X}\hat{\mathit{\beta }})$$

Taking the matrix derivative and setting to zero yields the Normal Equations:

$${\mathbf{X}}^{\prime }\mathbf{X}\hat{\mathit{\beta }}={\mathbf{X}}^{\prime }\mathbf{y}⟹{\hat{\mathit{\beta }}}_{OLS}=({\mathbf{X}}^{\prime }\mathbf{X}{)}^{-1}{\mathbf{X}}^{\prime }\mathbf{y}$$

For simple univariate regression ($y_i={\beta }_0+{\beta }_1{x}_i+u_i$):

$${\hat{\beta }}_1=\frac{\sum (x_i-\overline{x})(y_i-\overline{y})}{\sum (x_i-\overline{x}{)}^2}=\frac{\widehat{\text{Cov}}(x,y)}{\widehat{\text{Var}}(x)},{\hat{\beta }}_0=\overline{y}-{\hat{\beta }}_1\overline{x}$$

---

2. 🛡️ The Gauss-Markov Theorem & BLUE

Gauss-Markov Theorem

Under assumptions A1–A5, the OLS estimator ${\hat{\mathit{\beta }}}_{OLS}$ is the Best Linear Unbiased Estimator (BLUE) (it possesses minimum variance among all linear unbiased estimators).

2.1 The Five Classical Assumptions

1. A1 (Linear in Parameters): $\mathbf{y}=\mathbf{X}\mathit{\beta }+\mathbf{u}$.
2. A2 (Strict Exogeneity): $\mathbb{E}[u_i\mid \mathbf{X}]=0$ (error term has conditional mean zero).
3. A3 (No Perfect Multicollinearity): $\text{rank}(\mathbf{X})=k+1<N$ (${\mathbf{X}}^{\prime }\mathbf{X}$ is invertible).
4. A4 (Homoskedasticity): $\text{Var}(u_i\mid \mathbf{X})={\sigma }^2$ (constant error variance).
5. A5 (No Serial Autocorrelation): $\text{Cov}(u_i,u_j\mid \mathbf{X})=0$ for $i\ne j$.

If errors are also normally distributed ($u_i\sim \mathcal{N}(0,{\sigma }^2)$), $\hat{\mathit{\beta }}\sim \mathcal{N}(\mathit{\beta },{\sigma }^2({\mathbf{X}}^{\prime }\mathbf{X}{)}^{-1})$.

---

3. ⚠️ Endogeneity & Omitted Variable Bias (OVB)

When a regressor is correlated with the error term ($\text{Cov}(x_j,u)\ne 0$), OLS is biased and inconsistent.

3.1 Omitted Variable Bias Formula

Suppose the true model is $y={\beta }_0+{\beta }_1{x}_1+{\beta }_2{x}_2+u$, but $x_2$ (e.g. unobserved ability) is omitted from the estimated regression $y={\tilde{\beta }}_0+{\tilde{\beta }}_1{x}_1+e$:

$$\mathbb{E}[{\tilde{\beta }}_1]={\beta }_1+\underset{\text{Effect of omitted on }y}{\underbrace{{\beta }_2}}\times \underset{\text{Relationship between }{x}_1\text{ and omitted }{x}_2}{\underbrace{\frac{\text{Cov}(x_1,x_2)}{\text{Var}(x_1)}}}$$

                          Sign of Bias on β_1
                     Cov(x_1, x_2) > 0    Cov(x_1, x_2) < 0
  ──────────────────────────────────────────────────────────
  β_2 > 0 (Positive)    Positive Bias        Negative Bias
  β_2 < 0 (Negative)    Negative Bias        Positive Bias

3.2 Instrumental Variables ($IV$ / Two-Stage Least Squares)

An instrument $Z$ must satisfy:

1. Instrument Relevance: $\text{Cov}(Z,X)\ne 0$ (Strong first stage).
2. Instrument Exogeneity (Exclusion Restriction): $\text{Cov}(Z,u)=0$ ($Z$ affects $y$ only through $X$).

$${\hat{\beta }}_{IV}=\frac{\widehat{\text{Cov}}(Z,y)}{\widehat{\text{Cov}}(Z,X)}$$

---

4. 🎯 Olympiad-Level Worked Master Problem

Master Problem: Omitted Variable Bias Calculation

Problem: An econometrician estimates the wage equation:

$$\ln (\text{Wage})={\beta }_0+{\beta }_1\text{Education}+{\beta }_2\text{Ability}+u$$

where true ${\beta }_1=0.08$ ($8\mathrm{\%}$ return per year of schooling) and true ${\beta }_2=0.04$. The auxiliary regression of Ability on Education yields $\text{Ability}=2.0+0.5\text{Education}+ϵ$.

1. Calculate the probability limit of the short regression coefficient ${\tilde{\beta }}_1$ when Ability is omitted.
2. Calculate the percentage overstatement of the true return to education.

Step-by-Step Rigorous Solution:

1. Apply Omitted Variable Bias Equation:

   $$\text{plim }{\tilde{\beta }}_1={\beta }_1+{\beta }_2\times \frac{\text{Cov}(\text{Education},\text{Ability})}{\text{Var}(\text{Education})}$$

   The auxiliary slope is ${\hat{\gamma }}_1=\frac{\text{Cov}(\text{Education},\text{Ability})}{\text{Var}(\text{Education})}=0.5$.

2. Compute Expected Estimate ${\tilde{\beta }}_1$:

   $$\text{plim }{\tilde{\beta }}_1=0.08+(0.04\times 0.5)=0.08+0.02=0.10(10.0\mathrm{\%})$$

3. Compute Overstatement Percentage:

   $$\text{Overstatement}=\frac{0.10-0.08}{0.08}\times 100=\frac{0.02}{0.08}\times 100=25.0\mathrm{\%}$$

   Finding: Omitting unobserved ability leads to a $25\mathrm{\%}$ upward bias in the estimated economic return to schooling.