线性回归的普通最小二乘估计 |
您所在的位置：网站首页 › ols估计量的线性特性推导 › 线性回归的普通最小二乘估计 |
|
线性回归的普通最小二乘估计¶
本文推导了线性回归的普通最小二乘估计量的矩阵形式，并在一元线性回归的情境下给出了求和形式的表达式。 $$ Y=X \widehat{\beta}+e $$ \[ \beta^{O L S}=\left(X^{\prime} X\right)^{-1} X^{\prime} Y \]在一元线性回归的情境下： \[ \beta_1^{O L S} =\frac{\overline{X Y}-\overline{X} * \overline{Y}}{\overline{X^2}-\left(\overline{X}\right)^2} \] \[ \beta_0^{O L S} =\frac{\overline{X^2} * \overline{Y}-\overline{X} * \overline{X Y}}{\overline{X^2}-\left(\overline{X}\right)^2} \] 线性模型¶若我们想要用\(X_k \left(k=1,\ldots,K\right)\)来解释\(Y\)，且\(Y\)关于\(X\)是线性的，即： $$ Y_t=\beta_0+\beta_1 X_{t 1}+\beta_2 X_{t 2} +\cdots+\beta_K X_{t K} $$ Note 这里加入了截距项（或偏置项），因此需要加入\(\beta_0\)。 将各变量以矩阵形式表示如下： \[ Y=\underbrace{\left[\begin{array}{c} Y_1 \\ Y_2 \\ \ldots \\ Y_T \end{array}\right]}_{T \times 1}, X=\underbrace{\left[\begin{array}{ccccc} 1 & X_{11} & X_{12} & \ldots & X_{1 K} \\ 1 & X_{21} & X_{22} & \ldots & X_{2 K} \\ \ldots & \ldots & \ldots & \ldots & \ldots \\ 1 & X_{T 1} & X_{T 2} & \ldots & X_{T K} \end{array}\right]}_{T \times(K+1)}, \beta=\underbrace{\left[\begin{array}{c} \beta_0 \\ \beta_1 \\ \ldots \\ \beta_K \end{array}\right]}_{(K+1) \times 1}, u=\underbrace{\left[\begin{array}{c} u_1 \\ u_2 \\ \ldots \\ u_T \end{array}\right]}_{T \times 1} \]矩阵\(X\)中的每一行对应一个观测点，每一列对应一个解释变量。 简化为： $$ Y=X \beta+u\label{1}\tag{1} $$ 公式\(\eqref{1}\)是真实的线性模型，其中的\(\beta\)是真实存在但未知的。我们需要构造估计量来估计\(\beta\)： $$ Y=X \widehat{\beta}+e\tag{2}\label{2} $$ 回归系数的普通最小二乘估计量¶“普通最小二乘”的含义是：使得误差项的平方和最小。普通最小二乘估计量是如下最小化问题的解： $$ \underset{\widehat{\beta}}{\operatorname{argmin}} \sum_{t=1}^T e_{t} ^{2} = e^{\prime} e=(Y-X \widehat{\beta})^{\prime}(Y-X \widehat{\beta}) $$ 其中 \[ e=\underbrace{\left[\begin{array}{c} e_1 \\ e_2 \\ \ldots \\ e_T \end{array}\right]}_{T \times 1} \]解这个最小化问题的推导过程在 普通最小二乘法的矩阵形式推导 中已有介绍。这里简单回顾如下： $$ \min (Y-X \widehat{\beta})^{\prime}(Y-X \widehat{\beta}) $$ 由于 \(\frac{d\left(A^{\prime} A\right)}{d A}=2 A, \frac{d(C B)}{d B}=C^{\prime}\) $$ \begin{aligned} \frac{d(Y-X \widehat{\beta})^{\prime}(Y-X \widehat{\beta})}{d \widehat{\beta}} & =0 \\ (-X)^{\prime} 2(Y-X \widehat{\beta}) & =0 \\ X^{\prime}(Y-X \widehat{\beta}) & =0 \\ X^{\prime} Y-X^{\prime} X \widehat{\beta} & =0 \\ X^{\prime} X \widehat{\beta} & =X^{\prime} Y \end{aligned} $$ 若 \(\left(X^{\prime} X\right)^{-1}\) 存在，则 $$ \left(X^{\prime} X\right)^{-1} X^{\prime} X \widehat{\beta}=\left(X^{\prime} X\right)^{-1} X^{\prime} Y $$ \[ \Rightarrow \beta^{O L S}=\left(X^{\prime} X\right)^{-1} X^{\prime} Y \] 一元线性回归¶若只有一个解释变量，则 \[ Y=\underbrace{\left[\begin{array}{c} Y_1 \\ Y_2 \\ \ldots \\ Y_T \end{array}\right]}_{T \times 1}, X=\underbrace{\left[\begin{array}{cc} 1 & X_1 \\ 1 & X_2 \\ \ldots & \ldots \\ 1 & X_T \end{array}\right]}_{T \times 2}, u=\underbrace{\left[\begin{array}{c} u_1 \\ u_2 \\ \ldots \\ u_T \end{array}\right]}_{T \times 1}, \beta=\underbrace{\left[\begin{array}{c} \beta_0 \\ \beta_1 \end{array}\right]}_{2 \times 1} \] \[ \begin{aligned} \underbrace{\left[\begin{array}{c} \beta_0^{O L S} \\ \beta_1^{O L S} \end{array}\right]}_{2 \times 1}& =\underbrace{\left(\left[\begin{array}{cccc} 1 & 1 & \ldots & 1 \\ X_1 & X_2 & \ldots & X_T \end{array}\right]\left[\begin{array}{cc} 1 & X_1 \\ 1 & X_2 \\ \ldots & \ldots \\ 1 & X_T \end{array}\right]\right)^{-1}}_{2 \times 2} \underbrace{\left[\begin{array}{cccc} 1 & 1 & \ldots & 1 \\ X_1 & X_2 & \ldots & X_T \end{array}\right]\left[\begin{array}{c} Y_1 \\ Y_2 \\ \ldots \\ Y_T \end{array}\right]}_{2 \times 1} \\ & =\left(\left[\begin{array}{cc} T & \sum X_t \\ \sum X_t & \sum X_t^2 \end{array}\right]\right)^{-1}\left[\begin{array}{c} \sum Y_t \\ \sum X_t Y_t \end{array}\right] \\ & =\frac{1}{T \sum X_t^2-\left(\sum X_t\right)^2}\left[\begin{array}{cc} \sum X_t^2 & -\sum X_t \\ -\sum X_t & T \end{array}\right]\left[\begin{array}{c} \sum Y_t \\ \sum X_t Y_t \end{array}\right] \\ & =\frac{1}{T \sum X_t^2-\left(\sum X_t\right)^2}\left[\begin{array}{c} \sum X_t^2 \sum Y_t-\sum X_t \sum X_t Y_t \\ -\sum X_t \sum Y_t+T \sum X_t Y_t \end{array}\right] \\ & =\left[\begin{array}{c} \frac{\sum X_t^2 \sum Y_t-\sum X_t \sum X_t Y_t}{T \sum X_t^2-\left(\sum X_t\right)^2} \\ \frac{T \sum X_t Y_t-\sum X_t \sum Y_t}{T \sum X_t^2-\left(\sum X_t\right)^2} \end{array}\right] \\ & \end{aligned} \] \(\beta_1\)的估计¶ \[ \begin{aligned} \beta_1^{O L S} & =\frac{T \sum X_t Y_t-\sum X_t \sum Y_t}{T \sum X_t^2-\left(\sum X_t\right)^2} \\ & =\frac{\frac{1}{T^2}\left\{T \sum X_t Y_t-\sum X_t \sum Y_t\right\}}{\frac{1}{T^2}\left\{T \sum X_t^2-\left(\sum X_t\right)^2\right\}} \\ & =\frac{\frac{\sum X_t Y_t}{T}-\frac{\sum X_t}{T} \frac{\sum Y_t}{T}}{\frac{\sum X_t^2}{T}-\left(\frac{\sum X_t}{T}\right)^2} \\ & =\frac{\overline{X Y}-\overline{X} * \overline{Y}}{\overline{X^2}-\left(\overline{X}\right)^2} \end{aligned} \] \(\beta_0\)的估计¶ \[ \begin{aligned} \beta_0^{O L S} & =\frac{\sum X_t^2 \sum Y_t-\sum X_t \sum X_t Y_t}{T \sum X_t^2-\left(\sum X_t\right)^2} \\ & =\frac{\frac{1}{T^2}\left\{\sum X_t^2 \sum Y_t-\sum X_t \sum X_t Y_t\right\}}{\frac{1}{T^2}\left\{T \sum X_t^2-\left(\sum X_t\right)^2\right\}} \\ & =\frac{\frac{\sum X_t^2 \sum Y_t}{T}-\frac{\sum X_t}{T} \frac{\sum X_t Y_t}{T}}{\frac{\sum X_t^2}{T}-\left(\frac{\sum X_t}{T}\right)^2} \\ & =\frac{\overline{X^2} * \overline{Y}-\overline{X} * \overline{X Y}}{\overline{X^2}-\left(\overline{X}\right)^2} \end{aligned} \] 评论 |
今日新闻 |
推荐新闻 |
| CopyRight 2018-2019 办公设备维修网 版权所有 豫ICP备15022753号-3 |