15.1 The least squares method


Let us start from the simple explanation of the least-squares method. Let $\{ ({a}_{i}, x_{i} )\}_{{i}=1}^{n}$ be a sequence in the two dimensional real space ${\mathbb R}^2$. Let $\phi^{(\beta_1, \beta_2)}: {\mathbb R} \to {\mathbb R}$ be the simple function such that

\begin{align} {\mathbb R} \ni a \mapsto x= \phi^{(\beta_1, \beta_2)}({a}) =\beta_1 {a} + \beta_0 \in {\mathbb R} \tag{15.1} \end{align}

where the pair $(\beta_1, \beta_2) (\in {\mathbb R}^2 )$ is assumed to be unknown. Define the error $\sigma_{}^{}$ by

\begin{align} \sigma_{}^2 (\beta_1, \beta_2)= \frac{1}{{n}} \sum_{{i}=1}^n (x_{i} - \phi^{(\beta_1, \beta_2)}({a}_{i}))^2 \Big( = \frac{1}{{n}} \sum_{{i}=1}^{n} ( x_{i} -( \beta_1 {a}_{i} + \beta_0 ))^2 \Big) \tag{15.2} \end{align} Then, we have the following minimization problem:

Problem 15.1 [The least squares method]
$\quad$ Let $\{ ({a}_{i}, x_{i} )\}_{{i}=1}^{n}$ be a sequence in the two dimensional real space ${\mathbb R}^2$. \\ Find the $(\hat{\beta}_0, \hat{\beta}_1)$ $( \in {\mathbb R}^2 )$ such that \begin{align} \sigma_{}^2 (\hat{\beta}_0, \hat{\beta}_1) = \min_{(\beta_1, \beta_2) \in {\mathbb R}^2 } \sigma_{}^2 (\beta_1, \beta_2) \Big( = \min_{(\beta_1, \beta_2) \in {\mathbb R}^2 } \frac{1}{{n}} \sum_{{i}=1}^{n} ( x_{i} -( \beta_1 {a}_{i} + \beta_0 ))^2 \Big) \tag{15.3} \end{align} where $(\hat{\beta}_0, \hat{\beta}_1)$ is called "sample regression coefficients".



This is easily solved as follows. Taking partial derivatives with respect to $\beta_0$, $\beta_1 $, and equating the results to zero, gives the equations (i.e., "likelihood equations"),

\begin{align} & \frac{\partial \sigma_{}^2 (\beta_1, \beta_2)}{\partial \beta_0} = {\sum_{{i}=1}^{n} (x_{i} - \beta_0 - \beta_1 {a}_{i})}=0, \quad ({i}=1,...,{n}) \tag{15.4} \\ & \frac{\partial \sigma_{}^2 (\beta_1, \beta_2)}{\partial \beta_1} = {\sum_{{i}=1}^{n} (x_{i} - \beta_0 - \beta_1 {a}_{i}){a}_{i}}=0, \quad ({i}=1,...,{n}) \tag{15.5} \end{align} Solving it, we get that \begin{align} & \hat{\beta}_1 = \frac{s_{{a}x}}{s_{{a}{a}}}, \quad \hat{\beta}_0 = \overline{x}-\frac{s_{{a}x}}{s_{{a}{a}}}\overline{{a}}, \quad \hat{\sigma}^2 ( = \frac{1}{{n}} \sum_{{i}=1}^{n} ( x_{i} -( \hat{\beta}_1 {a}_{i} + \hat{\beta}_0 ))^2 \Big) = s_{xx} - \frac{s_{{a}x}^2}{s_{{a}{a}}} \tag{15.6} \end{align} where \begin{align} & {\bar {a}}=\frac{{a}_1 + \cdots + {a}_{n}}{{n}}, \qquad {\bar x}=\frac{x_1 + \cdots + x_{n}}{{n}},\qquad \tag{15.7} \\ & s_{{a}{a}}=\frac{({a}_1 -{\bar {a}})^2 + \cdots + ({a}_{n} - {\bar {a}})^2}{{n}}, \quad s_{xx}=\frac{(x_1 -{\bar x})^2 + \cdots + (x_{n} - {\bar x})^2}{{n}}, \quad \tag{15.8} \\ & s_{{a}x}=\frac{({a}_1 -{\bar {a}})(x_1 -{\bar x}) + \cdots + ({a}_{n} - {\bar {a}})(x_{n} - {\bar x})}{{n}}. \tag{15.9} \end{align}



Remark 15.2 [Applied mathematics]

Note that the above result is in (applied) mathematics, that is,

$\bullet$ the above is neither in statistics nor in quantum language.

Since quantum language says:

Everything should be dscribed by quantum language

the purpose of this chapter is to add a quantum linguistic story to Problem 15.1 (i.e., the least-squares method) in the framework of quantum language.