04_Computing Parameters Analytically

Computing Parameters Analytically

Normal Equation

Normal Equation is the second way of minimizing Cost Function, except Gradient Descent.

Method:
explicitly taking its derivatives with respect to the θj ’s, and setting them to zero.

The normal equation formula is given below:

there is something matter Matrix Derivative

Ng’s notes

Andrew的notes的公式推导，已经将 偏导数=0 纳入考虑。

1. 不同于
2. 两边左乘得到derivative，
3. 对两边乘上的逆

直观理解

的左乘：不一定是可逆方阵
至少是方阵（不一定可逆
但是用matlab的 pinv一定能够得到，最优解（不管是否可逆

In practice, when n exceeds 10,000 it might be a good time to go from a normal solution to an iterative process.

这里可能会需要一些 矩阵论 的知识。

Normal Equation Noninvertibility

Ifis noninvertible, the common causes might be having:

• Redundant features, where two features are very closely related (i.e. they are linearly dependent)
• Too many features (e.g. m ≤ n). In this case, delete some features or use “regularization” (to be explained in a later lesson).

Solutions to the above problems include deleting a feature that is linearly dependent with another or deleting one or more features when there are too many features.