03_Multivariate Linear Regression

Multivariate Linear Regression

Multiple Features

Linear regression with multiple variables

Remark: Note that for convenience reasons in this course we assume.

Gradient Descent for Multiple Features

Hypothesis:
Parameters:
Cost Function:

Gradient Descent Algorithm

Repeat{

}(simultaneously update for every)

Repeat until convergence:{

}

The following image compares gradient descent with one variable to gradient descent with multiple variables:

practical tricks

Feature Scaling

Main Idea: Make sure features are on a similar scale.
will descend quickly on small ranges and slowly on large ranges.

Method: Get every feature into approximately arange

Mean normalization

Replacewithto make features have approximately zero mean (Do not apply to).

E.g.

mean normalization formula:


is the average of feature i, andmean the range of value(the max - min) or the standard deviation (if u know)

Gradient Descent refer to Learning Rate

To recap:

1. ifis too small: slow convergence.
2. ifis too large: may not decrease on every iteration and thus may not converge.

Features and Polynomial Regression

The feature and form of hypothsis function can be improved in different way.
also can multiple features into one . For instance, we can combineandinto a new feature.

Common Polynomial Regrssion

Quadratic Function

Cubic Function

Square Root Function

Remark

One important thing to keep in mind is, if you choose your features this way then feature scaling becomes very important.

eg. ifhas range 1-1000 then range ofbecomes 1-1000000 and that ofbecomes 1-1000000000