Insight re Linear Regression (HW#2 Q5)
I've been out of school for a long time and so basic math is a bit slow to process. When it comes to the particular linear regression described in the lecture and employed in Q5, I had an insight to share:
This is not linear regression in 2D. It is linear regression in 3D. You are taking 100 points in the xy plane, to each of which is assigned a value z_i (determined by the target function that you defined by taking two NEW random points and generating a line through them). z_i is either +1 or 1. Then, by employing the pseudoinverse process outlined in the lecture, you are finding a 2D PLANE (w0 + w1*x1 + w2 *x2 = z) in the 3D xyz space by linear regression.
The intersection of this regression plane with the xy plane defines your hypothesis g, which mostly separates the N points by their z_i values. You get the equation of this line g by setting the value of z to zero (w0 + w1*x1 + w2 *x2 = 0).
