LFD Book Forum (http://book.caltech.edu/bookforum/index.php)
-   Homework 2 (http://book.caltech.edu/bookforum/forumdisplay.php?f=131)
-   -   Linear Regression Classification HW2 Q5/6 (http://book.caltech.edu/bookforum/showthread.php?t=4494)

 KMoff 08-19-2014 10:04 AM

Linear Regression Classification HW2 Q5/6

Hello, I am having trouble understanding the procedure for binary classification using linear regression.

For ordinary linear regression, as I understand it, we may compute the weights by taking the product pseudo-inverse matrix and the y-vector. In 2D, the line obtained by linear regression is then y = w0 + w1 * x.

Now for the binary case, instead of using the actual y-coordinate value of a data point, we use its binary classification relative to the target function. In this case, the only difference would be that the y-vector which is to be multiplied by the pseudo-inverse matrix consists only of +1 and -1 values.

However, when I try this I get an hypothesis line nearly perpendicular to the target function.

 KMoff 08-19-2014 12:53 PM

Re: Linear Regression Classification HW2 Q5/6

On further investigation. I now think that the y-vector should be the vector whose elements are sign(wf[0] + wf[1] * x) where the target function is y = wf[0] + wf[1] * x. I now have an E_in of about 0.13.

 sandeeps 10-18-2015 07:10 PM

Re: Linear Regression Classification HW2 Q5/6

I get average E_in as ~ 0.13, however the answer is shown as [Answer edited out by admin].
What I have done:
• Generated a random line (target function)
• Generated 100 random points; xn
• Computed yn using target function
• Computed w using the linear regression equation
• Computed Ein = mean(h(x) != yn) i.e. number of values incorrectly estimated by w
• Repeated the above steps 1000 times and averaged Ein

What have I done wrong?

 yaser 10-18-2015 09:53 PM

Re: Linear Regression Classification HW2 Q5/6

Quote:
 Originally Posted by sandeeps (Post 12134) I get average E_in as ~ 0.13, however the answer is shown as [Answer edited out by admin]. What I have done:Generated a random line (target function) Generated 100 random points; xn Computed yn using target function Computed w using the linear regression equation Computed Ein = mean(h(x) != yn) i.e. number of values incorrectly estimated by w Repeated the above steps 1000 times and averaged Ein What have I done wrong?
The steps seem fine. Can you visually check if the hypothesis line makes sense in a few of the runs?

BTW, if you want to discuss specific answers (chosen or excluded), you need to do so in a thread whose title starts with the warning *ANSWER* per the announcement above.

 sandeeps 10-18-2015 11:48 PM

Re: Linear Regression Classification HW2 Q5/6

Sorry for posting the answer in the comment.
I had a bug in the target function which was causing erroneous partitioning. Once I fixed it, things started working as expected. Thank you! :)

 All times are GMT -7. The time now is 06:52 PM.