LFD Book Forum

LFD Book Forum (http://book.caltech.edu/bookforum/index.php)
-   Homework 2 (http://book.caltech.edu/bookforum/forumdisplay.php?f=131)
-   -   Linear Regression Classification HW2 Q5/6 (http://book.caltech.edu/bookforum/showthread.php?t=4494)

KMoff 08-19-2014 11:04 AM

Linear Regression Classification HW2 Q5/6
 
Hello, I am having trouble understanding the procedure for binary classification using linear regression.

For ordinary linear regression, as I understand it, we may compute the weights by taking the product pseudo-inverse matrix and the y-vector. In 2D, the line obtained by linear regression is then y = w0 + w1 * x.

Now for the binary case, instead of using the actual y-coordinate value of a data point, we use its binary classification relative to the target function. In this case, the only difference would be that the y-vector which is to be multiplied by the pseudo-inverse matrix consists only of +1 and -1 values.

However, when I try this I get an hypothesis line nearly perpendicular to the target function.

Could someone please clarify? Thanks

KMoff 08-19-2014 01:53 PM

Re: Linear Regression Classification HW2 Q5/6
 
On further investigation. I now think that the y-vector should be the vector whose elements are sign(wf[0] + wf[1] * x) where the target function is y = wf[0] + wf[1] * x. I now have an E_in of about 0.13.

sandeeps 10-18-2015 08:10 PM

Re: Linear Regression Classification HW2 Q5/6
 
I get average E_in as ~ 0.13, however the answer is shown as [Answer edited out by admin].
What I have done:
  • Generated a random line (target function)
  • Generated 100 random points; xn
  • Computed yn using target function
  • Computed w using the linear regression equation
  • Computed Ein = mean(h(x) != yn) i.e. number of values incorrectly estimated by w
  • Repeated the above steps 1000 times and averaged Ein

What have I done wrong?

yaser 10-18-2015 10:53 PM

Re: Linear Regression Classification HW2 Q5/6
 
Quote:

Originally Posted by sandeeps (Post 12134)
I get average E_in as ~ 0.13, however the answer is shown as [Answer edited out by admin].
What I have done:
  • Generated a random line (target function)
  • Generated 100 random points; xn
  • Computed yn using target function
  • Computed w using the linear regression equation
  • Computed Ein = mean(h(x) != yn) i.e. number of values incorrectly estimated by w
  • Repeated the above steps 1000 times and averaged Ein

What have I done wrong?

The steps seem fine. Can you visually check if the hypothesis line makes sense in a few of the runs?

BTW, if you want to discuss specific answers (chosen or excluded), you need to do so in a thread whose title starts with the warning *ANSWER* per the announcement above.

sandeeps 10-19-2015 12:48 AM

Re: Linear Regression Classification HW2 Q5/6
 
Sorry for posting the answer in the comment.
I had a bug in the target function which was causing erroneous partitioning. Once I fixed it, things started working as expected. Thank you! :)


All times are GMT -7. The time now is 04:32 PM.

Powered by vBulletin® Version 3.8.3
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, and participants in the Learning From Data MOOC by Yaser S. Abu-Mostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.