LFD Book Forum PLA (question 7): inital weight is too weak hint
 Register FAQ Calendar Mark Forums Read

#1
01-20-2013, 02:57 PM
 dobrokot Junior Member Join Date: Jan 2013 Posts: 3
PLA (question 7): inital weight is too weak hint

If I take output of linear regression, I usually have a little (1-5 of 100) points which are misclassified. But it does not help PLA, because first addition Xi to W destabilize "almost good" value of W. I can even invert sign of L.R. output, it doesn't affect number of iterations (despite number of misclassified items on first iteration changes from 1-5 to 95-99, I verified)

But if I use (L.R. output)*N, it does help for PLA, greatly reduces number of iterations.

Is it supposed, that usage of linear regression output as W should greatly change number of iterations? If yes, does it means I should to search for other bugs in my code?

Output of L.R. should be used as is, or should be multiplied to make it "stronger" ?
#2
01-20-2013, 03:12 PM
 sanbt Member Join Date: Jan 2013 Posts: 35
Re: PLA (question 7): inital weight is too weak hint

Quote:
 Originally Posted by dobrokot If I take output of linear regression, I usually have a little (1-5 of 100) points which are misclassified. But it does not help PLA, because first addition Xi to W destabilize "almost good" value of W. I can even invert sign of L.R. output, it doesn't affect number of iterations (despite number of misclassified items on first iteration changes from 1-5 to 95-99, I verified) But if I use (L.R. output)*N, it does help for PLA, greatly reduces number of iterations. Is it supposed, that usage of linear regression output as W should greatly change number of iterations? If yes, does it means I should to search for other bugs in my code? Output of L.R. should be used as is, or should be multiplied to make it "stronger" ?
I think for this question you should set N=10 and just use output of linear regression. It should reduce the number if iterations of PLA. I tried it for N = 100 and agree with you that linear regression doesn't help improve iterations.
#3
01-20-2013, 07:50 PM
 Anne Paulson Senior Member Join Date: Jan 2013 Location: Silicon Valley Posts: 52
Re: PLA (question 7): inital weight is too weak hint

This issue is discussed in another thread. Short answer: scaling the weights by N might make a difference, but for the homework question, we should follow the directions as given and not scale.
#4
04-03-2013, 08:25 PM
 Michael Reach Senior Member Join Date: Apr 2013 Location: Baltimore, Maryland, USA Posts: 71
Re: PLA (question 7): inital weight is too weak hint

Anne! Welcome back! (I was in your Data Analysis course.) Are you like a TA here or something? I know you took this course already.
#5
04-10-2013, 11:46 AM
 Anne Paulson Senior Member Join Date: Jan 2013 Location: Silicon Valley Posts: 52
Re: PLA (question 7): inital weight is too weak hint

Hi Michael!

My post was from the last iteration of the course (look at the date). I'm not taking it this time. I only happened to see your post because I was randomly surfing around. Good luck to all the DA students taking this course now.
#6
04-10-2013, 12:11 PM
 Michael Reach Senior Member Join Date: Apr 2013 Location: Baltimore, Maryland, USA Posts: 71
Re: PLA (question 7): inital weight is too weak hint

Oh, my bad. I did see your exalted Senior Member status and I thought that meant T.A. or such.

 Thread Tools Display Modes Hybrid Mode

 Posting Rules You may not post new threads You may not post replies You may not post attachments You may not edit your posts BB code is On Smilies are On [IMG] code is On HTML code is Off Forum Rules
 Forum Jump User Control Panel Private Messages Subscriptions Who's Online Search Forums Forums Home General     General Discussion of Machine Learning     Free Additional Material         Dynamic e-Chapters         Dynamic e-Appendices Course Discussions     Online LFD course         General comments on the course         Homework 1         Homework 2         Homework 3         Homework 4         Homework 5         Homework 6         Homework 7         Homework 8         The Final         Create New Homework Problems Book Feedback - Learning From Data     General comments on the book     Chapter 1 - The Learning Problem     Chapter 2 - Training versus Testing     Chapter 3 - The Linear Model     Chapter 4 - Overfitting     Chapter 5 - Three Learning Principles     e-Chapter 6 - Similarity Based Methods     e-Chapter 7 - Neural Networks     e-Chapter 8 - Support Vector Machines     e-Chapter 9 - Learning Aides     Appendix and Notation     e-Appendices

All times are GMT -7. The time now is 08:09 AM.