LFD Book Forum PLA Optimization criteria
 User Name Remember Me? Password
 FAQ Calendar Mark Forums Read

 Thread Tools Display Modes
#1
01-19-2013, 12:28 PM
 cygnids Member Join Date: Jan 2013 Posts: 11
PLA Optimization criteria

The PLA algorithm, eqn. 1.3, can be used to partition linearly separable data. What I'm curious is to what optimization criteria underlies eqn. 1.3? The figures on pp. 6-7 show that for a 2D case we have the algorithm converge to some straight line decision boundary, and it is also qualitatively clear that many different straight-lines, would "work" equally well (ie give the same E_{in} error rate); however PLA converges to a specific solution. The PLA algorithm seems to provide both, an optimization criteria, and a method for solution too. The opt. criteria gives provides uniqueness. Can the optimization criteria underlying PLA (eqn 1.3) be spelled out explicitly? Thank you.
__________________
The whole is simpler than the sum of its parts. - Gibbs
#2
01-21-2013, 02:51 PM
 yaser Caltech Join Date: Aug 2009 Location: Pasadena, California, USA Posts: 1,477
Re: PLA Optimization criteria

Quote:
 Originally Posted by cygnids The PLA algorithm, eqn. 1.3, can be used to partition linearly separable data. What I'm curious is to what optimization criteria underlies eqn. 1.3? The figures on pp. 6-7 show that for a 2D case we have the algorithm converge to some straight line decision boundary, and it is also qualitatively clear that many different straight-lines, would "work" equally well (ie give the same E_{in} error rate); however PLA converges to a specific solution. The PLA algorithm seems to provide both, an optimization criteria, and a method for solution too. The opt. criteria gives provides uniqueness. Can the optimization criteria underlying PLA (eqn 1.3) be spelled out explicitly? Thank you.
The optimization criterion for the PLA can be viewed as an application of Stochastic Gradient Descent to a particular error measure (Exercise 3.10). This is really just an artificial way of looking at it. A genuine optimization criterion based on margins leads to support vector machines.
__________________
Where everyone thinks alike, no one thinks very much
#3
01-22-2013, 12:47 PM
 cygnids Member Join Date: Jan 2013 Posts: 11
Re: PLA Optimization criteria

A few weeks ago I recall having read that section on SGD, however the connection with PLA somehow slipped past. My sincere apologies. Then, I suppose I was trying keep my focus on ML paradigms & approaches, and much as optimization is part & parcel of ML, I think I tried not to get sidetracked with finer details of optimization. Lately, I've started re-reading the book, a bit more carefully, and find myself appreciating the whole, and the subtle, even more so than before! Thank you for taking the trouble of pointing out the section. I do appreciate it.
__________________
The whole is simpler than the sum of its parts. - Gibbs

 Thread Tools Display Modes Linear Mode

 Posting Rules You may not post new threads You may not post replies You may not post attachments You may not edit your posts BB code is On Smilies are On [IMG] code is On HTML code is Off Forum Rules
 Forum Jump User Control Panel Private Messages Subscriptions Who's Online Search Forums Forums Home General     General Discussion of Machine Learning     Free Additional Material         Dynamic e-Chapters         Dynamic e-Appendices Course Discussions     Online LFD course         General comments on the course         Homework 1         Homework 2         Homework 3         Homework 4         Homework 5         Homework 6         Homework 7         Homework 8         The Final         Create New Homework Problems Book Feedback - Learning From Data     General comments on the book     Chapter 1 - The Learning Problem     Chapter 2 - Training versus Testing     Chapter 3 - The Linear Model     Chapter 4 - Overfitting     Chapter 5 - Three Learning Principles     e-Chapter 6 - Similarity Based Methods     e-Chapter 7 - Neural Networks     e-Chapter 8 - Support Vector Machines     e-Chapter 9 - Learning Aides     Appendix and Notation     e-Appendices

All times are GMT -7. The time now is 10:38 PM.

 Contact Us - LFD Book - Top