LFD Book Forum Perceptron Learning Algorithm
 Register FAQ Calendar Mark Forums Read

#41
04-22-2012, 01:40 PM
 jmknapp Member Join Date: Apr 2012 Location: Columbus OH USA Posts: 14
Re: Perceptron Learning Algorithm

How is the PLA generalized to handling multi-category classification, e.g., the problem of classifying coins?
#42
04-22-2012, 07:51 PM
 htlin NTU Join Date: Aug 2009 Location: Taipei, Taiwan Posts: 601
Re: Perceptron Learning Algorithm

Quote:
 Originally Posted by jmknapp How is the PLA generalized to handling multi-category classification, e.g., the problem of classifying coins?
Interesting question. There are general techniques for extending from binary classification to -category one. For instance, one simple approach (one-versus-all decomposition) is to form the following yes/no questions: "Does an example belong to category k or not?" If the machine learns hypotheses that answer each of the questions correctly, you can combine the answers to form a multi-category prediction. Hope this helps.
__________________
When one teaches, two learn.
#43
04-22-2012, 07:53 PM
 htlin NTU Join Date: Aug 2009 Location: Taipei, Taiwan Posts: 601
Re: Perceptron Learning Algorithm

Quote:
 Originally Posted by zsero There is a point what I think is not clearly explained. It took me a long time to realize that it's not true that exactly one point gets corrected in each step. The true statement is that at most one point gets corrected in each step.
The "at most" part may also be tricky, because you may be using point A for correction but "happens" to correct B and C by moving in a good direction.
__________________
When one teaches, two learn.
#44
04-22-2012, 08:01 PM
 htlin NTU Join Date: Aug 2009 Location: Taipei, Taiwan Posts: 601
Re: Perceptron Learning Algorithm

Quote:
 Originally Posted by shockwavephysics I have been trying to figure out why updating using w -> w + y_n * x_n works at all. I looked up the relevant section in the text, and there are a series of questions for the student that hint at the answer. I followed that logic to it's conclusion and it does seem to show that updating in that way will always give a w that is better (for the misclassified point) than the previous w. However, I cannot figure out how one comes up with this formulation in the first place. Is there a reference to a derivation I can read?
You can read Problem 1.3 of the recommended textbook, which guides you through a simple proof. Roughly speaking, the proof says the PLA weights get more aligned with the underlying "target weights" after each update.
__________________
When one teaches, two learn.
#45
04-23-2012, 06:21 PM
 jmknapp Member Join Date: Apr 2012 Location: Columbus OH USA Posts: 14
Re: Perceptron Learning Algorithm

Thanks--the textbook should be arriving from Amazon tomorrow.
#46
10-10-2012, 09:51 PM
 ken47 Junior Member Join Date: Oct 2012 Posts: 3
Re: Perceptron Learning Algorithm

I adapted an implementation of the single-layer perceptron for Octave (open-source Matlab) that I found online. It runs incredibly slow compared to my nodeJS implementation, but one can see the progress graphically which I think is really cool. Gist attached.

http://gist.github.com/3870213.git
#47
01-12-2013, 02:18 AM
 foodcomazzz Junior Member Join Date: Jan 2013 Posts: 5
Re: Perceptron Learning Algorithm

Thanks so much prof. This thread is very detailed and useful!

 Thread Tools Display Modes Linear Mode

 Posting Rules You may not post new threads You may not post replies You may not post attachments You may not edit your posts BB code is On Smilies are On [IMG] code is On HTML code is Off Forum Rules
 Forum Jump User Control Panel Private Messages Subscriptions Who's Online Search Forums Forums Home General     General Discussion of Machine Learning     Free Additional Material         Dynamic e-Chapters         Dynamic e-Appendices Course Discussions     Online LFD course         General comments on the course         Homework 1         Homework 2         Homework 3         Homework 4         Homework 5         Homework 6         Homework 7         Homework 8         The Final         Create New Homework Problems Book Feedback - Learning From Data     General comments on the book     Chapter 1 - The Learning Problem     Chapter 2 - Training versus Testing     Chapter 3 - The Linear Model     Chapter 4 - Overfitting     Chapter 5 - Three Learning Principles     e-Chapter 6 - Similarity Based Methods     e-Chapter 7 - Neural Networks     e-Chapter 8 - Support Vector Machines     e-Chapter 9 - Learning Aides     Appendix and Notation     e-Appendices

All times are GMT -7. The time now is 10:13 AM.