LFD Book Forum

LFD Book Forum (http://book.caltech.edu/bookforum/index.php)
-   Homework 1 (http://book.caltech.edu/bookforum/forumdisplay.php?f=130)
-   -   PLA: Let share nice visualizations (http://book.caltech.edu/bookforum/showthread.php?t=3831)

clarkfitzg 04-06-2013 10:39 PM

Re: PLA: Let share nice visualizations
 
http://s23.postimg.org/98thps8dn/Perceptron_page_1.jpg

The black line is f and the green line is g.

chiraz 04-07-2013 05:46 AM

Re: PLA: Let share nice visualizations
 
Quote:

Originally Posted by theTB (Post 8682)
Here is a visualization of the algorithm with n=50 points. The point in the circle is the chosen misclassified point in the current iteration. The final yellow line is the selected hypothesis.



@theTB, cool animation, what language did you use for this animation?
I usually use Matlab but haven't attempted doing animations there..

Chiraz BenAbdelkader

chiraz 04-07-2013 05:54 AM

Re: PLA: Let share nice visualizations
 
@Barath, I like to look of your plot; what software did you use? I use Matlab and my plots are much more lame looking!

Thanks,
Chiraz BenAbdelkader

OlivierB 04-07-2013 06:45 AM

Re: PLA: Let share nice visualizations
 
I would share my graph too, though it is exactly as expected. But impossible to upload an image... No trace of the 'Managed Attachment' button mentioned in the FAQ..
And I read 'You may not post attachments' at the bottom of my page..
I would be grateful for some help on this uninteresting topic.

OlivierB 04-07-2013 07:02 AM

Re: PLA: Let share nice visualizations
 
I cannot upload images to the forum, maybe because i read 'You may not post attachments' at the bottom of my screen.... Please help.

clarkfitzg 04-07-2013 01:37 PM

Re: PLA: Let share nice visualizations
 
Quote:

Originally Posted by OlivierB (Post 10197)
I cannot upload images to the forum, maybe because i read 'You may not post attachments' at the bottom of my screen.... Please help.

Use an image hosting service like http://postimage.org/. Click on the image button and paste the URL to your image.

OlivierB 04-07-2013 04:31 PM

Re: PLA: Let share nice visualizations
 
@Clarkfitzg, thx !

Here is another graph of the PLA (N=50), plotted with octave.

http://s18.postimg.org/p9b7g9qex/PLA_convergence1.png

Remark: In terms of convergence speed, for 1000 runs, as requested in questions 7,9, I find surprisingly different results from one trial (1000 runs) to the next (I use the rand() from octave). In my view, the difference makes it debatable as to which answer is "correct" (since only one is correct officially). I would tick 2 letters if I could. Am I the only one facing this issue ? Have I made some obvious mistake ? Or is it just a consequence of the imperfect multi choice form of the homework (Let nobody misunderstand me, I find the subject very interesting, the course VERY well presented, and the professor passionate and VERY engaging. By the way THANK YOU for putting your class online ! :bow:) ?

yaser 04-07-2013 04:48 PM

Re: PLA: Let share nice visualizations
 
Quote:

Originally Posted by OlivierB (Post 10210)
Remark: In terms of convergence speed, for 1000 runs, as requested in questions 7,9, I find surprisingly different results from one trial (1000 runs) to the next (I use the rand() from octave).

Hi,

Do your 1000 runs have different target functions as well as training data?

OlivierB 04-08-2013 01:37 PM

Re: PLA: Let share nice visualizations
 
Oh ! Thank you for clarifying this point.

Actually no, I was randomly choosing one f and training set, and then running 1000 PLA on these.
The variation in convergence speed were coming from the randomly selected misclassified point at each step.

Indeed, on re-reading hw1.pdf more carefully, I see that you mention "In each run, choose a random line in the plane as your target function...".

Now for each of the 1000 runs, I randomly choose a single f and training set and then run on PLA on them.
The convergence speed is now very stable, and the questions 7 and 9 do not seem ambiguous any longer.

As I already have submitted, I know that I probably get the correct results for questions 7, 8, and 10. But not for question 9! So there is still something puzzling... If by experience you have an idea about my probable mistake, I would be grateful for your insight (In fact, I already am for your first answer).

Elroch 04-08-2013 03:19 PM

Re: PLA: Let share nice visualizations
 
Quote:

Originally Posted by OlivierB (Post 10235)
Oh ! Thank you for clarifying this point.

Actually no, I was randomly choosing one f and training set, and then running 1000 PLA on these.
The variation in convergence speed were coming from the randomly selected misclassified point at each step.

Indeed, on re-reading hw1.pdf more carefully, I see that you mention "In each run, choose a random line in the plane as your target function...".

Now for each of the 1000 runs, I randomly choose a single f and training set and then run on PLA on them.
The convergence speed is now very stable, and the questions 7 and 9 do not seem ambiguous any longer.

As I already have submitted, I know that I probably get the correct results for questions 7, 8, and 10. But not for question 9! So there is still something puzzling... If by experience you have an idea about my probable mistake, I would be grateful for your insight (In fact, I already am for your first answer).

There is a quantitative indication of one part of the variation in the number of iterations in the proof of convergence of the PLA algorithm in Problem 1.3 in the book. In some problems (i.e. two separable sets of points), a much larger range of lines will perform the classification than in others. If one randomly picked lines, the expected time to converge would vary in a predictable manner between problems. Difficult problems for such a random line picking algorithm (if it deserves the term) are also ones with a loose bound on convergence using PLA.

eg) a 100 point problem can be relatively easy sometimes.
http://i699.photobucket.com/albums/v...a/PLA100-1.png


All times are GMT -7. The time now is 02:09 AM.

Powered by vBulletin® Version 3.8.3
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.
The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, and participants in the Learning From Data MOOC by Yaser S. Abu-Mostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.