LFD Book Forum  

Go Back   LFD Book Forum > Course Discussions > Online LFD course > Homework 1

Reply
 
Thread Tools Display Modes
  #11  
Old 04-06-2013, 10:39 PM
clarkfitzg clarkfitzg is offline
Junior Member
 
Join Date: Apr 2013
Posts: 6
Default Re: PLA: Let share nice visualizations



The black line is f and the green line is g.
Reply With Quote
  #12  
Old 04-07-2013, 05:46 AM
chiraz chiraz is offline
Member
 
Join Date: Apr 2013
Location: Tunis, Tunisia
Posts: 12
Default Re: PLA: Let share nice visualizations

Quote:
Originally Posted by theTB View Post
Here is a visualization of the algorithm with n=50 points. The point in the circle is the chosen misclassified point in the current iteration. The final yellow line is the selected hypothesis.


@theTB, cool animation, what language did you use for this animation?
I usually use Matlab but haven't attempted doing animations there..

Chiraz BenAbdelkader
Reply With Quote
  #13  
Old 04-07-2013, 05:54 AM
chiraz chiraz is offline
Member
 
Join Date: Apr 2013
Location: Tunis, Tunisia
Posts: 12
Default Re: PLA: Let share nice visualizations

@Barath, I like to look of your plot; what software did you use? I use Matlab and my plots are much more lame looking!

Thanks,
Chiraz BenAbdelkader
Reply With Quote
  #14  
Old 04-07-2013, 06:45 AM
OlivierB OlivierB is offline
Member
 
Join Date: Apr 2013
Location: Paris
Posts: 16
Default Re: PLA: Let share nice visualizations

I would share my graph too, though it is exactly as expected. But impossible to upload an image... No trace of the 'Managed Attachment' button mentioned in the FAQ..
And I read 'You may not post attachments' at the bottom of my page..
I would be grateful for some help on this uninteresting topic.
Reply With Quote
  #15  
Old 04-07-2013, 07:02 AM
OlivierB OlivierB is offline
Member
 
Join Date: Apr 2013
Location: Paris
Posts: 16
Default Re: PLA: Let share nice visualizations

I cannot upload images to the forum, maybe because i read 'You may not post attachments' at the bottom of my screen.... Please help.
Reply With Quote
  #16  
Old 04-07-2013, 01:37 PM
clarkfitzg clarkfitzg is offline
Junior Member
 
Join Date: Apr 2013
Posts: 6
Default Re: PLA: Let share nice visualizations

Quote:
Originally Posted by OlivierB View Post
I cannot upload images to the forum, maybe because i read 'You may not post attachments' at the bottom of my screen.... Please help.
Use an image hosting service like http://postimage.org/. Click on the image button and paste the URL to your image.
Reply With Quote
  #17  
Old 04-07-2013, 04:31 PM
OlivierB OlivierB is offline
Member
 
Join Date: Apr 2013
Location: Paris
Posts: 16
Default Re: PLA: Let share nice visualizations

@Clarkfitzg, thx !

Here is another graph of the PLA (N=50), plotted with octave.



Remark: In terms of convergence speed, for 1000 runs, as requested in questions 7,9, I find surprisingly different results from one trial (1000 runs) to the next (I use the rand() from octave). In my view, the difference makes it debatable as to which answer is "correct" (since only one is correct officially). I would tick 2 letters if I could. Am I the only one facing this issue ? Have I made some obvious mistake ? Or is it just a consequence of the imperfect multi choice form of the homework (Let nobody misunderstand me, I find the subject very interesting, the course VERY well presented, and the professor passionate and VERY engaging. By the way THANK YOU for putting your class online ! ) ?
Reply With Quote
  #18  
Old 04-07-2013, 04:48 PM
yaser's Avatar
yaser yaser is offline
Caltech
 
Join Date: Aug 2009
Location: Pasadena, California, USA
Posts: 1,477
Default Re: PLA: Let share nice visualizations

Quote:
Originally Posted by OlivierB View Post
Remark: In terms of convergence speed, for 1000 runs, as requested in questions 7,9, I find surprisingly different results from one trial (1000 runs) to the next (I use the rand() from octave).
Hi,

Do your 1000 runs have different target functions as well as training data?
__________________
Where everyone thinks alike, no one thinks very much
Reply With Quote
  #19  
Old 04-08-2013, 01:37 PM
OlivierB OlivierB is offline
Member
 
Join Date: Apr 2013
Location: Paris
Posts: 16
Default Re: PLA: Let share nice visualizations

Oh ! Thank you for clarifying this point.

Actually no, I was randomly choosing one f and training set, and then running 1000 PLA on these.
The variation in convergence speed were coming from the randomly selected misclassified point at each step.

Indeed, on re-reading hw1.pdf more carefully, I see that you mention "In each run, choose a random line in the plane as your target function...".

Now for each of the 1000 runs, I randomly choose a single f and training set and then run on PLA on them.
The convergence speed is now very stable, and the questions 7 and 9 do not seem ambiguous any longer.

As I already have submitted, I know that I probably get the correct results for questions 7, 8, and 10. But not for question 9! So there is still something puzzling... If by experience you have an idea about my probable mistake, I would be grateful for your insight (In fact, I already am for your first answer).
Reply With Quote
  #20  
Old 04-08-2013, 03:19 PM
Elroch Elroch is offline
Invited Guest
 
Join Date: Mar 2013
Posts: 143
Smile Re: PLA: Let share nice visualizations

Quote:
Originally Posted by OlivierB View Post
Oh ! Thank you for clarifying this point.

Actually no, I was randomly choosing one f and training set, and then running 1000 PLA on these.
The variation in convergence speed were coming from the randomly selected misclassified point at each step.

Indeed, on re-reading hw1.pdf more carefully, I see that you mention "In each run, choose a random line in the plane as your target function...".

Now for each of the 1000 runs, I randomly choose a single f and training set and then run on PLA on them.
The convergence speed is now very stable, and the questions 7 and 9 do not seem ambiguous any longer.

As I already have submitted, I know that I probably get the correct results for questions 7, 8, and 10. But not for question 9! So there is still something puzzling... If by experience you have an idea about my probable mistake, I would be grateful for your insight (In fact, I already am for your first answer).
There is a quantitative indication of one part of the variation in the number of iterations in the proof of convergence of the PLA algorithm in Problem 1.3 in the book. In some problems (i.e. two separable sets of points), a much larger range of lines will perform the classification than in others. If one randomly picked lines, the expected time to converge would vary in a predictable manner between problems. Difficult problems for such a random line picking algorithm (if it deserves the term) are also ones with a loose bound on convergence using PLA.

eg) a 100 point problem can be relatively easy sometimes.

Last edited by Elroch; 04-08-2013 at 03:44 PM. Reason: Added illustrative example
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -7. The time now is 08:06 PM.


Powered by vBulletin® Version 3.8.3
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, and participants in the Learning From Data MOOC by Yaser S. Abu-Mostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.