LFD Book Forum  

Go Back   LFD Book Forum > Course Discussions > Online LFD course > Homework 1

Reply
 
Thread Tools Display Modes
  #1  
Old 07-12-2012, 04:21 PM
hesam.creative hesam.creative is offline
Junior Member
 
Join Date: Jul 2012
Posts: 7
Default I think Q9 average number of irritations is related to implementation of PLA?!

Hi every one!
I was wondering that the average number of irritations is kinda related to the way you exploit the PLA.Am I clear?
regards.
Reply With Quote
  #2  
Old 07-12-2012, 06:50 PM
yaser's Avatar
yaser yaser is offline
Caltech
 
Join Date: Aug 2009
Location: Pasadena, California, USA
Posts: 1,477
Default Re: I think Q9 average number of irritations is related to implementation of PLA?!

Quote:
Originally Posted by hesam.creative View Post
Hi every one!
I was wondering that the average number of irritations is kinda related to the way you exploit the PLA.Am I clear?
The homework specifies:

"Start the PLA with the weight vector {\bf w} being all zeros, and at each iteration have the algorithm choose a point randomly from the set of misclassi ed points."
__________________
Where everyone thinks alike, no one thinks very much
Reply With Quote
  #3  
Old 07-13-2012, 08:50 AM
hesam.creative hesam.creative is offline
Junior Member
 
Join Date: Jul 2012
Posts: 7
Default Re: I think Q9 average number of irritations is related to implementation of PLA?!

Thanks professor,I think I have read the old version.I came up whit this idea:
Is it justified to compare Q7 through 10 to the probabilistic model in lecture #2,bin and marbles,say,all the points in [1,1]*[1,-1] as all marbles in bin,random points as random green and red marbles(a sample drawn from bin).p(f(x)~=g(x)) as μ. however,in this model f is determined,while target function is never known to us.Moreover,M is infinite;however,this is related to generalization theory.
Reply With Quote
  #4  
Old 07-13-2012, 11:31 AM
yaser's Avatar
yaser yaser is offline
Caltech
 
Join Date: Aug 2009
Location: Pasadena, California, USA
Posts: 1,477
Default Re: I think Q9 average number of irritations is related to implementation of PLA?!

Quote:
Originally Posted by hesam.creative View Post
Is it justified to compare Q7 through 10 to the probabilistic model in lecture #2,bin and marbles,say,all the points in [1,1]*[1,-1] as all marbles in bin,random points as random green and red marbles(a sample drawn from bin).p(f(x)~=g(x)) as μ. however,in this model f is determined,while target function is never known to us.Moreover,M is infinite;however,this is related to generalization theory.
The theory will indeed address this question in detail in Lectures 5-7, and the perceptron case in particular will be analyzed in Lecture 7.
__________________
Where everyone thinks alike, no one thinks very much
Reply With Quote
  #5  
Old 07-14-2012, 05:54 PM
data_user data_user is offline
Junior Member
 
Join Date: Jul 2012
Posts: 6
Default Re: I think Q9 average number of irritations is related to implementation of PLA?!

Dear Yaser,

Thanks a lot for your very interesting and useful course.
I have a question on HW1,Q9.

I implemented PLA. The implementation looks correct: I answered Q7,Q8, and Q10 correctly. More importantly, I can see from a visualization (see below) that the PLA does its job. Here, dots are data, green line corresponds to the target function f, and 1000 yellow lines correspond to g_i, where i=1:1000.



In the first case, the average # of iterations is k1 (relatively small), in the second case it is k2 (relatively large), and k2 is approximately equal to 10*k1. Based on the first case, I had to chose one out 5 possible answers, while, based on the second case, I had to chose another. My (submitted) choice turned out to be wrong.

On the other hand, from the above figures, it is intuitively clear that to find g in the 2nd case is more difficult (yellow area is smaller), and, therefore, it should take more iterations for the PLA to converge. In general, isn't it true that the number of iterations significantly depends on the data? Say, it could be k but also it could be 10*k, depending on the sample x1,...,xN?

Am I missing something?
Reply With Quote
  #6  
Old 07-14-2012, 08:28 PM
yaser's Avatar
yaser yaser is offline
Caltech
 
Join Date: Aug 2009
Location: Pasadena, California, USA
Posts: 1,477
Default Re: I think Q9 average number of irritations is related to implementation of PLA?!

Quote:
Originally Posted by data_user View Post
In general, isn't it true that the number of iterations significantly depends on the data? Say, it could be k but also it could be 10*k, depending on the sample x1,...,xN?
You are correct. The number of iterations does depend on the sample. The averaging over a large number of randomly generated samples should take care of these variations, though. Did you get two different answers after averaging over two sets of samples each generated according the the problem specs?

BTW, as alluded to in the preamble of the homework, the goal of the question is to make sure that you go through the experiment, analyze it, and understand the algorithm well. The plots you included in your post look great and they show that the goal was achieved. Having said that, the answer in the solution key should certainly be the correct answer.
__________________
Where everyone thinks alike, no one thinks very much
Reply With Quote
  #7  
Old 07-15-2012, 04:08 AM
data_user data_user is offline
Junior Member
 
Join Date: Jul 2012
Posts: 6
Default Re: I think Q9 average number of irritations is related to implementation of PLA?!

Quote:
Originally Posted by yaser View Post
The averaging over a large number of randomly generated samples should take care of these variations, though. Did you get two different answers after averaging over two sets of samples each generated according the the problem specs?
Ah... I see my mistake now. I generated data x1,...,xn (uniformly on [-1,1]^2) and I fixed it forever. Then, I applied the PLA 1000 times to learn from the fixed data. (In every run, at each iteration, the PLA chooses the misclassified point randomly). In this way I obtain figure 1. Next, I generated new data x1',...,xn', fixed it, and applied the PLA 1000 times again. This resulted into figure 2.

Instead of fixing the sample, I should have generated new data for each PLA run!

Thank you! Looking forward to solving HW2
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -7. The time now is 08:38 PM.


Powered by vBulletin® Version 3.8.3
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, and participants in the Learning From Data MOOC by Yaser S. Abu-Mostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.