LFD Book Forum

LFD Book Forum (http://book.caltech.edu/bookforum/index.php)
-   Homework 1 (http://book.caltech.edu/bookforum/forumdisplay.php?f=130)
-   -   I think Q9 average number of irritations is related to implementation of PLA?! (http://book.caltech.edu/bookforum/showthread.php?t=846)

hesam.creative 07-12-2012 03:21 PM

I think Q9 average number of irritations is related to implementation of PLA?!
 
Hi every one!
I was wondering that the average number of irritations is kinda related to the way you exploit the PLA.Am I clear? :clueless:
regards.

yaser 07-12-2012 05:50 PM

Re: I think Q9 average number of irritations is related to implementation of PLA?!
 
Quote:

Originally Posted by hesam.creative (Post 3389)
Hi every one!
I was wondering that the average number of irritations is kinda related to the way you exploit the PLA.Am I clear?

The homework specifies:

"Start the PLA with the weight vector {\bf w} being all zeros, and at each iteration have the algorithm choose a point randomly from the set of misclassi ed points."

hesam.creative 07-13-2012 07:50 AM

Re: I think Q9 average number of irritations is related to implementation of PLA?!
 
Thanks professor,I think I have read the old version.I came up whit this idea:
Is it justified to compare Q7 through 10 to the probabilistic model in lecture #2,bin and marbles,say,all the points in [1,1]*[1,-1] as all marbles in bin,random points as random green and red marbles(a sample drawn from bin).p(f(x)~=g(x)) as μ. however,in this model f is determined,while target function is never known to us.Moreover,M is infinite;however,this is related to generalization theory.

yaser 07-13-2012 10:31 AM

Re: I think Q9 average number of irritations is related to implementation of PLA?!
 
Quote:

Originally Posted by hesam.creative (Post 3395)
Is it justified to compare Q7 through 10 to the probabilistic model in lecture #2,bin and marbles,say,all the points in [1,1]*[1,-1] as all marbles in bin,random points as random green and red marbles(a sample drawn from bin).p(f(x)~=g(x)) as μ. however,in this model f is determined,while target function is never known to us.Moreover,M is infinite;however,this is related to generalization theory.

The theory will indeed address this question in detail in Lectures 5-7, and the perceptron case in particular will be analyzed in Lecture 7.

data_user 07-14-2012 04:54 PM

Re: I think Q9 average number of irritations is related to implementation of PLA?!
 
Dear Yaser,

Thanks a lot for your very interesting and useful course.
I have a question on HW1,Q9.

I implemented PLA. The implementation looks correct: I answered Q7,Q8, and Q10 correctly. More importantly, I can see from a visualization (see below) that the PLA does its job. Here, dots are data, green line corresponds to the target function f, and 1000 yellow lines correspond to g_i, where i=1:1000.

http://s8.postimage.org/t3je98nv9/PLA.jpghttp://s13.postimage.org/mlhiubdif/PLA2.jpg

In the first case, the average # of iterations is k1 (relatively small), in the second case it is k2 (relatively large), and k2 is approximately equal to 10*k1. Based on the first case, I had to chose one out 5 possible answers, while, based on the second case, I had to chose another. My (submitted) choice turned out to be wrong.

On the other hand, from the above figures, it is intuitively clear that to find g in the 2nd case is more difficult (yellow area is smaller), and, therefore, it should take more iterations for the PLA to converge. In general, isn't it true that the number of iterations significantly depends on the data? Say, it could be k but also it could be 10*k, depending on the sample x1,...,xN?

Am I missing something?

yaser 07-14-2012 07:28 PM

Re: I think Q9 average number of irritations is related to implementation of PLA?!
 
Quote:

Originally Posted by data_user (Post 3403)
In general, isn't it true that the number of iterations significantly depends on the data? Say, it could be k but also it could be 10*k, depending on the sample x1,...,xN?

You are correct. The number of iterations does depend on the sample. The averaging over a large number of randomly generated samples should take care of these variations, though. Did you get two different answers after averaging over two sets of samples each generated according the the problem specs?

BTW, as alluded to in the preamble of the homework, the goal of the question is to make sure that you go through the experiment, analyze it, and understand the algorithm well. The plots you included in your post look great and they show that the goal was achieved. Having said that, the answer in the solution key should certainly be the correct answer.

data_user 07-15-2012 03:08 AM

Re: I think Q9 average number of irritations is related to implementation of PLA?!
 
Quote:

Originally Posted by yaser (Post 3404)
The averaging over a large number of randomly generated samples should take care of these variations, though. Did you get two different answers after averaging over two sets of samples each generated according the the problem specs?

Ah... I see my mistake now. I generated data x1,...,xn (uniformly on [-1,1]^2) and I fixed it forever. Then, I applied the PLA 1000 times to learn from the fixed data. (In every run, at each iteration, the PLA chooses the misclassified point randomly). In this way I obtain figure 1. Next, I generated new data x1',...,xn', fixed it, and applied the PLA 1000 times again. This resulted into figure 2.

Instead of fixing the sample, I should have generated new data for each PLA run!

Thank you! Looking forward to solving HW2 :)


All times are GMT -7. The time now is 12:32 PM.

Powered by vBulletin® Version 3.8.3
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.
The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, and participants in the Learning From Data MOOC by Yaser S. Abu-Mostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.