#1




HW 2 Problem 6
How is this different from problem 5 other than N=1000 and the fact that these simulated 'out of sample' points (E_out) are generated fresh ? I may be missing something but it seems to boil down to running the same program as in problem 5 with N=1000 for 1000 times; can someone clarify please ? thanks

#2




Re: HW 2 Problem 6
Quote:
Problem 5 asks about while Problem 6 asks about (an estimate of) . In both problems, ( stands for the number of training examples in our notation).
__________________
Where everyone thinks alike, no one thinks very much 
#3




Re: HW 2 Problem 6
It is my understanding that "fresh data" refers to crossvalidation data. Do we then compute Eout using the weights obtained in problem 5? When I do this, Eout < Ein. When I design the weights using the fresh data, Eout is approximately equal to Ein. Does this makes sense?

#4




Re: HW 2 Problem 6
Quote:
The final hypothesis is indeed the one whose weights were determined in Problem 5, where the training took place.
__________________
Where everyone thinks alike, no one thinks very much 
#5




Re: HW 2 Problem 6
I am confused here , I don't understand what is final hypothesis here.
There are 1000 target function and corresponding 1000 weight vectors/hypothesis in problem 5 . So for problem 6 , 1000 times I generate 1000 outofsample data and then for each weight vector and target function(from problem 5) I evaluate E_out for that outofsample data and finally average them. This is how I have done. I don't see final hypothesis here , what I am missing , any hint Could it be that in problem 5 there is supposed to be only one target function and many insample data ? If so then the final hypothesis/weights could be that produces minimum insample error E_in . Please clarify. Thanks a lot. 
#6




Re: HW 2 Problem 6
Quote:
__________________
Where everyone thinks alike, no one thinks very much 
#7




Re: HW 2 Problem 6
Thanks a lot. The statements about (i) N being the number of 'insample' training data in both problems and (ii) the freshly generated 1000 points being disjoint from the first set clarified the confusion I had.

#8




Re: HW 2 Problem 6
Thanks Professor yaser.

#9




Re: HW 2 Problem 6
When I generate new data and hypothesis for every single run of 1000 (as the problem suggests) I get stable outofsample result close to (slightly greater than) insample error.
When I estimate 1000 different outofsamples for one insample and single hypothesis I get very different average error rates with high variability from 0.01 to 0.13 Why so? 
#10




Re: HW 2 Problem 6
Quote:
__________________
Where everyone thinks alike, no one thinks very much 
Thread Tools  
Display Modes  

