LFD Book Forum Ques 16-18 How to find Ein?

#1
06-10-2012, 01:58 AM
 vmathur87 Junior Member Join Date: May 2012 Posts: 5
Ques 16-18 How to find Ein?

Sorry if my question is naive, but how do I find Ein for regular RBF? I got the weight vector w (of length k+1), but what do I do with it? I know I am missing something simple, but I just can't figure this out. Thanks for any help in advance!
#2
06-10-2012, 03:09 AM
 yaser Caltech Join Date: Aug 2009 Location: Pasadena, California, USA Posts: 1,476
Re: Ques 17-19 How to find Ein?

Quote:
 Originally Posted by vmathur87 Sorry if my question is naive, but how do I find Ein for regular RBF? I got the weight vector w (of length k+1), but what do I do with it? I know I am missing something simple, but I just can't figure this out. Thanks for any help in advance!
You have the weights so you have the formula for the hypothesis that RBF is implementing. All you need to do is apply it to each input in the data set and check if the value it returns disagrees with the corresponding output. The fraction of training data points on which it disagrees will be .
__________________
Where everyone thinks alike, no one thinks very much
#3
06-10-2012, 08:02 AM
 vmathur87 Junior Member Join Date: May 2012 Posts: 5
Re: Ques 17-19 How to find Ein?

Thanks, I understand that. But do you mean to apply the weights to each input according to the formula given on slide 14 of lecture 16? I think that since that formula was used to derive the weights (using phi), the weights would always satisfy it and the Ein will always be 0. Maybe this doesbn't make too much sene, but I am a little muddled up here.
#4
06-10-2012, 08:42 AM
 kkkkk Invited Guest Join Date: Mar 2012 Posts: 71
Re: Ques 17-19 How to find Ein?

Please see slide 19 of lecture 16 which shows h(x) for regular rbf with the bias b (not part of the summation, b is w0).
#5
06-11-2012, 07:25 AM
 vmathur87 Junior Member Join Date: May 2012 Posts: 5
Re: Ques 17-19 How to find Ein?

Thanks, I got it!
#6
06-06-2013, 05:39 PM
 alasdairj Member Join Date: Mar 2013 Posts: 12
Re: Ques 17-19 How to find Ein?

OK, I am a little confused. The RBF description does not mention a bias, but then it is introduced in slide 19 of lecture 16... how do we learn the bias?
#7
06-06-2013, 06:48 PM
 yaser Caltech Join Date: Aug 2009 Location: Pasadena, California, USA Posts: 1,476
Re: Ques 17-19 How to find Ein?

Quote:
 Originally Posted by alasdairj OK, I am a little confused. The RBF description does not mention a bias, but then it is introduced in slide 19 of lecture 16... how do we learn the bias?
The bias can be learned along with the weights using the pseudo-inverse method. Just consider the bias a parameter like the weights which happens to be multiplied by the constant 1, the same way was handled in linear regression.
__________________
Where everyone thinks alike, no one thinks very much
#8
06-07-2013, 05:05 AM
 alasdairj Member Join Date: Mar 2013 Posts: 12
Re: Ques 17-19 How to find Ein?

Thanks! I implemented this for the pseudo-inverse calculation, and also in the bias in the final calculation of h. What surprised me was that the Ein got slightly worse rather than better, which is not what I expected. Or do I have an implementation issue?
#9
06-07-2013, 11:38 AM
 yaser Caltech Join Date: Aug 2009 Location: Pasadena, California, USA Posts: 1,476
Re: Ques 17-19 How to find Ein?

Quote:
 Originally Posted by alasdairj Thanks! I implemented this for the pseudo-inverse calculation, and also in the bias in the final calculation of h. What surprised me was that the Ein got slightly worse rather than better, which is not what I expected. Or do I have an implementation issue?
In-sample error should not go up with the added parameter (out-of-sample error might), so I suspect a bug of some sort.
__________________
Where everyone thinks alike, no one thinks very much

 Thread Tools Display Modes Linear Mode

 Posting Rules You may not post new threads You may not post replies You may not post attachments You may not edit your posts BB code is On Smilies are On [IMG] code is On HTML code is Off Forum Rules
 Forum Jump User Control Panel Private Messages Subscriptions Who's Online Search Forums Forums Home General     General Discussion of Machine Learning     Free Additional Material         Dynamic e-Chapters         Dynamic e-Appendices Course Discussions     Online LFD course         General comments on the course         Homework 1         Homework 2         Homework 3         Homework 4         Homework 5         Homework 6         Homework 7         Homework 8         The Final         Create New Homework Problems Book Feedback - Learning From Data     General comments on the book     Chapter 1 - The Learning Problem     Chapter 2 - Training versus Testing     Chapter 3 - The Linear Model     Chapter 4 - Overfitting     Chapter 5 - Three Learning Principles     e-Chapter 6 - Similarity Based Methods     e-Chapter 7 - Neural Networks     e-Chapter 8 - Support Vector Machines     e-Chapter 9 - Learning Aides     Appendix and Notation     e-Appendices

All times are GMT -7. The time now is 03:25 AM.