LFD Book Forum

LFD Book Forum (http://book.caltech.edu/bookforum/index.php)
-   The Final (http://book.caltech.edu/bookforum/forumdisplay.php?f=138)
-   -   Ques 16-18 How to find Ein? (http://book.caltech.edu/bookforum/showthread.php?t=676)

vmathur87 06-10-2012 02:58 AM
Ques 16-18 How to find Ein?
 
Sorry if my question is naive, but how do I find Ein for regular RBF? I got the weight vector w (of length k+1), but what do I do with it? I know I am missing something simple, but I just can't figure this out. Thanks for any help in advance!

yaser 06-10-2012 04:09 AM
Re: Ques 17-19 How to find Ein?
 
Quote:
Originally Posted by vmathur87 (Post 2930)
Sorry if my question is naive, but how do I find Ein for regular RBF? I got the weight vector w (of length k+1), but what do I do with it? I know I am missing something simple, but I just can't figure this out. Thanks for any help in advance!
You have the weights so you have the formula for the hypothesis that RBF is implementing. All you need to do is apply it to each input in the data set and check if the value it returns disagrees with the corresponding output. The fraction of training data points on which it disagrees will be E_{\rm in}.

vmathur87 06-10-2012 09:02 AM
Re: Ques 17-19 How to find Ein?
 
Thanks, I understand that. But do you mean to apply the weights to each input according to the formula given on slide 14 of lecture 16? I think that since that formula was used to derive the weights (using phi), the weights would always satisfy it and the Ein will always be 0. Maybe this doesbn't make too much sene, but I am a little muddled up here.

kkkkk 06-10-2012 09:42 AM
Re: Ques 17-19 How to find Ein?
 
Please see slide 19 of lecture 16 which shows h(x) for regular rbf with the bias b (not part of the summation, b is w0).

vmathur87 06-11-2012 08:25 AM
Re: Ques 17-19 How to find Ein?
 
Thanks, I got it!

alasdairj 06-06-2013 06:39 PM
Re: Ques 17-19 How to find Ein?
 
OK, I am a little confused. The RBF description does not mention a bias, but then it is introduced in slide 19 of lecture 16... how do we learn the bias?

yaser 06-06-2013 07:48 PM
Re: Ques 17-19 How to find Ein?
 
Quote:
Originally Posted by alasdairj (Post 11039)
OK, I am a little confused. The RBF description does not mention a bias, but then it is introduced in slide 19 of lecture 16... how do we learn the bias?
The bias can be learned along with the weights using the pseudo-inverse method. Just consider the bias a parameter like the weights which happens to be multiplied by the constant 1, the same way w_0 was handled in linear regression.

alasdairj 06-07-2013 06:05 AM
Re: Ques 17-19 How to find Ein?
 
Thanks! I implemented this for the pseudo-inverse calculation, and also in the bias in the final calculation of h. What surprised me was that the Ein got slightly worse rather than better, which is not what I expected. Or do I have an implementation issue?

yaser 06-07-2013 12:38 PM
Re: Ques 17-19 How to find Ein?
 
Quote:
Originally Posted by alasdairj (Post 11042)
Thanks! I implemented this for the pseudo-inverse calculation, and also in the bias in the final calculation of h. What surprised me was that the Ein got slightly worse rather than better, which is not what I expected. Or do I have an implementation issue?
In-sample error should not go up with the added parameter (out-of-sample error might), so I suspect a bug of some sort.


All times are GMT -7. The time now is 11:53 PM.

Powered by vBulletin® Version 3.8.3
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.
The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, and participants in the Learning From Data MOOC by Yaser S. Abu-Mostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.