LFD Book Forum  

Go Back   LFD Book Forum > Course Discussions > Online LFD course > The Final

Reply
 
Thread Tools Display Modes
  #1  
Old 06-10-2012, 02:58 AM
vmathur87 vmathur87 is offline
Junior Member
 
Join Date: May 2012
Posts: 5
Default Ques 16-18 How to find Ein?

Sorry if my question is naive, but how do I find Ein for regular RBF? I got the weight vector w (of length k+1), but what do I do with it? I know I am missing something simple, but I just can't figure this out. Thanks for any help in advance!
Reply With Quote
  #2  
Old 06-10-2012, 04:09 AM
yaser's Avatar
yaser yaser is offline
Caltech
 
Join Date: Aug 2009
Location: Pasadena, California, USA
Posts: 1,477
Default Re: Ques 17-19 How to find Ein?

Quote:
Originally Posted by vmathur87 View Post
Sorry if my question is naive, but how do I find Ein for regular RBF? I got the weight vector w (of length k+1), but what do I do with it? I know I am missing something simple, but I just can't figure this out. Thanks for any help in advance!
You have the weights so you have the formula for the hypothesis that RBF is implementing. All you need to do is apply it to each input in the data set and check if the value it returns disagrees with the corresponding output. The fraction of training data points on which it disagrees will be E_{\rm in}.
__________________
Where everyone thinks alike, no one thinks very much
Reply With Quote
  #3  
Old 06-10-2012, 09:02 AM
vmathur87 vmathur87 is offline
Junior Member
 
Join Date: May 2012
Posts: 5
Default Re: Ques 17-19 How to find Ein?

Thanks, I understand that. But do you mean to apply the weights to each input according to the formula given on slide 14 of lecture 16? I think that since that formula was used to derive the weights (using phi), the weights would always satisfy it and the Ein will always be 0. Maybe this doesbn't make too much sene, but I am a little muddled up here.
Reply With Quote
  #4  
Old 06-10-2012, 09:42 AM
kkkkk kkkkk is offline
Invited Guest
 
Join Date: Mar 2012
Posts: 71
Default Re: Ques 17-19 How to find Ein?

Please see slide 19 of lecture 16 which shows h(x) for regular rbf with the bias b (not part of the summation, b is w0).
Reply With Quote
  #5  
Old 06-11-2012, 08:25 AM
vmathur87 vmathur87 is offline
Junior Member
 
Join Date: May 2012
Posts: 5
Default Re: Ques 17-19 How to find Ein?

Thanks, I got it!
Reply With Quote
  #6  
Old 06-06-2013, 06:39 PM
alasdairj alasdairj is offline
Member
 
Join Date: Mar 2013
Posts: 12
Default Re: Ques 17-19 How to find Ein?

OK, I am a little confused. The RBF description does not mention a bias, but then it is introduced in slide 19 of lecture 16... how do we learn the bias?
Reply With Quote
  #7  
Old 06-06-2013, 07:48 PM
yaser's Avatar
yaser yaser is offline
Caltech
 
Join Date: Aug 2009
Location: Pasadena, California, USA
Posts: 1,477
Default Re: Ques 17-19 How to find Ein?

Quote:
Originally Posted by alasdairj View Post
OK, I am a little confused. The RBF description does not mention a bias, but then it is introduced in slide 19 of lecture 16... how do we learn the bias?
The bias can be learned along with the weights using the pseudo-inverse method. Just consider the bias a parameter like the weights which happens to be multiplied by the constant 1, the same way w_0 was handled in linear regression.
__________________
Where everyone thinks alike, no one thinks very much
Reply With Quote
  #8  
Old 06-07-2013, 06:05 AM
alasdairj alasdairj is offline
Member
 
Join Date: Mar 2013
Posts: 12
Default Re: Ques 17-19 How to find Ein?

Thanks! I implemented this for the pseudo-inverse calculation, and also in the bias in the final calculation of h. What surprised me was that the Ein got slightly worse rather than better, which is not what I expected. Or do I have an implementation issue?
Reply With Quote
  #9  
Old 06-07-2013, 12:38 PM
yaser's Avatar
yaser yaser is offline
Caltech
 
Join Date: Aug 2009
Location: Pasadena, California, USA
Posts: 1,477
Default Re: Ques 17-19 How to find Ein?

Quote:
Originally Posted by alasdairj View Post
Thanks! I implemented this for the pseudo-inverse calculation, and also in the bias in the final calculation of h. What surprised me was that the Ein got slightly worse rather than better, which is not what I expected. Or do I have an implementation issue?
In-sample error should not go up with the added parameter (out-of-sample error might), so I suspect a bug of some sort.
__________________
Where everyone thinks alike, no one thinks very much
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -7. The time now is 04:05 AM.


Powered by vBulletin® Version 3.8.3
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, and participants in the Learning From Data MOOC by Yaser S. Abu-Mostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.