LFD Book Forum  

Go Back   LFD Book Forum > Course Discussions > Online LFD course > The Final

Reply
 
Thread Tools Display Modes
  #1  
Old 06-11-2013, 07:21 AM
Elroch Elroch is offline
Invited Guest
 
Join Date: Mar 2013
Posts: 143
Default Q16 and Q17 clarification

In these two questions there are the independent possibilities of E_{in} rising falling or staying the same and the same for E_{out}. In the latter case, while for the two inequalities the meaning is clear and based (with high confidence) on an adequate test sample, what the precise meaning would be for equality is not clear.

In any case, the options are not complete (there are 5 of 9), so presumably what is wanted is the greatest from those offered, regardless of whether it is actually the most frequent possibility?

In any case, could you clarify the precise meaning of the fifth option?
Reply With Quote
  #2  
Old 06-11-2013, 11:34 AM
jforbes jforbes is offline
Member
 
Join Date: Apr 2013
Posts: 12
Default Re: Q17 and Q18 clarification

Quote:
Originally Posted by Elroch View Post
In these two questions there are the independent possibilities of E_{in} rising falling or staying the same and the same for E_{out}. In the latter case, while for the two inequalities the meaning is clear and based (with high confidence) on an adequate test sample, what the precise meaning would be for equality is not clear.

In any case, the options are not complete (there are 5 of 9), so presumably what is wanted is the greatest from those offered, regardless of whether it is actually the most frequent possibility?

In any case, could you clarify the precise meaning of the fifth option?
I used the same test set to evaluate the out of sample error for the alternatives, so it was in principle quite possible to get the same E_{out}. I'm not sure if this was the appropriate approach.

In principle though, what you're doing with your test set is you're approximating the true value of E_{out}, which you only approach in the limit of an infinite test set, in which case the probability that you get exactly the same E_{out} should be vanishingly small unless your hypotheses were the same exact function, which won't happen in our situation.
Reply With Quote
  #3  
Old 06-11-2013, 11:59 AM
Elroch Elroch is offline
Invited Guest
 
Join Date: Mar 2013
Posts: 143
Default Re: Q17 and Q18 clarification

Quote:
Originally Posted by jforbes View Post
I used the same test set to evaluate the out of sample error for the alternatives, so it was in principle quite possible to get the same E_{out}. I'm not sure if this was the appropriate approach.

In principle though, what you're doing with your test set is you're approximating the true value of E_{out}, which you only approach in the limit of an infinite test set, in which case the probability that you get exactly the same E_{out} should be vanishingly small unless your hypotheses were the same exact function, which won't happen in our situation.
I realise that, and was using a large test set, so the situation did not arise. What I did have on one question was a lot of unchanged E_{in} and different E_{out}. As I result, I was uncomfortable with my answer (although it was correct).
Reply With Quote
  #4  
Old 06-11-2013, 01:39 PM
jforbes jforbes is offline
Member
 
Join Date: Apr 2013
Posts: 12
Default Re: Q17 and Q18 clarification

Quote:
Originally Posted by Elroch View Post
I realise that, and was using a large test set, so the situation did not arise. What I did have on one question was a lot of unchanged E_{in} and different E_{out}. As I result, I was uncomfortable with my answer (although it was correct).
Ah, good point. I didn't check all 9 conditions, just the 5 given. There was a clear winner every time, even with a not-ridiculously-large test set, so I didn't bother checking that scenario, which in retrospect seems like it could indeed arise frequently.
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -7. The time now is 05:17 PM.


Powered by vBulletin® Version 3.8.3
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, and participants in the Learning From Data MOOC by Yaser S. Abu-Mostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.