View Single Post
Old 06-11-2013, 11:59 AM
Elroch Elroch is offline
Invited Guest
Join Date: Mar 2013
Posts: 143
Default Re: Q17 and Q18 clarification

Originally Posted by jforbes View Post
I used the same test set to evaluate the out of sample error for the alternatives, so it was in principle quite possible to get the same E_{out}. I'm not sure if this was the appropriate approach.

In principle though, what you're doing with your test set is you're approximating the true value of E_{out}, which you only approach in the limit of an infinite test set, in which case the probability that you get exactly the same E_{out} should be vanishingly small unless your hypotheses were the same exact function, which won't happen in our situation.
I realise that, and was using a large test set, so the situation did not arise. What I did have on one question was a lot of unchanged E_{in} and different E_{out}. As I result, I was uncomfortable with my answer (although it was correct).
Reply With Quote