Re: Exercise 4.10
I think this can be a possible explanation for Exercise 4.10.(c):
When K is 1, then estimation of outof sample error by validation error is not ‘that’ good because of the penalty term. Thus, the model chosen from this poor estimate might not be the ‘best’ one. This explains Expectation[OutofSample Error of g^_(m*)] < Expectation[OutofSample Error of g_(m*)]. This situation somewhat improves as K increases.
Please let me know if this explanation is not correct.
