LFD Book Forum  

Go Back   LFD Book Forum > Book Feedback - Learning From Data > Chapter 4 - Overfitting

Thread Tools Display Modes
Prev Previous Post   Next Post Next
Old 10-23-2013, 12:15 PM
Sweater Monkey Sweater Monkey is offline
Junior Member
Join Date: Sep 2013
Posts: 6
Default Exercise 4.7

I feel like I'm overthinking Exercise 4.7 (b) and I am hoping for a little bit of insight.

My gut instinct says that Var[E_{\text{val}}(g^-)] = \frac{1}{K} (P[g^-(x),y])^2

I arrived at this idea by considering that the probability is similar to the standard deviation which is the square root of the variance so since:
Var[E_{\text{val}}(g^-)] = \frac{1}{K}Var_{x}[e(g^-(x),y)] and P[{g^-(x)}\neq{y}] = P[e(g^-(x),y)] does Var_{x}[e(g^-(x),y)] = P[{g^-(x)}\neq{y}]^2 ???

Then for part (c) on the exercise, assuming that the above is true, I used the notion that P[{g^-(x)}\neq{y}] \le 0.5 because if the probability of error were greater than 0.5 then the learned g would just flip its classification. Therefore this shows that for any g^- in a classification problem,
Var[E_{\text{val}}(g^-)] \le \frac{1}{K}0.5^2 and therefore:
Var[E_{\text{val}}(g^-)] \le \frac{1}{4K}

Any indication as to whether I'm working along the correct lines would be appreciated!
Reply With Quote

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT -7. The time now is 07:30 PM.

Powered by vBulletin® Version 3.8.3
Copyright ©2000 - 2022, Jelsoft Enterprises Ltd.
The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, and participants in the Learning From Data MOOC by Yaser S. Abu-Mostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.