LFD Book Forum  

Go Back   LFD Book Forum > Book Feedback - Learning From Data > Chapter 4 - Overfitting

Thread Tools Display Modes
Prev Previous Post   Next Post Next
Old 10-04-2017, 11:24 AM
DeSLex DeSLex is offline
Junior Member
Join Date: Jul 2017
Posts: 1
Default Problem 4.4: Interpretation

Can someone confirm if this interpretation of "experiment" in part (d) is correct? I think we have nested loops:

for each choice of Q sub f, N, and sigma
Define the normalizing constant c sup 2 = E sub a, x (f sup 2).
for each choice of coefficients {a sub q: q = 1,..., Q} from standard normal distributions
f(x) = sum from 1 to Q (a sub q L sub q (x)) / c
for n = 1 to N
y sub n = f(x sub n) + sigma epsilon sub n
Find E sub out (g sub 2) and E sub out (g sub 10)
The reason for all this detail is that I was unclear about what the conditional distribution p(y|x) might be. I think now that in each iteration of the second nested loop, we fix a choice of {a sub q}; given these coefficients, f becomes a deterministic function of x and the only randomness in y sub n is due to epsilon sub n.

As a result, we have a different joint distribution P(x, y) for each choice of {a sub q}. We also have a different target function for each {a sub q}.

The repeated experiments in (d), with fixed Q, N, and sigma, lead to one E sub out (g) for each set of coefficients {a sub q}. E sub out (g) is a function of the normal random vector (a sub 1, ..., a sub Q). The average of the out-of-sample errors is empirical mean of the distribution of E sub out.
Reply With Quote

joint distribution

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT -7. The time now is 10:40 AM.

Powered by vBulletin® Version 3.8.3
Copyright ©2000 - 2022, Jelsoft Enterprises Ltd.
The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, and participants in the Learning From Data MOOC by Yaser S. Abu-Mostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.