LFD Book Forum  

Go Back   LFD Book Forum > Book Feedback - Learning From Data > Chapter 3 - The Linear Model

Reply
 
Thread Tools Display Modes
  #11  
Old 10-07-2013, 09:27 AM
magdon's Avatar
magdon magdon is offline
RPI
 
Join Date: Aug 2009
Location: Troy, NY, USA.
Posts: 595
Default Re: Exercise 3.4

\hat y-y is not H\epsilon, but that is close. Recall y=Xw+\epsilon


Quote:
Originally Posted by aaoam View Post
I'm having a bit of difficulty with 3.4b. I take \hat(y) - y and multiply by (XX^T)^{-1}XX^T, which ends up reducing the expression to just H\epsilon. However, then I can't use 3.3c in simplifying 3.3c, which makes me think I did something wrong. Can somebody give me a pointer?

Also, it'd be great if there was instructions somewhere about how to post in math mode. Perhaps I just missed them?
__________________
Have faith in probability
Reply With Quote
  #12  
Old 10-07-2013, 07:48 PM
Sweater Monkey Sweater Monkey is offline
Junior Member
 
Join Date: Sep 2013
Posts: 6
Default Re: Exercise 3.4

Quote:
Originally Posted by magdon View Post
Yes, that is right. You have to be more careful but use similar reasoning with

\epsilon^TH\epsilon
Ahhhh, yes I see now why \epsilon^TH\epsilon doesn't have a factor of N! The trace of this matrix is just \sigma^2(d+1).

Thanks Professor
Reply With Quote
  #13  
Old 10-07-2013, 10:54 PM
smiling_assassin smiling_assassin is offline
Junior Member
 
Join Date: Oct 2013
Posts: 1
Default Re: Exercise 3.4

Quote:
Originally Posted by Sweater Monkey View Post
Ahhhh, yes I see now why \epsilon^TH\epsilon doesn't have a factor of N! The trace of this matrix is just \sigma^2(d+1).

Thanks Professor

But isn't H a N \times N matrix? So trace would be N instead of d+1? I know X is N\times(d+1). What am I missing?
Reply With Quote
  #14  
Old 10-08-2013, 07:39 AM
magdon's Avatar
magdon magdon is offline
RPI
 
Join Date: Aug 2009
Location: Troy, NY, USA.
Posts: 595
Default Re: Exercise 3.4

You are right, H is an N\times Nmatrix. But its trace is not N. You may consider looking through Exercise 3.3, and in particular, part (d) should be helpful.


Quote:
Originally Posted by smiling_assassin View Post
But isn't H a N \times N matrix? So trace would be N instead of d+1? I know X is N\times(d+1). What am I missing?
__________________
Have faith in probability
Reply With Quote
  #15  
Old 10-09-2013, 02:05 PM
meixingdg meixingdg is offline
Junior Member
 
Join Date: Sep 2013
Posts: 4
Default Re: Exercise 3.4

For part (c), would the result of (y-hat - y) (from part b) be Ein(wlin) in terms of epsilon, since (y-hat - y) is the in-sample error?
Reply With Quote
  #16  
Old 10-10-2013, 09:11 AM
magdon's Avatar
magdon magdon is offline
RPI
 
Join Date: Aug 2009
Location: Troy, NY, USA.
Posts: 595
Default Re: Exercise 3.4

y and y-hat are vectors. The norm-squared of (y-hat - y) divided by N is the in-sample error.

Quote:
Originally Posted by meixingdg View Post
For part (c), would the result of (y-hat - y) (from part b) be Ein(wlin) in terms of epsilon, since (y-hat - y) is the in-sample error?
__________________
Have faith in probability
Reply With Quote
  #17  
Old 11-10-2013, 04:27 PM
jamesclyeh jamesclyeh is offline
Junior Member
 
Join Date: Nov 2013
Posts: 1
Default Re: Exercise 3.4

Hi,

For part (a), in one of the last steps I did:
{\bf{y}}=H{\bf{y}}-H\epsilon
Rearrange: \hat{\bf{y}}={\bf{y}}+H\epsilon
Since {\bf{y}}=Xw^*, \hat{\bf{y}}=Xw^*+H\epsilon

Are these steps correct?
I found subbing {\bf{y}}=Xw^* back in a bit recursive because I previously solved for w^* and plugged that in to get {\bf{y}}=H{\bf{y}}-H\epsilon.

Also for (b)
Is the answer \hat{\bf{y}}-{\bf{y}} = (H-I)\epsilon <---I ll delete this once its confirmed.

Thanks,
James
Reply With Quote
  #18  
Old 11-17-2013, 03:22 AM
yaser's Avatar
yaser yaser is offline
Caltech
 
Join Date: Aug 2009
Location: Pasadena, California, USA
Posts: 1,477
Default Re: Exercise 3.4

Hi James,

I am slow in responding this term as I am attending to the edX forum, but here are my quick comments:

For part (a), why is {\bf{y}}=Xw^* (what happened to the added noise)?

For part (b), your formula is correct.
__________________
Where everyone thinks alike, no one thinks very much
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -7. The time now is 04:20 AM.


Powered by vBulletin® Version 3.8.3
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, and participants in the Learning From Data MOOC by Yaser S. Abu-Mostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.