Exercise 3.4 - Page 2

magdon · #11 10-07-2013, 09:27 AM

$\hat y-y$ is not $H\epsilon$ , but that is close. Recall $y=Xw+\epsilon$

Quote:

Originally Posted by aaoam

I'm having a bit of difficulty with 3.4b. I take \hat(y) - y and multiply by (XX^T)^{-1}XX^T, which ends up reducing the expression to just H\epsilon. However, then I can't use 3.3c in simplifying 3.3c, which makes me think I did something wrong. Can somebody give me a pointer?

Also, it'd be great if there was instructions somewhere about how to post in math mode. Perhaps I just missed them?

Sweater Monkey · #12 10-07-2013, 07:48 PM

Quote:

Originally Posted by magdon

Yes, that is right. You have to be more careful but use similar reasoning with

$\epsilon^TH\epsilon$

Ahhhh, yes I see now why $\epsilon^TH\epsilon$ doesn't have a factor of N! The trace of this matrix is just $\sigma^2(d+1)$ .

Thanks Professor

smiling_assassin · #13 10-07-2013, 10:54 PM

Quote:

Originally Posted by Sweater Monkey

Ahhhh, yes I see now why $\epsilon^TH\epsilon$ doesn't have a factor of N! The trace of this matrix is just $\sigma^2(d+1)$ .

Thanks Professor

But isn't

a $N \times N$ matrix? So trace would be

instead of

? I know

is $N\times(d+1)$ . What am I missing?

magdon · #14 10-08-2013, 07:39 AM

You are right,

is an $N\times N$ matrix. But its trace is not

. You may consider looking through Exercise 3.3, and in particular, part (d) should be helpful.

Quote:

Originally Posted by smiling_assassin

But isn't

a $N \times N$ matrix? So trace would be

instead of

? I know

is $N\times(d+1)$ . What am I missing?

meixingdg · #15 10-09-2013, 02:05 PM

For part (c), would the result of (y-hat - y) (from part b) be Ein(wlin) in terms of epsilon, since (y-hat - y) is the in-sample error?

magdon · #16 10-10-2013, 09:11 AM

y and y-hat are vectors. The norm-squared of (y-hat - y) divided by N is the in-sample error.

Quote:

Originally Posted by meixingdg

For part (c), would the result of (y-hat - y) (from part b) be Ein(wlin) in terms of epsilon, since (y-hat - y) is the in-sample error?

jamesclyeh · #17 11-10-2013, 04:27 PM

Hi,

For part (a), in one of the last steps I did:
${\bf{y}}=H{\bf{y}}-H\epsilon$
Rearrange: $\hat{\bf{y}}={\bf{y}}+H\epsilon$
Since ${\bf{y}}=Xw^*$ , $\hat{\bf{y}}=Xw^*+H\epsilon$

Are these steps correct?
I found subbing ${\bf{y}}=Xw^*$ back in a bit recursive because I previously solved for

and plugged that in to get ${\bf{y}}=H{\bf{y}}-H\epsilon$ .

Also for (b)
Is the answer $\hat{\bf{y}}-{\bf{y}} = (H-I)\epsilon$ <---I ll delete this once its confirmed.

Thanks,
James

yaser · #18 11-17-2013, 03:22 AM

Hi James,

I am slow in responding this term as I am attending to the edX forum, but here are my quick comments:

For part (a), why is ${\bf{y}}=Xw^*$ (what happened to the added noise)?

For part (b), your formula is correct.

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode