LFD Book Forum Query regarding Noise Targets lecture 4
 User Name Remember Me? Password
 FAQ Calendar Mark Forums Read

 Thread Tools Display Modes
#1
01-16-2013, 11:41 AM
 ripande Senior Member Join Date: Jan 2013 Posts: 71
Query regarding Noise Targets lecture 4

As per lecture 4 :- Slide 16/22

Noise target = Diterministic Target + Noise (This is clear)

However it then says :

Diterministic target f(x) = E(Y|X). Hence
y ( Noisy target) = E(Y|X) ( Diterministic target) + noise ( y-f(x) )

I do not understand the above statement. Did we not say that to accomodate for noisy targets we replace y = f(x) with a conditional distribution of y given x. Should then not the noisy target be defined by
E(Y|X) ?
#2
01-16-2013, 11:59 AM
 yaser Caltech Join Date: Aug 2009 Location: Pasadena, California, USA Posts: 1,477
Re: Query regarding Noise Targets lecture 4

Quote:
 Originally Posted by ripande As per lecture 4 :- Slide 16/22 Noise target = Diterministic Target + Noise (This is clear) However it then says : Diterministic target f(x) = E(Y|X). Hence y ( Noisy target) = E(Y|X) ( Diterministic target) + noise ( y-f(x) ) I do not understand the above statement. Did we not say that to accomodate for noisy targets we replace y = f(x) with a conditional distribution of y given x. Should then not the noisy target be defined by E(Y|X) ?
The noisy target is and indeed it is specified by . The equation decomposes the noisy target into a noiseless component plus a pure noise component.
__________________
Where everyone thinks alike, no one thinks very much
#3
01-16-2013, 12:08 PM
 ripande Senior Member Join Date: Jan 2013 Posts: 71
Re: Query regarding Noise Targets lecture 4

Is E(Y|X ) the noiseless component ? If yes, how ?
#4
01-16-2013, 12:11 PM
 yaser Caltech Join Date: Aug 2009 Location: Pasadena, California, USA Posts: 1,477
Re: Query regarding Noise Targets lecture 4

Quote:
 Originally Posted by ripande Is E(Y|X ) the noiseless component ? If yes, how?
Yes, it is. It is a deterministic function of . The noisy aspect has been integrated out (averaged out) by the expected value.
__________________
Where everyone thinks alike, no one thinks very much
#5
01-16-2013, 12:24 PM
 ripande Senior Member Join Date: Jan 2013 Posts: 71
Re: Query regarding Noise Targets lecture 4

Understood. Thanks prof Yaser
#6
04-13-2013, 10:45 AM
 jlaurentum Member Join Date: Apr 2013 Location: Venezuela Posts: 41
Re: Query regarding Noise Targets lecture 4

One question:

If is a random sample, it is clearly a random variable (or rather, a sequence of random variables).

If is also a random variable (the target), then wouldn't be a random variable and not a deterministic quantity? (unless and are independent variables in which case the conditional expectation collapses to a constant?)
#7
04-13-2013, 11:20 AM
 yaser Caltech Join Date: Aug 2009 Location: Pasadena, California, USA Posts: 1,477
Re: Query regarding Noise Targets lecture 4

Quote:
 Originally Posted by jlaurentum One question: If is a random sample, it is clearly a random variable (or rather, a sequence of random variables). If is also a random variable (the target), then wouldn't be a random variable and not a deterministic quantity? (unless and are independent variables in which case the conditional expectation collapses to a constant?)
Let me first use the class notation to make sure we are talking about the same thing. The data (sample) is where each is (often) a random vector (a vector of random variables) and each is a random variable. The probability distribution is the conditional distribution of that random variable given .

The conditional expectation, , is a deterministic quantity (more presisely, a deterministic function of ). As you point out, if and are statistically independent, that function is a constant (independent of ). However, even if and are not statistically independent, that function is still a deterministic, albeit non-constant, function of .
__________________
Where everyone thinks alike, no one thinks very much
#8
04-13-2013, 01:02 PM
 jlaurentum Member Join Date: Apr 2013 Location: Venezuela Posts: 41
Re: Query regarding Noise Targets lecture 4

Oh ok. I get it, Professor. It's like with the regression, where for a given linear model: , what we're actually doing is specifying .

 Thread Tools Display Modes Linear Mode

 Posting Rules You may not post new threads You may not post replies You may not post attachments You may not edit your posts BB code is On Smilies are On [IMG] code is On HTML code is Off Forum Rules
 Forum Jump User Control Panel Private Messages Subscriptions Who's Online Search Forums Forums Home General     General Discussion of Machine Learning     Free Additional Material         Dynamic e-Chapters         Dynamic e-Appendices Course Discussions     Online LFD course         General comments on the course         Homework 1         Homework 2         Homework 3         Homework 4         Homework 5         Homework 6         Homework 7         Homework 8         The Final         Create New Homework Problems Book Feedback - Learning From Data     General comments on the book     Chapter 1 - The Learning Problem     Chapter 2 - Training versus Testing     Chapter 3 - The Linear Model     Chapter 4 - Overfitting     Chapter 5 - Three Learning Principles     e-Chapter 6 - Similarity Based Methods     e-Chapter 7 - Neural Networks     e-Chapter 8 - Support Vector Machines     e-Chapter 9 - Learning Aides     Appendix and Notation     e-Appendices

All times are GMT -7. The time now is 08:34 PM.

 Contact Us - LFD Book - Top