Query regarding Noise Targets lecture 4

ripande · #1 01-16-2013, 11:41 AM

As per lecture 4 :- Slide 16/22

Noise target = Diterministic Target + Noise (This is clear)

However it then says :

Diterministic target f(x) = E(Y|X). Hence
y ( Noisy target) = E(Y|X) ( Diterministic target) + noise ( y-f(x) )

I do not understand the above statement. Did we not say that to accomodate for noisy targets we replace y = f(x) with a conditional distribution of y given x. Should then not the noisy target be defined by
E(Y|X) ?

yaser · #2 01-16-2013, 11:59 AM

Quote:

Originally Posted by ripande

As per lecture 4 :- Slide 16/22

Noise target = Diterministic Target + Noise (This is clear)

However it then says :

Diterministic target f(x) = E(Y|X). Hence
y ( Noisy target) = E(Y|X) ( Diterministic target) + noise ( y-f(x) )

I do not understand the above statement. Did we not say that to accomodate for noisy targets we replace y = f(x) with a conditional distribution of y given x. Should then not the noisy target be defined by
E(Y|X) ?

The noisy target is

and indeed it is specified by $P(y|{\bf x})$ . The equation decomposes the noisy target into a noiseless component plus a pure noise component.

ripande · #3 01-16-2013, 12:08 PM

Is E(Y|X ) the noiseless component ? If yes, how ?

yaser · #4 01-16-2013, 12:11 PM

Quote:

Originally Posted by ripande

Is E(Y|X ) the noiseless component ? If yes, how?

Yes, it is. It is a deterministic function of ${\bf x}$ . The noisy aspect has been integrated out (averaged out) by the expected value.

ripande · #5 01-16-2013, 12:24 PM

Understood. Thanks prof Yaser

jlaurentum · #6 04-13-2013, 10:45 AM

One question:

If

is a random sample, it is clearly a random variable (or rather, a sequence of random variables).

If

is also a random variable (the target), then wouldn't

be a random variable and not a deterministic quantity? (unless

and

are independent variables in which case the conditional expectation collapses to a constant?)

yaser · #7 04-13-2013, 11:20 AM

Quote:

Originally Posted by jlaurentum

One question:

If

is a random sample, it is clearly a random variable (or rather, a sequence of random variables).

If

is also a random variable (the target), then wouldn't

be a random variable and not a deterministic quantity? (unless

and

are independent variables in which case the conditional expectation collapses to a constant?)

Let me first use the class notation to make sure we are talking about the same thing. The data (sample) is $({\bf x}_1,y_1), \cdots, ({\bf x}_N,y_N)$ where each ${\bf x}_n$ is (often) a random vector (a vector of random variables) and each

is a random variable. The probability distribution $P(y|{\bf x})$ is the conditional distribution of that random variable given ${\bf x}$ .

The conditional expectation, ${\bf E}(y|{\bf x})$ , is a deterministic quantity (more presisely, a deterministic function of ${\bf x}$ ). As you point out, if ${\bf x}$ and

are statistically independent, that function is a constant (independent of ${\bf x}$ ). However, even if ${\bf x}$ and

are not statistically independent, that function is still a deterministic, albeit non-constant, function of ${\bf x}$ .

jlaurentum · #8 04-13-2013, 01:02 PM

Oh ok. I get it, Professor. It's like with the regression, where for a given linear model: $Y=\beta_0+\beta_1\cdot X_1 + \ldots$ , what we're actually doing is specifying $E(Y|X)=\beta_0+\beta_1\cdot X_1 + \ldots$ .

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode