P(x) vs. P(y |x)

[pouramini]

The book says:

While both distributions model probabilistic aspects
of x and y, the target distribution P(y | x) is what we are trying to learn,
while the input distribution P(x) only quantifies the relative importance of
the point x in gauging how well we have learned.

I didn't get it! what is actually the difference? and specially the usage of P(x).. My English is not well!

Does it mean P(x) is only used in creating training and test set? and it is used in the estimate provided by test set of E_out?

[henry2015]

P(x) = probability of x
P(y|x) = probability of y given x has happened already

[pouramini]

Originally Posted by henry2015

P(x) = probability of x
P(y|x) = probability of y given x has happened already

I know, but what they mean in learning! does my conclusion in the initial post correct?

[henry2015]

My understanding is that the text you quoted is talking about learning when the target function has noise.

Because the target function has noise, so given an input x, f(x) doesn't always give y.

Hence, in this case, if we want to apply machine learning, we want to conclude what the probability of y given x as the input -- i.e. P(y|x).

Hence, P(x) isn't used for creating training set. P(x) is just talking about the distribution of x.

"P(x) only quantifies the relative importance of the point x in gauging how well we have learned"

For instance, if P(x1) is very small, we can't say that we learn very very well when P(y1|x1) is close to 1. Because there are x2, x3, ... that they might appear more frequent than x1 (e.g. P(x2) is much greater than P(x1)). When P(y1|x1) is close to 1, we can only say that we learn very well about how x1 is used to predict y1. But we can't say anything about x2, x3....given P(x1) is relatively small.

Hope I don't confuse you more.

If any of my statement is flaw, I appreciate anyone's correction

[pouramini]

Thank you! however I still need the author or another one clarify the sentence more... when he says "gauging how well we learned", I think he speaks about the test set.

We also know that we should avoid sampling bias. Then in my opinion P(x) is used in training set to make it unbiased, not?

and we know any distribution we used in training set we should use in test set, then P(x) is used in the test set too.

However I still don't know if we know P(x) or not, is it known?!