role of P(X) ?
The Hoeffding bound for the model H in chapter one, only requires that
we make the assumption that the input examples are a random sample from the bin; so we can generalize the sample error. What role does the distribution on X play? It appears to me that we don't need it. (at least the way the issue of feasibility is setup in chapter 1) ie. true mismatch ~ sample mismatch. Thanks. 
Re: role of P(X) ?
So can you say that P(X) populates the bin and determines mu? In that case we would be sampling P(X); is this correct?

Re: role of P(X) ?
I see.
Example: is the target. is the input space. If we let 1. or 2. , where t(1) is the tdistribution with one degree of freedom. I know from my stat classes that in case 1. a linear model is actually "correct". (this is great since we usually know nothing about f) So in this case the distribution of X plays a role in selecting H, and hence reducing the in sample error. (assuming the quadratic loss fct.) Questions: So in either case 1. or 2. the interpretation/computation of the sample error is the same? I am a little confused since the overall true error (which we hope the sample error approximates) is defined based on the joint distribution of (X,Y); which depends on the distribution of X. Thanks. I hope this class/book can clear up some misconceptions about the theoretical framework of the learning problem once and for all :) 
Re: role of P(X) ?
Quote:
[This may appear to be a trivial assumption when sampling from some populations, but it is likely to be nontrivial in many cases where we are attempting to infer future behavior from past behavior in a system whose characteristics may change] 
Re: role of P(X) ?
Quote:

Re: role of P(X) ?
Quote:
Thanks prof Yaser's reply. A quick question, as we are sampling according to , how effect each ? In other words, determines or Sampling process or both? 
Re: role of P(X) ?
Quote:

Re: role of P(X) ?
Quote:
But isn't fixed when you choose a particular hypothesis h. [ Because number of red marbles is equal to the number of points in the input space where hypothesis ( h ) and target function ( f ) disagree. And this, in my opinion, has nothing to do with probability distribution function ] Please clarify. Thanks, Giridhar. 
Re: role of P(X) ?
Quote:
To take a simple example, Let's say that there are only two marbles in the bin, one red and one green, but the red marble has a higher probability of being picked than the green marble. In this case, is not 1/2 though the fraction of red marbles is 1/2. 
Re: role of P(X) ?
To add to professor's explanation, another way to put it is that the Hoeffding inequality as presented in the book applies to sequence of N i.i.d. Bernoullli random variables with parameter .
This condition can actually be relaxed to any N nonidentical but independent r.v.'s that almost surely take values on compact intervals. There's even a further generalization that does not even require independence, so long as the sequence is a Martingale with (a.s.) bounded increments (see AzumaHoeffding inequality). 
Re: role of P(X) ?
I had the same question, I still don't know if we need to know P(X) or not! I know it doesn't matter which P(x) is used but should we know it or not?
it seems we just should select examples randomly and independently (what it actually means?) How we can get sure we do as this? 
All times are GMT 7. The time now is 05:38 AM. 
Powered by vBulletin® Version 3.8.3
Copyright ©2000  2021, Jelsoft Enterprises Ltd.
The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. AbuMostafa, Malik MagdonIsmail, and HsuanTien Lin, and participants in the Learning From Data MOOC by Yaser S. AbuMostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.