General question on sampling bias

[palmipede]

Greetings!

Does one speak of sampling bias when the training data points come from the same distribution, e.g. normal, uniform, but are statistically dependent?

Say for example that a questionnaire is passed around via friendship links on Facebook. There is a chance that everyone might see the questionnaire over a long enough period of time but that period might be longer than the time allotted to the machine learning project.

[htlin]

Originally Posted by palmipede

Greetings!

Does one speak of sampling bias when the training data points come from the same distribution, e.g. normal, uniform, but are statistically dependent?

Say for example that a questionnaire is passed around via friendship links on Facebook. There is a chance that everyone might see the questionnaire over a long enough period of time but that period might be longer than the time allotted to the machine learning project.

While the situation you describe can cause bias that affects the generalization ability, I've never seen this kind of bias called "sampling bias", which was commonly reserved for non-matching distributions between training and testing.

There is an ongoing research topic called "Learning from non-IID data" which partially aims at making learning possible for the situation you describe. For instance,

http://www-connex.lip6.fr/~amini/ecml-wk-lniid.html

Hope this helps.