Here's how I think about

and

.

The in-sample performance

is how well you have converged on a solution to your given data.

in our notation. The given data is a *sample* of the real world domain on which

is defined. Therefore the in-sample performance of

is how well it works on the data set (how much is

).

The out-of-sample performance

, is how well

works on the rest of the world (not in our tiny sample). We have seen in several of the problems that we can estimate it by generating a whole new data set (often with many more sample points) and compare the performance of our

with the performance of the made up

.

Of course in a real situation we won't have

, only the knowledge(belief!?) that it exists. That's why we need the Hoeffding inequality, so we can at least bound

.