Problem 1.10 (b)

NewtoML · #1 04-01-2015, 07:06 PM

Hello,

I believe that the number of possible f that can generate D in a noiseless setting is infinite. For example, if I take a data set of 2 points, say (5,-1) and (3,1), I can come up with any number of functions that will generate these two points.

However, I'm confused as to how this reconciles with the example on p. 16 where the set of all possible target functions in the example is finite, namely 256. Is this because the input space X is limited to Boolean vectors in 3 dimensions?

Thanks

yaser · #2 04-02-2015, 10:53 PM

Quote:

Originally Posted by NewtoML

However, I'm confused as to how this reconciles with the example on p. 16 where the set of all possible target functions in the example is finite, namely 256. Is this because the input space X is limited to Boolean vectors in 3 dimensions?

Exactly, and the output is limited to a binary value, too.

c.thejaswi · #3 11-23-2015, 09:13 AM

Quote:

Originally Posted by yaser

Exactly, and the output is limited to a binary value, too.

Dear Professor,
However, it is no where mentioned that ${\cal X}$ is a space of binary strings, and "$f$" is a logical operation. Therefore, is it still true that number of $f'$s that can generate D is finite?

c.thejaswi · #4 11-23-2015, 09:18 AM

I am sorry for the confusion. I got it.
Thanks.

memoweb · #5 12-03-2015, 05:59 AM

Quote:

Originally Posted by c.thejaswi

I am sorry for the confusion. I got it.
Thanks.

thanks

MaciekLeks · #6 04-01-2016, 06:32 AM

IMHO, the answer to (b) is not an infinite number. As for part (a) the anwer is not $\frac{1}{2}$ , so here the anwer is not as simple as it seems.

My way of thinking is as follows:
We must recall that the assumption from (a) does not work here, and we still do not know function

. We also do not know the dimensionality of the datapoint in the input space $\mathcal{X}=\{\mathbf{x}_{1,}\mathbf{x}_{2},...,\mathbf{x}_{N},\mathbf{x}_{N+1},...,\mathbf{x}_{N+M}\}$ but we know that this input space is fixed (all $\mathbf{x}_{k}$ for $1\leq k\leq N+M$ are already set). In this case we have $\mathcal{D}$ of size N generated in a deterministic way, and $y_{n}=f(\mathbf{x}_{n})$ ( $1\leq n\leq N$ ) is not affected by any noise. So, how many possible $f:\mathcal{X}\mathcal{\rightarrow\mathcal{Y}}$ can 'generate' $\mathcal{D}$ ? The subtle point in this case is the assumption: “For a fixed $\mathcal{D}$ of size

”, which means that $\mathcal{D}=\{(\mathbf{x}_{1},y_{1}),(\mathbf{x}_{2},y_{2}),...,(\mathbf{x}_{N},y_{n})\}$ is already generated. We can calculate how many possible outputs $y_{n}$ for $1\leq n\leq N$ we can get? Only 1. But there are remaining $\{\mathbf{x}_{N+1},...,\mathbf{x}_{N+M}\}$ datapoints for which we can have $2^{M}$ possible values $\{-1,1\}$ . So, the anwer is $2^{M}$ .

Am I wrong?

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode