LFD Book Forum  

Go Back   LFD Book Forum > Book Feedback - Learning From Data > Chapter 1 - The Learning Problem

Reply
 
Thread Tools Display Modes
  #1  
Old 04-01-2015, 07:06 PM
NewtoML NewtoML is offline
Junior Member
 
Join Date: Mar 2015
Posts: 8
Default Problem 1.10 (b)

Hello,

I believe that the number of possible f that can generate D in a noiseless setting is infinite. For example, if I take a data set of 2 points, say (5,-1) and (3,1), I can come up with any number of functions that will generate these two points.

However, I'm confused as to how this reconciles with the example on p. 16 where the set of all possible target functions in the example is finite, namely 256. Is this because the input space X is limited to Boolean vectors in 3 dimensions?

Thanks
Reply With Quote
  #2  
Old 04-02-2015, 10:53 PM
yaser's Avatar
yaser yaser is offline
Caltech
 
Join Date: Aug 2009
Location: Pasadena, California, USA
Posts: 1,477
Default Re: Problem 1.10 (b)

Quote:
Originally Posted by NewtoML View Post
However, I'm confused as to how this reconciles with the example on p. 16 where the set of all possible target functions in the example is finite, namely 256. Is this because the input space X is limited to Boolean vectors in 3 dimensions?
Exactly, and the output is limited to a binary value, too.
__________________
Where everyone thinks alike, no one thinks very much
Reply With Quote
  #3  
Old 11-23-2015, 09:13 AM
c.thejaswi c.thejaswi is offline
Junior Member
 
Join Date: Nov 2015
Posts: 2
Default Re: Problem 1.10 (b)

Quote:
Originally Posted by yaser View Post
Exactly, and the output is limited to a binary value, too.
Dear Professor,
However, it is no where mentioned that ${\cal X}$ is a space of binary strings, and "$f$" is a logical operation. Therefore, is it still true that number of $f'$s that can generate D is finite?
Reply With Quote
  #4  
Old 11-23-2015, 09:18 AM
c.thejaswi c.thejaswi is offline
Junior Member
 
Join Date: Nov 2015
Posts: 2
Default Re: Problem 1.10 (b)

I am sorry for the confusion. I got it.
Thanks.
Reply With Quote
  #5  
Old 12-03-2015, 05:59 AM
memoweb memoweb is offline
Junior Member
 
Join Date: Dec 2015
Posts: 4
Default Re: Problem 1.10 (b)

Quote:
Originally Posted by c.thejaswi View Post
I am sorry for the confusion. I got it.
Thanks.
thanks
Reply With Quote
  #6  
Old 04-01-2016, 06:32 AM
MaciekLeks MaciekLeks is offline
Member
 
Join Date: Jan 2016
Location: Katowice, Upper Silesia, Poland
Posts: 17
Default Re: Problem 1.10 (b)

IMHO, the answer to (b) is not an infinite number. As for part (a) the anwer is not \frac{1}{2}, so here the anwer is not as simple as it seems.

My way of thinking is as follows:
We must recall that the assumption from (a) does not work here, and we still do not know function f. We also do not know the dimensionality of the datapoint in the input space \mathcal{X}=\{\mathbf{x}_{1,}\mathbf{x}_{2},...,\mathbf{x}_{N},\mathbf{x}_{N+1},...,\mathbf{x}_{N+M}\} but we know that this input space is fixed (all \mathbf{x}_{k} for 1\leq k\leq N+M are already set). In this case we have \mathcal{D} of size N generated in a deterministic way, and y_{n}=f(\mathbf{x}_{n})(1\leq n\leq N) is not affected by any noise. So, how many possible f:\mathcal{X}\mathcal{\rightarrow\mathcal{Y}} can 'generate' \mathcal{D}? The subtle point in this case is the assumption: “For a fixed \mathcal{D} of size N”, which means that \mathcal{D}=\{(\mathbf{x}_{1},y_{1}),(\mathbf{x}_{2},y_{2}),...,(\mathbf{x}_{N},y_{n})\} is already generated. We can calculate how many possible outputs y_{n} for 1\leq n\leq N we can get? Only 1. But there are remaining \{\mathbf{x}_{N+1},...,\mathbf{x}_{N+M}\} datapoints for which we can have 2^{M} possible values \{-1,1\}. So, the anwer is 2^{M}.

Am I wrong?
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -7. The time now is 03:29 AM.


Powered by vBulletin® Version 3.8.3
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, and participants in the Learning From Data MOOC by Yaser S. Abu-Mostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.