LFD Book Forum  

Go Back   LFD Book Forum > Course Discussions > Online LFD course > Homework 4

Reply
 
Thread Tools Display Modes
  #1  
Old 01-31-2013, 10:36 AM
ripande ripande is offline
Senior Member
 
Join Date: Jan 2013
Posts: 71
Default Calculating Average Hypothesis

I guess this question has been asked before, but I am not very clear on it.

I understand calculating the average hypothesis of a constant i.e. taking average of the constant values of differnet hypothesis.

But what does calculating average hypothesis of a hypothesis set which consists of lines in y= mx+c mean? How is it calculated ?
Reply With Quote
  #2  
Old 01-31-2013, 10:44 AM
ripande ripande is offline
Senior Member
 
Join Date: Jan 2013
Posts: 71
Default Re: Calculating Average Hypothesis

I read the earlier post on this question once again. The concept is clear to me now
Reply With Quote
  #3  
Old 02-01-2013, 10:06 AM
ripande ripande is offline
Senior Member
 
Join Date: Jan 2013
Posts: 71
Default Re: Calculating Average Hypothesis

I wanted to validate that what I have understood is correct.

1. I calculated the value of "a" for which the which minimizes the least square for two points ( x, sin(pi*x) ), x being between -1 and 1.

2. Repeated the above for 100 times and hence got 100 values of "a"

3. Then I chose a fresh point x3 between [-1, 1] and calculated the value of y3 = a*x3 for all 100 points

4. Calculated average value of y3 for 100 points, say y_avg.

5. Calculated "a" for avg hypothesis as : y_avg/x3

Is my method of calculating avg hypothesis correct ? Iam not very confident of the answer I am getting
Reply With Quote
  #4  
Old 02-01-2013, 11:11 AM
yaser's Avatar
yaser yaser is offline
Caltech
 
Join Date: Aug 2009
Location: Pasadena, California, USA
Posts: 1,477
Default Re: Calculating Average Hypothesis

Quote:
Originally Posted by ripande View Post
1. I calculated the value of "a" for which the which minimizes the least square for two points ( x, sin(pi*x) ), x being between -1 and 1.

2. Repeated the above for 100 times and hence got 100 values of "a"
These steps are correct (with ax instead of x in step 1) in calculating the final hypothesis g^{\cal D} for 100 different sets {\cal D}.

Quote:
3. Then I chose a fresh point x3 between [-1, 1] and calculated the value of y3 = a*x3 for all 100 points
This step evaluates g^{\cal D}(x_3) for each {\cal D} in the 100 runs. If x_3 is fixed for all 100 runs, this step can be used to evaluate the bias and variance at the point x_3 (namely {\bf bias}(x_3) and {\bf var}(x_3)).

Quote:
4. Calculated average value of y3 for 100 points, say y_avg.
If the 100 points are the same x_3 with different g^{\cal D}, then the average approximates {\bar g}(x_3). If the points are different, I am not sure about the utility of this quantity for the calculation of bias and variance.

Quote:
5. Calculated "a" for avg hypothesis as : y_avg/x3
You already have the different values of a for different data sets g^{\cal D} (these are the values of a that you used to calculated y_3 from x_3). Because the formula for the hypothesis is linear in a, you can directly calculate a of the average hypothesis by averaging all the a's. What you are suggesting is equivalent in this case.
__________________
Where everyone thinks alike, no one thinks very much
Reply With Quote
  #5  
Old 02-01-2013, 12:02 PM
ripande ripande is offline
Senior Member
 
Join Date: Jan 2013
Posts: 71
Default Re: Calculating Average Hypothesis

Thanks Prof Yaser for the clarification.

So, if understand correctly,since the hypothesis is linear in "a", I can directly take the average of the value "a" in all hypothesis to calculate the average hypothesis; and what I was doing was just equivalent to it.

What should be done in a hypothetical case where the value had not been linear in the hypothesis set?
Reply With Quote
  #6  
Old 02-01-2013, 12:28 PM
yaser's Avatar
yaser yaser is offline
Caltech
 
Join Date: Aug 2009
Location: Pasadena, California, USA
Posts: 1,477
Default Re: Calculating Average Hypothesis

Quote:
Originally Posted by ripande View Post
So, if understand correctly,since the hypothesis is linear in "a", I can directly take the average of the value "a" in all hypothesis to calculate the average hypothesis; and what I was doing was just equivalent to it.
Exactly.

Quote:
What should be done in a hypothetical case where the value had not been linear in the hypothesis set?
You average the hypothesis value g^{\cal D}({\bf x}) at each point {\bf x}, rather than average the parameters.
__________________
Where everyone thinks alike, no one thinks very much
Reply With Quote
  #7  
Old 02-01-2013, 08:24 PM
gah44 gah44 is offline
Invited Guest
 
Join Date: Jul 2012
Location: Seattle, WA
Posts: 153
Default Re: Calculating Average Hypothesis

To help make sure I understood this, I tried to determine the red line for H1 in Example 2.8. That is, the average hypotheses for h(x)=ax+b.

But I get a=0 and b=0, unlike the figure.

I average over a matrix of points, ignoring x1=x2 points.

Anything I should do differently?

I have:

a=(y1-y2)/(x1-x2)
b=(x1*y2-x2*y1)/(x1-x2)

but a averages to zero.
Reply With Quote
  #8  
Old 02-01-2013, 08:33 PM
gah44 gah44 is offline
Invited Guest
 
Join Date: Jul 2012
Location: Seattle, WA
Posts: 153
Default Re: Calculating Average Hypothesis

OK, I was summing the wrong variable.

It works now.

a=0.82 b=0

as it looks on the graph.

Now see if I can get the bias and variance for the graph.
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -7. The time now is 10:41 PM.


Powered by vBulletin® Version 3.8.3
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, and participants in the Learning From Data MOOC by Yaser S. Abu-Mostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.