LFD Book Forum Calculating Average Hypothesis
 User Name Remember Me? Password
 Register FAQ Calendar Mark Forums Read

 Thread Tools Display Modes
#1
01-31-2013, 10:36 AM
 ripande Senior Member Join Date: Jan 2013 Posts: 71
Calculating Average Hypothesis

I guess this question has been asked before, but I am not very clear on it.

I understand calculating the average hypothesis of a constant i.e. taking average of the constant values of differnet hypothesis.

But what does calculating average hypothesis of a hypothesis set which consists of lines in y= mx+c mean? How is it calculated ?
#2
01-31-2013, 10:44 AM
 ripande Senior Member Join Date: Jan 2013 Posts: 71
Re: Calculating Average Hypothesis

I read the earlier post on this question once again. The concept is clear to me now
#3
02-01-2013, 10:06 AM
 ripande Senior Member Join Date: Jan 2013 Posts: 71
Re: Calculating Average Hypothesis

I wanted to validate that what I have understood is correct.

1. I calculated the value of "a" for which the which minimizes the least square for two points ( x, sin(pi*x) ), x being between -1 and 1.

2. Repeated the above for 100 times and hence got 100 values of "a"

3. Then I chose a fresh point x3 between [-1, 1] and calculated the value of y3 = a*x3 for all 100 points

4. Calculated average value of y3 for 100 points, say y_avg.

5. Calculated "a" for avg hypothesis as : y_avg/x3

Is my method of calculating avg hypothesis correct ? Iam not very confident of the answer I am getting
#4
02-01-2013, 11:11 AM
 yaser Caltech Join Date: Aug 2009 Location: Pasadena, California, USA Posts: 1,478
Re: Calculating Average Hypothesis

Quote:
 Originally Posted by ripande 1. I calculated the value of "a" for which the which minimizes the least square for two points ( x, sin(pi*x) ), x being between -1 and 1. 2. Repeated the above for 100 times and hence got 100 values of "a"
These steps are correct (with instead of in step 1) in calculating the final hypothesis for 100 different sets .

Quote:
 3. Then I chose a fresh point x3 between [-1, 1] and calculated the value of y3 = a*x3 for all 100 points
This step evaluates for each in the 100 runs. If is fixed for all 100 runs, this step can be used to evaluate the bias and variance at the point (namely and ).

Quote:
 4. Calculated average value of y3 for 100 points, say y_avg.
If the 100 points are the same with different , then the average approximates . If the points are different, I am not sure about the utility of this quantity for the calculation of bias and variance.

Quote:
 5. Calculated "a" for avg hypothesis as : y_avg/x3
You already have the different values of for different data sets (these are the values of that you used to calculated from ). Because the formula for the hypothesis is linear in , you can directly calculate of the average hypothesis by averaging all the 's. What you are suggesting is equivalent in this case.
__________________
Where everyone thinks alike, no one thinks very much
#5
02-01-2013, 12:02 PM
 ripande Senior Member Join Date: Jan 2013 Posts: 71
Re: Calculating Average Hypothesis

Thanks Prof Yaser for the clarification.

So, if understand correctly,since the hypothesis is linear in "a", I can directly take the average of the value "a" in all hypothesis to calculate the average hypothesis; and what I was doing was just equivalent to it.

What should be done in a hypothetical case where the value had not been linear in the hypothesis set?
#6
02-01-2013, 12:28 PM
 yaser Caltech Join Date: Aug 2009 Location: Pasadena, California, USA Posts: 1,478
Re: Calculating Average Hypothesis

Quote:
 Originally Posted by ripande So, if understand correctly,since the hypothesis is linear in "a", I can directly take the average of the value "a" in all hypothesis to calculate the average hypothesis; and what I was doing was just equivalent to it.
Exactly.

Quote:
 What should be done in a hypothetical case where the value had not been linear in the hypothesis set?
You average the hypothesis value at each point , rather than average the parameters.
__________________
Where everyone thinks alike, no one thinks very much
#7
02-01-2013, 08:24 PM
 gah44 Invited Guest Join Date: Jul 2012 Location: Seattle, WA Posts: 153
Re: Calculating Average Hypothesis

To help make sure I understood this, I tried to determine the red line for H1 in Example 2.8. That is, the average hypotheses for h(x)=ax+b.

But I get a=0 and b=0, unlike the figure.

I average over a matrix of points, ignoring x1=x2 points.

Anything I should do differently?

I have:

a=(y1-y2)/(x1-x2)
b=(x1*y2-x2*y1)/(x1-x2)

but a averages to zero.
#8
02-01-2013, 08:33 PM
 gah44 Invited Guest Join Date: Jul 2012 Location: Seattle, WA Posts: 153
Re: Calculating Average Hypothesis

OK, I was summing the wrong variable.

It works now.

a=0.82 b=0

as it looks on the graph.

Now see if I can get the bias and variance for the graph.

 Thread Tools Display Modes Linear Mode

 Posting Rules You may not post new threads You may not post replies You may not post attachments You may not edit your posts BB code is On Smilies are On [IMG] code is On HTML code is Off Forum Rules
 Forum Jump User Control Panel Private Messages Subscriptions Who's Online Search Forums Forums Home General     General Discussion of Machine Learning     Free Additional Material         Dynamic e-Chapters         Dynamic e-Appendices Course Discussions     Online LFD course         General comments on the course         Homework 1         Homework 2         Homework 3         Homework 4         Homework 5         Homework 6         Homework 7         Homework 8         The Final         Create New Homework Problems Book Feedback - Learning From Data     General comments on the book     Chapter 1 - The Learning Problem     Chapter 2 - Training versus Testing     Chapter 3 - The Linear Model     Chapter 4 - Overfitting     Chapter 5 - Three Learning Principles     e-Chapter 6 - Similarity Based Methods     e-Chapter 7 - Neural Networks     e-Chapter 8 - Support Vector Machines     e-Chapter 9 - Learning Aides     Appendix and Notation     e-Appendices

All times are GMT -7. The time now is 02:44 PM.

 Contact Us - LFD Book - Top