LFD Book Forum

LFD Book Forum (http://book.caltech.edu/bookforum/index.php)
-   Homework 4 (http://book.caltech.edu/bookforum/forumdisplay.php?f=133)
-   -   Calculating Average Hypothesis (http://book.caltech.edu/bookforum/showthread.php?t=3929)

ripande 01-31-2013 10:36 AM

Calculating Average Hypothesis
 
I guess this question has been asked before, but I am not very clear on it.

I understand calculating the average hypothesis of a constant i.e. taking average of the constant values of differnet hypothesis.

But what does calculating average hypothesis of a hypothesis set which consists of lines in y= mx+c mean? How is it calculated ?

ripande 01-31-2013 10:44 AM

Re: Calculating Average Hypothesis
 
I read the earlier post on this question once again. The concept is clear to me now

ripande 02-01-2013 10:06 AM

Re: Calculating Average Hypothesis
 
I wanted to validate that what I have understood is correct.

1. I calculated the value of "a" for which the which minimizes the least square for two points ( x, sin(pi*x) ), x being between -1 and 1.

2. Repeated the above for 100 times and hence got 100 values of "a"

3. Then I chose a fresh point x3 between [-1, 1] and calculated the value of y3 = a*x3 for all 100 points

4. Calculated average value of y3 for 100 points, say y_avg.

5. Calculated "a" for avg hypothesis as : y_avg/x3

Is my method of calculating avg hypothesis correct ? Iam not very confident of the answer I am getting

yaser 02-01-2013 11:11 AM

Re: Calculating Average Hypothesis
 
Quote:

Originally Posted by ripande (Post 9106)
1. I calculated the value of "a" for which the which minimizes the least square for two points ( x, sin(pi*x) ), x being between -1 and 1.

2. Repeated the above for 100 times and hence got 100 values of "a"

These steps are correct (with ax instead of x in step 1) in calculating the final hypothesis g^{\cal D} for 100 different sets {\cal D}.

Quote:

3. Then I chose a fresh point x3 between [-1, 1] and calculated the value of y3 = a*x3 for all 100 points
This step evaluates g^{\cal D}(x_3) for each {\cal D} in the 100 runs. If x_3 is fixed for all 100 runs, this step can be used to evaluate the bias and variance at the point x_3 (namely {\bf bias}(x_3) and {\bf var}(x_3)).

Quote:

4. Calculated average value of y3 for 100 points, say y_avg.
If the 100 points are the same x_3 with different g^{\cal D}, then the average approximates {\bar g}(x_3). If the points are different, I am not sure about the utility of this quantity for the calculation of bias and variance.

Quote:

5. Calculated "a" for avg hypothesis as : y_avg/x3
You already have the different values of a for different data sets g^{\cal D} (these are the values of a that you used to calculated y_3 from x_3). Because the formula for the hypothesis is linear in a, you can directly calculate a of the average hypothesis by averaging all the a's. What you are suggesting is equivalent in this case.

ripande 02-01-2013 12:02 PM

Re: Calculating Average Hypothesis
 
Thanks Prof Yaser for the clarification.

So, if understand correctly,since the hypothesis is linear in "a", I can directly take the average of the value "a" in all hypothesis to calculate the average hypothesis; and what I was doing was just equivalent to it.

What should be done in a hypothetical case where the value had not been linear in the hypothesis set?

yaser 02-01-2013 12:28 PM

Re: Calculating Average Hypothesis
 
Quote:

Originally Posted by ripande (Post 9118)
So, if understand correctly,since the hypothesis is linear in "a", I can directly take the average of the value "a" in all hypothesis to calculate the average hypothesis; and what I was doing was just equivalent to it.

Exactly.

Quote:

What should be done in a hypothetical case where the value had not been linear in the hypothesis set?
You average the hypothesis value g^{\cal D}({\bf x}) at each point {\bf x}, rather than average the parameters.

gah44 02-01-2013 08:24 PM

Re: Calculating Average Hypothesis
 
To help make sure I understood this, I tried to determine the red line for H1 in Example 2.8. That is, the average hypotheses for h(x)=ax+b.

But I get a=0 and b=0, unlike the figure.

I average over a matrix of points, ignoring x1=x2 points.

Anything I should do differently?

I have:

a=(y1-y2)/(x1-x2)
b=(x1*y2-x2*y1)/(x1-x2)

but a averages to zero.

gah44 02-01-2013 08:33 PM

Re: Calculating Average Hypothesis
 
OK, I was summing the wrong variable.

It works now.

a=0.82 b=0

as it looks on the graph.

Now see if I can get the bias and variance for the graph.


All times are GMT -7. The time now is 08:26 AM.

Powered by vBulletin® Version 3.8.3
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, and participants in the Learning From Data MOOC by Yaser S. Abu-Mostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.