LFD Book Forum (http://book.caltech.edu/bookforum/index.php)
-   Homework 7 (http://book.caltech.edu/bookforum/forumdisplay.php?f=136)
-   -   REQUEST: Q7 How-To (http://book.caltech.edu/bookforum/showthread.php?t=4654)

 galo 01-27-2016 11:50 PM

REQUEST: Q7 How-To

Can anybody share the rundown to get the answer? I can't grasp a way to do it. Plus it gets frustrating because I feel like it's more of an algebra problem rather than a Machine Learning one.

 tddevlin 08-14-2016 03:56 PM

Re: REQUEST: Q7 How-To

**Spoiler Alert: This post contains the full solution**

First let's make sure we have the right picture.

https://i.imgsafe.org/0e07a8f7bb.png

So and are sitting on the -axis, while is somewhere to the right of the -axis at height 1. For this dataset, leave-one-out validation entails fitting our model to two of the points, then testing the fit on the third. Let's start with the constant model, . When we fit this model on two data points, will simply be the average of the -coordinates of the two points.
• Leaving out, we find . The error is .
• Leaving out, we also find . Again, .
• Finally, leaving out, . The error is .

The overall cross-validation error is the average of the three individual errors, , as you can verify. Looking ahead, we would like to find the value of that makes .

Let's turn to the linear model, . The easy case is when is left out. The resulting fitted line is simply and the error is .

Things get more complicated when is left out. We need to find the equation of the line through and . Using slope-intercept form and rearranging, you can check that the fitted line has slope equal to its intercept, . The error on is .

A similar derivation yields .

Putting it all together gives us . If we set this equal to 1/2 (the error from the constant model), we have a quadratic equation in one unknown, which we can solve using the quadratic formula (alternatively, dumping the whole equation into WolframAlpha gives you the roots directly).

Hope that helped!

 RJT_12 12-08-2021 04:01 AM

Re: REQUEST: Q7 How-To

Hi,

If we assume constant model and leave out point whit rho included i.e. (rho, 1), the mid value appears to be y axis. (mid point between (-1,0) and (1,0)).

The difference between y axis and the point (rho, 1) is rho and error term is rho^2. However, using this error for constant model does not lead to any of the answers.

However if assume error for leave out point (rho, 1) be 1, than I arrive at correct solution.