LFD Book Forum  

Go Back   LFD Book Forum > Course Discussions > Online LFD course > Homework 7

Thread Tools Display Modes
Old 01-27-2016, 11:50 PM
galo galo is offline
Junior Member
Join Date: Jan 2016
Posts: 7
Default REQUEST: Q7 How-To

Can anybody share the rundown to get the answer? I can't grasp a way to do it. Plus it gets frustrating because I feel like it's more of an algebra problem rather than a Machine Learning one.
Reply With Quote
Old 08-14-2016, 03:56 PM
tddevlin tddevlin is offline
Junior Member
Join Date: Feb 2016
Posts: 1
Default Re: REQUEST: Q7 How-To

**Spoiler Alert: This post contains the full solution**

First let's make sure we have the right picture.

So P_1 and P_2 are sitting on the x-axis, while P_3 is somewhere to the right of the y-axis at height 1. For this dataset, leave-one-out validation entails fitting our model to two of the points, then testing the fit on the third. Let's start with the constant model, h_0(x) = b. When we fit this model on two data points, b will simply be the average of the y-coordinates of the two points.
  • Leaving P_1 out, we find b = 1/2. The error is e_1 = (h_0(x_1) - y_1)^2 = (b - 0)^2 = (1/2)^2.
  • Leaving P_2 out, we also find b = 1/2. Again, e_2 = (h_0(x_2) - y_2)^2 = (b - 0)^2 = (1/2)^2.
  • Finally, leaving P_3 out, b = 0. The error is e_3 = (h_0(x_3) - y_3)^2 = (0 - 1)^2 = 1.

The overall cross-validation error is the average of the three individual errors, E_{\text{val}}(h_0) = \frac{e_1 + e_2 + e_3}{3} = 1/2, as you can verify. Looking ahead, we would like to find the value of \rho that makes E_{\text{val}}(h_1) = 1/2.

Let's turn to the linear model, h_1(x) = ax + b. The easy case is when P_3 is left out. The resulting fitted line is simply y = 0 and the error is e_3 = 1.

Things get more complicated when P_2 is left out. We need to find the equation of the line through P_1 and P_3. Using slope-intercept form and rearranging, you can check that the fitted line has slope equal to its intercept, a = b = \frac{1}{1 + \rho}. The error on P_2 is e_2 = (h_1(x_2) - y_2)^2 = (ax_2 + b - 0)^2 = \left( \frac{2}{1+\rho} \right)^2.

A similar derivation yields e_3 =  \left( \frac{-2}{\rho - 1} \right)^2.

Putting it all together gives us E_{\text{val}}(h_1) = \frac{1}{3} \left[ 1 + \left( \frac{2}{1+\rho} \right)^2+ \left( \frac{-2}{\rho - 1} \right)^2 \right]. If we set this equal to 1/2 (the error from the constant model), we have a quadratic equation in one unknown, which we can solve using the quadratic formula (alternatively, dumping the whole equation into WolframAlpha gives you the roots directly).

Hope that helped!
Reply With Quote
Old 12-08-2021, 04:01 AM
RJT_12 RJT_12 is offline
Junior Member
Join Date: Dec 2016
Posts: 8
Default Re: REQUEST: Q7 How-To


If we assume constant model and leave out point whit rho included i.e. (rho, 1), the mid value appears to be y axis. (mid point between (-1,0) and (1,0)).

The difference between y axis and the point (rho, 1) is rho and error term is rho^2. However, using this error for constant model does not lead to any of the answers.

However if assume error for leave out point (rho, 1) be 1, than I arrive at correct solution.

But I think rho^2 error is correct. Please help me to find where I make mistake,
Reply With Quote

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT -7. The time now is 10:49 PM.

Powered by vBulletin® Version 3.8.3
Copyright ©2000 - 2022, Jelsoft Enterprises Ltd.
The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, and participants in the Learning From Data MOOC by Yaser S. Abu-Mostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.