LFD Book Forum

LFD Book Forum (http://book.caltech.edu/bookforum/index.php)
-   Chapter 4 - Overfitting (http://book.caltech.edu/bookforum/forumdisplay.php?f=111)
-   -   Chapter 4 Problem 4.4 (http://book.caltech.edu/bookforum/showthread.php?t=4418)

GB449 09-17-2013 08:39 PM

Chapter 4 Problem 4.4
 
I have the following questions. I have a few more on using Legendre polynomials but I will read 4.3 first before asking :)

a) what does selecting x_1, x_2, ... x_n independently with P(x) mean? In earlier homework, we selected the points at random from [-1,1] which was clear to me - I used a random function with min = -1 and max = +1 but I am not clear what to do with P(x)

b) y_n is clear since it is f(x_n). Should we generate random numbers for each epsilon_n? If so, what are the min and max values for the random numbers?

Thank you!

magdon 09-19-2013 05:09 AM

Re: Chapter 4 Problem 4.4
 
Quote:

Originally Posted by GB449 (Post 11490)
a) what does selecting x_1, x_2, ... x_n independently with P(x) mean? In earlier homework, we selected the points at random from [-1,1] which was clear to me - I used a random function with min = -1 and max = +1 but I am not clear what to do with P(x)

In this problem P(x) is the uniform probability distribution on [-1,+1] and so what you did in the earlier homework is exactly what this means -- generate a random number between -1 and +1. In general, the way to generate x may not be uniformly random over [-1,+1]. It is P(\mathbf{x}) that specifies the input probability distribution. For example, if P(x) is the Gaussian distribution with mean 0 and variance 1, then you would generate each x_n from that Gaussian distribution.

Quote:

b) y_n is clear since it is f(x_n). Should we generate random numbers for each \epsilon_n? If so, what are the min and max values for the random numbers?
Note y_n=f(\mathbf{x}_n)+\sigma\epsilon_n. Each \epsilon_n should be an independent Gaussian random number with mean 0 and variance 1 (you then multiply this random number by \sigma).

GB449 09-21-2013 01:24 AM

Re: Chapter 4 Problem 4.4
 
Thank you for the reply! How can I generate Gaussian random numbers with mean 0 and variance 1? I am programming in C#.

magdon 09-21-2013 05:33 AM

Re: Chapter 4 Problem 4.4
 
Quote:

Originally Posted by GB449 (Post 11503)
Thank you for the reply! How can I generate Gaussian random numbers with mean 0 and variance 1? I am programming in C#.

This is a common task for which standard algorithms exist in most numerical packages. For example, in matlab there is a function randn. You can also do this using the Box-Muller transform. You may find this thread useful:

http://stackoverflow.com/questions/2...sian-variables

Kleber de Aguiar 12-03-2014 01:06 PM

Re: Chapter 4 Problem 4.4
 
Hello,

just another question about this problem:

What I supposed to do to select the coefficients ai, in order to respect the restriction of Ea,x [f^2] = 1 ?

Best regards,
Kleber

ypeels 01-12-2015 11:15 PM

Re: Chapter 4 Problem 4.4
 
Quick question: what does the "LAMi" at the beginning of the problem mean?

magdon 01-13-2015 07:05 AM

Re: Chapter 4 Problem 4.4
 
Spurious typo that snuck in, sorry.
Quote:

Originally Posted by ypeels (Post 11885)
Quick question: what does the "LAMi" at the beginning of the problem mean?


magdon 01-13-2015 07:06 AM

Re: Chapter 4 Problem 4.4
 
Rescale all the weights by a constant factor so that Ea,x [f^2] = 1.
Quote:

Originally Posted by Kleber de Aguiar (Post 11857)
Hello,

just another question about this problem:

What I supposed to do to select the coefficients ai, in order to respect the restriction of Ea,x [f^2] = 1 ?

Best regards,
Kleber


715073608 07-05-2018 11:52 PM

Re: Chapter 4 Problem 4.4
 
Dear professor,can you explain what does the question(e) mean and how can I draw the pictures on page 144?Thank you .


All times are GMT -7. The time now is 02:31 PM.

Powered by vBulletin® Version 3.8.3
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, and participants in the Learning From Data MOOC by Yaser S. Abu-Mostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.