LFD Book Forum - Chapter 4 Problem 4.4

LFD Book Forum (http://book.caltech.edu/bookforum/index.php)

- Chapter 4 - Overfitting (http://book.caltech.edu/bookforum/forumdisplay.php?f=111)

- - Chapter 4 Problem 4.4 (http://book.caltech.edu/bookforum/showthread.php?t=4418)

Chapter 4 Problem 4.4

I have the following questions. I have a few more on using Legendre polynomials but I will read 4.3 first before asking :)

a) what does selecting

independently with P(x) mean? In earlier homework, we selected the points at random from [-1,1] which was clear to me - I used a random function with min = -1 and max = +1 but I am not clear what to do with P(x)

b)

is clear since it is f(

). Should we generate random numbers for each

? If so, what are the min and max values for the random numbers?

Thank you!

Re: Chapter 4 Problem 4.4

Quote:

Originally Posted by GB449 (Post 11490)

a) what does selecting

In this problem P(x) is the uniform probability distribution on [-1,+1] and so what you did in the earlier homework is exactly what this means -- generate a random number between -1 and +1. In general, the way to generate

may not be uniformly random over [-1,+1]. It is $P(\mathbf{x})$ that specifies the input probability distribution. For example, if

is the Gaussian distribution with mean 0 and variance 1, then you would generate each

from that Gaussian distribution.

Quote:

is clear since it is f(

). Should we generate random numbers for each $\epsilon_n$ ? If so, what are the min and max values for the random numbers?

Note $y_n=f(\mathbf{x}_n)+\sigma\epsilon_n$ . Each $\epsilon_n$ should be an independent Gaussian random number with mean 0 and variance 1 (you then multiply this random number by $\sigma$ ).

Re: Chapter 4 Problem 4.4

Thank you for the reply! How can I generate Gaussian random numbers with mean 0 and variance 1? I am programming in C#.

Re: Chapter 4 Problem 4.4

Quote:

Originally Posted by GB449 (Post 11503)

Thank you for the reply! How can I generate Gaussian random numbers with mean 0 and variance 1? I am programming in C#.

This is a common task for which standard algorithms exist in most numerical packages. For example, in matlab there is a function

. You can also do this using the Box-Muller transform. You may find this thread useful:

http://stackoverflow.com/questions/2...sian-variables

Re: Chapter 4 Problem 4.4

Hello,

just another question about this problem:

What I supposed to do to select the coefficients ai, in order to respect the restriction of Ea,x [f^2] = 1 ?

Best regards,
Kleber

Re: Chapter 4 Problem 4.4

Quick question: what does the "LAMi" at the beginning of the problem mean?

Re: Chapter 4 Problem 4.4

Spurious typo that snuck in, sorry.

Quote:

Originally Posted by ypeels (Post 11885)

Quick question: what does the "LAMi" at the beginning of the problem mean?

Re: Chapter 4 Problem 4.4

Rescale all the weights by a constant factor so that Ea,x [f^2] = 1.

Quote:

Originally Posted by Kleber de Aguiar (Post 11857)

Hello,

just another question about this problem:

What I supposed to do to select the coefficients ai, in order to respect the restriction of Ea,x [f^2] = 1 ?

Best regards,
Kleber

Re: Chapter 4 Problem 4.4

Dear professor,can you explain what does the question(e) mean and how can I draw the pictures on page 144?Thank you .