LFD Book Forum  

Go Back   LFD Book Forum > Book Feedback - Learning From Data > Chapter 4 - Overfitting

Reply
 
Thread Tools Display Modes
  #1  
Old 09-17-2013, 07:39 PM
GB449 GB449 is offline
Member
 
Join Date: Aug 2013
Posts: 20
Default Chapter 4 Problem 4.4

I have the following questions. I have a few more on using Legendre polynomials but I will read 4.3 first before asking

a) what does selecting x_1, x_2, ... x_n independently with P(x) mean? In earlier homework, we selected the points at random from [-1,1] which was clear to me - I used a random function with min = -1 and max = +1 but I am not clear what to do with P(x)

b) y_n is clear since it is f(x_n). Should we generate random numbers for each epsilon_n? If so, what are the min and max values for the random numbers?

Thank you!
Reply With Quote
  #2  
Old 09-19-2013, 04:09 AM
magdon's Avatar
magdon magdon is offline
RPI
 
Join Date: Aug 2009
Location: Troy, NY, USA.
Posts: 595
Default Re: Chapter 4 Problem 4.4

Quote:
Originally Posted by GB449 View Post
a) what does selecting x_1, x_2, ... x_n independently with P(x) mean? In earlier homework, we selected the points at random from [-1,1] which was clear to me - I used a random function with min = -1 and max = +1 but I am not clear what to do with P(x)
In this problem P(x) is the uniform probability distribution on [-1,+1] and so what you did in the earlier homework is exactly what this means -- generate a random number between -1 and +1. In general, the way to generate x may not be uniformly random over [-1,+1]. It is P(\mathbf{x}) that specifies the input probability distribution. For example, if P(x) is the Gaussian distribution with mean 0 and variance 1, then you would generate each x_n from that Gaussian distribution.

Quote:
b) y_n is clear since it is f(x_n). Should we generate random numbers for each \epsilon_n? If so, what are the min and max values for the random numbers?
Note y_n=f(\mathbf{x}_n)+\sigma\epsilon_n. Each \epsilon_n should be an independent Gaussian random number with mean 0 and variance 1 (you then multiply this random number by \sigma).
__________________
Have faith in probability
Reply With Quote
  #3  
Old 09-21-2013, 12:24 AM
GB449 GB449 is offline
Member
 
Join Date: Aug 2013
Posts: 20
Default Re: Chapter 4 Problem 4.4

Thank you for the reply! How can I generate Gaussian random numbers with mean 0 and variance 1? I am programming in C#.
Reply With Quote
  #4  
Old 09-21-2013, 04:33 AM
magdon's Avatar
magdon magdon is offline
RPI
 
Join Date: Aug 2009
Location: Troy, NY, USA.
Posts: 595
Default Re: Chapter 4 Problem 4.4

Quote:
Originally Posted by GB449 View Post
Thank you for the reply! How can I generate Gaussian random numbers with mean 0 and variance 1? I am programming in C#.
This is a common task for which standard algorithms exist in most numerical packages. For example, in matlab there is a function randn. You can also do this using the Box-Muller transform. You may find this thread useful:

http://stackoverflow.com/questions/2...sian-variables
__________________
Have faith in probability
Reply With Quote
  #5  
Old 12-03-2014, 12:06 PM
Kleber de Aguiar Kleber de Aguiar is offline
Junior Member
 
Join Date: Dec 2014
Posts: 1
Thumbs down Re: Chapter 4 Problem 4.4

Hello,

just another question about this problem:

What I supposed to do to select the coefficients ai, in order to respect the restriction of Ea,x [f^2] = 1 ?

Best regards,
Kleber
Reply With Quote
  #6  
Old 01-12-2015, 10:15 PM
ypeels ypeels is offline
Member
 
Join Date: Dec 2014
Posts: 17
Default Re: Chapter 4 Problem 4.4

Quick question: what does the "LAMi" at the beginning of the problem mean?
Reply With Quote
  #7  
Old 01-13-2015, 06:05 AM
magdon's Avatar
magdon magdon is offline
RPI
 
Join Date: Aug 2009
Location: Troy, NY, USA.
Posts: 595
Default Re: Chapter 4 Problem 4.4

Spurious typo that snuck in, sorry.
Quote:
Originally Posted by ypeels View Post
Quick question: what does the "LAMi" at the beginning of the problem mean?
__________________
Have faith in probability
Reply With Quote
  #8  
Old 01-13-2015, 06:06 AM
magdon's Avatar
magdon magdon is offline
RPI
 
Join Date: Aug 2009
Location: Troy, NY, USA.
Posts: 595
Default Re: Chapter 4 Problem 4.4

Rescale all the weights by a constant factor so that Ea,x [f^2] = 1.
Quote:
Originally Posted by Kleber de Aguiar View Post
Hello,

just another question about this problem:

What I supposed to do to select the coefficients ai, in order to respect the restriction of Ea,x [f^2] = 1 ?

Best regards,
Kleber
__________________
Have faith in probability
Reply With Quote
  #9  
Old 07-05-2018, 10:52 PM
715073608 715073608 is offline
Member
 
Join Date: May 2018
Posts: 10
Default Re: Chapter 4 Problem 4.4

Dear professor,can you explain what does the question(e) mean and how can I draw the pictures on page 144?Thank you .
Reply With Quote
Reply

Tags
chapter 4, lecture 11

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -7. The time now is 03:19 AM.


Powered by vBulletin® Version 3.8.3
Copyright ©2000 - 2018, Jelsoft Enterprises Ltd.
The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, and participants in the Learning From Data MOOC by Yaser S. Abu-Mostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.