 LFD Book Forum Why would variance be non-zero?
 User Name Remember Me? Password
 FAQ Calendar Mark Forums Read

 Thread Tools Display Modes
#1
 samirbajaj Member Join Date: Jul 2012 Location: Silicon Valley Posts: 48 Why would variance be non-zero?

In question 6 on the homework, we are asked to compute the variance across all data sets.

If we are sampling uniformly from the interval [-1, 1] for the calculation of g_bar, as well as for each data set (g_d), why would the variance be anything but a very small quantity? In the general case, when the data sets are not drawn from a uniform distribution, a non-zero variance makes sense, but if there is sufficient overlap in the data sets, it makes intuitive sense that the variance should be close to zero.

I ask this because my simulation results support the above (potentially flawed) theory.

Please answer the question in general terms -- I don't care about the homework answer -- I was merely using that as an example.

Thanks for any input.

-Samir
#2 yaser Caltech Join Date: Aug 2009 Location: Pasadena, California, USA Posts: 1,477 Re: Why would variance be non-zero?

Quote:
 Originally Posted by samirbajaj If we are sampling uniformly from the interval [-1, 1] for the calculation of g_bar, as well as for each data set (g_d), why would the variance be anything but a very small quantity? In the general case, when the data sets are not drawn from a uniform distribution, a non-zero variance makes sense, but if there is sufficient overlap in the data sets, it makes intuitive sense that the variance should be close to zero. is the average of over different data sets . You will get different 's when you pick different 's, since the final hypothesis depends on the data set used for training. Therefore, there will be a variance that measures how different these 's are around their expected value (which does not depend on as gets integrated out in the calculation of ).

This argument holds for any probability distribution, uniform or not, that is used to generate the different 's.
__________________
Where everyone thinks alike, no one thinks very much
#3
 samirbajaj Member Join Date: Jul 2012 Location: Silicon Valley Posts: 48 Re: Why would variance be non-zero?

Thank you ... now that you explain it that way, it makes perfect sense. (Not sure what I was thinking...)

-Samir

 Tags variance Thread Tools Show Printable Version Email this Page Display Modes Linear Mode Switch to Hybrid Mode Switch to Threaded Mode Posting Rules You may not post new threads You may not post replies You may not post attachments You may not edit your posts BB code is On Smilies are On [IMG] code is On HTML code is Off Forum Rules
 Forum Jump User Control Panel Private Messages Subscriptions Who's Online Search Forums Forums Home General     General Discussion of Machine Learning     Free Additional Material         Dynamic e-Chapters         Dynamic e-Appendices Course Discussions     Online LFD course         General comments on the course         Homework 1         Homework 2         Homework 3         Homework 4         Homework 5         Homework 6         Homework 7         Homework 8         The Final         Create New Homework Problems Book Feedback - Learning From Data     General comments on the book     Chapter 1 - The Learning Problem     Chapter 2 - Training versus Testing     Chapter 3 - The Linear Model     Chapter 4 - Overfitting     Chapter 5 - Three Learning Principles     e-Chapter 6 - Similarity Based Methods     e-Chapter 7 - Neural Networks     e-Chapter 8 - Support Vector Machines     e-Chapter 9 - Learning Aides     Appendix and Notation     e-Appendices

All times are GMT -7. The time now is 12:47 PM.

 Contact Us - LFD Book - Top