LFD Book Forum Support Vector Machines, Kernel Functions, Data Snooping
 User Name Remember Me? Password
 FAQ Calendar Mark Forums Read

 Thread Tools Display Modes
#1
05-30-2012, 09:09 AM
 dbl001 Member Join Date: Apr 2012 Posts: 11
Support Vector Machines, Kernel Functions, Data Snooping

Hi,

When you select a kernel function for a support vector machine,
do you 'peek' at the data? How do you select the kernel?

If yes, does this count as 'data snooping'?

Thanks
#2
05-30-2012, 11:35 AM
 dudefromdayton Invited Guest Join Date: Apr 2012 Posts: 140
Re: Support Vector Machines, Kernel Functions, Data Snooping

This answer isn't perfect by any means, but the libsvm FAQ offers this answer. If you decide in advance that you'll stop there no matter what, you haven't snooped.

If you want to try several kernels, use your selection as a parameter for validation. Then you snoop in the same way you might choose C, etc.

If you have knowledge as to how the data are produced (as opposed to what the data are), this might help you pick a kernel. For example, if you expect without snooping that the response should be symmetric in two variables, pick a kernel such as RBF that takes advantage of it. Example: homework 2 problems 8-10 use an unknown f that is symmetric in x1 and x2.
#3
05-30-2012, 01:13 PM
 mikesakiandcp Member Join Date: Apr 2012 Posts: 31
Re: Support Vector Machines, Kernel Functions, Data Snooping

If you're sticking with the same learning algorithm, I think you can account for the amount of snooping you're doing by expanding your hypothesis set. For example, if we had the non-linearly separable case in class where the target function was a circle. We could start with H1, so we have weights for x0, x1, and x2. If we see poor generalization (high Ein or Eval), we can then go to H2, so now we have:
phi = { 1, x1, x2, x1^2, x1*x2, x2^2 }

Since H2 includes H1, these are counted towards the VC-dimension and we should be ok. Where we would get in trouble with snooping is doing something like realizing H1 didn't work and going to this:
phi = {1, x1^2, x1*x2, x2^2}

And not charging for using the x1 & x2, even though you already tested using them in a previous run

 Tags data snooping, kernel methods, support vector machines

 Thread Tools Display Modes Linear Mode

 Posting Rules You may not post new threads You may not post replies You may not post attachments You may not edit your posts BB code is On Smilies are On [IMG] code is On HTML code is Off Forum Rules
 Forum Jump User Control Panel Private Messages Subscriptions Who's Online Search Forums Forums Home General     General Discussion of Machine Learning     Free Additional Material         Dynamic e-Chapters         Dynamic e-Appendices Course Discussions     Online LFD course         General comments on the course         Homework 1         Homework 2         Homework 3         Homework 4         Homework 5         Homework 6         Homework 7         Homework 8         The Final         Create New Homework Problems Book Feedback - Learning From Data     General comments on the book     Chapter 1 - The Learning Problem     Chapter 2 - Training versus Testing     Chapter 3 - The Linear Model     Chapter 4 - Overfitting     Chapter 5 - Three Learning Principles     e-Chapter 6 - Similarity Based Methods     e-Chapter 7 - Neural Networks     e-Chapter 8 - Support Vector Machines     e-Chapter 9 - Learning Aides     Appendix and Notation     e-Appendices

All times are GMT -7. The time now is 06:35 AM.

 Contact Us - LFD Book - Top

Powered by vBulletin® Version 3.8.3
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.
The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, and participants in the Learning From Data MOOC by Yaser S. Abu-Mostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.