*ANSWER* Q13 about linearly separable by SVM
I would like to take this chance to express my sincere thanks to Prof. Yaser S. AbuMostafa. This is an extremely good class. I had watched at least twice for each of the video. There are still a lot to learn after the course.
Still, I have some doubts. E.g., isn't it the RBF can result in an hypothesis for an infinite dimension space ? If that is the case, then with 100 points in Q14, the worse case is the SVM + SBF ends up with 100 support vector. Am I right ? Then, the hard margin SVM actually can guarantee Ein=0 => guarantee linearly separable ? Why Q14 can end up with Ein != 0 ? And why it said Ein=0 is not linearly separable ? I read another thread about this but still cannot solve the above questions. Can anyone give me some hints ? Thanks in advance. 
Re: Q14 about linearly separable by SVM
You are right, but in the lectures we did not prove that for the RBF kernel, so it was worth exploring the question at least empirically.
In general, it is conceivable that a transformed infinitedimensional space may still fail to separate a finite data set. For instance, take every dimension in the infinitedimensional space to be a (possibly different) linear transformation of the original space. In this case, you would still be implementing just a linear classifier in the original space when you use a linear classifier in the transformed space, so you will fail to separate a set of points that is not linearly separable in the original space. 
Re: *ANSWER* Q14 about linearly separable by SVM
Quote:
Quote:
Quote:

Re: *ANSWER* Q14 about linearly separable by SVM
My thoughts on skwong's post:
(1) There are reasons why SVM is a major workhorse of machine learning, while PLA is mainly found early in machine learning courses and books (and the RBF regular form is another method that is not popular. EDIT: thanks, Yaser for the information that it used to be used more). And it's not mere fashion! In realistic linearly separable situations, SVM gives better generalisation than PLA. It also usually gives better generalisation than RBF regular form. It's that really matters, not ! The advantage over PLA would extend to PLA with nonlinear transforms. As well as that, for problems with not too many data points, SVM is computationally efficient. Moreover, soft margin SVM is a major tool for classification where there is either not enough data or noise (it's a struggle to get useful results from PLA for these: the pocket algorithm is a bit like taking shots in the dark here, whereas SVM heads straight to the global optimum solution). (2) see (1) (3) The w is simply a natural way of representing a hyperplane. The relationship to polynomials is that polynomial models become linear models when viewed in the transformed space (with dimensions for each power of x). This is worth studying. (4) Yes (5) I think you are right. RBF regular form tends not to generalise as well as SVM in realistic scenarios, hence people use SVM (spot the recurring theme? :) ) 
Re: *ANSWER* Q14 about linearly separable by SVM
Many many thanks to Yaser and Elroch.

All times are GMT 7. The time now is 01:15 AM. 
Powered by vBulletin® Version 3.8.3
Copyright ©2000  2021, Jelsoft Enterprises Ltd.
The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. AbuMostafa, Malik MagdonIsmail, and HsuanTien Lin, and participants in the Learning From Data MOOC by Yaser S. AbuMostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.