LFD Book Forum  

Go Back   LFD Book Forum > Course Discussions > Online LFD course > Homework 8

Reply
 
Thread Tools Display Modes
  #1  
Old 05-24-2013, 01:14 PM
Katie C. Katie C. is offline
Member
 
Join Date: Apr 2013
Posts: 17
Default computing w, b for soft margin SVM

When we compute w from the alphas for the soft-margin SVM which values of alpha do we include? only margin support vectors ( those with 0 < alpha_n < C)? or do we include all of them (0 < alpha_n <= C)?

Similarly, when computing b, can we use any support vector? or only margin support vectors?
Reply With Quote
  #2  
Old 05-24-2013, 05:31 PM
yaser's Avatar
yaser yaser is offline
Caltech
 
Join Date: Aug 2009
Location: Pasadena, California, USA
Posts: 1,477
Default Re: computing w, b for soft margin SVM

Quote:
Originally Posted by Katie C. View Post
When we compute w from the alphas for the soft-margin SVM which values of alpha do we include? only margin support vectors ( those with 0 < alpha_n < C)? or do we include all of them (0 < alpha_n <= C)?

Similarly, when computing b, can we use any support vector? or only margin support vectors?
1. All of them for computing {\bf w}, since any vector with \alpha>0 will contribute to the derived solution for {\bf w}.

2. Only margin SV's for computing b, since we need an equation, not an inequality, to solve for b after knowing {\bf w}.
__________________
Where everyone thinks alike, no one thinks very much
Reply With Quote
  #3  
Old 07-28-2013, 05:42 PM
hsolo hsolo is offline
Member
 
Join Date: Jul 2013
Posts: 12
Default Re: computing w, b for soft margin SVM

Quote:
Originally Posted by yaser View Post
1. All of them for computing {\bf w}, since any vector with \alpha>0 will contribute to the derived solution for {\bf w}.

2. Only margin SV's for computing b, since we need an equation, not an inequality, to solve for b after knowing {\bf w}.
Is the 'heuristic' number of parameters (the VC dimension proxy) to be used while reasoning about generalization then the number of margin support vectors << the number of all support vectors?

When we use kernel functions with soft SVMs (problem 2 etc), where there is no explicit w, does the above translate to :
* 1==> Use all support vectors to compute the sigma term in the hypothesis function g()
* 2==> Use only margin support vectors for b (which is also used in g()

I was wondering if this aspect was covered in the lecture or any of the additional material -- I seem to have missed.
Reply With Quote
  #4  
Old 07-29-2013, 01:01 AM
yaser's Avatar
yaser yaser is offline
Caltech
 
Join Date: Aug 2009
Location: Pasadena, California, USA
Posts: 1,477
Default Re: computing w, b for soft margin SVM

Quote:
Originally Posted by hsolo View Post
Is the 'heuristic' number of parameters (the VC dimension proxy) to be used while reasoning about generalization then the number of margin support vectors << the number of all support vectors?

When we use kernel functions with soft SVMs (problem 2 etc), where there is no explicit w, does the above translate to :
* 1==> Use all support vectors to compute the sigma term in the hypothesis function g()
* 2==> Use only margin support vectors for b (which is also used in g()

I was wondering if this aspect was covered in the lecture or any of the additional material -- I seem to have missed.
The computation of {\bf w} involves all support vectors, margin and otherwise, since it involves all \alpha's that are bigger than zero. Assuming {\bf w} has been computed, the computation of b, for both hard and soft margins, involves any one support vector (margin support vector in the case of soft margin) since it is based on solving the equation {\bf w}^{\rm T} {\bf x}_n + b = 1 for b.

In the case of kernels, the explicit evaluation of {\bf w} followed by taking an inner product with a point {\bf x} is replaced by evaluating the kernel with two arguments; one is a support vector (margin or otherwise) and the other is the point {\bf x}, and repeating that for all support vectors (margin or otherwise).
__________________
Where everyone thinks alike, no one thinks very much
Reply With Quote
  #5  
Old 03-04-2016, 07:04 AM
khohi khohi is offline
Member
 
Join Date: Dec 2015
Posts: 10
Default Re: computing w, b for soft margin SVM

thanks

الولادة الطبيعية
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -7. The time now is 03:00 PM.


Powered by vBulletin® Version 3.8.3
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, and participants in the Learning From Data MOOC by Yaser S. Abu-Mostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.