LFD Book Forum - computing w, b for soft margin SVM

LFD Book Forum (http://book.caltech.edu/bookforum/index.php)

- Homework 8 (http://book.caltech.edu/bookforum/forumdisplay.php?f=137)

- - computing w, b for soft margin SVM (http://book.caltech.edu/bookforum/showthread.php?t=4312)

computing w, b for soft margin SVM

When we compute w from the alphas for the soft-margin SVM which values of alpha do we include? only margin support vectors ( those with 0 < alpha_n < C)? or do we include all of them (0 < alpha_n <= C)?

Similarly, when computing b, can we use any support vector? or only margin support vectors?

Re: computing w, b for soft margin SVM

Quote:

Originally Posted by Katie C. (Post 10947)

1. All of them for computing ${\bf w}$ , since any vector with $\alpha>0$ will contribute to the derived solution for ${\bf w}$ .

2. Only margin SV's for computing

, since we need an equation, not an inequality, to solve for

after knowing ${\bf w}$ .

Re: computing w, b for soft margin SVM

Quote:

Originally Posted by yaser (Post 10949)

1. All of them for computing ${\bf w}$ , since any vector with $\alpha>0$ will contribute to the derived solution for ${\bf w}$ .

2. Only margin SV's for computing

, since we need an equation, not an inequality, to solve for

after knowing ${\bf w}$ .

Is the 'heuristic' number of parameters (the VC dimension proxy) to be used while reasoning about generalization then the number of margin support vectors << the number of all support vectors?

When we use kernel functions with soft SVMs (problem 2 etc), where there is no explicit w, does the above translate to :
* 1==> Use all support vectors to compute the sigma term in the hypothesis function g()
* 2==> Use only margin support vectors for b (which is also used in g()

I was wondering if this aspect was covered in the lecture or any of the additional material -- I seem to have missed.

Re: computing w, b for soft margin SVM

Quote:

Originally Posted by hsolo (Post 11301)

The computation of ${\bf w}$ involves all support vectors, margin and otherwise, since it involves all $\alpha$ 's that are bigger than zero. Assuming ${\bf w}$ has been computed, the computation of

, for both hard and soft margins, involves any one support vector (margin support vector in the case of soft margin) since it is based on solving the equation ${\bf w}^{\rm T} {\bf x}_n + b = 1$ for

.

In the case of kernels, the explicit evaluation of ${\bf w}$ followed by taking an inner product with a point ${\bf x}$ is replaced by evaluating the kernel with two arguments; one is a support vector (margin or otherwise) and the other is the point ${\bf x}$ , and repeating that for all support vectors (margin or otherwise).

Re: computing w, b for soft margin SVM