computing w, b for soft margin SVM

Katie C. · #1 05-24-2013, 01:14 PM

When we compute w from the alphas for the soft-margin SVM which values of alpha do we include? only margin support vectors ( those with 0 < alpha_n < C)? or do we include all of them (0 < alpha_n <= C)?

Similarly, when computing b, can we use any support vector? or only margin support vectors?

yaser · #2 05-24-2013, 05:31 PM

Quote:

Originally Posted by Katie C.

When we compute w from the alphas for the soft-margin SVM which values of alpha do we include? only margin support vectors ( those with 0 < alpha_n < C)? or do we include all of them (0 < alpha_n <= C)?

Similarly, when computing b, can we use any support vector? or only margin support vectors?

1. All of them for computing ${\bf w}$ , since any vector with $\alpha>0$ will contribute to the derived solution for ${\bf w}$ .

2. Only margin SV's for computing

, since we need an equation, not an inequality, to solve for

after knowing ${\bf w}$ .

hsolo · #3 07-28-2013, 05:42 PM

Quote:

Originally Posted by yaser

1. All of them for computing ${\bf w}$ , since any vector with $\alpha>0$ will contribute to the derived solution for ${\bf w}$ .

2. Only margin SV's for computing

, since we need an equation, not an inequality, to solve for

after knowing ${\bf w}$ .

Is the 'heuristic' number of parameters (the VC dimension proxy) to be used while reasoning about generalization then the number of margin support vectors << the number of all support vectors?

When we use kernel functions with soft SVMs (problem 2 etc), where there is no explicit w, does the above translate to :
* 1==> Use all support vectors to compute the sigma term in the hypothesis function g()
* 2==> Use only margin support vectors for b (which is also used in g()

I was wondering if this aspect was covered in the lecture or any of the additional material -- I seem to have missed.

yaser · #4 07-29-2013, 01:01 AM

Quote:

Originally Posted by hsolo

Is the 'heuristic' number of parameters (the VC dimension proxy) to be used while reasoning about generalization then the number of margin support vectors << the number of all support vectors?

When we use kernel functions with soft SVMs (problem 2 etc), where there is no explicit w, does the above translate to :
* 1==> Use all support vectors to compute the sigma term in the hypothesis function g()
* 2==> Use only margin support vectors for b (which is also used in g()

I was wondering if this aspect was covered in the lecture or any of the additional material -- I seem to have missed.

The computation of ${\bf w}$ involves all support vectors, margin and otherwise, since it involves all $\alpha$ 's that are bigger than zero. Assuming ${\bf w}$ has been computed, the computation of

, for both hard and soft margins, involves any one support vector (margin support vector in the case of soft margin) since it is based on solving the equation ${\bf w}^{\rm T} {\bf x}_n + b = 1$ for

.

In the case of kernels, the explicit evaluation of ${\bf w}$ followed by taking an inner product with a point ${\bf x}$ is replaced by evaluating the kernel with two arguments; one is a support vector (margin or otherwise) and the other is the point ${\bf x}$ , and repeating that for all support vectors (margin or otherwise).

khohi · #5 03-04-2016, 07:04 AM

thanks

الولادة الطبيعية

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode