RBF using a library package?
 

Hi all,

I have written some code in Octave to simulate the regular RBF model for questions 15 - 19 on the final.

However, mindful of the fact that I don't usually write bug-free code the first time :) I was wondering if anyone knew of a library/package like libsvm for regular RBF's that I could use to validate the results I'm getting.

Thanks for any pointers.

-Samir

PS - On an unrelated note, these simulations are bringing my 2-year-old Mac down to its knees...vectorized code or not.

Re: RBF using a library package?
 

Yes, there is no question that the amount of number crunching required is enormous -- I had thought I had a fast machine but now I know better.

I've learned a huge amount about constrained optimization and SVM by rolling my own code (I don't learn anything unless I do it hands on) but, truth be told, there aren't any simple test data sets out there that I have found. I just use a simple 2D triangular test pattern ( (-1,-1) (0,1) (1,-1) ) with a constant hypothesis at Y=0 as a baseline confirmation that the kernel classification process is correct.

I had the impression from earlier discussions that SVMLIB has various RBF options. There is a web site featuring a C# port of SVMLIB and the guy at that site had some test files he brought over from the SVMLIB site. There are many domain-specific SVMLIB data files at the SVMLIB site also.

Re: RBF using a library package?
 

I developed the code for these questions in R. For the SVM with RBF I call svmlib through an interface with R (in concrete, I use the library e1071, strange name...). This lib has options for working with the RBF kernel.

For the "regular" RBF, I implement step-by-step Lloyd´s algorithm, assemble the matrix (RBF lecture 16, slide 14) and calculate the trained w.

I use the same N=100 points with both algorithms, obviously. And classify with both methods (SVM and "regular").

Regarding time, R it's not too bad. Running a batch of 1000 sets of 100 points takes me perhaps 2 minutes or less (my home desktop has a low end AMD A8 3850 processor, with 8 GB RAM, nothing fancy; it's quad-core but only one core is used by R). I use 200 random points for testing (the same for all the 1000 batches with 100 points... random is random...).

Regarding correctness, I verified the code with some care, (about 100 lines in R, with some amount of vectorization, for the implementation of both SVM and regular RBF) and the results obtained for Ein, Eout, etc... seem reasonable. So, I cross my fingers...

I didn't look for a library implementing "regular" RBF. Perhaps there are some out there... but it seems to me that it is not a very popular approach to classification :) ...

Re: RBF using a library package?
 

Python with numpy can handle the creation of the array with distances (between centers/points) as well as the "fi" matrix with exponents (basic RBF) with very few operations (all heavy work done under the hood). Basically almost all the matrix operations without loops

Re: RBF using a library package?
 

@samirbajaj,

I was wondering if you will be able to get your Octave implementation of SVM with RBF working. For some weird reason my Octave implementation is not able to classify a linearly separable data which libsvm is able to do.

Re: RBF using a library package?
 

For SVM with RBF, I used libsvm.

-Samir

Re: RBF using a library package?
 

Seems there is http://octave.sourceforge.net/communications/index.html
but I don't know how to use it.

Re: RBF using a library package?
 

Some part of the implementation can leverage existing functions in R

kmeans function in R implements Lloyd's algorithm for RBF. It also has an option for specifying the initial starting points (or, we could let it choose randomly).

It returns the centers. We still need to find the weights and do the classification.

Re: RBF using a library package?
 

I think that is not related to the lloyd algorithm we are talking, please ignore the post. Sorry about that.

BTW, I think implementing the Lloyd algorithm is not that difficult to debug.