LFD Book Forum

LFD Book Forum (http://book.caltech.edu/bookforum/index.php)
-   The Final (http://book.caltech.edu/bookforum/forumdisplay.php?f=138)
-   -   RBF using a library package? (http://book.caltech.edu/bookforum/showthread.php?t=1437)

samirbajaj 09-07-2012 11:02 AM

RBF using a library package?
 
Hi all,

I have written some code in Octave to simulate the regular RBF model for questions 15 - 19 on the final.

However, mindful of the fact that I don't usually write bug-free code the first time :) I was wondering if anyone knew of a library/package like libsvm for regular RBF's that I could use to validate the results I'm getting.

Thanks for any pointers.

-Samir

PS - On an unrelated note, these simulations are bringing my 2-year-old Mac down to its knees...vectorized code or not.

munchkin 09-09-2012 03:47 PM

Re: RBF using a library package?
 
Quote:

Originally Posted by samirbajaj (Post 5015)

I was wondering if anyone knew of a library/package like libsvm for regular RBF's that I could use to validate the results I'm getting.

PS - On an unrelated note, these simulations are bringing my 2-year-old Mac down to its knees...vectorized code or not.

Yes, there is no question that the amount of number crunching required is enormous -- I had thought I had a fast machine but now I know better.

I've learned a huge amount about constrained optimization and SVM by rolling my own code (I don't learn anything unless I do it hands on) but, truth be told, there aren't any simple test data sets out there that I have found. I just use a simple 2D triangular test pattern ( (-1,-1) (0,1) (1,-1) ) with a constant hypothesis at Y=0 as a baseline confirmation that the kernel classification process is correct.

I had the impression from earlier discussions that SVMLIB has various RBF options. There is a web site featuring a C# port of SVMLIB and the guy at that site had some test files he brought over from the SVMLIB site. There are many domain-specific SVMLIB data files at the SVMLIB site also.

TonySuarez 09-12-2012 12:27 PM

Re: RBF using a library package?
 
I developed the code for these questions in R. For the SVM with RBF I call svmlib through an interface with R (in concrete, I use the library e1071, strange name...). This lib has options for working with the RBF kernel.

For the "regular" RBF, I implement step-by-step Lloyd´s algorithm, assemble the matrix \Phi (RBF lecture 16, slide 14) and calculate the trained w.

I use the same N=100 points with both algorithms, obviously. And classify with both methods (SVM and "regular").

Regarding time, R it's not too bad. Running a batch of 1000 sets of 100 points takes me perhaps 2 minutes or less (my home desktop has a low end AMD A8 3850 processor, with 8 GB RAM, nothing fancy; it's quad-core but only one core is used by R). I use 200 random points for testing (the same for all the 1000 batches with 100 points... random is random...).

Regarding correctness, I verified the code with some care, (about 100 lines in R, with some amount of vectorization, for the implementation of both SVM and regular RBF) and the results obtained for Ein, Eout, etc... seem reasonable. So, I cross my fingers...

I didn't look for a library implementing "regular" RBF. Perhaps there are some out there... but it seems to me that it is not a very popular approach to classification :) ...

Andrs 09-13-2012 09:54 AM

Re: RBF using a library package?
 
Python with numpy can handle the creation of the array with distances (between centers/points) as well as the "fi" matrix with exponents (basic RBF) with very few operations (all heavy work done under the hood). Basically almost all the matrix operations without loops

MLearning 09-14-2012 01:57 PM

Re: RBF using a library package?
 
Quote:

Originally Posted by samirbajaj (Post 5015)
Hi all,

I have written some code in Octave to simulate the regular RBF model for questions 15 - 19 on the final.

However, mindful of the fact that I don't usually write bug-free code the first time :) I was wondering if anyone knew of a library/package like libsvm for regular RBF's that I could use to validate the results I'm getting.

Thanks for any pointers.

-Samir

PS - On an unrelated note, these simulations are bringing my 2-year-old Mac down to its knees...vectorized code or not.

@samirbajaj,

I was wondering if you will be able to get your Octave implementation of SVM with RBF working. For some weird reason my Octave implementation is not able to classify a linearly separable data which libsvm is able to do.

samirbajaj 09-14-2012 04:47 PM

Re: RBF using a library package?
 
Quote:

Originally Posted by MLearning (Post 5284)
@samirbajaj,

I was wondering if you will be able to get your Octave implementation of SVM with RBF working. For some weird reason my Octave implementation is not able to classify a linearly separable data which libsvm is able to do.


For SVM with RBF, I used libsvm.

-Samir

skwong 06-09-2013 07:52 AM

Re: RBF using a library package?
 
Quote:

Originally Posted by samirbajaj (Post 5015)
Hi all,

I was wondering if anyone knew of a library/package like libsvm for regular RBF's that I could use to validate the results I'm getting.

Thanks for any pointers.

-Samir

PS - On an unrelated note, these simulations are bringing my 2-year-old Mac down to its knees...vectorized code or not.

Seems there is http://octave.sourceforge.net/communications/index.html
but I don't know how to use it.

bargava 06-09-2013 01:36 PM

Re: RBF using a library package?
 
Some part of the implementation can leverage existing functions in R

kmeans function in R implements Lloyd's algorithm for RBF. It also has an option for specifying the initial starting points (or, we could let it choose randomly).

It returns the centers. We still need to find the weights and do the classification.

skwong 06-09-2013 07:40 PM

Re: RBF using a library package?
 
Quote:

Originally Posted by skwong (Post 11047)
Seems there is http://octave.sourceforge.net/communications/index.html
but I don't know how to use it.

I think that is not related to the lloyd algorithm we are talking, please ignore the post. Sorry about that.

BTW, I think implementing the Lloyd algorithm is not that difficult to debug.


All times are GMT -7. The time now is 01:43 AM.

Powered by vBulletin® Version 3.8.3
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, and participants in the Learning From Data MOOC by Yaser S. Abu-Mostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.