LFD Book Forum  

Go Back   LFD Book Forum > Course Discussions > Online LFD course > The Final

Reply
 
Thread Tools Display Modes
  #1  
Old 09-07-2012, 11:02 AM
samirbajaj samirbajaj is offline
Member
 
Join Date: Jul 2012
Location: Silicon Valley
Posts: 48
Default RBF using a library package?

Hi all,

I have written some code in Octave to simulate the regular RBF model for questions 15 - 19 on the final.

However, mindful of the fact that I don't usually write bug-free code the first time I was wondering if anyone knew of a library/package like libsvm for regular RBF's that I could use to validate the results I'm getting.

Thanks for any pointers.

-Samir

PS - On an unrelated note, these simulations are bringing my 2-year-old Mac down to its knees...vectorized code or not.
Reply With Quote
  #2  
Old 09-09-2012, 03:47 PM
munchkin munchkin is offline
Member
 
Join Date: Jul 2012
Posts: 38
Default Re: RBF using a library package?

Quote:
Originally Posted by samirbajaj View Post

I was wondering if anyone knew of a library/package like libsvm for regular RBF's that I could use to validate the results I'm getting.

PS - On an unrelated note, these simulations are bringing my 2-year-old Mac down to its knees...vectorized code or not.
Yes, there is no question that the amount of number crunching required is enormous -- I had thought I had a fast machine but now I know better.

I've learned a huge amount about constrained optimization and SVM by rolling my own code (I don't learn anything unless I do it hands on) but, truth be told, there aren't any simple test data sets out there that I have found. I just use a simple 2D triangular test pattern ( (-1,-1) (0,1) (1,-1) ) with a constant hypothesis at Y=0 as a baseline confirmation that the kernel classification process is correct.

I had the impression from earlier discussions that SVMLIB has various RBF options. There is a web site featuring a C# port of SVMLIB and the guy at that site had some test files he brought over from the SVMLIB site. There are many domain-specific SVMLIB data files at the SVMLIB site also.
Reply With Quote
  #3  
Old 09-12-2012, 12:27 PM
TonySuarez TonySuarez is offline
Member
 
Join Date: Jul 2012
Location: Lisboa, Portugal
Posts: 35
Default Re: RBF using a library package?

I developed the code for these questions in R. For the SVM with RBF I call svmlib through an interface with R (in concrete, I use the library e1071, strange name...). This lib has options for working with the RBF kernel.

For the "regular" RBF, I implement step-by-step Lloyd´s algorithm, assemble the matrix \Phi (RBF lecture 16, slide 14) and calculate the trained w.

I use the same N=100 points with both algorithms, obviously. And classify with both methods (SVM and "regular").

Regarding time, R it's not too bad. Running a batch of 1000 sets of 100 points takes me perhaps 2 minutes or less (my home desktop has a low end AMD A8 3850 processor, with 8 GB RAM, nothing fancy; it's quad-core but only one core is used by R). I use 200 random points for testing (the same for all the 1000 batches with 100 points... random is random...).

Regarding correctness, I verified the code with some care, (about 100 lines in R, with some amount of vectorization, for the implementation of both SVM and regular RBF) and the results obtained for Ein, Eout, etc... seem reasonable. So, I cross my fingers...

I didn't look for a library implementing "regular" RBF. Perhaps there are some out there... but it seems to me that it is not a very popular approach to classification ...

Last edited by TonySuarez; 09-12-2012 at 03:25 PM. Reason: Remove edges in text.
Reply With Quote
  #4  
Old 09-13-2012, 09:54 AM
Andrs Andrs is offline
Member
 
Join Date: Jul 2012
Posts: 47
Default Re: RBF using a library package?

Python with numpy can handle the creation of the array with distances (between centers/points) as well as the "fi" matrix with exponents (basic RBF) with very few operations (all heavy work done under the hood). Basically almost all the matrix operations without loops
Reply With Quote
  #5  
Old 09-14-2012, 01:57 PM
MLearning MLearning is offline
Senior Member
 
Join Date: Jul 2012
Posts: 56
Default Re: RBF using a library package?

Quote:
Originally Posted by samirbajaj View Post
Hi all,

I have written some code in Octave to simulate the regular RBF model for questions 15 - 19 on the final.

However, mindful of the fact that I don't usually write bug-free code the first time I was wondering if anyone knew of a library/package like libsvm for regular RBF's that I could use to validate the results I'm getting.

Thanks for any pointers.

-Samir

PS - On an unrelated note, these simulations are bringing my 2-year-old Mac down to its knees...vectorized code or not.
@samirbajaj,

I was wondering if you will be able to get your Octave implementation of SVM with RBF working. For some weird reason my Octave implementation is not able to classify a linearly separable data which libsvm is able to do.
Reply With Quote
  #6  
Old 09-14-2012, 04:47 PM
samirbajaj samirbajaj is offline
Member
 
Join Date: Jul 2012
Location: Silicon Valley
Posts: 48
Default Re: RBF using a library package?

Quote:
Originally Posted by MLearning View Post
@samirbajaj,

I was wondering if you will be able to get your Octave implementation of SVM with RBF working. For some weird reason my Octave implementation is not able to classify a linearly separable data which libsvm is able to do.

For SVM with RBF, I used libsvm.

-Samir
Reply With Quote
  #7  
Old 06-09-2013, 07:52 AM
skwong skwong is offline
Member
 
Join Date: Apr 2013
Location: Hong Kong
Posts: 13
Default Re: RBF using a library package?

Quote:
Originally Posted by samirbajaj View Post
Hi all,

I was wondering if anyone knew of a library/package like libsvm for regular RBF's that I could use to validate the results I'm getting.

Thanks for any pointers.

-Samir

PS - On an unrelated note, these simulations are bringing my 2-year-old Mac down to its knees...vectorized code or not.
Seems there is http://octave.sourceforge.net/communications/index.html
but I don't know how to use it.
Reply With Quote
  #8  
Old 06-09-2013, 01:36 PM
bargava bargava is offline
Junior Member
 
Join Date: Apr 2013
Posts: 8
Default Re: RBF using a library package?

Some part of the implementation can leverage existing functions in R

kmeans function in R implements Lloyd's algorithm for RBF. It also has an option for specifying the initial starting points (or, we could let it choose randomly).

It returns the centers. We still need to find the weights and do the classification.
Reply With Quote
  #9  
Old 06-09-2013, 07:40 PM
skwong skwong is offline
Member
 
Join Date: Apr 2013
Location: Hong Kong
Posts: 13
Default Re: RBF using a library package?

Quote:
Originally Posted by skwong View Post
Seems there is http://octave.sourceforge.net/communications/index.html
but I don't know how to use it.
I think that is not related to the lloyd algorithm we are talking, please ignore the post. Sorry about that.

BTW, I think implementing the Lloyd algorithm is not that difficult to debug.
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -7. The time now is 07:20 PM.


Powered by vBulletin® Version 3.8.3
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, and participants in the Learning From Data MOOC by Yaser S. Abu-Mostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.