LFD Book Forum  

Go Back   LFD Book Forum > Course Discussions > Online LFD course > Homework 1

Reply
 
Thread Tools Display Modes
  #1  
Old 04-07-2013, 08:48 PM
Michael Reach Michael Reach is offline
Senior Member
 
Join Date: Apr 2013
Location: Baltimore, Maryland, USA
Posts: 71
Default PLA computing speed

I'm a little concerned by how long my computations are taking (in R) on my home computer: ten minutes or more for the 100 points 1000 times. Is that normal, or is this just the tip of the iceberg, and the later assignments will take years?
Reply With Quote
  #2  
Old 04-07-2013, 09:11 PM
yaser's Avatar
yaser yaser is offline
Caltech
 
Join Date: Aug 2009
Location: Pasadena, California, USA
Posts: 1,477
Default Re: PLA computing speed

Quote:
Originally Posted by Michael Reach View Post
is this just the tip of the iceberg, and the later assignments will take years?
No danger of that!
__________________
Where everyone thinks alike, no one thinks very much
Reply With Quote
  #3  
Old 04-08-2013, 05:15 PM
Elroch Elroch is offline
Invited Guest
 
Join Date: Mar 2013
Posts: 143
Default Re: PLA computing speed

Quote:
Originally Posted by Michael Reach View Post
I'm a little concerned by how long my computations are taking (in R) on my home computer: ten minutes or more for the 100 points 1000 times. Is that normal, or is this just the tip of the iceberg, and the later assignments will take years?
If it's any consolation, I'm using R as well, and I believe my program was slower than yours!
Reply With Quote
  #4  
Old 04-08-2013, 05:27 PM
IsidroHidalgo IsidroHidalgo is offline
Member
 
Join Date: Apr 2013
Location: Toledo (Spain)
Posts: 28
Default Re: PLA computing speed

My 1000 iterations of N=10 points have taked 45 min... R running in a Notebook
I'm choosing the first missclassified point. I wonder if taking the closest to f(x) in each iteration can improve the algorithm speed
Reply With Quote
  #5  
Old 04-08-2013, 08:29 PM
kafar kafar is offline
Junior Member
 
Join Date: Apr 2013
Posts: 6
Default Re: PLA computing speed

you probably want to switch to c++. it took just a couple minutes to run 10k N=100 runs. and i didn't really optimize the code.
Reply With Quote
  #6  
Old 04-08-2013, 11:17 PM
pyguy pyguy is offline
Junior Member
 
Join Date: Apr 2013
Posts: 5
Default Re: PLA computing speed

I've been using Python and NumPy. My solution for N=100 with 1000 iterations was taking 4 minutes previously, and changing to PyPy and NumPyPy took it down to just 5 seconds. NumPyPy support isn't 100% yet, but it's getting there and the performance benefits of PyPy can be very nice without having to change your code.
Reply With Quote
  #7  
Old 04-09-2013, 07:53 AM
jlaurentum jlaurentum is offline
Member
 
Join Date: Apr 2013
Location: Venezuela
Posts: 41
Default Re: PLA computing speed

Hello.

I used R too. I did 500 iterations for N=100 and it took forever. I used a while loop for the main part of the PLA. R is very inefficient with while loops and I don't see an easy way to vectorize the code. I also have the same question as Isidro - can the misclassification point chosen at each iteration affect the speed? Is there a better choice than just taking the first misclassified point you bump into?

Oh and btw, you guys did vectorize the classification for the N points with a good old sapply didnt you?
Reply With Quote
  #8  
Old 04-09-2013, 08:06 AM
IsidroHidalgo IsidroHidalgo is offline
Member
 
Join Date: Apr 2013
Location: Toledo (Spain)
Posts: 28
Default Re: PLA computing speed

I reach the code vectorization and it's great: 2 min for 1000 iterations with N=10 and 3 min for another 1000 with N=100.
jlaurentum: use apply to vectorization of sign: sign(apply(s.points, 1, "%*%", omega))
Reply With Quote
  #9  
Old 04-09-2013, 08:14 AM
jlaurentum jlaurentum is offline
Member
 
Join Date: Apr 2013
Location: Venezuela
Posts: 41
Default Re: PLA computing speed

Isidro:

I used sapply on the sign function with an inner product argument. However, my code is very slow because of this (in pseudo code)

while there are misclassified points:
update the weight vector according to the correct output for that misclassified point
predict all the outputs according to the new weight vector (in just one instruction with
the sapply)
end while


My problem with speed is due to the while. Or maybe taking the first misclassified point is not the best thing to do...
Reply With Quote
  #10  
Old 04-09-2013, 08:21 AM
Elroch Elroch is offline
Invited Guest
 
Join Date: Mar 2013
Posts: 143
Default Re: PLA computing speed

Quote:
Originally Posted by jlaurentum View Post
Hello.

I used R too. I did 500 iterations for N=100 and it took forever. I used a while loop for the main part of the PLA. R is very inefficient with while loops and I don't see an easy way to vectorize the code. I also have the same question as Isidro - can the misclassification point chosen at each iteration affect the speed? Is there a better choice than just taking the first misclassified point you bump into?

Oh and btw, you guys did vectorize the classification for the N points with a good old sapply didnt you?
Good point about sapply!

In answer to your question, I have observed empirically that the way you choose the misclassified point has quite a large effect on the convergence. Beyond the scope of the actual problem, we could attempt to improve on randomisation by using the information in the value of w.x when sign(w.x) is wrong. But should we prefer large modulus of w.x or small?
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -7. The time now is 04:28 AM.


Powered by vBulletin® Version 3.8.3
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, and participants in the Learning From Data MOOC by Yaser S. Abu-Mostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.