LFD Book Forum  

Go Back   LFD Book Forum > General > General Discussion of Machine Learning

Reply
 
Thread Tools Display Modes
  #1  
Old 07-15-2012, 03:58 AM
lorddoskias lorddoskias is offline
Junior Member
 
Join Date: Jul 2012
Posts: 9
Unhappy Problem with simple perceptron implmenetation

So I wanted to do a little bit of experimenting with the perceptron algorithm to understand it better and I came up with a simple match making scenario. Essentially i have a 100x2 training data matrix with the first feature being height between 165 and 185 and the second feature is weight (as in physical weight of a person ) between 60 and 80. My target function is a very simple one :

Code:
function [res] = target(X)

m = size(X, 1);
res = zeros(100, 1);

for j = 1:length(X)
 if(X(j,1) > 170 && X(j,2) > 65)
    res(j) = 1;
  else
    res(j) =  -1;
  end
end

end
It returns 1 if a person is taller than 170cm and heavier than 65 kilos and -1 otherwise. I use this function to generate the result of a X matrix which consists of random values withing the aforementioned boundaries.

The next logical step is to learn a hypothesis function g(X) which would act much in the same way as the target function - which I assume is unknown. So here is my implementation of the perceptron:

Code:
function [w] = perceptron(X, y)

X = [ones(size(X, 1), 1) X]; % add the bias term
w = zeros(size(X, 2), 1); %init weights to zero
m = length(y);
iterations = 0;
wrong = 0;

while(true)
iterations = iterations + 1;
wrong = 0;
for j = 1:m
   if (sign(X(j, :) * w) != y(j)) 
		w = w + (y(j)*X(j, :))';
		wrong = 1;
		break;
   endif
end %inner for 

if(wrong == 0) 
	break;
	endif

end  %outer loop 


end %function
Unfortunately it doesn't converge? Are my assumptions towards the problem wrong or do I have a fault in my implementation?
Reply With Quote
  #2  
Old 07-15-2012, 01:29 PM
JohnH JohnH is offline
Member
 
Join Date: Jul 2012
Posts: 43
Default Re: Problem with simple perceptron implmenetation

I haven't looked at your implementation of the perceptron; however, I think there will be a problem with convergence regardless of the implementation since the target function is not linearly separable.
Reply With Quote
  #3  
Old 07-16-2012, 07:15 AM
lorddoskias lorddoskias is offline
Junior Member
 
Join Date: Jul 2012
Posts: 9
Default Re: Problem with simple perceptron implmenetation

I think you are mistaken. It is linearly separable since the 170/65 act as the threshold i.e. that's where the linear separation occurs.
Reply With Quote
  #4  
Old 07-16-2012, 10:40 AM
JohnH JohnH is offline
Member
 
Join Date: Jul 2012
Posts: 43
Default Re: Problem with simple perceptron implmenetation

The function, height > 170 and weight > 65, is not linearly separable because it does not bisect the plane but instead defines an infinite rectangular region on the plane. There is no single line that separates this region from the rest of the plane.
Reply With Quote
  #5  
Old 07-16-2012, 10:01 PM
Ubermensch Ubermensch is offline
Junior Member
 
Join Date: Jun 2012
Posts: 4
Default Re: Problem with simple perceptron implmenetation

John is right. It can't be fit with a linear line. This is what I get with your data (the points are randomly chosen and sorry for not including the axis, legends and labels)



So it isn't a straight line. So it can't converge with a simple 1D perceptron
Reply With Quote
  #6  
Old 07-17-2012, 12:17 PM
lorddoskias lorddoskias is offline
Junior Member
 
Join Date: Jul 2012
Posts: 9
Default Re: Problem with simple perceptron implmenetation

Thanks for the answers. So if I have understood you correctly in this case I will need to train 2 perceptrons - one for each feature and then when a new sample is given run each of the perceptrons for height/weight and my final hypothesis would return 1 only if the 2 perceptrons return 1 on each respective feature? Is this correct?
Reply With Quote
  #7  
Old 07-17-2012, 01:53 PM
JohnH JohnH is offline
Member
 
Join Date: Jul 2012
Posts: 43
Default Re: Problem with simple perceptron implmenetation

At this point the course hasn't provided us with sufficient tools to address the problem you pose, but I think you are on the right track for a method of resolving the categories. Two perceptrons, one for height and one for weight, could feed a third perceptron that would perform the logical operation (AND is a linearly separable logical operation). This approach probably requires back-propogation in order to train the network.
Reply With Quote
Reply

Tags
perceptron, pla

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -7. The time now is 04:22 PM.


Powered by vBulletin® Version 3.8.3
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, and participants in the Learning From Data MOOC by Yaser S. Abu-Mostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.