Q10 Clarification required

[ripande]

I am not exactly sure of the intent of the question. So would like some clarification.

How do you define perceptron learning algorithm? Is it an algorithm that recalculates weights to classify training example one point at a time ?

I am asking this because to me perceptron learning algothm has two properties:

1. Recalculate weights to classify training example one point
2. adjust weights if a point is misclassified by adding w*x to the weight

Now if I change the mechanism of weight adjustment to SGD, does the algorithm still remain perceptron ?

In short, what is a formal definition of perceptron algorithm ?

[butterscotch]

2. adjust weights if a point is misclassified by adding w*x to the weight.

-> i think you might have typo-ed, w(t+1) = w(t) + y(t)*x(t) for perceptron.

1. Recalculate weights to classify training example one point

With the updated weights, recalculate y_n for each x_n in the training set, and determine which ones are misclassified.

You are looking for an error function that will essentially make the SGD behave like a perceptron in updating the weights.

[ripande]

Perceptron algorithm updates the weight only if the point is misclassified. The same should be true using SGD, correct ?

[colinpriest]

I think that the question wants us to choose which error measure, when implemented using the SGD method, would produce exactly the same change to weights as the perceptron learning algorithm taught back at the start of the course.
So if there is no error for the selected data point, then there is no change to the weights. If there is an error for the selected data point, then the weights are increased by yn * xn

Is that the understanding of others?

[yaser]

Originally Posted by colinpriest

I think that the question wants us to choose which error measure, when implemented using the SGD method, would produce exactly the same change to weights as the perceptron learning algorithm taught back at the start of the course.
So if there is no error for the selected data point, then there is no change to the weights. If there is an error for the selected data point, then the weights are increased by yn * xn

Is that the understanding of others?

This is the correct understanding.

[Michael Reach]

Ah - that makes sense. I had just looked for any old error function that will solve the Perceptron classification problem (one of them seemed like it would obviously work), and got the wrong answer.

[jlaurentum]

Hello all:

I answered this question incorrectly. I think my confusion arises from considering that the error function must be differentiable, because we are after all taking the gradient vector. In reading the following:

So if there is no error for the selected data point, then there is no change to the weights. If there is an error for the selected data point, then the weights are increased by yn * xn

I realize that it contains the answer if you read between the lines, but what if the implied answer is not differentiable?

[catherine]

I had the same doubt though i still chose e as it was the only option resulting in the expected behaviour, ie do not update the weights in case of no error.