Target function formulation in Q8

[ripande]

I guess this is pretty basic, but I am unclear on this, so would like some help. As per requiremets of question 8

f(x) = 1 for Y=+1
f(x) = 0 for Y = -1

Is my understanding correct ?

if yes, f(x) should be the step function of y*x. I am unable to formulate this in a form such that it is differentiable to use SGD. Can someone help me understand how to do it?

Thanks

[ripande]

Or do we just use formula to minimize mean square error

sum ( sq(y1 - wx1) + sq(y2 - wx2) ...) ?

[yaser]

Originally Posted by ripande

I guess this is pretty basic, but I am unclear on this, so would like some help. As per requiremets of question 8

f(x) = 1 for Y=+1
f(x) = 0 for Y = -1

Is my understanding correct ?

if yes, f(x) should be the step function of y*x. I am unable to formulate this in a form such that it is differentiable to use SGD. Can someone help me understand how to do it?

The target function in this case is indeed "hard threshold," but the hypotheses are "soft threshold" (they only approximate the target) and these are the ones that have the required smoothness properties for SGD. The goal of miniizing the cross entropy error measure is to get a hypothesis that approximately replicates , based on the noisy examples.

[ripande]

Understood. Thanks Prof Yaser. So in this case we can use the theta(s) as the sigmoid function as explained in the lecture, correct?

[yaser]

Originally Posted by ripande

Understood. Thanks Prof Yaser. So in this case we can use the theta(s) as the sigmoid function as explained in the lecture, correct?

Correct.