LFD Book Forum - View Single Post - How does regularized logistic regression regularize perceptron model?

htlin · #2 12-13-2017, 01:38 PM

Quote:

Originally Posted by ntvy95

In the case I use regularized logistic regression to train perceptron model with threshold $\frac{1}{2}$ , for regularized logistic regression it means that I have had soft-order constraint on

, however for perceptron model, as suggested in Exercise 4.6, it looks like soft-order constraint on

doesn't regularize perceptron model at all. Meanwhile in practice, I see that regularized logistic regression does regularize perceptron model and Andrew Ng's online course also suggests that regularized logistic regression fights against overfitting on classification problems.

What did I miss here?

Thank you very much in advance.

That is because the hard-classification output (sign) is not affected by the magnitude of $\mathbf{w}$ while the soft output ( $\theta$ ) is . Hope this helps.