LFD Book Forum (http://book.caltech.edu/bookforum/index.php)
-   Chapter 4 - Overfitting (http://book.caltech.edu/bookforum/forumdisplay.php?f=111)
-   -   How does regularized logistic regression regularize perceptron model? (http://book.caltech.edu/bookforum/showthread.php?t=4692)

 ntvy95 09-11-2016 01:04 AM

How does regularized logistic regression regularize perceptron model?

In the case I use regularized logistic regression to train perceptron model with threshold , for regularized logistic regression it means that I have had soft-order constraint on , however for perceptron model, as suggested in Exercise 4.6, it looks like soft-order constraint on doesn't regularize perceptron model at all. Meanwhile in practice, I see that regularized logistic regression does regularize perceptron model and Andrew Ng's online course also suggests that regularized logistic regression fights against overfitting on classification problems.

What did I miss here?

Thank you very much in advance.

 htlin 12-13-2017 02:38 PM

Re: How does regularized logistic regression regularize perceptron model?

Quote:
 Originally Posted by ntvy95 (Post 12429) In the case I use regularized logistic regression to train perceptron model with threshold , for regularized logistic regression it means that I have had soft-order constraint on , however for perceptron model, as suggested in Exercise 4.6, it looks like soft-order constraint on doesn't regularize perceptron model at all. Meanwhile in practice, I see that regularized logistic regression does regularize perceptron model and Andrew Ng's online course also suggests that regularized logistic regression fights against overfitting on classification problems. What did I miss here? Thank you very much in advance.
That is because the hard-classification output (sign) is not affected by the magnitude of while the soft output () is . Hope this helps.

 pdsubraa 01-03-2018 05:44 AM

Re: How does regularized logistic regression regularize perceptron model?

Happy New Year 2018! guys

Htlin - you always make my day precious with your solutions. No wonder you're great!

Thanks again!

 htlin 01-04-2018 12:48 AM

Re: How does regularized logistic regression regularize perceptron model?

Quote:
 Originally Posted by pdsubraa (Post 12894) Happy New Year 2018! guys Htlin - you always make my day precious with your solutions. No wonder you're great! Thanks again!
Thanks. Happy new year to everyone.

 All times are GMT -7. The time now is 08:12 AM.