LFD Book Forum (http://book.caltech.edu/bookforum/index.php)
-   Chapter 4 - Overfitting (http://book.caltech.edu/bookforum/forumdisplay.php?f=111)
-   -   How does regularized logistic regression regularize perceptron model? (http://book.caltech.edu/bookforum/showthread.php?t=4692)

 ntvy95 09-11-2016 01:04 AM

How does regularized logistic regression regularize perceptron model?

In the case I use regularized logistic regression to train perceptron model with threshold , for regularized logistic regression it means that I have had soft-order constraint on , however for perceptron model, as suggested in Exercise 4.6, it looks like soft-order constraint on doesn't regularize perceptron model at all. Meanwhile in practice, I see that regularized logistic regression does regularize perceptron model and Andrew Ng's online course also suggests that regularized logistic regression fights against overfitting on classification problems.

What did I miss here?

Thank you very much in advance.

 htlin 12-13-2017 02:38 PM

Re: How does regularized logistic regression regularize perceptron model?

Quote:
 Originally Posted by ntvy95 (Post 12429) In the case I use regularized logistic regression to train perceptron model with threshold , for regularized logistic regression it means that I have had soft-order constraint on , however for perceptron model, as suggested in Exercise 4.6, it looks like soft-order constraint on doesn't regularize perceptron model at all. Meanwhile in practice, I see that regularized logistic regression does regularize perceptron model and Andrew Ng's online course also suggests that regularized logistic regression fights against overfitting on classification problems. What did I miss here? Thank you very much in advance.
That is because the hard-classification output (sign) is not affected by the magnitude of while the soft output () is . Hope this helps.

 pdsubraa 01-03-2018 05:44 AM

Re: How does regularized logistic regression regularize perceptron model?

Happy New Year 2018! guys

Htlin - you always make my day precious with your solutions. No wonder you're great!

Thanks again!

 htlin 01-04-2018 12:48 AM

Re: How does regularized logistic regression regularize perceptron model?

Quote:
 Originally Posted by pdsubraa (Post 12894) Happy New Year 2018! guys Htlin - you always make my day precious with your solutions. No wonder you're great! Thanks again!
Thanks. Happy new year to everyone.

 All times are GMT -7. The time now is 05:14 AM.

Powered by vBulletin® Version 3.8.3
Copyright ©2000 - 2022, Jelsoft Enterprises Ltd.
The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, and participants in the Learning From Data MOOC by Yaser S. Abu-Mostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.