LFD Book Forum

LFD Book Forum (http://book.caltech.edu/bookforum/index.php)
-   Chapter 4 - Overfitting (http://book.caltech.edu/bookforum/forumdisplay.php?f=111)
-   -   How does regularized logistic regression regularize perceptron model? (http://book.caltech.edu/bookforum/showthread.php?t=4692)

ntvy95 09-11-2016 12:04 AM

How does regularized logistic regression regularize perceptron model?
 
In the case I use regularized logistic regression to train perceptron model with threshold \frac{1}{2}, for regularized logistic regression it means that I have had soft-order constraint on w, however for perceptron model, as suggested in Exercise 4.6, it looks like soft-order constraint on w doesn't regularize perceptron model at all. Meanwhile in practice, I see that regularized logistic regression does regularize perceptron model and Andrew Ng's online course also suggests that regularized logistic regression fights against overfitting on classification problems.

What did I miss here?

Thank you very much in advance.

htlin 12-13-2017 01:38 PM

Re: How does regularized logistic regression regularize perceptron model?
 
Quote:

Originally Posted by ntvy95 (Post 12429)
In the case I use regularized logistic regression to train perceptron model with threshold \frac{1}{2}, for regularized logistic regression it means that I have had soft-order constraint on w, however for perceptron model, as suggested in Exercise 4.6, it looks like soft-order constraint on w doesn't regularize perceptron model at all. Meanwhile in practice, I see that regularized logistic regression does regularize perceptron model and Andrew Ng's online course also suggests that regularized logistic regression fights against overfitting on classification problems.

What did I miss here?

Thank you very much in advance.

That is because the hard-classification output (sign) is not affected by the magnitude of \mathbf{w} while the soft output (\theta) is . Hope this helps.

pdsubraa 01-03-2018 04:44 AM

Re: How does regularized logistic regression regularize perceptron model?
 
Happy New Year 2018! guys

Htlin - you always make my day precious with your solutions. No wonder you're great!

Thanks again!

htlin 01-03-2018 11:48 PM

Re: How does regularized logistic regression regularize perceptron model?
 
Quote:

Originally Posted by pdsubraa (Post 12894)
Happy New Year 2018! guys

Htlin - you always make my day precious with your solutions. No wonder you're great!

Thanks again!

Thanks. Happy new year to everyone.


All times are GMT -7. The time now is 12:04 PM.

Powered by vBulletin® Version 3.8.3
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, and participants in the Learning From Data MOOC by Yaser S. Abu-Mostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.