View Single Post
  #6  
Old 10-12-2012, 02:16 PM
magdon's Avatar
magdon magdon is offline
RPI
 
Join Date: Aug 2009
Location: Troy, NY, USA.
Posts: 595
Default Re: Classifying Handwritten Digits: 1 vs. 5

Any one of these three can happen:

1) the linear regression weights are optimal
2) the linear regression weights are not optimal and the PLA/Pocket algorithm can improve the weights.
3) the linear regression weights are not optimal and the PLA/Pocket algorithm cannot improve the weights.

In practice, we will not know which case we are in because actually finding the optimal weights is an NP-hard combinatorial optimization problem.

However, no matter which case we are in, other than some extra CPU cycles, there is no harm done in running the pocket algorithm on the regression weights to see if they can be improved.


Quote:
Originally Posted by rpistu View Post
Hi Professor, you said that the weight vector w learnted from the Linear Regression could be suboptimal for classification. However, after run the pocket algorithm with 1,000,000 iteration, the w still not change, which means that the w learnt from the Linear Regression is optimal. Is that true? Maybe I made some mistake.
__________________
Have faith in probability
Reply With Quote