Any one of these three can happen:
1) the linear regression weights are optimal
2) the linear regression weights are not optimal and the PLA/Pocket algorithm can improve the weights.
3) the linear regression weights are not optimal and the PLA/Pocket algorithm cannot improve the weights.
In practice, we will not know which case we are in because actually finding the optimal weights is an NPhard combinatorial optimization problem.
However, no matter which case we are in, other than some extra CPU cycles, there is no harm done in running the pocket algorithm on the regression weights to see if they can be improved.
Quote:
Originally Posted by rpistu
Hi Professor, you said that the weight vector w learnted from the Linear Regression could be suboptimal for classification. However, after run the pocket algorithm with 1,000,000 iteration, the w still not change, which means that the w learnt from the Linear Regression is optimal. Is that true? Maybe I made some mistake.
