Thread: Problem 3.5
View Single Post
Old 07-30-2014, 08:39 AM
magdon's Avatar
magdon magdon is offline
Join Date: Aug 2009
Location: Troy, NY, USA.
Posts: 597
Default Re: Problem 3.5

There are 3 cases:

y_n\textbf{w}^T\textbf{x}_n>1 in which case E(\mathbf{x}_n)=0, which is differentiable.

y_n\textbf{w}^T\textbf{x}_n<1 in which case E(\mathbf{x}_n)=1-y_n\textbf{w}^T\textbf{x}_n, which is differentiable.

y_n\textbf{w}^T\textbf{x}_n=1 in which case E(\mathbf{x}_n)=0, but is not differentiable because the derivative is 0 or not depending on whether y_n\textbf{w}^T\textbf{x}_n approaches 1 from above or below.

Originally Posted by BojanVujatovic View Post
I have questions about this problem. What I get after analyzing E_n(\textbf{w}) is the following:

E_n(\textbf{w}) = 
0, &\text{sign}(\textbf{w}^T\textbf{x}_n)=y_n \Leftrightarrow  y_n\textbf{w}^T\textbf{x}_n \geq 0  \\ 
|y_n - \textbf{w}^T\textbf{x}_n|=1 - y_n\textbf{w}^T\textbf{x}_n, & \text{sign}(\textbf{w}^T\textbf{x}_n) \neq y_n \Leftrightarrow  y_n\textbf{w}^T\textbf{x}_n < 0 

The last line follows because:
1\cdot|y_n - \textbf{w}^T\textbf{x}_n| = |y_n||y_n-\textbf{w}^T\textbf{x}_n|
= |1 - y_n\textbf{w}^T\textbf{x}_n|=1 - y_n\textbf{w}^T\textbf{x}_n (since y_n\textbf{w}^T\textbf{x}_n < 0)

From this, I cannot see how E_n(\textbf{w}) is not continuous at \textbf{w}^T\textbf{x}_n = y_n. Instead I find that it is not continuous nor differentiable for \textbf{w}'s such that \textbf{w}^T\textbf{x}_n = 0.

Any help would be greatly appreciated.
Have faith in probability
Reply With Quote