Problem 3.5

BojanVujatovic · #1 07-26-2014, 04:07 PM

I have questions about this problem. What I get after analyzing $E_n(\textbf{w})$ is the following:

$E_n(\textbf{w}) = \begin{cases} 0, &\text{sign}(\textbf{w}^T\textbf{x}_n)=y_n \Leftrightarrow y_n\textbf{w}^T\textbf{x}_n \geq 0 \\ |y_n - \textbf{w}^T\textbf{x}_n|=1 - y_n\textbf{w}^T\textbf{x}_n, & \text{sign}(\textbf{w}^T\textbf{x}_n) \neq y_n \Leftrightarrow y_n\textbf{w}^T\textbf{x}_n < 0 \end{cases}$

The last line follows because:
$1\cdot|y_n - \textbf{w}^T\textbf{x}_n| = |y_n||y_n-\textbf{w}^T\textbf{x}_n| = |1 - y_n\textbf{w}^T\textbf{x}_n|=1 - y_n\textbf{w}^T\textbf{x}_n$ (since $y_n\textbf{w}^T\textbf{x}_n < 0$ )

From this, I cannot see how $E_n(\textbf{w})$ is not continuous at $\textbf{w}^T\textbf{x}_n = y_n$ . Instead I find that it is not continuous nor differentiable for $\textbf{w}$ 's such that $\textbf{w}^T\textbf{x}_n = 0$ .

Any help would be greatly appreciated.

magdon · #2 07-30-2014, 08:39 AM

There are 3 cases:

$y_n\textbf{w}^T\textbf{x}_n>1$ in which case $E(\mathbf{x}_n)=0$ , which is differentiable.

$y_n\textbf{w}^T\textbf{x}_n<1$ in which case $E(\mathbf{x}_n)=1-y_n\textbf{w}^T\textbf{x}_n$ , which is differentiable.

$y_n\textbf{w}^T\textbf{x}_n=1$ in which case $E(\mathbf{x}_n)=0$ , but is not differentiable because the derivative is 0 or not depending on whether $y_n\textbf{w}^T\textbf{x}_n$ approaches 1 from above or below.

Quote:

Originally Posted by BojanVujatovic

I have questions about this problem. What I get after analyzing $E_n(\textbf{w})$ is the following:

$E_n(\textbf{w}) = \begin{cases} 0, &\text{sign}(\textbf{w}^T\textbf{x}_n)=y_n \Leftrightarrow y_n\textbf{w}^T\textbf{x}_n \geq 0 \\ |y_n - \textbf{w}^T\textbf{x}_n|=1 - y_n\textbf{w}^T\textbf{x}_n, & \text{sign}(\textbf{w}^T\textbf{x}_n) \neq y_n \Leftrightarrow y_n\textbf{w}^T\textbf{x}_n < 0 \end{cases}$

The last line follows because:
$1\cdot|y_n - \textbf{w}^T\textbf{x}_n| = |y_n||y_n-\textbf{w}^T\textbf{x}_n| = |1 - y_n\textbf{w}^T\textbf{x}_n|=1 - y_n\textbf{w}^T\textbf{x}_n$ (since $y_n\textbf{w}^T\textbf{x}_n < 0$ )

From this, I cannot see how $E_n(\textbf{w})$ is not continuous at $\textbf{w}^T\textbf{x}_n = y_n$ . Instead I find that it is not continuous nor differentiable for $\textbf{w}$ 's such that $\textbf{w}^T\textbf{x}_n = 0$ .

Any help would be greatly appreciated.

BojanVujatovic · #3 08-03-2014, 09:23 PM

Thank you for your reply.

The only part I don't quite follow is:

Quote:

Originally Posted by magdon

$y_n\textbf{w}^T\textbf{x}_n<1$ in which case $E(\mathbf{x}_n)=1-y_n\textbf{w}^T\textbf{x}_n$ , which is differentiable.

As I see, and please correct me if I'm wrong, for $0 \leq y_n\textbf{w}^T\textbf{x}_n<1$ , the signal $\textbf{w}^T\textbf{x}_n$ and the output

agree.
(e.g. $\textbf{w}^T\textbf{x}_n = -0.5$ and $y_n = -1 \implies y_n\textbf{w}^T\textbf{x}_n = 0.5$ ).

Therefore, $\text{sign}(\textbf{w}^T\textbf{x}_n)=y_n$ and $[\!\![\text{sign}(\textbf{w}^T\textbf{x}_n)\neq y_n]\!\!] = 0$ , meaning also $E_n(\textbf{w}) = 0$ .

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode