LFD Book Forum

LFD Book Forum (http://book.caltech.edu/bookforum/index.php)
-   Chapter 3 - The Linear Model (http://book.caltech.edu/bookforum/forumdisplay.php?f=110)
-   -   Problem 3.5 (http://book.caltech.edu/bookforum/showthread.php?t=4489)

BojanVujatovic 07-26-2014 04:07 PM

Problem 3.5
 
I have questions about this problem. What I get after analyzing E_n(\textbf{w}) is the following:

E_n(\textbf{w}) = 
\begin{cases}
0, &\text{sign}(\textbf{w}^T\textbf{x}_n)=y_n \Leftrightarrow  y_n\textbf{w}^T\textbf{x}_n \geq 0  \\ 
|y_n - \textbf{w}^T\textbf{x}_n|=1 - y_n\textbf{w}^T\textbf{x}_n, & \text{sign}(\textbf{w}^T\textbf{x}_n) \neq y_n \Leftrightarrow  y_n\textbf{w}^T\textbf{x}_n < 0 
\end{cases}

The last line follows because:
1\cdot|y_n - \textbf{w}^T\textbf{x}_n| = |y_n||y_n-\textbf{w}^T\textbf{x}_n|
= |1 - y_n\textbf{w}^T\textbf{x}_n|=1 - y_n\textbf{w}^T\textbf{x}_n (since y_n\textbf{w}^T\textbf{x}_n < 0)

From this, I cannot see how E_n(\textbf{w}) is not continuous at \textbf{w}^T\textbf{x}_n = y_n. Instead I find that it is not continuous nor differentiable for \textbf{w}'s such that \textbf{w}^T\textbf{x}_n = 0.

Any help would be greatly appreciated.

magdon 07-30-2014 08:39 AM

Re: Problem 3.5
 
There are 3 cases:

y_n\textbf{w}^T\textbf{x}_n>1 in which case E(\mathbf{x}_n)=0, which is differentiable.

y_n\textbf{w}^T\textbf{x}_n<1 in which case E(\mathbf{x}_n)=1-y_n\textbf{w}^T\textbf{x}_n, which is differentiable.

y_n\textbf{w}^T\textbf{x}_n=1 in which case E(\mathbf{x}_n)=0, but is not differentiable because the derivative is 0 or not depending on whether y_n\textbf{w}^T\textbf{x}_n approaches 1 from above or below.



Quote:

Originally Posted by BojanVujatovic (Post 11698)
I have questions about this problem. What I get after analyzing E_n(\textbf{w}) is the following:

E_n(\textbf{w}) = 
\begin{cases}
0, &\text{sign}(\textbf{w}^T\textbf{x}_n)=y_n \Leftrightarrow  y_n\textbf{w}^T\textbf{x}_n \geq 0  \\ 
|y_n - \textbf{w}^T\textbf{x}_n|=1 - y_n\textbf{w}^T\textbf{x}_n, & \text{sign}(\textbf{w}^T\textbf{x}_n) \neq y_n \Leftrightarrow  y_n\textbf{w}^T\textbf{x}_n < 0 
\end{cases}

The last line follows because:
1\cdot|y_n - \textbf{w}^T\textbf{x}_n| = |y_n||y_n-\textbf{w}^T\textbf{x}_n|
= |1 - y_n\textbf{w}^T\textbf{x}_n|=1 - y_n\textbf{w}^T\textbf{x}_n (since y_n\textbf{w}^T\textbf{x}_n < 0)

From this, I cannot see how E_n(\textbf{w}) is not continuous at \textbf{w}^T\textbf{x}_n = y_n. Instead I find that it is not continuous nor differentiable for \textbf{w}'s such that \textbf{w}^T\textbf{x}_n = 0.

Any help would be greatly appreciated.


BojanVujatovic 08-03-2014 09:23 PM

Re: Problem 3.5
 
Thank you for your reply.

The only part I don't quite follow is:
Quote:

Originally Posted by magdon (Post 11699)
y_n\textbf{w}^T\textbf{x}_n<1 in which case E(\mathbf{x}_n)=1-y_n\textbf{w}^T\textbf{x}_n, which is differentiable.

As I see, and please correct me if I'm wrong, for 0 \leq y_n\textbf{w}^T\textbf{x}_n<1, the signal \textbf{w}^T\textbf{x}_n and the output y_n agree.
(e.g. \textbf{w}^T\textbf{x}_n = -0.5 and y_n = -1 \implies y_n\textbf{w}^T\textbf{x}_n = 0.5).

Therefore, \text{sign}(\textbf{w}^T\textbf{x}_n)=y_n and [\!\![\text{sign}(\textbf{w}^T\textbf{x}_n)\neq y_n]\!\!] = 0, meaning also E_n(\textbf{w}) = 0.


All times are GMT -7. The time now is 03:02 PM.

Powered by vBulletin® Version 3.8.3
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, and participants in the Learning From Data MOOC by Yaser S. Abu-Mostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.