backpropagation at the final layer

jianpan · #1 08-17-2012, 06:56 PM

since $\delta_1^{(L)}=\frac{\partial e(w)}{\partial s_1^{(L)}}$ , and $e(w)=(x_1^{(L)}-y_n)^2$ , and $x_1^{(L)}=\theta(s_1^{(L)})$ , I got $\delta_1^{(L)}=2*(x_1^{(L)}-y_n)*(1-\theta^2(s))$ . is this correct? I found a python program at wikipedia about backpropagation using the $\delta_1^{(L)}=(x_1^{(L)}-y_n)*(1-\theta^2(s))$ , and couldn't figure out where the 2 was dropped. Can someone confirm my derivation is correct? Thanks.

yaser · #2 08-17-2012, 09:12 PM

Quote:

Originally Posted by jianpan

since $\delta_1^{(L)}=\frac{\partial e(w)}{\partial s_1^{(L)}}$ , and $e(w)=(x_1^{(L)}-y_n)^2$ , and $x_1^{(L)}=\theta(s_1^{(L)})$ , I got $\delta_1^{(L)}=2*(x_1^{(L)}-y_n)*(1-\theta^2(s))$ . is this correct? I found a python program at wikipedia about backpropagation using the $\delta_1^{(L)}=(x_1^{(L)}-y_n)*(1-\theta^2(s))$ , and couldn't figure out where the 2 was dropped. Can someone confirm my derivation is correct? Thanks.

Some people define the error with a factor of $1 \over 2$ in it in anticipation of the differentiation, hence the discrepancy.

jianpan · #3 08-18-2012, 01:05 AM

Thanks for the clarification, Professor.

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode