View Single Post
Old 05-30-2013, 12:02 PM
yaser's Avatar
yaser yaser is offline
Join Date: Aug 2009
Location: Pasadena, California, USA
Posts: 1,478
Default Re: Gradient Descent on complex parameters (weights)

Originally Posted by Kais_M View Post
look at the complex parameter as a 2D vector of real numbers and compute derivative wrt that vector.. this why the # of parameters doubles. is this an "engineering" solution?? or is it really mathematically correct.
Say you apply the same principle of GD; that you are moving in the parameter space by a fixed-length step (in the direction that gives you the biggest change in your objective function, under linear approximation). If you take the size of a complex step to be the Euclidean size (magnitude of a complex number measured as the square root of the sum of its squared real and imaginary parts), then the approach quoted above would be the principled implementation of GD.
Where everyone thinks alike, no one thinks very much
Reply With Quote