View Single Post
  #7  
Old 05-30-2013, 12:02 PM
yaser's Avatar
yaser yaser is offline
Caltech
 
Join Date: Aug 2009
Location: Pasadena, California, USA
Posts: 1,477
Default Re: Gradient Descent on complex parameters (weights)

Quote:
Originally Posted by Kais_M View Post
look at the complex parameter as a 2D vector of real numbers and compute derivative wrt that vector.. this why the # of parameters doubles. is this an "engineering" solution?? or is it really mathematically correct.
Say you apply the same principle of GD; that you are moving in the parameter space by a fixed-length step (in the direction that gives you the biggest change in your objective function, under linear approximation). If you take the size of a complex step to be the Euclidean size (magnitude of a complex number measured as the square root of the sum of its squared real and imaginary parts), then the approach quoted above would be the principled implementation of GD.
__________________
Where everyone thinks alike, no one thinks very much
Reply With Quote