Re: Gradient Descent on complex parameters (weights)
The purpose of a quality function for comparing options, such as an error function, requires at the very least a total order on the range. It would be reasonable for this to be topological, rather than explicitly metric: this would do no harm to the ideas of minima, or comparisons between errors. [One indication of this was in the lectures, where an error function was replaced by its logarithm, in the knowledge that this would preserve the total order].
But as Yaser indicates, complex numbers lack such an order (at least a natural one), so can't be the range of an error function.
