Having spent some time on this, (this area of maths I am very weak)
I think the solution is:
Where
is a diagonal matrix. A bit like the Identity matrix but with weight values i.e.,
|
, 0, 0, ... 0|
|0,
, 0 ... 0|
|..................|
|0, 0, 0 ....
|
The bit that makes this tricky (for me) is the regularisation. I suppose I could test the above using this formula and then try the same using gradient descent (where I know it will be correct) if the values are close then the above can be considered correct (if I plug in largely varying values of lamba for testing).