Having spent some time on this, (this area of maths I am very weak)
I think the solution is:
Where
is a diagonal matrix. A bit like the Identity matrix but with weight values i.e.,

, 0, 0, ... 0
0,
, 0 ... 0
..................
0, 0, 0 ....

The bit that makes this tricky (for me) is the regularisation. I suppose I could test the above using this formula and then try the same using gradient descent (where I know it will be correct) if the values are close then the above can be considered correct (if I plug in largely varying values of lamba for testing).