LFD Book Forum - View Single Post

magdon · #14 08-27-2012, 01:08 PM

Yes, there is a closed form solution which is obtained by taking the $\alpha_t$ into the square:

$E_{in}=\sum_{t}\alpha_t(\mathbf{w}\cdot\mathbf{x}_t-y_t)^2=\sum_{t}(\mathbf{w}\cdot\mathbf{x}_t\sqrt{\alpha_t}-y_t\sqrt{\alpha_t})^2$

This is exactly an unscaled linear regression problem where you have rescaled each data point $(\mathbf{x}_t,y_t)$ by $\sqrt{\alpha_t}$ . So, after you rescale your data in this way, you can just run your old regression algorithm without the weightings.

Quote:

Originally Posted by itooam

Thank you for all your help it has been really appreciated. I have one final question, do you know if there is a closed form solution to

$E_{in}=\sum_{t}\alpha_t(\mathbf{w}\cdot\mathbf{x}_t-y_t)^2$

(assuming $\alpha$ is a vector with the same number of rows as x?)

i.e., the closed form solution as used for linear regression and regularization - copied from lecture notes is this:

$W_{reg} = (Z^{T} Z+\lambda I)^{-1}Z^Ty$

I am not sure where $\alpha$ would end up in the above, the derivation is beyond me mathematically?