LFD Book Forum

LFD Book Forum (http://book.caltech.edu/bookforum/index.php)
-   Chapter 3 - The Linear Model (http://book.caltech.edu/bookforum/forumdisplay.php?f=110)
-   -   Recency weighted regression (http://book.caltech.edu/bookforum/showthread.php?t=1090)

itooam 08-26-2012 02:00 AM

Re: Recency weighted regression
Suppose I could just transform the weight matrix A into a vector do a transpose then do a cross product (not sure how to present that in algebraic form but think that is the solution)!?

itooam 08-26-2012 02:09 AM

Re: Recency weighted regression
* I meant "inner" product above NOT cross product.

itooam 08-27-2012 03:41 AM

Re: Recency weighted regression
Scrap what I wrote above about large datasets causing havoc for the weight matrix. I found Octave already knows about such problems and has support for sparse matrices... very useful :)

magdon 08-27-2012 12:08 PM

Re: Recency weighted regression
Yes, there is a closed form solution which is obtained by taking the \alpha_t into the square:


This is exactly an unscaled linear regression problem where you have rescaled each data point (\mathbf{x}_t,y_t) by \sqrt{\alpha_t}. So, after you rescale your data in this way, you can just run your old regression algorithm without the weightings.


Originally Posted by itooam (Post 4423)
Thank you for all your help it has been really appreciated. I have one final question, do you know if there is a closed form solution to


(assuming \alpha is a vector with the same number of rows as x?)

i.e., the closed form solution as used for linear regression and regularization - copied from lecture notes is this:

W_{reg} = (Z^{T} Z+\lambda I)^{-1}Z^Ty

I am not sure where \alpha would end up in the above, the derivation is beyond me mathematically?

itooam 08-28-2012 03:15 AM

Re: Recency weighted regression
Thanks Magdon, I always manage to make things so much more complicated than they need to be. That equation you posted would have saved me hours - and it is so simple - why didn't I think of it? Instead I went the long way round, not a total loss though as has been a great learning curve for me :)

I tried your approach and compared to my workings (in one of my previous posts):
W_{reg} = (Z^{T} A Z+\lambda I)^{-1}Z^TAy

and for all my tests I am getting the same W_{reg}. So this is great news as confirms my formula was correct too :D.

Many thanks, I can't say enough how much your help is appreciated.

All times are GMT -7. The time now is 08:01 PM.

Powered by vBulletin® Version 3.8.3
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.
The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, and participants in the Learning From Data MOOC by Yaser S. Abu-Mostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.