LFD Book Forum (http://book.caltech.edu/bookforum/index.php)
-   Homework 2 (http://book.caltech.edu/bookforum/forumdisplay.php?f=131)
-   -   Multiplication matrix X and its transpose (http://book.caltech.edu/bookforum/showthread.php?t=4498)

 tarquilu 09-15-2014 04:33 PM

Multiplication matrix X and its transpose

Hello,

I'm trying to compute the pseudo-inverse of matrix X with dimensions [d+1 x N], where N is the number of data points. Lecture 3, slide 16 shows that the dimensions "shrink" when multiplying (X^T)X from [d+1 x N][N x d+1] to [d+1 x d+1].

Although LFD on page 86 states that "In practice, (X^T)T is invertible in most of the cases since N is often much bigger than d+1, so there will likely be d+1 linearly independent vectors xn," I don't know if this statement is meant to explain the reason for the simplification to [d+1 x d+1] dimensions. Moreover, when creating X and computing (X^T)T, I obtain a matrix with [N x N] dimensions, not [d+1 x d+1] dimensions.

Could some please clarify this for me? Thank you.

 yaser 09-16-2014 01:02 AM

Re: Multiplication matrix X and its transpose

Quote:
 Originally Posted by tarquilu (Post 11725) Hello, I'm trying to compute the pseudo-inverse of matrix X with dimensions [d+1 x N], where N is the number of data points. Lecture 3, slide 16 shows that the dimensions "shrink" when multiplying (X^T)X from [d+1 x N][N x d+1] to [d+1 x d+1]. Although LFD on page 86 states that "In practice, (X^T)T is invertible in most of the cases since N is often much bigger than d+1, so there will likely be d+1 linearly independent vectors xn," I don't know if this statement is meant to explain the reason for the simplification to [d+1 x d+1] dimensions. Moreover, when creating X and computing (X^T)T, I obtain a matrix with [N x N] dimensions, not [d+1 x d+1] dimensions. Could some please clarify this for me? Thank you.
1. X is in fact of dimensions N x d+1. The transpose is d+1 x N.

2. The remark about independent vectors is only meant to argue that the d+1 x d+1 matrix has full rank (d+1) since in principle it could be singular, with rank less than d+1 hence not invertible.

3. I am not clear on what (X^T)T is.

 tarquilu 09-21-2014 11:14 AM

Re: Multiplication matrix X and its transpose

Thank you professor. Sorry, I meant to write (X^T)X to symbolize the transpose of matrix X multiplied by X.

I now know why my matrix was not "shrinking." I was multiplying X(X^T) instead of (X^T)X so that is why I was not obtaining a [d+1 x d+1] dimensional matrix.

However, the g hypotheses I'm obtaining with w = (X^dagger)y are usually not close to target functions f... random, really. I'm obtaining an average Ein of approximately 0.5. Sometimes my code gives a g with an extremely large or small y-intercept value (e.g., 3.60288*10^16 - 0.5x). Any pointers would be appreciated. Thank you.

 yaser 09-24-2014 09:15 PM

Re: Multiplication matrix X and its transpose

Just want to point out that a regular session of the course is starting this week at edX:

http://www.edx.org/course/caltechx/c...ning-data-2516

I am attending to the discussion forum over there, so I'll check here less frequently during that session.

 tarquilu 10-14-2014 04:47 PM

Re: Multiplication matrix X and its transpose

I made a simple program coding error which was leading to the aberrant weights and Ein. Once I fixed the coding error, I obtained the correct weights and Ein.

 All times are GMT -7. The time now is 07:19 AM.