View Single Post
  #9  
Old 05-17-2013, 05:28 PM
Elroch Elroch is offline
Invited Guest
 
Join Date: Mar 2013
Posts: 143
Default Re: *ANSWER* questions w linear regression & weight decay

Quote:
Originally Posted by Michael Reach View Post
Thanks!

Yes, Elroch, I used the full range of lambda. I think my mistake is elsewhere.
If you're like me, you've probably made a silly error which has nothing to do with understanding the method.

ok, I'm going to expose most of my insult to the art of programming for these questions. Don't use it as a style guide (especially that nasty bit of unvectorised code. Also I suspect the as.matrix's may be superfluous.) The data format should be clear, I hope.
Code:
WeightDecayLinearRegressionSolver <- function(inputs, outputs, lambda) {  
  # note inputs have bias co-ordinate 
  # inputs is a matrix of 2d points (with a bias)
  # outputs is a vector providing a real valued function of those points
  if (isTRUE(all.equal(var(outputs), 0))) {  
    # This is the completely degenerate case, which occurs when trying to classify data of a single class
    result <- c(outputs[1], 0, 0)
  }
  else {
    result <- PseudoInverse(t(as.matrix(inputs)) %*% as.matrix(inputs) + diag(rep(lambda, length(inputs[1,]))) ) %*% t(as.matrix(inputs)) %*% outputs
  }
  result
}

PseudoInverse <- function(mat) {
  tmat <- t(as.matrix(mat))
  inv(tmat %*% as.matrix(mat)) %*% tmat
}

ClassificationError <- function(actual, predicted) {
  result = 0
  for(i in 1:length(actual)) {
    if(abs(actual[i] - predicted[i]) > 0.5) {
      result <- result + 1
    }
  }
  result/length(actual)
}
Reply With Quote