View Single Post
Old 05-17-2013, 05:28 PM
Elroch Elroch is offline
Invited Guest
Join Date: Mar 2013
Posts: 143
Default Re: *ANSWER* questions w linear regression & weight decay

Originally Posted by Michael Reach View Post

Yes, Elroch, I used the full range of lambda. I think my mistake is elsewhere.
If you're like me, you've probably made a silly error which has nothing to do with understanding the method.

ok, I'm going to expose most of my insult to the art of programming for these questions. Don't use it as a style guide (especially that nasty bit of unvectorised code. Also I suspect the as.matrix's may be superfluous.) The data format should be clear, I hope.
WeightDecayLinearRegressionSolver <- function(inputs, outputs, lambda) {  
  # note inputs have bias co-ordinate 
  # inputs is a matrix of 2d points (with a bias)
  # outputs is a vector providing a real valued function of those points
  if (isTRUE(all.equal(var(outputs), 0))) {  
    # This is the completely degenerate case, which occurs when trying to classify data of a single class
    result <- c(outputs[1], 0, 0)
  else {
    result <- PseudoInverse(t(as.matrix(inputs)) %*% as.matrix(inputs) + diag(rep(lambda, length(inputs[1,]))) ) %*% t(as.matrix(inputs)) %*% outputs

PseudoInverse <- function(mat) {
  tmat <- t(as.matrix(mat))
  inv(tmat %*% as.matrix(mat)) %*% tmat

ClassificationError <- function(actual, predicted) {
  result = 0
  for(i in 1:length(actual)) {
    if(abs(actual[i] - predicted[i]) > 0.5) {
      result <- result + 1
Reply With Quote