View Single Post
#9
 Elroch Invited Guest Join Date: Mar 2013 Posts: 143 Re: *ANSWER* questions w linear regression & weight decay

Quote:
 Originally Posted by Michael Reach Thanks! Yes, Elroch, I used the full range of lambda. I think my mistake is elsewhere.
If you're like me, you've probably made a silly error which has nothing to do with understanding the method.

ok, I'm going to expose most of my insult to the art of programming for these questions. Don't use it as a style guide (especially that nasty bit of unvectorised code. Also I suspect the as.matrix's may be superfluous.) The data format should be clear, I hope.
Code:
```WeightDecayLinearRegressionSolver <- function(inputs, outputs, lambda) {
# note inputs have bias co-ordinate
# inputs is a matrix of 2d points (with a bias)
# outputs is a vector providing a real valued function of those points
if (isTRUE(all.equal(var(outputs), 0))) {
# This is the completely degenerate case, which occurs when trying to classify data of a single class
result <- c(outputs, 0, 0)
}
else {
result <- PseudoInverse(t(as.matrix(inputs)) %*% as.matrix(inputs) + diag(rep(lambda, length(inputs[1,]))) ) %*% t(as.matrix(inputs)) %*% outputs
}
result
}

PseudoInverse <- function(mat) {
tmat <- t(as.matrix(mat))
inv(tmat %*% as.matrix(mat)) %*% tmat
}

ClassificationError <- function(actual, predicted) {
result = 0
for(i in 1:length(actual)) {
if(abs(actual[i] - predicted[i]) > 0.5) {
result <- result + 1
}
}
result/length(actual)
}```