
#1




*ANSWER* questions w linear regression & weight decay
I have been running the weight decay examples Q26, but haven't seen any real improvement in the outofsample error compared to no regularization at all. Is that just a feature of this particular problem, or should I recheck my calculations?
Unfortunately (or not), the answers I've been getting do appear as options on the multiple choices. 
#2




Re: *ANSWER* questions w linear regression & weight decay

#3




Re: *ANSWER* questions w linear regression & weight decay

#4




Re: *ANSWER* questions w linear regression & weight decay
Quote:
And yes, I have been using classification error, but that is a good point  I started using the regression residuals and such, but that mistake at least I caught. 
#5




Re: *ANSWER* questions w linear regression & weight decay
As I suspected, all my answers on these were wrong. Does anyone have code (R if possible) to show, that I could use for comparison? I'm suspecting my problem was something dumb; even the original linear regression was wrong, and I compared that one with the same answer from the R lm() function.
I'm especially concerned since HW 7 uses all the same data again  so I really need to track this down. 
#6




Re: *ANSWER* questions w linear regression & weight decay
Quote:

#7




Re: *ANSWER* questions w linear regression & weight decay
Michael:
Here you go: Code:
#READ IN THE FILES. datos1 < read.table("in.dta") names(datos1) < c("X1","X2","Y") datos2 < read.table("out.dta") names(datos2) < c("X1","X2","Y") #FOR THE FOLLOWING QUESTIONS, SET UP THE MATRIXES Z < with(datos1, cbind(rep(1,nrow(datos1)),X1,X2, X1^2,X2^2,X1*X2,abs(X1X2),abs(X1+X2)) ) Z < as.matrix(Z) Zout < with(datos2, cbind(rep(1,nrow(datos2)),X1,X2, X1^2,X2^2,X1*X2,abs(X1X2),abs(X1+X2)) ) Zout < as.matrix(Zout) #NOW FIT WITH WEIGHT DECAY USING LAMBDA=10^3 lambda < 10^(3) M < t(Z)%*%Z + diag(rep(8,1))*lambda w < solve(M)%*%t(Z)%*%datos1$Y Ym < as.numeric(sign(Z%*%w)) Ein < mean(datos1$Y!=Ym) Ym < as.numeric(sign(Zout%*%w)) Eout < mean(datos2$Y!=Ym) 
Thread Tools  
Display Modes  

