Re: SVMs versus NNs
1) the issue with logistic regression is that it is giving you a probability of class A rather than a classification...if you need that then it is (clearly) better . if all you are going to do is threshold the probability at 0.5 and classify then not.
2) I think SVMS are the method of choice unless you have large number of points...ie try that first....the issue is that NNETS basically have no optimisation procedure. You have to find a learning rate that jumps over the troughs and lands in a deep vallley... [imagine trying to descend a mountain range with your eyes closed]
3) I view SVMs as a linear classifier with a nice regularisation.... this is very important in classification, because as you saw in the lectures, there are many different linear classifiers that give you the same classifcation on the training set. Regression is different .. changing the straight line fit changes your MSE. I would say linear regression would be the natural analogue of SVMs ...[ and then you have to choose the regularisation term and nonlinear transformations]
