LFD Book Forum Neural Network - Model / Genetic Algorithm - Optimization

#1
05-07-2012, 06:52 AM
 dbl001 Member Join Date: Apr 2012 Posts: 11
Neural Network - Model / Genetic Algorithm - Optimization

Hi,

During the Q + A section in lecture 10, Prof. Abu-Mostafa mentioned that a neural network is a model, while a genetic algorithm is an optimization. Earlier in the lecture he said the neurons in a level 'were factors' and in the next layer, factors of factors.

Is a neural network a model instead of an optimization because the nodes are viewed as factors (even though the factors don't correspond to 'factors' we could assign meaning to in the real world)?

Thank You!
#2
05-07-2012, 03:08 PM
 markland Member Join Date: Apr 2012 Posts: 30
Re: Neural Network - Model / Genetic Algorithm - Optimization

That's not it. Think of it this way:

A neural network model is a layout (how many layers, how many nodes in each layer) along with the values of the weights (e.g. , , and so on for all the weights). The model describes a function, in that it takes an input, x, and produces an output, y.

During learning, the way the learning algorithm figures out what values to assign to the weights is by optimizing the error measure, in other words finding the weights that minimize Ein(w). The method of optimization (the learning algorithm) described in class is backpropagation, which is pretty standard but won't always find the lowest possible Ein (because it can get stuck in local minima). You are free to use other optimization techniques in hopes of finding lower Ein, and one such optimization technique is genetic algorithms. The reason GA isn't a model is because it doesn't specify how to turn x into y. It's just a technique for optimizing a function, in this case it optimizes Ein(w) in terms of w.

 Tags genetic algorithm, neural network

 Thread Tools Display Modes Linear Mode

 Posting Rules You may not post new threads You may not post replies You may not post attachments You may not edit your posts BB code is On Smilies are On [IMG] code is On HTML code is Off Forum Rules
 Forum Jump User Control Panel Private Messages Subscriptions Who's Online Search Forums Forums Home General     General Discussion of Machine Learning     Free Additional Material         Dynamic e-Chapters         Dynamic e-Appendices Course Discussions     Online LFD course         General comments on the course         Homework 1         Homework 2         Homework 3         Homework 4         Homework 5         Homework 6         Homework 7         Homework 8         The Final         Create New Homework Problems Book Feedback - Learning From Data     General comments on the book     Chapter 1 - The Learning Problem     Chapter 2 - Training versus Testing     Chapter 3 - The Linear Model     Chapter 4 - Overfitting     Chapter 5 - Three Learning Principles     e-Chapter 6 - Similarity Based Methods     e-Chapter 7 - Neural Networks     e-Chapter 8 - Support Vector Machines     e-Chapter 9 - Learning Aides     Appendix and Notation     e-Appendices

All times are GMT -7. The time now is 05:07 AM.