LFD Book Forum (http://book.caltech.edu/bookforum/index.php)
-   Homework 5 (http://book.caltech.edu/bookforum/forumdisplay.php?f=134)
-   -   Neural Network - Model / Genetic Algorithm - Optimization (http://book.caltech.edu/bookforum/showthread.php?t=457)

 dbl001 05-07-2012 06:52 AM

Neural Network - Model / Genetic Algorithm - Optimization

Hi,

During the Q + A section in lecture 10, Prof. Abu-Mostafa mentioned that a neural network is a model, while a genetic algorithm is an optimization. Earlier in the lecture he said the neurons in a level 'were factors' and in the next layer, factors of factors.

Is a neural network a model instead of an optimization because the nodes are viewed as factors (even though the factors don't correspond to 'factors' we could assign meaning to in the real world)?

Thank You!

 markland 05-07-2012 03:08 PM

Re: Neural Network - Model / Genetic Algorithm - Optimization

That's not it. Think of it this way:

A neural network model is a layout (how many layers, how many nodes in each layer) along with the values of the weights (e.g. , , and so on for all the weights). The model describes a function, in that it takes an input, x, and produces an output, y.

During learning, the way the learning algorithm figures out what values to assign to the weights is by optimizing the error measure, in other words finding the weights that minimize Ein(w). The method of optimization (the learning algorithm) described in class is backpropagation, which is pretty standard but won't always find the lowest possible Ein (because it can get stuck in local minima). You are free to use other optimization techniques in hopes of finding lower Ein, and one such optimization technique is genetic algorithms. The reason GA isn't a model is because it doesn't specify how to turn x into y. It's just a technique for optimizing a function, in this case it optimizes Ein(w) in terms of w.

 All times are GMT -7. The time now is 04:36 PM.