LFD Book Forum

LFD Book Forum (http://book.caltech.edu/bookforum/index.php)
-   Homework 5 (http://book.caltech.edu/bookforum/forumdisplay.php?f=134)
-   -   Neural Network - Model / Genetic Algorithm - Optimization (http://book.caltech.edu/bookforum/showthread.php?t=457)

dbl001 05-07-2012 06:52 AM

Neural Network - Model / Genetic Algorithm - Optimization

During the Q + A section in lecture 10, Prof. Abu-Mostafa mentioned that a neural network is a model, while a genetic algorithm is an optimization. Earlier in the lecture he said the neurons in a level 'were factors' and in the next layer, factors of factors.

Is a neural network a model instead of an optimization because the nodes are viewed as factors (even though the factors don't correspond to 'factors' we could assign meaning to in the real world)?

Thank You!

markland 05-07-2012 03:08 PM

Re: Neural Network - Model / Genetic Algorithm - Optimization
That's not it. Think of it this way:

A neural network model is a layout (how many layers, how many nodes in each layer) along with the values of the weights (e.g. w_{0,1}^{(1)} = 0.123, w_{0,2}^{(1)} = 0.456, and so on for all the weights). The model describes a function, in that it takes an input, x, and produces an output, y.

During learning, the way the learning algorithm figures out what values to assign to the weights is by optimizing the error measure, in other words finding the weights that minimize Ein(w). The method of optimization (the learning algorithm) described in class is backpropagation, which is pretty standard but won't always find the lowest possible Ein (because it can get stuck in local minima). You are free to use other optimization techniques in hopes of finding lower Ein, and one such optimization technique is genetic algorithms. The reason GA isn't a model is because it doesn't specify how to turn x into y. It's just a technique for optimizing a function, in this case it optimizes Ein(w) in terms of w.

All times are GMT -7. The time now is 04:36 PM.

Powered by vBulletin® Version 3.8.3
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.
The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, and participants in the Learning From Data MOOC by Yaser S. Abu-Mostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.