LFD Book Forum  

Go Back   LFD Book Forum > Course Discussions > Online LFD course > Homework 5

Reply
 
Thread Tools Display Modes
  #1  
Old 05-07-2012, 06:52 AM
dbl001 dbl001 is offline
Member
 
Join Date: Apr 2012
Posts: 11
Default Neural Network - Model / Genetic Algorithm - Optimization

Hi,

During the Q + A section in lecture 10, Prof. Abu-Mostafa mentioned that a neural network is a model, while a genetic algorithm is an optimization. Earlier in the lecture he said the neurons in a level 'were factors' and in the next layer, factors of factors.

Is a neural network a model instead of an optimization because the nodes are viewed as factors (even though the factors don't correspond to 'factors' we could assign meaning to in the real world)?

Thank You!
Reply With Quote
  #2  
Old 05-07-2012, 03:08 PM
markland markland is offline
Member
 
Join Date: Apr 2012
Posts: 30
Default Re: Neural Network - Model / Genetic Algorithm - Optimization

That's not it. Think of it this way:

A neural network model is a layout (how many layers, how many nodes in each layer) along with the values of the weights (e.g. w_{0,1}^{(1)} = 0.123, w_{0,2}^{(1)} = 0.456, and so on for all the weights). The model describes a function, in that it takes an input, x, and produces an output, y.

During learning, the way the learning algorithm figures out what values to assign to the weights is by optimizing the error measure, in other words finding the weights that minimize Ein(w). The method of optimization (the learning algorithm) described in class is backpropagation, which is pretty standard but won't always find the lowest possible Ein (because it can get stuck in local minima). You are free to use other optimization techniques in hopes of finding lower Ein, and one such optimization technique is genetic algorithms. The reason GA isn't a model is because it doesn't specify how to turn x into y. It's just a technique for optimizing a function, in this case it optimizes Ein(w) in terms of w.
Reply With Quote
Reply

Tags
genetic algorithm, neural network

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -7. The time now is 03:44 AM.


Powered by vBulletin® Version 3.8.3
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.
The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, and participants in the Learning From Data MOOC by Yaser S. Abu-Mostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.