Clarification on HW6-Q8
 

In Q8 it is asked the "...closest to the total number of operations required in a single iteration of backpropagation (using SGD on one data point)".

My question is if we have to account only the backpropagation step or, instead, one complete iteration of the backpropagation algorithm, which includes the forward propagation, the backpropagation and the SGD updating steps.

Thanks in advance.

TS

Re: Clarification on HW6-Q8
 

Forward, backward, and update.

Re: Clarification on HW6-Q8
 

Did I understand right that dimensions for input and hidden layer are given without constant term? (so, that in input layer, e.g., we have 5 +1 (const) nodes).

Re: Clarification on HW6-Q8
 

Correct, as the convention has the index going from to .

Re: Clarification on HW6-Q8
 

Thank you very much Professor for answering "on-the-fly" (as always).
TS

Re: Clarification on HW6-Q8
 

Only product of the form count as operations.

On backward we have:

and and +1 (constant) for layer (1). So we have the same 4 operations for 2-1 layer, and obviously 18 operations for 1-0 layer. Or we dont need to compute for constants ?

Re: Clarification on HW6-Q8
 

yes, we don't need to:

1. compute delta for the output (because it doesn't need any of the operations, counted as operation in this certain task)

2. compute deltas for constants (because they're constants)

3. compute deltas for input (because they're just features (x), and delta is a derivative of the error with respect to s)

So for computing deltas you only need 3 operations.

Re: Clarification on HW6-Q8
 

Thank you very much ! Enlgish isnt my native language and its hard to me to catch 100% information from video. Thanks for clarifying !

Re: Clarification on HW6-Q8
 

Thank you! Your answer preempted one of my questions. Specifically about the calculation of the final layer delta, as technically none of it seems to count as an operation under our definition.