arunmallya / arunmallya.github.io Goto Github PK

Hi arunmallya:

Recently I found your personal website and I learned a lot form your blog, thanks!

I found a typo error in your post "A Quick Introduction to Backpropagation". In the section
"Softmax Layer", the back propagation seems to be this, you missed an item.

Your final result is all right, so I think it is just a type error.

First of all: thank you for your writeups on Backprop in general and the LSTM backprop. It has been a huge help!
I have a problem with your post on LSTM-backprop not being displayed correctly. Right now I am on a Windows7 computer with Firefox, but it has the same problem on different browsers in Linux (Chrome, Firefox):

As you can see, the most interesting part - the equations - are always cut of. One can zoom out to 50%-60% but then the equations get quite small.

I'm not sure how the derivative of h^t-1 could be computed, since many equations in z^t relies on h^t-1. Do we compute each function's derivative and add them together to form the derivative of h^t-1?

c^t hat is a^t

I'm new to LSTMs, and I'm trying to derive the backpropagation of LSTMs. Your site was very helpful to me.

Thanks

In Input and Gate Computation - I you write that

Shouldn't it be ?
The same applies to f and o.

Could you explain the meaning of += in detail?

You said that

+= above is so that this gradient is added to gradient from time step (t+1) (calculated on next slide)

But there is no gradient from time step t+1 calculated on the next slide...
Thank you very much.

Hi,

in the equations noted in: http://arunmallya.github.io/writeups/nn/lstm/index.html#/10

For the derivative of i^hat, I think you took the derivative of sigmoid, but why isn't there any sigmoid function? Is there an error?

Thanks

I don't understand the equation in the red box.

In my opinion, f^t is also indirectly depend on c^(t-1), since f^t is depend on h^(t-1), and h^(t-1) is depend on c^(t-1). What's more, i^t and a^t are also indirectly depend on c^(t-1) for the same reason.

So I think the above equation may not be correct?

Am I right?

Thank you very much!