benbo / adagrad Goto Github PK

Hello, I definitely am not a professional at deep learning (unlike you), and I truly appreciate your implementation, it clarified my questions on the matter. I didn't understand why you added the fudge factor outside of the squaring of gti, on a website with the same naming as your implementation (https://ruder.io/optimizing-gradient-descent/#:~:text=In%20its%20update%20rule%2C%20Adagrad,%CF%B5%20%E2%8B%85%20g%20t%20%2C%20i%20.) the author added the fudge factor inside the squaring. Do you know if this matters, if it was your mistake or not and do you have any tips (I am trying to implement adagrad too!).
tysm.