shikumo / adasoptimizer Goto Github PK
View Code? Open in Web Editor NEWThis project forked from yanaieliyahu/adasoptimizer
ADAS is short for Adaptive Step Size, it's an optimizer that unlike other optimizers that just normalize the derivative, it fine-tunes the step size, truly making step size scheduling obsolete, achieving state-of-the-art training performance
License: MIT License