Understanding InverseSquareRootLR about test-completion-transformer HOT 1 CLOSED

Ah, I missed the negative sign! Should be the following in the decay phase:

• At step 10,000, the scheduler will return 0.01 / sqrt(10,000) = 1.
• At step 11,000, the scheduler will return 0.01 / sqrt(11,000) = 1.04880884817.
• At step 20,000, the scheduler will return 0.01 / sqrt(20,000) = 1.41421356237.
• At step 50,000, the scheduler will return 0.01 / sqrt(50,000) = 2.2360679775.
• At step 100,000 the scheduler will return 0.01 / sqrt(100,000) = 3.16227766017.
• At step 1,000,000 the scheduler will return 0.01 / sqrt(1,000,000) = 10.
• At step 10,000,000 the scheduler will return 0.01 / sqrt(10,000,000) = 31.6227766017.
• At step 100,000,000 the scheduler will return 0.01 / sqrt(100,000,000) = 100.

Your code makes perfect sense. Happy to delete these comments if you like or leave them in case they are helpful to others.

from test-completion-transformer.