Comments (3)
mmarius
commented on September 27, 2024
1
https://github.com/cybertronai/pytorch-lamb/blob/ff2245eaa458278b096e682a66c29a2d73f690d7/pytorch_lamb/lamb.py#L18
This line will throw the key error. Because state does not contain 'r', 'r1', and 'r2'. They are added by the step function.
from transformer-xl.
yaroslavvb
commented on September 27, 2024
1
should be fixed in master version of lamb
from transformer-xl.
yaroslavvb
commented on September 27, 2024
Can you give a more detailed stack trace? If state_dict is empty I would expect the log_lamb_rs to do nothing:
from transformer-xl.
Related Issues (20)
- local 8-GPU run hangs in .item after 3 days HOT 2
- unexpected keyword argument 'serialized_options' in some envs HOT 3
- feature: include jupyter notebook server for all runs HOT 1
- GPT-2 encoder breaks in new version of PyTorch/huggingface HOT 2
- Reduce Loss HOT 4
- Incorrect model loading HOT 2
- Generating text from the model HOT 1
- model vs. model_to_reset in evaluate_and_log HOT 1
- Correctly adjust LR with LAMP HOT 1
- How to determine max_tokens ? HOT 2
- Share PPL results HOT 1
- qkv computation HOT 1
- Module Not Found Error
- increase eval frequency
- Step time measurement missing proper barrier
- run eval and possibly checkpoint at end of training
- Multi-machine test is broken HOT 5
- babysitter job to automatically kill hung jobs
- Figure out how to install pytorch 1.1 in new env on AWS HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from transformer-xl.