Comments (2)
Could you share the command you ran that produced the NaN?
from mlx-examples.
I ran the following for 1K iterations and did not see a NaN:
python -m mlx_lm.lora --model microsoft/Phi-3-mini-4k-instruct --train --data ../lora/data --iters 1000
Closing for now. If you share a command that repros NaN then we can reopen and investigate.
from mlx-examples.
Related Issues (20)
- If we do not specify the specific LoRa configuration in the evaluate script, the program will automatically overwrite the default configuration to adapter_config.json.
- Model type phi3 not supported HOT 1
- [Feature Request] Support for QDoRA: Efficient quantized fine-tuning HOT 1
- Loss nan for phi-3 HOT 6
- Curl response got truncated HOT 1
- Model type openelm not supported HOT 2
- Seems like when generating, some memory usage cannot be correctly released. HOT 19
- Looks like llama.py sanitize_config is outdated HOT 3
- Colorize not working with phi-3 HOT 3
- Phi-3 q4 systematic wrong token in first date HOT 7
- [Feature request] A version of mlx_lm.utils.generate() that acts as an iterator HOT 2
- [BUG] OpenELM Quantization broken HOT 12
- TypeError: ModelArgs.__init__() missing 5 required positional arguments HOT 3
- Add a βscan-models to mlx_lm.server to check downloaded models HOT 5
- generate mlx-community/Meta-Llama-3-70B-Instruct-4bit doesn't halt at <|eot_id|> HOT 5
- Potential memory leak during Llama 3 8b model fine-tuning with LoRA HOT 9
- Bug due to Typo in starcoder2 model file
- Convert OpenELM to MLX compatible (ValueError: Unrecognized configuration class_) HOT 2
- Model doesn't know when to stop generating. HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. πππ
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google β€οΈ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from mlx-examples.