Comments (2)
Right so the parameter is actually named --max-tokens
now. This is just an upper limit on the maximum number of tokens. Usually the model stops when it encounters an end of sentence token before it hits the upper limit.
from mlx-examples.
Thanks a lot.
from mlx-examples.
Related Issues (20)
- Llama-3-8B-Instruct-Gradient-1048k-4bit not working? HOT 2
- Generating after LORA training CAN NOT Stop Properly HOT 3
- Issue with Fusing Models - Output is Bad HOT 2
- GatedRepoError: 401 Client Error; "You must be authenticated to access it." HOT 1
- [Feature Request] When generating using mlx_lm, specify data format HOT 2
- how to merge lora adapter to base model HOT 1
- delete and uninstall HOT 11
- KV Cache can only process more than self.step tokens if offset % step == 0 HOT 2
- Text to Speech MLX model. HOT 1
- SLM Example Code HOT 1
- Enhance load function to support model configuration editing HOT 1
- Support for full set of output formats - e.g. vtt, json and json-full HOT 2
- Whisper stutters HOT 8
- mlx 0.13 very slow with q8 and fp16 HOT 5
- Fine tuned a Mixtral-8x7B-Instruct-v0.1 model and unable to load with AutoModelForCausalLM HOT 1
- Phi-3-mini-4k-instruct : Failing to stop at <|end|> on generating the answer. HOT 5
- PaliGemma 4bit Quantization broken and Inference issues. HOT 27
- [Feature Request] Function Calling for mlx_lm.server HOT 4
- OS system requirement for mlx HOT 1
- 01-ai/Yi-1.5-9B-Chat got ValueError: Cannot instantiate this tokenizer from a slow version. HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from mlx-examples.