Additional parameters to mlx_lm lora? r, lora_alpha, lora_dropout, scale? about mlx-examples HOT 6 OPEN

It would be great adding the new LR Scheduler or even the optimizer, but parameters become too complex then.
Probably something like PR #235 to read from a config file would be better.

from mlx-examples.

Yes that's been on our list to add for a while.

Though, do you use mlx_lm.lora or just the original lora.py script? My preference is to update the package since that one has many more features.

from mlx-examples.

I always us mlx_lm now, it's becoming more powerful at each release 💪

from mlx-examples.

Agree I think we may need to start using a yaml config

from mlx-examples.

#235 is dated. I can rebase it to mlx-examples/main and update it (to support the parameters that have been added since I last worked on that PR) if there is interest. I have found it useful to pull parameters from yaml and override them from what is provided via the command line.

from mlx-examples.

I too migrated to mlx_lm.lora and end up using a shell script generator.
#235 would simplify the whole process as use cases for MLX grow and simplify
the tuning by having different models/confs/benchmark in parallel.

from mlx-examples.