Error in finetuning mistralai/Mixtral-8x7B-v0.1 using mlx_lm.lora about mlx-examples HOT 10 CLOSED

There is an issue compiling the model during the training step. Before it gets fixed, you can manually delete this line in the Lora trainer: https://github.com/ml-explore/mlx-examples/blob/main/llms/mlx_lm/tuner/trainer.py#L192
cc @awni I created a PR to remove the compile step in the Lora trainer, but I'm not sure if you plan on implementing a proper fix or not. Please feel free to close that PR if it doesn't fit the purpose.

from mlx-examples.

Thanks, it works now :)

from mlx-examples.

Line 192 removal does not work for me

Loading pretrained model
Trainable parameters: 0.005% (2.229M/46702.793M)
Loading datasets
Training
Starting training..., iters: 1000
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/Users/pro/mambaforge/envs/torch/lib/python3.11/site-packages/mlx_lm/lora.py", line 257, in <module>
    run(types.SimpleNamespace(**args))
  File "/Users/pro/mambaforge/envs/torch/lib/python3.11/site-packages/mlx_lm/lora.py", line 204, in run
    train(
  File "/Users/pro/mambaforge/envs/torch/lib/python3.11/site-packages/mlx_lm/tuner/trainer.py", line 219, in train
    lvalue, toks = step(batch)
                   ^^^^^^^^^^^
  File "/Users/pro/mambaforge/envs/torch/lib/python3.11/site-packages/mlx_lm/tuner/trainer.py", line 195, in step
    (lvalue, toks), grad = loss_value_and_grad(model, *batch)
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/pro/mambaforge/envs/torch/lib/python3.11/site-packages/mlx/nn/utils.py", line 34, in wrapped_value_grad_fn
    value, grad = value_grad_fn(model.trainable_parameters(), *args, **kwargs)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: [reshape] Cannot reshape array of size 32 into shape (32,4096).

from mlx-examples.

Very odd.. I'm unable to reproduce that problem. Which version of MLX are you using? Which version of MLX LM?

from mlx-examples.

mlx 0.8.0 mlx LM 0.4.0 . Got the same error as @Satyam7166-tech . This one is after line 192 fixing proposed by @mzbac

from mlx-examples.

Could you share the command?

from mlx-examples.

python -m mlx_lm.lora --model ./mixtral4b-mlx-instruct --adapter-file adapters-plucv3.npz --data ./datapluc --batch-size 2 --train --iters 1000 --lora-layers 16

from mlx-examples.

I see.. I think this was actually from the bug here that was introduced in 0.8 and just fixed.

Sorry about that. You can revert to 0.7 (or build MLX from source) until we have a patch release / 0.9 is out.

from mlx-examples.

I'm going to close this issue because we landed the compile fix. Everything is working on main so it will be a few days at most until this percolates into our relese.

Sorry for the noise here.. we're working on some regression testing for this model to keep us honest.

from mlx-examples.

This should work now with the latest MLX, if you do pip install -U mlx.

from mlx-examples.