Evaluate fine tuning only the sentiment head of sentiment analysis models about torch-control HOT 1 CLOSED

Added code to freeze all layers but the classification head of LLMs. When importing from transformers, differently named heads are created for various models and the layers to keep training need to be specified for each model.

When the model is allowed to tune fully, the training accuracy goes up to 0.9 but the val accuracy falls quickly after an initial spike as the model overfits and looses pretrained knowledge.

Training just the classification head is much more well behaved with the validation accuracy increasing up to a plateau. However, train accuracy is only 0.65 indicating underfitting.
TODO: Implement ability to add a custom classification head with specified complexity, and try training with more weights.

The BERT base was implemented in this branch, together with WIP code for RoBERTa and LLama-2. Turn off the latter as not implemented until finalized and merged. End scope creep here, and raise a separate issue to finalize.

from torch-control.