This is the official codes for Winner-Take-All Column Row Sampling for Memory Efficient Adaptation of Language Model. FOR REVIEW ONLY, DO NOT DISTRIBUTE

datasets==1.6.2 
scikit-learn==0.24.2 
tensorboard==2.5.0 
matplotlib==3.4.2 
transformers==4.6.0 
numpy==1.21.1 

pip sinstall -v -e .

Run DIVISION on T5 language models:

for dataset in ("rte" "mrpc" "stsb" "cola" "sst2" "qnli" "qqp" "mnli")
do
    for seed in (0 1 2)
    do
    bash scripts/approx_linear.sh 0 $dataset 1 1 1 1 1 1 8 4 division
    done
done

Our code is based on the official code of Ladder Site Tuning