Source code for NeurIPS 2022 Long paper: Fine-Tuning Pre-Trained Language Models Effectively by Optimizing Subnetworks Adaptively
- python: 3.8.10
- CUDA Version: 11.4
- torch==1.8.0
- datasets==1.8.0
- transformers==4.7.0
cd script
bash run_glue.sh
bash run_out_of_domain.sh
bash run_low_resource.sh
If you use this work or code, please kindly cite the following paper:
@inproceedings{haojiefine,
title={Fine-Tuning Pre-Trained Language Models Effectively by Optimizing Subnetworks Adaptively},
author={Haojie, Zhang and Li, Ge and Li, Jia and Zhang, Zhongjin and Zhu, Yuqi and Jin, Zhi},
booktitle={Advances in Neural Information Processing Systems}
}