zyxxmu / dsnot Goto Github PK
View Code? Open in Web Editor NEWOfficial Pytorch Implementation of Our Paper Accepted at ICLR 2024-- Dynamic Sparse No Training: Training-Free Fine-tuning for Sparse LLMs
Official Pytorch Implementation of Our Paper Accepted at ICLR 2024-- Dynamic Sparse No Training: Training-Free Fine-tuning for Sparse LLMs
When running DSNOT with SparseGPT on an OPT model, i get the error above from the following line:
Line 127 in 297993f
It seems like in the WrappedGPT, the H attribute is never set.
Looking at the code, this error probably also happens when running non-opt models, although i haven't tried.
The default argument values in main.py are starkly different from the ones in the README.
Which values should be used here in general, or does it vary widely per model and one has to tune them?
Thanks!
Jonas
Hi, thanks for your inspiring and brilliant work!
I notice that there is no code for the evaluation of zero-shot performance. I was wondering whether this part of code is the same as wanda ?
Hi,
I am interesting in your work.
Can you share 30% sparse llama2-7b?
I want to measure more metrics.
Thanks
Hi, we have some problem of reproducing some of the results in Table1:
CUDA_VISIBLE_DEVICES=4,5,6,7 python main.py \
--model /home/ma-user/modelarts/user-job-dir/DSnoT/ckpt/Llama-2-7b-hf \
--prune_method magnitude \
--initial_method magnitude \
--sparsity_ratio 0.6 \
--sparsity_type unstructured \
--max_cycle_time 50 \
--update_threshold 0.1 \
--pow_of_var_regrowing 1 \
--model_type 'llama'
model: /home/ma-user/modelarts/user-job-dir/DSnoT/ckpt/Llama-2-7b-hf
prune_method: magnitude
without_DSnoT: False
initial_method: magnitude
skip_layer mlp, skip_sub_layer no_skip
max_cycle_time: 50, update_threshold: 0.1
pow_of_var_pruning:1, pow_of_var_regrowing:1.0
without_same_sign:True
sparse pattern: unstructured
sample: 128
sparsity sanity check 0.6013, ppl: 1924.735595703125
CUDA_VISIBLE_DEVICES=4,5,6,7 python main.py \
--model /home/ma-user/modelarts/user-job-dir/DSnoT/ckpt/Llama-2-7b-hf \
--prune_method DSnoT \
--initial_method magnitude \
--sparsity_ratio 0.6 \
--sparsity_type unstructured \
--max_cycle_time 50 \
--update_threshold 0.1 \
--pow_of_var_regrowing 1 \
--model_type 'llama'
model: /home/ma-user/modelarts/user-job-dir/DSnoT/ckpt/Llama-2-7b-hf
prune_method: DSnoT
without_DSnoT: False
initial_method: magnitude
skip_layer mlp, skip_sub_layer no_skip
max_cycle_time: 50, update_threshold: 0.1
pow_of_var_pruning:1, pow_of_var_regrowing:1.0
without_same_sign:True
sparse pattern: unstructured
sample: 128
sparsity sanity check 0.5999, ppl: 3950.8154296875
CUDA_VISIBLE_DEVICES=4,5,6,7 python main.py \
--model /home/ma-user/modelarts/user-job-dir/DSnoT/ckpt/Llama-2-7b-hf \
--prune_method DSnoT \
--initial_method sparsegpt\
--sparsity_ratio 0.6 \
--sparsity_type unstructured \
--max_cycle_time 50 \
--update_threshold 0.1 \
--pow_of_var_regrowing 1 \
--model_type 'llama'
model: /home/ma-user/modelarts/user-job-dir/DSnoT/ckpt/Llama-2-7b-hf
prune_method: DSnoT
without_DSnoT: False
initial_method: sparsegpt
skip_layer mlp, skip_sub_layer no_skip
max_cycle_time: 50, update_threshold: 0.1
pow_of_var_pruning:1, pow_of_var_regrowing:1.0
without_same_sign:True
sparse pattern: unstructured
sample: 128
sparsity sanity check 0.5999, ppl: 122160.71875
We would greatly appreciate your assistance in helping us pinpoint the reasons behind these results and in verifying the correctness of our implementation of your code.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.